What is the chance of making a type 1 error, and can you elaborate on it? If we modify the sampling distribution by filtering out data below the mean, how will it affect the probability of a type 1 error?

Question

Anonymous · Accepted Answer

Type I errors are False Positives, rejecting the Null Hypothesis when it's
actually true. It's the risk of being over eager to see a signal. If we filter
out all the values below our mean, without actually effecting the population,
our results are not usable. We're cherry-picking and making our numbers
inflated, making the FP rate jump through the roof. If before the experiment we
pre-set that we'd only measure and observe results on a historic subportion of
the population and had both control and experimental groups and noted the
conditions, it'd be usable and there are some clipping heuristics (e.g. using
1st to 99th percentiles to avoid skew from outliers), but to do the picking
after results come in is bias and no longer represents the underlying population
(sample integrity is compromised). In business context, this would be something
akin to running a new version of a button or add, but only report reporting the
view-through rate for the top 50% of viewers or in finance to run a trading
strategy but to only report on the top 50% of profitable trades while dropping
the losers, another example of this cleaner to survivor bias, would be only
measuring quality of life outcomes for folks who survive a disease even though a
treatment may have a much lower survival rate (i don't think most folks would
take such a gamble)

Anonymous · Answer

Probability of type 1 error is the probability of rejecting a null hypothesis
when it is true. when we filter out all the values below the mean, the
distribution becomes more skewed and the results become more significant,
resulting in an increase in the type 1 error

Anonymous · Answer

The distribution is truncated (like removing values below the mean), the
variance decreases and the mean shifts. Test statistics may become larger than
expected under Ho.

This leads to rejecting Ho more often, increasing the actual Type I error rate
above the nominal α

1.

What is the chance of making a type 1 error, and can you elaborate on it? If we modify the sampling distribution by filtering out data below the mean, how will it affect the probability of a type 1 error?

Practice this question with AI

Go Premium

Community Answers

Unlock Full Access

Unlock Full Access

Unlock Full Access

Practice More Questions