The Importance of Effect Magnitude, Essay Example

As given in the question, this is a one-sided test because the company claims that it produces less cavities than other brands of toothpaste.  Thus, we are interested in exploring whether there were fewer cavities (one-sided).

The null hypothesis for this study is: The mean number of cavities between individuals that use the toothpaste and do not use the toothpaste are equal.

The alternate hypothesis for this study is: The mean number of cavities between individuals that use the toothpaste are less than those than do not use the toothpaste (one-sided hypothesis).

What are degrees of freedom? How are the calculated?

“Degrees of freedom” are a general term for the values in the calculation; it has also been defined as to the dimensionalities of parameter spaces.  They essentially are the way to track of how data is used to either estimate the answer or provide variability.  They are calculated differently for different statistical operations, but they commonly are counted based on the sample size used for the calculation.

What do inferential statistics allow you to infer?

The purpose of inferential statistics is to take a data sample, from which various statistics are estimated, and infer what the parameters from a population would ultimately be. So, for example, a sample of students from a school may be able to estimate the SAT scores for all students at that school (if the sample is representative); a clinical trial may infer the efficacy of a drug for the entire population.

What is the General Linear Model (GLM)? Why does it matter?

The General Linear Model (GLM) is a less strict model of the commonly-used linear regression.  GLM is important in that, while OLM is primarily used for data with an underlying normal distribution, GLM can be used with other distributions.  This is an important matter because the ability to generalize linear regression enlarges the ability to use it for a wider range of data.

Compare and contrast parametric and nonparametric statistics. Why and in what types of cases would you use one over the other?

Parametric statistics are used for data with an underlying normal distribution; non-parametric statistics are used for data without an underlying normal distribution.

If one has data greater than 30 observations that generally meets the assumptions, using the parametric statistics.  However, if you have a small number of data points, or you have reason to believe that the data is skewed, using a non-parametric statistic is recommended.

Why is it important to pay attention to the assumptions of the statistical test? What are your options if your dependent variable scores are not normally distributed?

The assumptions underpinning a statistical test are essentially what guarantees that one receives a valid result. They are essentially the “rules of the road.”  If you don’t pay attention to those assumptions, the result you receive may not approximate  to reality.

If one’s dependent variable scores are not normally distributed, the main option is to use one of the non-parametric statistical calculating methods.

What does p = .05 mean? What are some misconceptions about the meaning of p =.05? Why are they wrong? Should all research adhere to the p = .05 standard for significance? Why or why not?

A p-value of .05 means that there is a 5% chance that the result received is due to probabilistic chance rather than an actual difference between the phenomenon tested.   Many of the misconceptions surrounding the p-value have to do with what it actually means; that is, that the results are “significant.”

In fact, the .05 threshold, although gaining a cult following among scientists and social scientists alike, is a random cut-off that has no more inherent meaning than say a p-value of .06.  No; not all research should adhere to the p=.05 significance threshold.  The best policy is for researchers is to list out the p-value of the result and to have readers decide whether the results are significant  or not.

Compare and contrast the concepts of effect size and statistical significance

The concept of effect size is related to the size of the effect measured in the variables being manipulated.  For example, perhaps a biostatistical analysis looks at the impact on a new drug on lowering an individual’s blood pressure (say a reduction of two points/ ml); or an econometric study that looks at the impact of a dollar increase in taxes on consumption (say a reduction of $10 in personal consumption for ever $1 increase in taxes).  Those measures are effect sizes.

The issue of statistical significance is a question of: is the result due to probabilistic chance or is there an actual difference  in the phenomenon being examined.  So in the case of the drug’s impact on blood pressure, the question of statistical significance is really whether the drug achieved that decrease, or the decrease is due to chance.  A similar interpretation can be given of the econometric example.

These two concepts do interact.  For example, one may have achieved a large effect size but it is not statistically significant- that is, the effect measured is likely a function of chance rather than a true representation of a difference in the population.  On the other hand, one could have an extremely small effect size that is statistically significant, but in practice is not relevant for further research.

What is the difference between a statistically significant result and a clinically or real world significant result? Give examples of both.

The difference between a statistically significant result and a clinically significant result is important.  For example,  a clinical study has shown that a drug leads to, on average, a 1 point in HDL cholesterol among users.  This result has a p-value close to 0, meaning that it is likely due to the drug rather than the chance.  While this result is highly statistically significant, it is not clinically significant: this is because although it is likely due to the true effect of the drug, it is not clinically useful- a one reduction in HDL in any population is clinically significant where the medication would help individuals lower heart disease risk.

On the other hand, there may be a clinical study that shows that a statin has the impact of lowering HDL cholesterol, on average, 17.5 points.  This is a clinically significant result; in fact, an average reduction in bad cholesterol of 17.5 points could mean the difference between a heart attack and health to numerous individuals. While this result is clinically significant, the p-value of the relationship is .56- meaning that the majority chance is that this result is due to chance rather than the drug’s actual impact.  This is typically the case in introductory studies on a small number of patients.

What is NHST? Describe the assumptions of the model.

Null hypothesis statistics testing is a common methodology used in statistics.  The main assumptions are model: 1)normal distribution; 2) use of p-values.

Describe and explain three criticisms of NHST.

Three main criticisms of NHST include:

• The use of p-values is not always useful, especially for the research community. A significant p-value is the goal of all scientific researchers: that p-value, in theory, shows that their results actually exist rather than just chance.  However, the belief that a p-value can “prove” a hypothesis is often times false: This is especially true when a researcher is testing numerous variables (the multiple testing problem) and will get a significant p-value due to chance.  While this result will pass the significance test, it is not solid science.
• The NHST is biased (not robust) against rare events. The use of a small example can bias results in a way that leads to the rejection of the null hypothesis and acceptance of the alternate hypothesis.
• Many of the assumptions used are not realistic. For example, many of the statistical tests that incorporate NHST use assumptions regarding the underlying distribution of sample that are often not met, and even if they are met, they are not corrected for appropriately leading to an incorrect conclusion.

Describe and explain two alternatives to NHST. What do their proponents consider to be their advantages?

Two main alternatives to the NHST are : 1) the use of confidence intervals; 2) Bayesian statistics.

• Confidence Intervals- Reporting the effect size along with a confidence interval may be the easiest way to mitigate the weaknesses of the NHST.  That is, a confidence interval gives a range of values for which the effect size may be (with a certain probability) rather than the NHST’s method of saying that the alternate hypothesis is true.
• The second main analytical line used is Bayesian statistics. Bayesian statistics, unlike the frequentist-based NHST, does not try to prove a hypothesis, but gives a probability of something being right that integrates new information along the way.

Which type of analysis would best answer the research question you stated in Activity 1? Justify your answer.

I think Bayesian analysis would have best answered the research question stated in Activity 1 because the methodology allows for the implementation of new information along the way.

Sources

Schmidt, F. (2010). Detecting and correcting the lies that data tell. Perspectives on Psychological Science, 5(3) 233-242.

Carver, R. P. (1978, August). The case against statistical significance testing. Harvard Educational Review, 48(3), 378-399.

Chapter 5 – The Importance of Effect Magnitude. (2003). In Blackwell Handbook of Research Methods in Experimental Psychology. Jackson, S. (2012): Chapters 8, 10