Statistical Analysis for Vignette Data, Research Paper Example

ANOVA

Anova which stands for Analysis of Variance, it is method used in stats to calculate the differences of two or more than two means. The Anova used in this table is used in order to calculate each level of mean squares of independent variable (1). So, the mean square is more in the between the groups variable. It indicates that the data is more scattered in the first variable. Whereas in masters the table for Total2011 it has the total sum of square which is 52511,794 by adding the two groups i.e. within group and between group.

Multiple Comparisons

When we perfrom the statistical tests in large number , some P values will have the value less than 0.05, even all of the null hypothesis are true. The P value having 0.05 shows that there is 5% chance to get the observed result (4). From the data we can see that the significance of the mean difference is 0.05. The mean difference is calculated by subtracting I from J. In the chart the confidence interval is further divided in two categories which is upper bound and lower bound. So the value which has the highest significance indicates that they lie within the confidence interval curve (1). And the lowest significance doesn’t lie within the curve, it is outside the curve and we reject the values (5). In masters the forth totvignrec2008 has continuous 0 significance interval so the hypothesis of these values are rejected.

Total

The total chart shows that TotVignRec2008 has four variables that is 1,00, 2,00, 3,00, 4,00 and the number of values are also four which are 88, 83, 89, 87 and the subset for alpha is 0.05 which is more categorized into two columns indicated as 1 and 2 (4). If we compare these two columns the significance level of 2^nd column is highest which is 0.429 so the hypothesis must be accepted and it lies within the curve (6). In master the sample size is 48,990. The two subsets defines that the data of 1, 3 and 2 is different from the data of 4.

Graph

In the graph of bachelors there are two variables. Totvignrec2008 lies in X-axis (horizontal) whereas Means of total lies in Y-axis (Vertical). The line is not straight which means the data is scattered.

Regression

Variables entered/removed

Regression analysis used in statistics is the process to determine the relations among variables. It has various techniques for analyzing and modelling several variables. Focusing the relationsip within dependent variable or more than one independent variables (2). This chart shows two important columns of variables entered and variables removed. There is nothing in the column of variables removed and the variable entered contains totvign2008. It also tells that in the column of method the entered variables are taken (1). The masters explains the same in the regression.

Model summary

In the table of model summary, there are two parameters R and R square. R is basically used in correlation analysis also called correlation coefficient (1). This parameter is used to measure the linear relation for two variables. Its value ranges from -1.00 till +1.00. if the value is of 1.00 positive or negative it indicates that the correlation is positive (3). So the table shows 0,239 value and its is a perfect correlation. Whereas R square is used in Regression analysis which is also called coefficient of determination (4). This parameter is used to measure the strength of relation for two variables (1). R square Value ranges from 0 to 1 where 0 is poor predictor and 1 is excellent predictor. In the table the R square is 0,057 which is considered as the poor predictor. As in masters the R is 0.44 and R square is 0.194 so this shows that above model summary for bachelors explains same as in masters.

Anova

The Anova table shows that in the column of model regression is summing up with the residual and gives the total value (4). The sum of squares for regression has the value of 6877,011 adds with the sum of square of residual which is 113849,171 and we gets the result of total 120726,182. The total degree of freedom (df)  346 having significance level of 0.000 (3). For the masters the sum of square for both residual and regression is 52511,794 with the df of 195.

Coefficients

Once we are done with the evaluation of R square, it is necessary to calculate the coefficients which are standardized and un-standardized (5). The bachelor’s coefficients of beta can be positive or negative and also have a t-value which in the table is 4,565. The un-standardized coefficients are divided in two columns one is indicated as B and other is Standard Error. If the coefficient beta is not significant statistically (which means that value t is not significant) then we cannot interpret statistical significance (3). Here in the table the beta value is positive then the interpretation says that for every single unit will increase the dependent variable which here is total and this will be increased through un-standardized coefficients.(6) In masters the t value of TotVign2008 is 6,834 with the level of significance 0.000

Graph

The bachelors graph lies in between the horizontal axis which is TotVign2008 and vertical axis of Total. Here it shows that the data is very much scattered as the significance of level and the variation is very high (2). The points or the data are scattered randomly in the below graph, which shows that there lies no relation between these two variables (5). It further means that there is zero correlation or low correlation between two variables.

There is difference in graph of masters and bachelors, as comparing it with the bachelors, the student of masters have high number of total and the data shows is very much scattered and this indicated that there is zero correlation between these two variables(3).

Vignette

Anova

We are analyzing the anova in vignette test as we have two samples here and we are using the t-test to differentiate the sample means but the result sometimes can be unreliable in case is other than two samples. When the two means are compared then t-test will occur the same result (4). In this table it again describes about the two groups which are between groups and within groups but the only difference is the columns are sum of squares, mean square and in frequency. If comparing both the anovas it shows the decreasing in the values but keeping significance level constant (5).

Multiple Comparisons

We do this test when means equality is rejected. It is a popular way in investigating the cause when null hypothesis is rejected. It is a method which compare or examine the means or proportions of one or more pairs in same time (5). This approach to multiple comparison tells that to set the critical level for significance to 0.05 instead of that we use lower value of critical(6). If the tests are true for null hypothesis, for getting a result the probability will be new significant and the lower value for critical will be 0.05.

Total

Variance measures that how data should be distributed by its mean and expected value. Its not like range which only consider the extremes. The variance will look in all data points as shown in graph and it will determine the distribution (3). The difference in previous and this total is that in previous one the second column has the highest value, but here the significance level of 1^st column is highest which is 0,618 so there is un-equality in both groups (6).

Graph

The graph lies in between the horizontal axis which is vignrec22008 and vertical axis of Mean of Total. Here it shows that increase in Vignrec22008 will increase the mean of total. This shows that there is a positive relationship between these two variables.

In the below graph the point s or the data creates a straight line, so that means linear relationship in between these two variable is higher in correlation and stronger. It also shows the dependent relations between two variables as X axis is increasing with the increase of Y axis.

Regression

Variables Entered/Removed

This chart again shows two columns of variables entered and variables removed(4). There is again nothing in the column of variables removed and the variable entered contains vignette2008. It also tells that in the column of method the entered variables are taken (2).

Graph

This graph for bachelors has the horizontal axis which is Vignette2008 and vertical axis of Total. Here again it shows that the data is very much scattered and the points or the data are scattered randomly in the below graph, which again shows that there lies no relation between these two variables(1)

In the masters graph shown below, the data or the points are scattered but at the high value of total as comparing with the graph of bachelors nevertheless both determines the same relation between the two variables lying in X and Y axis.

Stepwise Regression

Variables Entered/Removed

In statistics the stepwise regression has models of regression where the option of predictive variable moved through an automatic procedure (2). Normally this has seuqence of t-test or f-tests. Here in this method of regression the key method is to use F=t-squared (5). There are two variables in variable entered one is totvign2008 and vignette 2008. For the first variable the probability of F to enter is 0.05 and the probability of F to remove is 0.100. Similarly the same is for the second variable

Excluded Variables

Co-linearity is when two variables of predictor for example x1 and x2 in multiple regression has no zero correlation and in the given table the tolerance is 0.869 whereas partial correlation has the value of 0.117 (3). Likewise in masters the partial correlation has the value of 0.127 which is slightly greater than in bachelors and the correlation tolerance is 0.664 which is smaller than the bachelors.

Full Progress Testing :

The progress testing for all students is 4 x per year , 200 MCQ’s taken by student of 5 medical schools in the Netherlands. the first about halfway through first semester, the second about halfway through second semester.

For both of the tests in year 2004 and the first test in year 2005 are the used tests that were identical to those used recently so that we could assess comparability between the students. There was consistently only a slight difference (usually within about 5%, with mean scores in the clinical disciplines sometimes higher, while scores in the basic medical sciences were consistently higher among the advanced clinical students, probably because of their repeated exposure to questions from the basic sciences in their previous progress tests). From the second test in 2005, The questions from the join construction , administration at the same time , same standards , same rules and regulation for the tests .

From 2004 through 2006, progress testing ran in this fashion. The tests were ‘compulsory’, but no penalties were imposed on students who failed to sit and no one other than the student him- or herself saw the individual results. Nearly 100% of students sat the tests. Results were reported as ‘raw’ percentages correct. There was no correction for ‘guessing’. The pattern of results was as expected: higher scores on a given test by classes further into the course and increasing scores by any class as it sat further tests. (See Figures  1 and  2 for recent examples.) It was left to individual students to respond to their own results, although they were encouraged to seek advice or help if they found that their results were consistently well below those of their classmates, either overall or for particular disciplines or subdisciplines. As part of the individual feedback, each student received a graph showing his or her total scores on all tests sat to date, superimposed on a zone showing the mean ± 1 standard deviation for his or her class for the same tests. (See  Figure 2 for an example.)

The most striking and unexpected observation from these first years of progress testing at was the very high correlation between students’ total scores on the last progress test in fifth year and their scores in the total written component of the fifth year final examination. This was despite the facts that:

• the progress test MCQs bore no resemblance to the fifth year exam questions (which were extended matching multiple choice and short answer types)
• the content of the progress test questions was not selected to match the objectives of the fifth year exam
• students did no preparation whatever for the progress test, while they most certainly did for the fifth year exam.

From the pilot study in 2003 through 2006, the correlation coefficient between results on the last progress test in fifth year and performance in the fifth year written final examination was between 0.74 and 0.79. It reached the stage of being able to tell students that, if their progress test results kept pace with the rest of their class and if they were able to score over about 65% overall on the last progress test before the fifth year exams, they could be confident that they would almost certainly pass the fifth year written exam.

Paradoxically, this ongoing consistently high correlation was a bit disturbing. We hope that it might fall somewhat as students discovered from their progress tests the areas in which they had deficiencies and proceeded to rectify these before the final exams. This did not seem to be happening.

Evaluating and Modifying the Progress Test

During 2006, the medical education support team undertook an evaluation of the progress tests. Students reported being generally satisfied with the tests (two-thirds of respondents favoured retaining the tests), but they were concerned by the amount of time required to complete the tests.

The number of questions per test has been halved, while still retaining the same blueprint. (In a split test experiment, we showed that cutting the numbers of questions had little effect on the predictive ability of the test at fifth year level.)

The change in format of the test has removed the previous complaints from teachers and students. In a more recent evaluation by the medical education support team, however, there are now some new ones. About 15% of students admit to looking up answers (on the internet or in books) before answering. While this might arguably be considered a form of learning, the students have repeatedly been reminded that the main purpose of the test is to gauge the student’s developing fund of knowledge in relation to that of the other students in his or her class. But the correlation between performance on the last progress test in fifth year and the written final exam score has fallen considerably since the change in 2007 (to about 0.65 in 2007 and further to about 0.53 in 2008). It would be nice to think that this was because students were using information from the progress tests  constructively, but the pattern of results suggests it is more likely to be the effect of some students looking up answers and others not taking the progress test so seriously (see Figures  3 and  4 for scatter diagrams from 2005 and 2008).

For the present, progress testing continues as it has since the start of 2007. But anything might happen. Watch this space!