Multivariable Models, Essay Example

Abstract:

This paper describes two different multiple regression analyses conducted on a data set of 26 variables: A one-step linear regression and a stepwise regression.  The one-step linear regression found 13 significant variables with a r-squared of roughly .64; the step-wise regression found 2 significant variables and a r-squared of roughly .56.

Introduction

This paper will explore multivariable linear regression from two different perspectives.  First, the paper will look at crafting a predictive model for car sales with 26 different variables (both categorical and continuous). Rather than attempting to run a model with all 26 variables, a step-wise regression model will be used to assess which variables are statistically significant in predicting car sales.  Second, this paper will look at four different papers to assess how scholars in different academic fields use multivariable regression models to help inform analysis.

Data Analysis

The data analysis portion of the project is based on a data set (“Car Sales.xls): The data set contains a total of 26 different variables (including nominal and continuous) with a total of 158 observations. The data is given to answer the following question: What vehicle characteristics are (generally) predictive of car sales in the data set.

Before choosing the model to analyze, it is good practice to first look at descriptive statistics composing the data set.  In order to accomplish this, basic descriptive statistics of the data set are listed in Appendix 1.  Looking at the descriptive statistics, there are a few minor, but not substantial issues: While the variables level of variance are all in an acceptable range, the number of observations for “resale” may be a concern that it is under powered vis-à-vis other chosen variables.  This shouldn’t be a large problem. however, because resale is a dependent variable (rather than a predictor).  Another important question is whether there are enough data points to roughly 26 variables. Using guidance found in the text book as a guide, this data set does not meet the criteria of having 15 times the number of independent predictors; it does, however, meet the less stringent criteria of having a total number of observations that is roughly 66 (40+26). Because it is not possible to gather more data points, statistical analysis will be performed on this data set.  In addition, potential outliers should be examined in order to understand if any data points may bias results.  After a running a case wide diagnostic on SPSS, five potential outliers were found for the dependent variable (sales).  Although these outliers were outside the two standard deviation range, they were left in the analysis in order to test how the characteristics of these cars explained the variance.

Figure 1- Outlier Analysis

Finally, the residuals of the analysis should be examined in order to understand if the underlying distribution is normal.  In order to assess this question, a p-p plot was produced (Appendix 4); overall, the variables follow a roughly normal pattern.

Another important pre-analysis exercise in multivariable regression analysis is understanding correlation between variables.  This is an important because if two variables share a high degree of correlation (usually defined as .8 or higher), multicollinearity may become an issue. Multicollinearity is when two or more variables share a high degree of correlation, and as a result, bias the coefficient estimators of the predictive variables. The Pearson correlation coefficient between sale price and the 26 variables, as well as resale price and the 26 variables is listed in Appendix B. In theory, variables with a high correlation that are significant will be included in the final model.

Once the preliminary descriptive exercises have been completed, the process of developing a predictive model begins.  There are numerous ways to conduct mulitiple regression- for this example, the model used allows the computer to choose the variables entered based on the correlation of the variables.  Although 25 variables were entered to assess which ones are significant in predicting the outcome, only 13  variables were ultimately selected to be included in the model (included below in Figure 1).

Figure 2- Multiple Regression Model

Overall, the thirteen selected variables were significant in predicting the sales of vehicles; the r-squared (or coefficient determination) for the model was quite high at .638, indicating that the variables selected  explained roughly 64% of the  variance in the dependent variable (sales).  The related variance statistics and coefficients are included in Appendix 3.

For didactic purposes, step wise regression was also run on the data set.  While the above model is built on a type of regression that includes all dependent variables at once and eliminates based on whether they are significant or not, the step wise regression produces a more parsimonious model; that is, it usually produces a model with fewer variables as it simultaneously enters the variable with the highest correlation into the model while selecting other variables if they lower the SSE and have a significant t-value.  The differentiation between the two different models is stark: While the initial multi-regression model selected 13 different variables with a coefficient determination of .634, the step-wise regression model selected two variables with a coefficient determination of  .564 (see Figure 3). While the initial multiple regression model has greater explanatory power, it is also quite difficult to fit the requisite 13 variables in the model.  Thus, the stepwise regression model provides a viable alternative that although possesses less explanatory power, is more parsimonious in that it only contains two variables.

Figure 3: Stepwise Regression

Use of Multiple Regressions in Academic Papers

A total of four different academic papers were examined that used different multiple regression techniques.  Johnson and Stahl-Moncada (2008) examine how restrictions in state Medicaid formularies (pharmaceutical lists) result in differences related to visits to the hospital and overall health care expenditures.  The  authors use Arizona Medicaid recipients and different formularies to understand the relationship between different drug consumption patterns and health outcomes.  There are two generalized linear models constructed: The independent variables included demographic variables such as age, sex, formulary restrictiveness, acute or long-term health plan recipient on the two outcomes (visits and health care expenditures).  The authors found that formularies with restrictive conditions experienced fewer visits and more hospitalizations- including greater expenditure on prescription drugs.  (Johnson & Stahl-Moncada, 2008).

Hyvonen  et al. (2010) use generalized linear regression in order to understand how mangers’ goal setting interacts with the perception of their psychosocial work environment.  In order to explore this relationship, two multinomial regression analyses were performed to investigate whether the components of their model (the ERI model-consisting of four independent variables- effort, reward, ERI-ratio, OVC) predicted membership to eight goal categories (Hyvonen et al, 2010).

Vijapurkat and Gotway (2001) extend the realm of generalized linear regression , which is usually used with data underpinned with a normalized distribution,  to non-Gaussian data.  The authors are then able to use that method in forecasting non-Gaussian time series data (Vijapurkat & Gotway, 2001).  There are three different statistical models tested: 1) a regression model with the latent process; 2) a regression model that used an approximation designed to make the necessary matrix inversions computationally easier; 3) A marginal quasi-likelihood regression approach.  Overall, the authors found that the quasi-likelihood predictor outperformed the other models, particularly in comparison to the size of mean squared errors.

Finally, Callen (2009) attempts to build on existing research that synthesizes and generalized the variance decomposition approach to firm level valuation.  In particular, Callen argues that shocks to returns are linear to earnings, under normal conditions (Callen, 2009).  In general, Callan builds on the existing model by adding error terms having  stochastic variances that impact current security returns, an extension on the VAR system (Callen, 2009).

References:

Callen, J.  (2009). Shocks to Shocks: A Theoretical Foundation for the Information Content of Earnings.  Contemporary Accounting Research, 26(1), 135-166.

Hyvonen, K., Feldt, T., Tolvanen, A. & Kinnunen, U. (2010).  Journal of Vocational Behavior. 76(3), 406-418.

Johnston, T.J. & Stahl-Moncada, S. (2008).  Medicaid Prescription Formulary Restrictions and Arthritis Treatment Costs.  American Journal of Public Health , 98(7), 1300-1305.

Hyvonen, K., Feldt, T., Tolvanen, A. & Kinnunen, U. (2010).  Journal of Vocational Behavior. 76(3), 406-418.

Vijapurkar, U. & Gotway, C.A. (2001).  Journal of Statistical Computation and Stimulation. 68(4), 321-329.

Appendix 1: Descriptive Statistics

Appendix 2: Correlations and Significance

Appendix 3