Correlation and Regression Analysis, Coursework Example

What is the research problem?

The larger research problem the study addresses is: how does the effectiveness of health care outcomes  differ when administered by different health professionals?

Why is the potential public health significance and impact of this research in healthcare?

This research paper’s potential public health significance and impact on health care is substantial.  The cost of health care in the United States is rising annually, far exceeding the rate of inflation. Part of the reason for the price increase are costs associated with labor; if doctors see every patient, there are questions regarding the cost and efficacy of such a decision.   If there was robust evidence, however, that nurse practitioners, which charge less, could deliver similar health outcomes at a lower cost, it would likely change how health care is delivered.

What are the specific aims of the research?

The specific aim of the research is to assess the differences of patient outcomes treated by the primary doctor or nurse practitioner.

Who is the target population for whom the inferences will be applied?

Judging from the study’s sample population, the target population was adult patients with a need for health care, particularly for primary care that was oversampled in this study.

What is the primary research question(s)?

The primary research question is: is the care offered by doctors and nurse practitioners in a similar environment with similar responsibilities equivalent?

How is the following applicable to the design and hypotheses?

A non-inferiority or equivalence trial aims to demonstrate that the experitreatment is not clinically worse than the comparator (an active control treatment) by more than a pre-specified small amount, known as the non-inferiority or equivalence margin.The new treatment is expected to have non-inferior or equivalent efficacy compared with the standard treatment but could have advantages in safety, convenience, or cost. The new treatment may also present an alternative or second-line therapy.

In healthcare, the goal may be to look at the degree of equivalence between something new and something established – rather than superiority.  The goal of these trials is to show that the new intervention or practice is statistically (and clinically) not inferior to the standard approved intervention or practice. Instead of testing for equivalence, the use of a non-inferiority experimental design is a useful technique to accomplish this goal.

In this study, the authors are looking at the degree of equivalence between the standard treatment (doctor care) versus nurse practitioners (the new treatment).  The research design is not meant to prove superiority due to the limitations of the study subjects as well as the emphasis on primary care.  The hypothesis ties into this goal by only looking at “similar” and “non similar” outcomes between the two health care practitioners. Further, statistical tests used in the analysis such as chi-squared do not prove superiority of treatment, they only look at differences in the results.  Ad hoc tests would be needed to further assess outcomes.

What is the primary research (alternative) hypothesis?

State others if there is more than one primary hypothesis.

The primary research hypothesis was: there would be similar patient outcomes when comparing nurse practitioners with physicians when both were servings as primary care providers with the same authority.

The alternate hypothesis was: there would not be similar patient outcomes when comparing nurse practitioners with physicians when both were servings as primary care providers with the same authority.

What is the research design?

The research design is a randomized trial.

Is this a cross-sectionalor longitudinal study?  Explain your answer.

This study is a cross-sectional study; that is, the study follows patients for a time period of 6 months assessing differences over a period of time.  A cross-sectional study is a study done at a point in time- all data is taken during one period.

What is the phase of research?

This is the initial phase of research to delineate whether there is a relationship between the variables.

Does the designadequately address the primary research hypothesis (or hypotheses)?

Yes; the hypothesis aims to measure the differential in patient outcomes between nursing practitioners and physicians.  In order to address that specific hypothesis, the study must control for a number of confounding factors such as differences in health condition, access to care, etc.  By using a randomized control trial design, the randomization eliminates these potential confounders.

What are advantages and disadvantages of the research design given the research question(s)?

The major advantage of the research design (as mentioned above) is the controlling for confounders in assessing the relationship between health provider and health outcome.

There are no ex ante disadvantages to using the RCT format over an observational study.  If the randomization is broken, or the blinds are not adhered to, the results of the RCT would essentially have the same weight as an observational study.

What are the primary outcome measures? Be specific

There are three primary measures: 1) patient satisfaction after initial appointment (based on 15-item questionnaire); 2) health status (Medical Outcomes Study Short-Form 36), satisfaction and physiologic test results (6 months later), and service utilization for 1 year after initial appointment by service provider.

How well do these outcomes address the primary research question?

These outcomes are quite comprehensive in assessing how patient’s health status may differ under the care of a primary care physician versus a nurse practitioner.  In particular, the measures include self-reported outcomes (patient satisfaction and health status), as well as more objective measures such as service utilization 1 year after initial appointment.

What are key secondary outcomes? Be specific.

The study did not name any secondary outcome measures.

What are the inclusion and exclusion criteria?

Inclusion criteria: If they had no current primary care provider at the time of recruitment and planned to be in the area for the next 6 months.

Exclusion criteria: The primary exclusion criteria was a subject not being part of the Columbia Presbyterian system and not in the system for the next six months.

What sampling method is applied?

Although not explicitly discussed in the text, one would assume a simple random sampling method was used along with randomization.  Indeed, if one identifies the larger population being tested as individuals attending Columbia Presbyterian for care, SRS would be the correct answer.  If one defined the general population as the general population in New York City, the answer would not be SRS.

What is the sample size?

A total of 1981 patients were randomized; a total of 1316 patients kept their primary care appointments and were considered enrolled in the study.

How well does the sample represent the study population?

The initial sample taken was likely a fair representation of the study population.  That is, the initial sample included individuals from the demographic and diagnostic groups the researchers were interested in as part of the larger population.

What kind of randomization method is applied?

Once patients were deemed eligible, they were blindly randomized to one of two groups: 1) nurse practitioner or the physician practices.  There is no further information presented on randomization techniques used.

Why is the randomized group allocation ratio different?

The randomized group allocation was initially different because there were more openings in the nurse practitioner arm.  That is, the ratio during the recruitment phase was 2:1; as the number enrolled in the nursing practitioner’s panel increased, the randomization ratio was brought back down to 1:1. The authors also state they factored in the uneven ratio in their power and statistical analysis.

What is the potential impact of a treatment allocation change during the middle of study?

A treatment allocation change in the middle of the study would essentially break the initial randomization by which the individuals were randomized into two different arms.  One could still keep those individuals in an intention to treat analysis by using data from the first half and imputing their data points during the second half based on other individuals in the randomized group- it would not be advisable, however, to use the same individual’s data in both arms of the study.

How many subjects are randomized to each group?

There were a total of 806 individuals randomized in the nurse practitioner group; there were a total of 510 individuals randomized in the physician group.

Do the authors provide a consort diagram? If yes, is it useful?

Yes; the authors provide a consort diagram on page 64.  The diagram is quite useful because it not only provides important information on randomization, but also provides information on the different arms: those allocated to the nurse practitioner and physician arms.

Is statistical power for the testing the primary hypotheses addressed in the article?

Yes, statistical power concerns are addressed on page 62.  The authors arrived at the needed sample size based on estimates needed to detect a difference of 5 points on a 100-point scale for the SF-36 scores on all scales when comparing 2 groups with repeated measures. The threshold of a 5 point difference is considered to be clinically and socially relevant.  The power (or beta) of the study was 80%- a benchmark for clinical studies.

Does the study have sufficient statistical power to address the primary hypothesis?

Yes, with 80% power, the study has sufficient statistical power to address the primary hypothesis.

How is intent-to-treat (ITT) defined?

Although the study uses aspects related to an intent-to-treat analysis, particularly for the 159 patients who either went to another clinic than assigned or to multiple care clinics, the concept is not explicitly defined in the study.

Is the ITT principle actually applied in the primary analysis? Explain your response.

Yes, the ITT analysis is applied in the primary analysis as explained for the 159 individuals who essentially broke the randomization but were still included in the analysis.

Are there any systematic baseline differences in two randomized groups?

Yes, there were systematic baseline differences between the two originally randomized groups.  There were a total of 1981 patients initially randomized to the two different groups.  Of the initial 1981 originally randomized, only 1316 patients kept the initial appointment: Although there were no baseline differences between the 1316 patients who were officially “enrolled” in the study, there were baseline differences between those who did not attend the first appointment and those who continued on in the study.

Is blindness or masking an issue in this study?

It is difficult to maintain a double blind trial in this trial because the individuals clearly know (or they should) if they are seeing a doctor or nurse practitioner.  The researchers and data analysts, however, should be blinded in order to prevent bias.  Thus, blindness is an issue if it introduced bias into the study.

Do you have concerns regarding fidelity of the intervention?  One way to think about this question is to consider the impact of variations in the health care practices by clinicians providing treatment in the NP and MD groups at the different clinical sites.

Yes, there are concerns over the fidelity of the intervention used in the study.  In particular, there may be physicians or nurse practitioners of varying abilities in different treatment sites.  This may introduce extra variability in the results that did not arise as a result of an actual difference in the patient’s condition.

Are there any other elements of the study that threaten the internal validity?

There are potential threats to internal validity that exist.  For example, if subjects in the nurse practitioner group sought advice from doctors who were friends or acquaintances, this would arguably bias the results if they sought medical care/advice out of the group.

What are potential strengths of the methodological approach applied?

The randomized control trial is the “gold standard” or research methodology.  This is because randomization eliminates potential known and unknown confounders in the subjects.  Without confounding, many observational studies are unable to make firm causal claims about the relationship being tested.

What is the statistical analytic method applied for the primary outcome(s)?

There were a number of statistical analytical methods applied to primary outcomes.

• Regarding the analysis of SF-36 health outcome survey, there were a total of four different analyses conducted. 1) Two sets of t-tests were conducted to assess differences in health status at the 6-month follow up period between the two groups (physicians and nursing practitioners); The researchers also conducted a “sub-group” analysis of the sickest patients (based on the bottom quartile) comparing scores at baseline and six-month increments.  Finally, a test was conducted to detect differences in service offerings from bundled to “fee for service.”
• For the analysis of health service utilization, chi squared and Poisson regression was conducted on the two groups at three different data points: 1) 6 months before recruitment; 2) 6 months after recruitment; 3) one year after the first primary care visit.

Does the analysis approach address the primary hypothesis?

Yes, the analysis approach, which draws heavily on t-tests and chi-squared tests, is based on detecting differences in the physician group versus the nurse practitioner group.  These statistical tests address the fundamental differences tested in the hypothesis.

How useful are the tables and figures used to summarize the study results?

Due to the complex nature of the randomization and statistical tests carried out, the tables and figures were helpful in understanding the results.

Are effect sizes for the key outcome provided?

Yes, the effect sizes for the main outcome variables are provided in the charts and table.

Are the statistical methods used to address the secondary hypotheses and analyses appropriate?

There are no explicit secondary hypotheses explored in this study.

Do have any concerns about the subgroup analysis in terms of sample size and power?

The subgroup analysis is not powered at the same level to detect a 5-point difference in SF-36 as the main analysis.  Indeed, the subgroup analysis only has 162 subjects in the nurse practitioner arm and 103 patients in the physician arm.  This may be enough to detect a very large difference, but would not detect a small difference and probably should not be extrapolated to a larger population.

How generalizable are the primary analysis results to the study population?

The external validity of the study is primarily based on the similarity of characteristics of the study population.  Overall, I would posit that study population has a fairly robust similarity to the general population- the only caveat would be regarding the individuals who dropped out after randomization.  Although there was no difference in service utilization, there were baseline differences.

Are the limitations of the study addressed in the conclusions or discussion section of the report?

Yes; the report addresses limitations on page 68 of the report.

Do you have any concerns related to causal inferences in this study?

The causal inference in the study is fairly strong. This is primarily a function of the randomization used in the study.  Although there may be reasons to question whether the randomization was broken or not ( only 65% of the original subjects remained randomized), the study’s internal validity is fairly robust.

Do you agree with the authors’ interpretation of the results and conclusions?

Yes, the authors make very conservative conclusions that center around one question: who provides primary care and ultimately at what price?  Although this research study cannot definitively answer that question in any way, it provides an important foundation for further inquiry in which situations nurse practitioners giving primary medical care may save time and money.

Do you have any concerns related to study validity?

Internally, there may be contamination effects; externally, the study may not generalize to the general population that receives health care.  In addition, a majority of health care costs are  related to specialists- a study examining specialists would be helpful.

What is your overall impression of the study? (4 points)

Please is your impression in the context of the overall study validity, inferences drawn from the findings, and the potential impact or importance of the study for healthcare.

First, the study addresses one of the most important questions in health care: How can escalating costs potentially be reduced through substituting cheaper labor inputs in health care? Within that question, the authors choose an important comparison: the care offered by doctors versus nurse practitioners.

Second, although the study may have some validity issues, the causal inference is stronger than observational studies.  From an internal validity perspective, it is possible that subjects may have crossed over or received outside advice (“contamination). From an external validity perspective, the study’s results may not generalize to every population, but it does provide a foundation for further inquiry.