University of Florida: Program Evaluation, Coursework Example

I have chosen the CHOICES program from the University of Florida for the program evaluation project. Alachua county officials in Florida have commissioned the Florida Survey Research Center at the University of Florida to conduct a program evaluation of CHOICES.

In the program that you have been studying, are they collecting data on an ongoing basis? If so, what type of data are they collecting, and with what instruments or tools? Are these appropriate choices? Provide support for your answer.

There are two different sections of the program evaluation program: 1) Needs assessment which happens at baseline; 2) Program evaluation which is done through telephone surveys on an ongoing basis.

There are three primary tools used to collect information for program evaluation purposes: 1) Telephone survey (Choices currently enrolled and disenrolled); 2) Focus groups (enrolled in Choices); 3) Structured Interviews for participating health care providers and main stakeholders.

This is a fairly robust program evaluation method: On the one hand, the program is receiving solid demographic and scaled survey information that can be analyzed via regression and factor analysis. On the other hand, the program is also receiving robust qualitative information from all stakeholders involved in the process- this gives greater granularity to the quantitative data.

If your program does not have ongoing data collection, identify one kind of measurement instrument or tool you would use, and the type of data, for measuring the outcomes of your program. Explain your choice. If a measurement instrument or tool is not appropriate, explain what individual questions or items would be used.

• Ongoing collection is practiced in the CHOICES study.
• Briefly explain the concepts of reliability and validity in terms of measurement. Then, for either the current or proposed data collection method you described, explain what steps you would take to establish the reliability and the validity of this measurement instrument or individual items
• Reliability and validity are important concepts in understanding how to accurately measure things. Reliability refers to the ability of a survey to capture a similar result if used again, or if questions were asked in a different way. Validity, in contrast, refers to whether the survey is actually capturing the subject; that is, validity captures if the survey is actually measuring whether CHOICE improves health access and health outcomes.

Of the three instruments used in program evaluation, the survey instrument is the most amenable to reliability and validity measures. There are a number of ways to test reliability in the survey: one way is to have the members do another survey (test re-test validity); another way is to rephrase the questions in another way to see if a different response is gained.

Regarding validity, face validity, construct validity, and content reliability of the survey can be attested to by experts in the subject area.