Program Background
The New Moves is a program designed to help adolescent girls to adopt healthy lifestyles. The program is designed to promote a healthy lifestyle through the development of helpful habits in the target populations. The majority of the focus students are from low-income families since these individuals tend to face the problem of excessive weight more often. The program started in 1996 in St. Paul Minnesota School District. In 2000 to 2001, an experimental study to assess the feasibility of implementing the program was conducted. The positive results of the study led to first implementation of New Moves program in 2005. The main purpose of the program is to address the psychological issues that obese girls encounter, such as low self-esteem, social isolation, stress, anxieties and depression (Neumark-Sztainer, 2005). In the current program, the stakeholders include the medical professionals, researchers, educators, the government, parents, and the obese girls.
The targeted population is 356 girls aged between 9 and 13. The funders include National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and National Center for Research Resources (NCRR) in conjunction with the ministry of health nutrition department. The program is administered during the physical education classes and should be carried out for nine months. There are individual coaching sessions that normally take 15-20 minutes. In addition to support nutrition classes. The program is based on a logic model that integrates trainers, food promotion, information about healthy lifestyle, physical activities, equipment, intellectual and capacity building processes.
The intended outcome is to make the girls fit, look good and motivated to continue with healthy lifestyles. This is achieved by increased physical exercise, counseling and motivational talks. The Heath Belief Model (HBM) forms the theoretical concept for the program. The theory stipulates that health behavior is determined by a personal belief and the strategies that are adopted to reduce the health risks or disease occurrence. For instance, in the New Moves the concept enhances peer support, goal setting and personal factors such as body image. This is in line with extensive formative and summative researches carried out in accordance with Social Cognitive Theory that have shown that the program improves the socio-environmental factors which help girls to adopt healthy lifestyles (Neumark-Sztainer et al., 2010). These changes results in increased physical activities, healthy eating and desired weight control behavior. Figure 1 below is the logic model for the program. Earlier evaluation studies carried out about New Moves rated it as an effective intervention for obese girls.
Evaluability Assessment
The HBM theory stipulates that health behavior is determined by a personal belief and the strategies that are adopted to reduce the health risks or disease occurrence. The current New Moves Program is designed to ensure that the girls adopt exercise and healthy eating habits in order to reduce obesity. The program is also based on behavioral change in order to ensure healthy lifestyles. Based on the program theory, secondary data analysis, and interviews with key participants such as the health instructors show that the program has been progressing as planned. The implementation of a program includes exploration, installation, initial implementation, full implementation and program sustainability (Msila & Setlhako, 2013). The New Moves program is in the sustainability stage, as the program has already been fully implemented. Sustainability of the program depends on the ability of the program to adapt to changes and priorities, and to attract continued funding.
The stability of the program is strengthened by the intended outcome. New Moves is designed in a participatory approach to help the girls to adopt enjoyable physical activities they can engage outside the class, and provides a supportive environment. The participatory nature of the program and the anticipated health needs contributes to its stability and reduces attrition rates. The New Moves has been received well by the girls, teachers, and parents. The girls are not coerced to participate in the program. In addition, the program has led to weight reduction among girls and boosted the girls’ confidence (Stame, 2004). The positive results achieved justify the costs for the evaluation to help program implementers determine the factors that contribute to success or failure (Stame, 2004). The data sources included primary sources, such as quantitative data obtained by measuring the weight of the participants on a weekly basis, and qualitative interviews. In addition, secondary sources of data were used, such as information on how similar programs have performed.
Purpose of Evaluation
The evaluation is intended to determine the impact of the program to the girls. It will help identify how the girls have responded to the program in terms of weight management practices and lifestyle changes. According to Fitzpatrick, Worthen and Sanders (2003), the findings of an evaluation help in establishing “what works” and “what does not work”. Thus, the findings will be used to establish the benefits of the program. The evaluation will also determine the training skills among the staff and their ability to deliver the services. The results will be applied to showcase the effectiveness of the program.
The parties interested in the results include the funders, the government agencies in charge of planning, the parents, civil society, the program implementers and United Nations agencies that deal with health and nutrition. In addition, the participating girls will need to know the progress of the program. Evaluations serve as tools to improve programs (Van & Brown, 1997). Thus, the information obtained can be used to streamline the future implementation of the program. Therefore, the evaluation is motivated by the key interest of finding out whether the program achieved the intended outcomes.
The Evaluand and Evaluation Purpose Statement
In the healthcare setting, the evaluand is the critical assessment, judgment and effectiveness of a project. The design of a good evaluation should start with the identification of the priorities and information needs of the key stakeholders (Msila & Setlhako, 2013). In New Moves, the factors being evaluated include acceptance level of the program, changes achieved, and training skill of the trainers. The evaluation will also focus on the complexity of the evaluand in terms of the timelines, the components of the program and the budget.
Therefore, the purpose statement of the New Moves evaluation is to establish whether the structural design of the program, the girls’ knowledge and behavioral change have improved as a result of the program and how the program has been embraced by the various stakeholders.
Evaluation Questions
The key evaluation questions that drive the evaluation are
- How many participants were recruited into the program and how did they respond to the intervention?
- To what extend were the expectations of the participants met?
- To what extent did the program decrease obesity?
- Which behavioral changes can be attributed to the New Moves program?
- To what extent does gaining knowledge about healthy lifestyle help to positive change?
The evaluation questions provide the specifications that help in the determination of the factors that are used to judge the progress of the program. The first question is about the program process. In the logic model, the question can be answered by the boxes for people and funders. The second question relates to the expected outcomes. Therefore, in the logic model, this question relates to various outcomes such as gaining knowledge about healthy lifestyles and the subsequent arrows that lead to decreased rate of students’ obesity. The other three questions are about the factors that facilitate the change and how they interlink to produce change as illustrated in the highlighted boxes in figure 1.
The third question (To what extent did the program decrease obesity?) attributes the outcomes to the program. This is critical in ascertaining that the changes obtained did not result from other external factors. The outcome can be ascertained by measuring the body weight and body mass index (BMI). The fourth question links the program process and the outcome. The question assesses how the programs processes lead to the expected change in lifestyle. The question probes the behavioral changes that can be attributed to the New Moves Program. Therefore, to get the answers to the questions, four parts of data will be measured. This will include multiple comparison groups, attendance of each participant’s activities and doses, frequency of attending the program and the log of doses for the three activities, self-report assessment. The fifth question links the program activities to the outcome. The question is designed to examine specifically how the sessions they contribute to the behavioral change. This is best ascertained through the indicators of lifestyle changes. The indicators for a healthy lifestyle include healthy eating habits, the level of physical exercise and self-esteem. In this case, self-reporting will be used as the strategy for data collection. Table 1 is a summary of the questions and data collection methods.
On the other hand, the first question is a program process question. The question is used in order to examine the stakeholders in the program and expected operation of the program. The second question is a program outcome. The question is used to probe the contribution of the program in changing the knowledge, behavior, health and the general practices of the students. Thus, it links the outcome to the expectation of the stakeholders. The evaluation questions were decided based on the preferences of the stakeholders, usefulness and the uncertainties in the program implementation.
Appropriateness of the Questions
The process/outcome questions are the most appropriate for both the formative and summative evaluation. According to Weiss (1997), the questions are normally framed based on the feasibility of the program. This is critical in determining whether the program is within the scope in terms of budget allocations and time. On the other hand, the outcome evaluation questions are aimed at establishing the consequences of the intervention. This helps in determining whether the real results have been achieved based on the timelines and budget allocation. For example, an assumption that $10,000 budget is required for the nine-month curriculum. Thus, the process/outcome questions will evaluate whether the program is running within the budgetary allocation and whether the set indicators are achieved within the stipulated timelines.
Evaluation Design and Methods
In order to answer the questions, both the quantitative and qualitative approaches will be used. The qualitative methods are used to provide the overview of the non-numerical variables. On the other hand, quantitative approaches entail the use of numerical data. The New Moves is intended to achieve weight loss among the girls. Therefore, the quantitative approach will be used to determine the weight changes over the nine months.
The evaluation design that that will be used is experimental design. The experimental designs normally have two types of study participants (Stame, 2004). The participants include the control group and the intervention group. At the end of the intervention, the outcome from intervention group is compared with the outcome from the control group. This ensures that the results are least biased.
Methods of Data Collection
The data collection will include individual interviews, use of structured questionnaires, observations, measurements, self-assessment reports and action research. The evaluation will rely on both the primary and secondary sources. The primary sources are data collection methods in which the information is collected directly from the study participants. Examples include the use of the interviews, questionnaires, self-assessment reports, observations, measuring body weight and BMI, and observations. The secondary sources include the data collection methods from other sources other than the participants. For example, use of recorded data from the internet or archival records.
Table 1 is a clarification of the data collection strategies in relation to the evaluation questions.
Table 1: Questions and Data Collection Strategy
The clients for the New Moves program will be involved in the evaluation process through a voluntary participatory manner. A written consent will be obtained from the clients. The willingness to participate will ensure that that the findings are objective.
Answering the Evaluation Questions
The data collected will be categorized into qualitative and quantitative data. The data will then be analyzed. This will provide the descriptive statistics that relate to the program process and outcome. Based on the obtained results, inferences will be drawn to answer the specific evaluation questions.
Options of Designs and Data Methods
There are different types of evolution designs. The designs include the randomized experiments, non-experiments, and quasi-experiments. The randomized experiments or the experimental designs normally include intervention and control groups (Fitzpatrick et al., 2003). Data can be collected at the start and the end of the study in order to make comparisons between the two groups. Quasi-experiments also have a control group and multiple measures. However, they lack random assignment. This complicates the statistical analysis. The other option is the non-experiment design. This design lacks the control group; as a result, there is no parameter for comparison. Thus, the inferences drawn may be biased. The viable option for the evaluation of the New Moves is the experimental design. The design provides the basis for comparisons, and the results are likely to be least biased.
The options for data methods include the use of primary or secondary data sources. Primary methods of data collection include the use of interviews, questionnaires, observations and focused group discussions. The secondary methods include the use of stored or recorded data. In the case of the New Moves, both the primary and secondary data collection methods provide a good option for gathering the evaluation data.
Limitations
The program is expected to run for nine months. In the course of the program, the school administration may alter the timetable. As a result, the projected time may be affected. Thus, some training sessions may not be completed within the nine months. The other challenge may relate to the budgetary allocations. The costs of the services such as traveling expenses are affected by economic changes. As a result, the cost of some items may increase and hence the program may run out of funds. In relation to the personnel, the busy schedules of the health professionals may not allow them to adhere to all training sessions. In addition, some personnel may drop out of the program. Based on the program design and purpose, there is an assumption that there will be no limitation in relation to data collection and political interferences.
In order to overcome the challenges, the program is tailor-made to fit the needs of the girls. Therefore, time issues can be addressed based on an individual basis. The budgetary projections are made based on factors such as foreseeable inflations. Thus, possible price changes are planned at the onset of the program. In order to ensure program continuity, the program will have many trainers. This implies that a trainer who departs will be replaced immediately.
References
Fitzpatrick, J., Worthen, B., & Sanders, J. (2003). Program evaluation: Alternative approaches and practical guideline. New York: Pearson Education Inc.
Msila, V., & Setlhako, A. (2013). Evaluation of Programs: Reading Carol H. Weiss. Pretoria: Research Publishing.
Neumark-Sztainer, D. (2005). Preventing weight-related problems in girls. Minneapolis: Guilford Press.
Neumark-Sztainer, D., Friend, S., Flattum, C., Hannan, P., Story, M., Bauer, K., & Petrich, C. (2010). New Moves-preventing weight-related problems in adolescent girls: a group- randomized study..American journal of preventive medicine, 39(5), 421-432.
Stame, N. (2004). Theory-based evaluation and types of complexity. Evaluation, 10 (1), 58-76.
Van, V., & Brown, K. (1997). Evaluability assessment: A tool for program development in corrections. Cincinnati: University of Cincinnati.
Weiss, C. (1997). Evaluation: Roles for the Evaluator. Upper Saddle River, NJ: Prentice Hall.