Evaluation of Service Learning Programs

To continue and expand funding from legislative initiatives that began in the 1990s, several organizations have collected and reported data regarding the positive impact of service-learning programs for students, community organizations, and recipients of service in communities. Unlike community service projects, service-learning endeavors connect students' service experiences to their school work. The article summarizes evaluation studies, describes strengths and weaknesses of program implementation, and references organizations and websites from which stakeholders can access current information about expanding and sustaining service-learning programs.

Keywords Civic Engagement; Civic Responsibility; Community-School Partnership; Community Service; Public Purpose of Education; Service Learning; Social Responsibility; Standards-Based Curriculum

Service Learning > Evaluation of Service Learning Programs

Overview

Service-learning takes community service a step further by connecting the service performed in the community to school work. Several initiatives were adopted in the 1990's to support increased student involvement in their communities and, by the end of the century, involving America's students in community service to prepare them for responsible citizenship was Goal 3 of the "National Education Goals for the year 2000" (Kleiner & Chapman, 1999).

Legislation & Funding

To support service-learning projects in communities throughout the U.S., the National and Community Service Act of 1990 established the Learn and Serve America Program to fund projects that enable students to make meaningful contributions to their community while strengthening their educational skills and developing an awareness of social responsibility. In the last decade, an increasing number of schools “have provided service-learning opportunities for students that connect their curriculum studies to service activities such as tutoring younger children, adopting a river, creating a museum exhibit, or conducting oral histories with senior citizens. In these and similar instructional activities, youth have simultaneously learned to serve and served to learn, becoming both better students and better citizens” (National Commission on Service Learning, 2002, p. 2).

The 1990 legislation created a program “to award grants to states, schools and community organizations to develop and implement service-learning. The National and Community Service Trust Act of 1993 expanded the federal role in service-learning and provided funds for every state to incorporate service-learning into schools. In 2000, $20 million in funds were distributed by the Learn and Serve America program to support local service-learning efforts. Other funding sources include the W.K. Kellogg Foundation, the Carnegie Corporation of New York, Dewitt Wallace-Reader's Digest Fund, The Ford Foundation, Ewing Marion Kauffman Foundation, Charles Stewart Mott Foundation and Surdna Foundation” ("Service-Learning at a Glance," 2007, p. 30).

Learn and Serve America (www.servicelearning.org) defines service-learning as a “teaching and learning strategy that integrates meaningful community service with the school curriculum to enrich the learning experience, teach civic responsibility, and strengthen communities” ("Service learning is," 2007). In the opening to a report by the National Commission on Service-Learning (NCSL), Chairperson John Glenn referred to an ancient saying to articulate the concept of service-learning -"I hear, I forget. I see, I remember. I do, I understand." In addition, he suggested that service-learning can add a fourth "R"-which stands for "responsibility"-to the traditional three R's of education (cited in National Commission on Service Learning, 2002).

Benefits of Service Learning

Brown (1998) contends that students gain confidence, competence, and the ability to empathize for others through service-learning. Additionally, they build employment skills by engaging in problem solving and by working cooperatively and collaboratively with others. Brown also explains that an important enhancer of service-learning is a stronger connection between a school and outside organizations that help and benefit the community. For example, community members who volunteer as partners in service-learning and those who receive service often modify their opinions about adolescents, seeing them as crucial resources and contributors rather than as troublemakers. Service providers join together to impact a larger community, refrain from replicating the same efforts, utilize resources effectively and contribute more broadly to the healthy development of youth.

The 2002 NCSL report also cites several studies indicating that service-learning:

• Reverses student disengagement from schooling by increasing their motivation to participate in school activities focused on identifying actual community needs;

• Reinforces the standards-based curriculum by providing a real-life context for learning;

• Promotes the public purpose of education by preparing students for citizenship;

• Builds on the growing willingness of students to become involved in service to their communities while adding an academic component to such service; and

• Contributes to young people's personal development by increasing their sense of responsibility and to their career development by building workplace skills (National Commission on Service Learning, 2002, p. 4).

In addition to Learn and Serve America, several organizations provide information and resources for beginning or enhancing service-learning programs. They include Learning in Deed (www.learningindeed.org); National Service-Learning Partnership (www.service-learningpartnership.org); and Youth Service America (www.ysa.org). Service-learning is first and foremost a locally-driven form of education, and the majority of power and decision-making occurs at the local level. Advocates for service-learning programs include community leaders, students, parents, school superintendents and teachers across the country.

Applications

Incorporating Evaluation Strategies

Creating an ongoing and effective service-learning program requires careful planning, evaluation, and revision based on the evaluation data. The Corporation for National and Community Service stresses the importance of ongoing program assessment:

Step 9: Assess and Evaluate Your Service Program

“Ensure that your evaluation assesses the outcomes of the service project for the youth, the community, and the organizations involved. Documentation and evaluation of the project will create a legacy for the individuals and organizations that participated. It will also point the way to the next project for your classroom, and may foster activities in other classrooms” (Corporation for National and Community Service, 2002, ¶ 4).

While service-learning has been demonstrated to be an effective educational strategy and can have positive effects on students' academic performance, social development, civic engagement, and career knowledge, any individual program may or may not be successful in achieving its desired goals.

Evaluation is a way to ensure that programs are effective, to understand which ones to keep up with, how to make them stronger, and which ones to terminate. The National Committee on Service Learning (2007) has made available the "Educators' guide to Service-Learning Program Evaluation" to facilitate the evaluation process.

Another evaluation resource is available from the Compendium of Assessment and Research Tools (CART) (www.cart.rmcdenver.com). The CART website provides information on research implements, tools, and guides that offer assistance to people who are interested in learning about the effects of service-learning, safe and drug-free communities, and school-based adolescent growth activities.

Analyzing Program Assessment Research

Billig (2000) prepared a summary of research on service-learning programs that were implemented between 1990 and 1999. At the time of her analysis, only a few studies revealed that no impact and no sustained impact occurred over time. Billig's report organized the research findings into six areas of impact, each supported by statements from the evaluative studies. Her report cites the research from which supporting summary statements were drawn.

Impact on Student Personal and Social Development

Service-learning has a positive effect on the personal development of public school youth, supported by these statements from program evaluations:

• Middle and high school students who engaged in quality service-learning programs showed increases in measures of social responsibility, communication and sense of educational competence;

• Students who engaged in service-learning ranked responsibility as a more important value and reported a higher sense of responsibility to their school than comparison groups;

• Students who engaged in service-learning were more likely to treat each other kindly, help each other and care about doing their best;

• Students who engaged in service-learning were more likely to increase their sense of self-esteem and self-efficacy;

• Students who engaged in service-learning were less likely to be referred to the office for disciplinary measures;

• Students who were engaged in service-learning were less likely to engage in behaviors that lead to pregnancy or arrest;

• Middle and elementary school students who participated in service-learning were better able to trust and be trusted by others, be reliable and accept responsibility;

• Students who participated in service-learning enjoyed helping others with projects, became more dependable for others and felt more comfortable communicating with ethnically diverse groups (Weiler, et al., 1998; Leming, 1998; Scales & Blyth, 1997; O'Bannon, 1999; Morgan & Streb, 1999; Berkas, 1997;Shaffer, 1993; Switzer et al., 1995, as cited in Billig, 2000, p. 11).

Impact on Civic Responsibility

Service-learning helps to develop students' sense of social responsibility and their citizenship skills, supported by these statements from program evaluations:

• Students who engaged in high quality service-learning programs showed an increase in the degree to which they felt aware of community needs, believed that they could make a difference and were committed to service now and later in life;

• High school students who participated in high quality service-learning programs developed more sophisticated understandings of socio-historical contexts, were likely to think about politics and morality in society and were likely to consider how to effect social change;

• Students who engaged in service-learning increased their understanding of how government works;

• High school students from five states who participated in high quality service-learning programs increased their political attentiveness, political knowledge and desire to become more politically active;

• Over 80 percent of participants in high quality service-learning programs felt that they had made a positive contribution to the community (Melchior, 1999, Berkas, 1997; Youniss, et al., 1997, Yates & Youniss, 1998; Morgan & Streb, 1999; O'Bannon, 1999, Cairn, 1999; Melchior, 1999; Billig & Conrad, 1997; Scales & Blyth, 1997, as cited in Billig, 2000).

According to Billig's research summary, service-learning also aids students in acquiring academic abilities and knowledge, career exploration and aspirations, greater mutual appreciation for teachers and students, and more positive outlooks on school, adolescents, and education by community members.

To support continued funding and to build upon the effectiveness of service-learning programs, the "Growing to Greatness (G2G)" organization was established to provide annual follow-up reports to the Billig 2000 research summary. G2G brings together a variety of service-learning information including state-by-state detailed accounts of service-learning activities, profiles of national organizations, and specific examples of how many different ways exist that can contribute to the helping of society and the community. Sponsored by the State Farm Companies Foundation and affiliated with the National Youth Leadership Council, news and reports from G2G can be accessed from www.nylc.org/programs.cfm.

Viewpoints

Learn & Serve America

An important and dominant goal for the national Learn and Serve program has been to expand permanent school and community-based service-learning programs by supporting the alliance between service-learning and academic curriculum and classroom teaching. Between 1994 and 1997, the Center for Human Resources at Brandeis University (1999) evaluated the national Learn and Serve School programs. Their study examined programs at seventeen schools nationwide. The Learn and Serve programs studied were picked to symbolize the capability of a well-designed, fully implemented service learning leadership. “All of the programs in the evaluation had been in operation for more than a year. While each program had its own strengths and weaknesses, together the programs represented serious efforts to bring the ideals of service-learning and the federal legislation into practice” (Center for Human Resources at Brandeis University, 1999, p. 2).

Results of the Brandeis study indicated that, generally, the Learn and Serve sites in the research conducted achieved their goals to create and expand service-learning programs. “In fifteen of the seventeen sites the service-learning programs were in operation through the follow-up year, and all fifteen appeared likely to continue beyond the end of their Learn and Serve grant period. The study of the seventeen sites highlights some of the difficulties involved in integrating service more broadly into the school curriculum. For most of the schools, and particularly those in which service was focused on a single class or program, involvement in service-learning was concentrated among a small group of teachers. Relatively few of the sites had organized efforts to expand the use of service-learning in the school, and few teachers had received any formal training” (Center for Human Resources at Brandeis University, 1999, p. 23).

The lack of broader integration did not appear to be the result of opposition to service-learning, but more likely the result of a host of barriers to institutional change in the schools. Based on interviews with teachers, program staff, and administrators, the Brandeis study reported these barriers:

• Lack of funds and available time for professional development (often less than one day per quarter);

• Competing professional development priorities;

• Concerns about meeting new content standards and graduation requirements;

• Lack of planning time for teachers;

• Logistical problems and inflexible school schedules; and

• A continued emphasis on "community service" over "service-learning" (Center for Human Resources at Brandeis University, 1999, p. 23).

The Brandeis study has several conclusions regarding policy and practice of service-learning. The results show that well-designed, fully-implemented programs have a strong and affirmative impact on their members and societies. The study recommended that the Corporation for National Service and its grantees must continue to emphasize the improvement of the value of local service-learning programs, whether through professional development and continuous work on growth and dissemination of work regarding "best practices."

The evaluation results also suggest “the need for continued research on the longer-term and cumulative impacts of service-learning. While the evaluation found clear short-term impacts from program participation, the findings from the follow-up study raise the question of how schools and communities can structure their programs to extend impact over a longer period of time” (Center for Human Resources at Brandeis University, 1999).

Terms & Concepts

Civic Engagement: This term refers to the work involved in creating a difference in the society and community of local, regional, or national communities to which one is a part of as well as in advancing the combination of education, abilities, values and inspiration to make that change.

Civic Responsibility: This term refers to working toward the betterment of one's society and surroundings through public volunteerism and other efforts to help the community and the lives of all the citizens involved.

Community-school Partnership: This term refers to community organizations and schools working together to plan and implement projects that benefit the members of the partnership and the entire community.

Community Service: This term refers to tasks a person does for the benefit of his/her community.

Public Purpose of Education: This term describes the belief that public education should teach children the skills and expertise necessary to succeed in school, the workforce, and life in general, including the development of skills needed to be a good citizen at the private and local level to the public and governmental level.

Service Learning: This term describes a strategized way of teaching that includes community volunteerism mixed with instruction and analysis that is intended to advance the educational experience, further the sense of civic obligation and aid, and improve communities.

Social Responsibility: This term describes an ethic theory that an entity (whether it is a government, corporation, organization or individual) has a duty to the society in which it operates.

Standards-based Curriculum: This term refers to a school curriculum that defines what students should know and be able to do; clear, measurable standards are used to guide all other educational system components including instructional content, assessments and professional development.

Bibliography

Abravanel, S. (2003). Building community through service-learning: The role of the community partner. Retrieved November 7, 2007 from http://www.servicelearning.duq.edu/documents/Role%20of%20Community%20Partner.pdf

Billig, S. (2000). The impacts of service-learning on youth, schools and communities: Research on K-12 school-based service-learning, 1990-1999. Retrieved November 7, 2007 from Lerningindeed.org, www.learningindeed.org/research/slresearch/slrsrchsy.html

Brown, B. (1998). Service learning: more than community service. Retrieved November

Brown, B. (1998). Service learning: more than community service. Retrieved November 7, 2007 fromhttp://www.ericdigests.org/1999-2/service.htm

Center for Human Resources at Brandeis University (1999). National evaluation of learn and serve America summary report. Retrieved November 7, 2007 from Learn and Serve America, http://www.learnandserve.gov/pdf/lsa_evaluation.pdf

Corporation for National and Community Service, (2002). Designing a service-learning program in ten steps. Retrieved November 7, 2007 from website http://nationalserviceresources.org/epicenter/practices/index.php?ep_action=view&ep_id=801

Educators' guide to service-learning program evaluation. (2007). Retrieved November 7, 2007 from Learn and Serve America Clearinghouse http://www.servicelearning.org/filemanager/download/37/EvaluationToolkit.pdf

Kleiner, B. & Chapman, C. (1999). Service-learning and community service among 6th- through 12th grade students in the United States: 1996 and 1999. Retrieved November 7, 2007 from U. S. Department of Education http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2000028

National Commission on Service Learning. (2002). Executive summary. Learning in deed: The Power of Service-Learning for American Schools. Retrieved November 7, 2007 from Learningindeed.org http://www.learningindeed.org/slcommission/executive_summary.pdf

National Youth Leadership Council. (2004). Growing to greatness: The state of service- learning project 2004 executive summary. Retrieved November 7, 2007 from http://www.nylc.org/rc_downloadfile.cfm?emoid=14:225&property=download&mode=download

Pierce, M. B., Havens, E. K., Poehlitz, M., & Ferris, A. M. (2012). Evaluation of a community nutrition service-learning program: Changes to student leadership and cultural competence. NACTA Journal, 56, 10-16. Retrieved December 15, 2013, from EBSCO Online Database Education Research Complete. http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=85114473&site=ehost-live

Service learning is. (2007). Retrieved November 7, 2007 from Learn and Serve America Clearinghouse, http://www.servicelearning.org/what_is_service-learning/service-learning_is/index.php

Service Learning at a Glance. (2007). Retrieved November 7, 2007 from Learningindeed.org http://learningindeed.org/tools/glance.html

Smith, B., Gahagan, J., McQuillin, S., Haywood, B., Cole, C., Bolton, C., & Wampler, M. (2011). The development of a service-learning program for first-year students based on the hallmarks of high quality service-learning and rigorous program evaluation. Innovative Higher Education, 36, 317-329. Retrieved December 15, 2013, from EBSCO Online Database Education Research Complete. http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=66903912&site=ehost-live

Suggested Reading

Billig, S. (2004). Heads, hearts, and hands: The research on K-12 service-learning. http://www.civicyouth.org/PopUps/Billig_Article2.pdf

National Commission on Service Learning (2002). Learning in deed: The power of service-learning for American schools. http://www.wkkf.org/pubs/PhilVol/Pub3679.pdf

National Youth Leadership Council (2003). State Farm state of service-learning project. Generator, 21. http://www.nylc.org/objects/inaction/initiatives/GenG2GIntroFINAL.pdf

Rizzo, M. & Brown, J. (2006). Building character through community service: Strategies to implement the missing element in education. Maryland: Rowman & Littlefield Education.

U.S. Department of Education, the Corporation for National and Community Service, and the Points of Light Foundation (2002). Students in Service to America: A Guidebook for Engaging Students in a Lifelong Habit of Service.http://www.studentsinservicetoamerica.org/download/guidebook.pdf