|COMM-ORG Papers||Volume 17, 2011||http://comm-org.wisc.edu|
April 5, 2011
A Report Evaluation submitted to the Trent Centre for Community Based Education and the U-links Centre for Community Based Education
Despite the popularity of community-based education (CBE), the effects of CBE projects on host organizations and their targeted communities have not been well examined. The Trent Center for Community Based Education (TCCBE) and U-links have facilitated more than 1000 projects since the 1990s in Peterborough and Haliburton area. In our research, we used the TCCBE and U-links databases to select and interview 17 host organizations, a faculty member/host, and a representative of both TCCBE and U-Links – the brokers to answer our research question: what the direct and contributing effects of CBE projects are to host organizations and targeted communities. Our results show that many host organizations had had positive experience with CBE projects, which help them to enhance their functionality, fundraise, and improve public recognition. However, there are still limitations with time, communication and quality that TCCBE and U-links can address. We also suggest that a separate study is conducted to examine in depth the effects of community-based education on host organizations’ targeted communities.
Since the 1980s, community based education has been growing in popularity in many academic institutions across North America. In these programs, students work with community projects and host organizations as part of their academic curriculum and earn credits. Community-based education is strongly supported by post-secondary institutions because it gives students a unique opportunity to gain real life experience and to balance and understanding of theory with practice. In addition, community-based education is also an effective tool for helping universities prepare a better quality workforce for job markets. Numerous researchers have discussed the experience of students and faculty with service-learning projects, yet little has been written about the effects of such projects on host organizations and the community.
The Trent Centre for Community-based Education (TCCBE) and U-links have
been connecting students with community organizations in Peterborough and
Haliburton since the 1990s with more than 600 projects completed. In this
study, we examine the direct and contributing effects of CBE projects on
host organizations and communities in both Peterborough and Haliburton.
In our research, we conducted an extensive literature review to explore what has been discussed with regard to service-learning and community based education. We used the substance of our literature review to both identify our research direction and compare and contrast scholars’ observations with our own findings. In our methodology we employed archival research using the websites, databases and other materials obtained from TCCBE and U-links, and interviews, which we conducted with host organizations in both Peterborough and Haliburton.
Of the 17 interviewed host organizations’ representatives, 12 expressed a very positive attitude towards TCCBE and U-links projects, 3 had mixed feelings and 2 described negative experiences. We were also able to identify the reasons for successes as well as causes for failures based on both interviews and archival research. The three most frequently mentioned elements for the success of a service-learning project were effective communication, well-defined expectations of host organization and students’ skills and behaviours. Besides successes, there are also limitations that future TCCBE’s and U-links’ projects can address such as ineffective communication, issues with short-term projects, inappropriate timelines, students’ research skills and limited organizational support. Though our research goes into depth about the host organizations’ experiences, we were not able to investigate the community impacts because of time constraints and the complex nature of the communities involved.
Based on our research findings, we make a number of suggestions to both host organizations and the two centres. For TCCBE and U-links, we recommend that their websites be better organized so that information can become more explicit to host organizations. Both centres can create a guide for host organizations to refer to before, during and after each project. On the same topic of publicity and communication, we suggest that TCCBE and U-links maintain the network of existing host organizations well, while seeking out more community organizations to take part in service-learning projects. Lastly, we strongly recommend a better program agreement that articulates clearer dispute settling rights for TCCBE and U-links when problems occur. We recommend that host organizations clearly determine their expectations so that they can screen potential students accordingly and communicate their expectations to the students effectively. We also suggest that host organizations stay in close contact with TCCBE/U-links to foster the relationship and keep each other on track about the students’ work.
It is our hope that this report will be of use to TCCBE and U-links. As we continually state throughout the report, change can occur all the time and with TCCBE and U-links who organize projects by year or by semester, positive changes are possible and implementable. We hope that in the near future, not only the quantity but also the quality of community-based education projects facilitated by these two brokers will rise robustly.
April 5th, 2011
Research team: Alicia McDermott, Linh Phan, Claire Ryan, and Maritza Queeley.
Community-based education, internships and service-learning programs are widely implemented in many higher educational institutions all over the world. The general purpose of these programs is to provide students with practical experiences in addition to the theoretical base taught in universities and colleges. Since community-based education programs involve a number of stakeholders including students, faculties, academies, host organizations and communities, the effects that programs have on each stakeholder merits detailed examination. However, existing literature on community-based education has been predominantly about the student and university experience. In this research project, we shift our focus away from the student and seek to investigate the direct and contributing effects that service-learning projects have on the host organizations and communities.
Our research focuses on the work of the Trent Centre for Community-Based Education (TCCBE) in Peterborough and U-links in Haliburton, Ontario, Canada. Both TCCBE and U-links are independent, non-profit organizations that have acted as brokers between Trent University students and local organizations to formulate community-based research, service-learning, and experiential opportunities since 1995 and 1999 respectively. Over the course of 15 years, more than 1000 projects have been created with over 600 completed. Similar to the larger service-learning context, little has been done with regard to investigating the effects of CBE projects on host organizations and communities. This is an important step for organizations like TCCBE and U-links that should help them to reflect on their current performance and to develop strategies to address any shortcomings in order to maximize projects’ effects on host organizations and communities in the future.
Our research report is organized as follows: We begin our report with a critical literature review on service-learning and its correlating issues. In this part, we use existing studies to examine the positions of the main stakeholders in community-based education projects including students, academic institutions and host organizations. Based on the key themes that we identified, we then tailor our methodology to answer the research question and explain our use of archival and semi-structured interviews in our evaluation.
Archival research gives us valuable statistics and helps us to follow up on the contents of our interviews. Our interviews, on the other hand, create an opportunity for the host organizations to share their perspective at a deeper level than what is usually done with end-of-project evaluations. In the methodology section, we also outline the obstacles that we anticipated and encountered during the field research. In Chapters 4 and 5, we present the outcome of our research about the effects of TCCBE and U-links program on host organizations and communities respectively. With regard to host organizations, we maintain that there have been both positive and negative experiences with CBE projects, and the determining factors are communications and students’ knowledge, skills and attitudes. We list a number of success stories that have been spoken highly of by host organizations as examples for future projects to potentially replicate. With regard to community, due to the complex nature of research needed and the time constraints, we were able to trace the community impact only indirectly via the interviews with host organization representatives. In chapter 6, we provide a list of recommendations for TCCBE, U-links and host organizations based on the findings of our research. Finally, our report ends with concluding remarks that summarize the contents, outcomes and implications of our research.
This research asks the question: What direct and contributing effects have the Trent Center for Community Based Education (TCCBE) and U-links had on host organizations and communities? Before one gets to the field and studies the effects, it is indispensable to unpack the concepts of “community” and “community based education” and examine the literature on organizations’ motivation for participating in community-based education programs. A further idea that merits attention is “impact evaluation” which comprises both the purpose and the methodology of our research. By drawing information and evidence from existing scholarly work, we seek to provide a theoretical overview and detailed discussion of each of the components of our research question. This will help develop an analytical framework that guides us through the rest of the research process.
Central to our research is the concept of community-based education. The idea of incorporating students into a community organization has gained strong support and has been widely implemented in higher educational institutions since the 1980s (Fisher, Michael & Simmons, 2005; Stoecker & Tryon, 2009). The emergence of such programs reflected a broader shift in approach to university education, from an academic ivory tower to a more harmonious combination of theory and practice (Fisher et al., 2005). In that context, community-based education programs are believed to be beneficial for both the students and the community (Stoecker & Tryon, 2009; Rosing & Hofman, 2010).
It must be recognized, however, that existing research on community-based education program, work placements and student internships is severely lopsided. The majority of academic literature and empirical studies are focused on the education and student side of the equation. Conversely, description and analysis of host organizations’ experiences are scanty in both quality and quantity (Stoecker & Tryon, 2009). The process of students engaging in off-campus field-based work while earning credits is referred to by some common names, such as community-based education, service-learning, work placement and to a more limited extent, internship. Without going deeply into the differences between these titles, all of them are used interchangeably in this research to denote students’ participation in TCCBE and U-links projects.
Many scholars agree that students who are involved in a work placement gain certain advantages. Sather, Weitz & Carlson (2007) described the main benefits of service-learning modality as “experiential, active, and collaborative learning; intensive writing and reading; ethical and value-centered education; peer learning; and integration of the community as a laboratory” (p. 63). In a study about the role of internship in higher education in Belgium, Scholz et al. (2004) observed that having a field placement substantially improves “students’ ability to operate independently” and “to deal with uncertainty” in the work place (p. 41). They thus suggest a complementary relationship between field-based practical experience and academic education, especially in the setting of environmental studies. An early work by Sternberg (1982) showed that internships raised real life concerns among students studying economics at the University of California- Berkeley and that the students are then more equipped to balance between theory and practice.
In a similar manner, community-based education has been discussed in the context of pedagogy, either as an innovation in teaching or as an effort to address the lack of pragmatic research in the higher education process (Fisher et al., 2005; Harris et al., 2010; Rosing & Hofman, 2010). According to Sather et al. (2007), service-learning is able to address many of faculty’s challenges in raising students’ awareness about macro issues of practice, research and policy. They thus capture the idea of community-based education as “an innovative teaching modality which seeks to integrate students’ learning in the classroom, with a practical application of community service” (p. 65). Describing his personal experience teaching “The Global Economy”, Sternberg (1982) maintained that instructors no longer limited their objectives to helping students to master knowledge about theoretical concepts. Thanks to the field-based learning component, the course purpose has been broadened to equip students with practical skills and real life awareness, which he alleged is the “most important from a pedagogical standpoint” (p. 47).
On the other hand, work placement is not simply a learning tool for students or a teaching methodology for instructors. It also has the potential to address broader and more profound issues of higher education. Reardon (1998, as cited in Rosing & Hofman, 2010) named these issues as a lack of pragmatic research and failure to prepare the needed socially responsible human resources for the economy and society. Reardon’s critique resonated with the earlier recognition about the theory-practice divide of higher education among education leaders in the United States in the 1980s (Fisher et al., 2005; Gearan, 2005). Many authors agree that service-learning has helped bridge that divide and enabled higher education to fulfil its role in education and training more effectively (Fisher et al., 2005; Gearan, 2005; Rosing & Hofman, 2010).
Contrary to the abundant analyses and rich descriptions of the benefits that work placement generates for students, faculties and universities, both the host organizations’ and communities’ experiences with such programs are understudied in scholarly research. The process of an organization contemplating, deciding and entering a contract to hire student interns either plays a minor role in the research efforts (Sather et al., 2007) or makes no appearance at all (Gearan, 2005). In their research about the impact of incorporating service-learning into a social work curriculum, Sather et al. (2007) provide a cursory glance into the process of establishing placement. In their study, the academic faculty contacted the community organizations about potential intern positions. The two parties then sat down and discussed which projects and activities could be assigned to students and whose personnel should be charged with supervising the students at work. Few details were provided to explain why the agency decided to partner with the academic institution. The university-centered way of implementing a community-based service-learning program is termed the “singleton approach” (Harris et al., 2010) and receives considerable criticism for rendering community organizations subservient to the educational institutions (Stoecker & Tryon, 2009; Harris et al., 2010; Rosing and Hofman, 2010). Harris et al. (2010) suggest the replacement of such a lopsided relationship by a multi-stakeholders approach that can respect and equally benefit all parties involved. Expressing similar concern, Stocker and Tryon (2009) propose that more research needs to be conducted to evaluate the effects of community-based education on host organizations and communities.
Of the scanty references in the literature to organization benefits and expectations from student interns, three general conclusions can be drawn. First, many of the organizations incorporate students’ work into their existing projects while others create new projects seen as appropriate for the recruited students. For example, the instructors of social work, shown in the study by Sather et al. (2007), approach the agency and together they develop potential projects for students. In Rosing and Hofman’s research (2010), DePaul University students’ placement was part of an existing project of the organization CO-OP in their battle against malnutrition and obesity. The creation of new projects or assignment of existing projects thus depends on whether the perceived needs of the organization are matched with the intent of course instructors and students’ interest.
The second observation is that there is a broad spectrum of motivation for organizations to offer placement opportunities. Bell and Carlson (2009, p.20) are among the first scholars to provide a detailed empirical analysis of organizations’ motivations. They identify the four categories of motives for organizations to participate in service-learning program. The altruistic motive encompasses the willingness among organization staffs to educate students about important issues. The long-term motive refers to organizational strategy in recruiting future workers and donors. According to a study by IRS Employment Review (Crail, 2006), it was noted that 78% of employers interviewed see placement and internships as an opportunity to reach potential employees while 75% look at the student placement position as a chance to contribute to the society. The third motive, according to Bell and Carlson (2009), involves organizations’ intent to facilitate their operation by adding human resources. Cupps and Olmosk (2008) agree that not only the student but the organization also benefit greatly, by having “a short-term additional skilled staff person, new perspectives, enthusiasm, someone who can bring unfinished projects to successful fruition, and perhaps, a recruit for a long-term position” (p. 310). Lastly, there are also community organizations that seek to make stronger connections with educational institutions through their participation in service-learning programs. By tapping into such important intellectual resources, many agencies hope to make use of research and innovation while building a stronger community, which comprises both the academy and the organization. It is important to note that these motives do not mutually exclude each other. Some organizations will identify with more than one of the motivations mentioned above or add more to the existing list. The catalogue of organizations’ motivations to take on students that Bell and Carlson created will help us assess the extent to which students’ work fulfils the organizations’ and the larger community’s needs.
The process of matching the students with the organizations and the actual workplaces is filled with complications. Cupps & Olmosk (2008) attempt to unpack the relationship between the intern and the agency, describing it as “far from simple and is filled with potential pitfalls and frustrations” (p. 303). They name a few of the common problems as a lack of communication with regard to both the student’s and the agency’s expectations; the inability to incorporate students into the busy pressure-filled workplace, and the underestimation or overestimation of students’ capabilities to contribute. Whether or not the students and the organization personnel can grow from this relationship has a powerful influence on the perceived outcome of the community-based education program. Harris et al. (2010, p. 551) further pointed out that if both the universities and the agencies strictly enforce their “singleton approach”, there is not only little space for mutual learning and benefits, but the actual placement may lead to tensions, resistance and other undesirable effects.
Given the benefits that placements bring about to the students and the host organizations, together with the complicated negotiating process between the education institutions and the community projects, independent programs such as U-links and TCCBE are extremely helpful. Their role can be visualized as that of a broker who matches the demand for human resources of organizations with support supplied by students; and the demand for practical learning of students with a workplace experience provided by the agency. Highlighting the importance of such brokers’ role, Rosing and Hofman (2010) point out that “collaboration among faculty and students from diverse fields is essential, but likely challenging without the support of a service-learning center” (p. 226). They also recognize that “most important, service-learning support centers provide an important role in sustaining relationships that can bring different research resources to community partners over an extended period of time” (p. 229). Pearce, Pearson and Cameron (2007) expand on this idea and argue that service-learning centers help broaden the focus of community-based education program to include the exchange, mobilization and utilization of knowledge, with the intent to mutually benefit the institution, and its local and regional communities. By directly matching the supply and demand between host organizations and students and facilitating a collaborative environment between community program and academia, the intermediary role of programs such as TCCBE and U-links is thus vital to the success and sustainability of community-based education.
Of equal importance to understanding community-based education programs is the unpacking of the word “community”. With so many organizations, projects and programs that include “community” in their name or mandate, it is not surprising that a clear definition of the term “community” is not easily apparent in the literature. In the case of community-based education, due to differences in scope, size and objectives, the targeted “community” of each project can vary greatly. The literature concerning community programs does however suggest different characteristics that a targeted community will likely hold. As well, the decision about what kind of community is served appears to be contingent on the goals of the program/project/organization. Because of the difficulties in ascertaining a single definition of a community, it is suggested that the term “targeted communities” – those that are sought out or served by an organization- would provide a more practical tool for evaluation.
There are many categories that can determine the character of a targeted community. Characteristics may overlap and complement each other and some or all of these characteristics may be present in a single community. A targeted community may be seen as:
Examples that show the targeted community fulfilling more than one of the above-mentioned criteria are numerous. Melhuish et al. (2007, p. 543) provide the example of children and their families in “disadvantaged communities” being the focus of a Sure Start Local Program that was implemented by the UK government. This targeted community is composed of more than one geographic area and is the target of “universal services” (Melhuish et al., 2007, p.543). Another community-based program described by Montenegro (2002) is an educational program put in place in Caracas, Venezuela for children in a slum area who did not attend school.
Krause’s (2002) study of community interventions in Chile provides an important warning against viewing a community as only a group of individuals. He argues that larger social issues are also at play within and around communities (Krause, 2002). If the greater society is not taken into account there may be consequences as the community interventionists “contradict themselves with regard to their concepts and actions” (Krause, 2002, p.567). This will often take place unless the target group is contextualized as part of the broader “community or a social structure”. This is essential if the causes of the problems leading to intervention are to be addressed (Krause, 2002, p. 567).
Screening and choosing communities wisely is of great importance to organizations when they plan where to place their efforts and their interns. Shenassa (2002) is perhaps one of the most committed supporters of greater attention to matching. She states that: “Whether community intervention and prevention programs deliver the goods or beget ills depends in great measure on whether the right participants are chosen for them.” (p. 197). This emphasis on thinking ahead is accompanied by the warning that things can and do change within a targeted community (Shenassa, 2002). Therefore it has been suggested that pilot programs be conducted as a justification for and evaluation of the program’s application in the specific community chosen (Shenassa, 2002, p.206).
A program for a targeted community also may belong to more than one of these categories. The Sure Start Local Programs (SSLPs) put into practice as described above by Melhuish et al. (2007, p.550) were intended to improve the social/emotional development of the children involved, provide attention/awareness to health issues within the community and to strengthen families and the community. Montenegro (2002, p.515) provides another example in the empowerment of a targeted community in the San José de la Urbina, a low-income neighbourhood of Caracas. This community showed initiative and strength as members argued for their right to better transportation services in the neighbourhood; finding success as their requests were granted within a few weeks (Montenegro, 2002).
The idea that community is not only the target of intervention but also the engine of change has been noted in much of the literature. Levy (2004, p. 192) suggests that there is a strong role for the community as an agent of change when the practitioner acts as a facilitator and not as a simple provider of aid. Levy (2004, p. 193) shows the importance in community actors as agents in creating their own changes and finding empowerment, by saying: “Community members and leaders, once organized through the practitioner’s facilitation, become their own advocates, mediators and mobilizers, and bring in experts as partners … ultimately to develop their own expertise.”
In summary, the issue of defining the term “community” may not be as important as noting the difference in characteristics that can compose the usual targeted community in community-based programs. While these characteristics vary in importance to a specific organization or program of implementation, it is worth noting that community-based programs also differ in their goals and ideology. These differences may be seen as a blessing or a hindrance. It is auspicious as a greater number of individuals may receive attention and aid when the providing organization has a more comprehensive view of how the community they want to work with is comprised. Conversely it may be that creating parameters that are too rigid for a targeted community may divide it from others that share similar need. Therefore, it is gratuitous when many similar community-based programs exist while a broader “community” of individuals is not reached. In the context of our research, the term “community” will be used to refer to the collection of targeted communities of each host organization; as well as the community constituted by all host organizations.
Every year an enormous amount of money is put into developing social programs, which can help the communities outlined above, and collecting information about what practices would make a program design more effective in achieving goals (Savedoff, 2006). This is what impact evaluation essentially aims to accomplish.
There are different meanings associated with evaluation, and impact evaluation asks about the difference between what happened with the program in existence, and what would have happened had the program not been implemented (Savedoff, 2006). Impact evaluations are best completed on programs that are aiming to expand or continue, which is why it is a relevant concept to our project with TCCBE and U-links. It is important to make a distinction between effects and impacts. According to Lichfield (1996), there are effects that happen to individual community sectors that are targeted by the programming, thus driving the change in its way of life, which is the impact. Therefore what Lichfield defines as impact may or may not have occurred as a result of the project. He also distinguishes between three different types of effects: economic, social, and on the natural environment. Due to the nature of our project, we will add cultural life as the fourth dimension to the category of impacts to be examined.
The most basic procedure for an impact evaluation is to understand the program's original goals, determine whether those goals have been met and figure out whether or not those effects are a direct cause of the program's existence (Baker, 2000). Impact evaluations can also look into the unintended consequences of a project, which is relevant to the assessment requested by TCCBE and U-links as well. Taking into account the unexpected outcomes of a program is beneficial since it will not limit the research to the dualistic verdict of negative or positive, and will instead enrich the findings and lessons learned. It is also pertinent that impact evaluations tell why or how there have been effects as a result of the program (Savedoff, 2006). This is where the term “impact” becomes problematic, since it is almost impossible to determine whether or not the changes identified have definitely been a result solely of the program or intervention. It has been suggested that the replacement of “direct and contributing effects” for “impact” is necessary, in order not to isolate the cause of changes as a single intervention. Such a replacement for “impact” is beneficial in guiding the conceptualization and the actual research, while the use of “evaluation” remains intact.
The lack of knowledge and certainty in how to most efficiently allocate resources within a program to reach the specified goals and the lack of effective impact evaluations being carried out are termed “the evaluation gap” by Savedoff (2006). According to the author, one of the reasons there is an evaluation gap is because there are not enough incentives and too many barriers to completing an impact evaluation. Often when impact evaluations are completed, they do not produce useful information because they lack scrupulous methods. Additionally, good impact evaluations require a lot of time and resources, which are not readily available. This is problematic because poorly done evaluations can mislead, and result in social development programs being carried out that may have minimal or even negative consequences that have been overlooked in the evaluation (Savedoff, 2006).
In order to make an evaluation effective, methods are suggested in the literature for collecting data for impact evaluation. Ravillion (2001) provides an extensive list on how to effectively assemble data for impact and effect evaluation; the ones listed here are most relevant to our research:
Our research project offers a unique opportunity to examine the outcomes of service-learning projects from the host organizations’ and communities’ perspectives. The initiatives implemented through the TCCBE and U-links do provide insights into things the community organizations and agencies see as important to the targeted communities as well as some of the expectations they hold regarding the community, themselves, and the greater society. We suggest that programs that have been implemented in Peterborough and Haliburton largely fit into the following categories:
Impact or effect evaluation is part of process of knowledge building, and the end result becomes a “collective good” (Savedoff, 2006). In the end, the evaluation of TCCBE and U-links projects should be something that allows the two organizations to gain insight into how their projects affect the host organizations and the communities, and what methods and means have been the most effective in inducing positive change. The recognition of evaluation as contributor to knowledge building resonates with Patton’s early work on utilization-focused evaluation. He maintained that:
“Evaluation research is only of use if one believes that some systematic information is better than none. Evaluation research has meaning only if one believes that a rough idea of the relationship between program activities and outcomes is preferable to relying entirely upon hope and good intentions. Evaluation research does not provide final answers, but it can provide direction. Thus, evaluation research does not lead to final statements about causal linkages, but can reduce uncertainty about such linkages. Therein lies its potential for utilization. (Patton, 1978, p. 180)”
This means that evaluation should be treated as being an imperfect process rather than being expected to produce absolute results and recommendations. The ultimate purpose is to assess the performance of the organizations, in this case TCCBE and U-links, and provide information about how and whether they have produced the desired effects.
Community-based education projects primarily aim to increase the quality, availability, and effectiveness of educational opportunities off-campus. The previously neglected study on host organizations’ and communities’ perspectives on such projects attaches a lot of significance to our project. First, it can help organizations to reflect and potentially redefine their goals and practice in order to ensure the effective function and attainment of their original missions, or revise their missions as needed. Second, CBE projects receive new students on an annual or semester- basis, which makes evaluation results and changes relatively easy to implement. Nafissatou, Faye, Moreau and Cabral (2004) give the example of CBE programs in Burkina Faso and Senegal, where evaluation and replications helped to improve programs both educationally and administratively while promoting and expanding programs to other regions and communities.
According to Rubin (2000), the main purpose for conducting and assessing community- based education is to provide immediate feedback, which is often used to make changes in a program in order to better respond to the needs and concerns of those involved. As much as short-term solutions can be identified, evaluation can also have a long term effect on program design. Gamm, Castillo and Williams (2004) argue that evaluation of educational and community-based programs serve to manage limited resources better and deliver more quality services to targeted populations by pointing out the pattern of success stories and modifying the program structure accordingly. Similarly, Raadt (2004) believes that evaluation can also identify inherent weaknesses such as incompetence, lack of caring, and dishonesty that can undermine the outcomes of the project. It has been suggested that long-term evaluation is needed to assess the essential contributing factors to the sustainability of educational and community programs (Garlick and Pryor, 2004). Unlike much of the existing literature on community-based education we seek to examine the direct and contributing effects that TCCBE and U-links have had on communities and host organizations. Our assessment is expected to address both the long-term performance of the two organizations over the past ten years and potentially lead to short-term and long-term changes. In the next section, we will introduce and explain the methodology we use to answer this research question.
What direct and/or contributing effects do TCCBE and U-Links projects have on host organizations and their targeted communities?
TCCBE and the U-Links program aim to connect students and faculty with local organizations to generate community-based research, service-learning and experimental education opportunities to enhance the social, environmental, cultural and economical health of communities. Based on our research, TCCBE and U-Links hope to obtain a clear sense of the effects of their programs on communities; which could be used to enhance the projects to better meet the needs of the communities for a more successful future.
Effects are defined as the positive and negative outcomes that were anticipated and unexpectedly created by/related to the projects students participated in when working for host organizations associated with TCCBE and/or U-Links. Effects were looked for in relation to the categories of social, cultural, environmental and economic.
Community here will refer to the targeted population of the projects implemented by host organizations affiliated with TCCBE and/or U-Links.
Host organizations are the organizations/groups that have hosted students to complete projects as part of the TCCBE program.
Community-based education has been defined as: the projects or programs facilitated by TCCBE and U-links in which students, mostly from Trent University, volunteer work or research at a host organization as part of their academic curriculum.
This research component of this project took place over the first three months of 2011. The locations for the project are broadly defined as the areas of Peterborough and Haliburton.
We compared the differing qualitative data provided within our interviews and archival research against each other and other outside literature regarding the impacts community-based/ service-learning programs have on the targeted communities in reference to their social, cultural, environmental and economic effects. Indicators of the various positive and negative effects became clearer throughout the research process. However, generally it can be said that positive effects are those described as successful in reaching the goals of the project and/or outcomes that have been beneficial to targeted communities, host organizations, student participants and/or TCCBE and U-Links. Negative effects will be broadly defined as the harmful or undesired outcomes that have harmed relationships between the above-mentioned groups and/or created unwelcome effects on any of the members of these groups. That being said, focus will primarily be placed on the positive and negative effects of past TCCBE and U-Links projects on the targeted communities and the host organizations. As existing literature on service-learning has focused mainly on effects of community-based/service-learning programs for students, we examined the effects on the other key stakeholders – targeted communities of projects and primarily the host organizations themselves.
Interviews, databases, and reports about/created by: TCCBE and U-Links staff, host organizations, and various communities targeted in TCCBE and U-Links projects provided the primary data for this research. Literature on evaluation and community-based/service-learning programs was used to back up the primary data.
TCCBE and U-Links project reports were split into the four categories of economic, social, cultural and environmental based on the project’s main focus. We then did random sampling to select at least four projects within each category. These numbers were altered as we discovered that some host organizations of selected projects were unable to participate. The past project files provided the initial links to the host organizations and their targeted communities.
Among the methods we chose for our evaluation research, archival research is the sole unobtrusive measure. Berg (2001) discussed at length archival research as a complementary tool to other methods, given the risks and rewards it involves. He has identified two types of archive resources - public and private. In the context of our research, most organizational archives were made available with free or restricted access by both TCCBE and U-Links. They are both in paper and electronic format and have been collected over the past 10 years of the organizations’ operations. There were two main types of official documents that we used for our research: past evaluation forms, and the databases of TCCBE and U-Links projects.
Past evaluation forms are mostly end-of-year surveys, filled out by faculty, students and host organizations. These surveys are in paper format and kept in order by year. The original purpose of these surveys was to assess the experience of each party involved in every TCCBE project, pinpointing the positive and negative aspects of the programs. In the past few years, there have been modifications to the questions and content of the surveys to better suit the TCCBE and U-Links strategic needs. The information stated on these forms is thus one of the best tools to document the direct effect of community education projects for an extended period of time.
In addition to evaluation forms, TCCBE and U-Links maintain databases with a detailed description of projects, students, and faculty’s contacts. . This information was vital when we started identifying the pattern of host organizations’ participation as well as for our selection of samples for our interviews. From this comprehensive database, we were able to observe which organizations have been receiving students for what types of work, whether they have a long-term commitment, and what type of projects students are more prone to adopt and what type of projects interest students most. Based on the information in the database, we also categorized the potential effects of projects into one of the four pillars: economic, social, environmental or cultural. Once these categories were determined, we selected a number of organizations to approach for more in-depth research in the form of interviews.
With all information processed, we ensured that ethical and confidential protocols have been strictly followed. Additionally, as Berg (2001) has pointed out, there are certain errors and biases associated with each archive resource, such as missing data or information based on organization’s self interests. In the process of research, we constantly had to remind ourselves that no single methodology will be sufficient by itself but they must complement each other to produce fruitful discussions and lead to suggestions for improvement. Archival research therefore was supplemented by interviews.
Archival research, while being extremely helpful, can only provide us with the information that has already been gathered. Though it can be manipulated into something new, on its own archival research is data already known. Unlike archival research, interviews can be used to create new information. An interview is a systematic way of talking and listening to people and is another way to collect data from individuals through conversations. Interviews provide in-depth information about a particular research issue or question and are a useful tool in collecting credible data from a particular source (Gillham, 2000). There are many reasons to use interviews for collecting data and using it as a research instrument. Gray (2004, p. 214) provides four main reasons:
The TCCBE and U-Links host organizations and other interviewees were asked to participate in semi-structured interviews. Semi-structured interviews are used to answer specific or direct questions related to the research topic and also to collect a range of personalized opinions and experiences for the research. A semi-structured form of interviewing was chosen because it has some degree of predetermined and organized order, but still ensures flexibility in the way issues are addressed by the interviewer (Gray, 2004). The questions asked in the interview are content-focused and deal with issues or areas judged by the interviewer to be significant to the research topic.
Semi-structured interviews are non-standardized and are frequently used in qualitative analysis. The strength of the semi-structured interview is that the researcher can probe deeper into the given situation (Gillham, 2000). In addition, the researcher can explain or rephrase questions if needed. As people’s opinions and experiences vary greatly between different classes, ethnicities, ages etc., interviews are used to counter the claims of those who presume to have created certain beliefs or views of their own (Gillham, 2000). Most interview questions allow for an open response, as opposed to a closed set of responses, allowing the interviewer to discover more about events or opinions from the interviewee. One of the major strengths of an interview is that it also allows the interviewer to discover what information is relevant. Interviews can be very useful in finding indepth information from a single source, or few sources.
In conclusion, both of the methods: archival research and interviews each have their own strengths and weaknesses. In this description the focus has been on the positive aspects of these methods. However, one method would not be sufficient on its own; archival research cannot give in-depth information, and interviews cannot provide a discussion of differing opinions alone. The data gathered from each method had to be examined in such a way that we could see how one can complement the other – thus providing a more complete response to the main research question and its sub-questions.
Problems arose in attaining a large enough sample size for reports in all of the four main categories of interest: social, cultural, environmental and economic. Thus some categories had fewer participants in the interviews than others. This was largely due to problems in securing enough participants for some of the interviews. Time constraints arose as differences in schedules were unavoidable. To handle the problem of time, we tried to select our samples as early as possible to provide us with more time to meet with and interview persons from each of the selected host organizations. Difficulties with time also caused us to drop the idea of focus groups from our research. Due to the problems in scheduling individual interviews we ran out of time to set up and run a focus group. We also found that the time limitation disallowed the opportunity to hold interviews or focus groups within the targeted communities. We feel that our research can be used to represent the host organization views; however, to reach the targeted communities will necessitate a new research project.
Our study seeks to assess the direct and contributing effects of TCCBE and U-links projects on host organizations and their communities. In previous chapters of this report, we have examined in detail the existing scholarly literature with regard to CBE programs and the stakeholders that are involved. We have also introduced the methodology which we used to approach the research problem: purposive sampling, interviews and archival research. In this chapter, we present the findings of our study by theme. We will first review the motivations of host organizations to participate in CBE projects. Based on these motivations, we evaluate whether CBE projects have been successful in meeting the expectations of the host and in creating positive change in the organizations. We also identify the limitations of CBE projects from the host organizations’ perspectives, and accordingly generate recommendations for future improvement of the programs. Finally, we argue that exploring the impacts of TCCBE and U-links projects on the community will require a separate study because of the complexity involved.
As we pointed out in Chapter 2, there is little systematic information in the current literature on why host organizations take up and continue their use of community-based education. The process of initiating a relationship between the host organizations and brokerage institutions has been scarcely examined as well. In our interviews we asked host organizations what their motivations are for beginning and continuing, or discontinuing, their use of the brokerage of TCCBE and U-Links. We were interested in understanding how these relationships began and why they have continued or not.
Figure 1: How CBE relationships are started (Sample size n = 17)
The responses to how the host organizations began their relationship can be split into five main categories. The first option is that the approach came from the host organization, as they asked TCCBE/U-Links for students’ aid in filling a need. This response was repeated by a third of the interviewed organizations. The second way the relationship was initiated was through TCCBE/U-Links approaching a host organization to place a student, which occurred in 17% of the responses. The third means was via the host organization having a pre-existing relationship with a faculty member and TCCBE/U-Links acting as an intermediary, which occurred only 6% of the time. The student approaching the host organization directly appeared in the responses 29% of the time. We also found that 33% of those interviewed were unsure of the environment in which the relationship began, or vaguely stated meeting TCCBE/ U-Links in other ways, such as open houses.
Our results differ from those of Sather et al. (2007), in which the academic faculty contacted the community organizations. This university-centered way of implementing a community-based service-learning program, the “singleton approach” (Harris et al., 2010), has been the target of numerous criticisms for rendering community organizations subservient to the educational institutions (Stoecker & Tryon, 2009; Harris et al., 2010; Rosing and Hofman, 2010). Therefore, TCCBE and U-links have contributed greatly to balance the university-community dynamics. They act as brokers by facilitating both the learning opportunities of students and the engagement of the host organization.
One key question that was included in our interview is about host organizations’ motivations for taking part in TCCBE/U-links programs. Our results reveal mixed motivations. First, a lack of resources – either in terms of finance or staff, was the primary reason why many host organizations seek placement students. For example, one host organization stated: “In the beginning we were completely volunteer-based and we needed more workers to get more research done.” This was not a singular response. Many other organizations echoed the need for human resources to reach their goals. This is especially true for grass roots organizations, which often lack funding and rely strongly on volunteer support.
Some host organizations also stated that student/youth leadership and participation in community work were important to their own mandates and goals. Therefore, they saw community-based education as a means to meeting these goals, and TCCBE/U-Links as actors that could foster these relationships. Host organizations often see these relationships with students as “win-win” in which the organizations mentor and teach students while students provide much needed support. This idea fits in well with the altruistic motive outlined by Bell and Carlson (2009). An example of this comes from the response: “TCCBE offers good opportunities for students to learn outside of the class room because the university has so many resources that can be linked to the community and it’s also the mandate of [the organization] to provide educational opportunities for the students.”
Another host stated: “[The organization] is a research organization and very volunteer oriented. TCCBE thus comes as a natural partner. A tradition of working with TCCBE also justifies our continuing engagement. Even without that engagement, I would search them out anyway.” This is an example of how host organizations prefer to create relationships with brokers such as TCCBE because of they are volunteer-based.
Host organizations also sometimes linked their motivation to a desire for public recognition. This recognition could be seen sometimes as a desire to educate the student or community of their existence and goals. As students connect their networks to the host, the organizations can expand their services to more community members. One interviewee mentions that “Another student organized [Conference name]. It was very successful and facilitated the relationship between community and university. The conference also helped raise the awareness and [organization name] profile, which is very important to us. We are often invisible in the community you know.” Students who do TCCBE and U-Links projects often produce tangible results which can help the host organization with future fundraising opportunities, and with the general accessibility of their programming.
As we have discussed in Chapter 2, there is often no single motivation for a host organization to participate in community-based learning programs. Our research provides further evidence to the truth of the multifaceted motivations of host organizations. For example, in one interview it was stated that not only had the university faculty been involved with TCCBE projects in the past, but additionally had an ongoing relationship with one of the staff there, and this allowed them to keep projects going.
The length of time a host continues their relationship could be suggestive of their trust and reliance on actors such as TCCBE and U-Links. According to the information retrieved from the TCCBE and U-links database, of the 197 host organizations that TCCBE has worked with- with or without complete projects - 113 have more than once participated in service-learning projects. With U-links, the corresponding figure is 43 out of 69 host organizations. Notably, the number of host organizations with at least 5 CBE projects is considerable: 38 for TCCBE and 16 for U-links.
This led us to a closer examination of the length of time that the host organizations have had a relationship with either organization in order to see if the brokerage became a long term relationship. As our sample primarily covered those with repeating engagement with TCCBE and U-links projects, we were able to identify the reasons behind that decision. The most frequently cited explanation is one of an established trust: host organizations have had past successful experiences with TCCBE and U-links projects and therefore would like to continue when they have projects in mind. However, even with some organizations who have worked with TCCBE and U-links in more than 10 projects, success stories are not always the case. In the sections that follow, we discuss both the positive effects of CBE projects on host organizations and to a certain extent, communities, as well as identify the limitations that the programs need to overcome.
Out of seventeen interviewed organizations, twelve responded with positive feedback and three responded with mixed outcomes (See Table 1). The effects that CBE projects have on host organizations can be documented in a number of areas: enhanced functionality; fundraising opportunities and improved public recognition. To best demonstrate the way CBE projects benefit hosts, we present specific cases where the impacts are most vividly seen.
Improved public recognition and community participation
Our findings showed that successes such as the above mentioned examples can be attributed to three factors. These include good communication, clear expectations of the host organizations, and appropriate student skills.
80% of the interviewed host organizations stressed the importance of communication when projects are being carried out. Communication is important because it brings a closer connection between organizations, students and TCCBE/U-Links; it creates a mutual playing field in which everyone’s understandings and objectives can be perceived; and it often determines the quality of the final product. All of the twelve organizations who expressed positive outcomes mentioned the importance of effective communication between students and TCCBE/U-Links as the defining factor for project successes. To provide the best picture of the communication issue in CBE programs, we divide it into two types: initial communication and communication throughout the process.
Initial communication is essential to the beginning of the project. First either staff from TCCBE and U-links or from host organizations will contact the other partner to discuss the idea of the project. The project proposal is then drafted by the host organization and promoted to students by TCCBE and U-links. Most organizations who have repeatedly hosted student to carry out their projects mentioned that once they got acquainted with the process, they proactively submit their proposals to TCCBE and U-links. However, many respondents also identify the critical role of TCCBE and U-links staff in initiating the projects. Interestingly, many organizations were introduced to TCCBE by personal connection. One staff at a Peterborough grass roots organization reports that: “In the first year of our operation, I got contacted by (TCCBE staff name) who told me about TCCBE... She was my neighbour; if I did not personally know (TCCBE staff name), we would never get any CBE students. Maybe the word needs to get out more so that more organizations can know about this great opportunity”.
Another host organization staff also expressed her gratitude when referring to TCCBE staff “I did not work with them [TCCBE staff] that much but before I worked with (TCCBE staff name) and she was excellent with great ideas. She is always ready to think about new ideas to start new projects, a very inspiring figure”. TCCBE and U-Links have done a great job in promoting service-learning in the Peterborough and Haliburton area, via the Knowledge in Action forum in March and the open house in September. Staffs’ networking skills also enables the formation of new projects and relationships.
Initial communication is not limited to what takes place between TCCBE and U-links staff and host organizations. It also includes the first meeting with students, the supervisor from the host organization, and TCCBE and U-links staff. At this meeting, students get to understand what is expected of them and the hosts get to decide whether a student is the right fit. This meeting is a critical point as it not only sets the impression, but more importantly, it determines whether the project is going to take shape.
Of equal importance to initial communication is communication throughout the process- both of which can determine the project outcome. Because supervisors participating in CBE have different capacities, some were able to arrange face-to-face meetings every few weeks and others let the schedule be more flexible to students. An on-campus organization staff mentioned that check-in meetings like these are necessary, where students can give updates to both the board and the staff and listen to the feedback. She recognized that being on campus is an advantage, since it makes it easier for students to attend meetings. However, this is not always the case and for some host organizations which take time to travel such as those in Haliburton, face-to-face meeting can be effectively aided by emails and other means of communication. Some supervisors think that students should be left to determine course and pace of their work. In this case, all the expectations are lined up at the first meeting and the student would give updates by phone, email or casual chat every few weeks as needed. It is thus apparent that the issue of communication can be addressed when both students and host organizations are proactive in their communication by providing mutual feedback and updates. The role of TCCBE and U-links is important not only in setting up the project and matching students with the placements, but also to facilitate this relationship throughout the process.
In cases that showed positive outcomes, expectations were often communicated and clearly defined. Host organizations that mentioned students showing exceptional working skills and exceeding their expectations were the cases where both parties are well aware of the expectations. Most host organizations expect students and the TCCBE/U-Links staff to be timely, communicative, and keen to learn and to bring new perspectives as well as a useful end product. Although some project products might not be seen as completely successful by interviewees, the fact that students grow and learn from the experience still make the project a partial success - as stated by one host organization staff. Another host organization member believed that partnerships with the people at TCCBE and young people are their best hope in changing attitudes. It was unclear whether host organizations expectations are clearly defined orally or written; however, as one host mentioned, expectations can be something very practical and TCCBE is a great interface between what is expected of host and students.
Host organizations believed that projects/programs helped the community by creating awareness, community building, outreach, and they help people with needs in the community; For example, with the provision of food. Because 100% of all host organizations as well as TCCBE and U-Links have some form of mandate linked to community well-being, it is also expected as one host stated, that TCCBE screen students in areas of community interest. As information from projects can be useful in areas such as poverty reduction and benefits of immigrant workers for not just one, but other community organizations, how organizations and TCCBE/U-Links convey their expectations plays a significant role in positive project outcomes.
Many hosts believed that whether students come ready with a skill set will determine the project outcomes. They trust students’ ability to do research and appreciate students’ willingness to help. For example, in a Black Bears Project the host organization stated that “The first project we did to draft the curriculum, the student convinced me that he was interested in the program, had a biology background and access to people in the teaching field that did provide assistance as well, and this Project was very successful”. Topic related courses not only helped students for the projects but also their writing, research, communication and social skills, especially if they are going to be working in the field or within the community.
Organizations have seen some positive results; this was one of the reasons why host organizations continue to be a part of CBE programs and to be linked with TCCBE and/or U-Links.The fact that most organizations don’t have the time to do a project themselves is one reason why CBE can helps. Through TCCBE/U-Links, the organizations are able to engage with students and gain exposure, which also promotes and develops their organizations. Students have enthusiasm and are often eager to learn. It is in cases such as these where the students are believed to have created a positive result over all, considering their time restrictions. If students can have a positive outlook on CBE programs and gain knowledge during their time spent doing service-learning, then the TCCBE/U-Links mandate will be fulfilled. Students tend to take what they learn and share their experiences outward. As one organization believed, “if they had a positive experience or learned something, then at the end of the day it can be filtered out into the community and can create a positive impact in the long run”. Hence, community impact is also important because not also do students, but projects inform them of many programs and information that is available to them for service-learning and to help them become more proactive individuals.
Community based education projects show positive outcomes where good communication, students’ skills, and defined expectations are exhibited. According to many host organizations, projects do have a positive connection with community well-being, shown through cultural, environmental, social and economic impacts. Host organizations believe that research-based projects help to shape community knowledge by bringing people together to reflect, celebrate and connect with each other. One organization’s mandate is working to protect the heritage of the community and the health of its environment. Hence, projects from this organization directly contribute to and promote the social aspect of one of TCCBE’s mandates. Also, CBE is helpful in making community links through TCCBE and U-Links and in promoting the host organizations.
Table 1: Host Organizations’ Assessment of Project Outcomes per Interviews conducted with 17 organizations
Overall successful projects
In the preceding sections, we have argued that many TCCBE and U-links projects have been beneficial to both organizations and the community. We identified the reasons for success, which include effective communication, defined expectation and students’ skills and willingness to contribution. However, the successful realization of those programs does not come problem-free. Past evaluations submitted by host organizations and interviews with students’ supervisors have revealed several challenges that may impede the success of CBE programs. In this part of research findings, we discuss in turn the issue of communication; organizational capacity; student research quality and timeline; and the short-term nature of the projects. Awareness of these issues and deliberate attempts to address them will likely result in more successes and meaningful student experience in CBE programs in the future.
The issue of communication and miscommunication tops the list of both reasons for success and causes for failure of service-learning projects. Most host organizations’ representatives mention the importance of communication in working with students even though it is not specifically included in our questionnaires. In this section, we also discuss the topic of communication by the two types mentioned above: initial communication and communication throughout the process.
With initial communication, the issue lies mainly with the first meeting that happens between the students and their host organization. It is the time when students get to decide whether they will work with the host, and the organization can express their expectations and select the right fit for their projects. When asked about the screening process, most organizations responded that they do not apply any strict measures. It is generally agreed that if students show interest in the program they will be accepted. One host organization regretted not selecting students more carefully and spoke with frustration:
“In the future, we will screen students more thoroughly. Students need to be knowledgeable, willing to learn and be interested in the course. There needs to be a more formal screening process, not just a meeting. It is almost like a job and you need to justify why you have the capacity and how you plan on learning to carry out the work”.
Though the selection of students is primarily the responsibility of the organization, TCCBE and U-links can offer support by developing guidelines for this process. Because recruiting service learners is different from hiring an employee, as we have explained in the literature review, organizations should be informed of the best practices to select their best match.
Communication throughout the work placement process often met presents challenges. It was not uncommon to hear feedback from the organization where students were enthusiastic at the first meeting and then quickly fell behind. The severity of the lack of correspondence varies: from not answering email to completely missing meetings and deadlines. We acknowledged above that different hosts set up check-in meeting in different ways, from a frequent face-to-face reunion to relying mainly on emails and phone calls. Regardless of the means, a lack of communication between two parties can lead to serious problems and undermine the relationship between service learners and their supervisors. One staff member of a host organization that has had a negative experience with CBE projects pointed out that:
“I sent them so many emails; let them know the key to the office, office hours, board room. I saw the work plan and that’s the only thing they sent before the report. I made comments and suggestions but they never revised the work plan. I sent a lot of resources through email but no response either. It costs time and energy and later we decide to hire somebody to make the guide instead of students. I offered training but they did not take it. Students got enthusiastic at the first meeting but became disengaged quickly. They did not take part in training, did not come with the required skill set and were not interested in learning either. The project was a failure and not what we hoped for.”
Such situations as the one above, reflects a lack of commitment more than a lack of communication. However, one could argue that if the students had explained to the host their lack of interest, or had TCCBE talk to the host, the issue could have been less devastating. Such negative experiences often lead to feelings of frustration and even distrust. Organizations feel that they are not treated the way they deserve, which further aggravates the gap between the university and the community.
On the other hand, some host organizations agreed that they might not be communicating their expectations clearly enough or early enough, though they try their best to facilitate communication. One organization member in Haliburton drew from her experience and said that “I have heard some other organizations complain...but I think it is just because there must be clear and effective communication. We need to be clear about the outcomes we are looking for and communication and feedback is needed throughout the project for everyone involved.”
Agreeing with this standpoint, another staff observed: “In the other failure the students were actually very efficient – they did everything I had laid out for them ...the problem was they took everything too literally. Their information was useful, but not very, they just took everything I said too literal and then told me it was my fault.”
Effective communication can enhance the quality of the project tremendously, and vice versa. The roles of TCCBE and U-links are vital in easing problems that arise. Many organizations, when problems occur, are at a loss as to whom they can contact and request support from when facing a dispute. Miscommunication or a lack of communication, as this research has pointed out, might not only hamper the project outcomes but also jeopardize the long term relationship between host organizations and the higher learning institution. Given the large number of projects in operation every year, TCCBE and U-links should work on increasing the number of staff who work towards meeting the needs of the parties involved, and monitoring or facilitating projects.
CBE can sometimes present a paradox. Many organizations choose to participate in CBE programs in the hopes of gaining extra manpower, since their existing personnel are overloaded and financial resources do not allow hiring extra staff. The ideal scenario is that the student performs well at what has been assigned; however, this is not always the case. Students can add to the burden of the regular staff of an organization by requiring a considerable amount of time and attention. When organizations consider starting a research project or hosting a student, they need to consider whether it is worth the extra work. One employment program coordinator explained: “I am careful about taking on student research. It takes time although the research is good. Students do not alleviate my everyday burden but they do the extra initiative.” The story of limited organizational capacity to accommodate CBE projects is a recurrent one. Many organizations have expressed their willingness to continue the engagement with both TCCBE and U-Links but have to reconsider from time to time due to a lack of manpower. A representative from a government office who has hosted many CBE projects shared that:
“I would like to do another project again, but I haven’t because the one we did do was very ambitious and it took a lot of time, so it’s finding the time to do that again. Also, I think it would be helpful to learn more about the types of projects students have done which were successful to see how big or small of a project we should be looking at, because with the first student it was a lot of work so with the second I expected a lot and I don’t know if I communicated that well.”
Some organizations were able to create an extra research coordinator position to supervise students’ research. Even for these organizations, however, relying on a temporary position of research coordinator can be problematic: “This year we do not have the research coordinator so it all depends on students - some have a great sense of what they have to do. Others need more guidance, which is a different type of relationship. Sometimes there are too many projects that we cannot manage. We try to limit to no more than 2 projects a year now. I would not do the 12 projects a year again.” The problem with organizations lacking capacity can partly be solved if students are aware of the situation and are proactive in their researching and learning process. However, not all students are ready and able to work with little guidance. The key factor to make the projects work is for organizations to evaluate their own capacity and assign work accordingly.
Quality of research
As discussed in previous sections, when deciding to host students, all organizations expect to have the project finished with successful results. However, as one professor pointed out in the interview, undergraduate students have certain limits when it comes to both knowledge and skills. He explained: “Many organizations understand that and usually we do not have contracts, nor guarantees for student’s work, except for the eligible grade of 76% and above. We make sure they have certain levels of education. It’s true that host organizations are busy but they need to understand that there’s a limit to students’ capacity”.
Despite the contributions students were able to make to host organizations and communities; there are circumstances when the quality does not meet the hope. For example, there was a project of urgent need to investigate the advocacy framework so that organization staffs can be informed and trained. However, the research ends up taking mere definitions from the website and was not of any help to the host organization. The host then resorted to hire an extra staff to do the same research, since they were pressed for time and the student’s research did not fulfill the expectation. There are also cases when research is not as accurate or as thorough as the organization wished to see. When the host verifies the result, mistakes were spotted and the report quality was questionable. Students’ ability to conduct research is strictly important and this issue need to be addressed for long-term benefits of both the students and the host organizations.
Timeline of the project
One of the most surprising yet interesting findings of our evaluation process is about the timeline of CBE projects. Host organizations’ complaints are often that too much time is spent on the ethics approval process and writing the project proposal, while too little time is spared for the actual research. While this issue does not apply to every single project, it is a constant frustration for organizations that need immediate support. One respondent suggests that “I don’t know how much they have improved since 2005-2006, but there was a lot of back and forth before things were actually on the way. So, before we got things together students had to get things lined up with signing and so on, but if this was done ahead of time students could have jumped right in the project - which would have been more helpful”. The employment coordinator who had two very successful projects also expressed his disagreement with the lengthy paper work process:
“Trent is really heavy on ethics; they make sure ethics need to be taken care of though we do not care much about it. Sometimes I get a bit paranoid about these ethics. Students have to deal with it themselves and it’s time consuming. Sometimes the privacy statement is unnecessary. We need contacts of business, not everything being kept anonymous.... Because of the ethics approval, they start surveying a bit late in the year: 2 months on forming the project and only 2 weeks in the survey. Students did great but we could be better off with more time spent on the actual survey process.”
Of the 16 organizations interviewed, 4 recommended cutting down on the bureaucracy process as the one difference that TCCBE and U-links can make to enhance the quality and experience of the program. Though the ethics approval process is strictly required by the higher education institution, efforts can be made from the faculty side to allocate time more effectively. For example, professors can move the deadline around so as to expand the actual research period, which can also partially address the aforementioned problem with research inaccuracy.
Short-term projects were occasionally mentioned by our interviewees and we received mixed opinions about them. Some organizations think that short projects are not a problem since the workload can be adjusted accordingly. Students do not always come in to do the research and instead, they can be assigned with more practical work such as creating a manual or updating promotional materials. Other organizations advocate that the same projects can be repeated over years so as to learn from various perspectives of students. It is also a good way to take advantage of the project done by previous students. There are, however, organizations that are enthusiastic about multi-year projects:
“It might be interesting to open up to multi-year projects, for example, if you were doing an evaluation that took you two years you would be able to accomplish more... I think that’s one of the weaknesses, in my experience, but it takes a long time for people to learn about the organization so it limits the impact of your organization or research. I think the opportunity to be involved in other ways with the organization instead of just related to research, like the example I gave above with Kelly, that student was more involved in other ways. And I can see that with other organization not just TCCBE, being involved more will improve any organization.”
Another organization voiced stronger opinions with regard to the value of long-term projects “There is no continuity between years. Over the summer months nothing happens and then in September we start all over again”. What we found from our research is therefore not consistent with the Stoecker and Tryon (2009) criticism of the short-term nature of service-learning projects. Since TCCBE and U-links programs are very diverse and not limited to research, the project content and workload can be adjusted accordingly to meet the time frame.
Part of our research question was to attempt to assess the impact that TCCBE and U-Links brokerage have had on the targeted communities of host organizations. As we have gone over in the literature review, the very definition of ‘community’ is open to interpretation, and is therefore difficult to assess. That being said, using specific methodologies we attempted to gather whatever information we could on the impact TCCBE and U-Links projects have had on the targeted communities. These are the findings from the eighteen interviews that we conducted.
We asked each interviewee if they thought that TCCBE and U-Links projects had a positive impact on the community, and why they did or did not think so. We also requested that they provide us with specific examples. Largely the responses we received reflect very positively on TCCBE and U-Links and on Community-Based Education in general. From the interviews we can ascertain that these organizations have a positive impact on the community because they often create relationships between community members and students that would not exist otherwise. Additionally, students produce tangible results at the end of their CBE project which can provide information and tools to increase public awareness about certain issues throughout Haliburton County. An example of this is the Water Stewardship program, within which students have been instrumental in further the educational components.
It was brought to our attention by one of the interviewees that the U-Links coordinator sits on a number of community development committees. It is linkages such as these that create a fruitful environment wherein students who connect with U-Links are more likely to become involved in the community. In addition to this, broker organizations are aware that the mandate of TCCBE and U-Links is consistent with enhancing community well-being. Specifically within Peterborough, TCCBE projects have had a positive impact by producing an Anti-Oppression manual that has had long-term, beneficial effects on multiple organizations. ‘Spaces of Racism’ (a report on racism in Peterborough) was noted as one project that had lasting effects on the community, because it was published in the Peterborough Examiner and brought local issues of racism into the spotlight. Furthermore, TCCBE and U-Links have potential to have a positive impact on the community because the host organizations often push to help Peterborough diversify, e.g. hiring immigrants etc.
There appears to be a strong connection between community well-being and CBE projects regarding issues such as accessibility and ethical issues, for example the Anti-Sweat Campaign at Trent, and the Ethical Purchasing Conference 2008. TCCBE and U-Links make connections with brokering organization, and this can be empowering because it lessens the possibility of burnout within an organization. On a similar note, TCCBE and U-Links provides organizations with capacity that they would not have otherwise, which enables them to complete projects faster and more efficiently. One interviewee stated, “I think that the links…with the people and young people are our best hope in changing [misinformed] attitudes and perceptions.” This links directly to community impacts because if a local organization is being provided with resources that enhance its capabilities, it can more effectively serve the needs of the surrounding community.
Community impacts do not seem to happen in isolation. It takes a combined effort of the host organization, the student, the community, service-learning, and horizontal linkages between organizations to make long-lasting positive impacts. It is important to keep in mind that in most cases it has been a combination of factors that has led to the community being positively impacted. The only negative response that we received in terms of the community impact was surrounding the very specific nature of CBE projects. There was a concern that because the projects are microcosms, they are only fulfilling the needs of a specific organization.
Doing research from within the community about the short- and long-term effects of TCCBE and U-Links projects would require a separate CBE project. The amount of work was such that we only had the resources to assess the impact of CBE projects on organizations, and the impact on the community through these organizations. The suggested study about community impact of TCCBE and U-links project will need to take into at account at least two issues. Firstly, as we have demonstrated in the literature review, it is strictly important to create a working definition for ‘community’ that can enable the measurement of effects. Specifically, community may range anywhere from the population of Peterborough and Haliburton, to the particular group of clients that the organization serve. Our research shows this wide spectrum, such as the school curriculum which serves elementary school students, the Workforce Integration Center which benefits both new immigrants and business owners in Peterborough. Projects like “Spaces of Racism”, on the other hand, have the potential to raise awareness in the public audience through the articles students published in the Peterborough’s Examiner. It is therefore necessary to make a distinction, not a separation, between types of communities which can be predicated upon detailed literature review and which should be made clear to host organizations at the interviews or focus group discussions. The second caution we want to make with regard to the future project on community impact is to limit the scope of study so as to produce a more quality and in-depth result. Each pillar we mentioned earlier about impact- economic, social, environmental, and social- will take more than one project to evaluate. We therefore suggest any future CBE project on community impact to focus primarily on one or two pillar at most. The topical focus will help the study to go in-depth given the time and resources of a CBE project. These two recommendations, along with our literature review on community impact, may be used as a good base to develop project proposal and inform the actual research work.
In the previous chapter, we identified the barriers that prevent TCCBE and U-links’ projects from fully realizing their success. They include the issues of poor or inadequate communication between students and the host organization, a shortage of organizational support, the quality of the research, the timeline of the project, and the short-term nature of CBE programs. Some of these problems pertain to the organizations or the students, while others can be significantly reduced if TCCBE and U-links play a stronger role. It is therefore recommended that:
In this research, we seek to identify the direct and contributing effects of TCCBE and U-links projects on host organizations and communities. We answer this question by using archival research and semi-structured interviews. For the most part TCCBE and U-Links have had positive effects on host organizations, as shown by enhanced functionality; public recognition; and fundraising. We also stress the importance of trust as the determining factor for a host organization to continue their engagement with community-based education projects.
With regard to community impact, the majority of host organizations do believe that these organizations have positive effects on the targeted communities. However, due to a lack of time and resources, and also due to the amorphous nature of target communities, we were unable to determine for certain from the perspectives of the communities themselves what impact projects have had.
We were also able to explore the reasons behind successful CBE projects and the limitations that some CBE projects share. The reasons for successes include effective communication; aptness of students’ skills; and organizations’ clearly defined expectations. Conversely, challenges to the success of CBE program includes poor communication; lack of organizations support; low quality of research; the short-term nature of most projects tied to the academic time-table and the resulting limited timeframe for the projects.
Based on these findings, we provide a detailed list of recommendations in chapter 6 for TCCBE, U-links and host organizations. The recommendations revolve around improving publicity and support provided to host organization; as well as the addition of training courses that equip students with knowledge on ethics and methods that can address the limitations of previous CBE projects. We hope that this report can provide TCCBE and U-Links with some important feedback, as well as a perspective on service-learning in Peterborough and Haliburton in general. This report might also be used in the future as a starting point for a CBE project that would assess the effects of TCCBE and U-Links based solely on the community.
Baker, Judy (2000). Evaluating the Impact of Development Projects on Poverty: A Washington, DC: The International Bank for Reconstruction and Development
Bell, S. & Carlson, R. (2009). Motivations of Community Organizations for Service-learning. In Stoecker & Tryon (Eds.) The Unheard Voice: Community Education and Service-learning, pp. 19-37. Philadelphia: Temple University
Berg, B. (2001).Qualitative Research Methods for the Social Science. Maine: Allyn & Bacon.
Crail, M. (2006). Students not just a source of cheap labour. In Personnel Today Issue, p. 1.
Cupps, S. & Olmosk, K. (2008). Developing Effective Internships Within Public Sector Organizations. In Public Personel Management 37(3), pp. 302-311.
Gamm, L., Castillo, G., and Williams, L. (2004). Education and Community-Based Programs in Rural Areas. In Gamm, L. and Hutchison, L. (eds.) Rural Healthy People: A companion document to Healthy People 2010, Volume 3. College Station, TX: The Texas A&M University System Health Science Center.
Garlick, S and Pryor, G. (2004). Benchmarking the University: Learning about improvement. Commonwealth of Australia: Report for the Commonwealth Department of Education Science and Training.
Gearan, M. (2005). Engaging the Communities: The Campus Compact model. In National Civic Review, pp. 32-40.
Gillham, B. (2000). The Research Interview.New York: Continuum.
Gray, D. E. (2004). Doing Research in the Real World. London: SAGE Publications.
Fisher, R.; Michael, F.; & Simmons, L. (2005). Understanding Contemporary University-Community Connections', Journal of Community Practice, 12(3), pp. 3-34.
Harris, L., Jones, M. & Coutts, S. (2010) 'Partnerships and learning communities in work integrated learning: designing a community services student placement program', Higher Education Research & Development, 29(5), pp. 547-559.
Krause, M. (2002). The Institutionalization of Community Interventions in Chile: Characteristics and Contradictions. In American Journal of Community Psychology 30(4), pp. 547-570.
Levy, H. (2004). Classic Texts. Community Development Journal, pp. 190-193.
Lichfield, N. (1996). Community Impact Evaluation.London: UCL Press, Web.
Maller, C., Kerachsky, S., and Thornton, C. (1992) Evaluation of the Economic Impacts of the Job Corps Program: Third Follow-up Report,Mathematical Policy Inc.
Melhuish, E., Belskey, J., Anning, A., Ball, M., Barnes, J., Romaniuk, H., Leyland, A. (2007). Variation in Community Intervention Programmes and Consequences for Children and Families: The Example of Sure Start Local Programmes. Journal of Child Psychology and Psychiatry 48(6) pp. 543–551.
Montenegro, M. (2002). Ideology and Community Social Psychology: Theoretical Considerations and Practical Implications. American Journal of Community Psychology 30(4) pp. 511-527.
Nafissatou, J., Faye, M., Moreau, A., and Cabral, J. (2004). The TOSTAN program evaluation of a Community Based Education program in Senegal. Population council: US agency for International development.
Patton, M. (1978). Utilization-FocusedEvaluation. California: SAGE Publications.
Pearce, J., Pearson, M., and Cameron, S. (2007). The Ivory Tower and beyond: Bradford University at the heart of its communities. ICPS-University of Bradford.
Raadt, J.D.R and Raadt, V. (2004). Normative Evaluation of Community Projects: A Multimodal Systems Approach. Journal of Systematic Practice and Action Research, 17(2).
Ravallion, M. (2001). The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation. In TheWorldBankEconomicReview15(1) pp. 115-140
Rosing, H. and Hofman, N. (2010). Notes From the Field: Service-learning and the Development of Multidisciplinary Community-Based Research Initiatives. In Journal of Community Practice 18(2) pp.213-232.
Sather, P., Weitz, B. & Carlson, P. (2007). Engaging Students in Macro Issues Through Community-Based Learning. In Journal of Teaching in Social Work, 27(3) pp.61- 79.
Savedoff, Levine, and Nancy Birdsall (2006). WhenWillWeEverLearn?ImprovingLivesThroughImpactEvaluation. Centre for Global Development, pp. 4-95.
Scholz, R.; Steiner, R. & Hansmann, R. (2004). Role of Internship in Higher Education in Environmental Sciences. In Journal of Research in Science Teaching 41(1) pp. 24-46.
Shenassa, E. (2002). Delivering the Goods: The Importance of Screening accuracy for Effective Community Intervention and Prevention. Journal of Community Psychology 30(2) pp. 197-210.
Sternberg, M. (1982). The Use of a Field-based Approach to Economic Education. Journal of Economic Education, pp. 47-50.
Stoecker, R. & Tryon, E. (2009). The Unheard Voice: Community
Education and Service-learning. Philadelphia: Temple University.
The research project is part of an International Development Studies course on Assessing Development Projects and was undertaken by four students: Claire Ryan, Alicia McDermott, Linh Phan and Maritza Queeley from September 2010 to April 2011.
We would like to give a sincere thanks to our instructor, Professor Michal Avram for her continuous support, informative tutorials and lecture materials that have prepared and allowed us to conduct the research and to write this final report. Professor Avram went out of her way to make sure our research and end product was of good quality and for that we are grateful. We would also like to thank the TCCBE and U-Link staff for their guidance and unwavering commitment to our needs and for allowing us the opportunity to be a part of such a valuable project. We were able to stay motivated because of their support and interest throughout the research process, in spite of their busy schedules and overwhelming workload.
This evaluation report would not have been possible without the compliance of the host organizations in Peterborough and Haliburton whom we interviewed. During the past months these organizations agreed to take time out of their busy schedules to meet and share with us their perspectives and stories. We would also like to thank our other classmates for their insightful questions and constructive inputs to improve our final report.
Last but not least we owe a great deal to our group members for taking initiative, meeting the deadlines and creating this quality evaluation report which will aid TCCBE and U-Links with future Community Based Education projects. Thank you all and it was great working with you.