Event: CES Webinar – Critical reflexivity when working with marginalized populations

Date: May 18th 2022, from 12 pm to 1 pm EDT

Register here or visit the post through the Canadian Evaluation Society website.

Reflective and reflexive practice is increasingly being recognized as a core competency within the profession of evaluation (CES, 2018). Critical reflexivity is suggested as a method by which evaluators can monitor their cognitive biases, better understand their privilege, and increase their awareness of power dynamics. The intended goal of critical reflexive practice is to reduce the risk that an evaluator uncritically, and usually unwittingly, perpetuates systemic oppressions during an evaluation and causes further harm. Furthermore, if the aim of a program or service is to be equitable, integrating a disciplined practice of reflection is increasingly being recognized as a method of practicing social justice and anti-oppression when evaluating those programs and services (Archibald, Neubauer & Brookfield, 2018; Caldwell & Bledsoe, 2018; Jewiss & Clark/Keefe, 2007; Hall, 2019; van Draanen, 2016).

Nick Petten (he/they) has been practicing critical reflexivity in their evaluation and believes that it can be used by evaluators to maintain an ethical stance and be culturally responsive to groups of people that do not fully identify with the dominant ideology in Western society. This practice has emphasized the need to create safe spaces for discourse in a system that is characterized by hyper-consumerism, imperialism, colonialism, cis-hetero-patriarchy, and white supremacy. Equally important to understanding how a system can be oppressive is understanding that all humans, with all their cognitive flaws, are universally deserving of love and compassion and are always subject to change.

Webinar focus:

In this webinar, participants will have an opportunity to learn more about some techniques and practices in integrating critical reflection and reflexivity in their evaluation practice. Furthermore, participants will discuss the role and impact of critical reflection and reflexivity on an individual’s practice, within the dynamics of both a team and the professional community of evaluators.

Biography of presenter:

Nick Petten is a researcher, consultant, and evaluator with over 10 years’ experience working with culturally diverse populations. Nick has in-depth and front-line experience working with children and adolescents with multiple needs and capabilities through parent education programs, early childhood education centres, youth programs, adolescent treatment centres and many other community programs. He is also dedicated to building the capacity of the organizations he works with through several volunteer and board engagements.

Learning objectives:

In addition, this webinar aims to strengthen capacities in line with the following Competencies for Canadian Evaluation Practice:

Domain 1.0 Reflective Practice: Competencies focus on knowledge of evaluation theory and practice; application of evaluation standards, guidelines, and ethics; and awareness of self, including reflection on the implications of one’s practice and the need for continuous learning and professional growth.

And specifically, Competency 1.8: Uses self-awareness and reflective thinking to improve own practice and to pursue purposeful learning and development as an evaluator.

  1. Recognizes and manages one’s own biases and strives to reduce their impact on evaluation practice.
  2. Identifies one’s shortcomings as an evaluation professional and actively seeks to strengthen them.
  3. Learns from personal successes and failures within projects.
  4. Pursues relevant opportunities for professional learning and development.

CES’s updated competencies on Reflective Practice and Ethics (https://evaluationcanada.ca/sites/default/files/crwg_revisions_clean_version_april_23.18.pdf)

If you have any questions please contact Vicky at secretariat@evaluationcanada.ca.

Webinar: Children and Adolescents Speaking Truth to Adults: Ethics and Participation

In April 2019, Nick Petten, along with a few colleagues working in the intersections of children’s rights and program development, presented their work to YouthPower, a USAID-funded education initiative. The webinar and a description can be viewed on the USAID website here.

Event Description:

In order to speak truth to power, children and adolescents have to overcome multiple obstacles. Despite several conventions supporting their right to participate in decisions that concern them (UN Convention on the Rights of the Child, Treaty of Lisbon), their involvement often remains limited. Children and adolescents’ meaningful engagement in research and evaluation is indeed challenging. It requires time and resources to adapt to evaluation tools and materials. It also involves specific ethical issues, such as informed parental consent and child assent.

This panel brought ethics to the forefront of their discussion on children’s participation. It considered how several gatekeepers (parents, schools, local authorities) can facilitate or interfere with their right to participate. The panel offered cross-national perspectives with examples from Canada, Europe and Low and Middle Income Countries and focused on ways to ensure meaningful participation, such as visual methods and robust consent processes.

Event: Critical Reflexivity for Program Evaluators

Nick Petten recently presented at an EvalCafe event on his work around critical reflexivity. The event description and links to resources can be accessed on EvalCafe’s event listing page on Meetup.com.

Content from the presentation can be accessed here for free.

Event description:

Reflective and reflexive practice is increasingly being recognized as a method by which evaluators can monitor their biases, better understand their privilege and increase their awareness of power dynamics. The intended goal of reflexive practice is to reduce the risk that an evaluator uncritically perpetuates systemic oppressions during the course of an evaluation. Furthermore, if the aim of a program or service is to be equitable, integrating a disciplined practice of reflection is increasingly being recognized as a method of practicing social justice and anti-oppression when evaluating those programs and services (Archibald, Neubauer & Brookfield, 2018; Caldwell & Bledsoe, 2018; Jewiss & Clark/Keefe, 2007; Hall, 2019; van Draanen, 2016).

Nick Petten has been practicing reflexivity in his evaluation practice and has opened this practice to investigation through a research project on team-based critical reflexivity. He also occupies some leadership positions within his professional associations, CES and AEA, and is aware of the increasing importance that is being placed on reflective practice and reflexivity as a core competency within the profession. His ecology of evidence tells him that reflective practice and critical reflexivity can be used by evaluators to maintain an ethical stance and be culturally responsive to groups of people that do not fully identify with the dominant ideology in contemporary Western society.

In this presentation, participants will have a chance to learn more about some techniques and practices in critical reflexivity and then discuss the role and impact of critical reflexivity on an individual’s practice, within the dynamics of a team and within the professional community of evaluators.

Those interested in attending are encouraged to read the following short articles:

– AEA’s Statement Regarding Racism and Inequality in our Society (https://www.eval.org/Full-Article/statement-from-the-aea-board-of-directors-regarding-racism-and-inequality-in-our-society)

– CES’s updated competencies on Reflective Practice and Ethics (https://evaluationcanada.ca/sites/default/files/crwg_revisions_clean_version_april_23.18.pdf)

Seeking Assent from Children in Research and Evaluation

Children and young people are fully capable of expressing and communicating their lived realities and that those realities are a valued focus of study. Just like any potentially vulnerable and marginalized population, conducting research with children requires an ethical understanding of how to involve them in the research and how not to perpetuate systemic injustices and discriminations that they often face in their daily lives.

Back to Basics

All research involves at least two main participants, the researcher and the subject. The relationship between researcher and subject is defined by the methodology of the research. The methodology is critical in determining any truth claim that a research study suggests. When researching childhood, a critical methodological concern is the power dynamics that occur between adults and children. Unequal power can compromise your data and severely distort truth. But more importantly, it can cause harm to your subject. In order to make any truth claims based on the scientific process, careful consideration must be given to the ethics of involving children in research so that power dynamics can become more equalized, while also acknowledging that they may never truly disappear.

The UN Convention on the Rights of the Child articulates various rights and responsibilities that children have. Article 3 states that the best interests of the child must be the primary concern in making decisions that may affect them—including on their participation in research. As holders of rights, even the youngest children are entitled to express their views. Young children are acutely sensitive to their surroundings and acquire an understanding of people, places and routines in their lives, along with the awareness of their own identity. They make choices and communicate their feelings, ideas and wishes in numerous ways, long before they are able to communicate through spoken or written language. It may be through drawings, body movements, or eye contact. Children even use silence to express themselves.

The new sociology of childhood is a scientific discipline that promotes the process of obtaining assent from children to participate in research. This discipline recognizes that children’s agency in research is based on the view that children are individuals who experience their worlds in unique ways, and that these unique experiences are a valued focus of study. So, what is assent? In addition to parents and guardians providing informed consent, seeking children’s assent is a process where children can exercise choice about their own research participation. There are different conceptions of assent and consent between the medical field and sociological studies—so, it is important to know how to use it and within which context.

So, how do you seek assent with children?

Tip #1: Researchers often balance written and visual information with verbal information. Knowing the individual child participants well enough to be able to make judgments about what information is necessary to facilitate their understanding is an ongoing process, also known as process assent. Respect for children as an individual also includes respect for their developing competencies. Competence is a developmental process. That means that the content and process of assent may have to be explained in different ways at different times. Develop an assent and accountability framework that will help you explain the process of research and its findings in a child-friendly manner and systematically ‘check-in’ with children about their participation in the research.

Tip #2: It is important that voluntary participation involves an active decision to take part—rather than failure to give dissent, or passive resignation. The voluntary nature of participation can also be undermined by things like obligations to parents or teachers. Familiar contexts, people and experiences can promote participation. However, familiarity can also put pressure on children to participate. Supportive contexts facilitate children’s developing competencies, while recognizing that their interests and understandings change. Develop protocols to use when seeking children’s assent that considers as many factors as possible about why they would answer in a particular way. 

Tip #3: Work with and build a relationship with children’s gatekeepers. In addition to parents as gatekeepers, there may also be educators or supervisors at early learning and child care centres. Gatekeepers provide a familiar and trusted mediator between participants and researchers. They may have knowledge about the social and cultural contexts and language protocols. Building a relationship will give the researcher a better sense of a child’s interests and preferences which provides opportunities to gauge their comfort with participation. These relationships also allow a child to get to know the researcher within supportive and safe interactions. Dedicate time to building relationships and having conversations with the gatekeepers and remember that such conversations need to strike a balance between providing critical information and too much detail that can lead to confusion.

Tip #4: Researchers can use reflexivity to analyze the personal, intersubjective and social processes which shape their research and acknowledge their role. Reflexivity includes critical and ethical reflection on the assumptions underpinning the research and the processes to obtain assent, conduct the research and interpret the findings. Researchers should also be able to explain in an understandable way how the research findings will be used. Children may have critical and ethical questions about who benefits from the research; what is their future involvement; and how will they be represented in the research findings. Engage in internal reflection through an explicit and systematic process throughout the research process and think about how the insights gained may influence how you converse with children and their gatekeepers.

I recently presented on a webinar for professionals in the social research and evaluation sector that was organized by YouthPower. Check out the video below for my section of the webinar including the question and answer period.

Online Resources

If you are starting to think about how to genuinely involve children’s participation in research, fortunately, there is lots of great reading material out there. By reading some of the great material on research with children you can begin to understand children’s position in research–and even in our adult-centric world. Meaningfully and authentically involving children’s participation in research can contribute immensely to the truths that you are trying to uncover in your research. Not only will your programs or studies be better off with a good understanding of your stakeholders, but you will be actively supporting and advocating for the realization of the UN Convention on the Rights of the Child–one of the most important and widely accepted international conventions.

Here is some material to get you started on you reading adventure:

Also, these organizations publish material that relates to involving children in research, monitoring and evaluations:

This post is cross posted with the following article from the American Evaluation Asssociation’s AEA365 Blog.

Academic Articles on Child Assent and Participation

Schiller, W., & Einarsdottir, J. (2009). Special Issue: Listening to young children’s voices in research – changing perspectives/changing relationships. Early Child Development and Care179(2), 125–130. http://doi.org/10.1080/03004430802666932 

Schnoor, O. (2012). Early childhood studies as vocal studies: Examining the social practices of “giving voice to children’s voices” in a crèche. Childhood20(4), 458–471. http://doi.org/10.1177/0907568212466902 

Spyrou, S. (2015). Researching children’s silences: Exploring the fullness of voice in childhood research. Childhood. http://doi.org/10.1177/0907568215571618 

Underwood, K., Chan, C., Koller, D., & Valeo, A. (2015). Understanding Young Children’s Capabilities: Approaches to Interviews with Young Children Experiencing Disability. Child Care in Practice0(0), 1–18. http://doi.org/10.1080/13575279.2015.1037249 

White, L. a. (2014). Understanding Canada’s Lack of Progress in Implementing the un Convention on the Rights of the Child. The International Journal of Children’s Rights22(1), 164–188. http://doi.org/10.1163/15718182-02201002 

Pyle, A., & Danniels, E. (2015). Using a picture book to gain assent in research with young children. Early Child Development and Care

Nussbaum, M., & Dixon, R. (2012). Children’s Rights and a Capabilities Approach: The Question of Special PriorityPublic Law and Legal Theory Working Papers

Mayes, E. (2015). Shifting research methods with a becoming-child ontology : Co- theorising puppet production with high school students. Childhood, 1–18. http://doi.org/10.1177/0907568215576526 

Matthews, S. H. (2007). A Window on the “ New ” Sociology of Childhood. Sociology The Journal Of The British Sociological Association1, 322–334. 

Hurley, J. C., & Underwood, M. K. (2002). Children’s understanding of their research rights before and after debriefing: informed assent, confidentiality, and stopping participation. Child Development73(1), 132–143. http://doi.org/10.1111/1467-8624.00396 

Hohti, R., & Karlsson, L. (2013). Lollipop stories: Listening to children’s voices in the classroom and narrative ethnographical research. Childhood, 0907568213496655-. http://doi.org/10.1177/0907568213496655 

Harcourt*, D., & Conroy, H. (2005). Informed assent: ethics and processes when researching with young children. Early Child Development and Care175(6), 567–577. http://doi.org/10.1080/03004430500131353 

Grimes, S. M., & Street, W. H. (2008). Researching the Researchers : Market Researchers, Child Subjects and the Problem of “ Informed ” Consent. International Journal of Internet Research Ethics1(1), 66–91. 

Frost, K. H., Lincoln, S. H., Norkett, E. M., Jin, M. X., Gonzalez-Heydrich, J., & D’Angelo, E. J. (2016). The Ethical Inclusion of Children With Psychotic Disorders in Research: Recommendations for an Educative, Multimodal Assent Process. Ethics & Behavior26(2), 163–175. http://doi.org/10.1080/10508422.2015.1063425 

Finelli, M., Rocca, C., Marchesi, V., & Maggi, S. (2014). Ask Them : Child Participation in the Development of Educational Services. Canadian Journal of Children’s Rights, 219–234. 

Dockett, S., & Perry, B. (2011). Researching with young children: Seeking assent. Child Indicators Research4(2), 231–247. http://doi.org/10.1007/s12187-010-9084-0 

Dockett, S., Einarsdóttir, J., & Perry, B. (2012). Young children’s decisions about research participation: opting out. International Journal of Early Years Education20(3), 244–256. http://doi.org/10.1080/09669760.2012.715405 

Dockett, S., Einarsdottir, J., & Perry, B. (2009). Researching With Children: Ethical Tensions. Journal of Early Childhood Research7(3), 283–298. http://doi.org/10.1177/1476718X09336971 

Di Santo, A., & Kenneally, N. (2014). A call for a shift in thinking: Viewing children as rights-holders in early childhood curriculum frameworks. Childhood Education90(6), 395–406. 

Bourke, R., & Loveridge, J. (2014). Exploring informed consent and dissent through children’s participation in educational research. International Journal of Research & Method in Education37(2), 151–165. http://doi.org/10.1080/1743727X.2013.817551 

Baines, P. (2011). Assent for children’s participation in research is incoherent and wrong. Archives of Disease in Childhood96(10), 960–962. http://doi.org/10.1136/adc.2011.211342 

Adcock, K. G., Hogan, S. M., Elci, O. U., & Mills, K. L. (2012). Clinical Investigation Do Illustrations Improve Children’s Comprehension of Assent Documents ? Journal of Pharmacological Therapy17(3).

AEA’s Updated Ethical Guidelines

Purpose of the Guiding Principles: The Guiding Principles reflect the core values of the American Evaluation Association (AEA) and are intended as a guide to the professional ethical conduct of evaluators.

As a member of AEA and on the leadership team of the Youth-Focused Evaluation Topical Interest Group (YFE TIG), Nick attempts to operationalize these guidelines into all evaluation projects and initiatives to ensure that all participants are not harmed during the course of the research and evaluation work. Even better than doing no harm, these guidelines can serve to demonstrate the benefits that come with research and evaluation work which, ultimately, are meant to serve the common good in society.

AEA members recently voted on whether to adopt these guidelines. Please follow this link to learn more about the guidelines including how they were developed.

New Resources on the YFE Website!  Child and Youth Participation in Speaking Truth to Power

New Resources on the YFE Website!
Child and Youth Participation in Speaking Truth to Power

Written and assembled by Nick Petten


Seeking children and young people’s perspectives, ideas, opinions, and experiences through participatory approaches in research and evaluation is a valued focus of study and has the ability to speak truth to power. The power that adults typical wield in society often speak on behalf of children and young people and may not truly represent them and, sometimes, completely misrepresent them in order to gain more power.

Children and young people are one of the most marginalized and oppressed groups in society. And yet, today’s adults expect that tomorrow’s adults (children) will solve some pretty wicked, global, complex and complicated challenges that we face as a civilization—climate change, growing income inequality, a resurgence of autocratic and nationalist governments and policies, to name a few. We pin our hopes and dreams on our children and yet deny them the chance to truly participate in society, whether we allow them to vote at a younger age, decide what to learn in the classroom, or design an evaluation or research project.

Thankfully, there are lots (and growing number) of academics, evaluators, thought-leaders and practitioners that are acting in the best interests of the child and young person by co-designing approaches, methods, and processes to genuinely include their participation in things that matter most to them, like how the local playground should be designed, choosing politicians that represent their best interest, or how national policies could be designed to benefit their adult lives and thus the future of humanity. From these approaches, methods, and processes, children and young people stand a better chance of ‘speaking truth’ to powers that haven’t traditionally represented them authentically or have even co-opted their ‘voice’ to progress their own agenda.

If you are someone that works with children and young people and are looking for ways to design, implement and/or evaluate programs and services for them, we’ve collected and curated a list of resources that could help you in your journey.

News: Published Article on Reflexivity

We are proud that one of our associates, Jenna van Draanen, has published an article on ‘Introducing Reflexivity to Evaluation Practice’ in the American Journal of Evaluation. The practice of reflexivity is something that we are actively exploring and employing in our evaluation and research work at Petten Consulting. We believe that it makes us more culturally competent and responsive to the complex needs of our clients. If you have access to scientific journals, please visit the Journal’s website and download the article.

Feel free to contact us with any questions. We’d love to hear what you think about the practice of reflexivity!

Abstract

There is currently a paucity of literature in the field of evaluation regarding the practice of reflection and reflexivity and a lack of available tools to guide this practice—yet using a reflexive model can enhance evaluation practice. This paper focuses on the methods and results of a reflexive inquiry that was conducted during a participatory evaluation of a project targeting homelessness and mental health issues. I employed an action plan composed of a conceptual model, critical questions, and intended activities. The field notes made throughout the reflexive inquiry were analyzed using thematic content analysis. Results clustered in categories of power and privilege, evaluation politics, the applicability of the action plan, and outcomes. In this case study, reflexivity increased my competence as an evaluation professional: The action plan helped maintain awareness of how my personal actions, thoughts, and personal values relate to broader evaluation values—and to identify incongruence. The results of the study uncovered hidden elements and heightened awareness of subtle dynamics requiring attention within the evaluation and created opportunities to challenge the influence of personal biases on the evaluation proceedings. This reflexive model allowed me to be a more responsive evaluator and can improve practice and professional development for other evaluators.

van Draanen, J. (2016). Introducing Reflexivity to Evaluation Practice An In-Depth Case Study. American Journal of Evaluation, 1098214016668401.

Report: Right To Play’s PAQE Program Baseline Evaluation 2016

The team at Petten Consulting (Nick, Jenna and Priya) worked with Right To Play, a child-oriented international development organization, to establish a baseline for the Play for the Advancement of Quality Education (PAQE) program that is operating in 8 countries around the world and funded by the Government of Canada. The PAQE program seeks to, “improve educational attainment and participation of girls and boys, aged 2-15 in eight countries with a child friendly approach to learning with play” (Global Affairs Canada, 2016).

Download the full report here [via Dropbox].

Report: UNICEF Evaluation Report – Peacebuilding, Education and Advocacy Programme

Nick Petten worked with an international team of evaluators at Avenir Analytics on the global evaluation of the Peacebuilding, Education and Advocacy (PBEA) programme in 14 conflict-afflicted countries around the world. Using an outcome harvest approach, the evaluation used mixed methods to evaluate the progress and impact of UNICEF’s and partnering organizations’ activities for peacebuilding, social cohesion and resilience, and education.

Download the report at UNICEF’s Global Evaluation Database

PBEA was designed to work at the nexus of education and conflict, to improve conflict sensitive programming and contribute to peacebuilding, social cohesion and resilience, using education as an entry point. Meaningful results were achieved in integrating conflict sensitivity and peacebuilding into education sector reform and other policies; building institutional capacities of UNICEF, governments and implementing partners for conflict sensitive programming and peacebuilding; building individual and community capacities to mitigate causes of conflict; and increasing access to conflict sensitive education. More results are expected in generating learning and evidence of what works in social services for peacebuilding. The evaluation made eight overarching conclusions.

UNICEF Evaluation Database: 2015 Global: Evaluation of UNICEF’s Peacebuilding, Education and Advocacy Programme (PBEA)