Groundtruthing with Youth
We recently completed a fun project where we sat with groups of youth at afterschool programs to get their input on survey design! The San Francisco Department of Children, Youth, and Their Families (DCYF) had historically implemented a Youth Survey to program participants of their youth-serving grantee organizations. With a refresh of their strategic plan and a new funding cycle, DCYF decided to revamp their long, cumbersome Youth Survey and Clarity was retained to implement a process to gather youth feedback on the survey and to make recommendations on a new survey design that would honor youth feedback and more accurately reflect their experiences.
Over the course of 23 input sessions, we spoke with 128 youth who gave generously of their insights and perspectives to improve this tool. One survey item in particular, which DCYF had relied upon for years, was called into question by the youth and new questions were created that will be tested this year. The youth development literature is clear that the presence of a “caring adult” in a youth’s life is critical to positive outcomes in academic achievement and engagement, emotional and behavioral wellbeing, social and interpersonal skills, identity development, and future orientation. As such, DCYF historically included survey items such as, “Program staff really care about me” or “Program staff believe I will be a success”, to which youth were asked to dis/agree on a 5-point scale.
In our input sessions, youth challenged such items, wondering how they were supposed to know whether staff cared or what they believed. Instead, as the natural scientists youth are, they suggested that items gauge behavioral cues that would indicate “care” that they could empirically observe. They wanted questions they “feel like they can answer” based on their own experience and observations, not speculation.
For example, one item asked whether program staff notice when they are upset. Youth indicated that they may not be able to observe whether staff notice if they are upset but they may be able to answer questions if asked about actions staff took that would indicate that staff noticed them, such as, “When I am upset about something, program staff are there for me”, or “Program staff approach me when I seem upset.”
Such deep insights these young people shared will profoundly change the meaning and accuracy of the data collected and inferences made from them. This is how listening to the communities you serve will help improve outcomes for everyone!
