Structural Topic Modeling helps us see patterns in what families with young children are saying about their experiences

Since April 2020, the RAPID Survey Project has been asking families with young children about their experiences and listening to what they have to say. We lift up caregivers’ voices in real time to inform policies and programs that support children, families, and communities.

RAPID collects these data through monthly online surveys that ask caregivers about what’s going well and where more support is needed. Open-ended survey questions offer space for caregivers to tell us about their experiences, in their own words. Two of the questions we use are: “What are the biggest challenges and concerns for you and your family right now?” and “What is helping you and your family the most?”

Caregivers are free to write anything they want in response to the questions. We analyze their answers to better understand their experiences and spot patterns in what we are hearing, such as connections between families’ economic circumstances and emotional well-being. Over the five-year span of the project, we have received more than 330,000 responses to open-ended questions.

Understanding patterns across caregivers’ written responses

To help make sense of this wealth of information, we use Structural Topic Modeling (STM). STM is an analysis method that looks for patterns in what people say or write in response to questions. Think of STM as bringing together a huge group of people in a room, asking them the same question that they answer all at once, finding the themes that run through everything that everyone is saying, and then counting how often each theme is expressed. STM allows us to analyze an extensive number of responses, identify the patterns that run through these responses, and track how frequently these patterns come up and how they are across families. And if performed on data gathered over time, STM allows us to identify changes in these patterns over time.

In STM, caregivers’ responses to an open-ended question is treated as a data point and every word is counted. Words that frequently appear together are used to determine the themes and then these themes are counted across all the responses being analyzed to determine their frequency and distribution. This approach allows us to tally how often specific concerns are mentioned, linking response frequency to a deeper understanding of the challenges caregivers face. By identifying these patterns, we gain valuable insights that can inform policy decisions and enhance support for families.

This method allows us to analyze a vast number of responses to bring attention to the range of worries, successes, and sources of support that families experience. Additionally, STM speeds up the analysis process, allowing us to efficiently review the survey responses and capture the perspectives of a wide array of caregivers. This efficiency helps us gain insights rapidly and share them with policymakers, ensuring there are timely responses to families’ needs.

Key learnings from using STM

STM analysis has helped us identify the following trends related to families’ physical health, economic stability, and emotional well-being:

• Many parents of young children are concerned about getting food for their families. Parents frequently raise concerns about food access in their open-ended responses. Parents most commonly report concerns about the high cost of food, not being able to afford enough food, losing food assistance, and low food quality and nutrition.
• Meeting basic needs is a heavier load for lower-income families. Lower-income caregivers are more likely than middle- and higher-income caregivers to write about difficulties accessing basic needs, including food.
• Child care is a top concern for families with young children. Since early on in RAPID, we have been asking, “What would you like your elected officials or other policymakers to know about how you and your family are doing and what you need during this time?” We have received almost 30,000 answers to this question alone over the last five years. In 2024, during an election year that had important implications for young children and their families, we published our STM findings to this open-ended question. Across all demographics, family structures, and income levels, the topic parents most wanted lawmakers to know about was the need for additional government support in helping meet child care needs. Another strong pattern that emerged from STM analysis was parents noting that lack of child care causes stress and can undermine the attentive, nurturing caregiving that decades of research shows supports children’s positive development.
• Child care providers who experience material hardship are more likely to also experience emotional distress. Analysis of RAPID fixed-response questions has shown that when providers experience material hardship, they also report higher levels of emotional distress (anxiety, depression, loneliness, and stress). STM analysis confirmed this finding and showed that when providers wrote about feeling anxious and depressed they were also likely to write about challenges affording essentials.
• Many families with young children shared common challenges and sources of support during the pandemic. Throughout the Covid-19 pandemic, we invited parents to write about their experiences. In analyzing these responses with STM, we found that families wrote about struggles balancing work and child care, experiences of emotional distress, and difficulty finding stable employment. We saw specific challenges across household income levels and races and ethnicities, including social isolation, difficulty getting support, therapy, and special-need services for children, and worries about staying safe. We also asked families what was helping them the most. The majority of caregivers wrote about support from family and friends and being able to work from home. Lower-income parents also wrote about how economic support from state and federal programs helped them to meet their family’s basic needs.

Conclusion

Together with the answers to fixed-response survey questions, using STM to analyze families’ written answers to open-ended questions helps us understand caregivers’ and families’ experiences more deeply and report findings more rapidly and responsively. This combined approach promotes real-time, data-informed policies and programs that support children, families, and communities.