Learning & Growing from a Null RCT: Dost Education

May 29, 2025

Over the last few years, Dost Education has tested the impact of their parenting program on young children’s development, through a “gold standard” randomized controlled trial (RCT)—and the results were perhaps surprising. Despite strong parental engagement in the program, they found no measurable change in child-level learning outcomes. This null result may sound disappointing, but for Dost – and for us at The Agency Fund – we see a precious moment for reflection, iteration, and continued exploration. In a sector where evidence of impact often drives scale and funding, it is critical for us to share and benefit from null results, just as we celebrate success. The research reshaped Dost’s approach, and gave them concrete next steps. It also updated our priors, and added nuance to our understanding of how to support parents as they nurture our next generation.
– Temina Madon, The Agency Fund

ECD’s Promise & Mixed Evidence Base

It is well-established that the first three years of life are crucial for child growth, health, and development. High-quality parenting programs that train parents in child stimulation and early childhood development (ECD) have long-term benefits. Landmark studies such as the Perry Preschool Project and Nurse-Family Partnership in the United States, and international models like Reach Up in Jamaica, the LEAPS program in Pakistan, and home-visiting initiatives in Colombia have shown lasting improvements in child cognitive development, earnings, and educational outcomes (Gertler et al., 2014; Attanasio et al., 2014; Yousafzai et al., 2014). Because these programs rely on in-person home visits, they have been difficult to scale to all families in need. In response, nonprofits and governments, in collaboration with researchers, have piloted low-cost, tech-enabled parenting programs in hopes of scaling them effectively.

While the results of these lower-cost, technology-assisted programs show potential, further innovation and research are required. While a study leveraging audio content for parents in Guatemala found positive effects on parent-child interaction and vocabulary (Arteaga & Trias, 2023), a recent randomized controlled trial (RCT) of a remote program using the Reach Up curriculum in Jamaica showed mixed results: the study found improvements in parenting activities and praise, but no impact on parent-child interaction or vocabulary (Smith et al., 2023). Another study in Syria and Jordan (Rafla et al., 2024) found no effects of a phone-based intervention, though this is possibly because the content prioritized health topics over child stimulation. In Ghana, a parenting program via SMS also had mixed results (Aurino & Wolf, 2024): for parents with some formal schooling, it improved engagement and children's skills; for parents without formal schooling, it worsened educational inequality.

Dost’s Learning Journey

Given this mixed evidence base, Dost has always taken an iterative approach to developing, testing, and refining our program based on engagement data and feedback. Our “phonecast” program – which aims to provide daily support to parents with kids aged birth to six years old – was initially designed to be delivered through pre-recorded audio messages. Covering 18 parenting topics, audio content ensured accessibility: parents could engage at their own pace and did not need to be able to read or have a smartphone to use Dost.

From 2017 to 2018, we developed the program in Delhi and Uttarakhand in North India, based on community demand and engagement data. In 2019, we vetted our theory of change with Harvard’s Center for the Developing Child. From 2020 to 2021, we conducted a parent survey study with 60 Decibels. In that study, 91% of survey participants reported feeling more confident as parents, and 94% reported better knowledge of managing their child’s behavior. Without Dost, only 1% had access to alternative parenting support. Then in 2022, we ran an internal video study that coded parent-child interactions, finding that our program strengthened parent scaffolding and led to socio-emotional improvements such as increased father engagement and reduced harsh discipline. In parallel, we launched a 2-year study with Sambodhi (an India-based evaluation org) surveying over 1,500 parents.

At this stage, we had what the private sector might call a strong “product-market fit”: parents were consistently engaging with Dost and thus, the program met a real need for families. Nonetheless, it was important to us to determine if these improvements translated to measurable child outcomes.

So in 2023, we embarked on an RCT, generously funded by The Agency Fund, to further pressure test Dost’s phonecast program and contribute to this nascent and growing evidence base.

Our RCT: Generating Insights and Learnings from Mixed Results

We designed an ambitious RCT1, aiming to measure the phonecast program’s impact on parental engagement and child learning outcomes. This study was restricted to the program we run for children 6-30 months old; it did not include children in the older age groups.

The results were a mix of promising, neutral, and disappointing findings:

High Engagement: Over 80% of participants completed at least five phonecasts, with half engaging for more than ten—a strong indicator of accessibility and appeal.
Unexpected Emotional Impacts: While we hoped for increased parental self-efficacy, caregivers’ confidence slightly declined, and anxiety rose—an outcome observed in other programs, like cash transfers for new mothers, where increased knowledge sometimes brings heightened responsibility and stress (Arteaga & Trias, 2023).
Null Child Outcomes: Child development indicators, including language and cognitive skills, remained unchanged. This outcome calls into question the potential and limitations of low-intensity, phone-based interventions in achieving significant developmental gains for children in a short time frame, without supplementary relational or in-person support.

These results raised an important question: How do we respond to a null result? Should a program be abandoned after one RCT, or should it be iterated upon and improved? What can others learn from this evidence?

Learning from a Null Result: Next Steps

Guided by research principles such as those in “So, You Got a Null Result, Now What?,” we identified two key hypotheses to continue testing, and we’ve already started adjusting and testing our approaches:

Measuring Beyond Child Development: The RCT assessed a curriculum that covered a broad age range (from birth to two years old) to make it easier to register families without too many pre-screening questions. Dost’s curriculum emphasized daily parenting challenges (e.g., discipline, engagement during chores), since our theory of change is that more empowered parents will boost child development outcomes. But the RCT did not assess these areas. The aforementioned Sambodhi study and other internal focus groups have highlighted that our program has broader impacts: parents’ use of violent discipline, for instance, decreased by 23% from baseline to endline. We are now expanding the scope of what we measure to more closely meet our theory of change and curriculum. One example: we’re understanding how an intentional gender lens reduces maternal burden and anxiety by encouraging fathers to take on more equitable caregiving roles. We have also incorporated tracking how caregivers use positive reinforcement to manage children’s behavior and engage in daily playful activities with their children.

Combining A/B Testing Tech with Human Support to Improve Child Development: Behavioral change does not happen in isolation. Unlike standalone tech interventions, successful early childhood programs often involve peer learning and community engagement. These can include caregiver groups, sessions led by Anganwadi (government-run daycare center) workers, and father-focused interventions. We are now testing parent groups and peer support as part of our model, just like we have tested many product-based features in the past. Over the next 12 months, we’ll continue A/B testing to identify the optimal mix of digital and human approaches. What is awesome is that we can leverage our product-led A/B testing approaches to extend to offline aspects of the program, as well. The digital component itself has also evolved: our new outcome-specific modules (e.g., “oral language development for 1-year-olds”), which align more with gold-standard developmental assessments, have shown strong promise. (Shout out to TalkTogether Oxford for co-designing this with us.) In an early pilot of this narrower curriculum, we found that it improved oral language development significantly over our initial approach.

Our revised approach integrates digital tools with community engagement to bridge the gap between parents’ needs, knowledge, practices, and desired child outcomes. We may implement another RCT in the future. But for now, we’re running a number of faster, smaller-scale A/B tests to assess and refine the effectiveness of our revised approach.

Moving Forward: Why Sharing Null Results Matters

Some people treat RCTs as a “one and done” validation tool—especially in fields like medicine, where, after rigorous trials, a single vaccine can be scaled universally with predictable results. But ECD and parenting interventions don’t work that way. Every child’s development trajectory is different, every family and community context is unique, and behavior change is deeply relational. ECD programs must be adaptive and responsive to the environments they serve—and yet, we still have to find ways to make them affordable and scalable.

Our experience underscores the importance of treating RCTs as part of an iterative learning process rather than a final judgment on an organization. Instead of seeing it as a setback, we’re treating our results as a roadmap for future exploration and research that will help us deliver on our commitment to getting kids ready for school. Sharing null results is critical for advancing the field—allowing others to learn from our experiences, refine their own approaches, and contribute to building more effective interventions.

We encourage our ecosystem to embrace research transparency, experiment boldly, and continue to refine programs based on evidence. By doing so, we can collectively advance the ECD field and ensure more parents and children receive the high-quality support they need to thrive.

Acknowledgements: Thank you to families who participated, the research teams, principal investigators, funders, and the Dost team.

Sneha Sheth is the Co-Founder of Dost Education and CEO from 2015-2024

Arteaga I, De Barros A, Ganimian IA, J-PAL South Asia. The challenges of scaling up effective child-rearing practices using technology in developing settings: Experimental evidence from India.