Why some policy programs fail before they start, and what human-AI collaboration in behavioral science can get right.

Juhi Jain, Takagi Yusuke

SECTOR

FARM

PROJECT TYPE

Experimentation & Field Methods

DOI

https://doi.org/10.62372/WORL9432

Location

Kenya

BEHAVIORAL THEME

Human-AI collaboration | Behavioral System Map Generator (BSMG)

OVERVIEW

This Busara Groundwork explores why some policy programs fail before they start, and what human-AI collaboration can get right in behavioral science. Drawing on Busara’s retrospective application of Godot Inc.’s Behavioral System Map Generator (BSMG) to two real-world projects, the thought piece examines where AI can accelerate behavioral diagnostics and intervention design, and where human expertise remains essential for contextual judgment. It argues for a model of collaboration in which AI provides speed, breadth, and structure while humans define the questions, assess evidence, interpret context, and make decisions.

Research Questions

How can AI support behavioral diagnostics and intervention design without replacing human expertise?
What can AI tools contribute to the early stages of behavioral science research and systems analysis?
What are the limitations of AI-generated behavioral diagnoses, particularly around context, evidence quality, and prioritization?
What should effective human-AI collaboration look like in behavioral science?

Methods

The thought piece draws on Busara’s experimental use of Godot Inc.’s Behavioral System Map Generator (BSMG). Busara retrospectively applied the tool to two real-world projects led by its Food Systems, Agriculture and Climate Resilience Management (FARM) team: a behavioral diagnostic on chisel plough adoption in Makueni and Kilifi, Kenya, and the development of a behavioral scorecard for agricultural technology adoption. The simulated AI outputs were compared with Busara’s human-led processes to examine where the tool could support, and where it fell short of, rigorous behavioral diagnostics.

THEMATIC AREAS

Key Findings

Rigorous behavioral diagnosis requires understanding the specific behavioral, social, institutional, and contextual factors shaping a problem. Skipping or shortening this process can lead programs to fund the wrong interventions.
AI can rapidly structure a problem space, generate initial behavioral system maps, identify potential leverage points, and suggest intervention pathways.
In the chisel plough case, the BSMG captured key drivers such as labor burden, cost perception, service reliability, peer influence, and trusted advisors, but did not adequately distinguish the differences between Makueni and Kilifi or the relative importance of each barrier.
In the AgTech scorecard case, the tool was useful for early scoping and for mapping barriers to COM-B constructs, but its prioritization logic conflated diagnosis with intervention selection by emphasizing feasibility rather than behavioral significance.
AI systems trained primarily on existing literature can reproduce the biases and blind spots within that evidence base, including the underrepresentation of African research and non-WEIRD populations.
The strongest model is not human versus AI, but human judgment at both ends of the process, with AI accelerating and structuring the work in between.
When properly designed, human-AI collaboration can improve efficiency, adaptability, reproducibility, and the ability to codify and reuse organizational knowledge.

Implications for Policy or Development

AI should be used to enhance, rather than replace, behavioral science expertise.
Development programs should not treat AI-generated diagnoses as definitive answers. Outputs need to be tested, refined, and interpreted through practitioner and stakeholder knowledge.
Funders should support the time and resources required for rigorous behavioral diagnostics rather than prioritizing speed and low-cost analysis alone.
AI-enabled behavioral tools need stronger mechanisms for incorporating context, evidence quality, and the uneven distribution of research across populations and settings.
Making AI-assisted analytical processes visible and documentable can support learning, reproducibility, and better decision-making.
Development organizations should involve practitioners and communities in problem framing, not only in implementation, so that AI-supported diagnostics remain grounded in the realities of the people they are intended to serve.