Jim Henson and his fellow Muppeteers perform as Bert & Ernie.
In my first post, I explained why we mustn’t let our fears prevent us from experimenting with evaluations that are neither purely quantitative nor based on random samples when trying to understand whether our social interventions are working.
Now, I want to share examples from visionaries who practice what I preach. The people you should know are Jim Henson, VI Hart, David McCandless, and Jonathon Harris.
Evaluations are a search for the truth about whether our work has changed lives. And I believe good evaluations are conceptually simple.
When Jim Henson was developing the design for Sesame Street, child psychologists and early education experts insisted that they knew the right way to do the show. Jim only cared about measuring whether each segment of video was good, and that kids were watching. You can’t teach kids anything if they aren’t paying attention, he argued.
His greatest innovation was a simple method to quantify whether kids were paying attention at each point during a test episode. They brought some five-year-olds into a play room and showed them visuals on two televisions simultaneously. One played an episode of Sesame Street, while the other played a random series of interesting images for seven seconds each (“the distractor“). Read the rest of this entry »