ARTSblog

Jim Henson and his fellow Muppeteers perform as Bert & Ernie.

In my first post, I explained why we mustn’t let our fears prevent us from experimenting with evaluations that are neither purely quantitative nor based on random samples when trying to understand whether our social interventions are working.

Now, I want to share examples from visionaries who practice what I preach. The people you should know are Jim Henson, VI Hart, David McCandless, and Jonathon Harris.

Evaluations are a search for the truth about whether our work has changed lives. And I believe good evaluations are conceptually simple.

When Jim Henson was developing the design for Sesame Street, child psychologists and early education experts insisted that they knew the right way to do the show. Jim only cared about measuring whether each segment of video was good, and that kids were watching. You can’t teach kids anything if they aren’t paying attention, he argued.

His greatest innovation was a simple method to quantify whether kids were paying attention at each point during a test episode. They brought some five-year-olds into a play room and showed them visuals on two televisions simultaneously. One played an episode of Sesame Street, while the other played a random series of interesting images for seven seconds each (“the distractor“). Read the rest of this entry »

Mark Rodriguez

Upon reviewing a blog entry about The Arts and Achievement in At-Risk Youth study released by the National Endowment for the Arts (NEA) earlier this month, I ran across a respondent who stated, “It’s great to have all of these studies, but how does it help me and my organization? How can small or midsized arts organizations measure their impact without the resources of large institutions like the NEA?”

The following shares the story of how Changing Worlds, a midsized Chicago-based educational arts nonprofit went from basic surveys and pre- and post-residency exercises to a longitudinal study that improved our practice, reaffirmed the quality of our program, and helped build an organizational culture of inquiry.

In 2003, I became the executive director of a small start-up nonprofit that had little to no infrastructure in place to assess its programs. We had lots of informal data and some feedback from program partners. I knew immediately that if we were going to grow, thrive and succeed, we had to identify our unique niche, solidify our program model and select program inquiry questions we wanted to explore.

From 2003–2008, we went through various renditions of evaluation tools and we even contracted with three independent evaluation consultants. After five years, we learned some new things, developed the basic capacity to measure the impact of our residency programs and invested lots of time. While this helped us gain insight into our short-term impact, it didn’t address the potential long-term impact and implications of our program. Read the rest of this entry »

Mary-Helen Rossi

Anyone with their eyes open today can’t help but wonder if those “gloom and doomers” might at least be partly right — should we be worried for our organizations’ survival?

And if so, with many arts organizations closing their doors, what we do to keep ours open?

CREATIVE PRAGMATISM

For decades now, arts programs have gotten funded based on their case studies (we all have terrific stories, don’t we?) and assertions as to the benefits of the arts. And why not? Those benefits are real, and incredibly valuable. But case studies and avowals aren’t exactly tangible and they just aren’t cutting it any more.

TIME FOR A CHANGE?

Let’s face it — human beings do not like to change, but I’m not willing to bet I’ll be okay if I don’t, are you?

Well then, how can we change — what’s the direction to head in? Read the rest of this entry »