“I cried after my first big film was released,” she admitted, in hushed tones over the phone. This was no traditional case of post-partum blues. This was an award-winning filmmaker, who had spent years of her life, thousands of her own dollars, and probably a relationship or two to bring her poignant, hybrid, social issue documentary to the light of day (or in her case, a night on PBS following a limited theatrical release). “I wanted to change the world with that film. But even though the critics liked it, and people saw it, I had no proof that anything changed. Silence. I still don’t know if it had any impact at all.”
I had phoned my friend (let’s call her Maya) because I wanted to see how the indie crowd at the Sundance Film Festival would respond to my question: “You know your film is making a difference when…” I first attended Sundance in the late-80s when I was a script reader in Hollywood, and participated during the decade I was at the PBS series POV. But this year in my current role leading the Active Voice Lab for Story & Strategy I did more listening than screening. As a lifelong believer in the power of story — and other creative work – to advance social change, these days I’m spending a lot of time trying to prove it.
I can admit it now: early in my career when I wrote the “Evaluation” part of grant proposals I would wince, concoct a few bullets about the kind of change I wanted to see, and keep my fingers crossed that the funder would be so dazzled by what we were actually able to accomplish programmatically that they wouldn’t feel compelled to go back and reconcile it with our originally-stated objectives. During the first 10 years at Active Voice we commissioned outside firms and social scientists to give us qualitative feedback about our process and our partners’ satisfaction. We weren’t trying to impress funders; we simply couldn’t afford to repeat mistakes. But it wasn’t until Active Voice brought on a full time evaluator that were we able to identify and actually measure the kind of shifts we think films can contribute to.
Maya couldn’t have known if her film had made a difference because there wasn’t a solid strategy in place to connect the story strategically to the movement she was hoping to serve. There were no indicators she could monitor to find out if the film was contributing to specific outcomes; so she couldn’t have made mid-course corrections in distribution, communications, and partnerships that would have led to more “impact.” This is now the core work of Active Voice, designing outcome-oriented strategies and implementing story-fueled, measurable campaigns.
No filmmakers actually cried when I circulated my “making a difference” question at Sundance soon after speaking with Maya, but boy, did I get an earful of responses. Predictably, some filmmakers reject the concept altogether because they draw a sharp line between “artist” and “advocate;” or as one said, “I tell stories; what people take away is up to them” But many talented filmmakers, like Maya, also define themselves as change agents. They are fully dedicated to their craft and want their artistic contributions to be authentic and tangible. They know that their vision and talent can help fuel social transformation in unique ways, and they would go to great lengths to try to make that happen if they had the right kind of map and markers along the way.
But they don’t necessarily think of this as evaluation. Like it or not, evaluation is a crusty kind of term. It has the ring of judgment – which does little for creativity. If you ask a filmmaker about evaluation, they will often grimace, as visions of foundation bureaucracy float into view. Sometimes our language, traditions, and yes, those pesky power dynamics keep us apart. But if you ask a filmmaker, “How do you know if your film is making a difference?” the conversation will likely become purposeful: What can be achieved? Who are we trying to reach? What resources will be needed? Why this story, now? To me, these sound like the questions that all good evaluation starts with. In other words, when it comes to measuring the impact of arts and culture, we may be surprised by how much funders, creatives, advocates, and communities have in common.
Maya’s film may have indeed had impact. Who knows how many dinnertime conversations followed, and how many attitudes shifted? Maybe organizations got their hands on the film and used it to raise money, visibility, and awareness? Perhaps policymakers took a second look at piece of legislation based on having seen her story. From my point of view, the connection between “arts and culture” and engaging people in solving urgent problems is too potent to guess about.