ARTSblog

Mark Rodriguez

Upon reviewing a blog entry about The Arts and Achievement in At-Risk Youth study released by the National Endowment for the Arts (NEA) earlier this month, I ran across a respondent who stated, “It’s great to have all of these studies, but how does it help me and my organization? How can small or midsized arts organizations measure their impact without the resources of large institutions like the NEA?”

The following shares the story of how Changing Worlds, a midsized Chicago-based educational arts nonprofit went from basic surveys and pre- and post-residency exercises to a longitudinal study that improved our practice, reaffirmed the quality of our program, and helped build an organizational culture of inquiry.

In 2003, I became the executive director of a small start-up nonprofit that had little to no infrastructure in place to assess its programs. We had lots of informal data and some feedback from program partners. I knew immediately that if we were going to grow, thrive and succeed, we had to identify our unique niche, solidify our program model and select program inquiry questions we wanted to explore.

From 2003–2008, we went through various renditions of evaluation tools and we even contracted with three independent evaluation consultants. After five years, we learned some new things, developed the basic capacity to measure the impact of our residency programs and invested lots of time. While this helped us gain insight into our short-term impact, it didn’t address the potential long-term impact and implications of our program. Read the rest of this entry »

Rebecca Yenawine

I have been a community arts practitioner in Baltimore City for the last 15 years.

After years of being asked by funders how my program evaluates its outcomes and answering with anecdotal stories and satisfaction survey results, I decided to try to find more meaningful ways tell the story of this work so that its potential for impact could be better understood and attract investment and resources. To this end, I began some small research projects.

In 2010, Zoe Reznick Gewanter and I conducted a study of 14 community arts practitioners. Practitioners were interviewed and asked how they define community arts, what their methods are and what outcomes they see as a result of the work. Here’s a video of that work:

After transcribing and coding their interviews, several clusters of outcomes emerged: Read the rest of this entry »

Chris Dwyer

I’ve had the good fortune to live in the same community for the past 27 years and the double good fortune to have participated over time in a wide range of arts-based initiatives in that community:

• site-based work related to immigration
• explorations of the history of a working class neighborhood now gentrified
• rediscovery and recognition of the paved-over African Burying ground in our white New England city
• perspectives on the challenges faced by seniors
• and the most widely known, The Shipyard Project, which was a two-year exploration through dance of the intertwined histories of a large naval military installation and a port city that have co-existed side-by-side for over two hundred plus years.

So, I’ve had an advantage that most project evaluators never experience, that is, a really longitudinal view of unfolding impacts.

Long after initiatives have concluded, I have seen relationships that began in a community arts project rekindle to tackle a new issue or witnessed policies that were the object of arts-informed debate finally take hold after several failed attempts. I have seen young people who were inspired by the arts planning of my generation decide to settle in the community and become the next generation of arts and community leaders.

The real impacts become visible many years after the evaluator has delivered the final report to the funders. Read the rest of this entry »

Marty Pottenger

Art At Work

Recently, I found myself sitting in a circle in Portland, ME, leading a group that includes the city manager, police chief, a leader in the Occupy Maine movement, one of the founders of Portland’s NAACP, leaders from the Sudanese and Congolese refugee communities, the president of a city union (CEBA), and a doctor active in public health, among others. The members of this group are impressive and diverse, but what we are sharing is more so.

In only seven minutes, 20 city and community leaders composed poems that draw upon their personal histories, the history of Portland, and those things they have witnessed in this place we all call home.

Increasing the Odds

All of Art At Work’s projects are designed to increase the odds that Portland and their partner cities (Holyoke, Northampton, and Providence in 2012), launch their own Art At Work will be better able to turn anticipated social and economic crises into opportunities by integrating creative engagement in their ‘way of doing business.’

This workshop was a part of Portland Works, another one of our experiments in figuring out how to harness the transformative power of art to achieve concrete community-based outcomes. These workshops bring together community and city leaders to create a dialogue and increase understanding between individuals and groups that often see one another as obstacles as opposed to allies. “It’s just brilliant,” says Mike Miles, the City of Portland’s director of human resources, “using art to break conceptions about who people are and what people do.”

Art At Work, of which Portland Works is just one part, is designed to improve municipal government through strategic arts projects involving city employees, elected officials, community leaders, and local artists. Read the rest of this entry »

Rachel Engh

Last week, I heard local artist Kinji Akagawa’s joyful chuckle as I stood still, swept up in his world while viewing his public art piece, Enjoyment of Nature, in Minneapolis. And he wasn’t even next to me. Instead, I was listening to a recording done by Akagawa for Sound Point, a collaboration by Minnesota Public Radio (MPR) and the City of Minneapolis.

Sound Point is a technologically innovative way for people in Minneapolis to connect with art, artists, and public space. After the (optional) recorded welcome by Mayor R.T. Rybak, the listener/viewer can experience 13 pieces of art, all within a two-mile radius. In these short recordings, artists explain the significance of the pieces’ spatial contexts and what they hope visitors will experience while viewing their work.

I stood, phone to my ear, for a whole three minutes, as I listened to Akagawa talk about his piece. He communicated his wish of creating a gathering place for people, either waiting for the bus or sitting in the sun sipping coffee, and even birds who can visit the bird bath.

One aspect of Akagawa’s built environment is a moonscape, depicting the moon’s movement over a month-long period. Akagawa notes that it honors the people who clean the city at night, many of whom are people of color and immigrants.

Next, I walked to the City’s Public Service Center where I found another Sound Point, Wing Young Huie’s Lake St., USA. A community photography project, it was originally a set of hundreds of black and white photos that was publicly displayed along six miles of Lake Street in Minneapolis. Now, some of them hang on the walls where the city planners pass every day. In his recording, Huie notes the importance of showing his photos in public spaces because they “reflect realities of so many different people.” Read the rest of this entry »

Marc Maxson

Think about the most fun you’ve had doing charity work. What was it that really appealed to you? Was it the smiling faces of kids playing a sport or painting a mural? Maybe it was the moment you realized someone’s life would be forever changed by the small token of love that a program enabled one person to give another.

Do you know what those moments have in common?

First, they are significant on an emotional, social, or metaphyiscal level—and so no traditional evaluation is well-suited to quantify them.

Second, these moments belong to those whose lives have changed. Your impact, as the person who helped make it happen, should not be the focus (unless you enjoy being self-centered and alone in the world).

So why do we continue to act as if “quantitative” surveys about our own “impact” are smart?

My decade working as a neuroscientist with actual “quantitative” data enables me to confidently dispel this notion once and for all. Here me out:

• Social change is social. That means it depends on people. Lots of them. People lie, especially in surveys, and often with the best intentions. Self-reports from people are not quantitative.
• So that’s why we have statistics, right? Inferential statistics depend on random sampling, and sampling is almost never random given the reasonable time and cost constraints placed on nonprofits.
• Even more alarming, statistics has no really solid way of telling if the sampling was done randomly.
• If random sampling is a problem, then results will not be reproducible over time and in different places. That’s why a lot of high-paid people interpret them and argue over methodology. But I think that’s a distraction from the core problem—which is our obsession with extrapolating from brief and tiny samples of life to broad and timeless descriptions of social change and impact.
• If you want quantitative data about people and social change, it’s probably more practical to transform our evaluation tools into a regular part of daily life—like Facebook or Google—so that we’re constantly looking at tens of thousands of bits of knowledge instead of just a few hundred. Read the rest of this entry »

Rachel Grossman

I developed my deep fondness for assessment over 12 years in theatre education and community programming and I bring that affinity into my work as an artistic leader for dog & pony dc, the administrative leader for Washington Improv Theater, and a “chief experience officer” focused on community building and civic discourse through arts participation.

Why am I fond of measurement?

As a box-checker, it provides a tremendous sense of accomplishment. As a lifelong learner, it allows reflection on choices I make and their effect…in order to make stronger/more interesting or daring choices in the future. As a manager, it supports the creation and execution of successful programming and initiatives.

I grew up as an arts educator early in the assessment and evaluation movement in regional theatre education.

I learned some valuable lessons:

• be realistic (you can only accomplish so much in 45 minutes with 30 third graders);
• plans can be adjusted (and improved) when you know the endgame;
• assessment is linked to impact and change;
• if you can observe it, you can measure it.

It was no surprise when I fell head-over-heels for Theatre Bay Area and Wolf Brown’s Intrinsic Impact study, which reaches beyond measuring success by ticket revenue and surveys that only ask if audience liked/not a show. Read the rest of this entry »

Mark Stern

Susan Seifert and I began the Social Impact of the Arts Project (SIAP) in 1994 in response to the attention that economic impact studies were gaining at the time.

We felt—in addition to their methodological flaws—that these studies captured only a fraction of the importance that the arts held for society. We committed ourselves to think through the theoretical and methodological issues involved in documenting the contribution that arts and cultural engagement have for community life.

Over the years, we’ve discovered many connections between the arts and social well-being, some of them quite surprising.

It turned out that the arts were associated with preserving ethnic and racial diversity in urban neighborhoods, lower rates of social distress, and reduced rates of ethnic and racial harassment. Perhaps most surprisingly, we found that the presence of cultural assets in urban neighborhoods was associated with economic improvements, including declines in poverty.

We used the concept of “natural” cultural districts to study neighborhoods where we found unplanned concentrations of arts organizations, cultural enterprises, artists, and cultural participants and documented that it was the social and civic engagement associated with the arts that seemed to drive these economic benefits and revitalization.

Over the past several years, we’ve been trying to re-conceptualize our findings and their meaning for the cultural community, urban public policy, and scholarship. Read the rest of this entry »

John Bare

Let’s start with two assertions:

• First, every meaningful social change movement for the last 1,000 years, at least, has been driven, in large or small part, by the arts.
• Second, many arts-based civic works contribute little or nothing to individuals, communities, or societies.

It boils down to this: You can’t produce great social change without the arts. But there’s no guarantee that every arts-based program accomplishes something.

As with all interventions, whether arts or education or agriculture, much ends up on life’s cutting-room floor—or, if not tossed, left as a relic. If great art alone would suffice, Woody Guthrie’s Plane Wreck at Los Gatos would have changed the American experience for immigrant farm workers.

Let’s circle back to the first assertion.

• Imagine what would have come of spiritual life in the last 2,000 years without the contribution of literature (pick a version of the Bible and say thanks to Gutenberg).
• Imagine the LGBT movement without the contribution of theater (see Charlotte circa 1996).
• Imagine the Civil Rights movement without Guy Carawan teaching We Shall Overcome to the Student Nonviolent Coordinating Committee at its founding in Raleigh in 1960.

Back in 2008, Guy’s wife, Candi, explained to me that Myles Horton had embedded music in every undertaking at the Highlander Folk School.

“Myles, before he founded Highlander, had been over to visit the Scandinavian folk schools. He had observed in Denmark that when people came together to work on problems, they did a lot of group singing. He kind of brought that idea back to Highlander. He was not a musician himself. But he was really supportive of anything that would help grassroots people feel stronger.” Read the rest of this entry »

Jen Gilomen

We’ve all had the experience of sitting in a dark theater and being moved by a compelling documentary story. And as documentary mediamakers, many of us have felt that power materialize during animated discussions that occur with and among audience members when the lights come up for the Q&A.

But how do we really know if our films are having an impact beyond the walls of the theater, and how do we know that our film is causing something besides “clicks” and “likes” online?

At Bay Area Video Coalition, we’ve come a long way in our understanding of impact evaluation and its purpose. It used to be that evaluation was another box to check off in order to satisfy the requirements of our funders. We collected surveys at the end of each training or program, and when funding allowed, we began to track our program participants and projects over a longer period of time.

Our thinking about the purpose of evaluation began to shift, however, when we received a multi-year grant from the National Science Foundation that included a funded, dedicated evaluator to help us design and implement an evaluation not just for reporting purposes, but to create feedback loops that would shape future programming throughout the program’s lifecycle.

Participating in the design of this evaluation freed us to shift our focus from one of conducting surveys and basic reporting (for others, usually as an afterthought) to one of viewing evaluation as an opportunity to better understand the real and long-term impact of our work—for ourselves, so we could become more effective. Read the rest of this entry »

Jon Pounds

I believe we need to be really careful about what results we claim public art produces. Inevitably, and understandably, we will be asked by someone to produce the evidence to back our claims.

Careless claims can be most difficult task prove and, unproven, confound the good efforts of us all.

My caution is not because I think public art does little; rather that some things we might believe (or hope) we do are difficult to prove.

There are recent examples of assessments of well-known cultural agencies that provided little or no support for the assumptions made about their work. Does that mean that the work is not valuable (or properly valued)…or that the assessment of its value is nearly impossible even when well financed and professionally investigated? Assessing public art is nothing like counting beans.

There are examples of attitudinal assessments that work for some cultural experiences—not so much public art.

If you assess attitudes before and after a theater performance, at the very least you are asking someone to reflect on an experience that is both visual and aural and one that they have invested some significant amount of time (and perhaps money) to experience. Similarly, if someone has gone to a museum, they have invested time (likely at least an hour) and money and have chosen the experience because they anticipate satisfaction of their desire. And, in both cases the producing agency can hope to see an increase in funding from annual memberships as a long-term form of assessment.

Can public art begin to match those conditions for assessment? No. Read the rest of this entry »

Shirley Sneve

When is the point in a project’s life that you can say that was success?

How do you know you’re making a difference—that your programming touches people’s lives and makes them think?

What does having fun and learning at the same time look like?

Native American Public Telecommunications (NAPT) is a national organization based in Lincoln, NE. We work with American Indian and Alaska Native media makers to deliver programming to PBS stations. Major funding comes from the Corporation for Public Broadcasting.

It’s a fine line we walk as we balance how much our organization and individual staff members give back to our local community, when the nation and over 560 federally recognized Tribes make up our “service area.”

We decided to do a local film festival.

With the Mary Reipma Ross Media Arts Center and other Nebraska venues, we brought 37 Native films (both features and documentaries) to the VisionMaker Film Festival last fall as our fourth biennial film festival. The six filmmakers that we brought to Nebraska spoke to high school and colleges groups, in addition to their Q & A session after the screening. Read the rest of this entry »

Kenneth Bailey

First let me say that we are still working to figure out how to evaluate the impact of Design Studio for Social Intervention (ds4si) and our various unusual approaches to creating change.

We would love to think with practitioners, funders, artists, and other allies on this issue and use it as a way to build solidarity, increase the breadth and creativity of evaluation approaches, and further spread any new insights we uncover. (Unfortunately, the current climate is one where evaluation frequently creates distance between practitioners and funders. It would be interesting to explore this as a cultural practice as well!)

One of the ways we want to explore evaluating cultural practice is in gauging an intervention’s resonance within the situation it’s hoping to affect. Resonance might be a way to make a distinction between our social interventions and those artist-led practices that aren’t informed by and in embedded solidarity with a population’s or organization’s desire for a particular change.

We aren’t saying that social practice art projects that aren’t deeply informed as such shouldn’t happen, but that they might be considered differently that those that are.

Let’s take the projects we designed working with youth activists on youth violence. We have designed and tested three different interventions since we started this work with youth over four years ago.

One of these interventions was aimed at disturbing the social practice of “grilling,” which is the term for the glare which takes place when two youth make eye contact and immediately infer danger from each other. To us, we found resonance when youth across the board stated that “you can’t stop the grill.” In fact, the emotional investment and intensity with which youth thought they couldn’t affect this practice was what made us pick it. Read the rest of this entry »

Joanna Chin

Growing interest in capturing impact of many types of programs has resulted in escalating discourse and developing practice-based theory about the social impacts of the arts. This current focus on understanding what difference we make builds on, and goes beyond Robert Putnam’s theory, which connected the power of arts and culture in creating social capital.

Across the board, researchers are exercising leadership in this area. For example:

• Alan Brown, in An Architecture of Value, has drawn out and interpreted key concepts from the RAND Corporation’s Gifts of the Muse report to advance a framework of public value centered in and building from the arts experience.
• Clayton Lord and Alan Brown, working with theater partners across the country, have devised indicators and scales to measure the intrinsic impact of experiencing theater.
• In the media arts, American University’s Center for Social Media has reviewed state of the art methodologies for the strategic design and evaluation of social issue documentary films in its Designing for Impact.
• Mark Stern and Susan Seifert at the Social Impact of the Arts Project (SIAP) have developed cultural clustering as both a methodology and a concept. The method involves integrating data on cultural assets into a geographic information system to produce a Cultural Asset Index that can be used to identify census block groups with the highest density of these assets. SIAP is developing a Creative Assets Mapping Database as a community and economic development tool.
• The Knight Foundation and Artplace are working to create vibrancy measures for communities, while the National Endowment for the Arts is looking for indicators to assess the impact of Our Town and other grant programs.

Animating Democracy, a program of Americans for the Arts, has been working to bring together these strands of thinking in the Impact section of our website; particularly, when artists are intentional and art is integrated with practices of civic engagement and social activism as catalysts, conveners, forums, and forms for change. Read the rest of this entry »