How do we know if the way we do our work is resonating? Who needs to know that? And what makes that knowledge meaningful and useful?
These questions are helpful for arts organizations when telling their stories of change, impact, and resilience; and they are central to evaluation. Evaluation, however, does not have to be a snooze fest only about metrics, indicators, and reports.
Unfortunately, evaluation has a bad reputation among organizations – especially in the arts. Evaluation often happens as a funding requirement. When the goal of evaluation is solely accountability for dollars spent, organizations carrying out and participating in evaluative work may feel resentful about being subjected to exhaustive surveys and meaningless reporting templates that contribute little to organizational learning.
There is so much more to evaluation than that! Evaluation can also help foster, support, and sustain a culture of learning and growth. At AND implementation (a Vancouver-based evaluation & research consulting firm) we are on a mission to make evaluation relevant, meaningful, and – dare we say – fun. When evaluation isn’t just an exchange of metrics for money, it can help organizations better sustain what is working (and fix what isn’t), craft and communicate their authentic impact story, and cultivate/honour important relationships within their communities. That’s the kind of evaluation we are committed to… and, because of this, we advocate for using arts-based methods in evaluation practice.
What are arts-based methods?
Arts-based methods are ways of generating information by pairing art-making with a facilitated discussion or interview. Here, the art surfaces important information that a traditional data tool, like a survey or interview, can’t. And, when done well, it can help make sharing feedback more meaningful and inclusive. At AND, we use a range of arts across visual, literary, multimedia, and performance formats to collect information about people’s experiences, perceptions, values, beliefs, feelings, hopes, and priorities. Find out more about our approach here.
Why use arts-based methods?
To drive effective decision-making, organizations need different tools to collect different information. For example, surveys are great when organizations want to know a few things from a lot of people – like satisfaction or engagement across a large membership base. But, because surveys rely on short standardized questions, they don’t do a good job of generating rich, nuanced, or in-depth insights into people’s values, beliefs, assumptions, feelings, and messy human experiences. And, with automatic requests for feedback popping up after every shopping experience, customer service encounter, and webinar, surveys have become a social nuisance.
Arts-based methods are helpful when organizations want to know about the deeper meaning or experience of their current offerings, get more nuanced feedback on potential offerings, and connect more intimately and intentionally with specific groups. We commonly use drawing, photography, and poetry as part of individual or group interviews to explore complex topics central to organizations’ guiding values and strategic plans. For example, we have facilitated art-making activities to understand what it takes to belong and feel included, access safe and meaningful services, and define what a buzzword means for a particular organization.
By making art and reflecting on the process instead of answering direct questions, participants can explore concepts indirectly, metaphorically, through abstract thoughts, and with more authenticity than a sliding scale or tick-box. They are also better able to recall stories, illustrative examples, and rich multi-sensory details about particular experiences. As a result, organizations better understand the impact they have had or what is needed to have their desired impact. And, the communities organizations most want to learn from are better engaged with a meaningful feedback experience, rather than another generic survey.
What’s more, introducing different tools to collect feedback sparks critical thinking about what organizations need to know, from who, how, and why.
When organizations are more intentional about data collection, rather than reliant on habit and convenience, they are more likely to increase data use and decrease data waste.
When & how can organizations use arts-based methods?
Arts-based methods can be used effectively in three ways.
1. In organizational planning
Rather than subject organizational staff to a boring meeting or exhaustive sticky-note brainstorm, arts-based methods can elicit fresh ideas, surface goals and ambitions, and align guiding values or principles. Arts can be effective in planning at both micro (e.g., program or service) and macro (e.g., operational) levels. For example, at AND, we use collage in our annual strategic planning to look back on where we have been and imagine where we’d like to be next. By deconstructing and reconstructing found materials, we are better able to articulate what to pause, start, and change in the upcoming year.Find out more about collage as a method for data collection here.
Practice tip for arts organizations: Planning is often the easiest way to incorporate arts-based methods into organizational learning. We often recommend starting small and with an easy win. For example, you could use an arts-based activity as an ice-breaker to normalize creativity at work. Or, at your next meeting to plan an upcoming program or event, consider using art to articulate the ideal outcome of your offering. Depending on what is familiar or comfortable for your team, you could give them 5 minutes in the meeting to choose a photo from their phone or doodle something on scrap paper. Then, everyone can share 1 reflection back from their piece that explains what they hope happens because of the program or event. By starting with your own team and at the level of a program or event, you can start getting a sense of the kinds of insights arts can unlock, how it feels to participate in art-making, and how much time you need. You may be surprised that arts-based methods do not need to be expensive, time-consuming, or complicated to be effective. Find out more about the surprising low-cost of arts-based methods here.
It seems simple, but staff at arts organizations or arts-focused funding agencies are often starved for creativity at work. A routine invitation to be imaginative (without the toxic positivity or forced fun vibes) can go a long way.
2. To collect feedback
Facilitating an art-making activity and then unpacking the meaning of that art with a facilitated discussion or interview can be a helpful way to collect feedback about your organization. If you are already planning a survey, chances are you might also benefit from more in-depth feedback about what your programs/services mean to different folks. This could be a helpful place to use arts to unpack specific meanings, values, experiences, and perceptions. But, like any data collection technique, it is important to consider the skillset and time required to do this work well. If you aren’t sure how to do this, don’t. Your friends at Mass Culture can help point you in the direction of some resources or experts to consult with first.
Practice tip for arts organizations: Again, starting small and with an easy win can help boost your organizational confidence in using arts to collect data. Instead of using arts for the first time to elicit feedback from members, start with your staff and volunteers. Invite them to use a familiar art-making technique to unpack their experience of running a program or event. Or, you could invite them to unpack a concept that is central to your mission and mandate, like what supporting the arts or a thriving arts sector means to them. This way, you can pilot techniques in-house, with lower risk, and with additional insights from your staff/volunteers about what is needed to comfortably and meaningfully participate in an arts-based feedback activity.
Practice tip for arts funders: We are starting to see more funding organizations include arts-based elements in their reporting requirements, such as art products created by participants to illustrate their feedback and stories written by organizational staff. While we love seeing funders move toward inclusive, respectful, and trust-based reporting models, premature arts-based or creative evaluation requirements can backfire when there are insufficient budgets and plans in place. Here are two things to consider before ditching tick-box evaluation templates. First, ensure your grant recipients have sufficient resources (budget, protected staff time, and support) to engage in more creative data collection and reporting. Creativity should be possible, not punishing. Second, decide how you will use the information gathered (and the art created). This goal can then guide what art form you choose and how the art-making activity is designed. This way, you ensure that the results of arts-based evaluation activities don’t go to waste.
3. To pause and reflect
It takes a lot of hard work to run an arts organization and support a thriving arts sector. So, it can be difficult to take the time to pause, reflect, and think critically about how things are going when so much is up in the air. Arts activities for reflexive practice can be a simple and soothing deep breath. And, it doesn’t need to be a whole thing. Try taking 5 minutes once a month or at key milestones to make art in relation to the prompt – what is working or what is not working. Putting 5 minutes for art into your calendar can help protect a little creative reflection from the daily grind. It also gives you an opportunity to try out different art forms and begin understanding the unique strengths and weaknesses of each before trying more ambitious activities.
Practice tip for anyone curious about arts-based methods: We make art all the time at AND to pause, get unstuck, work through a hard thing, think differently about a problem, or reflect on lessons learned. Taking 5 minutes to doodle, write a Haiku poem, or make an erasure or “black-out” poem from a failed or successful proposal are easy places to get started. There is always a pen and scrap of paper lying around, and it doesn’t take more than that to take a little artistic time out.
Last Thoughts
To us, good evaluation provides you with the data you need to confidently answer the questions that matter to you, your organization, and the communities you service. We know evaluation can be intimidating, boring, and not at the top of the organizational priority list. But it doesn’t have to be a huge chore. We hope this blog post encourages you to think about the role of evaluation in your organization. As you contemplate your path forward, consider how you can use arts-based methods to gather wisdom from the wonderful people who make up your organization, those who are a part of your membership base or community, and yourself. The world – including the evaluation world – needs more art.
Get to know Maya & Jennica
