Methodology
Robin Sokoloski and Jamie Gamble utilized developmental evaluation to engage both the researchers and participants from arts organizations in a reflective process over 10 months. They designed a series of intermittent exchanges where researchers and arts organizations shared their learnings at various stages. Additionally, Mass Culture provided each arts organization with a learning log (a Jamboard) with prompts to reflect on their use of the framework throughout the study. In 2024, Robin and Jamie convened a final session for participants to discuss the framework’s application and outcomes. This process supported arts organizations in developing their reflection pieces on how they integrated the Arts Impact Framework that they used into their work practices.
Key Findings
- Qualitative evaluation tools were a good fit for the participating organizations. Vicki Stroich of Caravan Farm Theatre shared in her reflection piece, “Unlike other tools that primarily measure quantitative metrics like emissions, this framework addressed qualitative aspects, such as how artists and audiences engage with the work and the resulting mindset or actions. This approach resonated with us because of our deep relationships with the land and community.”
- These frameworks require high-level support to implement. In all cases, there was direct involvement of the researcher who created the framework. This greatly enriched the process. The direct support of the researcher is particularly valuable given the novelty of the frameworks and the emerging practice of integrating them into organizational operations. This also had the benefit of providing an external, “critical-friend” perspective. A self-directed application of the frameworks has not yet been tested. Our hunch is that a self-directed application could be possible when there is high-readiness and high-interest in an organization. This is desirable given the time and resource limitations of having external support.
- The process helped the researchers gain valuable insights on how to refine their arts impact frameworks. For instance, Emma mentioned her intention to revise some of the language in the Living Climate Framework for the Arts that she designed. Additionally, she plans to update or include indicators that more explicitly address social justice.
- The findings from the application of these frameworks were used by an internal audience. The extent to which the frameworks help organizations to communicate about impact with external audiences (such as funders) has not yet been tested.
- It’s one thing to discuss your organization’s commitment to equity, access, and environmental sustainability (the social domain of each framework). However, applying these arts impact frameworks uncovered some challenges, including resistance, fear, and capacity constraints within arts organizations to dive deeply into self-examination and taking action on findings. Rachel Marks referred to this as “finding the edge and exploring ways in which we can push or ease ourselves past it.”
- These frameworks facilitate an ongoing and expansive process, where questions often lead to further inquiries, fostering deeper understanding. For instance, Katie Ingrey from the City of Ottawa remarked that the framework prompted her to consider, “What does it mean to be proactive rather than reactive when serving your community?” Shanice Bernicky, the researcher behind “Spiraling Outwardly Towards Equity,” reflected at the conclusion of the Use-Case Study, saying, “Each wave of social justice considerations, beginning in the mid-1980s with the revision of the Indian Act and the creation of the Multiculturalism Act, builds to a crescendo. I believe we are at a peak now with the focus on equity, diversity, and inclusion. However, waves can crash down just as quickly as they rise. So, how do we support organizations across the sector in sustaining their commitment to intentional equity work over time?”
Conclusion
This process demonstrated the value of the three frameworks being tested. They are each thoughtful and comprehensive in how they support organizations to review critical issues of climate, equity and accessibility. These frameworks point to the value of qualitative impact assessment tools and how they can be catalysts for radical thinking, allowing a deeper exploration of how the arts intersect with these environmental and social domains.
Through this process, participating organizations were empowered to move beyond simply measuring impact, encouraging them to rethink and challenge traditional notions of the arts’ role in shaping society. The frameworks demonstrated new possibilities for understanding and harnessing the power of the arts to drive social change.
Possible Steps Forward
Research is a living thing, and Mass Culture is committed to continuously testing these frameworks alongside their researchers/ authors, aiding in their refinement and connecting them with arts organizations eager to explore their potential. We recognize that the arts—and the many social and environmental domains they intersect—require robust frameworks. That’s why we are dedicated to securing resources and partnering with great people to develop new arts impact frameworks. Ultimately, we are driven by the belief that qualitative measurements are not only credible, but just as “measurable” as quantitative data.
Access in Counterpoint Use Case Study Reflection
Access in Counterpoint Use Case Study
Living Climate-Impact Framework for the Arts Use Case Study Reflection
Living Climate-Impact Framework for the Arts
Spiralling Outwardly for Equity in Public Arts Use Case Study Reflection
Spiralling Outwardly for Equity in Public Arts Use Case Study
