Ethical considerations

There are many pitfalls to relying too heavily on quantitative methods to gather data about participants. It can be easy to make assumptions about a person based on this data and the risks should be assessed.

For example, a postcode may indicate that a person resides in an area of economic deprivation, but this does not present any information about the individual’s economic status.

Quantitative methods of evaluation can tend to focus on negatives (deprivation, lack, deficit). Therefore evaluators should be mindful of the impression this focus on ‘lack’ gives to participants and whether this undermines the work of the project.

It is important to consider informing audiences about the evaluation you are undertaking. Leicester University’s “Encountering the Unexpected” project used consent forms and information sheets to explain what their evaluation was for, how it would be used and how people would be involved in accessible language.

When finding the right approach for gathering your qualitative data it is important to consider the impact it may have on participants and their needs should be central to the process.

“People who have encountered systems of care during their life may understandably have a negative response to data capture” says Jess Turtle of the Museum of Homelessness.

For new volunteers the museum uses a ‘getting to know you’ form which captures access requirements as well as basic information.

The museum takes a relaxed approach with these forms. “Participants may not fill in all details in the first instance,” says Jess. “We often get responses that say things like ‘not now’ or ‘I’ll tell you later’ and we accept this. It’s important to build trust before asking people to open up.”

It is important to think about your
participants when choosing your outcomes as there are ethical issues to
consider when selecting them.

Are your outcomes appropriate and
useful for all participants? The Museum of Homelessness takes its
ethical responsibilities very seriously and says they would be unlikely
to set outputs relating solely to employability, for example, unless
they were specified by participants themselves.

“Our team,
informed by lived experience of homelessness, has set outcomes such as
an increased sense of social connection or a sense of common purpose. We
don’t believe that employability is the only way an individual can be
of value to society and reflecting that in our outcomes sends an
important message” says Jess.

By working with participants to set outcomes at
the start of a project you can avoid these pitfalls and make the work a
two-way process.

It is also important to consider your own needs when undertaking qualitative data capture. Measuring social impact can be an emotional experience for those leading and undertaking the work.

Aneesa Riffat of the National Holocaust Centre and Museum explains: “In the main, the process of interviewing has been challenging. Risk assessments, pre-research, post-research and every other procedure can be put into place but ultimately you are dealing with trauma and traumatised memory and that is coming through the narrative.”

It is important to consider your participants very carefully when selecting an off-the-shelf method for measuring social impact. Not all toolkits are right for all audiences. Be mindful of the impact that a particular method might have on participants – does it undermine what the project is trying to achieve?

It is important to consider the emotional wellbeing of both participants and staff.

Partnering with experienced organisations who are used to dealing with issues your work is exploring is a good idea as you can benefit from their experience. It can also be a good idea to incorporate group reflection sessions or offer individual counselling to participants. These types of sessions, facilitated by a professional psychologist, can be a way to put safety mechanisms in place and are not as expensive as you might imagine.

Links and downloads

Measuring socially engaged practice: a toolkit for museums (pdf)

Measuring socially engaged practice – large print version (word)