A step-by-step guide to generating evidence

A step-by-step guide to generating evidence

If you have over £10,000 of National Lottery funding from us, for a project that lasts two years or more, we ask you to tell us how your project’s going every year - including what you’re learning.

To help you learn, you might want to do research or evaluation to generate good quality evidence. Read our evidence principles to find out what we mean by good quality evidence.

You can use part of your funding to generate evidence
The time and resources you need to generate evidence should be right for the size of your funded project. What you generate evidence on, how you go about it, and how you use it, is your call. Because it needs to be useful to you.

If we need specific information from your progress updates we'll let you know
But this guidance is to help you think about what evidence you need to help your organisation - not necessarily to report to funders.

If you're not sure how to get started
Follow our step-by-step guide to gathering evidence below.

Now you've got an overview of some evidence principles. It's time to start planning how you'll generate your evidence.

It can be tempting to jump ahead to thinking about how you’ll collect information
But if you answer these three questions first it’ll be much easier to work out how best to go about collecting the information you need.

Why do you need the evidence?

You might need evidence for a range of reasons – including to:

• understand your community better
• put people’s voices at the heart of what you do
• improve what you do and how you do it
• show the difference you make
• inform and influence others.

These are just examples. Give yourself time to think about why you need evidence, what it can help you with and what your priorities are.

Who’s the main audience?

Lots of different people might use the evidence that you generate. But, being clear about who the main audience is will help you plan better. Your audience could be:

• practitioners and volunteers
• people in your community
• your own organisation
• commissioners and funders
• policy makers.

Try to be as as specific as you can with who your audience is
This will help you understand the type of evidence that will be useful to them, when they'll need it, and how they might use it.

We're not your main audience
You might think of us, your funder, as your main audience. But we really want you to generate evidence that's useful to you. So don't try to generate evidence you think we'll like.

We’re really interested to hear about what you’re learning from evidence though – have a look at the guidance we’ve provided for your progress updates to find out more.

What are the key questions to answer?

Once you’re clear why you need the evidence and who it's for, you can work out the high-level questions to ask. These are the overarching questions that will guide your research – not the questions you’d use in a survey or an interview. So don’t give yourself too many.

How to figure out your key questions
Look to address important things you don't have enough information on. One good way to figure out your key questions is to ask your audiences what they want to know. But you might need to manage their expectations about what's possible.

Here’s some example questions:

• What's the health and wellbeing of children under five like in our community compared to other communities?
• What's important to local families in bringing up their children in their neighbourhoods?
• What difference would moving our service into the local community centre make to who we are working with?

There are different ways to generate evidence – it’s all about what fits best for your purpose, audience and questions. If you’d like some expert help with this, check out the organisations listed on our tools and resources page.

Think about what approaches would help you achieve your purpose

This will be very individual to you, but here are a few ideas:

Decide who'll do the work

You can generate evidence yourself, or you can pay someone external to help you. Or a mix of both. This will depend on things like your capacity, experience, affordability, and the type of research you're looking to do.

Whatever you decide, make sure the people doing the work have the skills and experience to do it well.

There are advantages to each approach

Advantages of paying someone else to do the work:

• People might talk more freely to someone independent.
• Someone independent might find it easier to identify areas for improvement.
• It can avoid conflicts of interest.
• They can bring additional specialist skills.
• Some, but not all, audiences look more favourably on externally generated evidence.

Advantages of doing the work yourself:

• You and your staff have in-depth knowledge of how your organisation works.
• You are sensitive to the needs of your project and its users.
• You can use the expertise of your own research/evaluation staff or allow staff to learn new skills.
• It encourages ongoing reflection and learning, which can make positive changes in your organisation.
• When done well, it can promote the culture of continuous improvement, trust and honesty.

If you decide to hire other people to help you, think about how you, your staff, partners, and service users can be closely involved. This will help make evidence useful to you. And can also build your experience and confidence for the future.

Think about the budget and resources you have available

The amount of resource you set aside should be right for the size of your project or organisation. You should also think about how important or useful the evidence will be to your work before you decide how much resource and budget you want to use.

These resources might be helpful in estimating what you’ll need:

• How to budget for self-evaluation – Evaluation Support Scotland
• Costing an evaluation – NCVO Knowhow

You might need time and resources to:

• plan and manage the work
• involve and support the people you work with to take part
• send anyone on training on research methods
• collect information
• analyse information and make sense of findings
• share the findings and act on the things you’ve learned.

Get in touch with your funding officer if you think there’s other support we could provide you with too.

Just like any workplan, your evidence plan should say what you’re going to do, how and when.

When creating your plan, think about:

• What are the specific pieces of information you need to answer your questions?
• How will you collect this information and who will you collect it from?
• How will you analyse and make sense of what you collect?
• When will you collect and analyse it, and who's responsible?
• How will you use and share this information?

The people you work with will have good ideas about when and how to collect information
Your volunteers might have suggestions about the specific bits of information you need too. Or support organisations can advise you on ways to analyse information.

If you’re paying someone else to do the work - you'll need to put it out to tender, look through all of the proposals you get back, and decide which company suits your needs best.

Make sure your evidence will reflect the people intended

If you’re collecting new information, it’s important to know if the people you're getting information from for your evidence work, represent the same group of people you want to find out about. Be clear about who you want to get involved, why, and what different experiences and perspectives they have.

Sometimes, because of the specific questions you want to answer, you might deliberately choose a particular group of participants, rather than one reflecting everyone that you work with.

Be aware of potential bias and work to avoid this

If your information is biased, you won’t be able to draw useful conclusions from your research.

For example, people taking part in your research might tell you what they think you want to hear or behave differently than they would normally. To try and address this, you could think about how you can make them comfortable. And make sure they know how their information will and won’t be used.

You can also bring different sources of information together to help strengthen and deepen your findings – for example bringing together observations from workers with feedback from participants.

Honestly reflecting on the ways your information could be biased, and thinking about how to tackle this, will improve the quality of your evidence. You could also ask people with different perspectives about where they think your ‘blind spots’ might be.

Plan for high quality analysis

Think about how you'll analyse what you've collected to produce findings. You'll want to have people involved with expertise in the kind of analysis you want to do.

Then think about how you'll make sense of the findings. Sometimes we can skew what findings mean, for example by:

• ignoring information because it doesn’t fit with your previous experience or assumptions
• believing that a particular point is more important than others because you agree with it
• saying that a finding applies more widely than you have evidence for.

Involving people with a range of different views to make sense of your findings, and deciding together what they mean, can improve the quality of your evidence.

Plan how you will protect everyone’s rights and interests, and their data.

Choose methods that work for the people involved
If you need to collect data from people, think about what you’re asking for. Don’t ask them for extra information you don’t need.

The same goes for staff collecting the information too. Make sure they have a good understanding about the people taking part, to help avoid risk or harm. 

Make sure people know what they’re signing up for
Always ask people if they want to take part in research and give them enough information to make an informed decision.

An information sheet is a good way to do this. Make this clear and simple so everyone can understand it. You can always use these as subheadings:

• Who we are and why we’re gathering this information.
• What we want you to do and how long it will take.
• We can give support with sensitive questions that could be triggering.
• We might record your responses or take photos.
• This is how we’ll use and share the information you give us.
• You don’t have to take part – it's okay to say no.
• You can change your mind and stop taking part any time.
• You don’t need to answer all of our questions – it's okay to say no.
• Here are your rights under GDPR .

For information on GDPR rights
Have a look at the Information Commissioner’s Office (ICO) advice for charities. This might help you tell your participants about their rights.

It’s helpful to get a written record of someone’s consent
This would just be a simple written consent form. It just means that you’ll have something concrete to look back on.

Some people will be put off by having to fill something in though. If that’s the case think about how else you can record that someone’s given their consent to take part.

Getting consent from children, young people and vulnerable adults
The NSPCC have lots of useful guidance around ethics, safety and avoiding harm when doing research involving children.

Make sure everyone can take part if they want to
Think about all of the reasons your participants might not be able to take part. Maybe they don’t have money for transport or childcare. Maybe they need someone to translate.

If we can solve the problem, we should. Paying for childcare, transport or hiring a translator are all things we can and should do in these circumstances.

Involve people in the planning of your research
If you involve your participants right from the start, they’ll tell you about any barriers during the planning stages. And might even offer some solutions so we can make sure the research is accessible.

Think carefully about how you represent different groups
Make sure they’re represented in a way that they’re comfortable with. And think about how to show the views of groups that are often under-represented.

Do all that you can not to cause any harm
Think about what harm people could experience by taking part. And then figure out how you’ll avoid or minimise this.     

Examples of the harm our research could cause, if we’re not careful
Someone’s expectations might be unrealistically raised after you ask them about their ideas for the future. So you should:

• avoid asking questions about things that can’t be changed
• be clear on what can and can’t be done with the findings.

Or someone might become anxious or distressed if they’re asked to discuss something sensitive. To help avoid this:

• train the people gathering the information
• have the appropriate disclosure checks in place – especially when gathering information from children or vulnerable adults
• provide support to people before and after taking part – for example signposting to relevant support.

This should link to your wider plans to keep people safe. See NCVO Knowhow for support with safeguarding more generally.

Keep people’s responses anonymous and their personal information safe
It doesn’t matter whether or not people mind if you share their data or personal details - you should still keep this secure and confidential. Have a look at the Information Commissioner’s Office (ICO) advice for charities.

Here are some things you can do to keep personal information safe:

• Store personal details or data in a secure location.
• Only give access to people who need to use it.
• Keep participants’ personal details separate from their responses.
• When using and sharing the findings don’t include details that could give away who they are or what they said.
• Only use the research findings for the purpose(s) that you told the participants about when they agreed to take part.

The Social Research Association also have further guidance on research ethics.

While you’re carrying out your plan, it's helpful to think about how you’re generating, using, and sharing evidence. Is there anything you need to change to make the evidence more useful? What evidence do you need next?

Things to keep thinking about as you generate evidence

• Quality of collection and analysis. Does it fit with what you set out in your plan? Do you need to change anything to make sure you are generating quality evidence? Do you need to adjust your resources?
• Keep up with changes in your project. Has anything changed in your project and does this mean you need to shift or adapt your evidence plan?
• Involve people in making sense of the findings. Involving people from your staff and volunteers, to partners and people you work with, can give added insight and help make sure your evidence is acted on.

Things to keep thinking about as you make use of evidence

• Use your evidence to inform your learning – find out more about how to embed learning in your work
• Look back on your original aims. How well is your evidence reaching your audience, and being useful to them?
• Adapt as needed. If your audiences, evidence uses, or timescales change then adapt your plan.
• Think ahead. What evidence do you need next? Have you found out about something you need to understand better?

Share your evidence to help others to use it too

• Communicate well. Is your evidence clear and well communicated for the intended audience? Writing a report about your evidence can make it transparent and credible. But not everyone loves a report. There are other ways to share evidence too - like blogs, films, events, podcasts, exhibitions etc.
• Share as you go – you don’t have to wait until the end of your project.
• Plan for research and data legacy. If, in a few years’ time someone wants to find out about the topic your project is on, how would they search for and access your findings? We have an evidence library on our website and might be able to share your findings there. If you want to share your findings with us, please email knowledge@tnlcommunityfund.org.uk