You have decided on some prevention projects and you know what it is you want to achieve. Let's say, you want to try to reduce the number of young people who feel they can't cope, or the number of adults who need social care when in theory you think it could be prevented. The big question is how do you know your project is working? Or even likely to work?
In Hertfordshire, we have been working with the council and voluntary sector services on how to demonstrate the effectiveness of prevention, ranging from drug testing on arrest for acquisitive and violent crime, to safety training for motorcyclists and mentoring young people with learning disabilities into employment. Not one project has been simple, but through various means we found they all had value.
The most straightforward way of doing this is to review the evidence and use it as a yardstick against your own practice. Not everything has good and reliable evidence that something works, but there are some things which do, for example, preventing recurring falls in older people. The evidence continues to support assessing multiple risk factors (from gait stability to hazards at home) and multiple points of intervention (removal of hazards, plus strength training, plus eye tests).
Begin by getting your public health team to hone down on what your project is trying to achieve, then assess the evidence of what works. By comparing how what you do sits with the evidence and making any changes to what you do are essential steps.
If there is no good quality published evidence, there are two reactions: first, you have to commission an external evaluation or you can’t prove anything. Wrong. Evaluation is a good discipline to practice, but doesn’t always mean commissioning external evaluations.
Second, what happens if you can’t prove anything? There are things you can do. The answer is not to jump to external evaluation straightaway. Prevention means changing something. Examine the change you are looking for. Define what it is you are trying to prevent. What behaviours are you looking for? What outcomes? For which population? Develop these with that population and take their views on what might work for them.
Then, turn this into your model for change. Identify which information will tell you what baseline you are at, for example, how many carers get mental health problems? Is it all types or do depression and anxiety predominate?
Next, identify exactly what and how you will measure as change – is it in outcome terms, or economic terms, or both?
Finally, run the project, collect and review the data and assess whether it works.
These are some of the steps in two free resources we have produced for Hertfordshire agencies that anyone can access for free. We produced our own toolkit to help local agencies and you can find some of these tools at: https://www.hertshealthevidence.org/evaluation-support/evaluation-support.aspx.
The evaluation toolkit we produced jointly with the Herts Sports Partnership also helps you identify the steps to take to determine what your project is doing for you at: https://sportinherts.org.uk/app/uploads/2018/05/Hertfordshire-Evaluation-Framework.pdf.
The biggest mistakes people make in prevention are not defining clearly enough what they want to achieve and not setting up a project in a way that they can measure whether it is doing what it is supposed to.
You don’t need a randomised controlled trial to determine whether your social project works. You do need a clear approach which is logically consistent and thorough. Your public health team should be able to help you with this.
Jim McManus is director of public health for Hertfordshire CC and vice president of the Association of Directors of Public Health