Thursday 19 July 2018
The Forward Trust is committed to assessing the effectiveness of our services through research and evaluation, and therefore will be working closely with the Justice Data Lab (JDL), a unique service from the Ministry of Justice that helps organisations to assess the impact of their work on reducing re-offending.
Our first submission to the JDL relates to our work with prisoners with a history of alcohol addiction, and we are delighted to report that our RAPt Alcohol Dependency Treatment Programme (ADTP) has reported a 1-year re-offending rate of 37%, significantly lower than predicted re-offending rates for alcohol dependent offenders who do not get access to treatment. However, it is unfortunate that we cannot claim ‘statistical validity’ for the ADTP impact on re-offending, due to technical challenges in adequately matching the characteristics of our programme clients with a control group of offenders.
The result is consistent with previous research into our programmes that are provided to prisoners who are dependent on drugs and/or alcohol – a group with higher than average predicted rates of re-offending. Other programmes that have been evaluated include our Substance Dependent Treatment Programme (SDTP) with a proven impact of reducing the re-offending rate of participants from a predicted 49% to 31%. By way of broader comparison, the most recent re-offending statistics released by the MOJ show a 64% re-offending rate for these sentenced to less than a year in prison. In summary, whilst establishing grounds for comparative analysis of the impact of our programmes is ‘work in progress’, it is clear that we are achieving lower than predicted re-offending rates for ‘high risk’ prisoners who are dependent on drugs and/or alcohol.
The findings explained:
How the Justice Data Lab works
The Justice Data Lab (JDL) provides an analysis which assesses the impact of rehabilitation interventions against various offending measures including the one year proven re-offending rate and the frequency of re-offending over one year.
To show whether an intervention has had an impact on re-offending the JDL will match the client group with a comparison group of other offenders who haven’t received the intervention but are matched as closely as possible in terms of level/type of offending, demographics etc.
The challenge of matching like with like
One of the key barriers to identifying whether our intervention has had an impact on re-offending has been the identification of a comparison group that matches the cohort of clients participating in the ADTP. Whilst we acknowledge the efforts that have gone into identifying an accurate comparison group, there have been some limitations to this due to the data currently held by the JDL.
To enter onto the ADTP all participants must be dependent on alcohol; however, currently the JDL is unable to match dependency (acknowledged as a key criminogenic factor) in the comparison group. The nearest match is frequency and level of alcohol use, which we would argue is very different from dependence. With almost 30 years’ experience as a substance misuse provider we believe that alcohol dependency is the key criminogenic factor and therefore any comparison group must also be shown to be alcohol dependent, to allow for a like-for-like comparison.
Previous research (Brunton-Smith & Hopkins, 2013) showed that a cohort attending low intensity drug or alcohol treatment programmes (and therefore likely to have presented with dependency) is associated with a one-year re-offending rate of 58%. Based on this, we are pleased that only 37% of dependent alcohol users engaging with the ADTP went on to re-offend.
In addition to this, the JDL is also currently unable to identify if any offenders in the comparison group have received any other alcohol interventions whilst in custody. It is highly likely that they will have received some form of alcohol intervention. Therefore, the comparison can’t show the difference between an individual who did receive an intervention (the ADTP) and someone who received no intervention at all.
Further work with the JDL
We understand the issues raised above have also been problematic for other substance misuse providers when submitting data to the JDL and as such we have agreed to submit further data to the JDL which may lead to a demonstration study in an attempt to assess more accurately the re-offending rates of drug/alcohol dependent prisoners who do not receive structured treatment.
We are committed to working with the JDL to build comparison groups that are as accurate as possible for the substance misuse field to identify those interventions that genuinely make an impact.
The full Justice Data Lab report can be found here.