I found an equation that fits recidivism data in Utah. I would like to publish it. A colleague has presented a challenge--how do I know this is the only form (or model) that can fit the data? I will show you my solution and ask, if you have the time, can you find an alternative?
I assumed a slightly adjusted decay function. A first group "I" has a risk P of returning to prison each month. A second group "II" have zero risk. The percentage of zero-risk people, Z, is a constant--they stay out of prison. The people at risk is (F-Z)%. (100% is the initial sample size.) F is the percent remaining free at time t (in months). Then the decay rate looks like this:
dF/dt = -P (F-Z)
I allowed that P may vary linearly with time: P =K+Lt, and I excluded the first point (t=0) as an outlier.
A solution is F = (100-Z)ab exp (t+ct^2) +Z
The constants to fit the data: a = 1.0611, b = 0.91615, c = 0.00545, Z = 33.00
This is a good least-squares fit from t = 1 to t = 34 months.
The implications of the model are that:
a. People at risk to recidivate are intractable (as they were managed in the system).
b. A large number of people released to parole, about 33%, have essentially zero risk.
I believe this is a very important insight into the nature of recidivism, but only if the model is the CORRECT model. Is there a different one that fits the data? I am listing my data below.
Month % Free on Parole
0.0, 100.0
1.0, 97.8
2.0, 91.9
3.0, 87.7
4.0, 83.1
5.0, 79.0
6.0, 75.3
7.0, 70.5
8.0, 67.2
9.0, 63.75
10.0, 61.4
11.0, 58.3
12.0, 55.6
13.0, 53.1
14.0, 51.9
15.0, 50.4
16.0, 48.8
17.0, 47.0
18.0, 45.6
19.0, 44.4
20.0, 43.1
21.0, 42.6
22.0, 41.25
23.0, 40.1
24.0, 39.4
25.0, 38.6
26.0, 38.25
27.0, 37.8
28.0, 37.5
29.0, 36.9
30.0, 36.6
31.0, 35.9
32.0, 35.6
33.0, 35.3
34.0, 35.0
I'm not sure what you have solved, so perhaps you might explain why your solution is useful, and what in particular it is describing that could be employed toward some end (perhaps some policy outcome). It is well know that recidivism reaches about 2/3rd after about 3 years. This result is so consistent that it is practically a constant (across time and space).
Hello, Michael,
Yes, it seems that recidivism is high everywhere--the major differences in the numbers seem to arise from different ways of reporting. I think I am bringing attention to three things:
1) The usual way of rating risk of people released from prison is to place them on a scale of varying risk. I think this is wrong. Justice systems believe that all parolees have substantial risk and need supervision. I am concluding from the model that about one-third of people released are no more likely to get into trouble than anyone outside and they should not be supervised--if you know who they are--there's the catch, so far the available assessments don't tell you.
2) Justice systems think that their supervision and programming classes help parolees to succeed. The model shows that, in Utah, the rehabilitation efforts are a complete failure. Inmates that have risk of returning are not getting any help from the system as it is--98% of the at-risk group were back within three years, and the remaining 2% had returned by five years.
3) Research over the last thirty years on measuring or predicting risk has been built on the assumption that risk is a continuous variable with a range from 0 to 1. I am finding that it is discontinuous with two rather discrete values. A surprise is that my model, which is absurdly simple, does so well.
I am hopeful that changing the paradigm will help corrections and rehabilitation systems to be more efficient. At this point, I am looking for reassurance that I am on the right track.
Ernie Rogers
Everything you are pointing out is already known. Your model is an equation for a trend that has occurred, and of course an equations can be fit to a known line/curve. What you are suggesting from your aggregate curve is something about individual behavior. But, what you can't predict, and what the models you are criticizing try and do, is predict the behavior of each individual. Whether or not those individual models are good (a question you raise) at predicting the outcome can be assessed by inspecting the results of those models for individuals, not from an equation that describes the macro-level decay function of a trend. So, the question here that would make you equation relevant might be whether you can fit your equation to individuals.
I see that I probably misused "exp" in the original post of the question. I corrected that. I hope this corrected statement can replace the original.
I found an equation that fits recidivism data in Utah. I would like to publish it. A colleague has presented a challenge--how do I know this is the only form (or model) that can fit the data? I will show you my solution and ask, if you have the time, can you find an alternative? I assumed a slightly adjusted decay function. A first group "I" has a risk P of returning to prison each month. A second group "II" have zero risk. The percentage of zero-risk people, Z, is a constant--they stay out of prison. The people at risk is (F-Z)%. (100% is the initial sample size.) F is the percent remaining free at time t (in months). Then the decay rate looks like this: dF/dt = -P (F-Z)I allowed that P may vary linearly with time: P =K+Lt, and I excluded the first point (t=0) as an outlier.A solution is
F = (100-Z)ab^(t+ct^2) +Z.
The constants to fit the data: a = 1.0611, b = 0.91615, c = 0.00545, Z = 33.00 This is a good least-squares fit from t = 1 to t = 34 months. The implications of the model are that:
a. People at risk to recidivate were intractable (as they were managed in the system).
b. A large number of people released to parole, about 33%, had essentially zero risk.
I believe this is a very important insight into the nature of recidivism, but only if the model is the CORRECT model. Is there a different one that fits the data (listed before) that does not include a zero-risk group?