21st
June 2004
Professor
Ben Heydecker
Centre
for Transport Studies
University
College London
Gower
Street
London
WC1E 6BT
Dear
Professor Heydecker,
Regression
to the mean benefit illusion
I refer
to the recently published document “The national safety camera programme:
Three-year evaluation report”, of which you are apparently lead author.
You
have done it again. You have allowed a large and unspecified “regression
to the mean” benefit illusion to remain incorporated in the report's headline
conclusions.
You
know very well that the magnitude of the benefit illusion could quite possibly
be larger than the entire benefit claimed for speed cameras. But that has
not stopped you from making unsupportable and unscientific claims.
The
report contains the words: “Because of this requirement to identify
the sites as suitable for this particular safety measure, and in particular
that the collision record was not the sole criterion for selection, the
established statistical phenomenon of regression to the mean (also known
as bias by selection) will not apply in full measure.”
But
the rules for speed camera placement contain the words: “There has been
a site survey by a road safety engineer and there are no other obvious,
practical measures to improve road safety along this stretch of road”.
I expect you appreciate that this rule strongly encourages the selection
of random sites, as opposed to genuine black spot sites, and may considerably
exacerbate the already large regression to the mean benefit illusion.
I remind
you of your words on BBC Radio Four in the programme “More or Less” broadcast
on 29th January 2004, discussing last year's report, when the following
exchange occurred:
Interviewer:
“So
it sounds as if you were clearly aware of this problem [regression to the
mean benefit illusion], but in the numbers you produced there's no adjustment
for it is there? So the 36% reduction in accidents – isn't it rather likely
that some of that would have occurred simply as a result of random behaviour
of accidents?”
Heydecker:
“Well,
that is certainly possible, but we did not attempt to make any correction
for that at the sites – there is a published correction that is available,
but in fact it is known to reduce the accuracy of the estimate because
the correction is less accurate than the original figure.”
You
know very well that there are a number of ways that you could have estimated
the magnitude of the regression to the mean error. For example, you could
have used sites that did not receive cameras as a control group.
There
are other peculiarities in the report, but the regression to the mean benefit
illusion is likely to be so large that other errors are barely worthy of
consideration.
You
are responsible for misinforming the public and the government about a
matter of public safety, and it is likely that this misinformation will
result directly in loss of life by misdirecting road safety efforts. You
are also bringing University College of London into disrepute. You must
resign and you must issue a public statement apologising for the error
and withdrawing the report.
Yours
sincerely
Paul
Smith
CC:
Tim Yeo MP, John Thurso MP, Alistair Darling MP, Lord Young of Graffham
(UCL), Professor Malcolm Grant (UCL), Professor Mervyn Stone (UCL), Professor
John Adams (UCL), Jon Moynihan (PA Consulting), Bruce Tindale (PA Consulting),
John Brignall (Numberwatch), Press and
http://www.safespeed.org.uk/heydecker2.html
[this
page] |
