Tuesday 1 February 2011

Events, dear boy, events

This isn't a post on web privacy. It's a post about www.police.co.uk and I'm afraid it isn't pretty.

Leyton on www.police.uk

News that the website was up hit Twitter last night and of course everyone beat a path to the website to have a play. After discovering it didn't recognise my postcode (I'm absolutely 100% sure that's not because no crimes have happened in this postcode, just trust me on this) I finally managed to get it to tell me that 468 crimes had happened in a radius around the centre of Accrington, 33 of which were violent crime. In one month. I was a bit shocked. So I had a look at Blackburn. Similar. Okay, that's reassuring I suppose, similar sized town, similar crime levels, right next to each other. Some perspective gained.

In the midst of this I'd posted on Twitter about the 468. Lots of other people went to have a look including @ruthkennedy who discovered that the end of her back garden, among the potatoes and leeks was a crime hotspot.

I initially wondered, when she said she thought it might be something to do with the station 200m away that they'd projected the co-ordinates wrong and not allowed for North moving. Then I wondered if it was just down to different forces using different GPS units with varying levels of accuracy causing peoples odd discoveries.

This morning it transpires that in an effort to 'anonymise' the data, all offences recorded in a street seem to be arbitrarily allocated a point and referred to in the downloadable data sheets from the site as 'near XXX road' rather than publishing the actual geographical point where the offence happened.

So what we've actually got with www.police.uk is neither one nor the other. Ruth looks like a crime overlord cos of all the crimes happening in her garden and we haven't got exact point data, but we haven't got first part of postcode data either e.g. BB5 crimes or NW1 crimes. Instead, we've got this weird halfway house thing where it's not accurate, but its inaccuracy almost renders it useless because we don't have any idea if every force uses the same parameters when picking these points, we don't know how they pick their points, we don't know what we don't know in terms of whether one house in particular is causing a considerable issue with anti-social behaviour for example, allowing me to go to my local Council and demand they do something about it.

And if you think that's militant, well according to the Home Office, I am supposed to be taking action and doing something with this data. It says so at the bottom of this page. Well actually it doesn't it says I should use the data to have meaningful interaction with my police force. But as I don't know so many things, I can't really because I don't actually know what this data is going to be useful for.

I don't agree, for the record, with publishing point data to the public. I would have, I think, picked arbitrary points not in peoples back gardens, but in the middle of roads. I would have made it quite clear that that's what I had done. Because the most worrying thing of all to me is that picking some random point along a road to allocate all incidents in that road to can mean crossing a postcode line. Because, as we're not in an augmented world quite yet, when I'm standing in a street, I can't see the postcode line from BB5 1 to BB5 2 for example. And so I could accidentally misallocate a bunch of offences to the wrong sub postcode.

But that doesn't matter, does it?

Except, actually, it might, because does anyone know how insurance companies algorithms work or what they're based on? Might it be this very same crime data? It's worked out on postcode, I'm pretty sure of that. And what about house prices? Lose £10,000 of your house because that point is the wrong side of a line? Oh never mind.

So my point (ha ha ha)? Pick either point data, or area data. Map to specific co-ordinates, or map to postcode sub areas. But please be accurate and please think about this and actually, really, build a consultation forum and ask people who know what they're talking about, like Adrian Short, who has written a fantastic blog post on this subject, what pitfalls you're heading for. They'd happily help, and I hate to hit where it hurts, but they'd tell you for free, more than likely. Might chip £50,000 off that £300,000 bill. In fact, if you gathered opendata interested people all into one place, you might find you could get the whole shebang done for £100,000. Worth doing, I'd say.

As for me? I'll be keeping an eye on that 468.

Some other interesting viewpoints on this subject:
Adrian Short
David Higgerson
Mike Rawlins
Paul Bradshaw
Mad Prof
News Lancashire on Glover Court, Preston, being most crime ridden street in England.


  1. Hi Louise,

    Thanks for that post - saves me writing something similar but with less style. I can't see why anyone would choose to do this the way they have. The grouping of crime reports to a particular arbitrary spot rather than an area is just mind-numbing. I am also flabbergasted that the Govt says they can't see any reason why it should affect house prices (not that I have any problem with house prices falling... as I don't own one) and that they don't think people should be more afraid as the result of the data.

    I thought the area I lived in was quite nice, pretty low on crime. Apparently I was wrong - so I checked back to see the place where I used to own a house - and either it has gone downhill rapidly since I sold (obvious, really, when you consider I moved away... ;-)) or a lot of the terrible violent crime and anti-social behaviour which gets reported was amazingly low key (I was aware of 5 incidents in 12 years, and lived next door-but-one to the sort of person who made sure you knew about *anything* which happened in the area). One of the incidents was one of the traffic wardens in the area maliciously ticketing my car when it was parked perfectly legally, so that probably wouldn't even have counted.

    It astounds me that this cost £300,000. I'm pretty confident that given the baseline data I know several people on Twitter who could, and would, have done a better job for the fun of it. To be fair, given it was for the Govt, I hope they would have charged *something*, but still.

    It seems that they have difficulty keeping it up, too. At the moment, the search page shows, but clicking search does nothing. That could be a badly handled roll-out, with an 'unexpected' number of people interested in the data (whodathunkit) or just a lack of capacity planning. Personally, if I had commissioned it, I would be looking for a decent chunk of that £300,000 back as it doesn't work, and reflects badly on the police/Home Office.

  2. The police.uk site is so bad it's criminal!

  3. P@> I am reminded of the launch of the er...was it 2001 census data a few years ago which similarly brought servers to their knees as 'unexpected' traffic hammered the site.

    There is a positive side to this though. Take That tickets going on sale probably didn't result in 75,000 hits per minute. There is hope for our society yet, I think.

    Anon> Very good ;)

  4. I have had a lot of trouble with the Metropolitan police force, Please look at my blog I have put I the paper work I recived from them under the freedom of info act, I reported been raped, they told him one year later they were going to interview him but they were going on holiday fist and could he hang around, sine then I have taken a foto of the same police office 6 times, they have targeted me with non-lethal weapons, they dont answer letter non body dose any thing about these out of controle b---------

  5. There is a way to do "Outer Postcode" analysis, but you've got to download the to data and do it yourself! (Not incredibly easy to the average net user)