The following post is entirely Hadley Beeman's (@hadleybeeman) fault. So much is, actually, when it comes to opendata and linked data - just Google her and you'll find it why. For the US orientated among you, Hadley is essentially our national lead on opendata in the UK, she's an American and she yay's. In public. Unashamedly. And that makes her just awesome because us Brits, frankly, we do not yay enough.
Anyway.
This tweet kicked off a waterfall of thoughts and feelings:
"Soundthing: Ambient music selection: just walking into a pub with your phone can bend the pub's playlist towards your Spotify prefs. #gsxsw"
The tag is also worth investigating - gsxsw was curated by Rewired State (Emma Mulqueeny) and held at the Guardian newspaper offices this weekend - it was a hack day with a bit of a difference. Hack days might be a Brit thing also - if so, Google it, there's some amazing stuff going from idea to actuality here at the moment under these banners.
So, the tweet led me to respond with the idea that not only would the pubs sound system bend to me and my mates Spotified taste in music, but that this would somewhere be recorded for future questioning - that somehow I would land in a new city and know where to find like minded music lovers because I could see on a map layer or something similar all the pubs which linked with my Spotify preferences by more than x percent, where I set the percentage. So, for example, and purely hypothetically (ahem) if I hated 80's music with a passion most people only reserved for politicians, I could avoid pubs which played any 80's music at all. Or, if I was ambivalent to ambient but absolutely loved techno, I could set the filter percentages accordingly and go and find my tribe. Or if I was feeling kinda brave and adventurous, I could change the settings randomly with a button and land somewhere entirely new and unlikely.
But, I thought, what if you took that further? What if you took that out of a pub environment, that linked data which told me where I should be depending on my musical preferences, and as Hadley suggested with food, you applied it to something else?
I'm going to segue a little and tell you a story, based purely on an experience I had yesterday. It's true and it's relevant, so bare with me a little if this comes across as self indulgent.
I've got quite severe tendonitis which flares up every now and again. It means my tendon contracts which then pulls on the muscle above it which dislikes it, which then has a knock on effect upwards again. Riding my bike fixes it, I've not ridden my bike for a bit thanks to snow and torrential rain. You try mountain biking in snow on tech Northern trails sometime. Anyway.
We went out for the day and parked up in Manchester, in a car park we don't normally use. The car park is essentially one of those 'this used to have something built on it (in this case Boddingtons Brewery), it now doesn't, someone decided to make money off the land by parking cars on it'. My other half led us towards a sneaky exit he knew which wasn't the official exit. Just by the exit out onto the street was a short but incredibly steep upward incline. My calf muscles combined have lost about 1/2 inch of flexibility. Lots of unladylike swearing ensued. For the next 30 minutes walking was faintly hideous. It eventually stopped when I relented and dived into a handy second hand bookshop to let them calm down a little and stop contracting.
It could have been avoided, those 30 minutes of pain, and I'll tell you how. Augmented linked data. Somewhere else out there, I have no doubt, is someone else with slightly broken calf muscles who's probably done exactly the same thing and was discomforted enough to want to ensure someone else didn't suffer the same ouchiness. Currently, there is no way for her to do that. Currently, there is no way for me to do that. I can't tell anyone. Putting it on a forum would be pointless, it would get lost in the noise. Putting it an email to someone would get me nowhere. I now know something about the environment in a car park in Manchester would could save someone else some pain and I can't share.
In each of our heads is data. We know things. Some of these things are small, so infinitesimally small that we might never think of sharing them or that they might be of worth to someone else, and some of those things are very big things. Some of those big things are only big to us and some of those big things would make the world a better place if we could share them with everyone.
Imagine, then, a way of sharing, if you wanted to, everything you ever knew about obstacles or opportunities. A world where on your GPS there was a layer you could switch on or off which linked in to the local councils database in the area you were passing through and told you where the major potholes were or into the local police database so you could know as a motorcyclist where the accident blackspots and blind corners were. Or, even, which gate you needed to slow down for as you passed it because the cows always came across the road for milking at 4:45pm and as a result the road would be slippy and possibly treacherous for the next few hours after.
Imagine a world where local knowledge was at your fingertips. Don't park down that side street because anyone who does gets broken into - or if you must remove the GPS suction circle from your windscreen because the only people who get broken into are the people who forget. Imagine a world where you knew which restaurants and pubs were genuinely child friendly because other parents had indicated it so and you could see this in real time when driving down a street without having to get your smartphone out of your pocket, because your GPS told you? No more relying on the restaurant owners claims to be child friendly when his definition includes a high chair but nothing else.
But most of all, to me, imagine a world where you could see issues before they arose. Where people with sensory overload problems could reference to other people with similar issues without ever needing to know their name, but only by linking with someone just because they have a similar intolerance, and following where they've been and had problems, but more importantly where they've been and not had any problems at all.
A world where I'd never have walked up the slope because someone with the same issue as me had a way of indicating somehow that people with x issue should use the gate on the flat at the other end, should walk the long way around, should not even think about attempting if in a wheelchair.
In my world, in my future, I believe these things will happen. I believe everyone will see through the same eyes I do - through eyes which see data layered across and streamed through reality in real time. One where my world is shaped by others experiences who are like me. One where there are no potholes to break your suspension and no steep inclines to break your muscles.
Don't tell me tech can't change the world for the better. Don't tell me maps don't matter and visualisation is pointless. Don't tell me these are pipe dreams. If no one dreams, reality will never be changed. Yes, thinking small allows people to JFDI. But sometimes, just sometimes, someone has got to think BIG.
Showing posts with label opendata. Show all posts
Showing posts with label opendata. Show all posts
Sunday, 13 February 2011
Wednesday, 29 December 2010
Data tools - reviews
Data visualisation & mapping tools are something I've been curious about for a long time but never quite had a few hours to devote to tracking them all down and playing with them. Well, someone else has tracked them all down, so I thought I'd spend a fun few hours (my definition of fun is not your definition of fun, I appreciate that), playing with them, reviewing them and putting the results here, if only so in 3 months time when I need a tool, I can come back to this and hopefully know which one to use without getting horribly confused.
Chartle
Purports to offer simplicity, uniquity and interactivity. I've got some arguments with this. You can't make claims like that and not expect me to be harsh. The welcome splash screen is an abomination, but we're not judging on design here....no wait, a website splash screen for a website helping to me design lovely charts and graphs shouldn't make my eyes bleed. -1 point. Also on the front screen is the warning of things to come - a) it's in Beta and b) improved import from Excel. So another -1 point because already I've got a sneaking suspicion the word simplicity is nothing but marketing speak. Click on Create your own and another window spawns. Click your type of graphic - pie, bar, line etc.
I'm not going to do a walkthrough for every tool, but things I noticed in producing the graphic below: Import means pointing something at a spreadsheet on my desktop and hitting Import, not copying & pasting, it's easy to forget to click the Import button, you have to hand type the Title which in this day & age seems terribly retro, on importing the data, the axis retain the descriptors from the example - you're not asked to change them. To change them, you click on Special. I don't class being able to name my axis from your default assignation Special. I call it Mandatory. And confusing as I distinctly remember clicking 'first row are headers' in the Import tab. Each key tap dynamically updates so people who hate Googles live searching are going to detest this, as are those of us on netbooks.
The biggest thing this falls down on, however, is that I can't specify or change any colours on the actual chart. My whole chart is blue. I might not like blue. I might want to have each bar in different colours, or group bars visually for some reason. I can't.
So there you have it - it took very little time to do but there's some serious tweaking to be done on this site before I'll go back and use it in anger. Once it's launched, however, I will be going back and reviewing again.
ColorBrewer
Load it up. Go on. What do you see? A map of something which looks vaguely like the US down to County level and a world of pain. Click on the How to use page and all becomes clear - it's colour diagnostics for your infographics to ensure the colour differentiation is great enough between colours sat next to each other that the infographic is of value and not a mass of 'oh my eyes'. Genius in other words. It's got print friendly and colour blind friendly options, you can overlay roads and you can pull all the suggested colour schemes out in RGB, CMYK or Hex. You can also Export your colours to Excel.
This is a simple tool, and quite a specific one, but it's done well with a good selection of options but without overcomplicating things. Will definitly be using it.
Dundas Data Visualisation
Paid for so not reviewed as I'm a casual user at this stage. Someone else I'm sure will review it - flag below if you come across one. Included here only for completeness.
Exhibit
Confusion reigned because the link from the Digital Research Tools Wiki goes to a landing page and not the tool itself. Once you've followed the yellow brick road, you get to Exhibit. Unfortunately, you get to a page showing you what Exhibit can do but absolutely no clue on how to begin creating such lovely looking things. And since I suspect that working it out is going to take days and not hours, it's not something for the casual user so there the review ends.
Flare
I've deliberately linked to the tutorial page, within which it explains a working flash development environment. Not suitable for a casual user again, going to take more than a few hours to work it out. Again.
Geocommons
Incredibly interesting looking web based GIS tool by the looks of it - however I tried to search for layers to add to my map and found difficulties finding anything UK based as there doesn't seem to bea filter, so tried exiting the add layer dialogue which promptly caused an adobe shockwave plugin crash. Which is where my patience ended, I'm afraid, as I suspect a lot of others would.
This is the point where I started to lose the will to live so I scanned down the list and picked a tool at random - the Internet Community Text Analyser. It beautifully sums up a number of issues which I will summarise at the bottom of this post.
The ICTA is a good summary of my morning so far. I wanted to analyse my Twitter stream on Loulouk, including @replies. The whole lot. ICTA says it can deal with RSS feeds. So I tried initially using Twitter's raw offering of rss and validated it in Google Reader. Because of the introduction of oath, it stopped displaying anything after November 2009. Okay, fine. So I do a bit of investigating and find Free My Feed. I enter the RSS from Twitter, my username, Loulouk, and my password. For the first 4 attempts it said no Feed found. In writing this, it's generated a freed feed which will not require oath. So I've gone to ICTA and entered that web address that Free my Feed gave me, entered a name for it and hit Import and Save. It parses - no error. Wonderful. Except it says 20 records. Yep, it's trying to tell me I've tweeted 20 times from my account. I've also got a webpage full of white space, with a few lines of text at the top but we'll ignore that for the moment. So on clicking Next, what do I find after a page where I'm supposed to remove something but I'm not quite sure what?
Nothing. Zeroes everywhere. No explanation that I should wait or click on the Analyze button then wait (because it appears as I should as after a few seconds things start spinning and loading, though I'm not entirely sure what). On loading, it tells me # of names found is 1. Which is impossible because on the previous screen, inside those 20 messages it clearly showed more than one name. So what does it mean when it says name? Click on the ? and you get:
Yes, apparently there is a Boris Bike station in the middle of the Atlantic. Handy. Do you know why it's there? Because the data which TLA give out on the docking station locations includes co-ordinates and the co-ordinates are in Eastings and Northings which means they are a type of co-ordinate projection called Universal Transverse Mercator. There are, quite literally, hundreds of different projection systems. The ones you will be familiar with are porobably the one which gives co-ordinates in degrees, minutes and seconds, which is referred to as Latitude and Longitude and finally the Ordnance Survey term of reference - the British National Grid which uses a sequence of letters and numbers to enable you to pinpoint your exact position.
I don't know what projection, what type of system, Open Heat Maps is using. There is no option in the process offered to change the projection. And this is why there are dots in places on the map which frankly would incur such a monumental penalty in being over the return time limit that it doesn't bear thinking about.
So what I have learned today?
A) A projection standard is going to become an issue far quicker than I had thought as more and more people without geography degrees (me) or GIS degrees (me) try and play around with data to make it mean something, say something, represent something.
B) Beta means beta means beta. Don't expect it to work. It's a nice surprise then when it actually does.
C) People compose lists on a regular basis which point to tools which they clearly have not tested in any way shape or form. Once you know this and accept this, frustration disappears.
D) Data standards are also going to be a massive issue outside of co-ordinate data. But the issues shown above in co-ordinate data are being replicated across swathes of the UK and US as people rush to publish their data without first questioning the validity of it, the usefulness of it, the integrity of it or how someone will take it and compare it to anything else without metadata standards which are adhered to.
Now if you'll excuse me, I'm off to try and retrieve some of my enthusiasm for examining, visualising, interrogating and spatially mapping data. It's waned somewhat.
Chartle
Purports to offer simplicity, uniquity and interactivity. I've got some arguments with this. You can't make claims like that and not expect me to be harsh. The welcome splash screen is an abomination, but we're not judging on design here....no wait, a website splash screen for a website helping to me design lovely charts and graphs shouldn't make my eyes bleed. -1 point. Also on the front screen is the warning of things to come - a) it's in Beta and b) improved import from Excel. So another -1 point because already I've got a sneaking suspicion the word simplicity is nothing but marketing speak. Click on Create your own and another window spawns. Click your type of graphic - pie, bar, line etc.
I'm not going to do a walkthrough for every tool, but things I noticed in producing the graphic below: Import means pointing something at a spreadsheet on my desktop and hitting Import, not copying & pasting, it's easy to forget to click the Import button, you have to hand type the Title which in this day & age seems terribly retro, on importing the data, the axis retain the descriptors from the example - you're not asked to change them. To change them, you click on Special. I don't class being able to name my axis from your default assignation Special. I call it Mandatory. And confusing as I distinctly remember clicking 'first row are headers' in the Import tab. Each key tap dynamically updates so people who hate Googles live searching are going to detest this, as are those of us on netbooks.
The biggest thing this falls down on, however, is that I can't specify or change any colours on the actual chart. My whole chart is blue. I might not like blue. I might want to have each bar in different colours, or group bars visually for some reason. I can't.
So there you have it - it took very little time to do but there's some serious tweaking to be done on this site before I'll go back and use it in anger. Once it's launched, however, I will be going back and reviewing again.
ColorBrewer
Load it up. Go on. What do you see? A map of something which looks vaguely like the US down to County level and a world of pain. Click on the How to use page and all becomes clear - it's colour diagnostics for your infographics to ensure the colour differentiation is great enough between colours sat next to each other that the infographic is of value and not a mass of 'oh my eyes'. Genius in other words. It's got print friendly and colour blind friendly options, you can overlay roads and you can pull all the suggested colour schemes out in RGB, CMYK or Hex. You can also Export your colours to Excel.
This is a simple tool, and quite a specific one, but it's done well with a good selection of options but without overcomplicating things. Will definitly be using it.
Dundas Data Visualisation
Paid for so not reviewed as I'm a casual user at this stage. Someone else I'm sure will review it - flag below if you come across one. Included here only for completeness.
Exhibit
Confusion reigned because the link from the Digital Research Tools Wiki goes to a landing page and not the tool itself. Once you've followed the yellow brick road, you get to Exhibit. Unfortunately, you get to a page showing you what Exhibit can do but absolutely no clue on how to begin creating such lovely looking things. And since I suspect that working it out is going to take days and not hours, it's not something for the casual user so there the review ends.
Flare
I've deliberately linked to the tutorial page, within which it explains a working flash development environment. Not suitable for a casual user again, going to take more than a few hours to work it out. Again.
Geocommons
Incredibly interesting looking web based GIS tool by the looks of it - however I tried to search for layers to add to my map and found difficulties finding anything UK based as there doesn't seem to bea filter, so tried exiting the add layer dialogue which promptly caused an adobe shockwave plugin crash. Which is where my patience ended, I'm afraid, as I suspect a lot of others would.
This is the point where I started to lose the will to live so I scanned down the list and picked a tool at random - the Internet Community Text Analyser. It beautifully sums up a number of issues which I will summarise at the bottom of this post.
The ICTA is a good summary of my morning so far. I wanted to analyse my Twitter stream on Loulouk, including @replies. The whole lot. ICTA says it can deal with RSS feeds. So I tried initially using Twitter's raw offering of rss and validated it in Google Reader. Because of the introduction of oath, it stopped displaying anything after November 2009. Okay, fine. So I do a bit of investigating and find Free My Feed. I enter the RSS from Twitter, my username, Loulouk, and my password. For the first 4 attempts it said no Feed found. In writing this, it's generated a freed feed which will not require oath. So I've gone to ICTA and entered that web address that Free my Feed gave me, entered a name for it and hit Import and Save. It parses - no error. Wonderful. Except it says 20 records. Yep, it's trying to tell me I've tweeted 20 times from my account. I've also got a webpage full of white space, with a few lines of text at the top but we'll ignore that for the moment. So on clicking Next, what do I find after a page where I'm supposed to remove something but I'm not quite sure what?
Nothing. Zeroes everywhere. No explanation that I should wait or click on the Analyze button then wait (because it appears as I should as after a few seconds things start spinning and loading, though I'm not entirely sure what). On loading, it tells me # of names found is 1. Which is impossible because on the previous screen, inside those 20 messages it clearly showed more than one name. So what does it mean when it says name? Click on the ? and you get:
And this, this is why I am coming to detest arbitrary lists of useful tools. Half of them don't work. Realy desperately do not work. Of the half which do and are out of Beta mode, 90% of them require you to be a 'developer' or have a damn good understanding of GIS - Open Heat Maps was another tool I had a quick play with - I had the csv on my desktop of the location of all the Boris Bike stations lying around so I tried to use that. This is what I got:
This is how many unique personal names that ICTA found in this dataset.
By clicking on this number, you can review all names found by ICTA, add or delete names as necessary.
Yes, apparently there is a Boris Bike station in the middle of the Atlantic. Handy. Do you know why it's there? Because the data which TLA give out on the docking station locations includes co-ordinates and the co-ordinates are in Eastings and Northings which means they are a type of co-ordinate projection called Universal Transverse Mercator. There are, quite literally, hundreds of different projection systems. The ones you will be familiar with are porobably the one which gives co-ordinates in degrees, minutes and seconds, which is referred to as Latitude and Longitude and finally the Ordnance Survey term of reference - the British National Grid which uses a sequence of letters and numbers to enable you to pinpoint your exact position.
I don't know what projection, what type of system, Open Heat Maps is using. There is no option in the process offered to change the projection. And this is why there are dots in places on the map which frankly would incur such a monumental penalty in being over the return time limit that it doesn't bear thinking about.
So what I have learned today?
A) A projection standard is going to become an issue far quicker than I had thought as more and more people without geography degrees (me) or GIS degrees (me) try and play around with data to make it mean something, say something, represent something.
B) Beta means beta means beta. Don't expect it to work. It's a nice surprise then when it actually does.
C) People compose lists on a regular basis which point to tools which they clearly have not tested in any way shape or form. Once you know this and accept this, frustration disappears.
D) Data standards are also going to be a massive issue outside of co-ordinate data. But the issues shown above in co-ordinate data are being replicated across swathes of the UK and US as people rush to publish their data without first questioning the validity of it, the usefulness of it, the integrity of it or how someone will take it and compare it to anything else without metadata standards which are adhered to.
Now if you'll excuse me, I'm off to try and retrieve some of my enthusiasm for examining, visualising, interrogating and spatially mapping data. It's waned somewhat.
Wednesday, 3 November 2010
GIS is to #opendata as.......
GIS (Geographic Information System) is to open data as Microsoft Word is to a bunch of incohesive words and letters. Or at least, when you've been working with GIS for a bit, that's how it looks. A slightly distorted view of course, because GIS wasn't involved in the 15 minute dalliance with some data I danced the other night, but nevertheless, data and GIS is intrinsically linked for me and so I thought I'd try and explain why - with a diagram!
Look ma, no hands! Now, as regular readers will know, I'm not very good at diagrams. Or data visualisations or mash ups or whatever the cool and funky kids are calling them these days - I'm not a cool and funky kid either so I wouldn't know. But this is a pitiful attempt at an explanation, nevertheless because letters are boring and pictures are shiny.
Open data is the end of the story as I hope I've made clear here. Operations is the start. Operations is what generates all the data to go out into the open. Operations means childrens services, it means street scene or refuse collection and street cleansing with some park maintenance in there as well depending on what you're calling it this year, it's the actual day to day stuff like how many fly tips we collected or how many parking tickets we issued or how many library books we lent. It's our bread and butter, it's what we do, it paints a picture in numbers of the service we provide to everyone on a day to day basis.
The data from all that operational activity currently goes in one direction but eventually, one day, will go in two as shown here. At the moment, it goes into management information and in a lot of cases it gets fed, if appropriate and in most cases it is, into the GIS server and processed by our GIS software to make easy to understand visual representations of the data which our Managers can use to make informed management decisions quickly and easily, because the data is not 64,000 rows of ascii (raw letters), but instead thematic mapping, showing them where their hotspots and notspots are, where they need to focus more resource and where there was a problem 12 months ago but now isn't and so they can move resource. It's done more often than 12 monthly, but for the purpose of this, we'll call it 12 months.
The appropriateness of the information going into GIS software is generally whether it's spatial. Spatial means, relational, means does it have a latitude or longitude on it, does it have GPS data attached to it, will mapping it spatially make sense for the data to mean something. In a lot of cases, well actually, in most cases it does, from school catchment areas crossed with deprivation indices crossed with academic achievement levels to a thousand and one other 'mash ups' as they're now called.
Research and intelligence and policy actually have a two way relationship with GIS. They pull data out of the datasets parked there by ops and they crunch it, create 'mash ups' and provide it to Directors and Heads of Service to inform them. They do trend analysis and many other complicated and funky things. Policy take this crunched data too and they build our strategic advice on it. They tell people in words of one syllable (and yes sometimes more) where we were, where we are and where we will be if things continue as they are, but also where we will be if they do not. They don't hold crystal balls, they assure me, but I'm not so sure. GIS and much other non-spatial data is their bread and butter, I think (someone will correct me if I am wrong here, I'm sure).
The datasets which drive all this, will one day go straight onto data.gov.uk once signed off too. INSPIRE says they will. Please read the link, if you've got this far, you need to know about the existence of this Directive.
Which brings me to the other side of open data and the things in data.gov.uk which will sit next to the Operations generated stuff. The cost of satisying Freedom of Information requests was quoted at me by someone at a conference recently from their Authority and I will not post here what it was but it was enough to make my jaw drop. If we are transparent where it comes to information, if we are open, the assumption is that we will actually never receive FOI's ever again, because it will all be easily found, therefore cutting out the middlemen of the poor Administrators and Officers tasked with fulfilling these requests and instead leaving them to do their main job roles. But for the moment, perhaps it would be a nice interim policy for people to put the results of FOI's onto the web automatically, in the assumption that if one person wants to request the information, then perhaps a second will too? It's obviously of interest to someone, right?
And then there's spending data. Generated by Operations but kept by Finance. This is left to last, because Government have almost wrapped this one up. All spend over £500 will be published by local government in January 2011. All NHS PCT spend over £25,000 is already online.
Here ends the tour of data within local government. If you got this far, you know as much as I do, almost. Which in the interests of openness, is exactly the way I believe it should be.
Look ma, no hands! Now, as regular readers will know, I'm not very good at diagrams. Or data visualisations or mash ups or whatever the cool and funky kids are calling them these days - I'm not a cool and funky kid either so I wouldn't know. But this is a pitiful attempt at an explanation, nevertheless because letters are boring and pictures are shiny.
Open data is the end of the story as I hope I've made clear here. Operations is the start. Operations is what generates all the data to go out into the open. Operations means childrens services, it means street scene or refuse collection and street cleansing with some park maintenance in there as well depending on what you're calling it this year, it's the actual day to day stuff like how many fly tips we collected or how many parking tickets we issued or how many library books we lent. It's our bread and butter, it's what we do, it paints a picture in numbers of the service we provide to everyone on a day to day basis.
The data from all that operational activity currently goes in one direction but eventually, one day, will go in two as shown here. At the moment, it goes into management information and in a lot of cases it gets fed, if appropriate and in most cases it is, into the GIS server and processed by our GIS software to make easy to understand visual representations of the data which our Managers can use to make informed management decisions quickly and easily, because the data is not 64,000 rows of ascii (raw letters), but instead thematic mapping, showing them where their hotspots and notspots are, where they need to focus more resource and where there was a problem 12 months ago but now isn't and so they can move resource. It's done more often than 12 monthly, but for the purpose of this, we'll call it 12 months.
The appropriateness of the information going into GIS software is generally whether it's spatial. Spatial means, relational, means does it have a latitude or longitude on it, does it have GPS data attached to it, will mapping it spatially make sense for the data to mean something. In a lot of cases, well actually, in most cases it does, from school catchment areas crossed with deprivation indices crossed with academic achievement levels to a thousand and one other 'mash ups' as they're now called.
Research and intelligence and policy actually have a two way relationship with GIS. They pull data out of the datasets parked there by ops and they crunch it, create 'mash ups' and provide it to Directors and Heads of Service to inform them. They do trend analysis and many other complicated and funky things. Policy take this crunched data too and they build our strategic advice on it. They tell people in words of one syllable (and yes sometimes more) where we were, where we are and where we will be if things continue as they are, but also where we will be if they do not. They don't hold crystal balls, they assure me, but I'm not so sure. GIS and much other non-spatial data is their bread and butter, I think (someone will correct me if I am wrong here, I'm sure).
The datasets which drive all this, will one day go straight onto data.gov.uk once signed off too. INSPIRE says they will. Please read the link, if you've got this far, you need to know about the existence of this Directive.
Which brings me to the other side of open data and the things in data.gov.uk which will sit next to the Operations generated stuff. The cost of satisying Freedom of Information requests was quoted at me by someone at a conference recently from their Authority and I will not post here what it was but it was enough to make my jaw drop. If we are transparent where it comes to information, if we are open, the assumption is that we will actually never receive FOI's ever again, because it will all be easily found, therefore cutting out the middlemen of the poor Administrators and Officers tasked with fulfilling these requests and instead leaving them to do their main job roles. But for the moment, perhaps it would be a nice interim policy for people to put the results of FOI's onto the web automatically, in the assumption that if one person wants to request the information, then perhaps a second will too? It's obviously of interest to someone, right?
And then there's spending data. Generated by Operations but kept by Finance. This is left to last, because Government have almost wrapped this one up. All spend over £500 will be published by local government in January 2011. All NHS PCT spend over £25,000 is already online.
Here ends the tour of data within local government. If you got this far, you know as much as I do, almost. Which in the interests of openness, is exactly the way I believe it should be.
Monday, 1 November 2010
15 minutes of brain
All it takes is 15 minutes.
Crime Stats From Datadotgovdotuk
You might need to download the spreadsheet or view it over at ScribD - I'm on a 10" netpad and it's not so great at displaying things properly.
This is why data.gov.uk is one of the best websites on the internet. This is why open data is a wonderful thing. This is what can be done when youn open up your data. When I've got more time, I'll cross reference this info with population info of each of the areas the forces cover. Then I might look at deprivation indices inside those areas. Or possibly, you know, someone else might who can make the data look prettier because, after all, this was a 15 minutes job. This is not me showing off. This is merely a 'this is what is possible in 15 minutes'.
My father always told me curiosity killed the cat. No it doesn't. It opens up whole new worlds of information and it kills no one to stick it in a .csv and give it to people like me to play with.
- Go to data.gov.uk
- Look at the most recently uploaded dataset
- Happens to be Local police recorded crime data, a breakdown force by force, offence type by offence from 1 March, 2002 until 1 March 2010 or thereabouts.
- Download the .csv
- Open the .csv
- Filter the data by Offence Group (in this case Drug Offences)
- No need to filter the data by Offence Sub-Group in this case, but you can
- C & P out into another Excel tab the Offence data
- Filter the data byYear (in this case 12 months to 1 March, 2003)
- C & P out into another Excel tab the years data (still only showing drug offences)
- Delete 12 months to column
- Delete Region Name column
- Delete Offence Group column
- Delete Offence Sub-Group column
- Highlight the two remaining columns and all data
- Create a graph
- Make the graph look pretty
- Upload it to ScribD
- Embed it in your blog
Crime Stats From Datadotgovdotuk
You might need to download the spreadsheet or view it over at ScribD - I'm on a 10" netpad and it's not so great at displaying things properly.
This is why data.gov.uk is one of the best websites on the internet. This is why open data is a wonderful thing. This is what can be done when youn open up your data. When I've got more time, I'll cross reference this info with population info of each of the areas the forces cover. Then I might look at deprivation indices inside those areas. Or possibly, you know, someone else might who can make the data look prettier because, after all, this was a 15 minutes job. This is not me showing off. This is merely a 'this is what is possible in 15 minutes'.
My father always told me curiosity killed the cat. No it doesn't. It opens up whole new worlds of information and it kills no one to stick it in a .csv and give it to people like me to play with.
Wednesday, 20 October 2010
Waterproof post-it notes
Waterproof post-it notes don't exist. But if they did? Here's what I'd do. No barriers, right?
Imagine crowd sourcing taken to somewhere near the limits. Town planning. How many town planners actually live in the town they plan, I wonder. How many drive around the roads they one way systemise, how many use the ramps and diversions. Imagine if you could crowd source all the people who regularly use the physical infrastructure of the town or city where you live. All the footfall which passes across the pavements, all the people who pass through your local shopping centres shop doors. All the people who look for litter bins and can't find one, all the people who walk the streets and look for benches and can't find them. The people who cross roads and have to wait 10 minutes for a break in traffic, the people who walk along the darkened path and wonder why there are no streetlights.
Ask them. Open up your streets and ask them. Invite the curious, the residents, the armchair intelligent, the planning geeks and the planning bods from other Boroughs. Ask them to tell you where they'd like the bins to be. Where the pedestrian crossing should be moved to, why that sign is always pointing the wrong way and where the drop in the kerb is in absolutely the wrong place for the electric wheelchairs running over it. If waterproof post-it notes existed, by the end of the day, you should have a multi-coloured environment of suggestions and commentary, positive and negative, telling you exactly how the users of the world you've created feel about it when they try and interact with it.
Because waterproof post-it notes don't exist for the moment - why not use Google maps? It's got a zoom level accurate enough to suggest where a bin or a bench should be moved to. It's got a comment facility.
Just saying.
Ask. I think you might be surprised how many people would answer.
Imagine crowd sourcing taken to somewhere near the limits. Town planning. How many town planners actually live in the town they plan, I wonder. How many drive around the roads they one way systemise, how many use the ramps and diversions. Imagine if you could crowd source all the people who regularly use the physical infrastructure of the town or city where you live. All the footfall which passes across the pavements, all the people who pass through your local shopping centres shop doors. All the people who look for litter bins and can't find one, all the people who walk the streets and look for benches and can't find them. The people who cross roads and have to wait 10 minutes for a break in traffic, the people who walk along the darkened path and wonder why there are no streetlights.
Ask them. Open up your streets and ask them. Invite the curious, the residents, the armchair intelligent, the planning geeks and the planning bods from other Boroughs. Ask them to tell you where they'd like the bins to be. Where the pedestrian crossing should be moved to, why that sign is always pointing the wrong way and where the drop in the kerb is in absolutely the wrong place for the electric wheelchairs running over it. If waterproof post-it notes existed, by the end of the day, you should have a multi-coloured environment of suggestions and commentary, positive and negative, telling you exactly how the users of the world you've created feel about it when they try and interact with it.
Because waterproof post-it notes don't exist for the moment - why not use Google maps? It's got a zoom level accurate enough to suggest where a bin or a bench should be moved to. It's got a comment facility.
Just saying.
Ask. I think you might be surprised how many people would answer.
Subscribe to:
Posts (Atom)