Mind the (data) gap!

In the beginning, there was just data. It sat there, on remote servers, locked away behind firewalls and closed doors. No one gave it much thought, really, everyone sitting in their little silos, reinventing the wheel every time they needed to write a query or build a visualisation.

Then there was the word. And the word was opendata and it was new and big and shiny. It was bandied and bannered, posted high for all to see. It was used as a light sabre - a weapon to slice through locks, penetrate firewalls, and cast aside the fears of those who curated the hidden. 'Transparency!' was the rallying cry - and many gathered to this banner - developers and programmers, shiny eyed dreamers and central government believers.

Data.gov.uk was born. Linked data and SPARQl were created. Metadata was mooted and the language evolved. Coders coded and corporations released data and all was well with the wor...

Stop.

Hands up who understood all that? No, really, hands up. I'm being serious now. Deadly serious, because this is actually quite a serious subject. Does anyone reading this actually understand anything I just wrote?

I'll freely confess. I was fine until SPARQl and linked data. I understand them as a 'concept' but I don't understand the technicalities of how to make my data, our data, this data linked. I've tried. I've watched YouTube videos, read the page on data.gov.uk but I'm afraid I've found the limit of my ability to understand something and I avoided the linked data session at the weekends unconference for exactly that reason. Everyone else seems to understand and be massively enthusiastic, and I feel stupid. So I'm admitting it.

I'm inspired, entirely, by Toby Blume. I take no credit for this stance. It is his entirely. He thinks I am knowledgeable about open data. And I supposed I am in that I understand the concept and theories behind it and I love data, love reading, it burrowing through it, trying to find the use, the meaning, the stories and the successes hidden deep inside it. But knowledgeable? No. Except from where he is standing, which he freely admits to being right at the beginning, I suppose I am. And from where I am stanidng, Hadley Beeman knows everything there is to know about open data. Emer Coleman. Smart people knowing smart things, and trying to share them and defend them with other smart people.

So, I am in the middle then. I can remember the beginning of knowing nothing and I can look across at another pool of people merrily discussing things I have no understanding of, though would love to. And there's got to be an acknowledgement that all three states exist, and someone has to go back and speak in plain English to those one or two steps back, because if we don't, if we really aren't prepared to bother to speak in plain English, well then we will never have access to Toby's organisations data, no matter how much he might want to provide it, because he doesn't understand even what it is, never mind the irritating technicalities surrounding the difference between a PDF and a CSV. In fact I guarantee you, at least one of those collections of three letters means absolutely nothing to Toby - and until you tell him why, he simply might not care.

Now Toby does care. A lot. I know he does, because he's wrestling with the same frustrations that I am. So how are we going to solve this knowledge gap? Who is going to step forward and explain in simple terms what each and every term means which gets flung around under the open data banner? Who is going to step forward from the developer camp and stop being snarky about all the stupid people who just don't get it, and make sure that those people do, so that the data stream continues, turning into a river and then a flood? Who will stand in the middle and negotiate between the two groups - explaining to one why patience is a virtue and Rome wasn't built in a day but we are getting somewhere, oh yes we are because at least Toby is asking the question - and those who are just like Toby and simply don't understand the possibilities, the capabilities, the implications, the assumptions, the potential?

Someone may well be doing all this already. Link me. Link Toby. Because we are not alone. We are lost. Lost in a sea of confusing terminology, which deters us from trying to do the things we know are good and right. We might be at different levels, Toby and I, but I really think we might both be at the same level of confused.

Utterly.

So can we put a temporary halt on the banner waving and can someone come and help the little people, please? Because, yes, we know government spending data isn't going to change the world, but if you'd only tell us what kind would, I could make a concerted effort to hunt it down for you and provide it in a nice shiny format for you. As long as if it's in something other than CSV you explain to me how to do it. Because JSON just looks like it's missing an A to me.

(Disclaimer: No Tobies were harmed during the making of this post. In fact I have been assured repeatedly that I have not misrepresented him in any way. Repeatedly. He's not clueless, it's important to point that out - he is aware of opendata and all its possibilities. Just struggling with the more technial stuff - and by that I mean the things we as geeks think are non technical, to others appear so. And they are in the majority, whether we like it or not.)