The Guardian (and others) produced their own websites that did a similar job and generally worked better. Since then, there have been further developments including the Legg report and further detail released (twice).

This means that the data added to WhatTheyClaimed.com has become out of date and incomplete. This presents a problem as it may confuse and mislead visitors to the site.

As a result, I’ve decided to turn off editing access to the site and plaster a ‘mothballed’ sign on every page. I’m probably going to archive the content shortly and take the site down entirely. Thank you to everyone who contributed to the website – it was fun.

On Friday, Charlie and I attended a hack day with a difference.  The ScraperWiki Hack and Hackers Hack Day was designed to bring designers, developers (hackers) and journalists (hacks) together to see what they could do.  While other hack days may have involved journalists in a media capacity, this event involved journalists from the beginning.

After an introduction to how the day would work, we all split out into groups and started to work on our projects.  I worked with Charlie, two other journalists (Ian and Annamarie) and another developer (Edmund) to investigate new election candidates in the safest seats in the country.  We mashed together data on constituencies, candidates and then did some good old fashioned journalism to fill in the blanks

Other teams worked away on other projects, only interrupted by the food and drink provided.  Finally, at the end of the day we sat down to present our work – eight projects in total.  It was amazing to see what different directions the different teams had gone in.

A few of the projects used data to begin investigating a story.  One group looked at the transport data released by data.gov.uk and mapped it onto constituencies; another group looked at awards paid out by Enterprise Ireland.  Richard Pope looked at the register of interests for London Assembly members and plotted how often and where they were taken out to lunch by third parties.  The largest group produced “They Write For You”, a mashup that demonstrates how many MPs write articles in newspapers.

Other projects were more focused towards building tools such as a system that allowed you to classify the output of data.gov.uk data (e.g. Excel, CSV, PDF, linked data etc.).  A last minute entry also wrote a scraper for UK bank holiday dates (a much needed tool for developers!).  A final fun project scraped all of the horoscopes for the last year and analysed the most used words for each star sign – producing some fantastic visualisations.

The day aimed to explore how hacks and hackers could / would work together.  I think the quality of the projects created show that it is a good match.  Journalists have an eye for a story that us developers might not have.  While we may enjoy throwing tools together and mashing up data, we don’t necessarily know what makes a good story.  Similarly, journalists don’t necessarily know or understand what developers do and how we do it.

Lessons were learned on both sides and show the growing importance of data based journalism.  As an experiment, I think it was a success and certainly should be repeated again.

If it is – I will be first on the list to attend!

Thanks must go to Channel 4/4IP for providing the venue for the day, Richard Pope and everyone from ScraperWiki for organising the event and helping us get working.

The past 36 hours haven’t been the best for those travelling with Eurostar. Five trains were stranded for hours in the Channel Tunnel without power, light, food, drink, heating or information. A further train became stuck near Ebbsfleet yesterday evening.

The focus today is on the cause of the incident and the safety procedures. However the issue that concerns me is how public service companies like Eurostar handle communication in such emergencies. Poor communication can lead to frustration, anger and even panic. I’ve experienced this in much less drastic circumstances when commuting to and from Cardiff by train.

Today the media are reporting that Eurostar passengers were given no information about what was going on. People at stations had similar experiences. Even the public and media had no idea of the unfolding events until late on Saturday morning.

It seems a standard practice for UK transport companies to ignore incidents, keep quiet, disappear and leave customers to their own devices. It doesn’t take much to keep people informed – even if there isn’t any ‘useful’ information to give them at the time. Even a simple “The track ahead is flooded; we are awaiting an engineer to confirm we can proceed” provides re-assurance that something is happening.

There are three distinct audiences in any incident, each with different information needs. Those directly involved (i.e. train passengers) who need to know when they will reach their destination. Those affected through waiting to travel or waiting to meet passengers, and everyone else (for example the media and general public).

Open communications to all three groups is key, but how can it be done effectively? Which channels should be used? Well, every possible channel actually. If you are preparing a message it doesn’t take much more effort to push this out on all available options

By example, recently I’ve read reports that Eurostar didn’t start updating Twitter until 14 hours after the incident began. Some have suggested that Twitters low usage justifies it being left until last. But for passengers on trains with phones and computers, this may be the only way that information can be easily received.  It only takes one person with access to disseminate the details.

Communication (or the lack of) is a big problem when it comes to any form of crisis. Yet the prevalence of new social media communication actually makes communication easier than ever. Businesses of all kind need to recognise this and plan to make best use of it. I hope that Eurostar and those providing public services learn the communication lessons of this weekend.

On a final note – Eurostar’s social media company behind (We Are Social) has come in for what I believe is unjustified criticism. They actually did a great job given difficult circumstances and went well beyond the requirements of any contractor. The poor communication lies squarely with Eurostar. You can read about “We Are Social’s” involvement on their blog and a interesting response from Grapevine Consulting about the criticism levelled on We Are Social.

Sometimes I come away from the office and struggle to see what we have actually achieved. When you are stuck into day to day work, it is quite easy to lose sight of the bigger picture.

This isn’t helped by the fact that part of our team’s aim does not have a tangible outcome. We spend a sizable portion of our energy on promoting our open principles within the organisation and converting people to our way of thinking. It isn’t an easy task, as I’m sure other people in this field will confirm but I’m starting to feel like we are making some headway.

The first report (of which I hope there will be many) has just gone online, in a format allowing the public to comment on each part of the document. The Consumer Focus website is now powered using the open source Wordpress Content Management System (joining the likes of bis.gov.uk and number10.gov.uk). Other developments are also in the pipeline for the next few months and all have stemmed from us ’spreading the gospel’.

This sort of work is going on across Government on many different levels, but recently there seems to have been a bit of a breakthrough. There is lots of things happening at the moment which you might find interesting. I’m going to briefly run through them.

Data.gov.uk

This is the UK version of the USA’s Data.gov and has been operating in a restricted beta for a few months. Government departments have been posting raw datasets for developers to use for free in their open projects. From sometime in January, the site will go live with over 1100 datasets from central government.

Ordnance Survey, NHS Data, Office of National Statistics Data

Work is underway to open up more datasets and make them freely available. These include health data from NHS Choices, mapping data from Ordnance Survey (OS) and statistics from the Office of National Statistics. Consultations are planned for early in the new year with aims to open them up by April 2010. There also seems to be some movement on publishing data of how the money allocated in the 2008 Pre-Budget report has been spent (possibly by spring, with further data released in the summer).

More information about the data releases can be found in the Government’s recent strategy ‘Putting the Frontline First: Smarter Government‘ and on the Government’s Digital Engagement blog.

Crime Maps

The various Police forces around the UK have released crime statistics using their new mapping service. I’ve had a bit of a play around and it looks pretty good. You can see lots of information about your local area and download the data in CSV format.

More information about the Crime Maps can be found on the Directgov website.

Post Code Data

A few months ago I wrote about the issue of post codes after ErnestMarples.com was forced to shut down by the Royal Mail. Since then, a campaign has been ongoing to set the post code free. Well, it looks like someone might have listened. As part of the opening up of OS data above, a consultation will be taking place shortly on the opening up of post code datasets, which could see the data released by April 2010.

More information about the release of post code data can be found on the Ernest Marples blog.

Digital Economy Bill

The Digital Economy Bill has recently been introduced to Parliament and is due to go to committee stage shortly. This bill contains a large chunk of the actions and recommendations from the Digital Britain Report earlier in the year (with a number of strange and scary additions). There is a lot of talk about this at the moment and the effect it will have on the digital life and innovation in the UK. You can follow the progress of the bill on the Parliament.UK website, or on twitter using the #debill hashtag. If you want more information on the bill and how it might affect us, there are a large number of blogs and websites dedicated to the issue.

Directgov

This brings me to one of the last bits for this entry, and I make no apologies for the shameless self-promotion here.

Direct.gov.uk is the UK Government’s official website for the general public. It aims to collect all the public facing information into one place and make it easy to find. Unfortunately, its usefulness and ability to achieve this aims has been called into question. Personally, I’ve never liked the website and can never find information I want from the site (in fact, I tried several times to find information relating to bits in this post but gave up).

Consumer Focus has recently published a report on the website that analyses how well it delivers on its promise. As I alluded to earlier, this report is available online in fully commentable format. We are encouraging everyone and anyone to send us their thoughts on Directgov, what the problems are and what can be improved. Similarly, if you think it is doing something right, we want to know that too. You can find out more about the report at http://directgov.consumerfocuslabs.org.

I believe there will be a series of workshops planned for early in the new year to identify ways of engaging people in digital public service development. You can sign up on the site to be kept informed (and I’ll no doubt be publishing the information here as well).

Looking back at everything that has been achieved and everything that has happened in the last nine months, I’m actually quite excited at what we can do by this time next year (election permitting of course!). I have my fingers crossed 2010 will continue making leaps towards open government.

When you move into a new area, how do you find out about the community you are going to be living in?  How do you find out about the community groups, the local services, the bus times?  Some of the more technology savvy amongst us might look towards the web in the hope that the information is available.  Others might wander around and talk to neighbours, a few people just won’t care.

In Govilon, we’ve tried to make this process easier.  We’ve launched a “Village Information Pack” (VIP) – a small A5 folder which is delivered when someone new moves into the village.  The pack contains a list of community groups and services offered in nearby villages (post offices, libraries, leisure centres etc).  It lists bus times, school term dates and opening times of the shop & pubs.  We also allow local businesses to put in a leaflet about their services.

The initial version of the pack was delivered to every house in the village, and updated ones are delivered by volunteers whenever we hear about someone moving in.  So far it has been well received.

Recently, the Government began the process of opening up its datasets in a similar way to Data.Gov in America.  The Newspaper Club has taken some of this data and produced a prototype of an information pack for people moving into a new area.  It gathers information about local services, environmental information and crime statistics from the Government’s datasets and presents them in a small newspaper. They’ve also gone a step further and included maps, travel times to certain locations, NHS information and council details.

This is a fantastic idea and makes really good use of data which was not previously available.  It shows what can be done when you make public data available for free. Something like this would really complement the village information pack we’ve created for Govilon.

You can find out more information and some pictures of the prototype on the Newspaper Club website.

Britt Selvitelle (Twitter) – The Future of Frontend Engineering – Learning from Twitter.

Britt talked about their front end platform and how they now see their website as “the web client” of twitter, in much the same way as TweetDeck is a desktop client and Tweetie is an iPhone client. They all connect to the Twitter Platform.  He also talked about “Twitter Labs” (working title), a way of releasing some of the tools they are developing internally.

View the notes

Simon Wardley (Freelance) – The Future of the Cloud.

I think I actually heard Simon talk at FOWA Dublin on a similar topic or at the very least a similar style – slides in the hundreds for just a 20 minute talk.

Simon pointed out that not many people really know what ‘cloud computing’ is, and that we are destined to have lots of problems if there is not a common interface between them all.  Some people develop using Amazon’s cloud APIs, others using Rackspace.  If one of these shuts down, you have to redevelop using a different set of APIs.  This is certainly a good point; interoperability between the different cloud providers is going to be a big future issue.

Sanj Matharu (Vodafone) & Joel Moss (Codaset) – Your App + Mobile Widgets = Awesome

This talk showed an example of how  to build an application for a mobile phone using HTML, CSS and JavaScript.  Fantastic – an easy way to build mobile applications at last.  But.  This is only for one particular type of phone on the Vodafone network.  OK, so is the iPhone, but the difference is number of users.  If we could produce an application using HTML, CSS & JavaScript for lots of different phones, I’d be very very interested.

Robin Christopherson (AbilityNet) – Live Demo of Screen Reader Issues

I’ve seen Robin talk 3 times now, and all of them have been fantastic.  It amazes me that even now, website accessibility is not thought about at all.  It isn’t seen as important, and in some cases, hasn’t been considered.  I’m going to write a further post about this shortly, so keep an eye out.  In the mean time, try using Facebook with a screen reader. Good luck

Last week I attended the Future of Web Apps conference in London.  For those who haven’t been before – this event is run by Carsonified (a web agency based in Bath) and covers the latest thinking around developing Internet applications.  I went along to the FOWA event in Dublin earlier in the year, but this one was much bigger and spread over two days.

The event covered topics such as how to market your web applications, new development methods to make development quicker, HTML5, accessibility and cloud computing.

For the benefit of those that couldn’t make it to FOWA, or want to refresh their memory, here are some of my (very) rough notes from day 1 of the event.

Kevin Rose (Digg) – Taking your Site from One to One Million Users

This talk was all about how to get the most out of your users, based on Kevin’s experience with Digg.  Communication is key, but it also helps to feed user’s ego a bit!

View the notes

Mike McDerment (FreshBooks) – Three Vital Marketing Systems for a Successful Web App

Mike’s talk outlined three elements a site needs to help you market your application: tracking, storage and reporting.  While Google Analytics does a fairly good job of the tracking and reporting, you can’t drill down to a per user basis (based on your user accounts).  Nothing quite makes up for having your own raw user data.

View the notes

Dustin Dias (Twitter) – The Future of JavaScript Design Patterns

Dustin talked about JavaScript with a surprising number of references to corn.  The thought was that JavaScript frameworks are not necessarily the best way to go as they frame and affect the way you work.  Either way, the best thing to do is love the language, because JavaScript is not going anywhere.

View the notes

Addison Berry (Lullabot) – Passion and Paychecks: Open Source Lessons

I don’t really have any notes from this talk, but Addison was talking about open source and money.  Essentially, it is a myth that there is no money to be made in open source.  It is also important to be passionate and happy about what you are doing.  If you are not passionate, and if the people who are working for you are not passionate, then you won’t do a good job.

Francisco Tolmasky (280 North) – Introducing Atlas: A Visual Development Tool for creating Web Applications

Again, no notes for this one, but Francisco showed off a new visual development tool.  280 North are the makers of Cappuccino, the open source framework for building applications.  Atlas is a visual editor for making these applications (there will be both Windows and Mac versions).  With Atlas, you can produce an application in minutes, and it looks fantastic.  Can’t wait for this to be released.

David Prager (Revision3) – Get Niche, Rich, and go Mainstream

David’s talk was all about how to start off your web application.  The overview is to pick a niche (or “nitch” as he pronounced it), and build out from there.

View the brief notes

Osama Bedier (PayPal) – Payments innovation will unlock the Web’s potential

Osama talked about micropayments and PayPal’s new APIs.  PayPal believe that micropayments will change the way the web operates.  Their new APIs will make this possible.

Chris Abad (Spymaster) – Advanced Web App Marketing Strategies

Chris talked about the annoying but very successful Spymaster game on Twitter and how they planned it.  It is essentially a standard viral marketing technique – make something interesting so people will share links (or better still, just post it to their twitter stream automatically).

Cat Lee (Facebook) – Going global: The Future of Facebook Connect

Facebook showed off the latest extension to Facebook Connect.  Now any website can make use of their crowdsourced translation engine to make their sites multi-lingual.  While the talk was pretty much a sales pitch for Facebook Connect, the topic was interesting.  I’ll be having a play around with this when I get the chance.

Bruce Lawson (Opera) – The Future of HTML5

This was probably one of the most interesting talks from the event.  Bruce showed off some of the new aspects of HTML5, explained the new tags, which browsers they work on, their accessibility and their backwards compatibility.  There is some fantastic stuff coming out of HTML5, but I can’t help but think it is a slightly backwards step and we could end up with horribly messy code with unclosed tags.  That is one of the reasons why I like XML and XHTML.  Anyway, HTML5 is not yet ready for production environments, so we will have to see how it is used in real life.

View the notes

Chris Thrope (The Guardian) – How The Guardian is using APIs, Frameworks and Tools to Build a “Mutalised” Newspaper

Chris talked about the Guardian’s efforts to embrace new technology.  Unlike some other newspapers, they are embracing the new way of dealing with things.  Chris also talked about how they built some of their online applications such as their MP expenses site and their twitter conversations engine.  I’m probably going to come back and write a seperate entry about the expenses site shortly, as I had a bit of my own experience in this area!  Chris showed off their Guardian DataStore and how people are using it to produce other data and statistics.

View the notes

I’ll be posting my notes from FOWA day 2 tomorrow.

The issue surrounds a small piece of text – the Post Code. It sits at the bottom of every address (similar to the Zip code in the USA). It is the holy grail of data in the UK. It maps each place in the country to a physical location. The Royal Mail uses it to assist with routing mail from A to B, developers use it to verify addresses and locate other services nearby. Unfortunately, access to this database costs thousands (approximately £3800 per website per year).

In the majority of cases, small not-for-profit websites or online experiments just can’t afford to pay out for this license. I know from experience – we don’t have a postcode lookup on CllrTweeps for exactly this reason. I can’t afford to pay out the best part of a sixth of my salary (before tax) each year for a postcode licence for a personal project.

Enter ErnestMarples.com – a website that provided access to the postcode database to not for-profit websites for free. A developer could submit a Post Code to the API and get the appropriate longitude and latitude back in return. Although the legality of the site was questionable, it was used by a large number of sites similar to CllrTweeps. These included PlanningAlerts.com, JobCentreProPlus, TheStraightChoice, HealthWhere and so on.

Planning Alerts used it to good effect. It allowed you to sign up for email alerts of planning applications that were submitted near you. How did it work this out? By mapping your Post Code to a physical location. The site was so successful that other sites have cropped up that disseminate its data through other communication channels.

Unfortunately yesterday, the Royal Mail threatened legal action against ErnestMarples and the site was taken down. As of yesterday, all the above sites have gone offline too. Sites of real value to people in the UK have been taken down because of the Royal Mail’s actions.

Now some people will argue that they shouldn’t have been using a dubious source for their data. But as I’ve explained before, they don’t really have much choice. It is a case of using a dodgy dubious source, or not building the site at all. None of these sites were run on a for-profit basis. None of them had a spare £4000 lying around to buy the license.

The Royal Mail claims that they were losing money because of the site (although they have yet to answer enquiries about how much they’ve actually lost). I’m sceptical about this – none of these sites could have afforded to buy the data anyway, so how they could have lost money I do not know!

Lets not forget that the database was created using public money through taxes paid for by the people of the UK. This is public data that is of benefit to the whole of the UK.

By closing down this site and refusing to provide a not-for-profit licence for their Post Code database, the Royal Mail is stifling innovation in the UK.

Last week I attended the Future of Web Apps conference in London. I was amazed at how behind we are in the UK compared to the USA when it comes to innovation. I’m now convinced it is partly due to the failure of organisations like the Royal Mail and Ordnance Survey to modernise. They are acting in the same way as the music industry and trying to stop innovation rather than cultivate a mutually beneficial environment.

I’m sorry to see PlanningAlerts and The Straight Choice die. But maybe it will be the kick up the backside that the UK needs to get its house in order.

Until the Royal Mail releases its Post Code database with a free (or very very cheap) not-for-profit license, we will never have a Digital Britain.

There has been a lot of talk on Twitter today about the launch of the new Birmingham City Council website.  Without a doubt, Birmingham have successfully launched one of the most delayed and expensive websites of the last few years.  After being delayed for 6 months (original launch date was expected to be March 2009), and going significantly over budget (costing a total of £2.8 million), you’d expect the site to be the best thing ever developed.

Unfortunately it isn’t.  The site is riddled with problems; suffers from poor build quality, lack of thinking, and technology that wouldn’t be out of place in the early part of this century.  Here are just a few of the problems I’ve come across after browsing the site for a little over 10 minutes:

• Absolutely no thought given to SEO
• Frequent broken incoming links
• Incomprehensible alt tags and title attributes for links and images
• Lack of validation on many pages
• No modern technology

In thought I’d take a very brief look at a few of these.

Search Engine Optimisation

There really is no excuse for not using SEO optimised URLs on a public site anymore.  Your URLs should be human readable and understandable.  This helps with not only search engine optimisation, but with your website’s users as well!

Unfortunately this website has stuck with the horrible model of using exceptionally long and completely indistinguishable URLs throughout the system.  Here is an example of one of the pages:

http://www.birmingham.gov.uk/cs/Satellite?c=Page&childpagename=Highways-and-Maintenance%2FPageLayout&cid=1223092719662&pagename=BCC%2FCommon%2FWrapper%2FInlineWrapper

I count this URL at 166 characters long – a simplified human readable URL for this site could be as low as 47 characters long (http://www.birmingham.gov.uk/transport/highways)

From the first URL, I have no idea where that page really goes; the second URL is much more obvious.

Broken Incoming Links

Almost all incoming links to the website have been broken with this re-launch.  Even the Google Sitelinks which appear when you search for Birmingham City Council no longer work.  This is not only bad for search engine optimisation, but any bookmarks or links to content on the site will no longer work.

Incomprehensible Alt & Title Attributes

This is probably the most disturbing part of this website.  While looking through the website, I’ve come across title tags such as “BNE /cs/Satellite/v?packedargs=AlphabetValue%3DA&rendermode=live”, “TLE contactus” and “BNE copy”.  These links go to the A-Z website index for the letter A, a contact us page, and the copyright statement respectively.  The first is quite obviously the most incomprehensible, but I’d love to know what the TLE and BNE stand for in the others.

Title attributes are really important – these are used by screen readers (the sorts of things used by those with no or poor visibility) to identify where a link goes.  Without a descriptive title tag, you have no idea where you are going.

Alt attributes are equally important – these describe the content of an image if the viewer can’t see it.  There are quite a few images around the site with no, or incomprehensible alt attributes.

Like the SEO URLs, there is absolutely no excuse for this – in fact, its purely negligent.

Lack of Modern Technology

This is less important, but I felt it had to be said.  There are no RSS feeds whatsoever – who builds a website with no RSS feeds?

Overview

This website cost a lot – significantly more than any other website I’ve come across.  It is badly built, suffers from accessibility issues, and it generally pretty poor.  To whoever signed off this development – you should be taken out and shot.  To the people of Birmingham – I’m sorry, but you’ve been had.  Your money has been wasted, and you should hold your council to account for this farce at the next election.

You can follow more discussions on this redesign on the Birmingham Post website, and by following the Twitter hash tags: #bccwebsite, #bcc and #fail.

This evening’s programme “investigated” the water quality of beaches in the UK – in particular looking at problems caused by sewage.  The reporter looked at the issue of “combined sewer overflows (CSOs)”.  These are essentially overflow pipes for the sewage system (similar to the overflow pipe in your sink) that releases pressure when there is no more capacity for water.  Instead of a pipe sticking out of your bathroom wall, these overflow pipes run from the sewage treatment works to a nearby watercourse (a river, stream or the sea).  These CSOs are designed to operate very rarely – approximately 10 times a year (at maximum) – usually after a heavy rainfall event.

Unfortunately, the cases highlighted tonight have clearly been discharging sewage significantly more often than this.  However, rather than looking at the cause of the issue, the programme instead decided to spend its time bashing the water companies, Environment Agency and local councils.  All while scaring people into thinking that 1 in 20 are going to get ill by swimming in the water contaminated with sewage.

Over the past few years, we have increasingly had very wet summers.  This has contributed to more and more discharges from the CSOs.  But why has this happened? Why are the sewage systems becoming overwhelmed more often?  What big change has happened over the last 10/20 years that might have caused this?  A lot of people will blame climate change, but it’s much more than that.

Tarmac, concrete and paving.  Over the last 10 years, our towns and cities have seen more of their green space covered in tarmac and concrete, people have paved over their driveways and gardens making it more difficult for rainwater to permeate into the ground.  Living spaces have been expanded with extensions and conservatories, reducing gardens.  Where water would previously have been absorbed by soil, it is now blocked by impermeable surfaces.  Where does all this water go? Straight down the drain and into the sewer.  In addition to this, lazy building methods and poor planning inspections have lead to guttering being connected to the sewer, rather than a soak-away which allows the rainwater to drain more naturally –and not exacerbate the problem every time it rains heavily.

When we have rainstorms, the sewers are filled to the brim with rainwater – completely overwhelming a system which was not designed for this purpose.  The end result is a discharge of rainwater and sewage into the sea through the CSOs.

So how can we reduce this problem (another topic not covered by tonight’s programme)?  If you have paved, tarmaced or concreted over your driveway/garden – rip it up.  Replace it with something more permeable like gravel or a permeable paving solution.  If your guttering is connected to the sewer, build a soak-away to allow the water to be absorbed naturally, or consider a Rainwater Harvesting system.  Also take a look at Sustainable Drainage Systems (SuDS) – there is lots of information on Wikipedia and SuDSWales.com.

By implementing these measures, we can significantly reduce the amount of water that ends up in the sewer – thus reducing the number of CSOs that happen following rainstorms.

You can rewatch the programme for 7 days on the BBC iPlayer.