If you dig a little deeper into the figures you can see that central government has spent £14m on Strategy & Planning, £15.8m on Design & Build, £23.8m on Hosting & Infrastructure and £9.7m on Testing (these are categories that COI requires under their TG128 standard). Despite their efforts (gathering this kind of data is by no means easy) I have concerns about the quality. For example:

• The Ministry of Defence spent £418,000 on Design & Build: however, that includes figures for their internal intranet whereas the others do not appear to include intranet costs
• A comment against the Forestry Commission figures numbers states “This is as close as I can make it. Won’t be too far out.”
• The Office of Fair Trading hosting costs cover both their website and internal intranet
• Numerous departments (Ministry of Justice, Audit Commission, Department for Environment, Food & Rural Affairs, GCHQ, Crown Prosecution Service and the Statistics Authority) show no costs for website hosting

Quite clearly there is a discrepancy in the ways each department has calculated their figures. There could be quite legitimate reasons for this, for example, hosting could be part of a much bigger IT contract, making it difficult to split out website costs. However it does raise questions as to the accuracy of the dataset as a whole.

Interestingly, the Crown Prosecution Service does not appear to have provided any costs (all fields are marked as zero); they have however answered an FOI request recently, revealing they paid nearly £122k last year.

That said the spending figure is horrifying. The government is spending way too much on websites. I can’t quite figure out how spending £35 million on BusinessLink is justified, or £10.4 million on Directgov for that matter, particularly when these sites were supposed to reduce the cost of public sector websites.

I would also be interested to see a breakdown of where all of this money is spent. I wonder how much of it goes to the big public sector outsourcing companies rather than SME web development agencies.

I agree that the costs need to be brought down, and some of that needs to be through closing websites and rationalising hosting infrastructures. However I think that there needs to be a proper look at why the costs for producing websites are so high – value for money is just not being achieved in a lot of these cases.

Simon Dickson from Puffbox has written more about these figures on his blog, as has Harry Metcalfe from the Dextrous Web.

There are lots of different opt-out services that exist, all of which have separate sign-up forms and require you to fill out the same details (such as name, address, email address), over and over again. We felt there was a better solution, and so we built a site that combines the forms into one and does the hard work of submissions for you. The initial version of StayPrivate.org incorporates the Telephone, Mail & Baby Mail Preference service offered by the Direct Marketing Association.

I’ve been working on this project on and off for the past seven months (although most of the work has happened in the last two months or so), so I’m really pleased to see it launch. Since we flipped the switch early on Thursday morning, we’ve had over 4500 registrations and submitted over 6500 applications to the DMA opt-out services. This is many more than we were originally anticipating for the first public release.

The day of launch was amazing. We had to respond to issues quickly – sometimes within a few minutes. We pushed out nearly 30 fixes through out the day and switched off parts of the site to reduce server load. Since then we’ve been reducing the queues of submissions to ensure that everyone who registered has been processed.

The next task is to go through the site and look at where we can make improvements. I-ll be spending the next few days reworking parts of the site and implementing some of the suggestions we-ve received.

If you want to know more about StayPrivate or want to keep track of my work on this project, take a look at the CF Labs blog.

Update: fixing a spelling mistake pointed out by @keeran.

Of course, with a new government brings a new direction. Steph Gray, a former civil servant working in the technology arena has written a fantastic overview of what might happen to public sector IT over the next parliament. While these potential plans seem to focus on large scale IT projects, we’ve already started to see the effects of the new government as it has attempted to make its mark.

Within hours of David Cameron stepping into Number 10, public sector websites underwent a huge change. Most policy generated for the previous Government was cleared from department websites and consigned to The National Archives, Twitter and social media accounts were updated, and in some cases entire websites were replaced.

DEFRA (Department for Environment, Food and Rural Affairs) launched a new, temporary website to differentiate between new and old policy. Department for Business, Innovation and Skills (BIS) on the other hand have left most of their policy in place, albeit with a warning stating that it relates to the previous government.

The Department for Culture, Media and Sport (DCMS) website saw one of the biggest changes with a fantastic new design. The Department for Children, Schools and Families (DCSF) also got a new website, but this time under the new name “Department for Education“.

Both of these departments now have brand new Twitter accounts – @dcms and @educationgovuk. Unfortunately, they both already had accounts prior to 12th May – raising questions as to why they needed new ones. The old DCMS account (@dcmsgovuk) still lives on, dormant and not updated. No one appears to have bothered to inform their existing followers of the presence of the new feed. The DCSF account (@dcsfgovuk) has suffered an even more drastic fate. An account which at one point had some 4000 followers was deleted along with all of its content.

Other departments handled their changes a bit better – Downing Street rebranded all of their social media accounts to “Number10Gov” for consistency. Their twitter account was renamed, leaving all of their followers and previous content intact. This, in my opinion, is a much better way of handling the change.

In general these immediate changes have been a mixed bag. Some have worked well – such as the changes to Number 10 and the new DCMS website. Others appear to have not been thought through – such as the new account for DCSF. Unfortunately however, there has been no consistency between departments or government as a whole.

This of course has been the first major change in government since the rapid growth of social media platforms, so quite clearly there was going to be a bit of trial and error. Hopefully next time the public sector will be a bit wiser when it comes to social media and not leave their followers hanging or disconnected.

The Guardian (and others) produced their own websites that did a similar job and generally worked better. Since then, there have been further developments including the Legg report and further detail released (twice).

This means that the data added to WhatTheyClaimed.com has become out of date and incomplete. This presents a problem as it may confuse and mislead visitors to the site.

As a result, I’ve decided to turn off editing access to the site and plaster a “mothballed” sign on every page. I’m probably going to archive the content shortly and take the site down entirely. Thank you to everyone who contributed to the website – it was fun.

On Friday, Charlie and I attended a hack day with a difference. The ScraperWiki Hack and Hackers Hack Day was designed to bring designers, developers (hackers) and journalists (hacks) together to see what they could do. While other hack days may have involved journalists in a media capacity, this event involved journalists from the beginning.

After an introduction to how the day would work, we all split out into groups and started to work on our projects. I worked with Charlie, two other journalists (Ian and Annamarie) and another developer (Edmund) to investigate new election candidates in the safest seats in the country. We mashed together data on constituencies, candidates and then did some good old fashioned journalism to fill in the blanks

Other teams worked away on other projects, only interrupted by the food and drink provided. Finally, at the end of the day we sat down to present our work – eight projects in total. It was amazing to see what different directions the different teams had gone in.

A few of the projects used data to begin investigating a story. One group looked at the transport data released by data.gov.uk and mapped it onto constituencies; another group looked at awards paid out by Enterprise Ireland. Richard Pope looked at the register of interests for London Assembly members and plotted how often and where they were taken out to lunch by third parties. The largest group produced “They Write For You“, a mashup that demonstrates how many MPs write articles in newspapers.

Other projects were more focused towards building tools such as a system that allowed you to classify the output of data.gov.uk data (e.g. Excel, CSV, PDF, linked data etc.). A last minute entry also wrote a scraper for UK bank holiday dates (a much needed tool for developers!). A final fun project scraped all of the horoscopes for the last year and analysed the most used words for each star sign -producing some fantastic visualisations.

The day aimed to explore how hacks and hackers could / would work together. I think the quality of the projects created show that it is a good match. Journalists have an eye for a story that us developers might not have. While we may enjoy throwing tools together and mashing up data, we don’t necessarily know what makes a good story. Similarly, journalists don’t necessarily know or understand what developers do and how we do it.

Lessons were learned on both sides and show the growing importance of data based journalism. As an experiment, I think it was a success and certainly should be repeated again.

If it is – I will be first on the list to attend!

Thanks must go to Channel 4/4IP for providing the venue for the day, Richard Pope and everyone from ScraperWiki for organising the event and helping us get working.

The past 36 hours haven’t been the best for those travelling with Eurostar. Five trains were stranded for hours in the Channel Tunnel without power, light, food, drink, heating or information. A further train became stuck near Ebbsfleet yesterday evening.

The focus today is on the cause of the incident and the safety procedures. However the issue that concerns me is how public service companies like Eurostar handle communication in such emergencies. Poor communication can lead to frustration, anger and even panic. I’ve experienced this in much less drastic circumstances when commuting to and from Cardiff by train.

Today the media are reporting that Eurostar passengers were given no information about what was going on. People at stations had similar experiences. Even the public and media had no idea of the unfolding events until late on Saturday morning.

It seems a standard practice for UK transport companies to ignore incidents, keep quiet, disappear and leave customers to their own devices. It doesn’t take much to keep people informed – even if there isn’t any “useful” information to give them at the time. Even a simple “The track ahead is flooded; we are awaiting an engineer to confirm we can proceed” provides re-assurance that something is happening.

There are three distinct audiences in any incident, each with different information needs. Those directly involved (i.e. train passengers) who need to know when they will reach their destination. Those affected through waiting to travel or waiting to meet passengers, and everyone else (for example the media and general public).

Open communications to all three groups is key, but how can it be done effectively? Which channels should be used? Well, every possible channel actually. If you are preparing a message it doesn’t take much more effort to push this out on all available options

By example, recently I’ve read reports that Eurostar didn’t start updating Twitter until 14 hours after the incident began. Some have suggested that Twitters low usage justifies it being left until last. But for passengers on trains with phones and computers, this may be the only way that information can be easily received. It only takes one person with access to disseminate the details.

Communication (or the lack of) is a big problem when it comes to any form of crisis. Yet the prevalence of new social media communication actually makes communication easier than ever. Businesses of all kind need to recognise this and plan to make best use of it. I hope that Eurostar and those providing public services learn the communication lessons of this weekend.

On a final note – Eurostar’s social media company behind (We Are Social) has come in for what I believe is unjustified criticism. They actually did a great job given difficult circumstances and went well beyond the requirements of any contractor. The poor communication lies squarely with Eurostar. You can read about “We Are Social’s” involvement on their blog and a interesting response from Grapevine Consulting about the criticism levelled on We Are Social.

Sometimes I come away from the office and struggle to see what we have actually achieved. When you are stuck into day to day work, it is quite easy to lose sight of the bigger picture.

This isn’t helped by the fact that part of our team’s aim does not have a tangible outcome. We spend a sizable portion of our energy on promoting our open principles within the organisation and converting people to our way of thinking. It isn’t an easy task, as I’m sure other people in this field will confirm but I’m starting to feel like we are making some headway.

The first report (of which I hope there will be many) has just gone online, in a format allowing the public to comment on each part of the document. The Consumer Focus website is now powered using the open source WordPress Content Management System (joining the likes of bis.gov.uk and number10.gov.uk). Other developments are also in the pipeline for the next few months and all have stemmed from us ‘spreading the gospel’.

This sort of work is going on across Government on many different levels, but recently there seems to have been a bit of a breakthrough. There is lots of things happening at the moment which you might find interesting. I’m going to briefly run through them.

Data.gov.uk

This is the UK version of the USA’s Data.gov and has been operating in a restricted beta for a few months. Government departments have been posting raw datasets for developers to use for free in their open projects. From sometime in January, the site will go live with over 1100 datasets from central government.

Ordnance Survey, NHS Data, Office of National Statistics Data

Work is underway to open up more datasets and make them freely available. These include health data from NHS Choices, mapping data from Ordnance Survey (OS) and statistics from the Office of National Statistics. Consultations are planned for early in the new year with aims to open them up by April 2010. There also seems to be some movement on publishing data of how the money allocated in the 2008 Pre-Budget report has been spent (possibly by spring, with further data released in the summer).

More information about the data releases can be found in the Government’s recent strategy ‘Putting the Frontline First: Smarter Government‘ and on the Government’s Digital Engagement blog.

Crime Maps

The various Police forces around the UK have released crime statistics using their new mapping service. I’ve had a bit of a play around and it looks pretty good. You can see lots of information about your local area and download the data in CSV format.

More information about the Crime Maps can be found on the Directgov website.

Post Code Data

A few months ago I wrote about the issue of post codes after ErnestMarples.com was forced to shut down by the Royal Mail. Since then, a campaign has been ongoing to set the post code free. Well, it looks like someone might have listened. As part of the opening up of OS data above, a consultation will be taking place shortly on the opening up of post code datasets, which could see the data released by April 2010.

More information about the release of post code data can be found on the Ernest Marples blog.

Digital Economy Bill

The Digital Economy Bill has recently been introduced to Parliament and is due to go to committee stage shortly. This bill contains a large chunk of the actions and recommendations from the Digital Britain Report earlier in the year (with a number of strange and scary additions). There is a lot of talk about this at the moment and the effect it will have on the digital life and innovation in the UK. You can follow the progress of the bill on the Parliament.UK website, or on twitter using the #debill hashtag. If you want more information on the bill and how it might affect us, there are a large number of blogs and websites dedicated to the issue.

Directgov

This brings me to one of the last bits for this entry, and I make no apologies for the shameless self-promotion here.

Direct.gov.uk is the UK Government’s official website for the general public. It aims to collect all the public facing information into one place and make it easy to find. Unfortunately, its usefulness and ability to achieve this aims has been called into question. Personally, I’ve never liked the website and can never find information I want from the site (in fact, I tried several times to find information relating to bits in this post but gave up).

Consumer Focus has recently published a report on the website that analyses how well it delivers on its promise. As I alluded to earlier, this report is available online in fully commentable format. We are encouraging everyone and anyone to send us their thoughts on Directgov, what the problems are and what can be improved. Similarly, if you think it is doing something right, we want to know that too. You can find out more about the report at http://directgov.consumerfocuslabs.org.

I believe there will be a series of workshops planned for early in the new year to identify ways of engaging people in digital public service development. You can sign up on the site to be kept informed (and I’ll no doubt be publishing the information here as well).

Looking back at everything that has been achieved and everything that has happened in the last nine months, I’m actually quite excited at what we can do by this time next year (election permitting of course!). I have my fingers crossed 2010 will continue making leaps towards open government.

In Govilon, we’ve tried to make this process easier. We’ve launched a “Village Information Pack” (VIP) – a small A5 folder which is delivered when someone new moves into the village. The pack contains a list of community groups and services offered in nearby villages (post offices, libraries, leisure centres etc). It lists bus times, school term dates and opening times of the shop & pubs. We also allow local businesses to put in a leaflet about their services.

The initial version of the pack was delivered to every house in the village, and updated ones are delivered by volunteers whenever we hear about someone moving in. So far it has been well received.

Recently, the Government began the process of opening up its datasets in a similar way to Data.Gov in America. The Newspaper Club has taken some of this data and produced a prototype of an information pack for people moving into a new area. It gathers information about local services, environmental information and crime statistics from the Government’s datasets and presents them in a small newspaper. They’ve also gone a step further and included maps, travel times to certain locations, NHS information and council details.

This is a fantastic idea and makes really good use of data which was not previously available. It shows what can be done when you make public data available for free. Something like this would really complement the village information pack we’ve created for Govilon.

You can find out more information and some pictures of the prototype on the Newspaper Club website.

Britt Selvitelle (Twitter) – The Future of Frontend Engineering – Learning from Twitter.

Britt talked about their front end platform and how they now see their website as “the web client” of twitter, in much the same way as TweetDeck is a desktop client and Tweetie is an iPhone client. They all connect to the Twitter Platform. He also talked about “Twitter Labs” (working title), a way of releasing some of the tools they are developing internally.

View the notes

Simon Wardley (Freelance) – The Future of the Cloud.

I think I actually heard Simon talk at FOWA Dublin on a similar topic or at the very least a similar style – slides in the hundreds for just a 20 minute talk.

Simon pointed out that not many people really know what “cloud computing” is, and that we are destined to have lots of problems if there is not a common interface between them all. Some people develop using Amazon’s cloud APIs, others using Rackspace. If one of these shuts down, you have to redevelop using a different set of APIs. This is certainly a good point; interoperability between the different cloud providers is going to be a big future issue.

Sanj Matharu (Vodafone) & Joel Moss (Codaset) – Your App + Mobile Widgets = Awesome

This talk showed an example of how to build an application for a mobile phone using HTML, CSS and JavaScript. Fantastic – an easy way to build mobile applications at last. But. This is only for one particular type of phone on the Vodafone network. OK, so is the iPhone, but the difference is number of users. If we could produce an application using HTML, CSS & JavaScript for lots of different phones, I’d be very very interested.

Robin Christopherson (AbilityNet) – Live Demo of Screen Reader Issues

I’ve seen Robin talk 3 times now, and all of them have been fantastic. It amazes me that even now, website accessibility is not thought about at all. It isn’t seen as important, and in some cases, hasn’t been considered. I’m going to write a further post about this shortly, so keep an eye out. In the mean time, try using Facebook with a screen reader. Good luck

The event covered topics such as how to market your web applications, new development methods to make development quicker, HTML5, accessibility and cloud computing.

For the benefit of those that couldn’t make it to FOWA, or want to refresh their memory, here are some of my (very) rough notes from day 1 of the event.

Kevin Rose (Digg) – Taking your Site from One to One Million Users

This talk was all about how to get the most out of your users, based on Kevin’s experience with Digg. Communication is key, but it also helps to feed user’s ego a bit!

View the notes

Mike McDerment (FreshBooks) – Three Vital Marketing Systems for a Successful Web App

Mike’s talk outlined three elements a site needs to help you market your application: tracking, storage and reporting. While Google Analytics does a fairly good job of the tracking and reporting, you can’t drill down to a per user basis (based on your user accounts). Nothing quite makes up for having your own raw user data.

View the notes

Dustin Dias (Twitter) – The Future of JavaScript Design Patterns

Dustin talked about JavaScript with a surprising number of references to corn. The thought was that JavaScript frameworks are not necessarily the best way to go as they frame and affect the way you work. Either way, the best thing to do is love the language, because JavaScript is not going anywhere.

View the notes

Addison Berry (Lullabot) – Passion and Paychecks: Open Source Lessons

I don’t really have any notes from this talk, but Addison was talking about open source and money. Essentially, it is a myth that there is no money to be made in open source. It is also important to be passionate and happy about what you are doing. If you are not passionate, and if the people who are working for you are not passionate, then you won’t do a good job.

Francisco Tolmasky (280 North) – Introducing Atlas: A Visual Development Tool for creating Web Applications

Again, no notes for this one, but Francisco showed off a new visual development tool. 280 North are the makers of Cappuccino, the open source framework for building applications. Atlas is a visual editor for making these applications (there will be both Windows and Mac versions). With Atlas, you can produce an application in minutes, and it looks fantastic. Can’t wait for this to be released.

David Prager (Revision3) – Get Niche, Rich, and go Mainstream

David’s talk was all about how to start off your web application. The overview is to pick a niche (or “nitch” as he pronounced it), and build out from there.

View the brief notes

Osama Bedier (PayPal) – Payments innovation will unlock the Web’s potential

Osama talked about micropayments and PayPal’s new APIs. PayPal believe that micropayments will change the way the web operates. Their new APIs will make this possible.

Chris Abad (Spymaster) – Advanced Web App Marketing Strategies

Chris talked about the annoying but very successful Spymaster game on Twitter and how they planned it. It is essentially a standard viral marketing technique – make something interesting so people will share links (or better still, just post it to their twitter stream automatically).

Cat Lee (Facebook) – Going global: The Future of Facebook Connect

Facebook showed off the latest extension to Facebook Connect. Now any website can make use of their crowdsourced translation engine to make their sites multi-lingual. While the talk was pretty much a sales pitch for Facebook Connect, the topic was interesting. I’ll be having a play around with this when I get the chance.

Bruce Lawson (Opera) – The Future of HTML5

This was probably one of the most interesting talks from the event. Bruce showed off some of the new aspects of HTML5, explained the new tags, which browsers they work on, their accessibility and their backwards compatibility. There is some fantastic stuff coming out of HTML5, but I can’t help but think it is a slightly backwards step and we could end up with horribly messy code with unclosed tags. That is one of the reasons why I like XML and XHTML. Anyway, HTML5 is not yet ready for production environments, so we will have to see how it is used in real life.

View the notes

Chris Thrope (The Guardian) – How The Guardian is using APIs, Frameworks and Tools to Build a “Mutalised” Newspaper

Chris talked about the Guardian’s efforts to embrace new technology. Unlike some other newspapers, they are embracing the new way of dealing with things. Chris also talked about how they built some of their online applications such as their MP expenses site and their twitter conversations engine. I’m probably going to come back and write a seperate entry about the expenses site shortly, as I had a bit of my own experience in this area. Chris showed off their Guardian DataStore and how people are using it to produce other data and statistics.

View the notes

I’ll be posting my notes from FOWA day 2 tomorrow.