The Guardian (and others) produced their own websites that did a similar job and generally worked better. Since then, there have been further developments including the Legg report and further detail released (twice).

This means that the data added to WhatTheyClaimed.com has become out of date and incomplete. This presents a problem as it may confuse and mislead visitors to the site.

As a result, I’ve decided to turn off editing access to the site and plaster a ‘mothballed’ sign on every page. I’m probably going to archive the content shortly and take the site down entirely. Thank you to everyone who contributed to the website – it was fun.

Sometimes I come away from the office and struggle to see what we have actually achieved. When you are stuck into day to day work, it is quite easy to lose sight of the bigger picture.

This isn’t helped by the fact that part of our team’s aim does not have a tangible outcome. We spend a sizable portion of our energy on promoting our open principles within the organisation and converting people to our way of thinking. It isn’t an easy task, as I’m sure other people in this field will confirm but I’m starting to feel like we are making some headway.

The first report (of which I hope there will be many) has just gone online, in a format allowing the public to comment on each part of the document. The Consumer Focus website is now powered using the open source Wordpress Content Management System (joining the likes of bis.gov.uk and number10.gov.uk). Other developments are also in the pipeline for the next few months and all have stemmed from us ’spreading the gospel’.

This sort of work is going on across Government on many different levels, but recently there seems to have been a bit of a breakthrough. There is lots of things happening at the moment which you might find interesting. I’m going to briefly run through them.

Data.gov.uk

This is the UK version of the USA’s Data.gov and has been operating in a restricted beta for a few months. Government departments have been posting raw datasets for developers to use for free in their open projects. From sometime in January, the site will go live with over 1100 datasets from central government.

Ordnance Survey, NHS Data, Office of National Statistics Data

Work is underway to open up more datasets and make them freely available. These include health data from NHS Choices, mapping data from Ordnance Survey (OS) and statistics from the Office of National Statistics. Consultations are planned for early in the new year with aims to open them up by April 2010. There also seems to be some movement on publishing data of how the money allocated in the 2008 Pre-Budget report has been spent (possibly by spring, with further data released in the summer).

More information about the data releases can be found in the Government’s recent strategy ‘Putting the Frontline First: Smarter Government‘ and on the Government’s Digital Engagement blog.

Crime Maps

The various Police forces around the UK have released crime statistics using their new mapping service. I’ve had a bit of a play around and it looks pretty good. You can see lots of information about your local area and download the data in CSV format.

More information about the Crime Maps can be found on the Directgov website.

Post Code Data

A few months ago I wrote about the issue of post codes after ErnestMarples.com was forced to shut down by the Royal Mail. Since then, a campaign has been ongoing to set the post code free. Well, it looks like someone might have listened. As part of the opening up of OS data above, a consultation will be taking place shortly on the opening up of post code datasets, which could see the data released by April 2010.

More information about the release of post code data can be found on the Ernest Marples blog.

Digital Economy Bill

The Digital Economy Bill has recently been introduced to Parliament and is due to go to committee stage shortly. This bill contains a large chunk of the actions and recommendations from the Digital Britain Report earlier in the year (with a number of strange and scary additions). There is a lot of talk about this at the moment and the effect it will have on the digital life and innovation in the UK. You can follow the progress of the bill on the Parliament.UK website, or on twitter using the #debill hashtag. If you want more information on the bill and how it might affect us, there are a large number of blogs and websites dedicated to the issue.

Directgov

This brings me to one of the last bits for this entry, and I make no apologies for the shameless self-promotion here.

Direct.gov.uk is the UK Government’s official website for the general public. It aims to collect all the public facing information into one place and make it easy to find. Unfortunately, its usefulness and ability to achieve this aims has been called into question. Personally, I’ve never liked the website and can never find information I want from the site (in fact, I tried several times to find information relating to bits in this post but gave up).

Consumer Focus has recently published a report on the website that analyses how well it delivers on its promise. As I alluded to earlier, this report is available online in fully commentable format. We are encouraging everyone and anyone to send us their thoughts on Directgov, what the problems are and what can be improved. Similarly, if you think it is doing something right, we want to know that too. You can find out more about the report at http://directgov.consumerfocuslabs.org.

I believe there will be a series of workshops planned for early in the new year to identify ways of engaging people in digital public service development. You can sign up on the site to be kept informed (and I’ll no doubt be publishing the information here as well).

Looking back at everything that has been achieved and everything that has happened in the last nine months, I’m actually quite excited at what we can do by this time next year (election permitting of course!). I have my fingers crossed 2010 will continue making leaps towards open government.

The issue surrounds a small piece of text – the Post Code. It sits at the bottom of every address (similar to the Zip code in the USA). It is the holy grail of data in the UK. It maps each place in the country to a physical location. The Royal Mail uses it to assist with routing mail from A to B, developers use it to verify addresses and locate other services nearby. Unfortunately, access to this database costs thousands (approximately £3800 per website per year).

In the majority of cases, small not-for-profit websites or online experiments just can’t afford to pay out for this license. I know from experience – we don’t have a postcode lookup on CllrTweeps for exactly this reason. I can’t afford to pay out the best part of a sixth of my salary (before tax) each year for a postcode licence for a personal project.

Enter ErnestMarples.com – a website that provided access to the postcode database to not for-profit websites for free. A developer could submit a Post Code to the API and get the appropriate longitude and latitude back in return. Although the legality of the site was questionable, it was used by a large number of sites similar to CllrTweeps. These included PlanningAlerts.com, JobCentreProPlus, TheStraightChoice, HealthWhere and so on.

Planning Alerts used it to good effect. It allowed you to sign up for email alerts of planning applications that were submitted near you. How did it work this out? By mapping your Post Code to a physical location. The site was so successful that other sites have cropped up that disseminate its data through other communication channels.

Unfortunately yesterday, the Royal Mail threatened legal action against ErnestMarples and the site was taken down. As of yesterday, all the above sites have gone offline too. Sites of real value to people in the UK have been taken down because of the Royal Mail’s actions.

Now some people will argue that they shouldn’t have been using a dubious source for their data. But as I’ve explained before, they don’t really have much choice. It is a case of using a dodgy dubious source, or not building the site at all. None of these sites were run on a for-profit basis. None of them had a spare £4000 lying around to buy the license.

The Royal Mail claims that they were losing money because of the site (although they have yet to answer enquiries about how much they’ve actually lost). I’m sceptical about this – none of these sites could have afforded to buy the data anyway, so how they could have lost money I do not know!

Lets not forget that the database was created using public money through taxes paid for by the people of the UK. This is public data that is of benefit to the whole of the UK.

By closing down this site and refusing to provide a not-for-profit licence for their Post Code database, the Royal Mail is stifling innovation in the UK.

Last week I attended the Future of Web Apps conference in London. I was amazed at how behind we are in the UK compared to the USA when it comes to innovation. I’m now convinced it is partly due to the failure of organisations like the Royal Mail and Ordnance Survey to modernise. They are acting in the same way as the music industry and trying to stop innovation rather than cultivate a mutually beneficial environment.

I’m sorry to see PlanningAlerts and The Straight Choice die. But maybe it will be the kick up the backside that the UK needs to get its house in order.

Until the Royal Mail releases its Post Code database with a free (or very very cheap) not-for-profit license, we will never have a Digital Britain.

There has been a lot of talk on Twitter today about the launch of the new Birmingham City Council website.  Without a doubt, Birmingham have successfully launched one of the most delayed and expensive websites of the last few years.  After being delayed for 6 months (original launch date was expected to be March 2009), and going significantly over budget (costing a total of £2.8 million), you’d expect the site to be the best thing ever developed.

Unfortunately it isn’t.  The site is riddled with problems; suffers from poor build quality, lack of thinking, and technology that wouldn’t be out of place in the early part of this century.  Here are just a few of the problems I’ve come across after browsing the site for a little over 10 minutes:

• Absolutely no thought given to SEO
• Frequent broken incoming links
• Incomprehensible alt tags and title attributes for links and images
• Lack of validation on many pages
• No modern technology

In thought I’d take a very brief look at a few of these.

Search Engine Optimisation

There really is no excuse for not using SEO optimised URLs on a public site anymore.  Your URLs should be human readable and understandable.  This helps with not only search engine optimisation, but with your website’s users as well!

Unfortunately this website has stuck with the horrible model of using exceptionally long and completely indistinguishable URLs throughout the system.  Here is an example of one of the pages:

http://www.birmingham.gov.uk/cs/Satellite?c=Page&childpagename=Highways-and-Maintenance%2FPageLayout&cid=1223092719662&pagename=BCC%2FCommon%2FWrapper%2FInlineWrapper

I count this URL at 166 characters long – a simplified human readable URL for this site could be as low as 47 characters long (http://www.birmingham.gov.uk/transport/highways)

From the first URL, I have no idea where that page really goes; the second URL is much more obvious.

Broken Incoming Links

Almost all incoming links to the website have been broken with this re-launch.  Even the Google Sitelinks which appear when you search for Birmingham City Council no longer work.  This is not only bad for search engine optimisation, but any bookmarks or links to content on the site will no longer work.

Incomprehensible Alt & Title Attributes

This is probably the most disturbing part of this website.  While looking through the website, I’ve come across title tags such as “BNE /cs/Satellite/v?packedargs=AlphabetValue%3DA&rendermode=live”, “TLE contactus” and “BNE copy”.  These links go to the A-Z website index for the letter A, a contact us page, and the copyright statement respectively.  The first is quite obviously the most incomprehensible, but I’d love to know what the TLE and BNE stand for in the others.

Title attributes are really important – these are used by screen readers (the sorts of things used by those with no or poor visibility) to identify where a link goes.  Without a descriptive title tag, you have no idea where you are going.

Alt attributes are equally important – these describe the content of an image if the viewer can’t see it.  There are quite a few images around the site with no, or incomprehensible alt attributes.

Like the SEO URLs, there is absolutely no excuse for this – in fact, its purely negligent.

Lack of Modern Technology

This is less important, but I felt it had to be said.  There are no RSS feeds whatsoever – who builds a website with no RSS feeds?

Overview

This website cost a lot – significantly more than any other website I’ve come across.  It is badly built, suffers from accessibility issues, and it generally pretty poor.  To whoever signed off this development – you should be taken out and shot.  To the people of Birmingham – I’m sorry, but you’ve been had.  Your money has been wasted, and you should hold your council to account for this farce at the next election.

You can follow more discussions on this redesign on the Birmingham Post website, and by following the Twitter hash tags: #bccwebsite, #bcc and #fail.

This evening’s programme “investigated” the water quality of beaches in the UK – in particular looking at problems caused by sewage.  The reporter looked at the issue of “combined sewer overflows (CSOs)”.  These are essentially overflow pipes for the sewage system (similar to the overflow pipe in your sink) that releases pressure when there is no more capacity for water.  Instead of a pipe sticking out of your bathroom wall, these overflow pipes run from the sewage treatment works to a nearby watercourse (a river, stream or the sea).  These CSOs are designed to operate very rarely – approximately 10 times a year (at maximum) – usually after a heavy rainfall event.

Unfortunately, the cases highlighted tonight have clearly been discharging sewage significantly more often than this.  However, rather than looking at the cause of the issue, the programme instead decided to spend its time bashing the water companies, Environment Agency and local councils.  All while scaring people into thinking that 1 in 20 are going to get ill by swimming in the water contaminated with sewage.

Over the past few years, we have increasingly had very wet summers.  This has contributed to more and more discharges from the CSOs.  But why has this happened? Why are the sewage systems becoming overwhelmed more often?  What big change has happened over the last 10/20 years that might have caused this?  A lot of people will blame climate change, but it’s much more than that.

Tarmac, concrete and paving.  Over the last 10 years, our towns and cities have seen more of their green space covered in tarmac and concrete, people have paved over their driveways and gardens making it more difficult for rainwater to permeate into the ground.  Living spaces have been expanded with extensions and conservatories, reducing gardens.  Where water would previously have been absorbed by soil, it is now blocked by impermeable surfaces.  Where does all this water go? Straight down the drain and into the sewer.  In addition to this, lazy building methods and poor planning inspections have lead to guttering being connected to the sewer, rather than a soak-away which allows the rainwater to drain more naturally –and not exacerbate the problem every time it rains heavily.

When we have rainstorms, the sewers are filled to the brim with rainwater – completely overwhelming a system which was not designed for this purpose.  The end result is a discharge of rainwater and sewage into the sea through the CSOs.

So how can we reduce this problem (another topic not covered by tonight’s programme)?  If you have paved, tarmaced or concreted over your driveway/garden – rip it up.  Replace it with something more permeable like gravel or a permeable paving solution.  If your guttering is connected to the sewer, build a soak-away to allow the water to be absorbed naturally, or consider a Rainwater Harvesting system.  Also take a look at Sustainable Drainage Systems (SuDS) – there is lots of information on Wikipedia and SuDSWales.com.

By implementing these measures, we can significantly reduce the amount of water that ends up in the sewer – thus reducing the number of CSOs that happen following rainstorms.

You can rewatch the programme for 7 days on the BBC iPlayer.

The website is based upon a system I built a few weeks ago to monitor our own expenses at Consumer Focus Labs . The site was designed to match the processes at Consumer Focus, but when Richard and I realised that MPs expenses were being published, we realised we could make use of the same codebase with a few minor changes.  I need to note at this point that the website was a personal project and not supported by Consumer Focus.

We rushed through these changes on Wednesday afternoon, sorted out some hosting and put the site up. Our initial plan was to get a few interested people to help us convert the data and start loading the information into the site bit by bit.

When the data was published yesterday morning, I realised how much of a mammoth task it was going to be. After a quick discussion, we decided to set up a generic username and password for the admin system so that lots of people could help convert the data.

Once MySociety posted the site on Twitter, everything went crazy. It very quickly became clear that the basic back end was a data entry nightmare and things needed to change. Some changes were quickly scoped out and implemented in between untimely distractions such as this and real work. We removed a number of fields from the forms including location since any location details had been completely redacted from the PDFs. I hacked some of the code to make things work and broke the site a few times in the process.

By 5.30 when I left the office to head home, I was absolutely shattered. I decided it was best to take a step back and take stock before making any further changes. I’d like to thank everyone for their messages of support through the day, and also everyone who sent in useful feedback. I haven’t been able to implement all of them yet, but hopefully a few more will be added over the next few days. I’d also like to thank the Guardian and Financial Times for mentioning the work we’ve done.

So, what lessons can be learned from this exercise?

Firstly and most importantly, plan! We threw the site together assuming the overall format of the data would be similar to the expenses claims made at Consumer Focus. In fact, it was much more complicated, and much more fragmented, with some (almost all!) crucial details missing. We also should have put much more effort into the data entry part of the site. While a basic system might be ok for internal use among a small number of people, it really isn’t appropriate for a crowd sourcing site- particularly when so many things can go wrong.

It is also worth considering that speed isn’t everything. We launched the site at 8.30am on Thursday. The Guardian waited until 3.30pm to launch their site and it was still successful. In hindsight we probably could have waited for a while and got things right first off.

Where next for WhatTheyClaimed.com?

I’m going to try and make some more changes over the next few days, in the hope of making things work a bit better. But in the long term, I don’t have the time to commit to the site. So if anyone is interested in taking over the project and giving it the attention it deserves (and needs!), then get in touch – whattheyclaimed@dafyddvaughan.co.uk. I’ve already had messages from a few people interested in working on the site.

Consumer Focus Labs will be publishing its expenses using the original system shortly – if you are interested in that then keep an eye on our website for the announcement. We’ll also be releasing the code for anyone to use to produce their own expenses websites. We hope that MPs will make good use of it to publish their expenses in the future.

Firstly I need to make a few disclaimers. Obviously the views expressed here are my own and not that of my employer – Consumer Focus (if you want the organisation’s view, you can see it here). I’ve also not read every paragraph in the 240 page report – some parts nearly sent me to sleep, so I skipped past them to some of the more interesting sections.

Overall, I can’t help but feel a little bit disappointed with the report. To me it doesn’t seem to go far enough – although a guaranteed 2Mbps broadband connection to every home in the UK makes a good headline, it’s poor compared to other countries such as South Korea (which is rolling out 1Gbps connections). It also skirts around some issues such as copyright fair use and leaves a lot to chance.

The report outlines 2 phases of broadband development in the UK. The first is the Universal Service Commitment (USC) which aims to provide a base to the whole country; the second is the Next Generation Broadband Network (NGB) which will provide for the future.

Universal Service Commitment (USC)

Digital Britain outlines its intention to provide every home in the UK with a minimum of 2 Mbps broadband by 2012. This will be part funded by the production of the NGB networks, with the rest funded through the underspend in the Digital Switchover budget. Initial thoughts are that this is great – everyone in the UK will have broadband of a ‘satisfactory’ quality. However, things soon start to fall apart.

Countries like South Korea are currently achieving 50 Mbps as a minimum, with some places getting as high as a 1 Gbps connection. When put in this context, 2 Mbps is exceptionally poor. Also, when you delve into a bit more detail, it turns out that this speed is actually optimistic.

Some homes will not be able to get broadband though traditional means, and so the Government is going to rely on mobile networks and satellite to provide the USC in these areas. The report suggests that 2 Mbps is unlikely for these houses, and connections of between 1 and 1.5 Mbps is more accurate. So actually, a 2 Mbps USC isn’t really correct.

Next Generation Broadband (NGB)

This is the next phase in broadband development and aims to provide broadband speeds of around 50 Mbps by using technologies such as Fibre to the Cabinet (FTTC) and Fibre to the Home (FTTH). Digital Britain expects that market investment will fund this deployment and anticipates that it will reach approximately two thirds of the population.

Assuming BT and the mobile networks do reach this target, that leaves about a third of the UK back on the not quite 2 Mbps USC broadband. This quite obviously is not acceptable, so a fund is being set up to provide NGB to this final third (called The Final Third Project).

The Final Third will be funded by a 50p per line per month levy on all fixed lines from next year. Contrary to some reports, this includes residential copper lines (like your BT phone line), business copper, ISDN and cable telephony lines (like those provided by Virgin Media). You will not have to pay this levy for mobile phones as they already pay license fees for network upgrades.

After all of this, it is expected that 90% of the population will be covered by NGB, leaving a poor 10% with USC speeds.

Illegal file sharing

This is of course a big bone of contention among many people. The interim report issued in January suggested the creation of a Digital Rights Agency which would be responsible for tracking illegal file sharers. This final version has moved away from this proposal and aims to stick with an industry lead effort, although supervised by Ofcom the industry regulator.

The report outlines a commitment to reduce illegal file sharing by 70-80%. This is a huge commitment, so they aren’t just leaving it to chance. If the industry effort of writing letters to account holders does not reduce numbers significantly within 12 months, Ofcom will be able to force ISPs to introduce ‘technical measures’. These might include blocking of sites, protocols, ports, bandwidth capping (speed and data transfer), bandwidth shaping and content identification and filtering.

However, for this to happen, Ofcom will need to ask Parliament to legislate.

Copyright fair use

Unfortunately this important issue gets all of a one paragraph mention in the document which makes no commitment to resolve the situation. As a result, it remains illegal to copy music you already own to your mp3 player.

Conclusion

As you can see, it is a bit disappointing. The USC isn’t quite the 2 Mbps they promised, NGB relies on the market to do the right thing, no clear direction on illegal file sharing and no movement at all on fair use.

I’ve not gone though everything in this post; there are a lot of other planned actions including the digital radio switchover, tax relief for video game makers, and the digital public service programme. Maybe next time!

Image courtesy of Media Guardian and Wordle.

Counting started yesterday after Thursday’s vote, and stretched into the early hours of the morning. Although we still have incomplete results (as I write this we are still waiting for Northern Ireland and Scotland), we have a pretty good picture of what has happened. Unfortunately, two parts of the North – Yorkshire and Humberside, and the North West have voted for the BNP, who now have 2 seats and for the first time, a proper platform and funding structure to support them in the future.

It is a dark day for the UK.

I’ve been tweeting the result announcements throughout the count and it was quite comforting to see the response of some of the other people on Twitter. A lot of people are disappointed that this has happened. How did it happen though? Why did they get in?

I received this message from someone I know last night while the results were coming through:

“No one gives a f about politics! Stop boring everyone!”

It is thoughts like this that has allowed the BNP to get into such a prominent position. While I understand that not everyone is as interested in politics as me, some people obviously do not understand how important it is. Politicians are responsible for making decisions that affect everything we do. People should care about who gets in and who doesn’t.

Part of the reason the BNP got in is not necessarily because more people voted for them (in fact, their percentage share only increased by 1.4%, which in reality is not that much), but because people didn’t vote for any of the other parties. In fact, with a turnout of just 35%, a lot of people obviously just didn’t vote at all.

I have a pretty strict viewpoint on not voting – if you don’t vote, you have no right to complain about the state of the country. So to everyone who didn’t vote in this election, you are as much to blame as the people who actually voted for the BNP.

Of course blame must party lie with the politicians as well – if they hadn’t been milking the system, we might not have been in this situation.

I understand how screen readers work and have tried out some myself, but seeing someone use one in a real situation was really humbling.  I can’t even begin to imagine how difficult it is to use a large part of the internet for someone who suffers from a disability that affects their ability to use a computer.

Robin showed off number of web apps that worked well with screen readers such as FixMyStreet.com, a service that allows you to report problems to your local council.  Another site that worked well was Google Maps (in particular the ability to get directions to a specific location).

What really surprised me however was how poor the accessibility was on new Downing Street website.  When the size of the text on the site is scaled up, the design just breaks completely.  For a public service website, this is quite disturbing, and really needs to be addressed ASAP by their web team.

I wonder how many other Government / public service sites do not consider these issues?

This brings up an interesting questions – how do schools inform parents and students that they are going to be closed? Traditionally, schools have informed local radio stations which have spread the news. This however comes with a number of problems – most importantly, not everyone can receive the local radio station. For example, one member of my family goes to Hereford Sixth Form College, which informs BBC Hereford & Worcester, and Wyvern FM of closures. However, here in Abergavenny, we receive neither of these stations.

The college could phone all parents – but with over 1700 students, that would be nearly impossible. Parents ringing the college would freeze up the phone lines. Maybe the schools could use the internet to pass this message? Hereford Sixth Form College does just that – an emergency message has been displayed on the homepage since Monday morning detailing the current status of the college.

What if there was one central resource for information on all schools? One website address for parents to remember – a website which was updated by each school to show their status. Tom Watson, MP for West Bromwich suggested this yesterday morning. He purchased schoolclosures.org.uk and challenged DirectGov (the UK’s government gateway) to produce the site ready for the next day (today).

DirectGov must have pulled out all the stops, because just 28 hours later, schoolclosures.org.uk was released as a beta to show how the system might work. This is fantastic work from a government department – not something you will usually hear from me. Being able to take an idea to a working system in just under 28 hours is no mean feat – but is something the government needs to get used to. Well done to everyone who worked on this!

Of course, no one system is going to solve the problem. But with all systems, schools, and government departments working as one, there is a real chance to make an event like this easier on parents and pupils.