The use of cookies on websites has become an increasingly hot topic over the past few months. A new EU law – specifically The Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2011 (PECR) – requires that websites ask for consent before setting cookies that aren’t “strictly necessary” for the operation of the website. We’ve put together this post to give you an overview of how we are approaching this law on the beta for the Single Domain.

Over that time, we’ve worked to make product recall information more useful and helped to make it easier to reduce cold calls and junk mail. I’ve been able to work with Consumer Focus’ energy team to publish complaints data on energy suppliers, the Wales team to publish performance data for the Royal Mail, and Passenger Focus to help open up rail performance information.

There is still a lot more to be done, however shortly I’m going to be taking a break from Consumer Focus and embarking on a new challenge.

I’m delighted to announce that from the end of this month I’ll be joining GovUK – the team in London working to produce a single website for government.

GovUK has already put together an initial prototype for the website (AlphaGov) – but they are now working to produce a much more in depth second (beta) phase.  Over the next few months, we’ll be putting together a public beta of the main citizen-facing aspects of Gov.uk.  We’ll also be developing a private beta of a shared  ’corporate’ publishing platform (for government departments) and a draft Global Experience Language for all government sites. The Cabinet Office has built up an impressive and extremely talented team to put the site together and I’m honoured to be able to join them.

If you want to know more about the project, take a look at the project’s blog and the initial prototype.

“The law was meant to protect the privacy of people using the Internet. To accomplish this, the EU made over 90% of websites illegal”
- Oliver Emberton, silktide.

Back in May I wrote about the new EU law on cookies that came into effect this year and will be enforced from May 2012.

On Monday, Matt Jukes pointed me to a free ebook (written by Oliver Emberton from silktide) about the law and what it means for websites within the EU.

I think this book is an essential read for anyone who develops or commissions websites. It should also be required reading for those who developed the law.

The more I look into this law, the more stupid & ridiculous it becomes. While I support what it is trying to do, the implementation is probably the worst possible solution. To use a common phrase, it is like using a sledgehammer to crack a nut. However, it’s actually more like using all the sledgehammers in the world plus a few Acme style anvils at the same time to try.

It is worth pointing out that the law doesn’t just effectively ban cookies. It also covers flash cookies, the HTML5 local storage facilities, java storage functions, silverlight and basically anything that stores data at the client end.

If this law is fully enforced, I think it could be the end of any form of innovation in European web development.

Essentially, this law blocks the use of cookies until a user has been asked for (and has given) informed consent. Francis Davey, the fantastic tech legal expert has written an in-depth blog about the changes.

The more I look into it, the more I think it will be an uphill struggle to fully implement this law. So many services use cookies such as embedded Vimeo videos, Facebook, Twitter and even WordPress. The law could make services like Google Analytics completely useless if the have to ask every time they are used. Of course Google is a US company and could potentially argue that EU law doesn’t apply (which would leave EU website owners with bit a problem).

If fully implemented, this law could significantly change the way websites work in Europe, and not necessarily for the better. However, at the same time, it could vastly increase the privacy of website users.

So, given that the law comes into force this week and there are so many problems challenges, what can website owners do? Well, the Information Commissioner (ICO) has published some useful guidance (pdf) which you should read. Luckily, they aren’t expecting full compliance just yet!

In the public sector, we’ve been given some temporary guidance on some small steps we can take. This includes auditing our use of cookies and producing a list of what they are, what they are used for and how long they last. This list should be published on the site along with the website privacy policy.

I produced a first version for work’s Recalled Products website yesterday and I’m working through the others over the next few days. If you want to copy / reuse / alter my example, go right ahead. I’d also appreciate any comments on how this could be improved.

Hopefully over the next few months, the ICO & government will put together some more comprehensive guidance on how website owners can move towards full compliance. For now though, start producing these lists of cookies!

If you dig a little deeper into the figures you can see that central government has spent £14m on Strategy & Planning, £15.8m on Design & Build, £23.8m on Hosting & Infrastructure and £9.7m on Testing (these are categories that COI requires under their TG128 standard). Despite their efforts (gathering this kind of data is by no means easy) I have concerns about the quality. For example:

• The Ministry of Defence spent £418,000 on Design & Build: however, that includes figures for their internal intranet whereas the others do not appear to include intranet costs
• A comment against the Forestry Commission figures numbers states “This is as close as I can make it. Won’t be too far out.”
• The Office of Fair Trading hosting costs cover both their website and internal intranet
• Numerous departments (Ministry of Justice, Audit Commission, Department for Environment, Food & Rural Affairs, GCHQ, Crown Prosecution Service and the Statistics Authority) show no costs for website hosting

Quite clearly there is a discrepancy in the ways each department has calculated their figures. There could be quite legitimate reasons for this, for example, hosting could be part of a much bigger IT contract, making it difficult to split out website costs. However it does raise questions as to the accuracy of the dataset as a whole.

Interestingly, the Crown Prosecution Service does not appear to have provided any costs (all fields are marked as zero); they have however answered an FOI request recently, revealing they paid nearly £122k last year.

That said the spending figure is horrifying. The government is spending way too much on websites. I can’t quite figure out how spending £35 million on BusinessLink is justified, or £10.4 million on Directgov for that matter, particularly when these sites were supposed to reduce the cost of public sector websites.

I would also be interested to see a breakdown of where all of this money is spent. I wonder how much of it goes to the big public sector outsourcing companies rather than SME web development agencies.

I agree that the costs need to be brought down, and some of that needs to be through closing websites and rationalising hosting infrastructures. However I think that there needs to be a proper look at why the costs for producing websites are so high – value for money is just not being achieved in a lot of these cases.

Simon Dickson from Puffbox has written more about these figures on his blog, as has Harry Metcalfe from the Dextrous Web.

There are lots of different opt-out services that exist, all of which have separate sign-up forms and require you to fill out the same details (such as name, address, email address), over and over again. We felt there was a better solution, and so we built a site that combines the forms into one and does the hard work of submissions for you. The initial version of StayPrivate.org incorporates the Telephone, Mail & Baby Mail Preference service offered by the Direct Marketing Association.

I’ve been working on this project on and off for the past seven months (although most of the work has happened in the last two months or so), so I’m really pleased to see it launch. Since we flipped the switch early on Thursday morning, we’ve had over 4500 registrations and submitted over 6500 applications to the DMA opt-out services. This is many more than we were originally anticipating for the first public release.

The day of launch was amazing. We had to respond to issues quickly – sometimes within a few minutes. We pushed out nearly 30 fixes through out the day and switched off parts of the site to reduce server load. Since then we’ve been reducing the queues of submissions to ensure that everyone who registered has been processed.

The next task is to go through the site and look at where we can make improvements. I-ll be spending the next few days reworking parts of the site and implementing some of the suggestions we-ve received.

If you want to know more about StayPrivate or want to keep track of my work on this project, take a look at the CF Labs blog.

Update: fixing a spelling mistake pointed out by @keeran.

Of course, with a new government brings a new direction. Steph Gray, a former civil servant working in the technology arena has written a fantastic overview of what might happen to public sector IT over the next parliament. While these potential plans seem to focus on large scale IT projects, we’ve already started to see the effects of the new government as it has attempted to make its mark.

Within hours of David Cameron stepping into Number 10, public sector websites underwent a huge change. Most policy generated for the previous Government was cleared from department websites and consigned to The National Archives, Twitter and social media accounts were updated, and in some cases entire websites were replaced.

DEFRA (Department for Environment, Food and Rural Affairs) launched a new, temporary website to differentiate between new and old policy. Department for Business, Innovation and Skills (BIS) on the other hand have left most of their policy in place, albeit with a warning stating that it relates to the previous government.

The Department for Culture, Media and Sport (DCMS) website saw one of the biggest changes with a fantastic new design. The Department for Children, Schools and Families (DCSF) also got a new website, but this time under the new name “Department for Education“.

Both of these departments now have brand new Twitter accounts – @dcms and @educationgovuk. Unfortunately, they both already had accounts prior to 12th May – raising questions as to why they needed new ones. The old DCMS account (@dcmsgovuk) still lives on, dormant and not updated. No one appears to have bothered to inform their existing followers of the presence of the new feed. The DCSF account (@dcsfgovuk) has suffered an even more drastic fate. An account which at one point had some 4000 followers was deleted along with all of its content.

Other departments handled their changes a bit better – Downing Street rebranded all of their social media accounts to “Number10Gov” for consistency. Their twitter account was renamed, leaving all of their followers and previous content intact. This, in my opinion, is a much better way of handling the change.

In general these immediate changes have been a mixed bag. Some have worked well – such as the changes to Number 10 and the new DCMS website. Others appear to have not been thought through – such as the new account for DCSF. Unfortunately however, there has been no consistency between departments or government as a whole.

This of course has been the first major change in government since the rapid growth of social media platforms, so quite clearly there was going to be a bit of trial and error. Hopefully next time the public sector will be a bit wiser when it comes to social media and not leave their followers hanging or disconnected.

The Guardian (and others) produced their own websites that did a similar job and generally worked better. Since then, there have been further developments including the Legg report and further detail released (twice).

This means that the data added to WhatTheyClaimed.com has become out of date and incomplete. This presents a problem as it may confuse and mislead visitors to the site.

As a result, I’ve decided to turn off editing access to the site and plaster a “mothballed” sign on every page. I’m probably going to archive the content shortly and take the site down entirely. Thank you to everyone who contributed to the website – it was fun.

On Friday, Charlie and I attended a hack day with a difference. The ScraperWiki Hack and Hackers Hack Day was designed to bring designers, developers (hackers) and journalists (hacks) together to see what they could do. While other hack days may have involved journalists in a media capacity, this event involved journalists from the beginning.

After an introduction to how the day would work, we all split out into groups and started to work on our projects. I worked with Charlie, two other journalists (Ian and Annamarie) and another developer (Edmund) to investigate new election candidates in the safest seats in the country. We mashed together data on constituencies, candidates and then did some good old fashioned journalism to fill in the blanks

Other teams worked away on other projects, only interrupted by the food and drink provided. Finally, at the end of the day we sat down to present our work – eight projects in total. It was amazing to see what different directions the different teams had gone in.

A few of the projects used data to begin investigating a story. One group looked at the transport data released by data.gov.uk and mapped it onto constituencies; another group looked at awards paid out by Enterprise Ireland. Richard Pope looked at the register of interests for London Assembly members and plotted how often and where they were taken out to lunch by third parties. The largest group produced “They Write For You“, a mashup that demonstrates how many MPs write articles in newspapers.

Other projects were more focused towards building tools such as a system that allowed you to classify the output of data.gov.uk data (e.g. Excel, CSV, PDF, linked data etc.). A last minute entry also wrote a scraper for UK bank holiday dates (a much needed tool for developers!). A final fun project scraped all of the horoscopes for the last year and analysed the most used words for each star sign -producing some fantastic visualisations.

The day aimed to explore how hacks and hackers could / would work together. I think the quality of the projects created show that it is a good match. Journalists have an eye for a story that us developers might not have. While we may enjoy throwing tools together and mashing up data, we don’t necessarily know what makes a good story. Similarly, journalists don’t necessarily know or understand what developers do and how we do it.

Lessons were learned on both sides and show the growing importance of data based journalism. As an experiment, I think it was a success and certainly should be repeated again.

If it is – I will be first on the list to attend!

Thanks must go to Channel 4/4IP for providing the venue for the day, Richard Pope and everyone from ScraperWiki for organising the event and helping us get working.

The past 36 hours haven’t been the best for those travelling with Eurostar. Five trains were stranded for hours in the Channel Tunnel without power, light, food, drink, heating or information. A further train became stuck near Ebbsfleet yesterday evening.

The focus today is on the cause of the incident and the safety procedures. However the issue that concerns me is how public service companies like Eurostar handle communication in such emergencies. Poor communication can lead to frustration, anger and even panic. I’ve experienced this in much less drastic circumstances when commuting to and from Cardiff by train.

Today the media are reporting that Eurostar passengers were given no information about what was going on. People at stations had similar experiences. Even the public and media had no idea of the unfolding events until late on Saturday morning.

It seems a standard practice for UK transport companies to ignore incidents, keep quiet, disappear and leave customers to their own devices. It doesn’t take much to keep people informed – even if there isn’t any “useful” information to give them at the time. Even a simple “The track ahead is flooded; we are awaiting an engineer to confirm we can proceed” provides re-assurance that something is happening.

There are three distinct audiences in any incident, each with different information needs. Those directly involved (i.e. train passengers) who need to know when they will reach their destination. Those affected through waiting to travel or waiting to meet passengers, and everyone else (for example the media and general public).

Open communications to all three groups is key, but how can it be done effectively? Which channels should be used? Well, every possible channel actually. If you are preparing a message it doesn’t take much more effort to push this out on all available options

By example, recently I’ve read reports that Eurostar didn’t start updating Twitter until 14 hours after the incident began. Some have suggested that Twitters low usage justifies it being left until last. But for passengers on trains with phones and computers, this may be the only way that information can be easily received. It only takes one person with access to disseminate the details.

Communication (or the lack of) is a big problem when it comes to any form of crisis. Yet the prevalence of new social media communication actually makes communication easier than ever. Businesses of all kind need to recognise this and plan to make best use of it. I hope that Eurostar and those providing public services learn the communication lessons of this weekend.

On a final note – Eurostar’s social media company behind (We Are Social) has come in for what I believe is unjustified criticism. They actually did a great job given difficult circumstances and went well beyond the requirements of any contractor. The poor communication lies squarely with Eurostar. You can read about “We Are Social’s” involvement on their blog and a interesting response from Grapevine Consulting about the criticism levelled on We Are Social.