The use of cookies on websites has become an increasingly hot topic over the past few months. A new EU law – specifically The Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2011 (PECR) – requires that websites ask for consent before setting cookies that aren’t “strictly necessary” for the operation of the website. We’ve put together this post to give you an overview of how we are approaching this law on the beta for the Single Domain.

Over that time, we’ve worked to make product recall information more useful and helped to make it easier to reduce cold calls and junk mail. I’ve been able to work with Consumer Focus’ energy team to publish complaints data on energy suppliers, the Wales team to publish performance data for the Royal Mail, and Passenger Focus to help open up rail performance information.

There is still a lot more to be done, however shortly I’m going to be taking a break from Consumer Focus and embarking on a new challenge.

I’m delighted to announce that from the end of this month I’ll be joining GovUK – the team in London working to produce a single website for government.

GovUK has already put together an initial prototype for the website (AlphaGov) – but they are now working to produce a much more in depth second (beta) phase.  Over the next few months, we’ll be putting together a public beta of the main citizen-facing aspects of Gov.uk.  We’ll also be developing a private beta of a shared  ’corporate’ publishing platform (for government departments) and a draft Global Experience Language for all government sites. The Cabinet Office has built up an impressive and extremely talented team to put the site together and I’m honoured to be able to join them.

If you want to know more about the project, take a look at the project’s blog and the initial prototype.

“The law was meant to protect the privacy of people using the Internet. To accomplish this, the EU made over 90% of websites illegal”
- Oliver Emberton, silktide.

Back in May I wrote about the new EU law on cookies that came into effect this year and will be enforced from May 2012.

On Monday, Matt Jukes pointed me to a free ebook (written by Oliver Emberton from silktide) about the law and what it means for websites within the EU.

I think this book is an essential read for anyone who develops or commissions websites. It should also be required reading for those who developed the law.

The more I look into this law, the more stupid & ridiculous it becomes. While I support what it is trying to do, the implementation is probably the worst possible solution. To use a common phrase, it is like using a sledgehammer to crack a nut. However, it’s actually more like using all the sledgehammers in the world plus a few Acme style anvils at the same time to try.

It is worth pointing out that the law doesn’t just effectively ban cookies. It also covers flash cookies, the HTML5 local storage facilities, java storage functions, silverlight and basically anything that stores data at the client end.

If this law is fully enforced, I think it could be the end of any form of innovation in European web development.

I’ve been a regular user of trains for nearly 10 years.  When I was in college, I had to use the train to get to classes.  Before I moved to Cardiff in December, I commuted to work by train every day.  Now, I travel by train to meetings in London on a pretty regular basis.  I like to think that I’m a bit of an expert at train travel.

One of the things that always intrigued me is the rail performance figures that are published every few weeks. Somewhere in a secluded corner of a station will be a poster, sometimes titled “Passenger Charter Figures”, that is supposed to tell you how many trains run ‘on time’.

Normally (at least every time I’ve ever looked), the train company seems to have achieved its targets.  I always thought this was quite strange given the number of times my train was delayed or didn’t show up at all.

It turns out that there are numerous loopholes and tricks* they can use to make their figures look good.  For example, on time doesn’t mean on time, it actually means the train arrives at its destination no more than 5 minutes late (10 minutes for intercity travel).  Note the use of ‘destination’.  The train could depart 30 minutes late and drop me off at my stop 25 minutes late; but as long as it reaches its destination less than 5 minutes late (which for my journey could be nearly 5 hours later), the train is classed as ‘on time’.  Timetables have lots of padding built in which helps with this.

Luckily, last year I got to work on a joint project with Passenger Focus (the rail passenger watchdog) on ways of making these figures more useful.  We produced a basic HTML prototype to show how more detailed performance figures could be used by the public to hold train companies to account.

Unfortunately, even though much more detailed performance information is collected (the time a train passes each monitoring point on the network is recorded by Network Rail), it isn’t published, let alone open for re-use.  It’s generally considered ‘commercially sensitive’ – which is ridiculous.  Even if we wanted to use the, in my view, crap existing figures we wouldn’t be able to – it isn’t open.

We demoed our prototype and our ideas to a few people including the Office for Rail Regulation (ORR) – the public body that sets the terms of rail company operator licences – in the hope that they would either force the rail companies (or Network Rail) to open up the data.

We are, of course, not the only ones who have been trying to get this data opened up. Many other individuals and groups have been campaigning to the plethora organisations involved. Some of them have been campaigning for a lot longer than we have.

Last week the government announced a huge number of public datasets to open up.  Amongst these, two important elements stand out:

• Office of Rail Regulation to increase the amount of data published relating to service performance and complaints by May 2012
• Rail timetable information to be published weekly by National Rail from December 2011.

Obviously, everyone involved in the various campaigns have managed to get the issue to register on the radar of ORR!  Publishing this data is a huge & welcome step forward.  Everyone involved deserves a pat on the back.

But, we’re not there yet. We now need to make sure that the published performance data is detailed enough to be useful to the public.  Then, once it’s published, we need to make sure there are tools available that use it, so that the public can finally start to hold their train companies to account.

* the train companies would say that their figures are independently audited and there are no tricks and loopholes.

Photo Credit: AndrewHA (Flickr CC)

Edit: added a few links.

Essentially, this law blocks the use of cookies until a user has been asked for (and has given) informed consent. Francis Davey, the fantastic tech legal expert has written an in-depth blog about the changes.

The more I look into it, the more I think it will be an uphill struggle to fully implement this law. So many services use cookies such as embedded Vimeo videos, Facebook, Twitter and even WordPress. The law could make services like Google Analytics completely useless if the have to ask every time they are used. Of course Google is a US company and could potentially argue that EU law doesn’t apply (which would leave EU website owners with bit a problem).

If fully implemented, this law could significantly change the way websites work in Europe, and not necessarily for the better. However, at the same time, it could vastly increase the privacy of website users.

So, given that the law comes into force this week and there are so many problems challenges, what can website owners do? Well, the Information Commissioner (ICO) has published some useful guidance (pdf) which you should read. Luckily, they aren’t expecting full compliance just yet!

In the public sector, we’ve been given some temporary guidance on some small steps we can take. This includes auditing our use of cookies and producing a list of what they are, what they are used for and how long they last. This list should be published on the site along with the website privacy policy.

I produced a first version for work’s Recalled Products website yesterday and I’m working through the others over the next few days. If you want to copy / reuse / alter my example, go right ahead. I’d also appreciate any comments on how this could be improved.

Hopefully over the next few months, the ICO & government will put together some more comprehensive guidance on how website owners can move towards full compliance. For now though, start producing these lists of cookies!

December 2011

In early December we launched a directory of post offices owned by and operated for communities across the UK.  Consumer Focus commissioned the Plunkett Foundation and Social Enterprise Coalition to compile the data which CF Labs then presented through this site.

The site allows users to search for community post offices in their area, find out where they are located and see what services they provide.  The site also provides some statistics on core activities and the data is exposed in JSON format.

A few days later we launched a service to publish ratings of how energy companies handle complaints.  We’d worked closely with our colleagues in the Energy team at Consumer Focus to design and develop a meaningful way of showing this data.  We produced a number of prototypes which were refined through several stages to a shortlist of 3 possible options.  These were then independently tested by TNS-BMRB in September which helped us to pick the final option.

The system ranks each of the ‘Big Six’ energy suppliers using stars – 0 for the worst performing companies, 5 for the best.  The site publishes a snapshot of the data (the last quarter) and also a more detailed historical view.  All the data is also published using JSON and CSV formats.

October 2010

In October we completed the second phase of the new Consumer Focus website.  Earlier in the year we made the move from an external proprietary content management system to a WordPress system managed in house by CF Labs.  The second phase of work involved a complete restructure of the content and a refreshed website template.

As with the new CF Labs in September, we used WordPress 3.0 and built new templates for each of the Consumer Focus Nations (GB, Wales, Scotland & Northern Ireland).  New areas were created for advice, policy work and publications.

We launched the site just in time to hear rumours that Consumer Focus was to be abolished in the Government’s “bonfire of the quangos”.  It turned out that the rumours were true.  The intention is to transfer some of our functions to Citizens Advice and Citizens Advice Scotland.  Even now, 4 months later, we still don’t know exactly when or how this transfer will happen.

November 2010

November was a quiet month – mostly spent finalising work on three projects that were due to launch before the year came to a close.

At the end of the month we launched the second phase of the Consumer Focus Energy Price Comparison (EPC) Service.  The EPC acts as a mini-price comparison system and gives you a rough idea of the price to expect when shopping around for an energy supplier.  Consumer Focus inherited this system from our predecessor Energywatch.  Earlier in 2010 we’d shifted the system to a new data supplier and rebuilt it so we could manage it in house to save money.

The second phase of development involved producing a brand new front end to make the system easier to use.  We gave it the CF Labs treatment and integrated it with the Consumer Focus website design.  We cleaned up the code, added a bunch of features and created bookmarkable URLs for every search.

Coming up…

• Community post offices
• Rating energy suppliers

As this recap covers such a long period of time, I’ve split it across several posts. First up is July, August & September.

July 2010

Throughout July I spent time working on the recently launched StayPrivate.org website.  As users sent in feedback we evaluated and tweaked the interface to reduce problems.

August 2010

As with July, plenty of time was spent evaluating and implementing changes to StayPrivate.org following user feedback.  Late in the month we blogged about some of the issues users were having and the reasons behind them.  We also launched a revamped FAQ to answer as many of the common questions as possible.

A large number of other projects also took up time during August –all of which launched before the end of 2010.

September 2010

In early September we launched a new website for CF Labs to showcase our work and host our blog.  The existing website included a heavy focus on ideas for potential future projects but we decided to change the focus to projects that had already been completed.

As the CF Labs site was built using WordPress, we developed a new template to take advantage of all the new features added into WordPress 3.0.  We used Custom Post Types to provide a page for each of our projects which includes an introduction to the work and associated images. We also used tags to show the latest three blog posts related to each project.

Also in September I stumbled across a fantastic open-source tool (Tweet Nest) for creating a searchable archive of a user’s tweets.  In the past, I’d used various methods to create such an archive but none of them worked as well as this.  You can see a working example using my tweets at http://tweetnest.daibach.co.uk.

I’m considering using this tool to create an archive of tweets from the various Consumer Focus twitter accounts.

Coming up…

• Launch of new Consumer Focus website
• Consumer Focus is abolished
• Comprehensive Spending Review
• Comparing energy prices

I’m disappointed. Over the past few months a concerted media campaign against public sector workers has been building. We are continually portrayed as being lazy and overpaid, not only by the ‘old’ media (by which I mean the traditional print media in general) but also by MPs and our ‘bosses’ in Government. An image has been created of thousands of staff sat around twiddling their thumbs while earning footballer-style fat salaries, not to mention huge benefits, fabulous working conditions and cushy pensions to boot – all at the taxpayer’s expense.

This image couldn’t be further from the truth. Pretty much every public sector employee I’ve met is hard working and passionate about the work they do – you’d have to be to put up with everything that gets thrown at us. Many are prepared to put in extra effort and spend their own time furthering causes such as open government data. Now don’t get me wrong, I’m sure there are some who aren’t as dedicated, but these will be in the minority.

As public sector workers, the media and some of the general public seem to think that we should additionally be super-heroes. We should be perfect, always motivated, and never have a view on anything. Unfortunately this is just not the case, and despite what some people might like you to think, we are human. Sometimes we make mistakes, sometimes we have trouble getting motivated, and sometimes we express ourselves – both in private and in public.

This is particularly the case when it comes to the cuts following the budget in June, the “bonfire of the quangos” and the Comprehensive Spending Review in October. Many of us have an axe hovering over our heads, yet we’re expected to continue as if nothing is happening. My employer, Consumer Focus, has been abolished. This means that at some point in the future (when isn’t yet confirmed), my job will cease to exist and I will be made redundant. Despite this, I’m expected to remain motivated and continue as normal.

This weekend, “journalists” from both the Daily Mail and the Independent have written articles* about Ms Sarah Baskerville (aka @baskers), a civil servant working for the Department for Transport. They have taken excerpts from her twitter stream to suggest that she drinks, turns up to work with hangovers, criticises government policy and is wasting public money.

I’ve never met Sarah, but I have followed her on twitter for a while and I don’t think I can articulate a better defence than has already been written by numerous others. This is a disgraceful personal attack on someone who has put a huge amount of effort into making government more open and transparent. Is it really that unusual to express boredom while attending a training course? – I’m sure we’ve all been there. Is it really wrong for someone to have a few drinks after work? – I’m sure journalists never do this. Is it really that bad to express dismay at so many people you know and work with losing their jobs?

I’m disappointed at these continued and demoralising attacks on the public sector and I’m concerned. I’m concerned that the Daily Mail and the Independent’s actions will lead to unwillingness from other public sector workers to express themselves online – which will mean a more closed and less transparent government.

I hope that the rest of the media will disown the actions of these two “journalists” and I hope that Sarah doesn’t have too much of a hard time at work in the morning.

* I don’t intend to link to either of these articles, but they can be easily found through your preferred search engine.

If you dig a little deeper into the figures you can see that central government has spent £14m on Strategy & Planning, £15.8m on Design & Build, £23.8m on Hosting & Infrastructure and £9.7m on Testing (these are categories that COI requires under their TG128 standard). Despite their efforts (gathering this kind of data is by no means easy) I have concerns about the quality. For example:

• The Ministry of Defence spent £418,000 on Design & Build: however, that includes figures for their internal intranet whereas the others do not appear to include intranet costs
• A comment against the Forestry Commission figures numbers states “This is as close as I can make it. Won’t be too far out.”
• The Office of Fair Trading hosting costs cover both their website and internal intranet
• Numerous departments (Ministry of Justice, Audit Commission, Department for Environment, Food & Rural Affairs, GCHQ, Crown Prosecution Service and the Statistics Authority) show no costs for website hosting

Quite clearly there is a discrepancy in the ways each department has calculated their figures. There could be quite legitimate reasons for this, for example, hosting could be part of a much bigger IT contract, making it difficult to split out website costs. However it does raise questions as to the accuracy of the dataset as a whole.

Interestingly, the Crown Prosecution Service does not appear to have provided any costs (all fields are marked as zero); they have however answered an FOI request recently, revealing they paid nearly £122k last year.

That said the spending figure is horrifying. The government is spending way too much on websites. I can’t quite figure out how spending £35 million on BusinessLink is justified, or £10.4 million on Directgov for that matter, particularly when these sites were supposed to reduce the cost of public sector websites.

I would also be interested to see a breakdown of where all of this money is spent. I wonder how much of it goes to the big public sector outsourcing companies rather than SME web development agencies.

I agree that the costs need to be brought down, and some of that needs to be through closing websites and rationalising hosting infrastructures. However I think that there needs to be a proper look at why the costs for producing websites are so high – value for money is just not being achieved in a lot of these cases.

Simon Dickson from Puffbox has written more about these figures on his blog, as has Harry Metcalfe from the Dextrous Web.