On Tuesday I attended a show and tell to see the work that’s been done over the last sprint. It’s great to see things starting to come together. We also had a general pre-sprint planning catch up and team retrospective.

On Wednesday we started the first round of user testing the online service. We tested with both DVLA staff in Swansea and members of the public in Birmingham. This round has highlighted some things we can change now to make the service simpler and confirmed some suspicions we had. It has also given us some things to investigate further – it’s important not to jump to conclusions from just one round of testing.

The team area now has a TV for us to display various dashboards. It’s taken us a little while to get this, but it’s an important step to help us make what we are doing more visible. I also had some productive conversations around meeting the service standard and using the Service Manual.

Wednesday was also the first time we’ve been able to get a decent portion of the team together out of working hours on a social basis.

As we started work before the service manual was released we now have some stories in our backlog to update our approach to a couple of things such as our look and feel (like using an alpha/beta header instead of the normal GOV.UK header). We’ve also moved some of our work on the data model into the main backlog now that we have a better idea of what we are trying to do.

Finally, Friday was spent enjoying a mixture of Sci-Fi and good food.

As Mike would say – onwards (sorry, couldn’t resist).

I probably won’t get round to writing these every week, but hopefully often enough that it makes sense.

So, here we go with Week #1 (29 Apr – 3 May 2013):

Those that follow me on twitter will probably have noticed that I’m splitting my time between GDS in London and the DVLA in Swansea. While at DVLA I’m assisting a team working on a new service to help drivers find out information about their driving licence. We’re are about half way though the project’s 7th sprint.

Driving entitlement data has been through years of updates and tweaks from both UK and EU legislative changes. This has lead to things becoming a bit complicated inside the existing databases. I’ve been helping the team work out which fields of data are still relevant and which have been superseeded so we can create a cleaner data model.

I’ve also been looking at how some of the internal APIs will be structured, figuring out what UI tweaks we want to make before the first rounds of user testing next week and reviewing the stories for next sprint.

Finally, I’ve been having some conversations about how we can provide evidence of the service meeting the Digital by Default Service Standard. Hopefully more on that next week.

Back at GDS, we’ve been looking at trying out a survey tool as a potential way of doing user testing for one of our projects. I’ve been figuring out what the implications are for our cookies approach and talking to the suppliers involved about them.

On Tuesday the final central government departments moved to GOV.UK. There was, as has become tradition, cake and beer. I’ve also put together a quick visualisation of every code change made to the InsideGov repository on Github since they started work in late 2011.

Finally, yesterday I attended the monthly technical architects workshop to go through the common issues we’re seeing on across the various transformation projects.

There is no explanation or notice explaining why you end up on that homepage and you would be forgiven for thinking that the sites and tools never existed. Every link ever made to any of those things are now broken.

It’s strange to see two years of your professional life just disappear. I can’t really blame Consumer Focus for this – they no longer have anyone with the skills or knowledge to keep the things running (and the organisation will close entirely very soon).

Unfortunately, this happens all too frequently online. Websites are rebuilt, things are moved, stuff is deleted without explanation or warning. Bits of our digital history are just lost forever. The internet is fragile.

Keeping everything operating and the same for the rest of eternity isn’t realistic.

Luckily, however, for UK government organisations The National Archives takes a copy of most government sites every couple of months (there are other similar projects aimed at non-gov sites). Even better, most of the things I worked on are sitting there in The National Archives (unfortunately, you have to know they are there because the redirection goes to the wrong place).

The Government Digital Service is currently migrating a large number of websites to GOV.UK. My colleague Paul Downey wrote about how they are putting lots of effort into stopping broken links and redirecting users as much as possible. Where a similar thing doesn’t exist on GOV.UK, they are notifying users and directing them to a copy in the archive.

From day 1, over 42 thousand URLs from Business Link were redirected. This number has grown continually since, most recently with the migration of the Department of Energy and Climate Change site to GOV.UK. GDS hasn’t always got it right and do occasionally find some URLs they’ve missed, but as soon as they realise, they are being added to the list and redirected properly.

If you turn off a website or radically change a site, it doesn’t take much to make sure it’s in The National Archives and redirect the URLs properly. That way you don’t break any links or bookmarks. Users don’t get lost or end up in dead ends. Redirecting isn’t difficult to do (particularly when just redirecting to the archives), it doesn’t cost much and doesn’t require any maintenance. It’s part of being a good citizen of the web.

Of course, this should just be the way things are done. For government sites, there are/were rules for handling URLs for website rebuilds and closures. This rule has been around for quite a while, but unfortunately tends to get ignored fairly frequently.

More redirects means fewer broken links, a less frustrating web for everyone and makes the web a little less fragile.

It wasn’t really practical to do this on a daily basis, either from a time or cost perspective (4+ hours a day travelling in addition to a normal working day and a monthly season ticket of than £1000). So I travelled down on a Monday morning, stayed in a hotel* (and a friend’s sofa) during the week then travelled back on a Friday evening.

This sort of commuting is not easy.

You don’t have really have a place that is yours during the week – a hotel room doesn’t feel like home since you are going to be moving out of it at the end of the week.

You have to live on hotel food or restaurants for breakfast and dinner every day – very rarely do hotels have kitchen facilities! This means you have to pick your hotel pretty carefully – you really don’t want to end up in a hotel with no decent food provision nearby.

You have to put up with crappy internet – I’d love to know how hotels can justify charging up to £10 a day for wifi access that’s almost as slow as dial-up. Luckily I had unlimited data and tethering on my phone.

Cost – ouch. Unless you are lucky enough for work to contribute to this commute (I wasn’t), paying for rent, travel, weekly hotel costs and eating out every evening isn’t cheap (although it is cheaper than a rail season ticket!)

It isn’t all bad. You have someone who will clean your room every day and you quickly find out how much “stuff” you really need – just the stuff you can carry in a smallish suitcase!

What I’m trying to convey is that this kind of commuting isn’t for everyone. It may be a great short term solution, but isn’t appropriate for anything long term. The travelling and hotels completely wore me out.

In July I gave up the commute, rented a flat and moved properly to London. Since then, it’s been great to have a permanent base in a fantastic location. I have my own space, great internet connection and have much more money to spend on things other than travel & food.

More recently though, I seem to have found myself doing another regular long distance commute – this time in the opposite direction to work with the DVLA in Swansea.

To avoid burning myself out, I’ve set myself a few ground rules which might be useful for other people in similar situations:

1. Don’t go for the most basic of basic hotels. A Travelodge may be cheap, but if you are staying in hotels regularly, you want something a bit more homely.
    
2. Pick a hotel and stick with it. At first I was in a different hotel every week – usually the cheapest I could find. Eventually I settled on one hotel and got to know the staff. They put me in the same room every week which helped it to feel a little more like my space.
    
3. Pick a hotel in a city centre. The cost of hotel food is vastly inflated and (in the case of several popular hotel chains) generally poor quality. Staying in a city centre gives you access to lots of different options for evening meals. Staying in a city centre also gives you the chance to interact with others – otherwise you can easily feel cut off from the world.
    
4. Try to balance your week out more – don’t do 5 days away, 2 days at home, try 4 days away, 3 days at home. I’m trying to do no more than 3 days a week away from home this time round.
    
5. Avoid travelling on Friday evenings – just don’t do it. It’s horrible.

Now, the next challenge is to get the mandatory travel booking system at work to agree to all of these rules!

* You may ask why I stayed in hotels rather than, say, a flat-share or other more medium-term solution? The simple answer is that I didn’t know how long I was going to be doing the commute. Originally, it was only going to be 4 months, but ended up lasting much much longer.

Photo above by mwanasimba on Flickr. Used under Creative Commons.

We do this using a postcode (which we geolocate with help from some open data and the open source MapIt service) and an open database of over 90 thousand URLs collated by councils and the Local Directgov team.

Unfortunately, with such a large database of URLs from such a large number of organisations (there are around 420ish councils in the UK), there are inevitably going to be some that don’t work. This could be because a website has been redone, or content moved around without redirects put in place, or it could just be a mistake when the URL was added to the database.

To help explore this huge dataset and identify problems before people come across a broken user journey, I’ve built a little tool.

This tool takes the Local Directgov URL dataset that we use on GOV.UK and runs various tests to try and work out if each URL works. Firstly, it checks the HTTP status code for the page – if this suggests it’s broken, the tool flags that URL as a problem to be looked at. Secondly, it checks the content of the page to see if it contains phrases like ‘Page not found’ or ‘Error 404’ – this helps identify problems when the website hasn’t been configured to send accurate HTTP status codes (which unfortunately if fairly common).

At the moment, there appear to be approximately 6,600 broken URLs in the dataset (around 9%).

The tool is a little rough around the edges, but it gives a good overview of some of the current issues in the dataset. I’m going to be adding further tests, reports and other features to make it more useful over the next couple of weeks.

While it’s still a work in progress, it’s using live data and is re-checking existing URLs and importing changes to the datasets on a regular basis to provide as accurate a picture as possible.

There are a couple of important limitations and caveats:

• It’s currently only checking the services that exist on GOV.UK – which accounts for something like 73 thousand URLs. The Local Directgov dataset contains another 20 thousand or so that aren’t being checked. You can see the services it’s checking here.
• It’s running on a rather underpowered server, so things might take some time to load
• I’m still adding further checks and reports (including CSV/JSON views) so I’m likely to break things fairly regularly – sorry!

Feel free to take a look and let me know what you think.

On Tuesday afternoon I got to issue the pull request to remove all the beta messages from the site, then a team worked through the early hours of Wednesday morning to redirect as many Directgov and Business Link URLs to GOV.UK as possible*.

On Wednesday, the site received over 1 million visits and regularly had over 10 thousand people using it at the same time. The teams at GDS monitored how the site was working, responded to feedback and made fixes where appropriate.

During the day we deployed a number of bug fixes to production – something that we’ve continued to do since. This is one of our principles at GDS – release early, release often. In the last couple of weeks, we’ve made over 80 releases to various elements of GOV.UK**. Each release could contain changes to multiple apps and multiple fixes.

GOV.UK isn’t perfect and it’s definitely not finished (It shouldn’t ever be considered finished). We’ll be looking at the feedback we’ve received and data we’ve collected to see where we can iterate the site and make improvements.

This is just the first step in what will be a long journey to improve the digital infrastructure of government. There is lots still to do including continuing to improve GOV.UK based on data and transforming existing government transactions to focus on user needs.

It’s taken a huge amount of effort from lots of people to get us where we are now – from developers, designers, delivery managers, product owners and editors to support staff, fact check co-ordinators and policy experts in government departments.

It’s been fantastic to work with an amazing team of talented and passionate people on this project. I’m looking forward to working with them again on the next challenge. After a much needed rest :-)

For now though – let me point you in the direction of my three favourite bits of GOV.UK. Our APIs, what we’ve used to build the site and the ICS/json versions of the dates for when the clocks change.

* We’ve redirected over 42 thousand URLs from Business Link alone. We’ve probably missed some – so let us know if you find something that should have been redirected and hasn’t.

** You can follow our releases to production on Twitter by following the GDS Badger of Deploy (I’ll explain the naming of this account in a future post).

This past week, we’ve released (and written about) the new homepage design, updated browse categories and the removal of the search ‘auto suggest‘ feature.  In addition, we’ve also made a few other, smaller & slightly less noticeable, changes to the site.

Where possible, I’m going to be writing about them and posting them either here or on the GDS dev blog.

The first of these, about the changes we’ve made to guides and 302 redirects has just been published:

The guides format on GOV.UK is generally used for longer, more in-depth content (example). It usually consists of two or more ‘parts’ spread over multiple pages, each of which has its own URL. Until recently, when you visited gov.uk/any-guide, we automatically redirected you (using a 302 redirect) to the URL for the first part, e.g. gov.uk/any-guide/part-1. This meant that if you bookmarked a guide, you’d usually bookmark the first part.

Read the full post.

Since then lots of things have happened, and many things have changed.  Here is a quick summary:

• The Government Digital Service (GDS) was created to bring together many bits of government that were working on similar things, including the project to create the new website
• We released a public beta of our proposed new website for citizens (to replace Directgov). We followed this with a beta of a platform to replace all central government websites and an alpha of our design principles
• I got involved in, wrote and talked about the use of cookies on websites and the new law that has caused a lot of fuss
• I got to go to Number 10
• I left Consumer Focus and joined GDS full time as a developer and tech lead
• I spent a huge amount of time travelling from South Wales to London and staying in hotels
• I eventually stopped this madness and relocated from Cardiff to a flat in East London
• We released the first batch of things for businesses on the public beta
• We announced the date on which GOV.UK will replace both Directgov and BusinessLink – the two biggest government websites.

Over the last year, this blog has been very much unloved and has gathered an unhealthy layer of dust.  It’s not for want of trying – there are several half finished posts that I just haven’t gotten around to finishing.  I’m hoping I can resurrect some of these, but I suspect many are now too out of date.

In the run up to the launch of GOV.UK, I’m hoping to write a bit more about what I’m doing.  If the last year is anything to go by, the next 12 months is going to be a lot of fun.

The use of cookies on websites has become an increasingly hot topic over the past few months. A new EU law – specifically The Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2011 (PECR) – requires that websites ask for consent before setting cookies that aren’t “strictly necessary” for the operation of the website. We’ve put together this post to give you an overview of how we are approaching this law on the beta for the Single Domain.

Over that time, we’ve worked to make product recall information more useful and helped to make it easier to reduce cold calls and junk mail. I’ve been able to work with Consumer Focus’ energy team to publish complaints data on energy suppliers, the Wales team to publish performance data for the Royal Mail, and Passenger Focus to help open up rail performance information.

There is still a lot more to be done, however shortly I’m going to be taking a break from Consumer Focus and embarking on a new challenge.

I’m delighted to announce that from the end of this month I’ll be joining GovUK – the team in London working to produce a single website for government.

GovUK has already put together an initial prototype for the website (AlphaGov) – but they are now working to produce a much more in depth second (beta) phase.  Over the next few months, we’ll be putting together a public beta of the main citizen-facing aspects of Gov.uk.  We’ll also be developing a private beta of a shared  ’corporate’ publishing platform (for government departments) and a draft Global Experience Language for all government sites. The Cabinet Office has built up an impressive and extremely talented team to put the site together and I’m honoured to be able to join them.

If you want to know more about the project, take a look at the project’s blog and the initial prototype.