Migrating thousands of pages to a new CMS with new design patterns

How do you choose a tech stack and have confidence it will perform long-term?

The project was split into two phases, following the double-diamond approach. In the initial “designing the right thing” phase, the big question was: “Which CMS is right for Suffolk Libraries?”.

We adopted a user-centred approach to technology. Rather than going tech spec first, we wanted to prioritise what would be best for the user – in this case the team at Suffolk Libraries – and working with them to find the CMS that addressed their needs, including allowing them to meet the needs of their customers.

We began by understanding the requirements of the two core user groups: content editors and library customers. We held a workshop with Leon to identify the non-functional and functional requirements for the project. The non-functional requirements described “how the system should work”, and acted as prioritised design principles for the project, guiding and framing future technical decisions. Functional requirements acted as a high-level list of user-led requirements we needed to hit in order for this project to be a success for its end users. We’ve written about this workshopping process in more detail here.

Following this, we could begin the task of profiling the many CMS solutions out there. In the world of software development, anything is technically possible given enough time, budget and developers. To accommodate that we created a five point scale ranging from -2, meaning “incredibly complex or time consuming to fulfil”, to 2, meaning “very achievable, and available out of the box”. Scoring the two types of requirements for 15+ CMS solutions against this, and multiplying the totals by a weighted ranking gave us an initial score for each option. Rather than viewing these results as a foregone conclusion, they became the starting point of a discussion with Suffolk Libraries, framed with a detailed technical research document.

The second highest scoring CMS, Statamic, appeared to fulfill most of the requirements for Suffolk Libraries. But to be totally certain this decision was the right one, we set out a series of prototyped exercises, testing to see if the CMS would do everything they needed it to.

With the previous website storing content in Markdown without a CMS, the most important test was to see if we could ingest the existing content into the new CMS. By writing a handful of content migration scripts, we were able to import a content type, reformat it into a shape suitable for Statamic, and save it into a CMS prototype. Giving the team direct access to explore and try out the prototype enabled them to realistically test the platform with their own content in a risk-free environment before committing to the solution.

How do you build strong foundations?

The second, “designing the thing right”, phase was more production focused. We started with an intense period of cross-discipline user experience design and development thinking to nail the overall approach at a holistic-level. Through design exploration and rapid prototyping in the browser, we were able to free up more time to explore and live-test solutions with Leon.

With numerous page templates, and 2000+ pages of content to migrate, the UX and visual design time was kept deliberately short and in concentrated bursts to allow us to focus our efforts on the technical challenge. The initial visual design exploration was focused on designing the foundations right, at an atomic level. Then, through a small selection of fully designed page templates, we were able to test the components in situ and be confident they would work in future page layouts.

By collaboratively designing a system that governs typography, space, grids and colour from the start, we were able to streamline a lot of micro decisions around hover states, component composition, and nuances in responsive behaviour. Solid foundations are not a replacement for good design throughout a project but they do provide a guided framework and shared language for designers and developers to rapidly create beautiful, efficient page designs.

Following the foundational design and key page templates, we designed the rest of the website in the browser, with regular check-ins and reviews with the project team and Suffolk Libraries, to ensure we were working in the smartest, most efficient way possible.

We maintained a strict policy on the number of components and variants we allowed into the new pattern library. The old site had countless variants of components, particularly those that rendered ‘lists’, and as we converted them into the new brand language, it would have been easy to migrate this technical debt into the new site. We had a critical eye throughout, constantly rationalising every pattern created to ensure we built a neater and more manageable system.

How do you allow content editors to express themselves through a CMS?

For any library, digital is a perfect way to provide community access to the content they offer. Suffolk Libraries wanted to move from a catalogue of links to a content management system that enabled them to have more conversations around what’s new, local or useful to more people. Though many pages needed to be purely functional to help customers find what they need quickly, there was a growing desire to tell more engaging stories through expressive and narrative-driven pages.

Taking a combination of existing content types and new ideas generated by the Suffolk Libraries content team, we wireframed a modular, block-based content creation model. Content editors would be able to create pages using a suite of site-wide components, allowing them to list events, forms, book recommendations, related posts, reviews, image galleries and more. To achieve this, the Statamic ‘Bard’ editor was used as the backbone for most pages on the website, allowing the editors to simply nest components within prose without any coding input.

The previous website system, written in Jekyll, very much dictated the the website's information architecture (IA). The new Statamic site puts the content team in control of the IA, allowing them to deliver a more logical, accessible site.

How do you handle events at scale?

Physical and virtual events are a key part of Suffolk Libraries’ offering. Alongside a wealth of recurring events, there are hundreds of one-off events, and even entire festivals.

Where the old website stopped at listing the recurrence of the regular events in a single HTML table separate to the one-off events, we decided it was important to treat them as first-class citizens in the new events system. From a user perspective, the systematic difference between an event that happens once or multiple times is irrelevant. It’s more useful to have access to a central listing of what’s happening, where, and when.

We first needed to understand the rules that governed event recurrence: every N days/weeks, every Nth week, as well as when events don’t happen: during school holidays, on certain days, between certain periods, etc. We translated those conditions into an admin interface that content editors could easily use and imported all the existing recurring events through a content migration script.

The bespoke events system was then able to calculate all future occurrences of a recurring event, taking into consideration when it wasn’t running, and plot them onto the main calendar alongside one-off events.

How do you run an effective, fully remote, creative project?

As a fully remote project, there were some challenges to overcome. Along with daily standups and constant sharing, we were able to keep moving forward through regular demos. These helped us decide if we’d done enough of something in order to move on to the next task. We used a combination of live video demos and Loom recordings which proved invaluable for sharing more complex sections of the system asynchronously, without scheduling lots of extra meetings.

Through a regularly prioritised shared backlog, there was total visibility of what was left to do, and when we predicted it would be completed. As new items came up, they were added either to the backlog, or moved into a post-project icebox, a phase that we were able to pick up after the website launched.

How do you ensure delight?

The biggest benefit of well-structured data in a content management system is the ability to create relationships between different content types. Once each entity is stored appropriately, it becomes possible to traverse between them.

Each library has a latitude and longitude attached to it, driving a ‘map view’ of all the libraries, allowing you to search by location to find the nearest libraries to you. In addition to this, we added a ‘nearest nearby libraries’ feature to every library details page. This queried all the libraries, calculated their distance to the current library using the haversine formula, and displayed the closest three on the page.

Another area ripe for improvement was the mobile libraries section. Suffolk Libraries runs 53 routes, visiting 800+ stops every month. The old site displayed these hundreds of stops in a single, rather lengthy HTML table. We knew the ideal experience would be to plot the routes on a map and allow a customer to type in their address to find the closest stop. Unfortunately, the stops were only stored in text form with no location data attached. Gathering 800 pieces of latitude and longitude data manually would be a huge task for the content team, so we tackled this problem in two stages of feature fidelity.

The first round added an autocomplete field to the page, allowing users to search through the list of stops by town or stop name. This involved no additional work for the content team, and improved the existing user experience while we tackled the next phase.

Secondly, we devised an alternative method of collecting 800 pieces of location data. Knowing that the mobile library drivers were already going to every stop once a month on their rounds, we proposed the creation of a small web app that recorded the location data every time they made a stop. Mobile internet coverage couldn’t be guaranteed for all routes, so we made a progressive web app that worked entirely offline. Using the indexedDB API (a client-side database), we could store the latitude and longitude for each stop in a structured data form, and export it as JSON to be imported back into the website.

By building with native web technologies, we were able to design, build, test and deploy this progressive web app in a week, ready to be used by the Suffolk Libraries mobile library team. Thanks to the web’s distribution model, drivers were able to use an offline-first application, simply by visiting the URL in their phone’s browser, and even install it to their home screens for a more native app-like experience. Even though this rapidly built app was only ever going to be a means to an end, it was still a very efficient way to capture the data set.

We’ve discussed this prototyping phase of the project in more detail on the Clearleft podcast.

How do you plan for go-live from the start?

Migrating an actively updated website is a challenge. The simplest approach is to ‘content freeze’ the old website for long enough to pull the old content across to the new system. Suffolk Libraries update their site multiple times a day, so this approach would have been a stretch at the best of times. But to compound matters, this project commenced at the beginning of the first pandemic lockdown, when all 44 physical libraries had shut and there was more onus than ever for their digital presence to be timely and accurate.

Instead, we chose to continue our approach from the first project phase, writing dozens of content migration scripts that provided a consistent, and importantly, repeatable way of moving content into the new IA structure. This meant the content editors could continue updating the previous website until the end of the project without a content freeze or any duplication of effort.

On go-live day, we were able to run the heavily tested content migration scripts against the old site’s content one final time, repoint the domain, and get the site live with zero downtime.