Managing Web Design Projects with Scrum

Photo credit: VFS Digital Design

Successful web design and development agencies share the same passions — we want to do good work and see our clients succeed. Because the web is a relatively new medium and is evolving every day, traditional software development processes can often get in the way. Designing and developing web sites requires methodologies that are as fluid as the medium. There are several Agile methodologies available, but Scrum is the best option for agencies. This is a story about our agency’s transition to Scrum, and how it made our work and our clients more successful.

Scrum defined

In a nutshell, Scrum is a software development process that enables developers to break down a project into much smaller iterative and incremental deliverables. It is designed to be flexible, which makes it ideal for web design and development projects — projects with a custom nature that are difficult to fully define before starting out. During design and development, client changes in scope are easily absorbed into the workflow.

The Sprint: Tasks are worked on for the duration of the sprint, resulting in deliverable features. The sprint cycle is repeated until the project is complete.
Photo credit: photo credit: verlaciudad

Instead of focusing on the entire project at once, you break down the first phase into smaller deliverables, called stories. Then you decide how many stories you want to develop in the next two or three weeks, what we call a sprint. The completion of each sprint informs how we go about the next one, repeating the process until the project is complete.

And because the process is more iterative, the client is more involved. For example, designers using the waterfall approach will often wait until the end of the design phase to present concepts to the client. Scrum breaks the design process down into smaller deliverables, where each step can be tested and reviewed, informing the overall design.

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
— The Agile Manifesto

There are more components to the Scrum software development process — self organizing teams, Scrum masters, and Product owners, to name a few — but, for this article we’ll focus on stories and sprints.

Why we switched to Scrum

We started out by reading a few books and attending a seminar to become Scrum certified. Once we felt comfortable adopting Scrum, we educated our clients about our new process during the initial estimating phase. We explained that the project would start out with a more general scope, to accommodate change. The only variable that would remain static was the total dollar amount of billable hours — we had to keep the project within the client’s budget.

“Agile processes harness change for the customer’s competitive advantage.”
— The Agile Manifesto

The initial client response was overwhelmingly positive. Using the Scrum web development process meant fewer hours trying to conjure up a detailed scope for a project they had difficulty envisioning in the first place. And it was comforting for them to know they could change the requirements at any time, as long as they stayed within their budget (or paid more). Adopting Scrum made the initial estimating almost painless and eased the development process, resulting in far fewer hiccups than previous projects managed with the waterfall model.

We rolled out the new Scrum web development process with one of our oldest clients, a fashion company that manufactured high end hand bags. They wanted to launch a new outlet website for selling excess inventory. We planned for the project to iterate through four sprints, each one being two-weeks long, for a total of 8 weeks. Then we broke down the site requirements into smaller stories, each one capable of being completed within a two-week sprint cycle.

The individual stories varied in size and scope. For example, the story for the visual design of the home page was rather broad and loosely defined. The inventory management feature, however, was broken down into four smaller stories, one for each sprint, to accommodate its complexity. The success of the site depended largely on this one feature, it required the ability to sell limited quantities of highly desirable items without overcommitting any stock to customer shopping carts. What these stories had in common is that each one would result in a tangible deliverable that could be reviewed and tested by both our team and the client to address address any shortcomings or failures as quickly as possible.

We kicked off the project with a sprint planning meeting where the team selected a list of stories they would commit themselves to completing the first sprint. The stories varied in size and complexity, but overall the list represented how much work our team could push through in two weeks. With the first sprint plan in place, we got started designing the website’s look and developing it’s features.

The project rolled along smoothly at first, and the fact that we were using Scrum seemed like more of a novelty than a necessity. Midway through the second sprint, however, Scrum became a lifesaver. We had a story in the second sprint to load test the inventory allocation feature. When the developers ran their tests they discovered that the shopping cart was allocating more inventory than it actually had, because of a race condition that only appeared under load.

Because we used Scrum we had taken a parallel approach to each story, testing each one individually before moving on to the next. This iterative testing process allowed us to fail quickly with fewer repercussions. Had we waited until the end of the project to do our testing, the discovery of this bug would have forced us to delay the website’s launch while we scrambled to fix it.

Worse yet, our usual ad-hoc approach to testing may not have discovered the bug at all, and it would have made it onto the live site — there it would have been a detriment to both our reputation and the client’s pocketbook. The longer a bug takes to be discovered, the more expensive it is to fix. Scrum places the emphasis on failing quickly because it’s the best way to reduce development time and protect the client’s budget.

Why the traditional “waterfall model” failed

The Healthcare.gov web site involved fifty-five different contractors and was managed using waterfall principles. The result was a broken web site that was nearly scrapped after launch and prevented millions of Americans from enrolling.

Perhaps the most well-known failure of the waterfall model is the Healthcare.gov web site. Fifty-five different agencies worked for twenty-two months on a site that underwent two weeks of testing right before it launched (source). The site quickly buckled when unveiled to the public.

The contractors were managed using a development process commonly known as the waterfall model. A process that kept the final product hidden from the end-user until the absolute last minute. The agencies building this site could not possibly have anticipated how the site would perform once live due to very little testing too late in the process. They would have fared much better had they involved the public as part of the testing process from the very start. Unfortunately, the process did not allow for that.

“Since the difficult launch of Healthcare.gov on October 1st, the process used to develop it, as well as other large and troubled government software projects, has been sharply criticized. This process, sometimes called a sequential-design process, is better known by its nickname: the “waterfall model.”
Don’t go Chasing Waterfalls
Rusty Foster
The New Yorker, October 28, 2013

Failures using the waterfall model are not exclusive to big agencies and big clients. We experienced many of its shortcomings in the years we employed the waterfall model with our own local clients. Like any other web design agency, we would approach each project from a high level and sequential approach. We spent hours in client meetings scoping out every detail of the project and building a timeline in Microsoft Project.

Changes in requirements, technology, and design trends are just a few of the variables that would change the course of a project during design and development. Whatever vision we and the client had created during those initial meetings was always compromised by the realities we encountered midstream.

Why Scrum is the best choice for building web sites

Developing a web site almost never goes exactly as planned. There are a number of different factors that can, and will, threaten to derail a project. Traditional web design project management approaches rely too much on interdependencies, allowing these threats to put a project behind schedule. Scrum, on the other hand, uses a flexible, parallel approach to simultaneously skirt and address these threats. In other words, Scrum allowed us to fail faster, giving us more time to resolve failures and move on. Here are some examples of interruptions we experienced, and how using Scrum got us around them.

Implementing third-party technology

The waterfall model

• Locking in an ironclad project scope
  Third party technology — CMS, blog, shopping cart, etc — needs to be researched and selected before the project starts.
• Not devoting enough time to research options
  It’s difficult to fully research a technology before the project begins. Poor decisions are often made due to a lack of time.
  • Wasting time implementing incompatible technology
    Hours can be lost before discovering a technology won’t work. It may be incompatible with the server or missing a key feature. The project can be delayed until a suitable replacement is found.

Scrum

• Delaying the research until the project requires it
  Allot a few hours of time for research to be done when the project requires it, not before it starts.
• Using a “spike” to evaluate technology
  Research can be done entirely in one sprint, known as a “Spike.” The goal is simply to research a few options and learn enough about each one to make a decision.
• Using the next sprint to deploy the technology
  The next sprint includes a task for implementing the selected technology. Because the research and deployment are delayed until more is known about the project, fewer obstacles are encountered

A natural evolution of the original plan

The waterfall model

• Building a web site before building a business
  Many tech startups require a web app but are not mature enough yet to fully articulate their business plan, much less their web needs. It’s near impossible to scope out a web project while a business is in its infancy.
• Developing a web site alongside a growing business
  Businesses need to evolve daily if they are going to react and incorporate market feedback. Trying to stick to an outdated scope makes it difficult to accommodate changes in the business plan.

Scrum

• Embracing project changes
  Instead of trying to commit to scope beforehand, openly acknowledge and embrace reasonable change during the project.
• Partnering with clients throughout the project
  The ability to interact with a partially developed web site is far more informative to the process than is waiting until the final product is delivered.
• Refining project scope with each sprint
  Scrum does not require throwing out the project scope. Developing a project that meets the client’s vision is still the end goal. Scrum just requires less information to get started, postponing the details until a sprint requires them.

Looking back

We did not have to adopt Scrum. We had plenty of clients lining up to hire our agency and our waterfall approach, although imperfect, worked often enough. Unfortunately, when the process did fail it was difficult to recover and not to blame ourselves or the client. In the long term we wanted to reduce the size and number of failures, make our clients happier, and produce better work. Scrum gave us the tools we needed to make that happen.

How to get started with Scrum

We got started by reading The Agile Samurai together as a company, holding a book club meeting each week to discuss what we learned and how we could apply it. After reading through the book, we decided to get certified as ScrumMasters through Mountain Goat Software, a two day event that we turned into a fun company retreat.

Whether it’s a book, a web site, or a training, there are more than enough resources available to get you started with Scrum. Here are just a few:

scrummethodology.com
www.scrumalliance.org/about-scrum/overview
www.mountaingoatsoftware.com/agile/scrum
www.scrum.org
agilemanifesto.org