Shift Left: Testing Earlier in Development
One candidate for best experience report at XP 2015 was “High Level Test Driven Development – Shift Left” by Kristian Bjerek-Gustuen, Emil Wiik Larsen, Tor Stålhane, and Torgeir Dingsøyr. Their report describes testing strategies and tactics used on a large-scale IT development project in Norway. These authors called it “shifting left” because they wanted to move testing to as early as possible in development.
Their project was complex along several dimensions.
Coordinated work among independent teams: Several vendors were simultaneously developing and delivering systems that communicated with each other via service interfaces. Vendors did detailed design and development including unit, integration and sprint system testing of their deliverables. This work had to be integrated by the company. They performed acceptance testing after every sprint, system integration testing which ran in parallel with the vendors’ sprint system tests, as well user acceptance testing.
Significant coding and testing effort: Over 100,000 hours of testing and development went into the release.
Time pressure and project criticality: The delivery date was fixed. The system was an extension of an existing system for processing payments. It had to be delivered on time and with high quality.
The duration of the project they described was roughly 40 weeks, if I read the report correctly: 12 3-week sprints followed by 6 weeks of system testing. Faced with a time crunch, an existing group of 3 maintenance scrum teams were scaled up to 11 teams over four months. The customer who was receiving the software system didn’t have fulltime resources to dedicate to the scrum teams who were comprised of these roles: scrum master, developer, functional architect, technical architect, and QA/tester. My guess is that the customer was called upon as needed to provide clarifications, offer feedback at sprint demos, and to do all necessary testing and integration testing.
To ensure that each team had efficient and sufficient testing and quality assurance, a dedicated QA person/tester was assigned to each team. This reminds me of Stephanie Savoia’s experience report at Marchex where they embedded testers into their dev teams.
In the case of this Shift left report, the dedicated QA person (I prefer the term quality advocate) seemed very busy and vital: they ensured the quality of design documentation; decided whether the implementation was testable; prepared test data; worked with devs and the Scrum master to ensure high quality code. They also made sure test activities were performed as early as possible and that the customer provided necessary clarifications to the dev team.
Bridge, go-between, quality advocate, tester extraordinaire!
During sprint demos, in addition to demonstrating new functionality, teams also provided information on how testing was performed and any issues they had encountered in testing the implemented functionality.
There is more in the report about how they managed testing and dependencies between teams, identified and tracked high-risk modules changes and defects to aid test and development planning, and introduced exploratory testing using interdisciplinary teams. But I digress.
Back to those busy QA advocates. Not surprisingly, the experience reporters mentioned in passing that the dedicated QA advocate became one of the most central resources for the teams.
It seems that they were deeply appreciated by the whole team. That’s important. Where they ever overwhelmed by their responsibilities? Were they overworked?
Any experience report never answers all questions I have. I still find them thought provoking. Even though I’d love to sit down and talk with experience reporters and ask them more questions.
One pattern Joe Yoder and I have written about in our Shifting From QA to AQ pattern collection is called Pair With A Quality Advocate. If you purposefully pair up devs or other folks with a QA advocate, their expertise can “rub off” on less skilled/experienced testers and developers. Steadily (and sometimes stealthily), a quality mindset gets infused into the entire team. You still need quality advocates, but everyone takes on more responsibility for quality. And that's a good thing.
- Previous: Future Commitment
- Next: Early Performance Testing at Agile 2015