How Large Is YOUR Retrospective?
Originally posted to the HBC Tech website on July 27th, 2017
Can you recall the size and length of your typical retrospective? If your team operates by The Scrum Guide, your retrospectives likely have less than ten people in one room and last about an hour for a two-week Sprint.
What if your current team is larger than a typical Scrum team and a retrospective period is longer than a month? What if the team members are distributed across locations, countries, time zones, and multiple third-party vendors? Is this retrospective doomed to fail? Not quite. These factors just add an additional complexity and call for a different facilitation approach.
Last month at HBC we facilitated a large-scale mid-project retrospective for a 60 people-project team. While this project certainly didn’t start as an agile project, bringing in an agile retrospective practice helped identify significant improvements. Here is how we did it.
From Inquiry to Buy-in
This all started with one of the project sub-teams reaching out with an inquiry: “Can you facilitate a retrospective for us?” That didn’t sound like anything major. We’ve been advocating for and facilitating retrospectives on various occasions at HBC: regular Sprint retrospectives, process retrospectives, new hire onboarding retrospectives, etc.
Further digging into a list of participants revealed that this retro would be unlike any others. We were about to pull together a group of 60 people from HBC and five consulting companies(!) In spite of working on the same project for a long time, these people never had a chance to step back and reflect on how they could work together differently.
In order to make it successful, we needed buy-in from the leadership team to bring the entire team (including consultants) into the retrospective. Our first intent was to bring everyone into the same space (physical and virtual) and facilitate a retrospective with Open Space Technology. The initial response wasn’t promising:
“We have another problem with this retro […] is concerned that it is all day and that the cost of doing this meeting is like $25K-$50K”
We had to go back and re-think the retrospective approach. How can we reduce the cost of this event without affecting the depth and breadth of the insights?
Options we considered
Thanks to the well-documented large retrospectives experiments by other agile practitioners, there was a number of options to evaluate:
1) Full project team, full-day, face-to-face, Open Space-style retro
3) Decentralized retrospectives using Innovation Games Online platform
4) Overall retrospective (LeSS framework)
Around the same time, I was fortunate to join a Retrospective Facilitator’s Gathering (RFG2017) - an annual event that brought together the most experienced retrospective facilitators from around the World. Learning from their experience as well as brainstorming together on the possible format was really helpful. Thank you Tobias Baier, Allan Jepsen, Joanne Perold, George Dinwiddie, and many others for sharing your insights! I was especially grateful for the in-depth conversation with Diana Larsen in which she pointed out that to
“Clarify the goal and commitment of the key stakeholders before you start designing how to run the retrospective.”
Back to the drawing board again! More conversations, clarifications, and convincing… With some modifications and adjustments, we finally were able to get the buy-in and moved forward with the retrospective.
What worked for us – a tiered format.
Individual team-level retrospectives
We had a mix of co-located and distributed sub-teams on this project and chose to enlist some help from multiple facilitators. To simplify data consolidation, each facilitator received a data-gathering format along with a sample retrospective facilitation plan. Each individual sub-team was asked to identify two types of action items: ones that they felt were in their power to address and others that required system-level thinking and support from the larger project community. The former was selected by the sub-teams and put in motion by their respective owners. The latter was passed to the main facilitator for analysis and aggregation to serve as a starting point for the final retrospective.
Final retrospective
For the final retrospective we brought together two types of participants:
1) Leads and delegates from individual sub-teams who participated actively at all times.
2) Senior leaders of the organization who joined in the last hour to review and support the team’s recommendations.
The goal of this workshop was to review the ideas from sub-teams, explore system-level improvements and get support from senior leadership to put the system-level changes into motion.
Retrospective plans
Each retrospective was structured according to the classic five-steps framework and included a number of activities selected from Retromat.
Example of an in-room sub-team retrospective (1 - 1.5 hours)
Set the Stage
We used a happiness histogram to get things started and get a sense of how the people felt about the overall project.
Instead of reading the Prime Directive once at the beginning with the team, we opted for displaying it in the room on a large poster as a visible reminder throughout the retrospective.
Gather Data
Everyone was instructed to think about the things they liked about the project (What worked well?) and the ones that could’ve been better (What didn’t work so well?). In a short time-boxed silent brainstorming each team member had to come up with at least two items in each category.
Next, we facilitated a pair-share activity in a “speed dating” format. Forming two lines, we asked participants to face each other and take turns discussing what each of them wrote on their post-its. After two minutes the partners were switched and the new pairs were formed to continue discussions with the new set of people.
At the end of the timebox, we asked the last pairs to walk together to the four posters on the wall and place their post-its into respective categories:
1) Worked Well/ Can’t control
2) Worked Well/Can control
3) Didn’t work so well/Can’t control
4) Didn’t work so well/ Can control
After performing affinity mapping and dot-voting the group selected the top three issues that they felt were in their control to address.
Generate Insights/Decide What To Do
Every selected issue got picked up by a self-organized sub-group. Using a template each sub-group designed a team-level experiment defining the action they propose to take, an observable behavior they expect to see after taking that action, and the specific measurement that will confirm the success of the experiment.
Close the Retro
We closed the retro by getting feedback on the retro format, taking photos of the insights generated by the team. These were passed on to the main facilitator for further analysis and preparation for the final retrospective event.
Modifications for distributed teams
For those teams that had remote team members or were fully distributed, we used a FunRetro tool. Flexibility to configure columns and the number of votes, along with easy user interface, fun colors, and free cost made this tool a good substitute for an in-room retrospective.
Final Retrospective (3 hours)
Once all individual sub-teams retrospectives were completed, we consolidated the project-level improvement proposals. These insights were reviewed, analyzed for trends and systemic issues, and then shared during Tier 2 Final Retrospective.
Set the stage
We used story cubes to reflect and share how each of the participants felt about this project. This is a fun way to run a check-in activity, equally effective with introverted and extraverted participants. The result is a collection of images that build a shared story about the project:
We also reviewed an aggregated happiness histogram from each individual sub-teams to learn about the mood of 60 people on this project.
Gather data
Since the retrospective period was very long, building a timeline together was really helpful in re-constructing the full view of the project. We asked participants to sort the events into the ones that had a positive impact on the project (placing them above the timeline) and the ones that had a negative impact on the project (placing them below the timeline). The insight we gained from this exercise alone was invaluable!
Generate Insights
Next, we paired the participants and asked them to walk to the consolidated recommendations posters. As a pair, they were tasked with selecting the most pressing issues and bringing them back for a follow-up discussion at their table.
Each table used the LeanCoffee format to vote on the selected issues, prioritize them into a discussion backlog and explore as many of them as the timebox allowed. Participants used roman voting as a way to decide if they are ready to move on to the next topic or need more discussion about the current one. Closing each discussion, participants recorded their recommended action. At the end of the timebox, all actions from each table were shared with the rest of the group to get feedback.
Decide What To Do/Close
In the final hour of the retrospective, the action owners shared their proposed next steps with the senior leadership team and reviewed the insights from the consolidated teams’ feedback.
Was this experiment successful? Absolutely! One of the biggest benefits of this retrospective was this collective experience of working across sub-teams and designing organizational improvements together.
Could we have done it better? You bet! As the project continues, we will be looking to run the retrospectives more frequently and will take into account things we learned in this experiment.
What did we learn?
Designing a retrospective of this size is a project in itself. You need to be clear about the vision, the stakeholders, and the success criteria for the retrospective.
Do your research, tap into the knowledge of the agile community, and get inspired by the experience of others. Take what you like and then adapt to make it work in the context of your organization.
Ask for help. Involve additional facilitators to get feedback, speed up the execution, and created a safe space for individual sub-teams.
Inclusion trumps exclusion. Invite consultants as well as full-time employees into your retrospective to better understand the project dynamic.
Beware of potential confusion around retrospective practice. Be ready to explain the benefits and highlight the differences between a retrospective and a postmortem.
Bringing senior leaders into the last hour of the final retrospective can negatively affect the dynamics of the discussions. Either work on prepping them better or plan on re-establishing the safe space after they join.
What would we like to do next?
Continue promoting the retrospective practice across the organization.
Offer a retrospective facilitator training to Scrum Masters, Agile Project Managers, and anyone who is interested in learning how to run an effective retro.
Establish a retrospective facilitator circle to help maintain and improve the practice for all teams.