Part 2: The effects of team and culture characteristics on coding
In part 1 we discussed the first part of our coding perspective on SCRUM/Agile. We focused on the effects of team size, team composition and team interdependencies and suggested solutions to common problems. In this part we will focus on the coding process itself and how it is affected by interruptions and priority shifts.
Scrum has a lot of rituals that pose a challenge to the coding process. Not only by interrupting the flow of coding but also because the content is too often not directly relevant to – or even worse – distracting from the developers work. We will discuss the challenges and suggest ways to deal with them so that their effects are minimized. We will end the blog with an overall summary of our findings and suggestions.
Difficulty in shifting focus
Coding is a high attention, high working memory intensive skill. Interruptions will mean that the developer will have to retrace his or her steps, before being able to continue coding. Depending on the complexity of the system and the experience of the programmer it can take up to 20 minutes each time to get back on track. The more often this happens the more time a developer will need to spend each day in getting back into the coding flow. Daily sprints, refinements, sprint planning, and retrospectives will all interrupt the developer’s coding process.
This means that too many meetings on a day will result in lower coding productivity. Simply limiting the time spent on each meeting (such as is done for stand-ups) will not decrease this effect since every interruption will be sufficient to create it. Minimizing the number of meetings depending on self-evaluation of whether each invitee can contribute to the meeting, scheduling them in one train of sequential meetings instead of distributed over the day, combined with the person leaving the meeting as soon as they feel they are not contributing in any useful way, minimizes the daily interruptions of the developer’s work and prolong the time solely dedicated to coding each day.
Issues with the planning and content of meetings
Because stand-ups are daily, to minimize their duration, especially in large teams, discussing too much detail in them needs to be avoided. For deep technical discussion or discussing organisational difficulties it is often not necessary for the entire team to be present. Breaking up in smaller groups after the stand-up allows for minimal interruptions and time spent on them.
During refinements we estimate the complexity of a story based on story points. This has been proven to be a difficult concept to grasp. Not every member of a team, not every team, has the same feeling for what exactly constitutes one story point. Mostly because its estimate is based on prior experience that can differ within a team (depending on the level of expertise) and across teams. As a result, variance within group estimates for stories can be high, and difficult to compare across teams. This makes it difficult to plan work even if the estimates of other teams are known. This problem is addressed within the team by the discussing the burndown chart of the sprint in the retrospective and comparing the estimates with the time spent on each story. Each sprint the estimate will improve consequently. Too often this step is not done, resulting in just a slight improvement in estimates over time. Improving the comparison of work estimates across teams is more difficult, even though we need it if we have interdependencies between teams. An estimate in hours or days, or any other measure that is not open for interpretation then seems to be the most viable solution.
Another issue often encountered during refinement is that the requirements in the stories are not detailed enough to refine the story properly nor to make an adequate estimate. But the product owner or scrum master insists on estimating them and refining them at a later moment because of outside pressure on the project. Of course, this results in wrong estimates, unrealistic sprint planning and the above-mentioned within-sprint shifts in priorities when the designs require additional work to be done first. As a result, at the end of the sprint the team will appear overbudget and over time even though the cause originated from an insufficient formulation of requirements. As clearly stated in the scrum guide, business requirements need to be perfectly clear from the start, so that the design of the implementation does not change during the building process. Any rework that needs to be done because of later changes unnecessarily puts the sprint goal at risk. The team should not accept any stories to be planned in the sprint if not all dependencies and requirements – both within the application and across the technical landscape- are fully clarified. This also means that the team is dependent on the business information analyst, functional and technical designers and architects to do so. We recommend pre-refinement meetups between the above-mentioned specialists so that each story on the backlog is actively prepared for refinement. That is, they make sure that all necessary info is present before the story is refined.
Atomic stories instead of a whole system approach
Cut down stories often take more time then when a system is built up more holistically from scratch. This is because they need to be integrated with previous solutions and their drawbacks and limitations must be considered, requiring extra analysis and build time. Also, it is not always possible to cut up a big issue in smaller parts, if it requires multiple components to be able to run. A complex system is more than its independent functioning parts, and SCRUM should provide a way to deal with this across the two or three weekly sprints. For example, delivered code could be placed on production continuously but not run and/or not connected. Or only approachable by users with a certain role (testers). And only activated and integrated when all components needed for a minimum viable product are supplied after several sprints.
Furthermore, we see that sequential releases of components of complex systems eventually can take up much time and effort than a more traditional big bang scenario. Especially when interdependencies and integrations with other systems are not discovered until the building stage. Again, this should be signalled much earlier by the people that have a helicopter view of the technical landscape of the organisation. Preferably in similar meetups of specialists as mentioned before (i.e., architects, business analysts and PO’s) when they are in the process of creating roadmaps for such changes. Again…pre-refinement preparation of stories should also help remedy such scenarios.
Changes in priorities during sprint
Development teams deal with requests coming from different stakeholders. We often see that – because of changing circumstances in the requirements of the business – the product owner decides that another story needs to be built first even though it is currently not in the sprint or planned for the next. Either because of management or board pressure or because of obstacles encountered in other teams. Although understandable, pulling new issues into a sprint makes it impossible to reach its target. Also, refinement of brand-new stories on the fly often suffers from an incomplete overview of all the requirements, risks and dependencies of the change in question.
Again, to avoid time and money wasted on rework, no stories should be designed or planned when there is doubt about their functional requirements. This is also made clear in the scrum guide. However, we often see that office politics result in ignoring this particularly important guideline. It implies that trying to implement SCRUM/Agile in such organisations is unrealistic and hence the method of working should be adjusted to accommodate the current culture that is simply there and will not change in the short run. Instead, we suggest that every change in priority and/or addition to the sprint’s target should be accompanied by a reconsideration of sprint timing and sprint target. Even a reset of the sprint should be possible if the remaining time will be insufficient to implement the changes. This should be made explicitly clear in both the planning and evaluation, and preferably be accompanied by a risk-analysis that studies the effect of the change in priorities on both the sprint goal, the team’s long-term goals and any of the interdependencies between different teams.
Conclusion
Although developers see drawbacks on how SCRUM/Agile is implemented at their organizations, they do also see the merit that the structured approach of working provides. That is, the emphasis on distinct roles and distinct responsibilities streamlines and focusses the development process. Stakeholder management and detailed analysis of business processes should not be placed with the technical staff as it takes up too much time and distracts from delivering code. It is also simply not their expertise.
Most developers suggest adjusting the approach to be more coder friendly. More efficient time management so that the number of coding hours each day in one go is maximised and the number of interruptions of the coding process are minimized seem the two most important take-aways of the information that we gathered at the workshop. We also argue that the identification of interdependencies of work between teams should be signalled at an earlier stage, preferably before creation of roadmaps and before the refinement of epics and stories takes place. As the SCRUM guide tells us, changes to the design of a story – both functional and technical- after refinement should be avoided at any cost. However, we frequently see that office politics or changing demands from the business jeopardizes this important pillar of SCRUM/Agile. An efficient way to deal with this reality -instead of ignoring it- would be to place a bigger emphasis on risk and time analysis and management in response to external changes that affect sprint targets and overall planning. And adjust the planning and evaluation accordingly. We of course hope that our suggestions are picked up by organizations;-).
Christa van Mierlo