[xyz-ihs snippet=”quickLinks-scaling-up”]

This is the third blog in the #scalingup series. Read the previous blog to find out more about our DevOps journey while scaling up.

What is Scrum ?

No Really, What is scrum ? How do you define Scrum ?

A Scrum, as you can see in the image below, and as per wikipedia, is the method of restarting play in a Rugby game that involves players packing closely together with their heads down and attempting to gain possession of the ball.

I am pretty sure that is not the definition that you wanted to see, at least not in this blog. But, bear with me for a couple of seconds. It was not a failed attempt at a bad joke. I just wanted you to see the imagery and get a feel of what is happening. Also I wanted to throw some light on the history of that name. How did a Rugby method give its name to the popular Agile methodology that we use almost throughout the software engineering world.

An oversimplified definition of scrum is that it happens to be a common Agile framework where actions are planned to be completed in timeboxed intervals known as Sprints and progress is tracked on a daily basis in short meetings known as Daily Scrums. This daily Scrum is a stand up meeting with team members standing in a rough circle and explaining their set of points. Since this arrangement looks like a civil version of the Rugby Scrum, both the daily meeting and the process take the name of this particular Rugby activity. The point of this is that the entire team meets, gets to know the statuses of all the activities being done, finds mechanisms by which anyone who is stuck gets help and any one should also be able to suggest improvements etc. Because of this nature, It also allows for course corrections and other changes needed to make sure that what was planned for in that Sprint is indeed completed within the same sprint. Everyone is present to tackle whatever situation is there at hand. Both in Rugby and in this Agile Methodology. Hence the name is shared in both the disciplines.

Scrum is probably the most popular of all the Agile Methodologies in the software engineering world now. It is popular because it gives you as a team manager and as a team member a lot of freedom and expects you to fill only the minimum amount of documentation. It also offers both stakeholders and developers opportunities to reconsider and change their minds about requirements, architecture and other stuff that is usually set in stone in an upfront manner in other older methodologies. If you need to change any of these, you only have to wait till the end of the sprint. It also has periodic feedback built into it which is facilitated by Sprint Review & Retrospectives & the availability of a dedicated Product owner gives you the ability do a lot of experimentation with your feature set before arriving at a final one.

And it is for these reasons that we chose scrum as our Agile Methodology of choice. And it made a lot of sense. After all, we had seen a lot of benefits of scrum in our previous organizations and have been able to make good use of the same here too. But the sailing was not smooth. At least not in the beginning. Here were the primary reasons.

• Non Constant Tasks during a Sprint: pCloudy is essentially a SaaS offering and that too in a startup frame of working. So this usually meant that some issues that crop up, mid sprint which might be important for some (important) customer, and the priority therefore gets bumped up. This would mean that developers have to strategically pivot and fix those issues leading many times obviously to the issues that were originally planned getting sidelined. So the usual 2 week sprint got broken all too frequently and this meant we had unfinished tasks at the end of each sprint, which was definitely not desirable.

 

• Sprint Length: The ideal sprint length is considered to be 2-3 weeks with the most popular length being 2 weeks. This is ideal because it is short enough for a feature to be completely done from end to end and yet be short enough for stakeholders to experiment with features and decide to pivot if needed. However with many customer related interruptions coming, we decided to reduce the length of a sprint to one week. The idea was that we would take in this kind of customer interruptions only in the next sprint, which never would be more than a week away, and with this whatever we planned for in the sprint would be completed. However what we say here was that a week was usually not enough time for a feature to be completely developed, let alone it being tested and deployed. This meant that at the end of the sprint, most of the features were not “done-done”. They would be in various stages of completion. Another issue we faced was the length of the review and retro meeting of one sprint and that of the planning meeting for the next one.

  So here is what we finally arrived at, which takes the best principles of Scrum but at the same time be able to be as Agile as possible. Remember, we are not here to be Agile Champions, So Agility is very highly desirable, but not indispensable. We did some modifications to our process, once small change at a time and kept doing those changes, here is what we look like now. But As our changes continue, I am pretty sure our methodology will not look like what it looks now even 6 months down the line. Here are the most salient items from our thought process.

 

• Sacrosanct Backlog:
  • Backlog Storage: Every Feature that we need to pick up is stored in a backlog. We used JIRA for some time and have now migrated to Trello. But the point is that our Agile Backlog is stored in a automated tool. Tomorrow, if our team’s complexity changes, we may chose something else, but the point to note is that team members are able to access the backlog and are able to see all the items they are required to work on and report all changes in statuses. With both JIRA we had used the Kanban like board that the system provides and Trello by default comes with this board. This board was one of the most important artifacts used by the team.

   

  • Also we changed our methodology in such a way that while the Backlog itself is sacrosanct, but we allowed ourselves to change the backlog of a sprint to ensure that our customers are not waiting. But unless something earth shattering happens, we do not interrupt a particular task before it is finished. So strictly speaking, we are not doing Scrum, we are in fact doing something more in line with Kanban with some trappings of Scrum like Planning, Review, Retro and of course the Daily Scrum. While we continue to internally call the daily meeting a Scrum out of habit, we also do not pretend that our process is Scrum. It is now a form of what many people call the “Scrumban” where the best features of Scrum and Kanban are merged together to form a practice that looks like both. We have found that at this time, this suites our business and development needs best.

 

• One Scrum / Many Scrums: We were once a smaller team and hence it made a lot of sense for the entire team to have a common scrum, but as the team became bigger, the scrums dragged longer and both heads and legs began to ache. We then divided the team based on the specific area that each of the team worked (Portal, Devices etc) and had each team have their own Scrum. Then all the Scrum Masters of each team had another “scrum of scrums” where the larger team could focus on larger goals. This made each scrum shorter and each scrum more relevant for each attendee.

 

• Management View: Previously since the team was small, the management found it easy to sync with each team member and follow their pace of tasks with a single tool like JIRA or Trello. However as the team grew, the management needed something more robust and flexible to track their day to day tasks. We started with a shared Google sheet that managed the status of each work item which was always updated and visible to management. The shared Google sheet while easy seemed a bit flimsy. We then moved to something a bit more formal with Atlassian Confluence. Now broader level tasks are looked at in Confluence, which break down to smaller issues in Trello for each developer to work on. The scrum masters make sure that Confluence is updated based on the status of the tasks in Trello. Also the confluence report has links to the trello issues.

So what is next and what is the way forward ? Well I do not want to answer that question. That question takes the fun out of the journey, the fun of not knowing what you will discover at the next bend, what you will invent at the next bend. However as I have kept saying, ours will be a journey where we continue to tweak our processes so that it fits us at every phase of our growth. We are still young as a company and still growing and we need to be able to continue tweaking, at least till we are in a few hundreds in terms of headcount. But that doesn’t mean we are blind about the future, and we have no idea what to expect. In fact on the contrary, every tweak we make also comes with the question, how long will this stay and what do we need to do when we outgrow this phase. This is a constant question that we put to ourselves and make sure we have answers. Whenever we don’t, we step back and replan.

Appendix

Before I sign off, let me also throw some light on the experiences we had with some of the tools that we used to track the development. Also a small tidbit about compliance.

Plain Excel / Google Sheets: A Spreadsheet software is the simplest and most flexible mechanism to track virtually anything. In fact it is this simplicity that tempts a lot of people to Run their Software Agility through a series of shared Excel sheets. We too had the same experience, but the only problem is that if you have to manage complex calculations and deductions, then the number tasks that you have to perform to keep your sheet in control becomes difficult to manage. This is not a problem if this is the main thing you do or if you have people to do it for you, otherwise it soon becomes very cumbersome.

JIRA: Jira is a globally popular Issue Management System which is very very helpful in tracking issues. It also deals well with dependencies and broken down tasks. It does what it does very well. But it has one issue that makes it a bit difficult for a fast paced startup. And that is it is very rigid and inflexible. And maybe it was designed that way and it was intentionally not made too flexible, so that there are not too many changes done. I wanted some changes in the workflow that was specific to the work that we were doing and I could not quickly do it. Remember altering JIRA was not the only or the most important work that we were doing. I tried a lot to understand the complexity of JIRA under the hood, but I could not achieve something as simple as adding as many more columns to the JIRA kanban board as I wanted. I know it could be done as I have seen something like what I wanted working elsewhere, but I simply did not have the patience to understand the arcane JIRA workflow editor and other such entities in it. I understand, this is probably a job for a JIRA admin, but we were not so big to recruit one to do the customization for us. So for now we went to another tool from the Atlassian stable. Probably we will revisit JIRA when we grow bigger. And that brings us to the tool we are using now.

Trello: We have right now all but settled on Trello. Trello is another project tracker from Atlassian. It is built around the Kanban board and hence allows you to do a lot of changes to the Kanban Board, including custom columns(we view them as states). We liked it a lot, since I can now track the exact (and many times specific to pCloudy) state each item is in like Backlog, Requirements, Design, Test Planning, Coding, Testing, Code Review, Ready to Deploy, Post deployment Sanity and others.

Compliance: One common thing with these tools is the amount of developer (and tester and devops engineer) compliance. The success of any of these tools depend on the readiness of the developer to actually update statuses in these or any tools. I have see this for the past 15 years that many developers, including me when I was a fresher, simply do not want to do this. Somehow they are ready to spend nights and weekends in office running behind an elusive bug, but spending 5 minutes updating the tool is many times anathema. You have to make your developers see that it is imperative to do the necessary status and other updates in these tools. If your developers do not do it, then none of these tools will be useful. In the end these tools as much for the developer as much as it is for a manager, but they will work only if they are used properly.

Share with me the lessons you have learnt from your daily scrum in the comments section. Keep a watch on this space for my next blog in the #scalingup series.