For a team to effectively share ownership of all the code, all the developers need to write the code in the same way, with rules that ensure the code communicates clearly.
The specifics of the coding standard aren’t all that important, just that all team members follow the same standard.
Here are some sample coding standards:
Tired team members make more mistakes. Airline pilots, truck drivers and other occupations that require sharp concentration mandate the maximum number of hours that can be worked. That doesn't apply to the software industry, and the overall quality of software reflects that fact.
Agile advocates only working overtime if it is effective, and certainly not for an extended period of time. The basis for this is to allow Agile teams to work at a pace that can be sustained for the long term without causing burnout. This improves the quality of the work performed, and improves the team's morale.
Of the 4 variables of project management – Resources, Scope, Quality, and Time – Sustainable Pace is intended to improve the control of Quality.
Continuous Integration is a practice in which the team checks in and builds the software system as often as possible. This keeps all the programmers on the same page, and enables very rapid progress. Perhaps surprisingly, integrating more frequently tends to eliminate integration problems that plague teams who integrate less often.
True Continuous Integration occurs when developers use the practice in concert with Test-Driven Development (TDD), integrating their work immediately after writing a unit test and just enough code to make the test pass. The cycle looks like this:
This leads to developers integrating very small changes very frequently – on the order of 5-10 minutes. This practice all but eliminates the issues such as complex merges with collisions that teams encounter when they integrate even once per day and certainly when they delay integration for longer periods of time.
There are several tools that assist in Continuous Integration:
Each of these tools can be configured to poll your code repository for changes, and automatically start the build process when a change is detected. They can also execute unit and accpetance tests, and report the results.
There is also the belief that a tool is overkill for implementing Continuous Integration. James Shore has published a Continuous Integration Checklist, that thoroughly describes the CI process without the use of special tool (excepting a rubber chicken!).
All of the system artifacts belong to the entire team. This allows the team to continuously move forward, avoiding bottlenecks caused by islands of knowledge in the team. This improves flow and thus productivity, and reduces the risk that work will be disrupted by staff turnover.
In order to allow all team members to share in the work, you want to avoid having very narrow specializations as much as possible. That doesn't preclude having people with some specialization, but the team members should be Generalizing Specialists as much as possible.
Remember, though, this is just a guideline – there will always be instances where very specific specialization is required. However, in most cases Generalizing Specialists will be more productive. As Robert Heinlein said in Time Enough For Love:
A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.
A relatively important practice in Agile is the use of multiple pairs of eyeballs to help ensure that the code that is being delivered is of the highest quality possible. Quality in the context means low defects, good design, adherence to local standards, and suitability to the task at hand.
There are several ways to achieve this goal:
In Pair Programming, all production code is written in pairs - two programmers working together at one machine. Pair programming has been shown by several experiments to produce better software at similar or lower cost than programmers working alone.
This is a rather controversial practice – the concept of not working alone is foreign to many programmers. However, Pair Programming acts as a real-time code review, and it has a distinct peer pressure aspect to it. Programmers are more likely to develop test-first, follow coding standards and refactor their code when there is someone working with them!
Research by Dr. Randall W. Jensen published in 1996 while he was working for the U.S. Air Force spoke of two-person (2P) teams. At the time of the report, 2P teams had been in use since 1975. Initially, there was concern that there would be lost productivity owing to having 2 people work on the same problem. However, the study showed productivity improvements of over 150% at the time scale of a month.
Similar research by Laurie Williams at North Carolina State University has indicated that there is a 15% overhead when programming in pairs, i.e. it takes 15% longer to develop the same code as when it is written by individual programmers. However, the 15% overhead is more than made up for by the reduction in defects and the higher overall design quality of the code. Also notice that the overhead is only 15%, as opposed to the expected 50%. This indicates that two programmers working together are indeed working faster than when they are alone.
Pair Programming, though, depends on total engagement by both people involved. If one simply acts as a passenger, it is indeed a waste of time. The social aspects can be uncomfortable for some people, although it can be argued that software development in general is a social exercise and it requires the level of collaboration that pairing provides.
In this model, developers may work alone but they must have another developer commit their code to source control. This means that the second developer will perform a quick code review, and work with the first developer if any issues are identified.
This provides the benefit of a second pair of eyes reviewing the code, but doesn’t provide the real-time feedback that Pair Programming does.
Most open source projects have a very small group or even a single person who is authorized to commit code to source control. There may be many, even hundreds, of people creating code and submitting it for review. Considering the number of highly successful open source projects, this model does have merit. For situations where the development people are widely distributed geographically, it can be particularly effective.
The assumption, though, is that the committers have time available to be able to review submissions. A committer must have the talent and in-depth knowledge of the system to be able to verify that submitted code is able to be committed. Because committers are reviewing, they aren’t able to work on as much production code where that talent and knowledge may be better applied.
The traditional form of code review involves multiple team members reading selected code line by line to critique the design and style and search for defects. There are processes and tools available to facilitate the review process, including a number of open source and commercial tools to help ensure adherence to coding standards and static analysis of the code for potential issues.
While this review method is not without merit, it’s current use has become generally limited to safetly and life-critical applications, with the other methods increasing in popularity.
Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving transformations. Each transformation (called a ‘refactoring’) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it’s less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.
Martin Fowler, from refactoring.com
In Agile, we constantly seek opportunities to simplify our code, removing duplication and making it more expressive of our intent. This is achieved through the process of Refactoring.
The use of automated Unit Tests supports Refactoring in that it is very simple to determine if a change to the code has affected any other parts of a system. As each change is made, the tests are run in full to ensure that the system as a whole is still in an operative state. This also promotes the value of Courage.
In object-oriented languages, Refactoring is used to put into effect good OO design principles. A classic example is the Single Responsibility Principle, in which a class should have only one reason to change. The process of Refactoring should result in numerous small classes with minimal dependencies and only a single reason for changing.
Refactoring is not to be confused with larger changes to code, such as Restructuring and Rewriting. Refactoring is a process that occurs every few minutes (or edits) and, as Martin Fowler states above, each small change has a cumulative effect of improving the design of the code.
Over time, code becomes more brittle, less cohesive and more tightly coupled as changes are made. Changes in team members will result in having people work on code that they didn’t write and thus don’t necessarily understand. As such, changes that they make may have unintented side-effects that may not manifest themselves immediately. The developers (and their managers and customers) fear making changes to existing code – the testing effort required to validate the changes is quite prohibitive. Similarly, managers have traditionally seen changes to existing code as unnecessary tweaking that costs money and produces little short-term value. Developers are told not to touch code that isn’t broken, regardless of its state.
This is known as Technical Debt.
What results is not unlike the debt on a high-interest credit card. If you only make the minimum payments on the credit card but continue to rack up charges, eventually the combination of high interest and the principal debt becomes overwhelming.
Refactoring is a way to deal with code debt. With most credit cards, if you pay off all of your purchases before the statement date then no interest is charged. The same applies to Refactoring – if you apply it in small increments supported by tests, the code debt is quite low and manageable. A system with low or no code debt can be modified easily since changes will be relatively isolated. A symptom of code debt is that a single change in one class requires changes to multiple areas of the code.
Effective Agile teams use a technique called Test-Driven Development (TDD), in which the unit tests are written before the code. Then, just enough production code is written to allow the test to compile, but not pass. The unit test is executed to ensure that it really does fail. If it passes at this point, then the unit test must have a problem. Then code is written until the unit test passes. The code and tests are then Refactored to eliminate duplication, minimize dependencies and better express the intent of the code.
While the concept of TDD sounds simple, it requires a considerable amount of discipline and focus, although this becomes much easier over time. The unit tests must be kept as small as possible so that the code required to make them pass is similarly small. The end result is a comprehensive test suite that can be executed at any time, and code that tends to be much cleaner since it has been focused from the start on simply making a test pass. This concept is known as coding by intention.
In Test-Driven Development, the tests are not used for verification of the code, but rather for specification, i.e. the tests detail how the code is supposed to behave. Indeed, people are now beginning to use the term Behaviour-Driven Development, with associated tools. Even the latest version of the quintessential Java testing tool, JUnit, no longer forces the developer to begin the test method names with ‘test’.
TDD follows a consistent rhythm or cycle as illustrated here:
It’s important to emphasize that this process occurs in very small steps. A single test and accompanying code are written, executed and integrated with the rest of the system. By making only tiny changes like this, collisions during the integration process are much less likely and much easier to resolve.
A program built using an Agile process should be the simplest program that meets the current requirements. There is not much building for the future. Instead, the focus is on providing business value. Of course it is necessary to ensure that a good design is used, and in Agile this is brought about through Refactoring.
Kent Beck provided 4 rules for Simple Design, in descending order of priority:
Simple means enough to allow you to keep moving. Don’t design for something that isn’t needed today. The use of Design Patterns also contributes to simplicity by using standard constructs and approaches that have been observed over many years of software development. Similarly, good object-oriented design principles will ensure that the code is simple and maintainable. A useful guide is the SOLID principles from Robert C. Martin:
Simple design is also a side-effect of Test Driven Development (TDD). Since the code is being written to make the unit tests pass, it tends to be more focused and much simpler.
One of the oft-cited myths about Agile is that there is no design at all, which simply isn’t true. What Agile promotes is enough design for the moment rather than trying to design the entire system up front that many traditional methodologies espouse. Additionally, Agile promotes the use of the simplest tools possible, i.e. CRC cards and whiteboards over round-trip modelling tools.
The simplest answer about the amount of design required is that it depends. It depends on the people on the team (as most issues do), and their level of experience and expertise, their ability to learn and the ways in which they view the world. There are people who can just see the abstractions in a system, where others need to work their way through a modelling exercise. As a result, each team will require different levels of design effort. The one overarching principle, though, is that a team should not design everything up front, and they should validate their designs with tests and code as soon as possible.
There are design decisions, that are perceived as difficult to change once they are made and thus people want to get them right up front. The selection of a programming language or platform (e.g. Java/J2EE, .NET, Ruby/Ruby on Rails/LAMP, etc.), or whether to use a web interface or GUI are examples of fundamental decisions that people believe are hard to change.
However, Martin Fowler makes a powerful argument that these assumptions are to an extent invalid, and indeed lead to more complexity in a system than is required.
The developers put a simple system into production early, and update it frequently on a very short cycle. This makes the software ‘visible’ to the Product Owner and Stakeholders, and is a much better indicator of concrete progress that updating a project plan. It also gived the Product Owner the opportunity to use the system much earlier than would be possible under traditional processes. As such, the Product Owner sees returns on their investment in the system much sooner.
Releasing early and often gives the Product Owner a better opportunity to steer the project, accommodating change as required and putting the project back on track if it is deviating from the Release Plan or Iteration Plan. This is a very powerful risk mitigation strategy, allowing the system to adapt to changing business requirements.
Note that releasing the software to the Product Owner doesn’t necessarily mean that it will be deployed to production. That is completely up to the Product Owner, and is subject to all of the scheduling, certification and deployment issues typical of any other software development project.
The use of the term “Velocity” is very deliberate in Agile. Velocity is defined as the rate of change of position over time along a straight line. It is a vector quantity, adding the dimension of direction to the scalar value of speed. This applies to a software project in the sense that you may be able to move fast, but you need to know which direction you’re going. As such, Project Velocity is a measure of the rate at which business value is being delivered on the project. It isn’t simply a measure of how much time the team puts in on that work or how much code is written, but rather a measurement of value from the Customer’s perspective. The unit of measure of business value is the User Story. To measure the Project Velocity you simply take the User Stories that were completed during an iteration, and obtain a total of the estimates for the Stories. The total number of Points (or whatever unit your team is using) is your team’s Project Velocity. For example:
This table shows the User Stories completed for an Iteration. Four Stories were completed and one was not. For the completed Stories, the total number of points is 8. Therefore, the team’s Velocity in that Iteration was 8. That value is then used to feed the planning process for the next Iteration. This is known as Yesterday´s Weather. The team´s Velocity is also used to show how they are progressing on the overall system, and to highlight if that progress is slower or faster over time. Quite often, Burn-down or Burn-up Charts are used to highlight this:
In the end, the Project Velocity always represents how much business value the team was able to deliver in a given Iteration, which is used to help determine how much they can deliver in the future.
Agile approaches prefer to use a simple estimation technique called Relative Estimation.
Consider this stack of books:
If you estimated the number of pages in each book, you would have some very precise numbers. You would also most likely be quite incorrect!
What if you took one book – for example the one on the bottom – and said it had a size value of 1. Then, you estimated the size of the rest of the books relative to that first one – for example, the book second from the bottom would be 2 or 3 times the size of the bottom one. The next book up, the thickest, would be about 5 times the size. While these estimates aren’t very precise, they are probably much more accurate than attempting to estimate the number of pages, and you can perform the estimation very quickly
We can apply this same approach to estimating Stories. We take a single Story and use it as the baseline – often one that isn’t the smallest but isn’t the largest either – and give it an arbitrary value of 2. We then take each Story that needs to be estimated and compare it against the baseline in terms of the total amount of effort required by the whole team to complete that Story to satisfy the team’s Definition of Done.
If the Story being estimated is considered to require more effort to complete than the baseline Story, give it a value of 3. If the effort is less, then give it a value of 1. If the effort is considerably larger and the team isn’t comfortable with a value of 3, you should look at ways to split the Story further until the smaller Stories can have values of 1, 2 or 3.
Note that the team members estimate the effort involved to implement the User Stories. This is an important point, because traditionally the team has had estimates imposed on them, or the estimation has been performed by a small group of experts. By ‘owning’ the estimates, the team members not only have a vested interest in meeting those estimates, but they also tend to make much more realistic estimates in the first place.
If the team doesn’t feel comfortable providing an estimate to a Story, it’s likely because they don’t know enough about it. In that case, they ask the Product Owner for more information. This can result in the developers having enough detail to provide the estimate, or that the Story may need to be split into multiple stories.
This is a sample user story for defining a Customer Discount Rate:
The problem with this Story is that it’s too large. There are a couple of concepts here that need to be split out into separate stories:
This provides just enough detail for everyone involved (Product Owner, Developers) to agree on the importance of the story, and for the Developers to create a reasonable estimate of the time required to implement it. It doesn’t delve into any specific details about how the system performs the action, any validations required, etc. These are left until the Developers are creating the programming tasks, and consult the Product Owner for more details.
Having said that, there still isn’t quite enough detail in the Preferred Customer story. During the Release Planning meeting, the Developers would likely ask the Product Owner, “When do you start counting the $50,000?”
With the answer in hand, the Story is updated by the Product Owner, and the Developers provide their estimate:
Agile teams strive for continuous improvement. Just as Refactoring is used to improve the design of the code, Retrospectives are used to improve how the team works. Traditionally, retrospectives were held at the end of a project and were commonly known as post-mortems. Other than the connotation that the project is dead, holding a retrospective at the end of project is quite simply too late.
In Agile, Retrospectives are used at two levels – at the end of each Iteration, and at the end of each Release. In both cases, the goal is to identify not only what didn’t work well for the team, but to acknowledge what did work well. It’s equally important to learn from one’s victories as it is to learn from one’s mistakes!
Retrospectives are always held with the mindset that every person on the project did the best that they could do at the time. This isn’t New Age blather intended to prevent people’s feelings from being hurt – it’s actually quite rare for people to actively sabotage projects! Retrospectives are also not a forum in which to exact revenge for perceived wrongdoing, or simply a blame session. They are used to help the team improve with each and every iteration and release, and are a crucial aspect to remaining Agile in the longer term.
Iteration retrospectives are typically held in the afternoon after demonstrating the latest features to the Product Owner. At this level, the meeting is relatively short, taking 1-2 hours.
The immediate team participates, and one person acts as a facilitator. Using a whiteboard or flipchart, each person in attendance may suggest items or issues that they feel the team did well, or needs to improve upon. The facilitator writes down the items, possibly asking questions for clarity’s sake. People may also suggest multiple items, or even the same item under both “did well” and “needs improvement”. At this point, the team does not discuss the items.
Once all team members have had a chance to suggest items, the team as a whole discusses the two lists. For the items that are deemed to be the most important, the team as a whole creates actions that will be used to address the items. These actions could be as simple as posting a progress chart on the wall daily, or possibly as significant as asking that the developers receive training in Test-Driven Development.
Retrospectives at the Release level are quite similar to Iteration Retrospectives in terms of how they are run. However, they can be considerably longer, often taking 1-2 days. Rather than just the immediate development team, all stakeholders may be invited to this retrospective.
Release retrospectives deal with more strategic issues, such as the team’s workspace, the composition of the team, the involvement of stakeholders, and dealing with teams or resources external to the project.
For both Iteration and Release retrospectives, the goal is the same: improve the performance of the team.
Each day, the team gathers for a very short meeting to discuss the current situation on the project. For this meeting, everyone stands in order to avoid long detailed discussions. This is the Standup Meeting, and is used to allow everyone to keep their finger on the pulse of the project. Attendance for this meeting is mandatory, but it is quite limited in length.
During the meeting each team member speaks in turn, answering these 3 questions:
Ensure that when people are answering these question, they don't get into the "gory details". All discussions that don't answer the "3 Questions" should be held outside of the standup.
As a rule of thumb, if your standups are taking more than 1 minute per attendee, then they are too long! You should know that you're doing well if they are taking 30 seconds per person.
The intention of this very limited discussion is to allow the entire team to know who is working on what, and how they're progressing. If the meeting is limited to 10-15 minutes, then team members are much more likey to attend. Despite its brevity, the standup meeting provides an amazing level of information to the team.
It is said that if you forecast the weather for the next day based on today's weather, you will be correct 70% of the time. This concept is used to determine how much work for which the developers can sign up in any given iteration. The total number of points for which they are allowed to sign up is only as many as were completed in the previous iteration.
This allows the developers to recover after a difficult iteration, and to deal with issues such as vacations and illness. If there is still time left in an iteration after all of the Stories and cleanup tasks have been completed, the developers ask the Customer to provide more Stories in order to fill the time.
The Project Velocity can be expected to fluctuate slightly over time. If there is a substantial change for more than one iteration, a Release Planning meeting should be held in order to re-estimate the remaining Stories and to and renegotiate the release plan.
Iterative Development is a key element to adding agility to the development process. The development schedule for a release should be divided into iterations of about 1-3 weeks in length. This length is kept constant, despite pressures to increase length in order to fit “one more feature” in.
The result of this constant iteration size is a rhythm that the process takes on. It provides predictability for the Product Owner, Managers, Developers and Testers alike. The use of shorter iterations also accommodates changes in a system’s requirements. Unless a change represents a complete paradigm shift, it is always scheduled for a later iteration based on its priority from the Product Owner’s point of view. By having shorter iterations, such a change is at worst a couple of weeks away from being implemented.
The programming tasks shouldn’t be scheduled in advance. Instead, these are fleshed out during the Iteration Planning Meeting at the beginning of each iteration.
An Iteration Planning Meeting is called at the beginning of each iteration to produce that iteration’s plan of programming tasks. User stories are chosen for this iteration by the Product Owner from the Release Plan in order of the most valuable to the customer first. Failed acceptance tests to be fixed are also selected.
The number and size of Stories selected depends on how much work was completed in the previous iteration. This concept is known as Yesterday´s Weather. This is a risk management activity that addresses the tendency to over-book an iteration, and accommodates estimates that were perhaps too optimistic. The actual value of the Stories completed in the previous iteration is referred to as the Project Velocity.
The selected Stories are then broken down by the developers into the programming tasks required to implement the Stories. These tasks are the detailed plan for the iteration, and should be 1 to 3 ideal programming days in duration. Ideal programming days are how long it would take to complete the task if there were no distractions. Tasks which are shorter than 1 day can be grouped together. Tasks which are longer than 3 days should be broken down even further.
Note that the estimates provided include the time for writing Tests and to perform Refactoring. These practices are critical to reducing the technical debt within the code, and allowing the team to continue to progress towards a completed system.
You should also resist the temptation to adjust estimates in order to “fit” more stories into an iteration. What often happens is that the initial estimate for a story won’t match the combined estimates of the tasks required to implement it. That’s actually a good safety net to catch stories that were larger than first thought, and the general rule of thumb is to change the story’s estimate to match the tasks. That may result in more hours, days or points than specified by Yesterday´s Weather. If that’s the case, the Iteration Plan must be adjusted by removing stories (and possibly adding smaller ones) in order to fall within the bounds of the previous iteration´s actual amount of work completed.
If the team happens to complete all of an iteration´s work before the end of the iteration, they approach the Product Owner to ask for more work that will fit the remaining time. This often happens when a previous iteration had a particularly low amount of completed work, possibly due to illness, vacation, etc. It is generally better to use the lower estimate based on Yesterday´s Weather and add Stories, than it is to attempt to account for the differences and try to fit more work in the iteration.
The development team needs to release incremental versions of the system to the customers as often as possible. Release Planning is used to discover small units of functionality that make good business sense and can be released into the customer’s environment as soon as is possible, and makes sense to the business. This is critical to getting valuable feedback in time to have an impact on the system’s development. The longer you wait to introduce an important feature to the system’s users, the less time you will have to fix it.
After User stories have been written you can use a release planning meeting to create a release plan. The release plan specifies exactly which user stories are going to be implemented for each system release and dates for those releases. This gives a set of user stories for customers to choose from during the iteration planning meeting to be implemented during the next iteration. These selected stories are then translated into individual programming tasks to be implemented during the iteration to complete the stories.
The acceptance criteria for the User Stories are also considered at this point. These acceptance tests are run during the iteration, and subsequent iterations to verify when the Stories are finished correctly and continue to work correctly.
When the project velocity changes dramatically for a couple iterations, or in any case after several iterations, schedule a release planning meeting with the customer and create a new release plan.
A release planning meeting is used to create a Release Plan, which lays out the overall project. The release plan is then used to create iteration plans for each individual iteration.
It is important for technical people to make the technical decisions and business people to make the business decisions. The essence of the planning meeting is for the development team to estimate each user story, and have the Product Owner decide the relative priority of the Stories.
User stories are printed or written on cards. Together programmers and customers move the cards around on a large table to create a set of stories to be implemented as the first (or next) release. A useable, testable system that makes good business sense delivered early is desired. You may plan by time or by scope. The Project Velocity is used to determine either how many stories can be implemented before a given date (time) or how long a set of stories will take to finish (scope). When planning by time multiply the number of iterations by the project velocity to determine how many user stories can be completed. When planning by scope divide the total weeks of estimated user stories by the project velocity to determine how many iterations until the release is ready. Individual iterations are planned in detail just before each iteration begins and not in advance.
For a team new to Agile, it may seem that the amount of work scheduled in a Release is small. They key difference is that each individual User Story is completed to the point of being ready to ship – this is the Done State. By working this way there is real, tangible progress and the rate at which the team was able to deliver Stories can be used to forecast their progress in the future (Yesterday’s Weather).
Another key point about the Release Plan is that it is going to change. The Plan only represents that the team knew at that single point in time, and any changes in the business that occur or anything the people learn about the system as it is being developed will be factored in. This is crucial to being Agile – you must embrace this change rather than fighting it! That doesn’t mean that every single change will be accommodated, but they will be considered and if the Product Owner deems something a high enough priority then it should be added.
That said, you can’t simply add more to a Release – you need to remove something of equal size in order to make room. So, if a new User Story is added that has an estimate of 2 points the Product Owner must select a Story or Stories of equal size to be removed.
This is scope-based management, where the Release Date remains fixed and scope becomes variable. It is possible that the the Release Date can be changed to allow more functionality to be added. The point, though, is that you can’t simply add more functionality to be delivered in the same amount of time.
There are situations in which the developers simply don’t know enough about a Story to be able to provide an estimate. In that case, they would perform what’s known as a Spike. The Spike is simply a quick, throwaway proof of concept to allow the developers to gain an insight into what’s really required to implement a story. The result is that the developers can then provide a proper estimate for the story. Note, though that a spike should be limited to no more than a week in duration, and ideally just a day or two.
This is real-world example where a spike was used when the developers didn’t have enough knowledge to make an estimate:
The mainframe system has been in existence for many years, and the Customer and management wish to leverage the all of the work that went into building the tax calculations (and no one wants to attempt to rewrite those calculations!). The mainframe system is a typical COBOL application, while the new application is web-based.
The problem is that the developers have no idea how or even if they can get the information from the mainframe system. There was an attempt by another team using one method, but it proved to be error-prone and not scalable.
So, rather than attempt to give an unreliable estimate on the story, the team decides to take a day or two to do some experiments to determine if it is indeed feasible to get the tx information directly, and if so to try a couple of different approaches.
In the end, there was a reasonably simple method to communicate with the mainframe system via CICS. By the end of the first day the developers were able to connect to and retrieve the information from the mainframe.
Then, the team did a very important thing. They threw away the code used to test the concept, and provided the Product Owner with a reliable estimate for the time it would take to build the actual feature. This included all of the requisite testing effort.
Quite often it’s difficult to get started writing User Stories for the first time. Here are some sample User Stories that may provide some guidance. In general, though, there are a few guidelines you can follow to make a good story:
A User Story is a short description of the behavior of the system from the point of view of the Product Owner. They are in the format of about three sentences of text written in the Customer’s terminology without technical jargon. In simplistic terms, user stories are the response to, “Tell me the stories of what the system will do. Write down the name of the story and a paragraph or two.”
Another way to view a Story is that it’s a reminder for a future conversation between the Product Owner and the Development Team. There should be enough detail in the Story to allow everyone to remember what the Story is about, but no more. The conversation takes place during Iteration Planning, at which time the Programmers ask questions of the Product Owner in order to flesh out the details of the Story, and then brainstorm the Tasks required to implement it.
A good mnemonic to remember is Ron Jeffries’ 3 C’s: Card, Conversation, Confirmation.
Often, the stories are written on 3″ x 5″ index cards, although an electronic copy may be used. Since index cards are somewhat small, they automatically limit the amount of writing that can be put on them (which is a good thing). This forces the Product Owner to focus on making the text clear and concise, while being as simple as possible.
Story cards are a great conceptual tool. When they are laid out on a table, the Product Owner can visualize the entire system. Since the cards can be easily moved around, the Product Owner can ‘see’ the system from different perspectives. They also work well during development, providing a very concrete reference to how much has been completed, and how much work is left.
Once the Stories have been estimated and the initial Project Velocity set, the Product Owner can make tradeoffs about what to do early and what to do late and how various proposed releases relate to concrete dates.
There has to be at least one story for each major feature in the system, and it must always be written by the users. The programmers shouldn’t write the stories, but they have conversations with the users that are then attached to the stories together with pointers to supporting documentation.
By initiating and using this process, the balance of power between the Product Owner and Developers is much more even. The Product Owner needs to feel ownership of and take responsibility for the care and maintenance of the Stories. This allows them to feel comfortable making priority decisions about the Stories, add new ones, and add new detail to existing Stories as development progresses.
User Stories also drive the creation of the Acceptance Tests. One or more automated acceptance tests must be created to verify that the user story has been correctly implemented.
A typical misunderstanding with user stories is how they differ from traditional requirements specifications. The biggest difference is in the level of detail. User stories should only provide enough detail to make a reasonably low risk estimate of how long the story will take to implement. When the time comes to implement the story developers will go to the Product Owner and receive a detailed description of the requirements face to face.
Another difference between stories and a requirements document is a focus on user needs. Details of specific technology, data base layout, and algorithms should be avoided. Stories should be focused on user needs and benefits as opposed to specifying GUI layouts.
Use cases tend to be complex and formal such that the Product Owner doesn’t want to touch them. This leads to development asking all the questions and writing down all the answers and taking responsibility for the resulting body of work. The Customer is reduced to sitting on the other side of the table and pointing. Using stories provides a form of expression that is more approachable than a formal use case.
One approach that has been suggested is to define the Stories during the Planning Process, and then create a Use Case for each Story at the beginning of an Iteration. This allows the deferral of specifying the details of a Story until they’re really needed. Some would argue that Use Cases aren’t really needed, although it may be a requirement for your organization. At any rate, Stories and Use Cases aren’t mutually exclusive.
Plans are worthless, planning is essential.
Gen. Dwight D. Eisenhower
One of the myths about Agile is that it is disorganized coding with no documentation or regard to planning. Nothing could be further from the truth. In fact, planning is one of the more important aspects of Agile, and occurs all the time. On any agile project, you will perform considerably more planning over the lifetime of the project than with traditional processes.
The Agile planning process allows the Product Owner to define the business value of desired features. Cost estimates provided by the development team are used to choose what needs to be done and what needs to be deferred. The effect of this planning process is that it is easy to steer the project to success.
The planning process in Agile can be described a list of steps that are repeated for each release:
Lack of user involvement traditionally has been the No. 1 reason for project failure. Conversely, it has been the leading contributor to project success. Even when delivered on time and on budget, a project can fail if it doesn't meet user needs or expectations.
Software Magazine, Feb. 2001, Collaborating on Project Success
Based on the Extreme CHAOS 2001 Report from The Standish Group.
Agile teams deal with the issue of business and user involvement by having a person who is empowered to specify the User Stories, set their priority and to answer questions about them in order to clarify the real requirements. Ideally, this Product Owner is co-located with the development team, although an even better approach is to locate the team with the Product Owner.
The effect of having the Product Owner working so closely with the development team is two-fold:
Because of its importance in ensuring the success of a project, the Product Owner role requires certain traits in the person who fills it. They must be able to make informed decisions quickly and not waffle on their answers. If they don't know the answer to a question, they need to be able to say so and then follow through with getting the answer as quickly as possible. They need to know the business that will be supported by the system quite well. They also need vision.
For a small system with a focused group of users, having an end-user as the Product Owner is fine. When a system is larger, for example if it has a large user base or crosses several lines of business, the Product Owner needs to have an overall strategic vision of how the system is going to be used to support all of its users. This requires someone at perhaps a management level, or they have a reporting relationship with C-level management. This is required because the issues faced by this Product Owner increase considerably with the number of lines of business in the system. Often, these issues require resolution from a higher level of management, and having a Product Owner either at that level or with a direct reporting relationship to that level will speed the resolution process.
It's quite rare that the Product Owner for a system is able to devote all of their time to the product ownership role. As a result the Product Owner role filled by a single person doesn't scale very well as a system grows or if the system has multiple stakeholders, each with their own competing priorities. Some strategies to help are:
The Product Owner role is the single most important and difficult on any type of project, but is especially important for an Agile one. However, by having a dedicated person located with the development team, the chances for success increase tremendously.
Agile teams use Retrospectives at two levels – at the end of each Iteration, and at the end of each Release. In both cases, the goal is to identify not only what didn't work well for the team, but to acknowledge what did work well. It's equally important to learn from one's victories as it is to learn from one's mistakes!
Retrospectives are always held with the mindset that every person on the project did the best that they could do at the time. This isn't New Age blather intended to prevent people's feelings from being hurt – it's actually quite rare for people to actively sabotage projects! Retrospectives are also not a forum in which to exact revenge for perceived wrongdoing, or simply a blame session. They are used to help the team improve with each and every iteration and release, and are a crucial aspect to remaining Agile in the longer term.
Many people have said that embracing change in Agile leads to scope creep and churning, and in the end nothing gets delivered. Consider this analogy:
You’re renovating a bathroom, and you planned to change the sink & vanity, reuse your existing tub & shower, and put ceramic tile on the floor. After replacing the sink & vanity, you decide that you do indeed want to replace the tub & shower after all. In order to stay within your budget, you decide to forego the ceramic flooring, and install less expensive vinyl instead.
Those are changes, NOT scope creep.
While doing all this you realize that since the new vanity has a granite countertop, and you want granite counters in the kitchen as well.
THAT is scope creep.
Automating tests at the unit and acceptance level are the "grease that helps the wheel turn" on Agile teams. Does automated testing take extra time? When you're first starting, yes. However, you must consider automated testing an investment that will bring significant returns in the medium and long term, at the expense of some short term effort.
If you make that investment, you will find:
So put simply, automated tests enable your team to be agile.
Testing at the developer level is used to provide tactical verification of the software, i.e. at the class and method level. Developers write tests for individual methods, working at a very low level with the code. This is also sometimes referred to as Unit Testing, however that term has certain connotations that came with it from more traditional software development processes. So, we prefer the term Microtesting in order to better communicate the level and scope of the activity.
To support developer testing, there are many commercial and open source testing frameworks available. The most popular are generically called 'xUnit' (from names such as JUnit for Java and NUnit for .NET), which allow the developers to write their tests in code that can be executed with a single click of a button.
A comprehensive list of the available developer testing frameworks for numerous languages can be found on Wikipedia.
Story Testing is the more strategic level of tests used for a system. These tests verify that the User Stories have been implemented as required by the Customer. Ideally, these tests are automated in order to provide an easy way to verify the state of the entire system. Note, however, that "through the GUI" record and playback testing tools are not recommended for this purpose since their tests tend to be quite brittle in the face of a changing user interface.
Story tests may be written using tools such as xUnit, FIT, FitNesse, Selenium, Watir, RSpec, Cucumber, and a number of other open-source and commercial tools.
Integration testing is a more controversial subject in Agile circles. There are some who believe that the investment of time & effort that goes into creating reasonable tests doesn't come with a good return. Integration tests tend to take much longer to run, and thus are less likely to be executed. Furthermore, the functionality covered by the integration tests should ideally have already been covered by the Story Tests.
Where Integration Testing does have merit is when Agile techniques are being applied to existing systems with a significant legacy code base. The Integration Tests form a foundation from which the developers can safely build new functionality.
Any code base will contain build up Technical Debt over time. This is even more true for code that does not have any automated tests.
You can easily see the symptoms of Technical Debt:
Technical Debt, like credit card debt, must be managed and paid down in order to be able to move forward. Developers must be empowered to identify debt and fix it immediately – the short term hit on velocity will pay off handsomely over time!
Knowing when a Story or Feature is ready to be put into production, the Done State, is critical. It doesn't matter if the developers believe that they're done, or even if QA says a feature is done, if the the business people for whom the feature is being built don't think that the feature is going to do what they need it to do. So, done means:
This forces the team to work on very small pieces of the functionality at a time, and also allows everyone involved to see real, tangible progress.
