When you hire a team to create software for you, the amount of effort put into initial planning is often what determines the project’s ultimate success or failure. An essential yet often overlooked part of that planning is the methodology chosen to manage the project itself. As a formal discipline, project management has been around for generations. But radical new techniques for managing projects—especially software projects—have been evolving at breakneck speed for the last two decades. Selecting the wrong management approach can lead to blown budgets, delayed delivery, low-quality software, and even outright failure.
So how do you choose the right project management technique? Are the latest methods the best, or should we stick to well-tested methods of the past? And what about popular project management terms like “Agile” and “Waterfall”—how do they fit into the big picture? When does Waterfall win over Agile?
Different Project Types → Different Project Management
The single most important thing we want you to take away from this article is this: One size does not fit all when managing software projects! Your team must thoughtfully select a “best fit” project management approach aligned with the specific situation you face.
I’m feeling hungry, so let me tell a story related to this topic about a local restaurant that serves delicious and handcrafted soup using only the best, locally sourced ingredients.
Imagine you’re the chef in that restaurant and need to make soup for tonight’s menu. If you make the same soup every day, then you know all about the recipe. You know what ingredients you need, which you don’t, the right amount of heat to use, and how long to simmer. You deeply understand what the final soup should taste like (the goal) and the process for creating it (the solution). These terms “solution” and “goal” are abstract and apply equally well in software development or soup development. The goal is the end product we want to achieve. The solution is how we get there: the tools, ingredients, process, data architecture, and a dash of salt. As the chef, you need to gather the ingredients and convert them into a delightful meal. Tonight’s menu is a soup you have made 100 times, and it turns out that every ingredient you need is already in the kitchen. Even your favorite pot and knife are clean and ready to use. You are on comfortable and familiar ground and could probably make the soup with your eyes closed. Since you have absolute clarity about both the goal and the solution, you know exactly how to organize your day and your staff to achieve the goal of tonight’s soup.
Here is a diagram exploring two key variables: goal and solution. Each can have two values: clear and not clear. Notice how these two variables, each with two possible values, create a simple matrix with four quadrants. Each quadrant represents a “best fit” project management approach. Every possible project that you can conceive will fit into one of these quadrants at any point in time:
Pay particular attention to the bottom two quadrants, where the goal is clear. The bottom half of the matrix is the realm of both Traditional Project Management (also known as Waterfall) and Adaptive Project Management (also known as Agile). The majority of projects fall into one of these two categories. This matrix and way of thinking about project management are ideas that were developed by Robert K. Wysocki in his book Effective Project Management: Traditional, Agile, Extreme, Hybrid, a 600-page textbook on project management well worth investigating.
Waterfall Project Management
When the goal and the solution are well understood, the project is an excellent candidate to manage using traditional, linear project management approaches, often called Waterfall Project Management. Waterfall projects follow a sequential set of discrete and well-defined steps to achieve their goals.
Now, let’s imagine that your restaurant offers a different soup every day based on what vegetables are available. Your team goes to the morning farmers’ market and picks whatever ingredients look best that day. Today it is dill, fennel, and potatoes. Yesterday it was beets and cabbage. In this type of soup du jour scenario, you might require an alternative approach to project management. The goal is still clear, but you don’t know the solution to that goal. Will you need to sauté, steam, or roast? It depends on the veggies! You need to be more flexible in your approach to managing your project since the recipe changes daily, based on the vegetables (and resources) at your disposal. Agile Project Management is a more appropriate approach than Waterfall for complex projects like this that benefit from just-in-time planning.
Extreme Project Management: When the Goal is Not Clear
The two quadrants at the top of the diagram occur less frequently but are still worth understanding. In these situations, the goal is not clear. In these cases, your project management strategy needs to be even more flexible! If you’ve been to the market and found the fresh ingredients you want to use, but you don’t know what you’ll make with them, then you have the components of the solution but no defined goal (other than something to eat). And, if you have no ingredients and no idea what to make, then you’re just brainstorming ideas. In that case, you are in the world of experimentation—essentially R&D—which requires a different project management strategy that focuses a great deal on collaboration and communication.
Agile Project Management
As you move across the spectrum from certainty to uncertainty in your endeavors, each step into complexity requires more human creativity, communication, and thoughtfulness. If, for example, I know the solution and the goal, then the process is straightforward. I’ll still require expertise, structure, and management, but much is known upfront and can be preplanned. I can write a complete detailed specification defining resources, processes, and procedures. I can prepare the stakeholders with a well-defined timeline of deliverables.
If I know the goal but not the solution, I might need some inventiveness, creativity, and ingenuity. The project management strategy needs to be more iterative, flexible, and adaptive to consider new information as we learn the domain and discover new requirements. But the complete set of steps from beginning to end is not well defined, and priorities may need to change during the project. Imagine that we are creating a new mobile app to collect illness data during a pandemic. We know the goal (the data we’re collecting), but we don’t yet see the form the mobile app will take. We might find that the user experience or features we have imagined are not straightforward and need to change after receiving feedback.
Agile Project Management has advantages over the more traditional Waterfall for more ambiguous projects with shorter and more flexible timelines. We can plan as we go, and we anticipate and welcome change. We have all of the significant responsibilities collected in our Backlog. We can still change the tasks that go into our Sprints based on new and evolving User Stories and changing priorities.
Without a well-defined goal or a well-defined solution, projects fall into the “Extreme Project Management” category. Many R&D projects fall into this space. These “Extreme” projects are a unique form of Agile that is even more adaptive. They are hyper-iterative, ultra-flexible, and require techniques that fit this specific situation full of unknowns. These are the “it would be great to be able to” projects. It would be great to cure malaria, but we don’t know how or what form that would take. (Are we talking about stopping the transmission or healing those already sick with the disease? What about reducing the mortality rate without curing the disease?) We might start down the malaria project path and realize that we have made a discovery along the way that defines a new goal. We now have a solution and can reevaluate how we manage the project. Extreme Project Management is similar to Agile, except we need to revise both our imagined goal and our imagined solution as we learn and uncover the details.
Waterfall or Agile?
As we said in the beginning, there is no one-size-fits-all approach to project management. The “best fit” management methodology is dependent on the amount of ambiguity and risk involved in the process and outcome. Agile is not always the best methodology to create a software product on time and within budget. On the other hand, a Waterfall approach may be the best path, given a clear set of customer requirements, a well understood and concrete set of deliverables, and a great deal of experience in the project’s domain.
This article was written as a joint effort between Michael Marmor and Dave Kearney. Dave has been building and scaling companies for 20+ years—driving performance through business-aligned technology strategy, product innovation, and data integration. As a CEO, CTO, and entrepreneur, he is skilled in solving real-world problems for startups through Fortune 500 organizations. He empowers teams to deliver robust, scalable infrastructures, and powerful solutions that automate processes, improve performance outcomes, and set the stage for exponential growth. You can reach Dave at his website Seamless.Partners.
