Data Whomp

---

01

Why Most Projects Struggle To Be Completed On Time and On Budget

Yemen K Dickinson | May 13, 2025

Projects are how things get done—this is not earth-shattering information, especially if you are reading this. Whether your team is developing official government security software or you are making dinner for friends, the process is the process. It involves managing time, cost, and scope—the triple constraint. But why do so many projects struggle to be completed when expected? Dinner for friends is scheduled at 7 o’clock, but inevitably is not served until 7:30 PM. What went wrong? Whether it is the roast that took longer to cook, or the friend who is always late, there is always something that we did not plan for or underestimate.
Having the project run over schedule and/or over budget results in not just hungry guests, but could mean millions of dollars lost or a security breach. According to PMBOK, the project manager’s primary purpose is to lead the team and ensure the project achieves its objectives by planning, organizing, and directing the project work. Seems like the planning, organizing, and directing are a little off most of the time if these projects are not finished on schedule. How does this happen? And what is the best plan to help mitigate the overrun? 

Nobel Prize-winning psychologist Daniel Kahneman, in his book “Thinking Fast and Slow,” coined the term planning fallacy—the overly optimistic plan that is eager for approval, rather than realistic completion. This is all too common in project management and seen from projects big and small, whether a stakeholder who is eager for a great quarterly report to quiet investors, or the contractor who promises the house addition in 60 days, just to get the job. These plans are not comprehensive and exhaustive in their approach. In “Why Good Projects Fail,” a Harvard Business Review article by Nadim F. Matta and Ronald N. Ashkenas, the authors mention a similar phenomenon called white space, where planners leave gaps or fail to plan. Think of the dinner roast that was not done by 7 o’clock. Usually, the simple calculation of lbs/hour would have been enough, but this time your partner forgot to thaw the roast the night before, thus resulting in a longer cook time. 

When scope creep occurs (the ever-expanding unauthorized additional features), there is no secret as to why your project is over schedule and budget. One of your friends showed up with some buddies from work, and now you have to feed 8 instead of 5. The big question is, how do you manage it? In many respects, combating scope creep starts in the planning phase, and it is no wonder the planning fallacy comes back to bite us. We should plan for the executive who always seems to add unauthorized features by perhaps getting him more involved in the planning phase, if possible. PMI suggests that an effective Scope Management plan can help clarify the roles and responsibilities, and allows for any additional features that must be added in an appropriate manner that considers time and budget.

So, what are the best ways to keep your project on time and within budget? A 2025 MIT Sloan Management Review article covers reference class forecasting (RCF), which won Daniel Kahneman the Nobel Prize in 2002. This forecasting method is for addressing uncertainty and is an excellent tool for more accurate predictions. It uses an “outside view,” where we learn from similarities of the reference class (the historical situation or project) to improve the accuracy of timelines, budgets, or resource estimates, while the inside view relies on the goals or gut, leading to missed milestones and more money.

The next time you are planning a friend’s dinner, refer to an outside view. You know the one friend who always brings extra co-workers, and the other is at least an hour late. Rely less on your feelings of creating the perfect dinner without using the historical information that will inevitably have you eating 2 hours late and spending extra money out of pocket.