Evolution of Monolithic Systems
The Entropy of Software in Tech Startups
Preface
In my journey through various lifecycles of tech startups, I have observed a recurring pattern of emergent monolithic systems during the early period of a platform’s life. These initial monolithic systems typically start out with good reasons for making smart tradeoffs to optimize for speed. However, for those companies that are resilient and lucky enough to hit product market fit, the demand for scale can arrive overnight. In these moments of scale, the compound interest from tech debt over time will cause enough inertia to significantly slow progress. The friction of flipping from economies of speed to economies of scale are where we find the “Heroic Monolith” that finds itself become a “Distributed Villain”.
You either die a hero or you live long enough to see yourself become the villain.
— Harvey Dent
In this article, I will share my perspective from over two decades of first hand experience with a variety of monolithic systems building web based commercial software. Identifying these signs are crucial to ensuring that an organization can scale successfully and minimize growing pains. My hope is that sharing some hard fought wisdom can provide value for readers to avoid these costly mistakes.
The Original Monolith
At the beginning of every startup is the initial engineer who is tasked with bringing “it” to market. To thrive in ambiguity while executing through adversity — all for the passion to deliver on a mission. The pressure for survival means that moving fast is prioritized over luxuries like ideal technical foundations. After all, the entire endeavor could be over next week so all tradeoffs are made to optimize for speed of delivery.
All of these speed tradeoffs are made in one person’s mind, optimizing the decision-making process to convince only one person — themselves. So with good reasoning, typically one or all of these technical monolithic structures are born.
Technical Monoliths
- Single Repo — co-locating your source code in a single repository
- Single Application — deliver all features in a single application
- Single Service — all functionality exists in a single back end service
- Single Database — all database tables exist in a single schema
For most cases, I believe this is a sound strategy that puts just enough investment to get something out to market. However, without guardrails, technical debt can grow exponentially in proportion to the rate that the company is growing in new team members or number of new features. Here are a few key investments that I highly recommend considering early on to prepare yourself so that future refactoring costs are kept at a minimum.
Key Design Investments
- Engineering Mindset — understand requirements from the frontend to the back end but execute in reverse; build back to front.
- Domain Driven Design—start with purpose built entities between domains and aggregates. (ref)
- Modularized Code —separate your concerns in preparation for refactoring the code out or remote hosting of shared packages. (ref)
- Database Schemas — separate schemas per bounded context.
- Decoupled Service Deployments — regularly ensure you can deploy each component independently.
The Organizational Monolith — A Growing Team
At this point, if the Minimum Viable Product (MVP) gains traction, then the company will likely receive funding and a growth period will occur. Invariably, that means more people will join the team and the vision from the initial engineer needs to be shared.
Early on, when the team is small (2–15 people), typically processes are loose as the overhead in aligning on conventions is manageable. However, as the team grows to the point where it is not efficient to have a single group working on everything, a decision is made to break up and form smaller, more nimble teams to divide and conquer against company goals.
Early in any company’s lifecycle, there is an allure to start in monolithic structures with a mindset to keep things simple (KISS Principle) and then stay lean on every decision (YAGNI Mindset) by delivering the bare minimum (MVP) ; cutting corners on quality along the way.
However, this is an illusion of ignorance because your field of vision only sees as far as the horizon and you are unable to see around the corner. It takes some foresight to realize what is coming and make preparations to proactively meet those those future challenges or be faced with a chain of endless reactive behaviors.
Then the challenge is that the longer you let complexity grow without paying down the tech debt, you start walking towards that scenario where productivity falls off a cliff.
At this stage, the key technical investments to build on are against the original monoliths. If you take these steps sooner, then the cost is cheaper but if you delay, this debt will grow exponentially in proportion to how fast your company is growing.
Key Technical Investments
- Remote Publishing — start publishing shared code to a remote repository despite short term increase in overhead.
- Decoupled Deployments —refactor independently deployable back end services into their own repos.
- Organizational Design — you have a chance to capitalize on the low cost of proactively designing the team for success now, otherwise you will suffer from letting the organization’s structure dictate the type of software you will delivery. (ref)
- Documentation — put time into a wiki with thoughtful information architecture to start building an organizational muscle for writing. Tribal knowledge does not scale and you will see loss of team cohesion.
Seven Technical Monoliths
In this section, I will highlight seven of the most common technical monolithic patterns that I have encountered at startups.
If there is no growth, you don’t need architecture
But if there is, you will be grateful for investing early.
Particularly for organizations that are growing, it is important to watch out for signs when the monolithic strategy needs to adapt to a new scaled environment. The primary technical monoliths that teams need to watch out for are centered around code organization, number of features, deployment strategies and data management.
Classic Monoliths
- (A) Monorepos
- (B) Monolithic Applications
- (C) Monolithic Services
- (D) Monolithic Databases
Advanced Monoliths
- (E) Distributed Monoliths
- (F) Cloud Native Monoliths
- (G) GraphQL Monoliths
Below, we will dissect the anatomy of each monolithic type with cautionary sign posts and remediation advice.
(A) Monorepos
The first type of monolithic structure is the large, single monorepo. To be clear, I am not talking about a small monorepo with limited scope. Rather I am making a case that a single repo to have your entire codebase will eventually become unmanageable with a large team that is working inside of it.
While colocating code has many benefits in its early stages, there is eventually a threshold where those benefits work against you because there are too many concerns all co-mingled together.
- Coupled Concerns — without a distributed mindset, shared code is co-located and lack of versioning when referenced directly means any dependencies are forced into upgrades.
- Governance Challenges — the larger the surface area of the codebase, the slower governing activities become.
- Enabling Diversity — with a large enough scope and diversity of workloads, variations will emerge that start to add concerns into the shared space (think about data engineering code emerging from a product engineering codebase)
There is no hard and fast rule on when you should be making the transition if you started off with a monorepo. Your answer will be dependent on your situation as you are always trading off against paying down future productivity with near term priorities.
The point of the illustration above is for you to establish threshold markers with your team so that you can periodically revisit the decision and look for opportunities to pivot. Without this understanding, the cost of delay will run away from you.
(B) Monolithic Application
In my experience, monolithic applications are on the rare side as most applications are maintainable. I define maintainable as a function of being able to manage changes in a reasonable amount of time which includes building, packaging, deploying, automated and manual tests.
When applications are small, the entire delivery cycle is fast. However, as you add on more features, the increased scope of an application can go from a few minutes to a few hours and before you know it, you are waiting days at a time with a high degree of coordination across a number of teams. The gradual growth is like The Tale of the Boiling Frog which warns us of our psychological tendency to tolerate suffering until there are catastrophic consequences.
So the metric to keep an eye on is how long your delivery cycles take and how many teams need to be involved. And as the time takes longer, the team takes on more risk to do “big bang” releases due to the sheer number of changes. It creates a virtuous cycle of ballooning deploy times and humans to test everything.
Investing in a strategy to break up the application into independently deployable units would allow teams to work and deploy independently. Carving out a coarse set of functionality while trying to minimize a poor user experience and facilitate independent deployments.
(C) Monolithic Service
Monolithic Services represent a scenario where back end API Services are built without foundational strategies like Domain Driven Design. This can result in a codebase with mixed concerns that grow into a Big Ball of Mud Architecture. This gradually leads to slow down as leveraging code is penalized with system stability issues because of poor boundaries — changing behavior in one part of the system can have unforeseen issues in another part of your system.
This makes future decoupling work much harder because the foundation of these back end architecture has flawed entities. Typical smells I look for include the following.
- Entities that have fields that are adjectives
- Entities that have fields that are verbs
- Entities that have a suspicious large number of fields
- Lack of layers in code
- Inconsistent access patterns
The unfortunate situation is that these challenges are typically more structural in nature and these are the most expensive to course correct. Not only do you need to redraw the domains but now you have to refactor code and perform data migrations.
The best advice I have is to dedicate a team to take inventory of all of your back end systems, prioritize the parts of the system that are harming the business operations or blocking future business opportunities to fix.
- Deploy a WET Strategy (Write Everything Twice) to extract your functionality into its own service.
- Use the Strangler Fig Pattern to refactor references in the Ball of Mud to use the newly extracted service.
- Finish it with DRY (Don’t Repeat Yourself) by removing the original code.
It is worth noting that not all technical debt is worth paying off. Deprioritize the rest and only work on it has more business value than other work.
(D) Monolithic Database
One of the most common monolithic patterns gone wrong is the shared monolithic database with a single giant schema. The earliest shortcut that I have personally been guilty of is to establish an honor system that we will not mix our domain concerns. Inevitably, there will be a new hire or an existing team member who has to make a decision to hit a deadline or abide by the original agreement. The problems of this anti-pattern spans potential security, privacy and costly refactors.
One solution is to create separate credentials and isolate on the schema level. This way, you do not prematurely scale the number of databases which can be very costly. The extra rigor here will help establish a pattern that will facilitate faster and safer delivery in the future with just a few extra days upfront to set up.
(E) Distributed Monolith
Distributed monoliths occur when your code is split out and deployed yet there are no structured rules around how dependencies are managed — resulting in tight coupling between services. This is the proverbial Big Ball of Mud Architecture where it becomes impossible to reason through how changing one part of the system will impact its dependencies. Over time, development gets slower and every deployment makes the system more brittle.
A good strategy to start to untangle this large ball of yarn is to establish guiding principles around the directed flow of dependencies and data. An example would be in a simple 3-tier system for a web application, you might lay down logically bounded layers.
- Rule 1: services optimized for aggregate business logic
- Rule 2: services optimized for domain based business rules and data persistence
- Rule 3: traffic always flows south and does not skip layers
By having these rules in place, your team can scale independently by fitting their uses cases in the appropriate lane.
(F) Cloud Native Monoliths
In the modern cloud native landscape, it has never been easier for a single developer to bring up an entire tech stack with a single command. These all-in-one wonders can deploy applications, services and all of its requisite infrastructure.
With great power comes a greater chance for a cloud native monolith.
The challenge with these modern vendor abstractions are that when they are overused in the place of traditional computing techniques. The result is a Big Ball of Infrastructure Mud that pushes the problem space into the infrastructure team.
In the illustration above, we run into issues when the dependencies or flow of data jumps out of one context and creates a tight infrastructure coupling into another. When this occurs and the pattern replicates, you start to see new type of distributed monolith emerge — one that is not only based on software domain coupling but also on the cloud infrastructure level.
The recommendation is to create hard rules not to leverage frameworks for any core components that are meant to be leveraged or shared. The moment these requirements arise, you should invest into decoupling your code away from your infrastructure.
(G) GraphQL Federated Monolith
While traditional API architecture can grow large with the number of available endpoints, this type of monolith is specific to GraphQL because of the power that the technology has for traversing the entire API. So the final monolith I want to highlight is when GraphQL Federation is used as the a single monolithic API for everything.
The challenge comes when federation grows to the point of being ungovernable. Over time as you add more APIs, you have to govern those changes to ensure that you did not introduce something that you wouldn’t want combined with another part of the graph.
If you have a handful of subgraphs, you could fit those few endpoints in your head to govern all possible traversal routes but at some point it becomes unmanageable. Some other team is going introduce something in a rush for their delivery that opens risk to another part of the graph that has not changed for years. In order to govern effectively, everyone that is publishing to the graph has to review everything — creating a high degree of cognitive drag on the organization.
The recommendation is if there is a desire to leverage federation, avoid the tendency to throw everything into a monolithic Supergraph. Instead, draw boundaries of intent around the purpose of each federated graph. These could be drawn by application, application functionality, scoped around security isolation and more.
Closing Thoughts
Monolithic structures hold great power to get things started. The cautionary tale here is that there are no free lunches especially when your company is successful and scales up. When complexity grows, you should be ready to restructure your systems so that your teams don’t get bogged down by the inertia of monoliths.
Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.
— Melvin E. Conway
The observation here is that while Conway’s Law is typically discussed within the framing of organizational design but I believe it goes both ways. The monolithic design of the system could be forcing bottlenecks in the flow of communication and changes in the organization.
