My experience is the opposite. I worked at SendGrid for a decade and we scaled the engineering org from a dozen to over 500 operating at scale sending billions of messages daily. Micro services. Well, services. The word micro messes people up.
I have also worked at half a dozen other shops with various architectures.
In every monolith, people violate separation of concerns and things are tightly coupled. I have only ever seen good engineering velocity happen when teams are decoupled from one another. I have only seen this happen in a (micro) service architecture.
Overall, in my own assessment, the decision to stick with a monolith has slowed down velocity and placed limits on scale at every other company I have been at and require changes towards decoupled services to be able to ship with any kind of velocity.
The place I just left took 2 years, over 50 teams, and over 150 individual contributors to launch a product that required us to move over an interface for sending messages from ORM querysets to DTOs. We needed to unlock our ability to start rearchitecting the modules because before it was impossible to know the actual edges of the system and how it used the data. This was incredibly expensive and hard and would never have happened but for the ability to reach into other's domains and making assumptions making things hard.
Don't couple systems. Micro services are the only arch I have seen successfully do this.
You say that every monolith you’ve seen has devolved into bad engineering — coupling, crossing boundaries. What was missing that could have stopped this? A missing service boundary you’d say, but also a lack of engineering leadership or lack of others’ experience? No code review? A commercial non-founder CEO pushing for results at the expense of codebase serviceability? Using a low-ceremony language (no types, no interfaces)?
You can stop coupling by enforcing boundaries. Repository boundaries are extremely solid ones. Too solid for some people, making it unnecessarily hard to coordinate changes on either side of the boundary. Barely solid enough for others, where it’s clearly too dangerous to let their devs work without solid brick walls keeping their hackers apart.
Coupling, smudged boundaries, and incoherence are symptoms of something more fundamental than simply we didn’t use services like we should have. If everyone’s getting colds off each other it’s because of bad hygiene in the office. Sure, you could force them to remain in their cubicles or stay at home but you could also teach them to wash their hands!
Generalizations don't help almost any discussion. Even if that's 100% of what you ever saw, many people have seen mixed (or entirely in the other extreme).
> In every monolith, people violate separation of concerns and things are tightly coupled. I have only ever seen good engineering velocity happen when teams are decoupled from one another. I have only seen this happen in a (micro) service architecture.
I would write this off as indifferent or incompetent tech leadership. Even languages that people call obscure -- like Elixir that I mostly work with -- have excellent libraries and tools that can help you draw enforceable boundaries. Put that in CI or even in pre-commit hook. Job done.
Why was that never done?
Of course people will default to the easier route. It's on tech leadership to keep them to higher standards.
Funny you mention Elixir. At one company, we passed around Ecto querysets. It started when the company was smaller. Then someone needed a little bit of analytics. And a few years of organic growth later and the system was bogged down. Queries where joining all over the place, and separating out the analytics from everything else was, again, a major undertaking.
I would love to see a counter example in real life at an org with over a dozen teams. A well working monolith and a well working monorepo are like unicorns; I don't believe they exist and everyone is talking about and trying to sell their mutant goat as one.
I am not selling you anything so you're starting off from a wrong premise.
What I said is that you should consider your experience prone to a bubble environment and as such it's very anecdotal. So is mine (a mix), granted. Which only means that neither extreme dominates out there. Likely a normal bell curve distribution.
What I did say (along with others) was that a little bit of technical discipline -- accentuating on "little" here -- nullifies the stated benefits of microservice architecture.
And it seems to me that the microservice architecture was chosen to overcome organizational problems, not technical ones.
My experience doesn't align with yours. I worked at SendGrid for over a decade and they were on the (micro) service train. I was on call for all dev teams on a rotation for a couple of years and later just for my team.
I have seen like a dozen security updates like you describe.
This was at a fintech and we took every single little vuln with the utmost priority. Triaged by severity of course, but everything had a ticking clock.
We didn't just have multiple security teams, we had multiple security orgs. If you didn't stay in compliance with VULN SLAs, you'd get a talking to.
We also had to frequently roll secrets. If the secrets didn't support auto-rotation, that was also a deployment (with other steps).
We also had to deploy our apps if they were stale. It's dangerous not to deploy your app every month or two, because who knows if stale builds introduced some kind of brittleness? Perhaps a change to some net library you didn't deploy caused the app not to tolerate traffic spikes. And it's been six months and there are several such library changes.
This Segment team was 3 people and 140 services. Microservices are best at solving org coordination issues where teams step on each other. This is a case of a team stepping on itself.
I left Twilio in 2018. I spent a decade at SendGrid. I spent a small time in Segment.
The shitty arch is not a point against (micro)services. SendGrid, another Twilio property, uses (micro)services to great effect. Services there were fully independently deployable.
The grandson of the 10th US president died this year. The US is barely three generations old. That guy could say his grandfather shook hands with Thomas Jefferson.
If you have never played the backwards-lifetime game, you should.
Take your current age and work backwards that same number of days, months and years before your birth. Every year something else remarkable is added.
At my current backwards age, World War II is the best part of two decades away; the UK is still recovering from World War I. Rocket 88, the first rock and roll song, won't be written for nearly another three decades. Women still can't vote in the UK, the Wall Street Crash is several years away.
When my father (who knew one of the most important men in medical history in his younger days and who was working in medicine not long after the NHS was founded) died, his backwards age reached back before the germ theory of modern medicine.
Another interesting game is to use your "oocyte age" — about 32 weeks before your mother was born is roughly when the oocyte developed that led to your egg. In my case this too is before World War II started.
If you look at this proposal and reject it, i question your experience. My experience is not doing this leads to codebases so intertwined that organizations grind to a halt.
My experience is in the SaaS world, working with orgs from a few dozen to several thousand contributors. When there are a couple dozen teams, a system not designed to separate out concerns will require too much coordinated efforts to develop against.
reply