Vasilios Syrakis - system designA simple blog made with Zola and DuckquillZola2026-07-11T00:00:00+00:00https://vsyrakis.dev/tags/system-design/atom.xmlHow to Avoid Sharding Yourself2026-07-11T00:00:00+00:002026-07-11T00:00:00+00:00
Vasilios Syrakis
https://vsyrakis.dev/blog/how-to-avoid-sharding-yourself/<p>I watched a video named <a rel="external" href="https://www.youtube.com/watch?v=rxR4nIQ0hCk">The Problem Sharding a Database
Solves</a> by <a rel="external" href="https://www.youtube.com/@WebDevCody">Web Dev
Cody</a>.<br />
He has quite a large number of subscribers, his video popped up on my feed, and
I enjoyed the video.</p>
<p>It’s about splitting up your database and the advantage of doing so, but it
introduces a small problem and that is the question of how to ensure that each
user accesses the correct database shard.</p>
<img class="no-hover"alt="screenshot of cody's video"src="/img/cody.png"/><small>
Also, the diagramming tool he's using is cool and if anyone knows what it is -
please tell me!
</small>
<p>However, one thing stuck out to me that I wanted to intercept in order to
provide advice on, based on my own real professional experience.</p>
<h2 id="what-caught-my-attention"><a class="zola-anchor" href="#what-caught-my-attention" aria-label="Anchor link for: what-caught-my-attention"><i class="icon"></i></a>
What caught my attention</h2>
<p>Cody delivers on what I think is the point of his video - to outline the advantage of sharding.
But there is an example for how to implement routing a tenant to the shards and
this is what I’ll be providing advice on, based on my professional experience.</p>
<p>Cody describes a naive approach where you take some kind of tenant identifier
and hash it, which gives you a deterministic key which maps to the shard that
the tenant lives on.</p>
<p>He then rightly points out that if you pick this approach, as the number and
size of tenants grows you’ve now made it harder to rebalance your shards, or
migrate tenants, because when you insert new shards, the deterministic hashing
essentially breaks.</p>
<p>This leads Cody to describe a possibly more elegant approach called “virtual
buckets” where instead of a hash being a 1:1 mapping between a tenant and a
shard, you take the modulus of the hash which maps a range of tenants to a
shard instead.</p>
<h2 id="algorithmic-routing-can-be-an-anti-pattern"><a class="zola-anchor" href="#algorithmic-routing-can-be-an-anti-pattern" aria-label="Anchor link for: algorithmic-routing-can-be-an-anti-pattern"><i class="icon"></i></a>
Algorithmic Routing can be an Anti-Pattern</h2>
<p>These methods of routing tenants to shards aren’t broken, but let’s call them anti-patterns.</p>
<p>First, let’s talk about why hashing is an attractive option.</p>
<ul>
<li>It’s stateless</li>
<li>It’s fast</li>
<li>It doesn’t need a database lookup</li>
</ul>
<p>But it leaves you open to <strong>operational pain</strong>, even with the use of virtual buckets.</p>
<h4 id="one-tenant-within-a-bucket-can-dwarf-other-tenants"><a class="zola-anchor" href="#one-tenant-within-a-bucket-can-dwarf-other-tenants" aria-label="Anchor link for: one-tenant-within-a-bucket-can-dwarf-other-tenants"><i class="icon"></i></a>
One tenant within a bucket can dwarf other tenants</h4>
<p>If we just assume that the Pareto principle will play out eventually, you’re
going to have one tenant which consumes 80% of a single shards capacity.</p>
<p>In order to rebalance them, you need to change their identifier. Now you have
to choose between accepting the noisy-neighbour problem where a huge tenant
fights for dominance with the other small tenants on the shard, or you add a
special-case to your hashing algorithm that lives on forever, and most likely
grows, needing to be maintained by operators.</p>
<h4 id="no-flexibility-for-isolation"><a class="zola-anchor" href="#no-flexibility-for-isolation" aria-label="Anchor link for: no-flexibility-for-isolation"><i class="icon"></i></a>
No flexibility for isolation</h4>
<p>This is semi-related to the above, essentially, if you wanted to have
different shards, like a “quarantine” shard for misbehaving tenants, or a “free
tier/trial” shard, or an “important enterprise customer” shard with only 1
tenant on it, you again have to special-case the algorithm.</p>
<p>These requirements will predictably appear as you grow and get more customers.<br />
Maybe they won’t, but we can take care of this with very little operational
friction as we’ll see later.</p>
<h4 id="rebalancing-friction-doesn-t-go-away"><a class="zola-anchor" href="#rebalancing-friction-doesn-t-go-away" aria-label="Anchor link for: rebalancing-friction-doesn-t-go-away"><i class="icon"></i></a>
Rebalancing friction doesn’t go away</h4>
<p>Cody does point out that even when you add new shards, you still might need to
rebalance tenants, but you can maybe do this with some background process or
lazy migration that writes to both shards at the same time.</p>
<p>For some businesses these might be acceptable operational events that someone
has to take care of, perhaps after-hours or during scheduled downtime.</p>
<p>Note: you <em>can</em> mitigate this particular downside with <a rel="external" href="https://en.wikipedia.org/wiki/Consistent_hashing">Consistent
Hashing</a> but the others are
still unsolved.</p>
<h2 id="use-a-lookup-table-for-tenant-routing"><a class="zola-anchor" href="#use-a-lookup-table-for-tenant-routing" aria-label="Anchor link for: use-a-lookup-table-for-tenant-routing"><i class="icon"></i></a>
Use a Lookup Table for Tenant Routing</h2>
<img class="no-hover"alt="diagram of architecture that uses a lookup table"src="/img/lookup-table.png"/>
<p>I know it’s appealing to avoid adding infrastructure to solve a problem, but I
think this situation justifies the addition of a database to act as a lookup
table or directory for your tenants.</p>
<p>For the vast majority of businesses, a dedicated Redis or Postgres instance can
serve this purpose and take you very far.</p>
<p>Yes, it can be a single point of failure, and when it goes down so does your
entire app probably, but do a little back of the napkin calculation on the
probability of redis (in a cluster, with high-availability), becoming suddenly
unavailable for any serious amount of time. It probably won’t go under 3 nines
of availability.</p>
<h4 id="surgical-rebalancing"><a class="zola-anchor" href="#surgical-rebalancing" aria-label="Anchor link for: surgical-rebalancing"><i class="icon"></i></a>
Surgical Rebalancing</h4>
<p>Your rebalancing goes from a stressful situation of changing the algorithm,
propagating it to your gateways and praying, to a 5 step process that an intern
can execute without bringing the company down.</p>
<ol>
<li>Identify a big noisy tenant</li>
<li>Provision a new shard</li>
<li>Run a background job to copy data, stream new writes in the meantime</li>
<li>Switch to the new shard atomically</li>
<li>Wait a week and delete the old copy</li>
</ol>
<h4 id="better-isolation"><a class="zola-anchor" href="#better-isolation" aria-label="Anchor link for: better-isolation"><i class="icon"></i></a>
Better Isolation</h4>
<p>Before, changing the way hashing works meant that you were changing the
algorithm for <em>everyone</em>.</p>
<p>One mistake and everyone is cooked.</p>
<p>With a lookup table, you’re changing a single record.</p>
<p>You also unlock the scenario discussed above where you can move your enterprise
customers to their own shard, or throw the naughty tenants into quarantine so
they don’t annoy everyone else.</p>
<h2 id="fives-nines-of-availability-no-i-said-nine-fives"><a class="zola-anchor" href="#fives-nines-of-availability-no-i-said-nine-fives" aria-label="Anchor link for: fives-nines-of-availability-no-i-said-nine-fives"><i class="icon"></i></a>
Fives nines of availability? No, I said nine fives…</h2>
<p>If you’ve somehow reached the point where that lookup table is no longer
sufficient (and you’re still reading this blog? lol), or you just feel
uncomfortable with having to make a lookup on every request to a single point
of failure, there’s a pattern you can employ to scale things further and bring
the probability of an outage to a number approximating zero.</p>
<p>You can actually read about
<sup>
<a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/aws-scaling-multi-region-low-latency-service">1</a>
<a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/atlassian-critical-services-above-six-nines-of-availability">2</a>
</sup>
how Atlassian solved this exact problem using the
<a rel="external" href="https://en.wikipedia.org/wiki/Command_Query_Responsibility_Segregation">CQRS pattern</a>.
However, I’m going to explain it here even more concisely.</p>
<p>CQRS, or “Command Query Responsibility Segregation”, can enable a caching
pattern where distributed nodes are able to keep a local copy of the data that
they can query, cutting the latency of tenant lookups to practically zero, and
allowing them to continue to operate while the central lookup table is offline.</p>
<p>Put even more simply:</p>
<ol>
<li>Something writes to the central lookup table (the Command)</li>
<li>The central lookup table publishes an event to a broker</li>
<li>The nodes pick up the event (the Query)</li>
</ol>
<img class="no-hover"alt="diagram of an architecture that demonstrates cqrs"src="/img/cqrs-example.png"/>
<p>An example of a broker could be an SNS topic, or a version number on an API, or
whatever else you want to use, as long as it’s cheap for nodes to access and
backed by something highly available.</p>
<h3 id="a-contrived-example"><a class="zola-anchor" href="#a-contrived-example" aria-label="Anchor link for: a-contrived-example"><i class="icon"></i></a>
A Contrived example</h3>
<p>Imagine an isolated scenario on a single machine where there’s a database, an
application with its own sqlite cache, and a file containing a number to
indicate what version the data is at currently.</p>
<p>When a row is added or changed in the database, it increments the number in the
file. The application has a background thread checking the number in the file,
and notices that it’s been incremented, so it replaces the sqlite cache
atomically.</p>
<p>This entire time, the application has been able to read from its local sqlite
database uninterrupted.</p>
<p>The same concept applies to your distributed tenant routing. The central lookup
table acts as the source of truth, but your application nodes or API gateways
can maintain their own fast, resilient local caches of the routing rules.</p>
<h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a>
Conclusion</h2>
<p>While clever routing to shards might seem appealing at first, saving you an
extra database lookup and avoiding a new piece of infrastructure, it introduces
operational complexity as your application scales and your tenants grow.</p>
<p>A lookup table is dead simple, painless, and gives you flexibility for later
on. If you need the scale, you can add it later with zero downtime.</p>
<p>Often, the “boring” infrastructure choice is exactly what you need to avoid
sharding yourself.</p>