![]() ![]() Perhaps we could variable-length encode some pieces of metadata, or remove others primarily used for debugging and etc. We investigated and explored a number of options.Īmong them was an option to optimize Edgestore’s existing data model, shaving off some bytes here and there on schema overhead. In the meantime, we wanted to avoid the next round of splits, giving us roughly 2 years to solve Edgestore’s capacity crunch. We began to build a replacement metadata storage engine, called Panda, underneath Edgestore that would support incremental capacity expansion, allowing us to add just a handful of clusters at a time and dynamically move subsets of data to the new clusters, but we knew this would be a multi-engineer-year project to build, validate, and migrate. Our levers for capacity expansion were quickly closing and we were heading into a capacity crunch. Second, buying and setting up so many machines at once is an all-hands-on-deck project for our hardware, datacenter, networking, and database teams and is hard to scale.Īrchitecturally, Edgestore would also only be able to perform 2 more splits, after which we would reach 1 shard per cluster with no more splits available. First, the financial cost for the exponentially larger and larger splits is a non-trivial amount of money to earmark. ![]() This was logistically problematic from a couple of perspectives. Increasing usage of Edgestore from various products, coupled with the growth of Dropbox itself, was causing the rate of splits to increase. This increased Edgestore’s physical footprint from around 1500 database hosts to more than 3000 database hosts after accounting for local and geo-diverse replicas.įrom our capacity projections at the time, we predicted the need to split yet again in under just 2 years. Capacity CrunchĪround fall of 2017, we split Edgestore from 256 clusters with 8 shards per cluster to 512 clusters with 4 shards per cluster. This means that capacity expansion most often occurs by splitting the entire fleet, doubling the physical cost of Edgestore each time. Assuming data is relatively evenly distributed among the hosts, when one cluster hits 80% disk space and needs to be split, most other clusters will also need to be split. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |