A View From Egnyte Engineering

Scale and ScalabilityBy Krishna SankarThe other day we were observing that scale is a tough master but a delightful slave. If scale rides you, there is no place to hide. As Joe Louis puts it, "You can run but you can't hide." But once you master scale, things start coming together.The whole philosophical musing started as we embarked on fitting in a new Object Store Layer. It is not that our current infrastructure is bursting at the seams, but we wanted to get ahead of the scale that we will face in the very near future.Conceptually an object layer is concise:

• It accepts objects from multiple applications and stores them.
• It has a metadata service that manages the distribution as well as location and access.
• Given an object ID, it returns the object to the application.

Life would be much easier if the feature set ended here. Fortunately or unfortunately, life is never that simple.

An object store also needs to guarantee integrity of the objects it holds against coordinated and random failures, even byzantine failures, which adds replication, object auditor, rebalance, storage failure recovery and so forth. One interesting characteristic about scale is that one has to expect and embrace failure and work in spite of it.

So an innocent excursion into object store suddenly turns into a universe of databases, multiple sharding, consistent hashing, monitoring, coordinated consensus, caching, Fisher-Yates shuffle, soft deletes and smart object keys.

We will visit these and other interesting challenges from the land of scale, coming to your favorite blog this summer.