How we optimized Azure Cosmos DB in our application storage platform, Habitat, to manage unprecedented growth.
All OpenAI products require fast, reliable access to data, provided by our storage platform, Habitat. It has to respond in milliseconds to over 1 billion users across 40 geographic regions. Our infrastructure faced an extraordinary challenge: supporting explosive growth and new use cases while continuing to improve reliability, performance, and product developer velocity. Solving that challenge required rethinking how our online storage platform was designed and operated. We constantly evaluated tradeoffs, following the pattern of: first become reliable enough for mission-critical product traffic, then fast enough for global users, and, finally, to deftly operate at massive scale.
This two-part series explains how we developed Habitat from a simple client library and database into a globally distributed storage platform serving over 500 petabytes of data. Part I focused on Habitat’s evolution from a Python client to a service. It also covered the techniques that allowed us to push a Python-based service far beyond what many would consider practical.
In this post, Part II, we walk through how we scaled the underlying storage component: Azure Cosmos DB. Through a close working partnership with the Azure team, we arrived at a viable multi-tenancy approach of co-managed containers, with excellent read performance at massive, global scale.
Habitat is the online storage platform we built so OpenAI products can quickly and reliably access needed information.
As companies scale, infra teams often evolve from running single-tenant systems to multi-tenant ones.
Source link
