At Exa, we are building one of the few true web indexes by aggregating and serving knowledge from across the web, along with private and synthetic sources. We optimize the index for semantic search, grounding agents and humans in true, high-quality data. We use multi-tower semantic machine learning (ML) models to embed web data and query data into the same vector space, so users can express complex queries in natural language alongside traditional keyword search. The keystone technology behind this is an in-house vector database. It must ingest as much high-quality web data as possible and embed all of it. The unit economics of embedding therefore underlie everything we do. At the scale of tens or hundreds of billions of documents, finding the hardware that delivers the best performance per dollar isn’t a tuning detail. Any per-document cost improvement compounds into a significant number.
This post details how we optimized our embedding infrastructure for cheaper, more flexible inference workloads on AWS Neuron hardware (AWS Inferentia2 and AWS Trainium2), and how an agentic development workflow drove the optimization.
Inference runs on an asynchronous pipeline, but it must be high-volume and fast to serve recent data quickly and shrink the time between discovery and serving.
Every structured web document we rank as high enough quality must be embedded before it can be served. This figure shows how raw web data moves from Crawlers into an asynchronous pipeline of Parse, Moderate and filter, then Embed, with the result served from a vector database.
Source link







