These are the two AI tools I used to ensure mine ran smoothly, efficiently and retained the right knowledge.
During the World Cup this year, I built Rune Score, a sports analytics chatbot. From the outside, it looks like a simple chatbot. Under the hood, it’s a distributed system that has to handle the freshness of sports data, user rephrasing, extraction failures, LLM call costs, streaming, memory, fallbacks and security. What I learned building it applies to almost any AI product that has to stay fast, cheap and accurate at the same time.
Two quick definitions before we go further, since neither tool is exactly a household name. Redis is “an open-source, in-memory data structure store used as a database, cache, message broker and streaming engine,” most commonly used in AI products as a fast cache that stores one exact answer under one exact label. RAG, short for retrieval-augmented generation, is defined by AWS as “a technique used to augment a large language model (LLM) with external data, such as a company’s internal documents.” Instead of matching an exact label, it searches for a broader knowledge base using meaning rather than exact wording. Redis is precise and cheap but rigid. RAG is more flexible and durable, but costs more to run.
My first move wasn’t a vector database, it was a Redis cache built around a simple labeling rule: Identical requests get an identical key, something like the football matchup Argentina vs. Spain. The savings were immediate. As I documented in my LinkedIn article on Rune Score , “without the cache, a fresh extraction followed by an analysis can consume up to 23,000 input tokens: scraped content, context, history.
Source link







