Google announced a faster way to generate high-quality query fan-outs that can scale to production-ready search.

Google has announced a new query fan-out framework that is faster, less computationally expensive and delivers higher quality fan-outs. The new system is said to deliver “production ready” search at scale.

The new system, called Retrieve-for-Train-Diffusion (R4T Diffusion Model), is a three-stage setup that combines reinforcement learning (RL) training, synthetic data generation, and a small generative neural network (a 53.9M-parameter diffusion model).

What they did was train a model on what computationally expensive query fan-out behavior looks like, save examples of high-quality query fan-out outputs, then train a significantly smaller model to copy the behavior of the larger model.

R4T generates better query fan-outs because it’s trained to identify useful aspects of the original search query. It keeps the fan-outs relevant to that query but with diversity in that it avoids generating redundant synonyms.

The researchers explain that the weighting of the model during training optimizes it:

“For our open-ended abstract retrieval tasks, this composite reward is a weighted balance of three competing pillars:

What Google’s researchers did was use an approach called distillation. Neural-network knowledge distillation, a landmark idea that ex-Googler Jeff Dean helped pioneer in 2015, is the transference of a large model’s behavior to a significantly smaller model. This is done by training a smaller model on the outputs of a larger model, giving the smaller model the ability to accomplish virtually everything the larger model can do but at significantly less computational cost.

Source link