This post was co-authored with Khaled Thabit, Meriem Boughrara, and Mohamed Desouky from Property Finder.

The engineering team at Property Finder, a digital real estate application, discovered that 96.8 percent of their evaluation labels had been inherited directly from a single model’s predictions rather than verified by human experts. Their ground truth was systematically skewed toward the incumbent model, making every alternative appear to underperform. The labels also had errors, but the bias was what blocked objective model comparison. They uncovered this when migrating their vision AI workload to a more cost-effective option on Amazon Bedrock.

Property Finder processes tens of thousands of property images every day across the Middle East and North Africa (MENA). Their system evaluates 11 quality attributes per image. These cover watermark detection, room classification, furnishing status, photo quality, and construction indicators. That’s thousands of new property listings with an average of 15 images each. All need quality assessment before going live.

Property Finder needed a more cost-effective option. But measured against those labels, any alternative was at a disadvantage.

The hardest part was not finding a better model. It was discovering that their evaluation methodology itself was the blocker.

In this post, we share how Property Finder identified evaluation bias, built unbiased ground truth, and applied evolutionary prompt optimization to select the right model on Amazon Bedrock, achieving higher accuracy at 86 percent lower cost. We walk through the methodology and iteration examples, with lessons you can apply to your own workloads.

The following figure illustrates the prompt optimization loop that Property Finder and the AWS Generative AI Innovation Center team used to iteratively improve classification accuracy on Amazon Bedrock.

Source link