When we started building a real-time sports odds platform, we thought the biggest challenge would be scraping bookmaker websites.

We were wrong.

Scraping turned out to be just the first 10% of the problem.

The real challenge was building an infrastructure capable of collecting, processing, normalizing, and delivering thousands of live updates every second without falling behind.

Here are five things we learned along the way.