Crawl, chunk, embed and serve semantic search over a site — with a re-index job that only re-embeds the pages that actually changed.
The hard part of site search is not the search. It is the second run: five thousand pages, of which twenty changed, and an index job that must not spend an hour and a bill re-embedding the other four thousand nine hundred and eighty. Content hashing per chunk turns that into a job that finishes in seconds.
Four jobs, not one script
Separate them, because they fail differently and run on different schedules.
Job
Description
A real production pipeline that ships SEO content at scale. The architecture, the code, and the trade-offs I learned the hard way…
You're building a semantic search feature for a B2B SaaS product. The corpus: 4 million support...
Building a semantic search engine for an e-commerce catalogue doesn't require a team of PhDs or a...
Search returns things you have to read. An answer engine reads them for you. I built a search engine...
Bottom line: for a beginner ask-your-docs feature in a SaaS help center, I would start with...
AI answer engines — Google's AI Overviews, ChatGPT Search, Perplexity, Gemini — increasingly answer...
