Why the standard ML playbook breaks in financial services - and a practical architecture for recommendation engines that actually survive production.
If you've built a recommendation system before, you've probably started with collaborative filtering. It worked beautifully on your MovieLens dataset. The precision@K looked great. Your matrix factorization converged cleanly. You felt like you understood the problem.
Then you tried to apply the same thinking to a fintech product - and watched it quietly fall apart.
The interaction matrix was 99.7% empty. Half your users had touched exactly one product in three years. A new savings account launched mid-sprint and the model had no idea what to do with it. Then compliance came back and asked you to explain, in plain English, why your system recommended a high-risk investment product to a retiree. "Because users with similar embedding vectors also engaged with it" didn't go over well.
Here's what we learned building and shipping these systems for financial services clients across lending, banking, and investment platforms: the failure modes were always the same, and they were structural, not incidental. Collaborative filtering's core assumptions - abundant interaction data, stable item catalogs, and no obligation to explain yourself - simply don't hold in this domain.
