A recommender that has never heard a note of music can be excellent, and one that understands the audio perfectly can be useless. The two approaches fail in opposite directions, and knowing which failure you have is the whole of the design decision.
What collaborative filtering actually uses
Collaborative filtering models the user–item interaction matrix and nothing else. Rows are listeners, columns are tracks, entries are plays or skips or likes. Matrix factorisation learns a low-dimensional latent vector for each user and each track such that their inner product reconstructs the observed interactions; similar tracks end up with similar vectors because similar people played them.
Nothing in that procedure knows what the track sounds like. That is a feature, not a limitation: the latent space encodes co-listening, which captures things audio cannot — that two tracks belong to the same scene, appeared in the same film, are played at the same kind of party, or were released by the same label to the same audience. A cover version in an entirely different genre can sit next to the original because the same people seek out both. No audio model will ever recover that, because the information is not in the signal.
