Understanding "Voice" — Breaking Down the Components
When trying to convert your recorded voice into someone else's, the first question that comes to mind is:
What exactly is a "voice"?
Even if the same words are spoken, different speakers produce different sounds. Even the same speaker produces different sounds depending on what they're saying. Adding intonation changes it further. In other words, speech is a signal composed of multiple independent pieces of information mixed together — at least: who is speaking (speaker identity), what is being said (content), and how it's being said (prosody).
The core challenge of voice conversion (VC) lies here. If you naively process speech to replace only the speaker identity, the content and intonation often get altered as well. In this article, we'll explain how Seed-VC, which we adopted for the backend of our "voice design" app, solves this problem — by separating and handling information using four modules, while walking through the actual model loading and inference code.
