Here is the problem, and it is easier to state than most people expect. End-to-end encryption scrambles a message so only the recipient can read it.

That works perfectly, provided you were handed the recipient’s real key. The app fetches that key from a server.

If the server hands you a different key, the encryption still functions exactly as designed. It just encrypts your message to somebody else.

The attack that survives perfect encryption

Signal describes the threat plainly. A key gets swapped out without the owner knowing, for example if somebody compromised Signal itself.