Background
When streaming AI chat responses, Server-Sent Events (SSE) are commonly used. They are also adopted by APIs from OpenAI and Anthropic, as well as by MCP server responses.
In fact, I implemented several AI chat projects that modified responses from AI platforms while streaming them to the browser. In doing so, I encountered an issue where SSE did not work because of certain intermediary proxies and load balancers, such as AWS App Runner.
After taking a closer look at the SSE specification, I no longer felt that using SSE was right when the purpose was not actually event notification. The API's block data itself is JSON. This is also the same format as the structured logging sent to services such as CloudWatch Logs today (I had already been working on structuring application logs as JSON).
Moving from SSE to JSON Lines Chunked Communication
