I recently read Jérémie Rodon's excellent article On-Demand Archives on S3, where he describes an elegant Rust solution for zipping 3,000 × 5MB files from S3 within a single Lambda function.
His approach is impressive: streaming a ZIP archive through a custom Rotating Slab Buffer, saturating bandwidth with concurrent downloads, all within 512MB of RAM. The result: 3 minutes 35 seconds.
I thought it was a good challenge to reach better performance. His article ends with an open invitation: "do you think you can do better with your favorite language?" Well, my favorite language is not Rust nor Go nor.. however, I'm fluent in serverless ;) so I took a different angle entirely.
A Different Approach: Why Not Parallelize the Problem?
Jérémie's constraint was a single Lambda. That's elegant, but it means you're bound by one machine's network bandwidth (~600 Mbps). No matter how perfect your streaming is, physics wins: 15GB at 600 Mbps ≈ 200 seconds minimum.
