Full-scale streaming evaluation without the infrastructure liftFor a team trying to find out whether Snowpipe Streaming holds up at 1M+ TPS, you want the shortest path to a real answer. Streaming at scale has a reputation. It's the workload teams put off, because standing it up entails S3 buckets, an EKS cluster, Kafka topics, IAM policies, a security review and sign-off from three platform teams who each have their own backlogs. That setup can require additional infrastructure provisioning and coordination before testing begins. Proving these pipelines work at the scale and handle production volume is usually its own project.In our demo environment, we were able to set up and run this evaluation in an afternoon. You can use our Snowpipe Streaming High-Performance Architecture to stream data into Apache Iceberg™ format at over 1M TPS, entirely inside Snowflake infrastructure. A container running in Snowpark Container Services (SPCS) generates the load, streams it through the SDK and lands it in a Snowflake-managed Iceberg table. That makes it ready to query in Snowflake right away, with role-based access control (RBAC), lineage and masking all applied as it arrives. In our test environment, we stood up the demo, tested it end to end, measured throughput and tore it down in an afternoon, as we did in our test.This post will walk you through how to actually build an end to end streaming to Iceberg demo, using this GitHub Repo. This example is in python but you can test the SDK options (Java, Python, Node, REST), so that you can credibly recommend this to your business with real results to show.Architecture diagram: The patternWe wanted to build a demo where everything runs inside Snowflake infrastructure. Multiple Docker containers running in Snowpark Container Services (SPCS) generate synthetic test data at your target transactions per second and stream it using the Snowpipe Streaming SDK. It lands in a Snowflake-managed Iceberg table: open format, queryable within seconds of landing, governed in Snowflake. Authentication is easy: SPCS injects a short-lived OAuth token into the container telling the Streaming SDK to use it. You don't need to manage any secrets.Figure 1: Snowpark Container Service to IcebergWhy SPCS to generate the load?To be clear, Snowpipe Streaming doesn't need SPCS. The SDK runs anywhere: your laptop, an EC2 box, a Kubernetes pod.Putting it in SPCS is what gets you to the test faster since the service runs entirely on infrastructure Snowflake already manages, with nothing new to provision or additional sign-offs to manage for the test itself.Landing directly into IcebergThe target is a Snowflake-managed Iceberg table. Rows stream in through the SDK and land as Parquet with Iceberg metadata, managed by Snowflake. Point it at your own S3 external volume later if you want, or leave it managed after the evaluation. The code stays the same either way.For the evaluation it means less setup. A table on Snowflake managed storage skips the external volume, storage integration and cloud storage permissions you'd otherwise have to configure first.When you're ready to productionize, you've got three clear paths:Snowflake-managed Iceberg with an external volume: Parquet files are stored in your S3 bucket. An external engine reads the Parquet files directly from S3, using the Horizon REST Catalog for metadata.Snowflake-managed Iceberg on Snowflake storage: The files are in Snowflake's internal storage. This option avoids configuring an external volume and associated cloud storage. Leverages Iceberg features for compatibility going forward.Native Snowflake table: No separately managed external storage infrastructure is required for this configuration.Demo: See it runningFigure 2: Demo of end to end streamingWhen you run the demo script you can see the SPCS service starts, the producer begins generating the load, and the consumer streams it into the Iceberg table.You can switch to Snowsight, and see the row count climb here as well — millions of rows, queryable as they land.The code: Building it yourselfTo help understand how this all works, here is some pseudocode to help visualize how this works. The full code is in the repo, and there are several alternative demos linked below.Start with a Snowflake-managed Iceberg table and a pipe:-- 1. The target table