Why Sequential Reads Beat Random Reads in Postgres at Scale

We had a large table - hundreds of gigabytes - that needed to be fully exported and processed in batches. The pipeline fetched rows from Postgres in batches of 2000, processed them, and pushed results downstream.

It was slow. Much slower than expected. The bottleneck wasn't the processing. It was Postgres. Specifically, how we were reading from it.

How Postgres Stores Data

Before getting to the fix, you need to understand three things: pages, slots, and sequence columns.

Pages are fixed 8KB chunks on disk. Everything Postgres stores lives inside pages.

Why Sequential Reads Beat Random Reads in Postgres at Scale

Related reading

PostgreSQL for Data Engineers: Indexes, Bulk Loads, and the Patterns That…

Multi-block buffered reads in PostgreSQL 19 (IO combine & prefetch)

Running Postgres at Scale: Lessons Learned

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale Effortlessly?

Three Ways to Set Up CDC from Postgres to ClickHouse

Your Postgres already knows why it's slow — you just have to ask it

Related reading

PostgreSQL for Data Engineers: Indexes, Bulk Loads, and the Patterns That…

Multi-block buffered reads in PostgreSQL 19 (IO combine & prefetch)

Running Postgres at Scale: Lessons Learned

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale Effortlessly?

Three Ways to Set Up CDC from Postgres to ClickHouse

Your Postgres already knows why it's slow — you just have to ask it

Related reading

PostgreSQL for Data Engineers: Indexes, Bulk Loads, and the Patterns That…

Multi-block buffered reads in PostgreSQL 19 (IO combine & prefetch)

Running Postgres at Scale: Lessons Learned

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale *Effortlessly*?

Three Ways to Set Up CDC from Postgres to ClickHouse

Your Postgres already knows why it's slow — you just have to ask it

Related reading

PostgreSQL for Data Engineers: Indexes, Bulk Loads, and the Patterns That…

Multi-block buffered reads in PostgreSQL 19 (IO combine & prefetch)

Running Postgres at Scale: Lessons Learned

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale *Effortlessly*?

Three Ways to Set Up CDC from Postgres to ClickHouse

Your Postgres already knows why it's slow — you just have to ask it

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale Effortlessly?

Multigres v0.1 Alpha: Is Supabase Finally Making Postgres Scale Effortlessly?