Shared Buffers and OS Page Cache in PostgreSQL: What Are They and How to Configure Them?

The Harsh Reality of Database Performance

Your application is slow. The development team is putting on the pressure. The CTO wants answers. At the end of the day, the blame falls on the database’s performance. But the problem isn’t PostgreSQL. The cause, most of the time, lies in a neglected configuration, a skipped optimization, or something as fundamental as Shared Buffers and the OS Page Cache.

HTI Tecnologia, a specialist in database consulting and support, knows this well. We’ve served dozens of medium and large companies, and the story is always the same: the success of an application depends on the health of its database. And for PostgreSQL, understanding the interaction between Shared Buffers and the OS Page Cache is the starting point for any serious optimization.

In this article, we’ll take a deep dive into these two critical components, demystify their functions, and, most importantly, show you how to configure them correctly to guarantee the performance, availability, and security of your environment. Get ready to challenge everything you thought you knew about PostgreSQL database optimization.

What Are Shared Buffers and the OS Page Cache in PostgreSQL?

To understand how these two components work together, imagine the following:

Shared Buffers (or shared buffers): This is PostgreSQL’s own memory area. It’s where the database stores data pages that have been read from the disk. The goal is to avoid repetitive disk reads, which are slow operations. When your application requests data, PostgreSQL first looks in the Shared Buffers. If the data is there, the response is almost instantaneous. If not, PostgreSQL has to go to the disk, which is the performance bottleneck we want to avoid.
OS Page Cache (or operating system page cache): This is the disk cache of the operating system (OS) itself. The OS stores recently read file pages in its RAM. PostgreSQL is a process like any other, and the pages of its files (tablespaces, WAL files, etc.) can be stored in the OS Page Cache.

The big issue is that these two caches compete for the same resource: RAM. PostgreSQL uses one part for its Shared Buffers, and the operating system uses the rest for the OS Page Cache. The challenge is to find the ideal balance so that both caches work complementarily, not competitively.

How Do Shared Buffers and the OS Page Cache Interact?

The interaction between the two is a topic of intense debate. The official PostgreSQL documentation recommends keeping Shared Buffers small (usually 25% of the total RAM), allowing the OS to manage the rest of the memory for the OS Page Cache. The logic behind this is that the OS is already extremely efficient at managing RAM for I/O operations. It caches both what is read and what is written (write buffers), optimizing the flow of data to the disk.

However, there’s a point of friction: PostgreSQL doesn’t know what’s in the OS Page Cache, and vice versa. This can lead to a situation where the same data page is stored in both the Shared Buffers and the OS Page Cache, consuming double the memory.

This is why HTI Tecnologia takes a more pragmatic approach. Instead of blindly following the 25% rule, we analyze the client’s workload pattern. A database with a heavy read load and frequently repeated data might benefit from a larger Shared Buffers, while an application with random reads that accesses a massive amount of data might perform better with a smaller Shared Buffers, giving more space to the OS Page Cache.

Here are the key points of the interaction:

When PostgreSQL needs to read a data page, it first checks the Shared Buffers.
If it’s not there, it asks the operating system to read the page from the disk.
At that moment, the operating system will check its OS Page Cache. If the page is there, it provides it to PostgreSQL almost instantly.
If the page is not in the OS Page Cache, the OS finally reads it from the physical disk.

The goal, therefore, is to have the largest amount of data possible in RAM, minimizing disk access, which is the slowest operation.

Configuring Shared Buffers: The Magical Formula (That Doesn’t Exist)

Configuring shared_buffers in the postgresql.conf file is one of the most crucial tasks. Unfortunately, there is no universal formula. The ideal value depends on:

Total RAM size: The more RAM, the higher the chance that a large shared_buffers value will be beneficial.
Workload type:
- Read-heavy: Workloads with many SELECT queries can benefit from a larger shared_buffers.
- Write-heavy: For workloads with many INSERTs, UPDATEs, and DELETEs, a very high shared_buffers value can cause checkpoint issues.
Amount of memory allocated for the OS Page Cache: Remember, they compete.

The Golden Rule (with caution):

The most common recommendation is 25% of the total RAM. However, HTI Tecnologia often experiments with values between 15% and 40% in controlled environments, monitoring the results with observability tools. The key is continuous database optimization.

Example configuration in postgresql.conf:

shared_buffers = 1GB

It’s important to note that for servers with more than 32GB of RAM, the benefit of increasing shared_buffers above 8GB or 16GB can be marginal. In these cases, the OS Page Cache becomes increasingly relevant for optimizing I/O.

Optimizing the OS Page Cache: Letting the OS Do the Work

Unlike shared_buffers, you don’t have a parameter in postgresql.conf to control the size of the OS Page Cache. Its management is the responsibility of the operating system itself. However, there are ways to influence its behavior, ensuring it has enough space to operate.

Here are some practical tips:

Monitor RAM usage: Use tools like htop, free -m, or vmstat to monitor free memory and the amount of memory used by the OS Page Cache (usually indicated as cache or buff/cache). A healthy amount of free memory for the cache is a good sign.
Limit other processes: Make sure other services and processes on the server aren’t consuming most of the RAM, leaving little or no memory for the OS Page Cache and for PostgreSQL.
Don’t overload Shared Buffers: If you set shared_buffers to 80% of your RAM, you are, in practice, suffocating the OS Page Cache and limiting its ability to optimize I/O operations.

Examples of commands for OS Page Cache monitoring:

free -h 

htop 

vmstat 2 5

What’s the Sweet Spot?

The ideal point is when shared_buffers is large enough to keep the most frequently accessed data blocks in memory, and the OS Page Cache has space to store disk pages that aren’t in shared_buffers but have been accessed recently.

In HTI Tecnologia’s production environments, we use a combination of monitoring tools and workload analysis to determine the ideal memory allocation. A single command line won’t solve it; continuous performance analysis is necessary.

The Human Factor: The Risk of Not Having a Specialist DBA

We’ve reached a crucial point. The topic of this article is technical, complex, and requires deep knowledge of operating systems, database architecture, and the specifics of PostgreSQL. An error in the shared_buffers configuration can lead to:

Poor performance: Slow queries, timeouts, and degradation of the user experience.
Lack of availability: I/O problems that can lead to database crashes.
Failures in migrations and upgrades: An incorrect configuration can become a nightmare in a new version of PostgreSQL.

Many companies try to manage their database internally, assigning the task to a developer or an infrastructure professional without specialized database administration knowledge. The result? Performance bottlenecks, security risks, and the loss of countless hours trying to solve problems that an experienced DBA would resolve in minutes.

DBA outsourcing is the answer to this problem. HTI Tecnologia offers consulting, support, and 24/7 maintenance for databases. With a team of senior DBAs, we guarantee operational continuity, risk reduction, and the optimization your business needs. Our specialists have already dealt with the most complex PostgreSQL performance issues and know exactly how to configure Shared Buffers and the OS Page Cache for your specific workload.

Don’t underestimate the importance of professional database management. Leaving your database configuration in the hands of amateurs is a risk your company simply cannot afford to take.

The Path to Performance Excellence

PostgreSQL optimization, starting with the configuration of Shared Buffers and the OS Page Cache, is not a trivial task. It requires knowledge, experience, and, above all, a holistic view of your application’s architecture. The true secret isn’t in a single line of code, but in a deep understanding of how the database, the operating system, and the workload interact.

HTI Tecnologia is here to be your partner on this journey. Instead of spending internal time and resources to solve performance problems, why not leave the management of your databases to those who do it professionally?

If your company depends on PostgreSQL to operate, HTI Tecnologia offers the expertise needed to ensure your database is a growth engine, not a bottleneck. Our PostgreSQL support services are designed to optimize, protect, and maintain your environment at its maximum capacity.

Don’t risk the performance and security of your database. Talk to an HTI Tecnologia specialist and discover how DBA outsourcing can transform your company’s infrastructure, ensuring more performance, availability, and peace of mind.

Schedule a meeting now and stop losing money on performance bottlenecks.

Schedule a meeting here