Proxmox cache modes and which one should I use?

When using Proxmox in my homelab, one of the parameters that will most affect my IOPS metrics may be the disk cache mode. In this article I will try to explain how Proxmox’s cache works, the multiple modes it has, and how to choose the best one for each scenario. But first…


What are IOPS?

IOPS (Input/Output Operations Per Second, pronounced i-ops) is a common benchmark performance measure for computer storage devices.

It gives us, therefore, an idea of how many operations the unit will be able to satisfy in a given time. The higher this number is, the more capacity the disk will have to be able to satisfy the needs of our homelab.

Then… it is clear that the higher our IOPS number is, the better our storage will respond. However, it will not only depend on the type of storage we are going to use (HDD vs SSD), but also other factors such as the network bandwidth (in case of using SAN), the hardware used to connect the disks, the storage controllers, the OS background operations or the system configuration. This last one is the one we are going to deal with today.


Disk cache in Proxmox

In the Proxmox wiki we can find a section dedicated to disk caching. Although a bit chaotic, it gives us the keys to know which modes are available to us and what advantages each one gives us. I summarize it here:

cache=none

The host page cache is bypassed and I/O happens directly between QEMU-KVM and the actual storage device.

cache=writethrough

This mode causes QEMU-KVM to interact with the disk image file or block device in a way that writes are reported as completed ONLY when the data has been committed to the storage device.

cache=writeback

This mode causes QEMU-KVM to interact with the disk image file or block device in a way that the host page cache is used and writes are reported to the guest as completed when placed in the host page cache, BEFORE they are committed to the storage device.

cache=directsync

This mode causes QEMU-KVM to interact with the disk image file or block device in a way that writes are reported as completed ONLY when the data has been committed to the storage device, AND when it is also desirable to bypass the host page cache.

cache=unsafe

This mode is similar to the cache=writeback mode discussed above.

The key aspect of this “unsafe” mode, is that all flush commands from the guests are ignored. Using this mode implies that the user has accepted the trade-off of performance over risk of data loss in the event of a host failure.


So… which one is better for me?

As a general rule, high end systems typically perform best with cache = none, because of the reduced data copying that occurs. But looking at safety, these are the conclusions:

cache = writethrough, cache = none, cache = directsync

These are the safest modes, and considered equally safe, given that the guest operating system is “modern and well behaved”, which means that it uses flushes as needed.

cache = writeback

This mode relies on the guest to send flush commands as needed to maintain data integrity within its disk image.

It should be noted that because there is a window of time between the time a write is reported as completed, and that write being committed to the storage device, this mode exposes the guest to data loss in the unlikely event of a host failure.

cache = unsafe

This mode is similar to writeback caching except the guest flush commands are ignored resulting in a higher risk of data loss due to host failure.


Some interesting resources on the subject:

Hello, world!

Hello, dear stranger,

Welcome to my blog, or whatever you want to call it. I define this as my notebook, a place to talk about the things I discover, learn and carry out in my day-to-day life.
My name is David, I’m 20 years old and I live in Valencia (Spain), a fantastic place for the entrepreneurial ecosystem, to sunbathe or eat paella. I’m a full-stack programmer, a PHP fan, a gamer, an Apple hater and I have a cat. His name is Charlie.

I’m mainly dedicated to solve problems, through the development of digital tools, mainly for companies. However, my self-taught streak makes me stay up late messing around with my lab at home. I consider myself lucky to be working on something that is also my hobby.

Although my main obsession is with code, I also have a strong interest in network and systems administration, Kubernetes, artificial intelligence or blockchain. I also enjoy developing small video game ideas in my spare time, trying out technologies I don’t use in my day and night, assembling DIY toys for my cat or searching for electronics on eBay.

Although I’m a man of few words, my idea is to use this blog as a personal challenge to improve my expression, as well as a notebook to document my learning and experiments. This way I can always go back and review my notes, and anyone who finds interest in my adventures can learn from my mistakes.

Well, I think that’s enough to be my first publication. See you soon!