A program on my Raspberry Pi was reading data on disk, performing operations, and writing the result on disk. It did so at an unusually slow speed. The problem could either be that the CPU was too underpowered to perform the operations it needed or the disk was too slow during read, write, or both.

I asked colleagues for opinions, and one of them mentioned that htop could orient me in the right direction. The time a CPU spends waiting for an I/O device such as a disk is known as the I/O wait. If that wait time is superior to 10%, then the CPU spends a lot of time waiting for data from the I/O device, so the disk would likely be the bottleneck. If the wait time remains low, then the CPU is likely the bottleneck.

By default htop doesn't show the wait time. By pressing F2 I can access htop's configuration. There I can use the right arrow to move to the Display options, select Detailed CPU time (System/IO-Wait/Hard-IRQ/Soft-IRQ/Steal/Guest) , and press Space to enable it.

I can then press the left arrow to get back to the options menu, and move to Meters . Using the right arrow I can go to the rightmost column, select CPUs (1/1): all CPUs by pressing Enter, move it to one of the two columns, and press Enter when I'm done. With it still selected, I can press Enter to alternate through the different visualisations. The most useful to me is the [Text] one.

I can do the same with Disk IO to track the global read / write speed, and Blank to make the whole set-up more readable.

With htop configured like this, I can trigger my slow program again see that the CPU is not waiting for the disk. All CPUs have a wa of 0%

If you know more useful tools I should know about when chasing bottlenecks, or if you think I got something wrong, please email me at thib@ergaster.org!