What Unit Is Used to Display Redis CPU Usage

What unit is used to display Redis CPU usage

It is the CPU time (expressed in seconds) accumulated since the launch of the Redis instance, as reported by the getrusage() call.

Calculate % Processor Utilization in Redis

You need to read the value twice, calculate the delta, and divide by the time elapsed between the two reads. That should give you the cpu usage in % for that duration.

Why is lpop increasing Redis CPU usage?

Using timed BLPOP instead of LPOP to avoid massive cpu
usage

committed 7 days ago

commit 54315dd9fe56a13b8aba2d2a8868fc48dfbb5795

machinery/v1/brokers/redis.go

- itemBytes, err := conn.Do("LPOP", redisBroker.config.DefaultQueue)
+ itemBytes, err := conn.Do("BLPOP", redisBroker.config.DefaultQueue, "1")

Use the latest version of machinery/v1/brokers/redis.go
which changes LPOP to BLPOP.

Reference: Redis commands: BLPOP

redis memory and cpu spikes

From experimenting further with this and reading about redis persistence, I think the following observations can be made:

• When using RDB (the default settings), redis will fork each time a save operation is triggered, which (by default) is set to once every 15 minutes as a minimum. When more writes to Redis are performed, then RDB writes are as frequent as once every 60 seconds.
• Each fork will use a "copy-on-write" memory allocation, which means that whilst memory won't actually double - it will appear so on tools like ps, htop and the like.
• The fork itself can be quite a cpu-intensive operation, particularly on xen-based virtual hosts (which is what we're using currently).
• The write operation seems to completely overwrite the existing RDB file. It does not write only the changes, but rather dumps the entire dataset to disk.

So on a modest virtual host with 4Gb RAM and data set of around 750Mb (at the time I posted the question), this starts to become rather "expensive". We observed those CPU/Memory spikes, as well as increased IO, even under fairly moderate load / redis usage.

So to answer my own question - this does seem to be the "expected" behaviour.

As for improving the situation, we opted to switch our configuration to use a combination of RDB and AOF. AOF (Append Only File), does appear to only write changes to disk. You can (and should) still configure the AOF file to rewrite (using auto-aof-rewrite-percentage and auto-aof-rewrite-min-size settings). It is also advisable to still use RDB for snapshots. In this configuration however, you can probably do full rewrites / snapshots less frequently and still maintain pretty-good performance and even-better durability.

---

Related Topics