Running Pydio Cells in docker on linux. Out of the blue Pydio Cells became dog slow. Checking logs was presented with a recurring error of:
nats - Slow Consumer Detected: WriteDeadline of 2s Exceeded…
Restarting the container cleared up the issue.
Any suggestions on how to chase down the root cause of the issue?
I completely agree. The only containers running on this hardware are the cells, MySQL, and traefik containers (MySQL only used by this cells container). The cells container was consuming 500% of the CPU until a restart at which point CPU consumption dropped into single digits and the recurring nats error went away.
Is there a more verbose logging option in cells to give a better understanding of what caused the nats error should it come back?
No, as nats is the communication layer for all inter-services requests, it’s just saying : your system is going super slow.
If that happens again, first think would be to “ps aux” / “top” / “htop” to detect which Cells process is hogging the CPU. Eventually if you are motivated, you can also send a “kill -SIGUSR1 $PROCESS_ID” which will print a trace of all goroutines in that process. Maybe that can give us some infos…
Had a similar issue today however the mysql container was the gross CPU abuser instead of the cells container (500% vs over 100% for cells). MySQL is spun up with its default configuration with the exception of the character_set_server = utf8mb4 and collation_server = utf8mb4_unicode_ci
Is there a published guideline for recommended parameterize of SQL for use will Cells? buffer size? io threads?