solving rsyslog write errors
When rsyslog reports a write error, it includes the operating-system generated error message. It should hopefully give you a clue what the problem cause was. Unfortunately from time to time to root cause is not obvious.
In this case please check the following potential causes:
- Was OS/rsyslog config change applied but rsyslog not restarted? Rsyslog configuration changes are only applied when rsyslog is restarted. Similarly, many operating system process limitations (like file size and several permission settings) are only applied if process is restarted. If in doubt, do a restart of rsyslog. Doing so can potentially save you a lot of time.
- Is rsyslog configured to drop privileges? If so, the user or group dropped to may simply not have the right permission. Try to comment out the privilege drop to see if this is the root cause.
- Does SELinux prevent rsyslog to access the file? This is often the case if you write to non-standard locations. To check if this is the cause, you can disable SELinux on the system. If it then works, you know the root cause. But please do not run with SELinux disabled. Instead, configure it correctly.
- Are you using something similar to SELinux? For example AppArmor on Ubuntu? Investigate and check if it causes the trouble.
- Do you run rsyslog via systemd? Are there any limits specified in the service file? Most modern Linuxes use systemd, so this is for sure a place to check.
- Are there any global limits specified in the system configuration? Note: systemd ignores them, so if you use systemd, your really need to check the systemd configuration and rsyslog’s unit file!
- Are there any file system limitations?
- Did the system (temporarily) run out of space? This could especially be the case for intermittent problems.
This list probably is not conclusive but should give you a good idea of known trouble spots.
For a quick but rough check to find the culprit, you can run rsyslog in an interactive terminal window. Use the root account and do not drop privileges. If it works there, chances are pretty good that some other operating system component is causing the trouble.