How to increase your logs
maximum string size with log shippers

Did you come across the unpleasant phenomenon of truncated logs? This usually happens when sending logs to a log management solution, and can be happening because of either technical or platform limitations.

So I’ll cover in this article the existing technical limitations to enlarging your log message size with log shippers, and depending on the technology used, and when possible, how to overcome them.

SysLog

SysLog comes with a hardcoded limitation of 1024 characters per line logged. Any longer log message will be truncated.
You’ll need to find and modify the source file to modify this existing limit.
Find the following line #define MAXLINE 1024 /* maximum line length */ and set the numerical value to the one you need. Don’t forget to make obj && make depend && make && make install and to restart /etc/rc.d/syslogd restart.

You can otherwise chose to work with RSysLog as string size can be directly set in the configuration file.

RSysLog

RSysLog default log string size limitation is the same as the one for SysLog : 1024 characters per line logged. It can however be directly modified in the configuration file with the following command:

$MaxMessageSize 20k

Please note this parameter must be set in the configuration file before the network declaration (therefore on top of the configuration file).

NXLog

If your log messages are bigger than 1MB, it can lead NXLog to failure. Therefore some truncation might be added in the configuration file if you feel it might be your case. Although, if you have log messages bigger than 1MB, it might be worth reviewing your logging strategy, as 1MB is already 3 times bigger than Shakespeare’s Hamlet (175K characters)!

You can thus prescribe a limit within the config file:
// Limit maximum message size to just less than 1MB; or NXLog dies with: ERROR string limit (1048576 bytes) reached
res += @" Exec if $Message $Message = substr($raw_event, 0, 1040000);" + Environment.NewLine;

And if you have the entreprise edition, the StringLimit parameter can be used. In any cases, the limit of 1MB per message cannot be overcome for NXLog.

Related Posts

Get notified when new content is published