I've had much better results from using Syslog-NG along with php-Syslog-NG.<p>Syslog-NG can already split your log files into subdirectories with the hostname of each server, but it also has the capability of redirecting messages to named pipes. This is great because you can pipe it into mysql and stuff all of your log messages in a database. Combine with a php front-end and now your developers and sysadmins can search logs intelligently across multiple servers, and get really fine-grained on their search strings. Want to tail the output from all Tomcat servers in your app server pool looking for a specific string? Go right ahead.
One big advantage of rsyslog over syslog-ng is that you can spool messages to disk if the remote syslog server is down (syslog-ng only offers this in their 'enterprise' paid version).
I've been pushing to implement this for our application, but I'm told that we used to, and had to turn it off because it would saturate the IO of the logging server.<p>Has anyone else experienced this? Is it just a simple configuration tuning problem?
Protip: Use syslog-ng.<p>Besides longer log message (arbitrarily long, with a recompile) and reliable delivery, it obviates my main use for logrotate, since it can be configured to write to a filename (including directory) based on time, date, or other variables.
If you're interested in more volume and flexibility, check out Flume, a new open-source project from Cloudera (the Hadoop/logging experts). Solid software and community behind it. <a href="http://archive.cloudera.com/cdh/3/flume/UserGuide.html" rel="nofollow">http://archive.cloudera.com/cdh/3/flume/UserGuide.html</a>