Summary: Centralized log file server

From: Malloy, Jim <>
Date: Tue Jan 15 2002 - 11:24:46 EST
Many, many responses.  My thanks to all.

All responses are included below (in no particular order).  Short version:

        Use syslog
        Use rsync or rdist 

        NFS mount or use scp or rsh - makes scripting easier and provides
        security (in the case of rsh and scp).

        Plan space, archive time requirements first


Karl Vogel []
   I'd use scp/ssh instead.

R. Lonstein []
        You could accomplish this with cron, but I've been down that road
        as the SA and as the developer and have the following comments:
            + Rotate your syslog files based upon age and size. Plan for
              space. Archive at least nightly, preferrably at rotation. I
              particularly like the BSD implementation of newsyslog,
              is barely adequate. See the following links...
            + Pull, don't push.
            + Use ssh with a public key/private key for each host configured
              perform a single command (like run rsync, below).
            + Use rsync for application logs.
            + Require that errors, warnings, etc. be emitted to syslog. Use
              script to tail application logs and relay them under 'local'
              of syslog.

        The most important advice I can give is:
            + Plan & prioritize first (then review the plan). Don't collect
              of logs if no one will read them. If someone is expected to
              them keep them as small as is useful. Better still, automate
              combing the logs and then do random spot checks. Determine how
              long you hold logs, do backups, provide access. You will end
              dealing with these eventually and it's easy up-front.

David Foster []
        Any logs coming from syslog can be configured on remote systems
        to be saved on those systems *and* the log server.

        For other logs, I'd suggest using 'rsync' to transfer them, it's
        a great utility for keeping directory trees sync'd up. Also, better
        to use SSH instead of RSH as the transfer shell, for security

                RSYNC: File distribution:

                OpenSSH : Open Source Secure Shell
                        CERT Doc :
                        RSAref : (not neces

                        Windows Client:
                        Windows SCP:
                        Mac Client:
                        Mac Client:
                        Scanssh: (Scan subnet for SSH server protocols)
                        Replacing telnet/rlogin/rsh with SSH:
                        VanDyke Tech. VShell: SSH2 server on Windows

        Dave Foster


        I keep my log files on a central server. I have the log staging
        available via NFS and use a cron script to copy the logs over to it
each night
        or each weekend. Right after copying the logs over, I cat /dev/null
> to zero out the log file. About once each year I write
the central
        log archive to tape for long term storage.


Paul LaMadeleine

        If all you want to do is replicate the logs on a center server (I'd
        in hostname specific directories) that I'd use rsync.  You can check
it out


Mark Wishneusky []

        Those sound good...  Also, what about mounting the drives
temporarily (in a
        script at like 3:00 a.m.), copying everything, and then umounting
        again.  Another could be to use rcp.

Omar Onsi []

        try the "rdist" command. read about it, i think it is what you are
        it is used to synchronize files on all nodes from one server.

Jeffrey C. Keyser []

        Rather than ftp, rsync or scp the logs. They'll be encrypted during
        transmission and if you configure it properly, you won't be prompted
        a password. Therefore, you won't need to write an expect script.

Kevin P. Inscoe []

        Rather then ftp the logs if those applications could write to the
        syslogd it would sure make life a *lot* easier. You can use facility
        switches in /etc/syslogd.conf to direct multiple outputs to multiple
        files but going through syslogd is how I would try to do it. It is a
        basic syscall in C and available in Perl as well. You could even
write a
        wrapper and put it in a shell script. You  are aware of course you
        push syslog entries to a central server which is also a good
        practice. Make it so you cannot login to the log server(s) except
        console and remove ftpd you would have the makings of a fortress
        there. This way if your systems are compromised you will always have
        logs and they can't be modified. :-)

Christian J Hedemark []
        Set up MySQL (RDBMS) with a next generation syslog facility, and use
        as the default logging facility for as many applications as you can.
        way your developers can have a robust SQL interface to all of the
        Forget the few hours lag.  We're talking fractions of a second.

        For any other applications that can't/won't do syslog you can set up
a perl
        script with DBI to move the contents of the logs into the same SQL

Jeff Horwitz []
        maybe you've already considered this and didn't mention it, but have
        considered sharing the log directory via NFS?
sunmanagers mailing list
Received on Tue Jan 22 17:58:05 2002

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:42:32 EST