syslog-ng

syslog-ng配置

March 13th, 2012
绚丽也尘埃 Leave a comment
Go to comments

syslog-ng的配置非常简单直观，于是乎配置好看看怎么用它实时收集日志。有两台服务器，一台服务器盯着error_log文件，一旦发现其有新数据，立即将日志发到另一台收集服务器。

client服务器配置如下。

?

1 2 3 4 5 6 7 8 9

#将文件作为src source s_blender_error_log {         file("/home/admin/kingso/logs/error_log_blender"); }; #收集到的日志发送给172.25.61.92 destination d_blender_error_log { tcp("172.25.61.92" port(514)); }; log { source(s_blender_error_log); destination(d_blender_error_log);};

server服务器配置如下。

?

01 02 03 04 05 06 07 08 09 10 11

#监听514端口 source s_kingso {         tcp(ip(0.0.0.0) port(514)); }; #设置输入文件及其相关的属性 destination d_kingso {         file("/home/admin/error.log" owner(admin) group(admin) perm(0755)); }; log { source(s_kingso); destination(d_kingso); };

最后收集到的日志如下。

?

1	Mar 13 20:32:56 s061090.cm5 moximoximoximoximoxi

syslog-ng有一个很有意思的功能，那就是在destination里面支持program，可以对符合条件的日志支持操作，比如报警等等。下面这个配置会将所有包含error的日志输出到一个指定的文件中。为了提高性能，建议这个程序从标准输入读入数据，如果没有数据就阻塞等待，不要轻易退出。

?

1 2 3

destination d_kingso {         program("grep error >> /tmp/eerroorr.log"); };

filter支持内容过滤，比如要把服务器上所有core信息都收集到一台服务器上，只要判断kern信息中是否有segfault就可以，配置如下。

?

1 2 3 4

filter f_kingso_core {     facility(kern) and     match("segfault" value("MESSAGE")); }

如果要用syslog-ng收集大集群的日志，需要设置下max-connections这个参数，它的默认值是10，实在是太小了。如果不改这个配置，netstat会出现syslog-ng导致的大量TIME_WAIT。将其放大之后，可以算一下ESTABLISHED的链接数是不是和集群服务器数量相同。

syslog-ng服务器默认情况下收集到的日志是：”Apr 1 17:42:33 hostname blabla”。月份后面可能有一个空格（大于10号）也可能有两个空格（小于10号），这样对消息的切分就会有问题，这个时候必须通过template来规范消息的格式。syslog-ng支持定义一个template模版，以后的template都可以使用改模版，非常方便。下面这个配置就是program和file两个destination共用一个t_kingso模版。

?

1 2 3 4 5 6

template t_kingso { template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$FULLHOST\t$MSGHDR$MSG\n"); }; destination d_kingso_blender {     program("exec python -u /home/henshao/log_monitor/log_monitor.py -c /home/henshao/log_monitor/log_monitor.conf -m blender >> /tmp/log_monitor.log 2>&1" template(t_kingso) );     file( "/home/admin/logs/blender.log.$S_YEAR-$S_MONTH-$S_DAY" create_dirs(yes) owner(admin) group(admin) perm(0755) template(t_kingso) ); };

我发现RHEL5上默认安装了syslogd、syslog-ng和rsyslog。syslogd历史最为悠久，wikipedia上的资料上讲它当初是sendmail项目的一部分，因为非常有用，所以逐渐成为所有unix-like系统的标准组件。syslog-ng则是syslogd的升级版本，是一个公司开发的，除了开源免费版还有收费的高级版，有一篇不错的文章： 专访syslog-ng 2.0开发人员Balazs Scheidler。rsyslog则最为年轻，作者想做一个log系统和syslog-ng竞争。

推荐一些不错的资料。

1、Syslog-ng

2、syslog-ng v2.0 reference manual

 

 

 

Syslog-ng安装及配置(2010-04-26 13:37:46)

转载▼

标签： 杂谈

分类： linux

注:当前使用版本为syslog-ng-2.0.9

//使用以下RPM或是使用eventlog的源码包

rpm -ivh eventlog-0.2.5-1.el5.i386.rpm

rpm -ivh eventlog-devel-0.2.5-1.el4.i386.rpm

./configure --prefix=/usr/local/syslog-ng --sysconfdir=/etc

make && make install

mkfifo /tmp/mysql.pipe  //创建mysql管道,接收到的日志写入该管道,通过期写入Mysql

chkconfig --add syslog-ng

chkconfig syslog-ng on

chkconfig --add sqlsyslogd

chkconfig sqlsyslogd on

web界面采用php-syslog-ng来查看日志

以下是所用到的配置文件及服务文件

Syslog-ng.conf配置文件

/etc/syslog-ng.conf

#

# configuration file for syslog-ng, customized for remote logging

#

source s_internal { internal(); };

destination d_syslognglog { file("/var/log/syslog-ng.log"); };

log { source(s_internal); destination(d_syslognglog); };

# Local sources, filters and destinations are commented out

# If you want to replace sysklogd simply uncomment the following

# parts and disable sysklogd

#

# Local sources

#

#source s_local {

#      unix-dgram("/dev/log");

#      file("/proc/kmsg" log_prefix "kernel:");

#};

#

# Local filters

#

#filter f_messages { level(info..emerg); };

#filter f_secure { facility(authpriv); };

#filter f_mail { facility(mail); };

#filter f_cron { facility(cron); };

#filter f_emerg { level(emerg); };

#filter f_spooler { level(crit..emerg) and facility(uucp, news); };

#filter f_local7 { facility(local7); };

#

# Local destinations

#

#destination d_messages { file("/var/log/messages"); };

#destination d_secure { file("/var/log/secure"); };

#destination d_maillog { file("/var/log/maillog"); };

#destination d_cron { file("/var/log/cron"); };

#destination d_console { usertty("root"); };

#destination d_spooler { file("/var/log/spooler"); };

#destination d_bootlog { file("/var/log/boot.log"); };

#

# Local logs - order DOES matter !

#

#log { source(s_local); filter(f_emerg); destination(d_console); };

#log { source(s_local); filter(f_secure); destination(d_secure); flags(final); };

#log { source(s_local); filter(f_maillog); destination(d_maillog); flags(final); };

#log { source(s_local); filter(f_cron); destination(d_cron); flags(final); };

#log { source(s_local); filter(f_spooler); destination(d_spooler); };

#log { source(s_local); filter(f_local7); destination(d_bootlog); };

#log { source(s_local); filter(f_messages); destination(d_messages); };

# Remote logging

source s_remote {

      tcp(ip(0.0.0.0) port(514));

      udp(ip(0.0.0.0) port(514));

};

options

{

chain_hostnames(no);

create_dirs (no);

dir_perm(0755);

dns_cache(yes);

keep_hostname(yes);

log_fifo_size(2048);

log_msg_size(8192);

long_hostnames(on);

perm(0644);

stats(3600);

sync(0);

time_reopen (10);

use_dns(yes);

use_fqdn(yes);

};

destination d_separatedbyhosts { pipe("/tmp/mysql.pipe" template("INSERT INTO logs

    (host, facility, priority, level, tag, datetime, program, msg)

    VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); };

log { source(s_remote); destination(d_separatedbyhosts); };

#----------------------------------------------------------------------

# Sources

#----------------------------------------------------------------------

# For Linux

#----------------------------------------------------------------------

source s_stream

{ unix-stream("/dev/log"); };

source s_internal

{ internal(); };

source s_kernel

{ pipe("/proc/kmsg" log_prefix("kernel: ")); };

source s_tcp

{ tcp(port(4800) keep-alive(yes) max_connections(100)); };

#----------------------------------------------------------------------

# Piping method

#----------------------------------------------------------------------

destination database { pipe("/tmp/mysql.pipe" template("INSERT INTO logs

    (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); };

#----------------------------------------------------------------------

# Logging to a database

#----------------------------------------------------------------------

log { source(s_stream);

source(s_internal);

source(s_kernel); destination(database); };

syslog-ng服务文件

/etc/init.d/syslog-ng

###############################################################

#

# Program: syslog-ng init script for Red Hat

#

###############################################################

# the following information is for use by chkconfig

# if you are want to manage this through chkconfig (as you should), you must

# first must add syslog-ng to chkconfig's list of startup scripts it

# manages by typing:

#

# chkconfig --add syslog-ng

#

# DO NOT CHANGE THESE LINES (unless you know what you are doing)

# chkconfig: 2345 12 88

# description: syslog-ng is the next generation of the syslog daemon. \

# syslog-ng gives you the flexibility of logging not only by facility and \

# severity, but also by host, message content, date, etc. it can also replace \

# klogd's function of logging kernel messages

#

# This following block of lines is correct, do not change! (for more info, see

# http://www.linuxbase.org/spec/refspecs/...facilname.html)

### BEGIN INIT INFO

# Provides: $syslog

### END INIT INFO

###############################################################

#

# This is an init script for syslog-ng on the Linux platform.

#

# It totally relies on the Redhat function library and works the same

# way as other typical Redhat init scripts.

#

#

# Platforms (tested): Linux (Redhat 7.3)

#

#

# Author: Gregor Binder <gbinder@sysfive.com>

# Changed: October 10, 2000

#

# Last Changed: September 27, 2002

# Updated by: Diane Davidowicz

# changes: Brought the start script up to snuff as far as compliance

# with managing the startup script through chkconfig;

# added PATH variable ability to hook in path to syslog-ng (if

# its necessary); converted init script format to the

# standard init script format in Red Hat (7.3 to be exact)

# including using the /etc/sysconfig/syslog-ng file to

# managed the arguments to syslog-ng without changing this

# script, and disabled klogd but noted where and under what

# conditions it should be enabled. HAPPY LOGGING.

#

# Copyright &copy; 2000 by sysfive.com GmbH, All rights reserved.

#

#

###############################################################

#

# configuration

#

INIT_PROG="/usr/local/syslog-ng/sbin/syslog-ng" # Full path to daemon

INIT_OPTS="" # options passed to daemon

#

# Source Redhat function library.

#

. /etc/rc.d/init.d/functions

# Tack on path to syslog-ng if not already in PATH

SYSLOGNG_PATH=":/usr/local/syslog-ng/sbin"

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/syslog-ng/sbin

INIT_NAME=`basename "$INIT_PROG"`

# /etc/sysconfig/ is the standard way to pull in options for a daemon to use.

# Source config

if [ -f /etc/sysconfig/syslog-ng ] ; then

. /etc/sysconfig/syslog-ng

else

SYSLOGNG_OPTIONS=

fi

RETVAL=0

umask 077

ulimit -c 0

# See how we were called.

start() {

echo -n "Starting $INIT_PROG: "

#daemon $INIT_PROG $SYSLOGNG_OPTIONS

daemon --check $INIT_PROG "$INIT_PROG $INIT_OPTS"

RETVAL=$?

echo

[ $RETVAL -eq 0 ] && touch "/var/lock/subsys/${INIT_NAME}"

return $RETVAL

}

stop() {

echo -n "Stopping $INIT_PROG: "

killproc $INIT_PROG

RETVAL=$?

echo

[ $RETVAL -eq 0 ] && rm -f "/var/lock/subsys/${INIT_NAME}"

return $RETVAL

}

rhstatus() {

status $INIT_PROG

}

restart() {

stop

start

}

case "$1" in

start)

start

;;

stop)

stop

;;

status)

rhstatus

;;

restart|reload)

restart

;;

condrestart)

[ -f /var/lock/subsys/syslog-ng ] && restart || :

;;

*)

echo $"Usage: $0 {start|stop|status|restart|reload}"

exit 1

esac

exit $?

实时把日志导入数据库的服务文件

/etc/init.d/sqlsyslogd

#!/bin/bash

#

# sqlsyslogd This is a daemon that takes syslog-ng input and pipe it into

# a MySQL database.

#

# chkconfig: 2345 98 10

# description: sqlsyslogd bridges syslog-ng and mysql.

# author: Josh Kuo Thu 2004/08/12 13:21:56 PDT

. /etc/rc.d/init.d/functions

case "$1" in

start)

if [ -x /tmp/mysql.pipe ]; then

mkfifo /tmp/mysql.pipe

else

# if the service is already running, do not start another one

PIDS=`pidofproc mysql`

if [ "$PIDS" ]; then

echo "sqlsyslogd is already running."

exit 1

fi

mysql -u 用户名 -h localhost –p密码 数据名 < /tmp/mysql.pipe &

#If you need to collect apache logs into mysql, uncomment the following two lines.

#tail -f /usr/local/apache2/logs/access_log | logger -p info -t apache &

#tail -f /usr/local/apache2/logs/error_log | logger -p notice -t apache &

fi

;;

stop )

killproc mysql

#If you need to collect apache logs into mysql, uncomment the next line.

#killproc tail

;;

*)

echo "Usage: sqlsyslogd {start|stop}"

exit 1;

esac

exit 0;