LAMP+logzilla+sphinx+syslog-ng实现集中日志管理(第一版)[原创] 
一、前言
目前查看系统日志比较被动，遇到系统不正常或故障时才会主动去检查服务器系统日志，这样一来不能及时了解系统的运行情况，因此部署Logzilla+sphine+syslog-ng来弥补这不足。以下为安装、部署平台详细步骤。(Logzilla是什么新东西？其实前身就是php-syslog-ng，引用作者的话“Php-syslog-ng is now known as LogZilla. Same owner, better code :-)”)
二、平台初始化
#yum install libdbi* libnet
#cpan Date::Calc Text::LevenshteinXS String::CRC32
三、下载相关包
#cd /home/install
#mkdir logzilla;cd logzilla
#wget http://www.balabit.com/downloads/files/eventlog/0.2/eventlog_0.2.9.tar.gz
#wget http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.0.3/setups/rhel-5-i386/syslog-ng-3.0.3-1.rhel5.i386.rpm
四、开始安装
# cp eventlog_0.2.9.tar.gz /usr/src/redhat/SOURCES/
# tar zxvf eventlog_0.2.9.tar.gz
# cd eventlog-0.2.9/
# rpmbuild --ba eventlog.spec.bb
# cd /usr/src/redhat/RPMS/x86_64
# rpm -Uvh libevtlog*
#cd /home/install/logzilla
#rpm -Uvh syslog-ng-3.0.3-1.rhel5.i386.rpm
五、安装logzilla
#cd /www/webroot/
#wget http://php-syslog-ng.googlecode.com/files/logzilla_3.0.85.tgz
#tar -zxvf logzilla_3.0.85.tgz
#cd logzilla/scripts
#./install.pl
(根据实际情况来回应就OK了)
===================
LogZilla Installation
====================
Enter the MySQL root username [root]:
Enter the password for root [mysql]:
Database to install to [syslog]:
Database table to install to [logs]:
Enter the name of the MySQL server [127.0.0.1]:
Enter the port of the MySQL server [3306]:
Enter the name to create as the owner of the logs database [syslogadmin]:
Enter the password for the syslogadmin user [syslogadmin]:
Enter the name to create as the WEBSITE owner [admin]:
Enter the password for admin [admin]:
Enter your email address [cdukes@cdukes.com]:
Enter a name for your website [The home of LogZilla]:
Enter the base url for your site (include trailing slash) [/logs/]: /
Where should log files be stored? [/var/log/logzilla]:
How long should I keep old logs? (in days) [30]:
========================================
Path Updates
========================================
Getting ready to replace paths in all files with "/www/webroot/logzilla"
Ok to continue? [y]:
Updating file paths
Modifying ../scripts/db_insert.pl
Modifying ../scripts/contrib/system_configs/logzilla.crontab
Modifying ../scripts/contrib/system_configs/syslog-ng.conf
Modifying ../scripts/contrib/system_configs/logzilla.apache
Modifying ../sphinx/indexer.sh
Modifying ../sphinx/sphinx.conf
Updating log paths
Modifying ../scripts/contrib/system_configs/logzilla.crontab
Modifying ../scripts/contrib/system_configs/logzilla.logrotate
====================
Database Installation
====================
All data will be installed into the syslog database
Ok to continue? [y]:
====================
Config.php generation
====================
Generating /www/webroot/logzilla/html/config/config.php
Ok to continue? [y]:
====================
System files
====================
Adding LogZilla logrotate.d file to /etc/logrotate.d
Ok to continue? [y]:
Where is your syslog-ng.conf file located? [/etc/syslog-ng/syslog-ng.conf]: /opt/syslog-ng/etc/syslog-ng.conf
Adding syslog-ng configuration to /opt/syslog-ng/etc/syslog-ng.conf
Ok to continue? [y]:
Found 1 sources
Which source definition would you like to use? [s_all]:
LogZilla installation complete...
Note: you may need to enable the MySQL Event Scheduler in your /etc/my.cnf file.
Please visit http://forum.logzilla.info/index.php/topic,71.0.html for more information.
Also, please visit http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0#UDP_Buffers to learn how to increase your UDP buffer size (otherwise you may drop messages).
Please run /etc/init.d/syslog-ng restart
六、安装 Sphinx
#cd logzilla/sphinx/src
#tar xzvf sphinx-0.9.9.tar.gz
#cd sphinx-0.9.9
#./configure --prefix `pwd`/../..
#make && make install
#cd /www/webroot/logzilla/sphinx
#vi sphinx.conf
#!/usr/bin/php 替换成实际php的位置，如#!/usr/local/php/bin/php
#./indexer.sh full
#bin/searchd
提示：
ERROR: index 'idx_logs': sql_query_pre[0]: Column 'max_id' cannot be null (DSN=mysql://syslogadmin:***@127.0.0.1:3306/syslog).
如果看到该信息，属正常，因为现在还没有数据:)
添加作业：
30 0 1 * */www/webroot/logzilla/sphinx/indexer.sh full >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
*/5 * * * */www/webroot/logzilla/sphinx/indexer.sh delta >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
0 0 * * */www/webroot/logzilla/sphinx/indexer.sh merge >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
添加rc.local
#vi /etc/rc.local
/www/webroot/logzilla/sphinx/bin/searchd -c /www/webroot/logzilla/sphinx/sphinx.conf
七、配置Mysql
激活event_scheduler
八、修改Syslog-ng配置
v#i /opt/syslog-ng/etc/syslog-ng.conf
(略)
九、修改apache配置
#vi httpd.conf
重启apache服务：/etc/init.d/apache2 restart
十、IONCube授权
http://127.0.0.1/logs/login.php
下载源码包ioncube_loaders_lin_x86-64.tar.gz
#mkdir -p /usr/local/ioncube
#tar -zxvf ioncube_loaders_lin_x86-64.tar.gz
#cd ioncube
#cp * /usr/local/ioncube/
#vi /usr/local/php/lib/php.ini
zend_extension = /usr/local/ioncube/ioncube_loader_lin_5.2.so
#/etc/init.d/apache2 restart
十一、安装完毕
http://127.0.0.1/logs/login.php
根据install.pl配置的管理员帐号密码进行登录。
1、MainPage
2、StatPage
十二、客户端配置
#vi /etc/syslog.conf
在最后添加以下，其中syslog.admin.com.cn为主机域名，也可以直接用IP代替。
*.emerg;*.err;*.warning @syslog.admin.com.cn
#/etc/init.d/syslog restart
测试：logger -p local4.err "This is a local.err test message."
参考文献：
http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0
如大家有什么疑问或感兴趣的话题可以通过weibo与我交流：http://t.qq.com/yorkoliu
目前查看系统日志比较被动，遇到系统不正常或故障时才会主动去检查服务器系统日志，这样一来不能及时了解系统的运行情况，因此部署Logzilla+sphine+syslog-ng来弥补这不足。以下为安装、部署平台详细步骤。(Logzilla是什么新东西？其实前身就是php-syslog-ng，引用作者的话“Php-syslog-ng is now known as LogZilla. Same owner, better code :-)”)
二、平台初始化
#yum install libdbi* libnet
#cpan Date::Calc Text::LevenshteinXS String::CRC32
三、下载相关包
#cd /home/install
#mkdir logzilla;cd logzilla
#wget http://www.balabit.com/downloads/files/eventlog/0.2/eventlog_0.2.9.tar.gz
#wget http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.0.3/setups/rhel-5-i386/syslog-ng-3.0.3-1.rhel5.i386.rpm
四、开始安装
# cp eventlog_0.2.9.tar.gz /usr/src/redhat/SOURCES/
# tar zxvf eventlog_0.2.9.tar.gz
# cd eventlog-0.2.9/
# rpmbuild --ba eventlog.spec.bb
# cd /usr/src/redhat/RPMS/x86_64
# rpm -Uvh libevtlog*
#cd /home/install/logzilla
#rpm -Uvh syslog-ng-3.0.3-1.rhel5.i386.rpm
五、安装logzilla
#cd /www/webroot/
#wget http://php-syslog-ng.googlecode.com/files/logzilla_3.0.85.tgz
#tar -zxvf logzilla_3.0.85.tgz
#cd logzilla/scripts
#./install.pl
(根据实际情况来回应就OK了)
引用
===================
LogZilla Installation
====================
Enter the MySQL root username [root]:
Enter the password for root [mysql]:
Database to install to [syslog]:
Database table to install to [logs]:
Enter the name of the MySQL server [127.0.0.1]:
Enter the port of the MySQL server [3306]:
Enter the name to create as the owner of the logs database [syslogadmin]:
Enter the password for the syslogadmin user [syslogadmin]:
Enter the name to create as the WEBSITE owner [admin]:
Enter the password for admin [admin]:
Enter your email address [cdukes@cdukes.com]:
Enter a name for your website [The home of LogZilla]:
Enter the base url for your site (include trailing slash) [/logs/]: /
Where should log files be stored? [/var/log/logzilla]:
How long should I keep old logs? (in days) [30]:
========================================
Path Updates
========================================
Getting ready to replace paths in all files with "/www/webroot/logzilla"
Ok to continue? [y]:
Updating file paths
Modifying ../scripts/db_insert.pl
Modifying ../scripts/contrib/system_configs/logzilla.crontab
Modifying ../scripts/contrib/system_configs/syslog-ng.conf
Modifying ../scripts/contrib/system_configs/logzilla.apache
Modifying ../sphinx/indexer.sh
Modifying ../sphinx/sphinx.conf
Updating log paths
Modifying ../scripts/contrib/system_configs/logzilla.crontab
Modifying ../scripts/contrib/system_configs/logzilla.logrotate
====================
Database Installation
====================
All data will be installed into the syslog database
Ok to continue? [y]:
====================
Config.php generation
====================
Generating /www/webroot/logzilla/html/config/config.php
Ok to continue? [y]:
====================
System files
====================
Adding LogZilla logrotate.d file to /etc/logrotate.d
Ok to continue? [y]:
Where is your syslog-ng.conf file located? [/etc/syslog-ng/syslog-ng.conf]: /opt/syslog-ng/etc/syslog-ng.conf
Adding syslog-ng configuration to /opt/syslog-ng/etc/syslog-ng.conf
Ok to continue? [y]:
Found 1 sources
Which source definition would you like to use? [s_all]:
LogZilla installation complete...
Note: you may need to enable the MySQL Event Scheduler in your /etc/my.cnf file.
Please visit http://forum.logzilla.info/index.php/topic,71.0.html for more information.
Also, please visit http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0#UDP_Buffers to learn how to increase your UDP buffer size (otherwise you may drop messages).
Please run /etc/init.d/syslog-ng restart
六、安装 Sphinx
#cd logzilla/sphinx/src
#tar xzvf sphinx-0.9.9.tar.gz
#cd sphinx-0.9.9
#./configure --prefix `pwd`/../..
#make && make install
#cd /www/webroot/logzilla/sphinx
#vi sphinx.conf
#!/usr/bin/php 替换成实际php的位置，如#!/usr/local/php/bin/php
#./indexer.sh full
#bin/searchd
提示：
ERROR: index 'idx_logs': sql_query_pre[0]: Column 'max_id' cannot be null (DSN=mysql://syslogadmin:***@127.0.0.1:3306/syslog).
如果看到该信息，属正常，因为现在还没有数据:)
添加作业：
引用
30 0 1 * */www/webroot/logzilla/sphinx/indexer.sh full >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
*/5 * * * */www/webroot/logzilla/sphinx/indexer.sh delta >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
0 0 * * */www/webroot/logzilla/sphinx/indexer.sh merge >>/www/webroot/logzilla/sphinx/log/sphinx_indexer.log 2>&1
添加rc.local
#vi /etc/rc.local
引用
/www/webroot/logzilla/sphinx/bin/searchd -c /www/webroot/logzilla/sphinx/sphinx.conf
七、配置Mysql
激活event_scheduler
八、修改Syslog-ng配置
v#i /opt/syslog-ng/etc/syslog-ng.conf
(略)
九、修改apache配置
#vi httpd.conf
重启apache服务：/etc/init.d/apache2 restart
十、IONCube授权
http://127.0.0.1/logs/login.php
下载源码包ioncube_loaders_lin_x86-64.tar.gz
#mkdir -p /usr/local/ioncube
#tar -zxvf ioncube_loaders_lin_x86-64.tar.gz
#cd ioncube
#cp * /usr/local/ioncube/
#vi /usr/local/php/lib/php.ini
zend_extension = /usr/local/ioncube/ioncube_loader_lin_5.2.so
#/etc/init.d/apache2 restart
十一、安装完毕
http://127.0.0.1/logs/login.php
根据install.pl配置的管理员帐号密码进行登录。
1、MainPage
2、StatPage
十二、客户端配置
#vi /etc/syslog.conf
在最后添加以下，其中syslog.admin.com.cn为主机域名，也可以直接用IP代替。
引用
*.emerg;*.err;*.warning @syslog.admin.com.cn
#/etc/init.d/syslog restart
测试：logger -p local4.err "This is a local.err test message."
参考文献：
http://nms.gdd.net/index.php/Install_Guide_for_LogZilla_v3.0
如大家有什么疑问或感兴趣的话题可以通过weibo与我交流：http://t.qq.com/yorkoliu
python IP 处理
LAMP+logzill
Can't locate Text/LevenshteinXS.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8 .) at ./install.pl line 29.
BEGIN failed--compilation aborted at ./install.pl line 29