ElasticSearch, Kibana, and Fluentd 的简略部署实例

ElasticSearch, Kibana, and Fluentd 的简单部署实例
参考：http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd

要求：JDK 1.6及以上，ruby 1.9.2及以上(最后附安装过程mac os)



一、安装ElasticSearch：
1、下载：http://www.elasticsearch.org/download/，本人下载的是：elasticsearch-0.90.3.tar.gz 包

                                                           [FAILED]
file2.txt
# /usr/lib64/fluent/ruby/bin/gem environment
~~~~~中略~~~~~~~~~~~~~
  - GEM PATHS:
     - /usr/local/rvm/gems/ruby-1.9.3-p327
     - /usr/local/rvm/gems/ruby-1.9.3-p327@global
~~~~~中略~~~~~~~~~~~~~
file3.txt
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems
file4.txt

GEM_HOME="/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/"
GEM_PATH="/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/"
/usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-mysql


-----------------------------------------------------
三个节点：192.168.1.4，192.168.1.5，192.168.1.6；

fluentd的配置，
  
192.168.1.4
********************************************************************  
  
  <source>
    type http
    port 8888
  </source>
  
  <source>
    type forward
    port 24224
    bind 0.0.0.0
  </source>
  
  <source>
    type tail
    format /^ *(?<level>[^ ]* )(?<type>[^ ]* )(?<time>[^ ]* [^ ]* )(?<message>.*)$/
    path /data/cassandra/log/system.log
    pos_file /root/logserver/fluentd/logpos/system.pos
    tag 192.168.1.4
  </source>
  
  <match 192.**>
  type copy
  <store>
   type file
   path /root/logserver/fluentd/log/cassandra
  </store>
  <store>
   type elasticsearch
   logstash_format true
   flush_interval 10s
        include_tag_key true
        tag_key host
  </store>
  </match>
  
  <match debug.**>
    type stdout
  </match>
  
  
192.168.1.5/6  
********************************************************************
  <source>
    type http
    port 8888
  </source>
  <source>
    type tail
    format /^ *(?<level>[^ ]* )(?<type>[^ ]* )(?<time>[^ ]* [^ ]* )(?<message>.*)$/
    path /data/cassandra/log/system.log
    pos_file /root/logserver/fluentd/logpos/system.pos
    tag 192.168.1.5
  </source>
  <match debug.**>
    type stdout
  </match>
  <match 192.**>
    type forward
    send_timeout 60s
    recover_wait 10s
    heartbeat_interval 10s
    phi_threshold 8
    hard_timeout 60s
    <server>
      name icecrown
      host 192.168.1.4
      port 24224
      weight 60
    </server>
    <secondary>
      type file
      path /root/logserver/fluentd/forward-failed
    </secondary>
  </match>
  ********************************************************************
--------------ok-------------------------------------



安装相关资料
ruby install:
1、rvm:的安装（http://stackoverflow.com/questions/11677771/rvm-command-not-found-mac-ox）
或者（this worked for me):
$ rvm use 1.9.3 --default


linux 上的安装：http://tecadmin.net/how-to-install-ruby-2-0-0-on-centos-6-using-rvm/#