Logstash indexer not updating elasticsearch with search-guard enabled

Hello,

I’m running these versions for our two node (server) ELK stack on CentOS 6.7.

Logstash 1.5.3

Elasticsearch 1.7.1

Kibana 4.0.1

Redis 2.8.6

search-guard 1.7.3.0

I installed bin/plugin -i com.floragunn/search-guard/1.7.3.0. Then configured the elasticsearch.yml, kibana.yml and the authentication works where I can log into elasticsearch and kibana. I’m using just the basic authentication setup in the https://github.com/floragunncom/search-guard/blob/master/searchguard_config_example_1.yml example to see if I can get it working.

My issue is that the Logstash indexer isn’t updating elasticsearch. I tried this in the indexer.conf file and the log says “Error: User and password parameters are not supported for ‘node’. Change the protocol to ‘http’ if you need them.”

output {

elasticsearch {

cluster => “elasticsearch_preprod”

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:20:36.798000-0500”, :message=>“Unknown setting ‘cluster’ for elasticsearch_http”, :level=>:error}.

output {

elasticsearch_http {

cluster => “elasticsearch_preprod”

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:24:00.313000-0500”, :message=>“Invalid setting for elasticsearch_http output plugin:\n\n output {\n elasticsearch_http {\n # This setting must be a string\n # Expected string, got ["172.26.2.209:9200", "172.26.2.240:9200"]\n host => ["172.26.2.209:9200", "172.26.2.240:9200"]\n …\n }\n }”, :level=>:error}

output {

elasticsearch_http {

host => [“172.26.2.209:9200”, “172.26.2.240:9200”]

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:25:04.250000-0500”, :message=>“Unknown setting ‘hosts’ for elasticsearch_http”, :level=>:error}

output {

elasticsearch_http {

hosts => [“172.26.2.209:9200”, “172.26.2.240:9200”]

user => “admin”

password => “secret”

}

}

So I’m looking for suggestions on how to configure so the indexer will update elasticsearch in my cluster. Thank you.

I figured it out. Below is what was needed for the indexer to connect with the example I originally provided.

output {

elasticsearch {

protocol => http

cluster => “elasticsearch_prod”

user => admin

password => secret

}

}

···

On Thursday, March 17, 2016 at 2:35:23 PM UTC-5, Cds Support wrote:

Hello,

I’m running these versions for our two node (server) ELK stack on CentOS 6.7.

Logstash 1.5.3

Elasticsearch 1.7.1

Kibana 4.0.1

Redis 2.8.6

search-guard 1.7.3.0

I installed bin/plugin -i com.floragunn/search-guard/1.7.3.0. Then configured the elasticsearch.yml, kibana.yml and the authentication works where I can log into elasticsearch and kibana. I’m using just the basic authentication setup in the https://github.com/floragunncom/search-guard/blob/master/searchguard_config_example_1.yml example to see if I can get it working.

My issue is that the Logstash indexer isn’t updating elasticsearch. I tried this in the indexer.conf file and the log says “Error: User and password parameters are not supported for ‘node’. Change the protocol to ‘http’ if you need them.”

output {

elasticsearch {

cluster => “elasticsearch_preprod”

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:20:36.798000-0500”, :message=>“Unknown setting ‘cluster’ for elasticsearch_http”, :level=>:error}.

output {

elasticsearch_http {

cluster => “elasticsearch_preprod”

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:24:00.313000-0500”, :message=>“Invalid setting for elasticsearch_http output plugin:\n\n output {\n elasticsearch_http {\n # This setting must be a string\n # Expected string, got ["172.26.2.209:9200", "172.26.2.240:9200"]\n host => ["172.26.2.209:9200", "172.26.2.240:9200"]\n …\n }\n }”, :level=>:error}

output {

elasticsearch_http {

host => [“172.26.2.209:9200”, “172.26.2.240:9200”]

user => “admin”

password => “secret”

}

}

Then I tried with this configuration below and I get {:timestamp=>“2016-03-17T14:25:04.250000-0500”, :message=>“Unknown setting ‘hosts’ for elasticsearch_http”, :level=>:error}

output {

elasticsearch_http {

hosts => [“172.26.2.209:9200”, “172.26.2.240:9200”]

user => “admin”

password => “secret”

}

}

So I’m looking for suggestions on how to configure so the indexer will update elasticsearch in my cluster. Thank you.