# Trouble with elasticsearch-filter-plugin **URL:** \ **Category:** Search Guard\ **Created:** [September 10, 2017, 1:17pm UTC](https://forum.search-guard.com/t/trouble-with-elasticsearch-filter-plugin/597 "2017-09-10T13:17:55Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![cito.ets](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@cito.ets](https://forum.search-guard.com/u/cito.ets)\ **Post date:** [September 10, 2017, 1:17pm UTC](https://forum.search-guard.com/t/trouble-with-elasticsearch-filter-plugin/597/1 "2017-09-10T13:17:55Z")
Hi, A bit new into SSL/TLS. I do have managed to get Logstash to work with Searchguard/Elasticsearch (elasticsearch output) on a test server but when trying to use the **elasticsearch filter plugin** I get a ConnectionFailed warning in Logstash but nothing from Elasticsearch. logstash.conf filter { … elasticsearch { hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] index =\> “logstash-index-ref” user =\> “logstash” password =\> “\*\*\*\*\*” ssl =\> true query =\> “BusinessEmail:%{user}” fields =\> { “FirstName” =\> “FirstName” } } … } `` output { elasticsearch { hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] index =\> “logstash-%{+YYYY.MM.dd}” user =\> “logstash” password =\> “\*\*\*\*\*” ssl =\> true ssl\_certificate\_verification =\> false truststore =\> “/etc/elasticsearch/truststore.jks” truststore\_password =\> “changeit” } stdout { codec =\> rubydebug } } `` sg\_roles.yml sg\_logstash: cluster: - indices:admin/template/get - indices:admin/template/put - indices:data/write/bulk\* indices: ‘logstash-_': '_’: - CRUD - CREATE\_INDEX ‘_beat_’: ‘\*’: - CRUD - CREATE\_INDEX `` I’m using the **search-guard demo**. Security works across Logstash-\>ES-\>Kibana, except I really couldn’t get the elasticsearch filter plugin to work. logstash-plain.log [2017-09-10T13:31:19,688][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“BusinessEmail:-”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:47.805788Z _ **.** _._ **.** _ - 2017-09-07T04:37:47.808586Z 3.798ms HTTP/1.1 401 Unauthorized Content-Type: application/json;charset=UTF-8 { “responseCode”: 401 } 2017-09-07T04:37:47.809702Z 4.914ms , :error=\>#Faraday::ConnectionFailed} [2017-09-10T13:31:19,736][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“[BusinessEmail:test@test.com](mailto:BusinessEmail:test@test.com)”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:48.216151Z _ **.** _._ **.** _ [test@test.com](mailto:test@test.com) 2017-09-07T04:37:48.852590Z 637.439ms HTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 { “responseCode”: 200 } 2017-09-07T04:37:48.853740Z 638.589ms , :error=\>#Faraday::ConnectionFailed} `` Anyone have an idea? The elasticsearch-filter-plugin doc doesn’t have the other security options in their elasticsearch output API. Cinto ---
**Author:** ![searchguard\_google\_group](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/searchguard_google_group/32/462_2.png) [@searchguard\_google\_group](https://forum.search-guard.com/u/searchguard_google_group)\ **Post date:** [September 10, 2017, 5:11pm UTC](https://forum.search-guard.com/t/trouble-with-elasticsearch-filter-plugin/597/2 "2017-09-10T17:11:05Z")
this seems like a logstash issue so please file an issue here [Issues · elastic/logstash · GitHub](https://github.com/elastic/logstash/issues) or [Logstash - Discuss the Elastic Stack](https://discuss.elastic.co/c/logstash) > **···** > > On Sunday, 10 September 2017 15:17:55 UTC+2, c…s@gm…il.com wrote: > > > Hi, > > > A bit new into SSL/TLS. I do have managed to get Logstash to work with Searchguard/Elasticsearch (elasticsearch output) on a test server but when trying to use the **elasticsearch filter plugin** I get a ConnectionFailed warning in Logstash but nothing from Elasticsearch. > > > > > > logstash.conf > > > filter { > > … > > elasticsearch { > > hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] > > index =\> “logstash-index-ref” > > user =\> “logstash” > > password =\> “\*\*\*\*\*” > > ssl =\> true > > query =\> “BusinessEmail:%{user}” > > fields =\> { “FirstName” =\> “FirstName” } > > } > > … > > } > > > `` > > > > > > output { > > elasticsearch { > > hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] > > index =\> “logstash-%{+YYYY.MM.dd}” > > user =\> “logstash” > > password =\> “\*\*\*\*\*” > > ssl =\> true > > ssl\_certificate\_verification =\> false > > truststore =\> “/etc/elasticsearch/truststore.jks” > > truststore\_password =\> “changeit” > > } > > stdout { codec =\> rubydebug } > > } > > > `` > > > > > > sg\_roles.yml > > > sg\_logstash: > > cluster: > > - indices:admin/template/get > > - indices:admin/template/put > > - indices:data/write/bulk\* > > indices: > > ‘logstash-_': > > '_’: > > - CRUD > > - CREATE\_INDEX > > ‘_beat_’: > > ‘\*’: > > - CRUD > > - CREATE\_INDEX > > > `` > > > > > > > > > I’m using the **search-guard demo**. Security works across Logstash-\>ES-\>Kibana, except I really couldn’t get the elasticsearch filter plugin to work. > > > > > > logstash-plain.log > > > [2017-09-10T13:31:19,688][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“BusinessEmail:-”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:47.805788Z _ **.** _._ **.** _ - > > 2017-09-07T04:37:47.808586Z 3.798ms > > HTTP/1.1 401 Unauthorized > > Content-Type: application/json;charset=UTF-8 > > > > { > > “responseCode”: 401 > > } > > 2017-09-07T04:37:47.809702Z 4.914ms > > > > , :error=\>#Faraday::ConnectionFailed} > > [2017-09-10T13:31:19,736][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“[BusinessEmail:test@test.com](mailto:BusinessEmail:test@test.com)”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:48.216151Z _ **.** _._ **.** _ [test@test.com](mailto:test@test.com) > > 2017-09-07T04:37:48.852590Z 637.439ms > > HTTP/1.1 200 OK > > Content-Type: application/json;charset=UTF-8 > > > > { > > “responseCode”: 200 > > } > > 2017-09-07T04:37:48.853740Z 638.589ms > > > > , :error=\>#Faraday::ConnectionFailed} > > > `` > > > > > > > > > Anyone have an idea? The elasticsearch-filter-plugin doc doesn’t have the other security options in their elasticsearch output API. > > > > > > Cinto ---
**Author:** ![cito.ets](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@cito.ets](https://forum.search-guard.com/u/cito.ets)\ **Post date:** [September 11, 2017, 9:57am UTC](https://forum.search-guard.com/t/trouble-with-elasticsearch-filter-plugin/597/3 "2017-09-11T09:57:28Z")
Thanks. Looked more into this, probably related to: > **[Logstash Lines: Introducing a benchmarking tool for Logstash](https://www.elastic.co/blog/logstash-lines-2017-08-01)** > > > I'm trying to use the elastic cloud as my filter source. I have a config like: > … > > filter { > elasticsearch { > hosts =\> \["https://FOUND\_SERVER.us-east-1.aws.found.io:9243"\] > ssl =\> true > user =\> "elastic" > password =\> "changeme" > } > } > > This doesn't work. I tried also uninstalling the elasticsearch input plugin, which was pinning elasticsearch ruby client library to 1.0 so that installing the elasticsearch filter would install the 5.0 version of the ruby client library, but it didn't change anything. > > Using http and ssl =\> false and port 9200 works fine for accessing the cluster so it does appear to be some sort of bug in the filter code. > **···** > > On Monday, September 11, 2017 at 1:11:05 AM UTC+8, Search Guard wrote: > > > this seems like a logstash issue so please file an issue here [https://github.com/elastic/logstash/issues](https://github.com/elastic/logstash/issues) or [https://discuss.elastic.co/c/logstash](https://discuss.elastic.co/c/logstash) > > > > On Sunday, 10 September 2017 15:17:55 UTC+2, c…s@gm…[il.com](http://il.com) wrote: > > > > > Hi, > > > > A bit new into SSL/TLS. I do have managed to get Logstash to work with Searchguard/Elasticsearch (elasticsearch output) on a test server but when trying to use the **elasticsearch filter plugin** I get a ConnectionFailed warning in Logstash but nothing from Elasticsearch. > > > > > > > > logstash.conf > > > > filter { > > > … > > > elasticsearch { > > > hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] > > > index =\> “logstash-index-ref” > > > user =\> “logstash” > > > password =\> “\*\*\*\*\*” > > > ssl =\> true > > > query =\> “BusinessEmail:%{user}” > > > fields =\> { “FirstName” =\> “FirstName” } > > > } > > > … > > > } > > > > `` > > > > > > > > output { > > > elasticsearch { > > > hosts =\> [“[https://127.0.0.1:9200](https://127.0.0.1:9200)”] > > > index =\> “logstash-%{+YYYY.MM.dd}” > > > user =\> “logstash” > > > password =\> “\*\*\*\*\*” > > > ssl =\> true > > > ssl\_certificate\_verification =\> false > > > truststore =\> “/etc/elasticsearch/truststore.jks” > > > truststore\_password =\> “changeit” > > > } > > > stdout { codec =\> rubydebug } > > > } > > > > `` > > > > > > > > sg\_roles.yml > > > > sg\_logstash: > > > cluster: > > > - indices:admin/template/get > > > - indices:admin/template/put > > > - indices:data/write/bulk\* > > > indices: > > > ‘logstash-_': > > > '_’: > > > - CRUD > > > - CREATE\_INDEX > > > ‘_beat_’: > > > ‘\*’: > > > - CRUD > > > - CREATE\_INDEX > > > > `` > > > > > > > > > > > > I’m using the **search-guard demo**. Security works across Logstash-\>ES-\>Kibana, except I really couldn’t get the elasticsearch filter plugin to work. > > > > > > > > logstash-plain.log > > > > [2017-09-10T13:31:19,688][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“BusinessEmail:-”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:47.805788Z _ **.** _._ **.** _ - > > > 2017-09-07T04:37:47.808586Z 3.798ms > > > HTTP/1.1 401 Unauthorized > > > Content-Type: application/json;charset=UTF-8 > > > > > > { > > > “responseCode”: 401 > > > } > > > 2017-09-07T04:37:47.809702Z 4.914ms > > > > > > , :error=\>#Faraday::ConnectionFailed} > > > [2017-09-10T13:31:19,736][WARN][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=\>“logstash-index-ref”, :query=\>“[BusinessE...@test.com](mailto:BusinessE...@test.com)”, :event=\>2017-09-10T12:30:54.824Z ubuntu 2017-09-07T04:37:48.216151Z _ **.** _._ **.** _ [test@test.com](mailto:test@test.com) > > > 2017-09-07T04:37:48.852590Z 637.439ms > > > HTTP/1.1 200 OK > > > Content-Type: application/json;charset=UTF-8 > > > > > > { > > > “responseCode”: 200 > > > } > > > 2017-09-07T04:37:48.853740Z 638.589ms > > > > > > , :error=\>#Faraday::ConnectionFailed} > > > > `` > > > > > > > > > > > > Anyone have an idea? The elasticsearch-filter-plugin doc doesn’t have the other security options in their elasticsearch output API. > > > > > > > > Cinto