# Slow indexing with searchguard **URL:** \ **Category:** Search Guard\ **Created:** [June 10, 2016, 4:31pm UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186 "2016-06-10T16:31:10Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Jainin\_Shah](https://avatars.discourse-cdn.com/v4/letter/j/46a35a/32.png) [@Jainin\_Shah](https://forum.search-guard.com/u/Jainin_Shah)\ **Post date:** [June 10, 2016, 4:31pm UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186/1 "2016-06-10T16:31:10Z")
Hi, We have installed searchguard 2.3.2-beta2 in our elasticsearch cluster. Everything went well with the installation. But what we noticed is that bulk indexing (from hdfs to es particularly) is quite slow after enabling searchguard in the cluster. For ex, we were indexing 50 million document in our cluster, which would generally take no more than few hours (usually between 2-3), but after installing searchguard the same indexing took more than 9 hours. Is that an expected behavior while indexing with searchguard? and/or Is that fixed (or improved) in the RC1 release? Thank you Jainin Shah ---
**Author:** ![searchguard\_google\_group](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/searchguard_google_group/32/462_2.png) [@searchguard\_google\_group](https://forum.search-guard.com/u/searchguard_google_group)\ **Post date:** [June 10, 2016, 6:03pm UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186/2 "2016-06-10T18:03:13Z")
This is not expected. Can you post your configuration (elasticsearch.yml and sg\_config.yml) and explain a little bit more in detail how your ingestion work and what kind of authentication you use. Forn example: Do you use HTTP or Java Client for indexing? If HTTP: Do you have SSL enabled and if so do you use native OpenSSL support? Maybe SSL is slowing down the whole thing. Beta1 introduced caching so hitting the authentication backend should normally not cause any pain. Please provide also informations for the folowing questions: - How many nodes do you have in your cluster? - JVM vendor and version - Operation system vendor and version - OpenSSL version Thx > **···** > > > Am 10.06.2016 um 18:31 schrieb Jainin Shah \: > > > > Hi, > > > > We have installed searchguard 2.3.2-beta2 in our elasticsearch cluster. Everything went well with the installation. But what we noticed is that bulk indexing (from hdfs to es particularly) is quite slow after enabling searchguard in the cluster. > > > > For ex, we were indexing 50 million document in our cluster, which would generally take no more than few hours (usually between 2-3), but after installing searchguard the same indexing took more than 9 hours. > > > > Is that an expected behavior while indexing with searchguard? and/or > > Is that fixed (or improved) in the RC1 release? > > > > Thank you > > > > Jainin Shah > > > > -- > > You received this message because you are subscribed to the Google Groups "Search Guard" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com. > > To post to this group, send email to search-guard@googlegroups.com. > > To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com\](https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com%5C). > > For more options, visit [https://groups.google.com/d/optout\](https://groups.google.com/d/optout%5C). ---
**Author:** ![Jainin\_Shah](https://avatars.discourse-cdn.com/v4/letter/j/46a35a/32.png) [@Jainin\_Shah](https://forum.search-guard.com/u/Jainin_Shah)\ **Post date:** [June 10, 2016, 7:54pm UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186/3 "2016-06-10T19:54:01Z")
Thanks for the prompt reply. To answer your questions: - How many nodes do you have in your cluster? : 10 - JVM vendor and version : java version “1.7.0\_75” Java™ SE Runtime Environment (build 1.7.0\_75-b13) Java HotSpot™ 64-Bit Server VM (build 24.75-b04, mixed mode) - Operation system vendor and version CentOS release 6.8 (Final) - OpenSSL version OpenSSL 1.0.1e-fips 11 Feb 2013 Also, attached are the elasticsearch.yml file and sg\_config.yml file. sg\_config.yml is exactly the same as it comes with the package. And we do not use java client to index, it’s going through http using elasticsearch-spark. And we use basic authentication (username and password) with searchguard. [sg\_config.yml](https://forum.search-guard.com/uploads/short-url/2ygVxjsrIyoNU6fEvVRiuht9rOd.yml) (1.19 KB) [elasricsearch.yml](https://forum.search-guard.com/uploads/short-url/MEl9Uoifdn8FRrosmiIlFzA5Ic.yml) (2.05 KB) > **···** > > On Friday, June 10, 2016 at 2:03:14 PM UTC-4, SG wrote: > > > This is not expected. Can you post your configuration (elasticsearch.yml and sg\_config.yml) and explain a little bit more in detail how your ingestion work and what kind of authentication you use. > > > > Forn example: Do you use HTTP or Java Client for indexing? If HTTP: Do you have SSL enabled and if so do you use native OpenSSL support? > > > > Maybe SSL is slowing down the whole thing. Beta1 introduced caching so hitting the authentication backend should normally not cause any pain. > > > > Please provide also informations for the folowing questions: > > > > - How many nodes do you have in your cluster? > > > > - JVM vendor and version > > > > - Operation system vendor and version > > > > - OpenSSL version > > > > Thx > > > > > Am 10.06.2016 um 18:31 schrieb Jainin Shah [jai...@tresata.com](mailto:jai...@tresata.com): > > > > > Hi, > > > > > We have installed searchguard 2.3.2-beta2 in our elasticsearch cluster. Everything went well with the installation. But what we noticed is that bulk indexing (from hdfs to es particularly) is quite slow after enabling searchguard in the cluster. > > > > > For ex, we were indexing 50 million document in our cluster, which would generally take no more than few hours (usually between 2-3), but after installing searchguard the same indexing took more than 9 hours. > > > > > Is that an expected behavior while indexing with searchguard? and/or > > > > > Is that fixed (or improved) in the RC1 release? > > > > > Thank you > > > > > Jainin Shah > > > > > – > > > You received this message because you are subscribed to the Google Groups “Search Guard” group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [search-guard...@googlegroups.com](mailto:search-guard...@googlegroups.com). > > > > > To post to this group, send email to [search...@googlegroups.com](mailto:search...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com](https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com). > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![searchguard\_google\_group](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/searchguard_google_group/32/462_2.png) [@searchguard\_google\_group](https://forum.search-guard.com/u/searchguard_google_group)\ **Post date:** [June 11, 2016, 8:12am UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186/4 "2016-06-11T08:12:40Z")
Can you try RC1 because it contains SSL related fixes which should improve speed and fix a possible memory leak with OpenSSL (Beware: config has slightly changed for RC1) I see from you config file that you NOT using OpenSSL (enable\_openssl\_if\_available: false). We strongly recommend to let OpenSSL handle SSL because Java built-in SSL support in slow (especially in Java 7, therefore you should also consider upgrading to Oracle Java 8) See here how to enable OpenSSL support for Search Guard. > **[GitHub - floragunncom/search-guard-ssl: Search Guard SSL Plugin](https://github.com/floragunncom/search-guard-ssl)** > > Search Guard SSL Plugin. Contribute to floragunncom/search-guard-ssl development by creating an account on GitHub. Another question: It seems do that you not encrypt your HTTP traffic. Plain HTTP together with basic authentication means that the plaintext password will be sent over the network. Consider setting searchguard.ssl.http.enabled: true to cope with that. > **···** > > > Am 10.06.2016 um 21:54 schrieb Jainin Shah \: > > > > Thanks for the prompt reply. > > > > To answer your questions: > > > > - How many nodes do you have in your cluster? : 10 > > > > - JVM vendor and version : > > java version "1.7.0\_75" > > Java(TM) SE Runtime Environment (build 1.7.0\_75-b13) > > Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode) > > > > - Operation system vendor and version > > CentOS release 6.8 (Final) > > > > - OpenSSL version > > OpenSSL 1.0.1e-fips 11 Feb 2013 > > > > Also, attached are the elasticsearch.yml file and sg\_config.yml file. sg\_config.yml is exactly the same as it comes with the package. > > > > And we do not use java client to index, it's going through http using elasticsearch-spark. And we use basic authentication (username and password) with searchguard. > > > > On Friday, June 10, 2016 at 2:03:14 PM UTC-4, SG wrote: > > This is not expected. Can you post your configuration (elasticsearch.yml and sg\_config.yml) and explain a little bit more in detail how your ingestion work and what kind of authentication you use. > > Forn example: Do you use HTTP or Java Client for indexing? If HTTP: Do you have SSL enabled and if so do you use native OpenSSL support? > > Maybe SSL is slowing down the whole thing. Beta1 introduced caching so hitting the authentication backend should normally not cause any pain. > > > > Please provide also informations for the folowing questions: > > - How many nodes do you have in your cluster? > > - JVM vendor and version > > - Operation system vendor and version > > - OpenSSL version > > > > Thx > > > > \> Am 10.06.2016 um 18:31 schrieb Jainin Shah \: > > \> > > \> Hi, > > \> > > \> We have installed searchguard 2.3.2-beta2 in our elasticsearch cluster. Everything went well with the installation. But what we noticed is that bulk indexing (from hdfs to es particularly) is quite slow after enabling searchguard in the cluster. > > \> > > \> For ex, we were indexing 50 million document in our cluster, which would generally take no more than few hours (usually between 2-3), but after installing searchguard the same indexing took more than 9 hours. > > \> > > \> Is that an expected behavior while indexing with searchguard? and/or > > \> Is that fixed (or improved) in the RC1 release? > > \> > > \> Thank you > > \> > > \> Jainin Shah > > \> > > \> -- > > \> You received this message because you are subscribed to the Google Groups "Search Guard" group. > > \> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com. > > \> To post to this group, send email to search...@googlegroups.com. > > \> To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com\](https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com%5C). > > \> For more options, visit [https://groups.google.com/d/optout\](https://groups.google.com/d/optout%5C). > > > > -- > > You received this message because you are subscribed to the Google Groups "Search Guard" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com. > > To post to this group, send email to search-guard@googlegroups.com. > > To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/c31c23da-503b-4095-9c9a-a326b1beef8c%40googlegroups.com\](https://groups.google.com/d/msgid/search-guard/c31c23da-503b-4095-9c9a-a326b1beef8c%40googlegroups.com%5C). > > For more options, visit [https://groups.google.com/d/optout\](https://groups.google.com/d/optout%5C). > > \\ ---
**Author:** ![Jainin\_Shah](https://avatars.discourse-cdn.com/v4/letter/j/46a35a/32.png) [@Jainin\_Shah](https://forum.search-guard.com/u/Jainin_Shah)\ **Post date:** [June 12, 2016, 4:54pm UTC](https://forum.search-guard.com/t/slow-indexing-with-searchguard/186/5 "2016-06-12T16:54:09Z")
Thanks. Will give RC1 and OpenSSL a try and see if that increases that speed. > **···** > > On Saturday, June 11, 2016 at 4:12:38 AM UTC-4, SG wrote: > > > Can you try RC1 because it contains SSL related fixes which should improve speed and fix a possible memory leak with OpenSSL (Beware: config has slightly changed for RC1) > > > > I see from you config file that you NOT using OpenSSL (enable\_openssl\_if\_available: false). We strongly recommend to let OpenSSL handle SSL because Java built-in SSL support in slow > > (especially in Java 7, therefore you should also consider upgrading to Oracle Java 8) > > > > See here how to enable OpenSSL support for Search Guard. > > > > [https://github.com/floragunncom/search-guard-ssl/wiki/Open-SSL-setup/f618d0efa96cc9d598b28439b25464047c5cc98a](https://github.com/floragunncom/search-guard-ssl/wiki/Open-SSL-setup/f618d0efa96cc9d598b28439b25464047c5cc98a) > > > > Another question: It seems do that you not encrypt your HTTP traffic. Plain HTTP together with basic authentication means that the plaintext password will be sent over the network. > > > > Consider setting searchguard.ssl.http.enabled: true to cope with that. > > > > > Am 10.06.2016 um 21:54 schrieb Jainin Shah [jai...@tresata.com](mailto:jai...@tresata.com): > > > > > Thanks for the prompt reply. > > > > > To answer your questions: > > > > > - How many nodes do you have in your cluster? : 10 > > > > > - JVM vendor and version : > > > > > java version “1.7.0\_75” > > > > > Java™ SE Runtime Environment (build 1.7.0\_75-b13) > > > > > Java HotSpot™ 64-Bit Server VM (build 24.75-b04, mixed mode) > > > > > - Operation system vendor and version > > > CentOS release 6.8 (Final) > > > > > - OpenSSL version > > > OpenSSL 1.0.1e-fips 11 Feb 2013 > > > > > Also, attached are the elasticsearch.yml file and sg\_config.yml file. sg\_config.yml is exactly the same as it comes with the package. > > > > > And we do not use java client to index, it’s going through http using elasticsearch-spark. And we use basic authentication (username and password) with searchguard. > > > > > On Friday, June 10, 2016 at 2:03:14 PM UTC-4, SG wrote: > > > > > This is not expected. Can you post your configuration (elasticsearch.yml and sg\_config.yml) and explain a little bit more in detail how your ingestion work and what kind of authentication you use. > > > Forn example: Do you use HTTP or Java Client for indexing? If HTTP: Do you have SSL enabled and if so do you use native OpenSSL support? > > > Maybe SSL is slowing down the whole thing. Beta1 introduced caching so hitting the authentication backend should normally not cause any pain. > > > > > > Please provide also informations for the folowing questions: > > > > > > - How many nodes do you have in your cluster? > > > - JVM vendor and version > > > - Operation system vendor and version > > > - OpenSSL version > > > > > > Thx > > > > > > > Am 10.06.2016 um 18:31 schrieb Jainin Shah [jai...@tresata.com](mailto:jai...@tresata.com): > > > > > > > > Hi, > > > > > > > > We have installed searchguard 2.3.2-beta2 in our elasticsearch cluster. Everything went well with the installation. But what we noticed is that bulk indexing (from hdfs to es particularly) is quite slow after enabling searchguard in the cluster. > > > > > > > > For ex, we were indexing 50 million document in our cluster, which would generally take no more than few hours (usually between 2-3), but after installing searchguard the same indexing took more than 9 hours. > > > > > > > > Is that an expected behavior while indexing with searchguard? and/or > > > > Is that fixed (or improved) in the RC1 release? > > > > > > > > Thank you > > > > > > > > Jainin Shah > > > > > > > > – > > > > You received this message because you are subscribed to the Google Groups “Search Guard” group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [search-guard...@googlegroups.com](mailto:search-guard...@googlegroups.com). > > > > To post to this group, send email to [search...@googlegroups.com](mailto:search...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com](https://groups.google.com/d/msgid/search-guard/aea72d49-2f6e-40c2-8d97-464744316fd6%40googlegroups.com). > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > – > > > You received this message because you are subscribed to the Google Groups “Search Guard” group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [search-guard...@googlegroups.com](mailto:search-guard...@googlegroups.com). > > > > > To post to this group, send email to [search...@googlegroups.com](mailto:search...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/search-guard/c31c23da-503b-4095-9c9a-a326b1beef8c%40googlegroups.com](https://groups.google.com/d/msgid/search-guard/c31c23da-503b-4095-9c9a-a326b1beef8c%40googlegroups.com). > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > \\