Enabling OpenSSL with Elasticsearch 6.2.1

Hi,

I’m trying to get Elasticsearch to start with OpenSSL, but I’m receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing …
[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]
[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config
[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]
[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- and truststore files are resolved relatively
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

``

Based on http://docs.search-guard.com/latest/openssl, I’ve installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I’m using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version

openjdk version “1.8.0_161”

OpenJDK Runtime Environment (build 1.8.0_161-b14)

OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl

openssl-libs-1.0.2k-8.el7.x86_64

openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr

apr-1.4.8-3.el7_4.1.x86_64

``

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard

searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem

searchguard.ssl.transport.pemkey_password: KEY_PASS

searchguard.ssl.transport.pemcert_filepath: node.crt.pem

searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemkey_filepath: node.key.pem

searchguard.ssl.http.pemkey_password: KEY_PASS

searchguard.ssl.http.pemcert_filepath: node.crt.pem

searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.http.clientauth_mode: OPTIONAL

searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

``

Thanks!

Dan

···

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version

  • Installed and used enterprise modules, if any

  • JVM version and operating system version

  • Search Guard configuration files

  • Elasticsearch log messages on debug level

  • Other installed Elasticsearch or Kibana plugins, if any

Additional information. Interesting how ssl_openssl_available is set to false, even though I have OpenSSL and apr packages installed, as far as I can tell.

{
  "principal" : null,
  "peer_certificates" : "0",
  "ssl_protocol" : "TLSv1.2",
  "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "ssl_openssl_available" : false,
  "ssl_openssl_version" : -1,
  "ssl_openssl_version_string" : null,
  "ssl_openssl_non_available_cause" : "java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]",
  "ssl_openssl_supports_key_manager_factory" : false,
  "ssl_openssl_supports_hostname_validation" : false,
  "ssl_provider_http" : "JDK",
  "ssl_provider_transport_server" : "JDK",
  "ssl_provider_transport_client" : "JDK"
}

``

Noting that with the same set up, Elasticsearch 6.1.3 seems to find the OpenSSL library.

{

“principal” : null,

“peer_certificates” : “0”,

“ssl_protocol” : “TLSv1.2”,

“ssl_cipher” : “TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256”,

“ssl_openssl_available” : true,

“ssl_openssl_version” : 268443839,

“ssl_openssl_version_string” : “OpenSSL 1.0.2k-fips 26 Jan 2017”,

“ssl_openssl_non_available_cause” : “”,

“ssl_openssl_supports_key_manager_factory” : true,

“ssl_openssl_supports_hostname_validation” : true,

“ssl_provider_http” : “OPENSSL”,

“ssl_provider_transport_server” : “OPENSSL”,

“ssl_provider_transport_client” : “OPENSSL”

}

``

···

On Tuesday, February 20, 2018 at 11:36:06 AM UTC-5, dc...@galileo.io wrote:

Hi,

I’m trying to get Elasticsearch to start with OpenSSL, but I’m receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing …
[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]
[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config
[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]
[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- and truststore files are resolved relatively
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

``

Based on http://docs.search-guard.com/latest/openssl, I’ve installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I’m using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version

openjdk version “1.8.0_161”

OpenJDK Runtime Environment (build 1.8.0_161-b14)

OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl

openssl-libs-1.0.2k-8.el7.x86_64

openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr

apr-1.4.8-3.el7_4.1.x86_64

``

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard

searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem

searchguard.ssl.transport.pemkey_password: KEY_PASS

searchguard.ssl.transport.pemcert_filepath: node.crt.pem

searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemkey_filepath: node.key.pem

searchguard.ssl.http.pemkey_password: KEY_PASS

searchguard.ssl.http.pemcert_filepath: node.crt.pem

searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.http.clientauth_mode: OPTIONAL

searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

``

Thanks!

Dan

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

you need the 2.0.7 version tcnative instead of 2.0.5, i updated the docs accordingly.
So pls try https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.7.Final-fedora-linux-x86_64.jar
See https://github.com/floragunncom/search-guard-docs/blob/6.x/_docs/tls_openssl.md for all download possibilities (dynamic and static)

···

Am 20.02.2018 um 17:49 schrieb dchan@galileo.io:

Additional information

{
  "principal" : null,
  "peer_certificates" : "0",
  "ssl_protocol" : "TLSv1.2",
  "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "ssl_openssl_available" : false,
  "ssl_openssl_version" : -1,
  "ssl_openssl_version_string" : null,
  "ssl_openssl_non_available_cause" : "java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]",
  "ssl_openssl_supports_key_manager_factory" : false,
  "ssl_openssl_supports_hostname_validation" : false,
  "ssl_provider_http" : "JDK",
  "ssl_provider_transport_server" : "JDK",
  "ssl_provider_transport_client" : "JDK"
}

On Tuesday, February 20, 2018 at 11:36:06 AM UTC-5, dc...@galileo.io wrote:
Hi,

I'm trying to get Elasticsearch to start with OpenSSL, but I'm receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing ...
[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]
[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config
[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-inJDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64,netty_tcnative_linux_x86_64_fedora, netty_tcnative]
[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- andtruststore files are resolved relatively
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

Based on http://docs.search-guard.com/latest/openssl, I've installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I'm using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version
openjdk version "1.8.0_161"
OpenJDK Runtime Environment (build 1.8.0_161-b14)
OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl
openssl-libs-1.0.2k-8.el7.x86_64
openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr
apr-1.4.8-3.el7_4.1.x86_64

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard
searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem
searchguard.ssl.transport.pemkey_password: __KEY_PASS__
searchguard.ssl.transport.pemcert_filepath: node.crt.pem
searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemkey_filepath: node.key.pem
searchguard.ssl.http.pemkey_password: __KEY_PASS__
searchguard.ssl.http.pemcert_filepath: node.crt.pem
searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]

Thanks!
Dan

--

When asking questions, please provide the following information:

* Search Guard and Elasticsearch version
* Installed and used enterprise modules, if any
* JVM version and operating system version
* Search Guard configuration files
* Elasticsearch log messages on debug level
* Other installed Elasticsearch or Kibana plugins, if any

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/62e28c04-944a-4bd4-9979-55d496adc0e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thanks for the quick reply! Using 2.0.7 version of tcnative with Elasticsearch 6.2.1 works on my setup.

Thank you for helping me out with the problem. In addition, will http://docs.search-guard.com/latest/openssl be updated as well?

···

On Wednesday, February 21, 2018 at 4:09:49 PM UTC-5, Search Guard wrote:

you need the 2.0.7 version tcnative instead of 2.0.5, i updated the docs accordingly.

So pls try https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.7.Final-fedora-linux-x86_64.jar

See https://github.com/floragunncom/search-guard-docs/blob/6.x/_docs/tls_openssl.md for all download possibilities (dynamic and static)

Am 20.02.2018 um 17:49 schrieb dc...@galileo.io:

Additional information

{

“principal” : null,

“peer_certificates” : “0”,

“ssl_protocol” : “TLSv1.2”,

“ssl_cipher” : “TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256”,

“ssl_openssl_available” : false,

“ssl_openssl_version” : -1,

“ssl_openssl_version_string” : null,

“ssl_openssl_non_available_cause” : “java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]”,

“ssl_openssl_supports_key_manager_factory” : false,

“ssl_openssl_supports_hostname_validation” : false,

“ssl_provider_http” : “JDK”,

“ssl_provider_transport_server” : “JDK”,

“ssl_provider_transport_client” : “JDK”

}

On Tuesday, February 20, 2018 at 11:36:06 AM UTC-5, dc...@galileo.io wrote:

Hi,

I’m trying to get Elasticsearch to start with OpenSSL, but I’m receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing …

[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]

[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]

[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config

[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-inJDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64,netty_tcnative_linux_x86_64_fedora, netty_tcnative]

[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- andtruststore files are resolved relatively

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]

[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

Based on http://docs.search-guard.com/latest/openssl, I’ve installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I’m using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version

openjdk version “1.8.0_161”

OpenJDK Runtime Environment (build 1.8.0_161-b14)

OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl

openssl-libs-1.0.2k-8.el7.x86_64

openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr

apr-1.4.8-3.el7_4.1.x86_64

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard

searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem

searchguard.ssl.transport.pemkey_password: KEY_PASS

searchguard.ssl.transport.pemcert_filepath: node.crt.pem

searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemkey_filepath: node.key.pem

searchguard.ssl.http.pemkey_password: KEY_PASS

searchguard.ssl.http.pemcert_filepath: node.crt.pem

searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.http.clientauth_mode: OPTIONAL

searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

Thanks!

Dan

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any


You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/62e28c04-944a-4bd4-9979-55d496adc0e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Sure, there’s a major update this week including also this correction.

···

On Wednesday, February 21, 2018 at 11:28:37 PM UTC+1, dchan@galileo.io wrote:

Thanks for the quick reply! Using 2.0.7 version of tcnative with Elasticsearch 6.2.1 works on my setup.

On Wednesday, February 21, 2018 at 4:09:49 PM UTC-5, Search Guard wrote:

you need the 2.0.7 version tcnative instead of 2.0.5, i updated the docs accordingly.

So pls try https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.7.Final-fedora-linux-x86_64.jar

See https://github.com/floragunncom/search-guard-docs/blob/6.x/_docs/tls_openssl.md for all download possibilities (dynamic and static)

Am 20.02.2018 um 17:49 schrieb dc...@galileo.io:

Additional information

{

“principal” : null,

“peer_certificates” : “0”,

“ssl_protocol” : “TLSv1.2”,

“ssl_cipher” : “TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256”,

“ssl_openssl_available” : false,

“ssl_openssl_version” : -1,

“ssl_openssl_version_string” : null,

“ssl_openssl_non_available_cause” : “java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]”,

“ssl_openssl_supports_key_manager_factory” : false,

“ssl_openssl_supports_hostname_validation” : false,

“ssl_provider_http” : “JDK”,

“ssl_provider_transport_server” : “JDK”,

“ssl_provider_transport_client” : “JDK”

}

On Tuesday, February 20, 2018 at 11:36:06 AM UTC-5, dc...@galileo.io wrote:

Hi,

I’m trying to get Elasticsearch to start with OpenSSL, but I’m receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing …

[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]

[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]

[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]

[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config

[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-inJDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64,netty_tcnative_linux_x86_64_fedora, netty_tcnative]

[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- andtruststore files are resolved relatively

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK

[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]

[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

Based on http://docs.search-guard.com/latest/openssl, I’ve installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I’m using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version

openjdk version “1.8.0_161”

OpenJDK Runtime Environment (build 1.8.0_161-b14)

OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl

openssl-libs-1.0.2k-8.el7.x86_64

openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr

apr-1.4.8-3.el7_4.1.x86_64

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard

searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem

searchguard.ssl.transport.pemkey_password: KEY_PASS

searchguard.ssl.transport.pemcert_filepath: node.crt.pem

searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemkey_filepath: node.key.pem

searchguard.ssl.http.pemkey_password: KEY_PASS

searchguard.ssl.http.pemcert_filepath: node.crt.pem

searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.http.clientauth_mode: OPTIONAL

searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

Thanks!

Dan

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any


You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/62e28c04-944a-4bd4-9979-55d496adc0e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Thank you for helping me out with the problem. In addition, will http://docs.search-guard.com/latest/openssl be updated as well?