Enabling OpenSSL with Elasticsearch 6.2.1

Dan_Chan · March 29, 2019, 1:10pm

Hi,

I’m trying to get Elasticsearch to start with OpenSSL, but I’m receiving an error when starting Elasticsearch.

[2018-02-20T16:29:59,742][INFO ][o.e.n.Node ] initializing …
[2018-02-20T16:29:59,847][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [6.9gb], net total_space [7.7gb], types [ext4]
[2018-02-20T16:29:59,848][INFO ][o.e.e.NodeEnvironment ] [9KwCLS3] heap size [1.7gb], compressed ordinary object pointers [true]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] node name [9KwCLS3] derived from node ID [9KwCLS3GQC67P4liY3drtw]; set [node.name] to override
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] version[6.2.1], pid[112], build[7299dc3/2018-02-07T19:34:26.990113Z], OS[Linux/4.9.76-3.78.amzn1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
[2018-02-20T16:29:59,860][INFO ][o.e.n.Node ] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.gHCP8Nfa, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1778m, -Xms1778m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
[2018-02-20T16:30:00,952][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /usr/share/elasticsearch/config
[2018-02-20T16:30:01,058][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]
[2018-02-20T16:30:01,348][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /usr/share/elasticsearch/config/, from there the key- and truststore files are resolved relatively
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK
[2018-02-20T16:30:01,527][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
[2018-02-20T16:30:01,528][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

``

Based on http://docs.search-guard.com/latest/openssl, I’ve installed openssl, apr, and the appropriate jar file into plugins/search-guard-6. https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.0.2-dynamic-2.0.5.Final-fedora-linux-x86_64.jar

Q: What should I change to get Open SSL to be made available to Elasticsearch?

More information below, for context to my ES environment.

I’m using Elasticsearch 6.2.1 and Search Guard 6.2.1-21.0. This is the open source version.

[elasticsearch@15e41b0c864c ~]$ java -version

openjdk version “1.8.0_161”

OpenJDK Runtime Environment (build 1.8.0_161-b14)

OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

[elasticsearch@15e41b0c864c ~]$ cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep openssl

openssl-libs-1.0.2k-8.el7.x86_64

openssl-1.0.2k-8.el7.x86_64

[elasticsearch@15e41b0c864c ~]$ rpm -qa | grep apr

apr-1.4.8-3.el7_4.1.x86_64

``

I have Elasticsearch plugins installed for discovery-ec2 and repository-s3.

My elasticsearch.yml configuration for Search Guard

searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemkey_filepath: node.key.pem

searchguard.ssl.transport.pemkey_password: KEY_PASS

searchguard.ssl.transport.pemcert_filepath: node.crt.pem

searchguard.ssl.transport.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemkey_filepath: node.key.pem

searchguard.ssl.http.pemkey_password: KEY_PASS

searchguard.ssl.http.pemcert_filepath: node.crt.pem

searchguard.ssl.http.pemtrustedcas_filepath: signing-ca.pem

searchguard.ssl.http.clientauth_mode: OPTIONAL

searchguard.ssl.http.enable_openssl_if_available: true

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

``

Thanks!

Dan

···

–

When asking questions, please provide the following information:

Search Guard and Elasticsearch version
Installed and used enterprise modules, if any
JVM version and operating system version
Search Guard configuration files
Elasticsearch log messages on debug level
Other installed Elasticsearch or Kibana plugins, if any

Dan_Chan · March 29, 2019, 1:13pm

Additional information. Interesting how ssl_openssl_available is set to false, even though I have OpenSSL and apr packages installed, as far as I can tell.

{
  "principal" : null,
  "peer_certificates" : "0",
  "ssl_protocol" : "TLSv1.2",
  "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "ssl_openssl_available" : false,
  "ssl_openssl_version" : -1,
  "ssl_openssl_version_string" : null,
  "ssl_openssl_non_available_cause" : "java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative]",
  "ssl_openssl_supports_key_manager_factory" : false,
  "ssl_openssl_supports_hostname_validation" : false,
  "ssl_provider_http" : "JDK",
  "ssl_provider_transport_server" : "JDK",
  "ssl_provider_transport_client" : "JDK"
}

``

Noting that with the same set up, Elasticsearch 6.1.3 seems to find the OpenSSL library.

{

“principal” : null,

“peer_certificates” : “0”,

“ssl_protocol” : “TLSv1.2”,

“ssl_cipher” : “TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256”,

“ssl_openssl_available” : true,

“ssl_openssl_version” : 268443839,

“ssl_openssl_version_string” : “OpenSSL 1.0.2k-fips 26 Jan 2017”,

“ssl_openssl_non_available_cause” : “”,

“ssl_openssl_supports_key_manager_factory” : true,

“ssl_openssl_supports_hostname_validation” : true,

“ssl_provider_http” : “OPENSSL”,

“ssl_provider_transport_server” : “OPENSSL”,

“ssl_provider_transport_client” : “OPENSSL”

}

``

···

On Tuesday, February 20, 2018 at 11:36:06 AM UTC-5, dc...@galileo.io wrote:

Hi,