HTTP/REST layer SSL

Search Guard
1

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:

We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it’s not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) … 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:

  • CN=zzz, OU=zz, O=zz, L=zz, C=zz

searchguard.ssl.http.enable_openssl_if_available: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.enabled_protocols:

  • TLSv1

  • SSLv3

searchguard.ssl.http.enforce_hostname_verification: false

searchguard.ssl.http.keystore_alias: alias

searchguard.ssl.http.keystore_filepath: keystore

searchguard.ssl.http.keystore_password: password

searchguard.ssl.http.resolve_hostname: false

#searchguard.ssl.http.truststore_alias: alias

#searchguard.ssl.http.truststore_filepath: truststore

#searchguard.ssl.http.truststore_password: password

searchguard.ssl.http.clientauth_mode: NONE

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_alias: alias

searchguard.ssl.transport.keystore_filepath: keystore

searchguard.ssl.transport.keystore_password: password

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.transport.truststore_alias: alias

searchguard.ssl.transport.truststore_filepath: keystore

searchguard.ssl.transport.truststore_password: password

security.manager.enabled: true

searchguard.ssl.transport.keystore_type: JKS

searchguard.ssl.transport.truststore_type: JKS

searchguard.ssl.http.keystore_type: JKS

searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013

And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

2

Thats nothing we can do on the "server side". We need to provide a server certificate (thats mandatory for SSL) and its up to the client (thats elasticsearch-spark, more specific: org.elasticsearch.hadoop.rest.RestClient) to validate it or not to validate it.
Its totally optional for the client to provide a client-certificate and your configuration is good for this point: searchguard.ssl.http.clientauth_mode: NONE

Maybe setting 'es.net.ssl.cert.allow.self.signed: true' will help you.

Note: If you want to use OpenSSL you have to set 'searchguard.ssl.http.enable_openssl_if_available: true'
Another note: You have configured SSLv3 to be allowed - This is a security risk, we recommend TLSv1.1 or TLSv1.2 only.

···

Am 29.06.2016 um 19:59 schrieb Jainin Shah <jainin@tresata.com>:

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:
We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it's not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) ... 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:
- CN=zzz, OU=zz, O=zz, L=zz, C=zz
searchguard.ssl.http.enable_openssl_if_available: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.enabled_protocols:
  - TLSv1
  - SSLv3
searchguard.ssl.http.enforce_hostname_verification: false
searchguard.ssl.http.keystore_alias: alias
searchguard.ssl.http.keystore_filepath: keystore
searchguard.ssl.http.keystore_password: password
searchguard.ssl.http.resolve_hostname: false
#searchguard.ssl.http.truststore_alias: alias
#searchguard.ssl.http.truststore_filepath: truststore
#searchguard.ssl.http.truststore_password: password
searchguard.ssl.http.clientauth_mode: NONE
searchguard.ssl.transport.enable_openssl_if_available: true
searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.keystore_alias: alias
searchguard.ssl.transport.keystore_filepath: keystore
searchguard.ssl.transport.keystore_password: password
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.truststore_alias: alias
searchguard.ssl.transport.truststore_filepath: keystore
searchguard.ssl.transport.truststore_password: password
security.manager.enabled: true
searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.http.keystore_type: JKS
searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/50fe847b-7d27-45fa-a7ee-fab69a954e6e%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

3

Thanks for the quick reply, if I enable openssl, it gives this error in the DEBUG mode for elasticsearch ( I followed this link https://github.com/floragunncom/search-guard-ssl-docs/blob/master/openssl.md, statically linked)

[2016-06-29 14:48:27,295][DEBUG][io.netty.util.internal.NativeLibraryLoader] Unable to load the library: netty-tcnative-linux-x86_64.

java.lang.UnsatisfiedLinkError: no netty-tcnative-linux-x86_64 in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)

at java.lang.Runtime.loadLibrary0(Runtime.java:849)

at java.lang.System.loadLibrary(System.java:1088)

at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:189)

at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:161)

at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:238)

at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:65)

at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin$1.run(SearchGuardSSLPlugin.java:70)

at java.security.AccessController.doPrivileged(Native Method)

at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuardSSLPlugin.java:66)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:472)

at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432)

at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:129)

at org.elasticsearch.node.Node.<init>(Node.java:158)

at org.elasticsearch.node.Node.<init>(Node.java:140)

at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

[2016-06-29 14:48:27,296][DEBUG][io.netty.util.internal.NativeLibraryLoader] Unable to load the library: netty-tcnative-linux-x86_64-fedora.

java.lang.UnsatisfiedLinkError: no netty-tcnative-linux-x86_64-fedora in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)

at java.lang.Runtime.loadLibrary0(Runtime.java:849)

at java.lang.System.loadLibrary(System.java:1088)

at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:189)

at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:161)

at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:238)

at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:65)

at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin$1.run(SearchGuardSSLPlugin.java:70)

at java.security.AccessController.doPrivileged(Native Method)

at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuardSSLPlugin.java:66)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:472)

at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432)

at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:129)

at org.elasticsearch.node.Node.<init>(Node.java:158)

at org.elasticsearch.node.Node.<init>(Node.java:140)

at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

[2016-06-29 14:48:27,333][DEBUG][io.netty.util.internal.NativeLibraryLoader] Unable to load the library: netty-tcnative.

java.lang.UnsatisfiedLinkError: /tmp/libnetty-tcnative4169558319741447025.so: /lib64/libc.so.6: version `GLIBC_2.14’ not found (required by /tmp/libnetty-tcnative4169558319741447025.so)

Using es.net.ssl.cert.allow.self.signed: true did not help, I get the same error. It seems the error is about elasticsearch complaining that it is not able to find the correct path for the certificate.

···

On Wed, Jun 29, 2016 at 2:44 PM, SG info@search-guard.com wrote:

Thats nothing we can do on the “server side”. We need to provide a server certificate (thats mandatory for SSL) and its up to the client (thats elasticsearch-spark, more specific: org.elasticsearch.hadoop.rest.RestClient) to validate it or not to validate it.

Its totally optional for the client to provide a client-certificate and your configuration is good for this point: searchguard.ssl.http.clientauth_mode: NONE

Maybe setting ‘es.net.ssl.cert.allow.self.signed: true’ will help you.

Note: If you want to use OpenSSL you have to set ‘searchguard.ssl.http.enable_openssl_if_available: true’

Another note: You have configured SSLv3 to be allowed - This is a security risk, we recommend TLSv1.1 or TLSv1.2 only.

Am 29.06.2016 um 19:59 schrieb Jainin Shah jainin@tresata.com:

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:

We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it’s not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) … 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:

  • CN=zzz, OU=zz, O=zz, L=zz, C=zz

searchguard.ssl.http.enable_openssl_if_available: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.enabled_protocols:

  • TLSv1
  • SSLv3

searchguard.ssl.http.enforce_hostname_verification: false

searchguard.ssl.http.keystore_alias: alias

searchguard.ssl.http.keystore_filepath: keystore

searchguard.ssl.http.keystore_password: password

searchguard.ssl.http.resolve_hostname: false

#searchguard.ssl.http.truststore_alias: alias

#searchguard.ssl.http.truststore_filepath: truststore

#searchguard.ssl.http.truststore_password: password

searchguard.ssl.http.clientauth_mode: NONE

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_alias: alias

searchguard.ssl.transport.keystore_filepath: keystore

searchguard.ssl.transport.keystore_password: password

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.transport.truststore_alias: alias

searchguard.ssl.transport.truststore_filepath: keystore

searchguard.ssl.transport.truststore_password: password

security.manager.enabled: true

searchguard.ssl.transport.keystore_type: JKS

searchguard.ssl.transport.truststore_type: JKS

searchguard.ssl.http.keystore_type: JKS

searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013

And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

–

You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/50fe847b-7d27-45fa-a7ee-fab69a954e6e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to a topic in the Google Groups “Search Guard” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/DrTaAnIkRmo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/F9CA4694-171D-42C4-AD60-8D825FCAA460%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.

4

Also, in the elasticsearch log, we get this error while indexing/loading data.

[2016-06-29 16:00:57,751][ERROR][com.floragunn.searchguard.http.SearchGuardHttpServerTransport] [Hussar] SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1639)

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1607)

at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1776)

at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1068)

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:890)

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:764)

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)

at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1218)

at org.jboss.netty.handler.ssl.SslHandler.decode(SslHandler.java:852)

at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)

at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)

at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)

at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)

at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)

at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)

at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)

at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)

at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)

at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

It seems the server is expecting a certificate to be passed.

···

On Wed, Jun 29, 2016 at 2:44 PM, SG info@search-guard.com wrote:

Thats nothing we can do on the “server side”. We need to provide a server certificate (thats mandatory for SSL) and its up to the client (thats elasticsearch-spark, more specific: org.elasticsearch.hadoop.rest.RestClient) to validate it or not to validate it.

Its totally optional for the client to provide a client-certificate and your configuration is good for this point: searchguard.ssl.http.clientauth_mode: NONE

Maybe setting ‘es.net.ssl.cert.allow.self.signed: true’ will help you.

Note: If you want to use OpenSSL you have to set ‘searchguard.ssl.http.enable_openssl_if_available: true’

Another note: You have configured SSLv3 to be allowed - This is a security risk, we recommend TLSv1.1 or TLSv1.2 only.

Am 29.06.2016 um 19:59 schrieb Jainin Shah jainin@tresata.com:

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:

We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it’s not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) … 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:

  • CN=zzz, OU=zz, O=zz, L=zz, C=zz

searchguard.ssl.http.enable_openssl_if_available: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.enabled_protocols:

  • TLSv1
  • SSLv3

searchguard.ssl.http.enforce_hostname_verification: false

searchguard.ssl.http.keystore_alias: alias

searchguard.ssl.http.keystore_filepath: keystore

searchguard.ssl.http.keystore_password: password

searchguard.ssl.http.resolve_hostname: false

#searchguard.ssl.http.truststore_alias: alias

#searchguard.ssl.http.truststore_filepath: truststore

#searchguard.ssl.http.truststore_password: password

searchguard.ssl.http.clientauth_mode: NONE

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_alias: alias

searchguard.ssl.transport.keystore_filepath: keystore

searchguard.ssl.transport.keystore_password: password

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.transport.truststore_alias: alias

searchguard.ssl.transport.truststore_filepath: keystore

searchguard.ssl.transport.truststore_password: password

security.manager.enabled: true

searchguard.ssl.transport.keystore_type: JKS

searchguard.ssl.transport.truststore_type: JKS

searchguard.ssl.http.keystore_type: JKS

searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013

And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

–

You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/50fe847b-7d27-45fa-a7ee-fab69a954e6e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to a topic in the Google Groups “Search Guard” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/DrTaAnIkRmo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/F9CA4694-171D-42C4-AD60-8D825FCAA460%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.

5

We changed our client and it works now.

Thanks

···

On Wed, Jun 29, 2016 at 2:44 PM, SG info@search-guard.com wrote:

Thats nothing we can do on the “server side”. We need to provide a server certificate (thats mandatory for SSL) and its up to the client (thats elasticsearch-spark, more specific: org.elasticsearch.hadoop.rest.RestClient) to validate it or not to validate it.

Its totally optional for the client to provide a client-certificate and your configuration is good for this point: searchguard.ssl.http.clientauth_mode: NONE

Maybe setting ‘es.net.ssl.cert.allow.self.signed: true’ will help you.

Note: If you want to use OpenSSL you have to set ‘searchguard.ssl.http.enable_openssl_if_available: true’

Another note: You have configured SSLv3 to be allowed - This is a security risk, we recommend TLSv1.1 or TLSv1.2 only.

Am 29.06.2016 um 19:59 schrieb Jainin Shah jainin@tresata.com:

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:

We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it’s not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) … 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:

  • CN=zzz, OU=zz, O=zz, L=zz, C=zz

searchguard.ssl.http.enable_openssl_if_available: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.enabled_protocols:

  • TLSv1
  • SSLv3

searchguard.ssl.http.enforce_hostname_verification: false

searchguard.ssl.http.keystore_alias: alias

searchguard.ssl.http.keystore_filepath: keystore

searchguard.ssl.http.keystore_password: password

searchguard.ssl.http.resolve_hostname: false

#searchguard.ssl.http.truststore_alias: alias

#searchguard.ssl.http.truststore_filepath: truststore

#searchguard.ssl.http.truststore_password: password

searchguard.ssl.http.clientauth_mode: NONE

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_alias: alias

searchguard.ssl.transport.keystore_filepath: keystore

searchguard.ssl.transport.keystore_password: password

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.transport.truststore_alias: alias

searchguard.ssl.transport.truststore_filepath: keystore

searchguard.ssl.transport.truststore_password: password

security.manager.enabled: true

searchguard.ssl.transport.keystore_type: JKS

searchguard.ssl.transport.truststore_type: JKS

searchguard.ssl.http.keystore_type: JKS

searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013

And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

–

You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/50fe847b-7d27-45fa-a7ee-fab69a954e6e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to a topic in the Google Groups “Search Guard” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/DrTaAnIkRmo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/F9CA4694-171D-42C4-AD60-8D825FCAA460%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.

6

Hi,

Can you please mention the changes you made to make this work.

Am facing similar issue writing to Elasticsearch via Hive.

Using Elasticsearch v2.3.5

Searchguard 2 v 2.3.5.10

Searchguard SSL v2.3.5.20

Hive v1.1

Thanks.

···

On Friday, July 1, 2016 at 9:32:31 AM UTC-4, Jainin Shah wrote:

We changed our client and it works now.

Thanks

On Wed, Jun 29, 2016 at 2:44 PM, SG in...@search-guard.com wrote:

Thats nothing we can do on the “server side”. We need to provide a server certificate (thats mandatory for SSL) and its up to the client (thats elasticsearch-spark, more specific: org.elasticsearch.hadoop.rest.RestClient) to validate it or not to validate it.

Its totally optional for the client to provide a client-certificate and your configuration is good for this point: searchguard.ssl.http.clientauth_mode: NONE

Maybe setting ‘es.net.ssl.cert.allow.self.signed: true’ will help you.

Note: If you want to use OpenSSL you have to set ‘searchguard.ssl.http.enable_openssl_if_available: true’

Another note: You have configured SSLv3 to be allowed - This is a security risk, we recommend TLSv1.1 or TLSv1.2 only.

Am 29.06.2016 um 19:59 schrieb Jainin Shah jai...@tresata.com:

Hi,

Have a question regarding the HTTP/REST layer SSL with es

Background:

We are using the latest stable release of search guard for securing elasticsearch. Once es is secure, we use elasticsearch-spark to load data into elasticsearch from HDFS. When we have not enabled HTTPS for rest layer, the data gets indexed just fine. But when we try to index/load data when HTTPS is enabled for REST layer, it gives below error. This occurs because it lack a shared certificate root, if all boxes had self-signed certificates with a common root it would work I think. But it’s not practical for us to do that in dev/staging environment.

org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:418) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:422) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:122) at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:568) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:178) … 38 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.jav

Question

Is there anyway, we can turn this off (similar to what we do with curl -k command)

This is our elasticsearch.yml file

searchguard.authcz.admin_dn:

  • CN=zzz, OU=zz, O=zz, L=zz, C=zz

searchguard.ssl.http.enable_openssl_if_available: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.enabled_protocols:

  • TLSv1
  • SSLv3

searchguard.ssl.http.enforce_hostname_verification: false

searchguard.ssl.http.keystore_alias: alias

searchguard.ssl.http.keystore_filepath: keystore

searchguard.ssl.http.keystore_password: password

searchguard.ssl.http.resolve_hostname: false

#searchguard.ssl.http.truststore_alias: alias

#searchguard.ssl.http.truststore_filepath: truststore

#searchguard.ssl.http.truststore_password: password

searchguard.ssl.http.clientauth_mode: NONE

searchguard.ssl.transport.enable_openssl_if_available: true

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_alias: alias

searchguard.ssl.transport.keystore_filepath: keystore

searchguard.ssl.transport.keystore_password: password

searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.transport.truststore_alias: alias

searchguard.ssl.transport.truststore_filepath: keystore

searchguard.ssl.transport.truststore_password: password

security.manager.enabled: true

searchguard.ssl.transport.keystore_type: JKS

searchguard.ssl.transport.truststore_type: JKS

searchguard.ssl.http.keystore_type: JKS

searchguard.ssl.http.truststore_type: JKS

We use this OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013

And Java 8 right now is not an option for us as we have othe programs that depend on Java 7.

Thanks for your help

Jainin Shah

–

You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/50fe847b-7d27-45fa-a7ee-fab69a954e6e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to a topic in the Google Groups “Search Guard” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/DrTaAnIkRmo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/F9CA4694-171D-42C4-AD60-8D825FCAA460%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.