search-guard-ssl javax.net.ssl.SSLHandshakeException on a 2 node cluster

Hi there,

I’m trying to install the search-guard-ssl plugin, v. 6.2.22 for an ES 6.2 - 2 node cluster(10.1.1.1 and 10.2.2.2) . I installed the plugin and then ran the example.sh(default parameters) script which generated a whole bunch of certificates. I copied over node-0-signed.pem, node-0.key.pem and root-ca.pem in to the config directory of the node I generated them on(10.1.1.1), as well as “secure copy” them to the other node as well.

Here’s the elasticsearch.yml settings(same on both):

searchguard.ssl.transport.enabled: true

searchguard.ssl.transport.pemcert_filepath: node-0-signed.pem

searchguard.ssl.transport.pemkey_filepath: node-0.key.pem

searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemcert_filepath: node-0-signed.pem

searchguard.ssl.http.pemkey_filepath: node-0.key.pem

searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

Upon starting the node where I copied over the the .pem files I am getting the following errors:

[2018-04-12T23:39:15,276][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [JVl5tOt] SSL Problem General SSLEngine problem

javax.net.ssl.SSLHandshakeException: General SSLEngine problem

at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431) ~[?:1.8.0_112]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) ~[?:1.8.0_112]

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813) ~[?:1.8.0_112]

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:1.8.0_112]

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_112]

at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:281) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1215) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1127) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1162) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.16.Final.jar:4.1.16.Final]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_112]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) ~[?:1.8.0_112]

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) ~[?:1.8.0_112]

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[?:1.8.0_112]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369) ~[?:1.8.0_112]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1364) ~[?:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1272) ~[?:?]

… 19 more

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) ~[?:1.8.0_112]

at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) ~[?:1.8.0_112]

at sun.security.validator.Validator.validate(Validator.java:260) ~[?:1.8.0_112]

at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[?:1.8.0_112]

at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281) ~[?:1.8.0_112]

at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136) ~[?:1.8.0_112]

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496) ~[?:1.8.0_112]

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[?:1.8.0_112]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369) ~[?:1.8.0_112]

Definitely feels like it’s not able to verify the signature or possibly IP’s on the cerificate - Did I misunderstand how to configure searchguard ? How can I get this 2 node cluster working ?

I just installed searchguard-ssl and not searchguard, since just needed TLS between nodes.

try

searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.pemcert_filepath: node-0.crt.pem
searchguard.ssl.transport.pemkey_filepath: node-0.key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: node-0.crt.pem
searchguard.ssl.http.pemkey_filepath: node-0.key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

···

Am 13.04.2018 um 09:13 schrieb Neeraj Prashar <neeraj.prashar@gmail.com>:

Hi there,

I'm trying to install the search-guard-ssl plugin, v. 6.2.22 for an ES 6.2 - 2 node cluster(10.1.1.1 and 10.2.2.2) . I installed the plugin and then ran the example.sh(default parameters) script which generated a whole bunch of certificates. I copied over node-0-signed.pem, node-0.key.pem and root-ca.pem in to the config directory of the node I generated them on(10.1.1.1), as well as "secure copy" them to the other node as well.

Here's the elasticsearch.yml settings(same on both):

searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.pemcert_filepath: node-0-signed.pem
searchguard.ssl.transport.pemkey_filepath: node-0.key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: node-0-signed.pem
searchguard.ssl.http.pemkey_filepath: node-0.key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

Upon starting the node where I copied over the the .pem files I am getting the following errors:

[2018-04-12T23:39:15,276][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [JVl5tOt] SSL Problem General SSLEngine problem
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
        at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431) ~[?:1.8.0_112]
        at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) ~[?:1.8.0_112]
        at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813) ~[?:1.8.0_112]
        at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:1.8.0_112]
        at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_112]
        at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:281) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1215) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1127) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1162) ~[netty-handler-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.16.Final.jar:4.1.16.Final]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_112]
        at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) ~[?:1.8.0_112]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) ~[?:1.8.0_112]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[?:1.8.0_112]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369) ~[?:1.8.0_112]
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1364) ~[?:?]
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1272) ~[?:?]
        ... 19 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) ~[?:1.8.0_112]
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) ~[?:1.8.0_112]
        at sun.security.validator.Validator.validate(Validator.java:260) ~[?:1.8.0_112]
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[?:1.8.0_112]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281) ~[?:1.8.0_112]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136) ~[?:1.8.0_112]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496) ~[?:1.8.0_112]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[?:1.8.0_112]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
        at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369) ~[?:1.8.0_112]

Definitely feels like it's not able to verify the signature or possibly IP's on the cerificate - Did I misunderstand how to configure searchguard ? How can I get this 2 node cluster working ?

I just installed searchguard-ssl and not searchguard, since just needed TLS between nodes.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a13ee60a-2e49-4f33-add0-8eb93fc9b6d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.