Got certificate_unknown after configured searchguard

Hi,

I configured Search Guard on my cluster. And I got this error. I have tested with offline TLS tool. The certificate is valid.

U8015996-MAC-BOOK:searchguard worapojc$ tools/sgtlsdiag.sh -ca …/files/searchguard/tls/root-ca.pem -crt …/files/searchguard/tls/10.49.116.129.pem

WARNING: JAVA_HOME not set, will use /usr/bin/java

···

========================================================================

…/files/searchguard/tls/10.49.116.129.pem


Certificate 1


        SHA1 FPR: e2dcccb4117928ce56f6af6614f194d7e36ba09b

         MD5 FPR: 13f77accfbfcd7f66d0a82bb56ea3560

Subject DN [RFC2253]: CN=10.49.116.129,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

   Serial Number: 1552408688812

Issuer DN [RFC2253]: CN=Signing Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

      Not Before: Tue Mar 12 23:38:10 ICT 2019

       Not After: Thu Mar 11 23:38:10 ICT 2021

       Key Usage: digitalSignature nonRepudiation keyEncipherment

Signature Algorithm: SHA256WITHRSA

         Version: 3

Extended Key Usage: id_kp_serverAuth id_kp_clientAuth

Basic Constraints: -1

            SAN:

              iPAddress: 10.49.116.129

Certificate 2


        SHA1 FPR: c861bdd81d59e02751f77bf1edfe0bbf72d226f7

         MD5 FPR: 86ea5be9723c8ffe00d560eac7d66bcb

Subject DN [RFC2253]: CN=Signing Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

   Serial Number: 2

Issuer DN [RFC2253]: CN=Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

      Not Before: Tue Mar 12 23:38:10 ICT 2019

       Not After: Thu Mar 11 23:38:10 ICT 2021

       Key Usage: digitalSignature keyCertSign cRLSign

Signature Algorithm: SHA256WITHRSA

         Version: 3

Extended Key Usage: null

Basic Constraints: 0

            SAN: (none)

Trust anchor:

C=US,DC=Service Tools,DC=Service Excellence,O=Refinitiv Company, Inc.,OU=Technology Development,CN=Compass Monitoring Events

``

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version

24.1 and 6.6.0

  • Installed and used enterprise modules, if any

No

  • JVM version and operating system version

1.8

  • Search Guard configuration files

  • Elasticsearch log messages on debug level

  • Other installed Elasticsearch or Kibana plugins, if any

[2019-03-12T17:33:23,363][INFO ][o.e.n.Node ] [10.49.113.25-master-node-0]initialized

[2019-03-12T17:33:23,363][INFO ][o.e.n.Node ] [10.49.113.25-master-node-0]starting …

[2019-03-12T17:33:23,774][INFO ][o.e.t.TransportService ] [10.49.113.25-master-node-0]publish_address {10.49.113.25:9300}, bound_addresses {127.0.0.1:9300}, {10.49.113.25:9300}

[2019-03-12T17:33:24,136][WARN ][i.n.c.AbstractChannelHandlerContext] [10.49.113.25-master-node-0]An exception ‘java.lang.NullPointerException’ [enable DEBUG level for full stacktrace] was thrown by a user handler’s exceptionCaught() method while handling the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]

    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

Caused by: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]

    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]

    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    ... 15 more

[2019-03-12T17:33:24,143][WARN ][i.n.c.AbstractChannelHandlerContext] [10.49.113.25-master-node-0]An exception ‘java.lang.NullPointerException’ [enable DEBUG level for full stacktrace] was thrown by a user handler’s exceptionCaught() method while handling the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:405) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:372) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:355) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1054) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:826) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-common-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]

    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

Caused by: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]

    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]

    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    ... 17 more

[2019-03-12T17:33:24,151][INFO ][o.e.b.BootstrapChecks ] [10.49.113.25-master-node-0]bound or publishing to a non-loopback address, enforcing bootstrap checks

[2019-03-12T17:33:24,163][INFO ][c.f.s.c.IndexBaseConfigurationRepository] [10.49.113.25-master-node-0]Check if searchguard index exists …

[2019-03-12T17:33:24,169][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [10.49.113.25-master-node-0]no known master node, scheduling a retry

[2019-03-12T17:33:24,335][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [10.49.113.25-master-node-0]SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]

    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]

    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]

    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]

    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]

    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

``

Can you please post your elasticsearch.yml configuration?

···

On Tuesday, March 12, 2019 at 6:39:24 PM UTC+1, Worapoj Chokeanankun wrote:

Hi,

I configured Search Guard on my cluster. And I got this error. I have tested with offline TLS tool. The certificate is valid.

U8015996-MAC-BOOK:searchguard worapojc$ tools/sgtlsdiag.sh -ca …/files/searchguard/tls/root-ca.pem -crt …/files/searchguard/tls/10.49.116.129.pem

WARNING: JAVA_HOME not set, will use /usr/bin/java

========================================================================

…/files/searchguard/tls/10.49.116.129.pem


Certificate 1


        SHA1 FPR: e2dcccb4117928ce56f6af6614f194d7e36ba09b
         MD5 FPR: 13f77accfbfcd7f66d0a82bb56ea3560

Subject DN [RFC2253]: CN=10.49.116.129,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

   Serial Number: 1552408688812

Issuer DN [RFC2253]: CN=Signing Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

      Not Before: Tue Mar 12 23:38:10 ICT 2019
       Not After: Thu Mar 11 23:38:10 ICT 2021
       Key Usage: digitalSignature nonRepudiation keyEncipherment

Signature Algorithm: SHA256WITHRSA

         Version: 3

Extended Key Usage: id_kp_serverAuth id_kp_clientAuth

Basic Constraints: -1

            SAN:
              iPAddress: 10.49.116.129

Certificate 2


        SHA1 FPR: c861bdd81d59e02751f77bf1edfe0bbf72d226f7
         MD5 FPR: 86ea5be9723c8ffe00d560eac7d66bcb

Subject DN [RFC2253]: CN=Signing Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

   Serial Number: 2

Issuer DN [RFC2253]: CN=Compass Monitoring Events,OU=Technology Development,O=Refinitiv Company, Inc.,DC=Service Excellence,DC=Service Tools,C=US

      Not Before: Tue Mar 12 23:38:10 ICT 2019
       Not After: Thu Mar 11 23:38:10 ICT 2021
       Key Usage: digitalSignature keyCertSign cRLSign

Signature Algorithm: SHA256WITHRSA

         Version: 3

Extended Key Usage: null

Basic Constraints: 0

            SAN: (none)

Trust anchor:

C=US,DC=Service Tools,DC=Service Excellence,O=Refinitiv Company, Inc.,OU=Technology Development,CN=Compass Monitoring Events

``

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version

24.1 and 6.6.0

  • Installed and used enterprise modules, if any

No

  • JVM version and operating system version

1.8

  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

[2019-03-12T17:33:23,363][INFO ][o.e.n.Node ] [10.49.113.25-master-node-0]initialized

[2019-03-12T17:33:23,363][INFO ][o.e.n.Node ] [10.49.113.25-master-node-0]starting …

[2019-03-12T17:33:23,774][INFO ][o.e.t.TransportService ] [10.49.113.25-master-node-0]publish_address {10.49.113.25:9300}, bound_addresses {127.0.0.1:9300}, {10.49.113.25:9300}

[2019-03-12T17:33:24,136][WARN ][i.n.c.AbstractChannelHandlerContext] [10.49.113.25-master-node-0]An exception ‘java.lang.NullPointerException’ [enable DEBUG level for full stacktrace] was thrown by a user handler’s exceptionCaught() method while handling the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

Caused by: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]
    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    ... 15 more

[2019-03-12T17:33:24,143][WARN ][i.n.c.AbstractChannelHandlerContext] [10.49.113.25-master-node-0]An exception ‘java.lang.NullPointerException’ [enable DEBUG level for full stacktrace] was thrown by a user handler’s exceptionCaught() method while handling the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:405) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:372) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:355) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1054) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:826) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-common-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

Caused by: javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]
    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    ... 17 more

[2019-03-12T17:33:24,151][INFO ][o.e.b.BootstrapChecks ] [10.49.113.25-master-node-0]bound or publishing to a non-loopback address, enforcing bootstrap checks

[2019-03-12T17:33:24,163][INFO ][c.f.s.c.IndexBaseConfigurationRepository] [10.49.113.25-master-node-0]Check if searchguard index exists …

[2019-03-12T17:33:24,169][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [10.49.113.25-master-node-0]no known master node, scheduling a retry

[2019-03-12T17:33:24,335][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [10.49.113.25-master-node-0]SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[?:?]
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[?:?]
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_191]
    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

``

I attached the the log and configuration file.

elasticsearch.yml (1.85 KB)

cluster-2019-03-13-9.log.gz (2.06 MB)

How did you create the certificates?

···

On Wednesday, 13 March 2019 05:06:29 UTC+1, Worapoj Chokeanankun wrote:

I attached the the log and configuration file.

Thank you, I have restarted all nodes in my cluster. It works now.