Unknown certificate error after installing new transport certificates

Hi,

I’ve tried to replace demo certificates on the cluster. I used offline tls tool to generate new node and ca certificates. Verified all certificates are valid.

After making suggested changes in the elasticsearch.yml, started one node while all other nodes were down, I see the SSL certificate error:

[2018-11-28T04:24:45,396][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [es-1] SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]

at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800

Environment:

Ubuntu 16.04.3 LTS, Elasticsearch 6.1, SG plugin 6.1, Oracle JDK 1.8

Tried: generation pem certificates with intermediate CA and without it, changing the data.path (meaning to start new node).

Verified generated certificates:

openssl verify -verbose -CAfile root-ca.pem es-1.pem

es-1.pem: OK

Appreciate any clue about this issue. Attached the configuration file.

elasticsearch.yml (3.05 KB)

At first glance, the config seems valid. Can you please:

  1. Add the complete ES log file when the node comes up

  2. Use the diagnose mode of the TLS tool to verify the setup

But as far as I can see from the logfile, I suspect there is another instance of Elasticsearch running somwhere. The issue you are seeing is happening on the transport layer of ES. This indicates that another node tries to connect to this node, and the certificates do not match. Or that sgadmin is executed with the old root CA.

···

On Wednesday, November 28, 2018 at 5:51:19 AM UTC+1, Oleks Yaremenko wrote:

Hi,

I’ve tried to replace demo certificates on the cluster. I used offline tls tool to generate new node and ca certificates. Verified all certificates are valid.

After making suggested changes in the elasticsearch.yml, started one node while all other nodes were down, I see the SSL certificate error:

[2018-11-28T04:24:45,396][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [es-1] SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]

at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800

Environment:

Ubuntu 16.04.3 LTS, Elasticsearch 6.1, SG plugin 6.1, Oracle JDK 1.8

Tried: generation pem certificates with intermediate CA and without it, changing the data.path (meaning to start new node).

Verified generated certificates:

openssl verify -verbose -CAfile root-ca.pem es-1.pem

es-1.pem: OK

Appreciate any clue about this issue. Attached the configuration file.

Jochen,
You were right. Some process tried to connect to the nodes on the transport port. After I changed the port, I don’t see such errors. Cluster works with new certificates as expected.

Thank you.

···

On Wednesday, November 28, 2018 at 3:33:48 AM UTC-7, Jochen Kressin wrote:

At first glance, the config seems valid. Can you please:

  1. Add the complete ES log file when the node comes up
  1. Use the diagnose mode of the TLS tool to verify the setup

https://docs.search-guard.com/latest/offline-tls-tool#validating-certificates

But as far as I can see from the logfile, I suspect there is another instance of Elasticsearch running somwhere. The issue you are seeing is happening on the transport layer of ES. This indicates that another node tries to connect to this node, and the certificates do not match. Or that sgadmin is executed with the old root CA.

On Wednesday, November 28, 2018 at 5:51:19 AM UTC+1, Oleks Yaremenko wrote:

Hi,

I’ve tried to replace demo certificates on the cluster. I used offline tls tool to generate new node and ca certificates. Verified all certificates are valid.

After making suggested changes in the elasticsearch.yml, started one node while all other nodes were down, I see the SSL certificate error:

[2018-11-28T04:24:45,396][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [es-1] SSL Problem Received fatal alert: certificate_unknown

javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]

at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]

at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800

Environment:

Ubuntu 16.04.3 LTS, Elasticsearch 6.1, SG plugin 6.1, Oracle JDK 1.8

Tried: generation pem certificates with intermediate CA and without it, changing the data.path (meaning to start new node).

Verified generated certificates:

openssl verify -verbose -CAfile root-ca.pem es-1.pem

es-1.pem: OK

Appreciate any clue about this issue. Attached the configuration file.