Cannot add node - TLS error

[wyllys]

Elasticsearch 6.7
SearchGuard: 6.7.1-24.3, enterprise licensed

Im having no luck adding a new node to my cluster. No matter what combination of nodes_dn I use, the server reports the same error every time the new node tries to do discovery:

 [2019-05-07T11:20:42,064][ERROR][c.f.s.t.SearchGuardRequestHandler] [isZPOIE] ElasticsearchException[Illegal parameter in http or transport request found.
This means that one node is trying to connect to another with a non-node certificate (no OID or searchguard.nodes_dn incorrect configured) or that someone is spoofing requests. Check your TLS certificate setup as described here: See http://docs.search-guard.com/latest/troubleshooting-tls]

I would like to just get this working with a wildcard as the nodes_dn but even that does not work.

Master node (works fine by itself) - ktelastic.domain.com

 searchguard.ssl.transport.pemcert_filepath: ssl/ktelastic.crt
 searchguard.ssl.transport.pemkey_filepath: ssl/ktelastic.key
 searchguard.ssl.transport.pemtrustedcas_filepath: ssl/domain.com.chained.crt
 searchguard.ssl.transport.enforce_hostname_verification: false
 searchguard.ssl.transport.resolve_hostname: false
 searchguard.nodes_dn:
   - 'CN=*.domain.com'

New node (fails to join):

discovery.zen.ping.unicast.hosts:
  - ktelastic.domain.com
searchguard.ssl.transport.pemcert_filepath: ssl/ktelastic2.crt
searchguard.ssl.transport.pemkey_filepath: ssl/ktelastic2.key
searchguard.ssl.transport.pemtrustedcas_filepath: ssl/domain.com.chained.crt
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false

The node certificate ktelastic2.crt has Issuer that looks like:
Subject: CN=ktelastic2.domain.com

Any ideas?

[jkressin]

For the nodes_dn entry, you need to configure the distinguished name of the node certificate, not the issuer.

So can you please check the full DN of your ktelastic2.crt certificate first? You can do that with OpenSSL or our TLS tool for example:

TLS troubleshooting

You should see (at least) two entries like:

Owner: CN=node-0.example.com, OU=SSL, O=Test, L=Test, C=DE
Issuer: CN=Example Com Inc. Signing CA, OU=Example Com Inc. Signing CA, O=Example Com Inc., DC=example, DC=com

You need to use the Owner entry for configuring the nodes_dn.

[wyllys]

Using openssl to view the cert, I selected the “Subject” DN and put it in the nodes_dn list - didnt work.
I also put in wildcards in the DN - didnt work.

Ultimately, I ended up making new node certificates that included the RID 1.2.3.4.5.5 in the SAN field and that seems to work fine. I still dont know why the nodes_dn list wasnt working, especially with a wildcard entry.