Certificates used with xpack are not reusable for searchguard

Search Guard
1

Hey guys.

I was using elasticsearch and kibana using xpack before. There I generated the certificates and root ca using the certutil tool, which elastichsearch provides. i have created certificates in .crt and .key suffix

Now I am trying to setup searchguard. It looks like I cannot simply reuse the existing certificates am I right? In the logs I have following errors:

[2020-06-03T12:49:49,888][ERROR][c.f.s.s.DefaultSearchGuardKeyStore] [elastic01] Your keystore or PEM does not contain a key. If you specified a key password, try removing it. If you did not specify a key password, perhaps you need to if the key is in fact password-protected. Maybe you just confused keys and certificates.

Caused by: org.elasticsearch.ElasticsearchSecurityException: Error while initializing transport SSL layer from PEM: java.lang.IllegalArgumentException: File does not contain valid private key: /home/usesr/bin/elasticsearch/config/certs/elastic01/elastic01.key

Caused by: java.security.KeyException: could not find a PKCS #8 private key in input stream

Do I need to recreate all the certificates for my nodes or can I just format them to anything so searchguard can read them?

I tried to convert they private key to pkcs8 format but stll receive following error that the does not contain a valid private key

Edit:

This is my elastic config.

searchguard.ssl.transport.pemcert_filepath: certs/elastic01/elastic01.crt
searchguard.ssl.transport.pemkey_filepath: certs/elastic01/elastic01.key
searchguard.ssl.transport.pemkey_password: changeme
searchguard.ssl.transport.pemtrustedcas_filepath: certs/ca.crt
searchguard.ssl.transport.enforce_hostname_verification: true

searchguard.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: certs/elastic01/elastic01.crt
searchguard.ssl.http.pemkey_filepath: certs/elastic01/elastic01.key
searchguard.ssl.http.pemkey_password: changeme
searchguard.ssl.http.pemtrustedcas_filepath: certs/ca.crt
2

Hi @Kosmonafft

  1. You have the TLS hostname verification enabled. Make sure all the nodes have hostnames specified in their certificates. Or disable the hostname verification and resolution:
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false

Or create new certificates using the SearchGuard TLS tool Offline TLS Tool | Security for Elasticsearch | Search Guard

  1. You also need to configure the nodes certificates to let Search Guard properly identify the inter-node requests, for example

elasticsearch.yml

node.name: node1.example.com
searchguard.nodes_dn:
- CN=node1.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node2.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node3.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
3

Also, make sure all the nodes certificates have the following extended usages: TLS Web Server Authentication, TLS Web Client Authentication. You can verify it like this

$ openssl x509 -text -in search-guard-tlstool/out/node1.pem | grep -i extended -A 5
            X509v3 Extended Key Usage: critical
                TLS Web Server Authentication, TLS Web Client Authentication
                ...
4

I dont understand why I need to provide this:

- CN=node1.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node2.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node3.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com

THis information is already a part of the certificate and when a node connects to another node it checks if its certificate is signed by the same ca provided in the config or not? Can i leave this out, because this is part of the certificate ?

5

You are right, for validating the certificate you would not need to provide this information.

However, this section makes sure that only trusted nodes can actually join your cluster. Without this information, you could use any certificate that validates correctly against your root CA to add nodes to the cluster. Defining what certificates are deemed valid node certificates protects the cluster from the following attack scenario: An attacker has no direct access to the machines where the nodes are running, but he has obtained a valid cert. The attacker could then spin up another data node, let it join your cluster, and then data will get replicated to this (malicious) node. Which is now in the hands of the attacker.

While it may sound a bit esoteric, this attack pattern has already been successfully used “in the wild”, so it is a real thing.

But - having said that, we support wildcards and regular expressions here. So, of course, you do not need to list every node separately, but you would rather use something like:

- CN=*.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com

Many customers use dedicated CNs for their ELK installation, like:

- CN=elk-*.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com

Or, if you are adventurous :wink: you could even use

- CN=*

but that would allow any certificate to be used as a node certificate.

6

Thank you for the clarification