elasticsearch.yml "searchguard.ssl.transport.keystore_filepath" java keystore in a subfolder

Search Guard
1

Hi,

we setup our Elastic Stack with the official Puppet module and install the SG plugin.

The Puppet module default behaviour is to set up the elasticsearch config path under

/etc/elasticsearch//elasticsearch.yml

The tree output from our elastic node:

root@elk1:/# tree /etc/elasticsearch/

/etc/elasticsearch/

├── elk1

│ ├── elasticsearch.yml

│ ├── jvm.options

│ ├── log4j2.properties

│ ├── logging.yml

│ ├── elk1.jks

│ ├── ca.jks

│ └── scripts → /usr/share/elasticsearch/scripts

├── scripts

├── ssl

│ ├── https.crt

│ ├── https.key

└── x-pack

├── xxx

Our elasticsearch.yml:

root@elk1:/# cat /etc/elasticsearch/elk1/elasticsearch.yml

cluster.name: elk

http.port: 9200

network.host: 10.2.3.4

node.name: elk1

path.data: “/var/lib/elasticsearch/elk1”

path.logs: “/var/log/elasticsearch/elk1”

searchguard.authcz.admin_dn: “- CN=admin”

searchguard.ssl.http.enable_openssl_if_available: ‘true’

searchguard.ssl.http.enabled: ‘true’

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.transport.enable_openssl_if_available: ‘true’

searchguard.ssl.transport.enforce_hostname_verification: ‘false’

searchguard.ssl.transport.keystore_filepath: ‘elk1/elk1.jks’

searchguard.ssl.transport.keystore_password: xxx

searchguard.ssl.transport.truststore_filepath: ‘elk1/ca.jks’

searchguard.ssl.transport.truststore_password: xxx

xpack.ml.enabled: ‘false’

xpack.monitoring.enabled: ‘true’

xpack.security.enabled: ‘false’

xpack.watcher.enabled: ‘false’

Now we have problems width the paramters “searchguard.ssl.transport.keystore_filepath” and “searchguard.ssl.transport.truststore_filepath” in the elasticsearch.yml.

The log output from elasticsearch:

Caused by: org.elasticsearch.ElasticsearchException: searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.

But we set these parameters in the config above. Why are there problems?

We checked the premissions of the Java Keystore and the are okay.

We try it to set an absolute path “/etc/elasticsearch/elk1/ca.jks” or “./elk1/ca.jks” or “ca.jks”. Nothing works.

If we copy the whole config files from /etc/elasticsearch/elk1/* to /etc/elasticsearch/ and change the paths, it works as expected.

And if we remove the SG plugin, elasticsearch works too as expected.

How do we have to set these parameters if the elasticsearch.yml is in a subfolder?

Thanks

2

Additional we read the tls configuration doku on GitHub for SG.
SG writes here for these two parameters:

relative to the config/ directory (mandatory)

We dont know exactly what this means?

3

Search Guard, during startup, prints the location out from which the config is loaded (all relative to the config/ folder).

···

On Tuesday, 24 October 2017 13:57:03 UTC+2, Joe wrote:

Hi,

we setup our Elastic Stack with the official Puppet module and install the SG plugin.

The Puppet module default behaviour is to set up the elasticsearch config path under

/etc/elasticsearch//elasticsearch.yml

The tree output from our elastic node:

root@elk1:/# tree /etc/elasticsearch/

/etc/elasticsearch/

├── elk1

│ ├── elasticsearch.yml

│ ├── jvm.options

│ ├── log4j2.properties

│ ├── logging.yml

│ ├── elk1.jks

│ ├── ca.jks

│ └── scripts → /usr/share/elasticsearch/scripts

├── scripts

├── ssl

│ ├── https.crt

│ ├── https.key

└── x-pack

├── xxx

Our elasticsearch.yml:

root@elk1:/# cat /etc/elasticsearch/elk1/elasticsearch.yml

cluster.name: elk

http.port: 9200

network.host: 10.2.3.4

node.name: elk1

path.data: “/var/lib/elasticsearch/elk1”

path.logs: “/var/log/elasticsearch/elk1”

searchguard.authcz.admin_dn: “- CN=admin”

searchguard.ssl.http.enable_openssl_if_available: ‘true’

searchguard.ssl.http.enabled: ‘true’

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.transport.enable_openssl_if_available: ‘true’

searchguard.ssl.transport.enforce_hostname_verification: ‘false’

searchguard.ssl.transport.keystore_filepath: ‘elk1/elk1.jks’

searchguard.ssl.transport.keystore_password: xxx

searchguard.ssl.transport.truststore_filepath: ‘elk1/ca.jks’

searchguard.ssl.transport.truststore_password: xxx

xpack.ml.enabled: ‘false’

xpack.monitoring.enabled: ‘true’

xpack.security.enabled: ‘false’

xpack.watcher.enabled: ‘false’

Now we have problems width the paramters “searchguard.ssl.transport.keystore_filepath” and “searchguard.ssl.transport.truststore_filepath” in the elasticsearch.yml.

The log output from elasticsearch:

Caused by: org.elasticsearch.ElasticsearchException: searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.

But we set these parameters in the config above. Why are there problems?

We checked the premissions of the Java Keystore and the are okay.

We try it to set an absolute path “/etc/elasticsearch/elk1/ca.jks” or “./elk1/ca.jks” or “ca.jks”. Nothing works.

If we copy the whole config files from /etc/elasticsearch/elk1/* to /etc/elasticsearch/ and change the paths, it works as expected.

And if we remove the SG plugin, elasticsearch works too as expected.

How do we have to set these parameters if the elasticsearch.yml is in a subfolder?

Thanks

“searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.” means that none of those properties are set in elasticsearch.yml. If yoi have set them anyway than it seems that a different elasticsearch.yml gets loaded (simply check this by make the one you believe to gets loaded invalid yml and test if elasticsearch starts despite of this)

4

this one

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

needs to be relative, for example

searchguard.ssl.http.pemcert_filepath: “https.crt”

searchguard.ssl.http.pemkey_filepath: “https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “https.crt”

and all of those files need to be reside in config/ dir which is typically /etc/elasticsearch or in your case maybe etc/elasticsearch/

···

On Tuesday, 24 October 2017 15:15:42 UTC+2, Search Guard wrote:

Search Guard, during startup, prints the location out from which the config is loaded (all relative to the config/ folder).

On Tuesday, 24 October 2017 13:57:03 UTC+2, Joe wrote:

Hi,

we setup our Elastic Stack with the official Puppet module and install the SG plugin.

The Puppet module default behaviour is to set up the elasticsearch config path under

/etc/elasticsearch//elasticsearch.yml

The tree output from our elastic node:

root@elk1:/# tree /etc/elasticsearch/

/etc/elasticsearch/

├── elk1

│ ├── elasticsearch.yml

│ ├── jvm.options

│ ├── log4j2.properties

│ ├── logging.yml

│ ├── elk1.jks

│ ├── ca.jks

│ └── scripts → /usr/share/elasticsearch/scripts

├── scripts

├── ssl

│ ├── https.crt

│ ├── https.key

└── x-pack

├── xxx

Our elasticsearch.yml:

root@elk1:/# cat /etc/elasticsearch/elk1/elasticsearch.yml

cluster.name: elk

http.port: 9200

network.host: 10.2.3.4

node.name: elk1

path.data: “/var/lib/elasticsearch/elk1”

path.logs: “/var/log/elasticsearch/elk1”

searchguard.authcz.admin_dn: “- CN=admin”

searchguard.ssl.http.enable_openssl_if_available: ‘true’

searchguard.ssl.http.enabled: ‘true’

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.transport.enable_openssl_if_available: ‘true’

searchguard.ssl.transport.enforce_hostname_verification: ‘false’

searchguard.ssl.transport.keystore_filepath: ‘elk1/elk1.jks’

searchguard.ssl.transport.keystore_password: xxx

searchguard.ssl.transport.truststore_filepath: ‘elk1/ca.jks’

searchguard.ssl.transport.truststore_password: xxx

xpack.ml.enabled: ‘false’

xpack.monitoring.enabled: ‘true’

xpack.security.enabled: ‘false’

xpack.watcher.enabled: ‘false’

Now we have problems width the paramters “searchguard.ssl.transport.keystore_filepath” and “searchguard.ssl.transport.truststore_filepath” in the elasticsearch.yml.

The log output from elasticsearch:

Caused by: org.elasticsearch.ElasticsearchException: searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.

But we set these parameters in the config above. Why are there problems?

We checked the premissions of the Java Keystore and the are okay.

We try it to set an absolute path “/etc/elasticsearch/elk1/ca.jks” or “./elk1/ca.jks” or “ca.jks”. Nothing works.

If we copy the whole config files from /etc/elasticsearch/elk1/* to /etc/elasticsearch/ and change the paths, it works as expected.

And if we remove the SG plugin, elasticsearch works too as expected.

How do we have to set these parameters if the elasticsearch.yml is in a subfolder?

Thanks

“searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.” means that none of those properties are set in elasticsearch.yml. If yoi have set them anyway than it seems that a different elasticsearch.yml gets loaded (simply check this by make the one you believe to gets loaded invalid yml and test if elasticsearch starts despite of this)

5

oh, by the way, this is wrong (syntax wise):

searchguard.authcz.admin_dn: “- CN=admin”

it needs to be

searchguard.authcz.admin_dn:

  • “CN=admin”

see https://github.com/floragunncom/search-guard/blob/5.6.0/sgconfig/elasticsearch.yml.example

If you need a working installation as a kind of blueprint pls refer to GitHub - floragunncom/search-guard: Search Guard Plugin - Security for Elasticsearch

If you need working certificates pls refer to Security and Alerting for Elasticsearch and Kibana | Search Guard

···

On Tuesday, 24 October 2017 15:17:33 UTC+2, Search Guard wrote:

this one

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

needs to be relative, for example

searchguard.ssl.http.pemcert_filepath: “https.crt”

searchguard.ssl.http.pemkey_filepath: “https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “https.crt”

and all of those files need to be reside in config/ dir which is typically /etc/elasticsearch or in your case maybe etc/elasticsearch/

On Tuesday, 24 October 2017 15:15:42 UTC+2, Search Guard wrote:

Search Guard, during startup, prints the location out from which the config is loaded (all relative to the config/ folder).

On Tuesday, 24 October 2017 13:57:03 UTC+2, Joe wrote:

Hi,

we setup our Elastic Stack with the official Puppet module and install the SG plugin.

The Puppet module default behaviour is to set up the elasticsearch config path under

/etc/elasticsearch//elasticsearch.yml

The tree output from our elastic node:

root@elk1:/# tree /etc/elasticsearch/

/etc/elasticsearch/

├── elk1

│ ├── elasticsearch.yml

│ ├── jvm.options

│ ├── log4j2.properties

│ ├── logging.yml

│ ├── elk1.jks

│ ├── ca.jks

│ └── scripts → /usr/share/elasticsearch/scripts

├── scripts

├── ssl

│ ├── https.crt

│ ├── https.key

└── x-pack

├── xxx

Our elasticsearch.yml:

root@elk1:/# cat /etc/elasticsearch/elk1/elasticsearch.yml

cluster.name: elk

http.port: 9200

network.host: 10.2.3.4

node.name: elk1

path.data: “/var/lib/elasticsearch/elk1”

path.logs: “/var/log/elasticsearch/elk1”

searchguard.authcz.admin_dn: “- CN=admin”

searchguard.ssl.http.enable_openssl_if_available: ‘true’

searchguard.ssl.http.enabled: ‘true’

searchguard.ssl.http.pemcert_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.http.pemkey_filepath: “/etc/elasticsearch/ssl/https.key”

searchguard.ssl.http.pemtrustedcas_filepath: “/etc/elasticsearch/ssl/https.crt”

searchguard.ssl.transport.enable_openssl_if_available: ‘true’

searchguard.ssl.transport.enforce_hostname_verification: ‘false’

searchguard.ssl.transport.keystore_filepath: ‘elk1/elk1.jks’

searchguard.ssl.transport.keystore_password: xxx

searchguard.ssl.transport.truststore_filepath: ‘elk1/ca.jks’

searchguard.ssl.transport.truststore_password: xxx

xpack.ml.enabled: ‘false’

xpack.monitoring.enabled: ‘true’

xpack.security.enabled: ‘false’

xpack.watcher.enabled: ‘false’

Now we have problems width the paramters “searchguard.ssl.transport.keystore_filepath” and “searchguard.ssl.transport.truststore_filepath” in the elasticsearch.yml.

The log output from elasticsearch:

Caused by: org.elasticsearch.ElasticsearchException: searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.

But we set these parameters in the config above. Why are there problems?

We checked the premissions of the Java Keystore and the are okay.

We try it to set an absolute path “/etc/elasticsearch/elk1/ca.jks” or “./elk1/ca.jks” or “ca.jks”. Nothing works.

If we copy the whole config files from /etc/elasticsearch/elk1/* to /etc/elasticsearch/ and change the paths, it works as expected.

And if we remove the SG plugin, elasticsearch works too as expected.

How do we have to set these parameters if the elasticsearch.yml is in a subfolder?

Thanks

“searchguard.ssl.transport.keystore_filepath or searchguard.ssl.transport.pemkey_filepath must be set if transport ssl is reqested.” means that none of those properties are set in elasticsearch.yml. If yoi have set them anyway than it seems that a different elasticsearch.yml gets loaded (simply check this by make the one you believe to gets loaded invalid yml and test if elasticsearch starts despite of this)