Issues in configuring searchguard.ssl.transport.pemkey_password

Elasticsearch version: : 7.0.1
Describe the issue: - Unable to configure self signed certs (in pem format) for searchguard.ssl.transport when key is encrypted.

When I generate certificates with -nodes flag (i.e. avoiding encryption of key), I am able to setup SG successfully.

openssl req -x509 -newkey rsa:4096 -keyout node.key.pem -out node.crt.pem -days 365 -subj “/C=IN/CN=elasticsearch.shiv” -nodes ----> Working fine

When the same cmds are used without nodes flag (with encryption of key), the setup is failing.

Steps used to generate self-signed certs:

  1. Node cert (passphrase of key when prompted - “eskey”)

openssl req -x509 -newkey rsa:4096 -keyout node.key.pem -out node.crt.pem -days 365 -subj “/C=IN/CN=elasticsearch.shiv”

  1. Admin cert (passphrase of key - “adminkey” )

openssl req -x509 -newkey rsa:4096 -keyout admin.key.pem -out admin.cert.pem -days 365 -subj “/C=IN/CN=admin”

  1. Rootca

cat node.cert.pem admin.cert.pem > root-ca.pem

Provide configuration:

searchguard.ssl.transport.pemkey_filepath: "/etc/elasticsearch/certs/node.key.pem"
searchguard.ssl.transport.pemcert_filepath: "/etc/elasticsearch/certs/node.crt.pem"
searchguard.ssl.transport.pemtrustedcas_filepath:"/etc/elasticsearch/certs/root-ca.pem"
searchguard.ssl.transport.pemkey_password: "eskey"

Provide logs:
Elasticsearch logs:

{“type”:“log”,“host”:“es-shiv-elk-elasticsearch-master-0”,“level”:“INFO”,“systemid”:“4636c00bfc3849e0be179bc71cef17f8”,“system”:“elk”,“time”: “2020-06-16T13:53:33.243Z”,“logger”:“c.f.s.s.DefaultSearchGuardKeyStore”,“timezone”:“UTC”,“marker”:"[es-shiv-elk-elasticsearch-master-0] “,“log”:“Config directory is /etc/elasticsearch/, from there the key- and truststore files are resolved relatively”}
{“type”:“log”,“host”:“es-shiv-elk-elasticsearch-master-0”,“level”:“ERROR”,“systemid”:“4636c00bfc3849e0be179bc71cef17f8”,“system”:“elk”,“time”: “2020-06-16T13:53:33.427Z”,“logger”:“c.f.s.s.DefaultSearchGuardKeyStore”,“timezone”:“UTC”,“marker”:”[es-shiv-elk-elasticsearch-master-0] “,“log”:“Your keystore or PEM does not contain a key. If you specified a key password, try removing it. If you did not specify a key password, perhaps you need to if the key is in fact password-protected. Maybe you just confused keys and certificates.”}
{“type”:“log”,“host”:“es-shiv-elk-elasticsearch-master-0”,“level”:“WARN”,“systemid”:“4636c00bfc3849e0be179bc71cef17f8”,“system”:“elk”,“time”: “2020-06-16T13:53:33.713Z”,“logger”:“o.e.b.ElasticsearchUncaughtExceptionHandler”,“timezone”:“UTC”,“marker”:”[es-shiv-elk-elasticsearch-master-0] ",“log”:“uncaught exception in thread [main]”}
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.0.1.jar:7.0.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.0.1.jar:7.0.1]
Caused by: java.lang.IllegalStateException: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:614) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: org.elasticsearch.ElasticsearchSecurityException: Error while initializing transport SSL layer from PEM: java.lang.IllegalArgumentException: File does not contain valid private key: /etc/elasticsearch/certs/node.key.pem
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:351) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.(DefaultSearchGuardKeyStore.java:151) ~[?:?]
at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.(SearchGuardSSLPlugin.java:194) ~[?:?]
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:211) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: java.lang.IllegalArgumentException: File does not contain valid private key: /etc/elasticsearch/certs/node.key.pem

But I checked the validity of the key -

$ openssl rsa -in node.key.pem -check
Enter pass phrase for es-key.pem: ( “eskey”)
RSA key ok
writing RSA key…

Since one log msg suggested to remove key password if provided, I removed the parameter searchguard.ssl.transport.pemkey_password: “eskey” from the configuration and restarted. With this, I get this error -

Caused by: java.lang.IllegalArgumentException: File does not contain valid private key: /etc/elasticsearch/certs/node.key.pem
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:270) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServerContext(DefaultSearchGuardKeyStore.java:737) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:338) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.(DefaultSearchGuardKeyStore.java:151) ~[?:?]
at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.(SearchGuardSSLPlugin.java:194) ~[?:?]
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:211) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: java.security.spec.InvalidKeySpecException: Neither RSA, DSA nor EC worked
at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1046) ~[?:?]
at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1015) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:268) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServerContext(DefaultSearchGuardKeyStore.java:737) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:338) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.(DefaultSearchGuardKeyStore.java:151) ~[?:?]
at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.(SearchGuardSSLPlugin.java:194) ~[?:?]
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:211) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: IOException : DER input, Integer tag error
at sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java:169) ~[?:?]
at java.security.KeyFactory.generatePrivate(KeyFactory.java:390) ~[?:?]
at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1044) ~[?:?]
at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1015) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:268) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServerContext(DefaultSearchGuardKeyStore.java:737) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:338) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.(DefaultSearchGuardKeyStore.java:151) ~[?:?]
at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.(SearchGuardSSLPlugin.java:194) ~[?:?]
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:211) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more
Caused by: java.security.InvalidKeyException: IOException : DER input, Integer tag error
at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:350) ~[?:?]
at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:355) ~[?:?]
at sun.security.ec.ECPrivateKeyImpl.(ECPrivateKeyImpl.java:74) ~[?:?]
at sun.security.ec.ECKeyFactory.implGeneratePrivate(ECKeyFactory.java:237) ~[?:?]
at sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java:165) ~[?:?]
at java.security.KeyFactory.generatePrivate(KeyFactory.java:390) ~[?:?]
at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1044) ~[?:?]
at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1015) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:268) ~[?:?]
at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServerContext(DefaultSearchGuardKeyStore.java:737) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:338) ~[?:?]
at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.(DefaultSearchGuardKeyStore.java:151) ~[?:?]
at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.(SearchGuardSSLPlugin.java:194) ~[?:?]
at com.floragunn.searchguard.SearchGuardPlugin.(SearchGuardPlugin.java:211) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:306) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
… 6 more

Can someone pls explain the right way to configure searchguard.ssl.transport.pemkey_password parameter? And if the steps used to generate/configure certs should be modified?

Thanks!

First, create the TLS certificates using the Search Guard TLS tool docs/tls.md · master · search-guard / labs · GitLab

Then the config for a cluster node can be:
elasticsearch.yml

node.name: sgssl-0.example.com
network.host: 0.0.0.0
http.port: 9200
cluster.name: es-docker-cluster
discovery.seed_hosts: ["sgssl-1.example.com", "sgssl-2.example.com"]
cluster.initial_master_nodes: ["sgssl-0.example.com", "sgssl-1.example.com", "sgssl-2.example.com"]
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3

xpack.security.enabled: false

searchguard.allow_default_init_sgindex: true
searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]
# This is a configuration snippet for the node node1
# This snippet needs to be inserted into the file config/elasticsearch.yml of the respective node.
# If the config file already contains SearchGuard configuration, this needs to be replaced.
# Furthermore, you need to copy the files referenced below into the same directory.
# Please refer to http://docs.search-guard.com/latest/configuring-tls for further configuration of your installation.


searchguard.ssl.transport.pemcert_filepath: node1.pem
searchguard.ssl.transport.pemkey_filepath: node1.key
searchguard.ssl.transport.pemkey_password: 6bbhtBZakqUT
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: node1_http.pem
searchguard.ssl.http.pemkey_filepath: node1_http.key
searchguard.ssl.http.pemkey_password: dGaWWFn8pt33
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.nodes_dn:
- CN=node1.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node2.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
- CN=node3.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
searchguard.authcz.admin_dn:
- CN=kirk.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com

You cant try it in docker using this repo Files · master · search-guard / labs · GitLab

This error may arise from this Java bug Loading...

The solution is to use the v1 encryption scheme. With openssl use the -v1 flag, for example openssl pkcs8 -in key.pem -topk8 -out enckey.pem -v1 PBE-SHA1-3DES