Search Guard not initialized (SG11). See https://github.com/floragunncom/search-guard-docs/blob/mast

Hi,

  • Search Guard (5.6.4-18) and Elasticsearch (5.6.4)
  • Using Search guard community edition
  • Oracle JVM 1.8.0
  • No Kibana and other plugins

I m generating the certificates using offline Search Guard tools which are provided by Search Guard. Please find the below information used for generating certificates.

ca:
root:
dn: CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com, Inc.,DC=searchblox,DC=com
keysize: 2048
validityDays: 3650
pkPassword: auto
file: root-ca.pem
nodes:

Herewith I have attached the generated certificates which are generated by Search Guard Tools.

Find the elasticsearch.yml config below

cluster.name: searchblox
node.name: searchblox-node-1
indices.fielddata.cache.size: 40%
http.enabled: true
elasticfence.disabled: false
elasticfence.root.password: searchblox
index.refresh_interval: 4s
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.pemcert_filepath: searchblox-node-1.pem
searchguard.ssl.transport.pemkey_filepath: searchblox-node-1.key
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: searchblox-node-1.pem
searchguard.ssl.http.pemkey_filepath: searchblox-node-1.key
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de
    searchguard.nodes_dn:
  • CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com, Inc.,DC=searchblox,DC=com’
    ######## End Search Guard Demo Configuration ########

After the above configuration, I started the product I tried to approach the https://localhost:9200/_cat/indices url. I got the “Search Guard not initialized (SG11). See https://github.com/floragunncom/search-guard-docs/blob/master/sgadmin.md” the message

I tried to initiate Search Guard using sgadmin script, I received the below error.

Command: sh sgadmin.sh -cd …/sgconfig -key …/…/kirk.key -cert …/…/kirk.pem -cacert …/…/root-ca.pem -icl -nhnv --diagnose --accept-red-cluster -ff

Error message:
WARNING: JAVA_HOME not set, will use /usr/bin/java
Search Guard Admin v5
Will connect to localhost:9300 … done
1256 [main] INFO c.f.s.SearchGuardPlugin - Clustername: elasticsearch

LICENSE NOTICE Search Guard

If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See https://floragunn.com/searchguard-validate-license)

  • Kibana Multitenancy
  • LDAP authentication/authorization
  • Active Directory authentication/authorization
  • REST Management API
  • JSON Web Token (JWT) authentication/authorization
  • Kerberos authentication/authorization
  • Document- and Fieldlevel Security (DLS/FLS)
  • Auditlogging
    In case of any doubt mail to sales@floragunn.com
    ###################################
    1284 [main] INFO c.f.s.SearchGuardPlugin - Node [client] is a transportClient: true/tribeNode: false/tribeNodeClient: false
    1285 [main] INFO c.f.s.SearchGuardPlugin - FLS/DLS module not available
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.version: 1.8.0_151
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vendor: Oracle Corporation
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.version: 1.8
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.vendor: Oracle Corporation
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.name: Java Virtual Machine Specification
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.name: Java HotSpot™ 64-Bit Server VM
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.vendor: Oracle Corporation
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.version: 1.8
    1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.vendor: Oracle Corporation
    1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.name: Java Platform API Specification
    1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.name: Mac OS X
    1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.arch: x86_64
    1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.version: 10.13.2
    1463 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for https [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
    1466 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for transport [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
    1467 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Config directory is /Users/sheik/SearchBloxDev/gitsourcce/build/libs/exploded/searchblox-9.0.war/WEB-INF/lib/tools/, from there the key- and truststore files are resolved relatively
    1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - AES-256 not supported, max key length for AES is 128 bit… That is not an issue, it just limits possible encryption strength. To enable AES 256 install ‘Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files’
    1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportClientProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
    1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportServerProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
    1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTPProvider:null with ciphers
    1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransport protocols [TLSv1.2, TLSv1.1]
    1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTP protocols [TLSv1.2, TLSv1.1]
    1589 [main] INFO o.e.p.PluginsService - no modules loaded
    1590 [main] INFO o.e.p.PluginsService - loaded plugin [com.floragunn.searchguard.SearchGuardPlugin]
    1591 [main] INFO o.e.p.PluginsService - loaded plugin [org.elasticsearch.transport.Netty4Plugin]
    3166 [main] INFO o.e.c.t.TransportClientNodesService - failed to get node info for {#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}, disconnecting…
    org.elasticsearch.transport.RemoteTransportException: [searchblox-node-1][127.0.0.1:9300][cluster:monitor/nodes/liveness]
    Caused by: org.elasticsearch.ElasticsearchSecurityException: Cannot authenticate null
    at com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:176) ~[search-guard-5-5.6.4-18.jar:?]
    at com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:140) ~[search-guard-ssl-5.6.4-23.jar:5.6.4-23]
    at com.floragunn.searchguard.SearchGuardPlugin$4$1.messageReceived(SearchGuardPlugin.java:423) ~[search-guard-5-5.6.4-18.jar:?]
    at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1553) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1510) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1393) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-client-5.6.4.jar:5.6.4]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[netty-common-4.1.13.Final.jar:4.1.13.Final]
    at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_151]
    Failfast is activated
    Diagnostic trace written to: /Users/sheik/SearchBloxDev/gitsourcce/build/libs/exploded/searchblox-9.0.war/WEB-INF/lib/tools/sgadmin_diag_trace_2018-Jun-13_14-20-23.txt
    Contacting elasticsearch cluster ‘elasticsearch’ …
    ERR: Cannot retrieve cluster state due to: None of the configured nodes are available: [{#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}].
    Root cause: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}]] (org.elasticsearch.client.transport.NoNodeAvailableException/org.elasticsearch.client.transport.NoNodeAvailableException)
    • Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you need to check your clustername as well as hostnames in your SSL certificates)
    • Make also sure that your keystore or cert is a client certificate (not a node certificate) and configured properly in elasticsearch.yml
    • If this is not working, try running sgadmin.sh with --diagnose and see diagnose trace log file)
    • Add --accept-red-cluster to allow sgadmin to operate on a red cluster.

Please help me to resolve this situation.

Best,

-Sheik

kirk.key (1.66 KB)

kirk.pem (1.57 KB)

searchblox-node-1.pem (1.62 KB)

searchblox-node-1.key (1.66 KB)

root-ca.key (1.76 KB)

root-ca.pem (1.36 KB)

Seems your elasticsearch.yml is not correct, should look like

searchguard.nodes_dn:
- 'CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com'
searchguard.authcz.admin_dn:
- 'CN=kirk.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com'

The entry:

searchguard.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

seems to be a leftover from the demo installation?

···

Am 13.06.2018 um 10:52 schrieb sheik.syedali@searchblox.com:

Hi,
  • Search Guard (5.6.4-18) and Elasticsearch (5.6.4)
  • Using Search guard community edition
  • Oracle JVM 1.8.0
  • No Kibana and other plugins
I m generating the certificates using offline Search Guard tools which are provided by Search Guard. Please find the below information used for generating certificates.

ca:
   root:
      dn: CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com
      keysize: 2048
      validityDays: 3650
      pkPassword: auto
      file: root-ca.pem
nodes:
  - name: searchblox-node-1
    dn: CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com
clients:
  - name: sheik
    dn: CN=sheik.example.com,OU=Ops,O=Sheik Com\, Inc.,DC=example,DC=com
  - name: kirk
    dn: CN=kirk.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
    admin: true

Herewith I have attached the generated certificates which are generated by Search Guard Tools.

Find the elasticsearch.yml config below

cluster.name: searchblox
node.name: searchblox-node-1
indices.fielddata.cache.size: 40%
http.enabled: true
elasticfence.disabled: false
elasticfence.root.password: searchblox
index.refresh_interval: 4s
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.pemcert_filepath: searchblox-node-1.pem
searchguard.ssl.transport.pemkey_filepath: searchblox-node-1.key
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: searchblox-node-1.pem
searchguard.ssl.http.pemkey_filepath: searchblox-node-1.key
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de
searchguard.nodes_dn:
  - 'CN=root.ca.searchblox.com,OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com'
######## End Search Guard Demo Configuration ########

After the above configuration, I started the product I tried to approach the https://localhost:9200/_cat/indices url. I got the "Search Guard not initialized (SG11). See https://github.com/floragunncom/search-guard-docs/blob/master/sgadmin.md" the message

I tried to initiate Search Guard using sgadmin script, I received the below error.

Command: sh sgadmin.sh -cd ../sgconfig -key ../../kirk.key -cert ../../kirk.pem -cacert ../../root-ca.pem -icl -nhnv --diagnose --accept-red-cluster -ff

Error message:
WARNING: JAVA_HOME not set, will use /usr/bin/java
Search Guard Admin v5
Will connect to localhost:9300 ... done
1256 [main] INFO c.f.s.SearchGuardPlugin - Clustername: elasticsearch
### LICENSE NOTICE Search Guard ###
If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See https://floragunn.com/searchguard-validate-license)
* Kibana Multitenancy
* LDAP authentication/authorization
* Active Directory authentication/authorization
* REST Management API
* JSON Web Token (JWT) authentication/authorization
* Kerberos authentication/authorization
* Document- and Fieldlevel Security (DLS/FLS)
* Auditlogging
In case of any doubt mail to <sales@floragunn.com>
###################################
1284 [main] INFO c.f.s.SearchGuardPlugin - Node [_client_] is a transportClient: true/tribeNode: false/tribeNodeClient: false
1285 [main] INFO c.f.s.SearchGuardPlugin - FLS/DLS module not available
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.version: 1.8.0_151
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vendor: Oracle Corporation
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.version: 1.8
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.vendor: Oracle Corporation
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.name: Java Virtual Machine Specification
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.name: Java HotSpot(TM) 64-Bit Server VM
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.vendor: Oracle Corporation
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.version: 1.8
1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.vendor: Oracle Corporation
1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.name: Java Platform API Specification
1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.name: Mac OS X
1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.arch: x86_64
1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.version: 10.13.2
1463 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for https [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
1466 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for transport [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
1467 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Config directory is /Users/sheik/SearchBloxDev/gitsourcce/build/libs/exploded/searchblox-9.0.war/WEB-INF/lib/tools/, from there the key- and truststore files are resolved relatively
1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - AES-256 not supported, max key length for AES is 128 bit.. That is not an issue, it just limits possible encryption strength. To enable AES 256 install 'Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files'
1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportClientProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportServerProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTPProvider:null with ciphers
1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransport protocols [TLSv1.2, TLSv1.1]
1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTP protocols [TLSv1.2, TLSv1.1]
1589 [main] INFO o.e.p.PluginsService - no modules loaded
1590 [main] INFO o.e.p.PluginsService - loaded plugin [com.floragunn.searchguard.SearchGuardPlugin]
1591 [main] INFO o.e.p.PluginsService - loaded plugin [org.elasticsearch.transport.Netty4Plugin]
3166 [main] INFO o.e.c.t.TransportClientNodesService - failed to get node info for {#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}, disconnecting...
org.elasticsearch.transport.RemoteTransportException: [searchblox-node-1][127.0.0.1:9300][cluster:monitor/nodes/liveness]
Caused by: org.elasticsearch.ElasticsearchSecurityException: Cannot authenticate null
  at com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:176) ~[search-guard-5-5.6.4-18.jar:?]
  at com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:140) ~[search-guard-ssl-5.6.4-23.jar:5.6.4-23]
  at com.floragunn.searchguard.SearchGuardPlugin$4$1.messageReceived(SearchGuardPlugin.java:423) ~[search-guard-5-5.6.4-18.jar:?]
  at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1553) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1510) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1393) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-client-5.6.4.jar:5.6.4]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[netty-common-4.1.13.Final.jar:4.1.13.Final]
  at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_151]
Failfast is activated
Diagnostic trace written to: /Users/sheik/SearchBloxDev/gitsourcce/build/libs/exploded/searchblox-9.0.war/WEB-INF/lib/tools/sgadmin_diag_trace_2018-Jun-13_14-20-23.txt
Contacting elasticsearch cluster 'elasticsearch' ...
ERR: Cannot retrieve cluster state due to: None of the configured nodes are available: [{#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}].
  Root cause: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}]] (org.elasticsearch.client.transport.NoNodeAvailableException/org.elasticsearch.client.transport.NoNodeAvailableException)
   * Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you need to check your clustername as well as hostnames in your SSL certificates)
   * Make also sure that your keystore or cert is a client certificate (not a node certificate) and configured properly in elasticsearch.yml
   * If this is not working, try running sgadmin.sh with --diagnose and see diagnose trace log file)
   * Add --accept-red-cluster to allow sgadmin to operate on a red cluster.

Please help me to resolve this situation.

Best,
-Sheik

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/935d670a-66f4-4adc-af7f-dc6b1978ec27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<kirk.key><kirk.pem><searchblox-node-1.pem><searchblox-node-1.key><root-ca.key><root-ca.pem>

Thanks for your quick reply. It works now.

···

On Wed, Jun 13, 2018 at 2:56 PM, SG info@search-guard.com wrote:

Am 13.06.2018 um 10:52 schrieb sheik.syedali@searchblox.com:

Hi,

  • Search Guard (5.6.4-18) and Elasticsearch (5.6.4)
  • Using Search guard community edition
  • Oracle JVM 1.8.0
  • No Kibana and other plugins

I m generating the certificates using offline Search Guard tools which are provided by Search Guard. Please find the below information used for generating certificates.

ca:

root:

  dn: CN=[root.ca.searchblox.com](http://root.ca.searchblox.com),OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com
  keysize: 2048
  validityDays: 3650
  pkPassword: auto
  file: root-ca.pem

nodes:

  • name: searchblox-node-1
dn: CN=[root.ca.searchblox.com](http://root.ca.searchblox.com),OU=CA,O=SearchBlox Com\, Inc.,DC=searchblox,DC=com

clients:

  • name: sheik
dn: CN=[sheik.example.com](http://sheik.example.com),OU=Ops,O=Sheik Com\, Inc.,DC=example,DC=com
  • name: kirk
dn: CN=[kirk.example.com](http://kirk.example.com),OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
admin: true

Herewith I have attached the generated certificates which are generated by Search Guard Tools.

Find the elasticsearch.yml config below

cluster.name: searchblox

node.name: searchblox-node-1

indices.fielddata.cache.size: 40%

http.enabled: true

elasticfence.disabled: false

elasticfence.root.password: searchblox

index.refresh_interval: 4s

######## Start Search Guard Demo Configuration ########

searchguard.ssl.transport.pemcert_filepath: searchblox-node-1.pem

searchguard.ssl.transport.pemkey_filepath: searchblox-node-1.key

searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemcert_filepath: searchblox-node-1.pem

searchguard.ssl.http.pemkey_filepath: searchblox-node-1.key

searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

searchguard.nodes_dn:

######## End Search Guard Demo Configuration ########

After the above configuration, I started the product I tried to approach the https://localhost:9200/_cat/indices url. I got the “Search Guard not initialized (SG11). See https://github.com/floragunncom/search-guard-docs/blob/master/sgadmin.md” the message

I tried to initiate Search Guard using sgadmin script, I received the below error.

Command: sh sgadmin.sh -cd …/sgconfig -key …/…/kirk.key -cert …/…/kirk.pem -cacert …/…/root-ca.pem -icl -nhnv --diagnose --accept-red-cluster -ff

Error message:

WARNING: JAVA_HOME not set, will use /usr/bin/java

Search Guard Admin v5

Will connect to localhost:9300 … done

1256 [main] INFO c.f.s.SearchGuardPlugin - Clustername: elasticsearch

LICENSE NOTICE Search Guard

If you use one or more of the following features in production

make sure you have a valid Search Guard license

(See https://floragunn.com/searchguard-validate-license)

  • Kibana Multitenancy
  • LDAP authentication/authorization
  • Active Directory authentication/authorization
  • REST Management API
  • JSON Web Token (JWT) authentication/authorization
  • Kerberos authentication/authorization
  • Document- and Fieldlevel Security (DLS/FLS)
  • Auditlogging

In case of any doubt mail to sales@floragunn.com

###################################

1284 [main] INFO c.f.s.SearchGuardPlugin - Node [client] is a transportClient: true/tribeNode: false/tribeNodeClient: false

1285 [main] INFO c.f.s.SearchGuardPlugin - FLS/DLS module not available

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.version: 1.8.0_151

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vendor: Oracle Corporation

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.version: 1.8

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.vendor: Oracle Corporation

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.specification.name: Java Virtual Machine Specification

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.name: Java HotSpot™ 64-Bit Server VM

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.vm.vendor: Oracle Corporation

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.version: 1.8

1317 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.vendor: Oracle Corporation

1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - java.specification.name: Java Platform API Specification

1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.name: Mac OS X

1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.arch: x86_64

1318 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - os.version: 10.13.2

1463 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for https [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]

1466 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - JVM supports the following 57 ciphers for transport [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]

1467 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - Config directory is /Users/sheik/SearchBloxDev/gitsourcce/build/libs/exploded/searchblox-9.0.war/WEB-INF/lib/tools/, from there the key- and truststore files are resolved relatively

1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - AES-256 not supported, max key length for AES is 128 bit… That is not an issue, it just limits possible encryption strength. To enable AES 256 install ‘Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files’

1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportClientProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]

1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransportServerProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]

1587 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTPProvider:null with ciphers

1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslTransport protocols [TLSv1.2, TLSv1.1]

1588 [main] INFO c.f.s.s.DefaultSearchGuardKeyStore - sslHTTP protocols [TLSv1.2, TLSv1.1]

1589 [main] INFO o.e.p.PluginsService - no modules loaded

1590 [main] INFO o.e.p.PluginsService - loaded plugin [com.floragunn.searchguard.SearchGuardPlugin]

1591 [main] INFO o.e.p.PluginsService - loaded plugin [org.elasticsearch.transport.Netty4Plugin]

3166 [main] INFO o.e.c.t.TransportClientNodesService - failed to get node info for {#transport#-1}{uCZ8UuIqQXWIujCL59Fw8w}{localhost}{127.0.0.1:9300}, disconnecting…

org.elasticsearch.transport.RemoteTransportException: [searchblox-node-1][127.0.0.1:9300][cluster:monitor/nodes/liveness]

Caused by: org.elasticsearch.ElasticsearchSecurityException: Cannot authenticate null

  at com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:176) ~[search-guard-5-5.6.4-18.jar:?]
  at com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:140) ~[search-guard-ssl-5.6.4-23.jar:5.6.4-23]
  at com.floragunn.searchguard.SearchGuardPlugin$4$1.messageReceived(SearchGuardPlugin.java:423) ~[search-guard-5-5.6.4-18.jar:?]
  at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1553) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1510) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1393) ~[elasticsearch-5.6.4.jar:5.6.4]
  at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-client-5.6.4.jar:5.6.4]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]
  at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-transport-4.1.13.Final.jar:4.1.13.Final]

Seems your elasticsearch.yml is not correct, should look like

searchguard.nodes_dn:

searchguard.authcz.admin_dn:

The entry:

searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

seems to be a leftover from the demo installation?