Hi everyone,
I’m trying to deploy searchguard in a 3 nodes cluster.
I already did it on a 2 nodes (both local) with success.
I generated nodes certificates and client nodes certificates for applications connected to elasticsearch.
All nodes seem to start correctly and discover each other.
I tried to call elasticsearch using curl and a login/password user (defined in sg_internal_users.yml) and it works fine (I got a result with correct credentials, and Unauthorized when i use a wrong password).
But then I tried to import my generated keystore and truststore in an application using SearchGuard plugin in its transport client (java), and got the following result in the target elasticsearch node logs :
[2017-11-22T17:53:14,525][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] SSL Problem Received fatal alert: certificate_unknown
javax.net.ssl.SSLException: Received fatal alert: certificate_unknown
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_144]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:254) ~[netty-handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1156) ~[netty-handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1078) ~[netty-handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
``
This setup was working in my test setup, but here I can’t get it to work.
I found no way of logging ssl infos. I added -Djavax.net.debug=all in every location possible (jvm.options and directly in elasticsearch script), but got no ssl debug data.
The only thing i get is searhguard debug info
[2017-11-22T17:52:35,887][INFO ][o.e.n.Node ] [ppjbies1] JVM arguments [-Xms2g, -Xmx2g, -Djavax.net.debug=all, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Djavax.net.debug=all, -Des.path.home=/opt/es-home]
[2017-11-22T17:52:36,679][INFO ][c.f.s.SearchGuardPlugin ] Clustername: pp-company-es-cluster
[2017-11-22T17:52:36,679][WARN ][c.f.s.SearchGuardPlugin ]
LICENSE NOTICE Search Guard
If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See Licensing | Search Guard Community, Enterprise and Compliance Edition)
-
Kibana Multitenancy
-
LDAP authentication/authorization
-
Active Directory authentication/authorization
-
REST Management API
-
JSON Web Token (JWT) authentication/authorization
-
Kerberos authentication/authorization
-
Document- and Fieldlevel Security (DLS/FLS)
-
Auditlogging
In case of any doubt mail to sales@floragunn.com
···
###################################
LICENSE NOTICE Search Guard
If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See Licensing | Search Guard Community, Enterprise and Compliance Edition)
-
Kibana Multitenancy
-
LDAP authentication/authorization
-
Active Directory authentication/authorization
-
REST Management API
-
JSON Web Token (JWT) authentication/authorization
-
Kerberos authentication/authorization
-
Document- and Fieldlevel Security (DLS/FLS)
-
Auditlogging
In case of any doubt mail to sales@floragunn.com
###################################
[2017-11-22T17:52:36,680][WARN ][c.f.s.SearchGuardPlugin ] Consider setting -Djdk.tls.rejectClientInitiatedRenegotiation=true to prevent DoS attacks through client side initiated TLS renegotiation.
Consider setting -Djdk.tls.rejectClientInitiatedRenegotiation=true to prevent DoS attacks through client side initiated TLS renegotiation.
[2017-11-22T17:52:36,711][INFO ][c.f.s.SearchGuardPlugin ] Node [ppjbies1] is a transportClient: false/tribeNode: false/tribeNodeClient: false
[2017-11-22T17:52:36,712][INFO ][c.f.s.SearchGuardPlugin ] FLS/DLS module not available
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Open SSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.version: 1.8.0_144
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vendor: Oracle Corporation
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vm.specification.version: 1.8
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vm.specification.vendor: Oracle Corporation
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vm.specification.name: Java Virtual Machine Specification
[2017-11-22T17:52:36,725][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vm.name: Java HotSpot™ 64-Bit Server VM
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.vm.vendor: Oracle Corporation
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.specification.version: 1.8
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.specification.vendor: Oracle Corporation
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] java.specification.name: Java Platform API Specification
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] os.name: Linux
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] os.arch: amd64
[2017-11-22T17:52:36,726][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] os.version: 2.6.32-642.11.1.el6.x86_64
[2017-11-22T17:52:36,919][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] JVM supports the following 57 ciphers for https [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
[2017-11-22T17:52:36,931][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] JVM supports the following 57 ciphers for transport [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_DH_anon_WITH_AES_128_GCM_SHA256, TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_RSA_WITH_NULL_SHA256, TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_NULL_SHA, SSL_RSA_WITH_NULL_SHA, TLS_ECDH_ECDSA_WITH_NULL_SHA, TLS_ECDH_RSA_WITH_NULL_SHA, TLS_ECDH_anon_WITH_NULL_SHA, SSL_RSA_WITH_NULL_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
[2017-11-22T17:52:36,932][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /opt/es-data/config/, from there the key- and truststore files are resolved relatively
[2017-11-22T17:52:36,932][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.keystore_filepath is /opt/es-data/config/ppjbies1-keystore.jks
[2017-11-22T17:52:36,932][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved /opt/es-data/config/ppjbies1-keystore.jks to /opt/es-data/config/ppjbies1-keystore.jks against /opt/es-data/config
[2017-11-22T17:52:36,933][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.truststore_filepath is /opt/es-data/config/truststore.jks
[2017-11-22T17:52:36,933][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved /opt/es-data/config/truststore.jks to /opt/es-data/config/truststore.jks against /opt/es-data/config
[2017-11-22T17:52:36,937][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,937][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: is a certificate entry?false/is a key entry?true
[2017-11-22T17:52:36,937][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: chain len 3
[2017-11-22T17:52:36,938][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,938][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Signing CA, OU=company Signing CA, O=company, DC=company, DC=com of type 0 → false
[2017-11-22T17:52:36,938][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,938][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: single cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,939][INFO ][c.f.s.s.u.SSLCertificateHelper] No alias given, use the first one: ppjbies1
[2017-11-22T17:52:36,939][WARN ][c.f.s.s.u.SSLCertificateHelper] Certificate chain for alias ppjbies1 contains a root certificate
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: is a certificate entry?false/is a key entry?true
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: chain len 3
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Signing CA, OU=company Signing CA, O=company, DC=company, DC=com of type 0 → false
[2017-11-22T17:52:36,939][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,940][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: single cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,941][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,941][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias root-ca-chain: is a certificate entry?true/is a key entry?false
[2017-11-22T17:52:36,941][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias root-ca-chain: single cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,941][DEBUG][c.f.s.s.u.SSLCertificateHelper] No alias given, will trust all of the certificates in the store
[2017-11-22T17:52:36,973][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.keystore_filepath is /opt/es-data/config/ppjbies1-keystore.jks
[2017-11-22T17:52:36,974][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved /opt/es-data/config/ppjbies1-keystore.jks to /opt/es-data/config/ppjbies1-keystore.jks against /opt/es-data/config
[2017-11-22T17:52:36,974][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] HTTPS client auth mode OPTIONAL
[2017-11-22T17:52:36,974][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,974][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: is a certificate entry?false/is a key entry?true
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: chain len 3
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Signing CA, OU=company Signing CA, O=company, DC=company, DC=com of type 0 → false
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: single cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,975][INFO ][c.f.s.s.u.SSLCertificateHelper] No alias given, use the first one: ppjbies1
[2017-11-22T17:52:36,975][WARN ][c.f.s.s.u.SSLCertificateHelper] Certificate chain for alias ppjbies1 contains a root certificate
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,975][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: is a certificate entry?false/is a key entry?true
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: chain len 3
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Signing CA, OU=company Signing CA, O=company, DC=company, DC=com of type 0 → false
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.u.SSLCertificateHelper] cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias ppjbies1: single cert CN=ppjbies1.services.company.com, OU=APO, O=company, L=France, C=FR of type -1 → false
[2017-11-22T17:52:36,976][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.truststore_filepath is /opt/es-data/config/truststore.jks
[2017-11-22T17:52:36,977][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved /opt/es-data/config/truststore.jks to /opt/es-data/config/truststore.jks against /opt/es-data/config
[2017-11-22T17:52:36,977][DEBUG][c.f.s.s.u.SSLCertificateHelper] Keystore has 1 entries/aliases
[2017-11-22T17:52:36,977][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias root-ca-chain: is a certificate entry?true/is a key entry?false
[2017-11-22T17:52:36,978][DEBUG][c.f.s.s.u.SSLCertificateHelper] Alias root-ca-chain: single cert CN=company Root CA, OU=company Root CA, O=company, DC=company, DC=com of type 2147483647 → true
[2017-11-22T17:52:36,978][DEBUG][c.f.s.s.u.SSLCertificateHelper] No alias given, will trust all of the certificates in the store
[2017-11-22T17:52:36,983][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] AES-256 not supported, max key length for AES is 128 bit… That is not an issue, it just limits possible encryption strength. To enable AES 256 install ‘Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files’
[2017-11-22T17:52:36,983][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportClientProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
[2017-11-22T17:52:36,983][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportServerProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
[2017-11-22T17:52:36,984][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] sslHTTPProvider:JDK with ciphers [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
[2017-11-22T17:52:36,984][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] sslTransport protocols [TLSv1.2, TLSv1.1]
[2017-11-22T17:52:36,984][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] sslHTTP protocols [TLSv1.2, TLSv1.1]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [aggs-matrix-stats]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [ingest-common]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [lang-expression]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [lang-groovy]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [lang-mustache]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [lang-painless]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [parent-join]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [percolator]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [reindex]
[2017-11-22T17:52:36,987][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [transport-netty3]
[2017-11-22T17:52:36,988][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded module [transport-netty4]
[2017-11-22T17:52:36,988][INFO ][o.e.p.PluginsService ] [ppjbies1] loaded plugin [search-guard-5]
[2017-11-22T17:52:38,566][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin com.floragunn.searchguard.SearchGuardPlugin
[2017-11-22T17:52:38,617][INFO ][c.f.s.SearchGuardPlugin ] FLS/DLS valve not bound (noop) due to java.lang.ClassNotFoundException: com.floragunn.searchguard.configuration.DlsFlsValveImpl
[2017-11-22T17:52:38,618][INFO ][c.f.s.SearchGuardPlugin ] Auditlog not available due to java.lang.ClassNotFoundException: com.floragunn.searchguard.auditlog.impl.AuditLogImpl
[2017-11-22T17:52:38,619][DEBUG][c.f.s.SearchGuardPlugin ] Using com.floragunn.searchguard.transport.DefaultInterClusterRequestEvaluator as intercluster request evaluator class
[2017-11-22T17:52:38,620][INFO ][c.f.s.SearchGuardPlugin ] Privileges interceptor not bound (noop) due to java.lang.ClassNotFoundException: com.floragunn.searchguard.configuration.PrivilegesInterceptorImpl
[2017-11-22T17:52:38,629][DEBUG][c.f.s.c.AdminDNs ] CN=sgadmin,OU=APO,O=company,L=France,C=FR is registered as an admin dn
[2017-11-22T17:52:38,631][DEBUG][c.f.s.c.AdminDNs ] Loaded 1 admin DN’s [CN=sgadmin,OU=APO,O=company,L=France,C=FR]
[2017-11-22T17:52:38,632][DEBUG][c.f.s.c.AdminDNs ] Loaded 0 impersonation DN’s {}
[2017-11-22T17:52:38,645][DEBUG][c.f.s.c.ConfigurationLoader] Index is: searchguard
[2017-11-22T17:52:38,646][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Subscribe on configuration changes by type config with listener com.floragunn.searchguard.http.XFFResolver@32120956
[2017-11-22T17:52:38,674][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Subscribe on configuration changes by type config with listener com.floragunn.searchguard.auth.BackendRegistry@67770b37
[2017-11-22T17:52:38,717][DEBUG][c.f.s.h.SearchGuardHttpServerTransport] [ppjbies1] using max_chunk_size[8kb], max_header_size[8kb], max_initial_line_length[4kb], max_content_length[100mb], receive_predictor[64kb->64kb], pipelining[true], pipelining_max_events[10000]
[2017-11-22T17:52:38,739][INFO ][o.e.d.DiscoveryModule ] [ppjbies1] using discovery type [zen]
[2017-11-22T17:52:39,342][INFO ][o.e.n.Node ] [ppjbies1] initialized
[2017-11-22T17:52:39,343][INFO ][o.e.n.Node ] [ppjbies1] starting …
[2017-11-22T17:52:39,413][DEBUG][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] using profile[default], worker_count[8], port[9300-9400], bind_host[null], publish_host[null], compress[false], connect_timeout[30s], connections_per_node[2/3/6/1/1], receive_predictor[64kb->64kb]
[2017-11-22T17:52:39,419][DEBUG][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] binding server bootstrap to: [0.0.0.0]
[2017-11-22T17:52:39,498][DEBUG][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] Bound profile [default] to address {0.0.0.0:9300}
[2017-11-22T17:52:39,500][INFO ][o.e.t.TransportService ] [ppjbies1] publish_address {172.21.22.38:9300}, bound_addresses {0.0.0.0:9300}
[2017-11-22T17:52:39,510][INFO ][o.e.b.BootstrapChecks ] [ppjbies1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-11-22T17:52:39,515][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Check if searchguard index exists …
[2017-11-22T17:52:39,522][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [ppjbies1] no known master node, scheduling a retry
[2017-11-22T17:52:59,361][DEBUG][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] connected to node [{ppjessbi}{FVHpTmZURWGQs1yxLhu0kw}{cFtADFUYSEWk_mY6nrTCQQ}{172.21.22.37}{172.21.22.37:9300}]
[2017-11-22T17:52:59,617][DEBUG][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] connected to node [{ppjbies2}{jhcY9XfoTIikmatGsYGpZQ}{w8W6-t8JSRuyZFf00gvXLQ}{172.21.22.39}{172.21.22.39:9300}]
[2017-11-22T17:52:59,620][INFO ][o.e.c.s.ClusterSettings ] [ppjbies1] updating [cluster.routing.allocation.enable] from [ALL] to [all]
[2017-11-22T17:52:59,926][DEBUG][c.f.s.h.SearchGuardHttpServerTransport] [ppjbies1] Bound http to address {0.0.0.0:9200}
[2017-11-22T17:52:59,928][INFO ][c.f.s.h.SearchGuardHttpServerTransport] [ppjbies1] publish_address {172.21.22.38:9200}, bound_addresses {0.0.0.0:9200}
[2017-11-22T17:52:59,929][INFO ][o.e.n.Node ] [ppjbies1] started
[2017-11-22T17:53:00,698][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Node started, try to initialize it. Wait for at least yellow cluster state…
[2017-11-22T17:53:00,762][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Try to load config …
[2017-11-22T17:53:02,080][DEBUG][c.f.s.c.ConfigurationLoader] Received config for config (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=4
[2017-11-22T17:53:02,087][DEBUG][c.f.s.c.ConfigurationLoader] Received config for roles (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=3
[2017-11-22T17:53:02,091][DEBUG][c.f.s.c.ConfigurationLoader] Received config for rolesmapping (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=2
[2017-11-22T17:53:02,093][DEBUG][c.f.s.c.ConfigurationLoader] Received config for internalusers (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=1
[2017-11-22T17:53:02,096][DEBUG][c.f.s.c.ConfigurationLoader] Received config for actiongroups (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=0
[2017-11-22T17:53:02,096][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Retrieved [rolesmapping, config, internalusers, actiongroups, roles] configs
[2017-11-22T17:53:03,177][DEBUG][c.f.s.c.ConfigurationLoader] Received config for config (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=4
[2017-11-22T17:53:03,177][DEBUG][c.f.s.c.ConfigurationLoader] Received config for roles (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=3
[2017-11-22T17:53:03,178][DEBUG][c.f.s.c.ConfigurationLoader] Received config for rolesmapping (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=2
[2017-11-22T17:53:03,178][DEBUG][c.f.s.c.ConfigurationLoader] Received config for internalusers (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=1
[2017-11-22T17:53:03,181][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Notify com.floragunn.searchguard.http.XFFResolver@32120956 listener about change configuration with type config
[2017-11-22T17:53:03,181][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Notify com.floragunn.searchguard.auth.BackendRegistry@67770b37 listener about change configuration with type config
[2017-11-22T17:53:03,184][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node ‘ppjbies1’ initialized
[2017-11-22T17:53:03,184][DEBUG][c.f.s.c.ConfigurationLoader] Received config for actiongroups (of [config, roles, rolesmapping, internalusers, actiongroups]) with current latch value=0
[2017-11-22T17:53:04,225][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [ppjbies1] SSL Problem Received fatal alert: certificate_unknown
``
And i configured each node this way, configured users, roles, actions and so on, then installed them using sgadmin.sh, whcih finished with success message.
######## Start Search Guard Configuration ########
searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.keystore_filepath: /opt/es-data/config/ppjessbi-keystore.jks
searchguard.ssl.transport.truststore_filepath: /opt/es-data/config/truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: /opt/es-data/config/ppjessbi-keystore.jks
searchguard.ssl.http.truststore_filepath: /opt/es-data/config/truststore.jks
The setting below informs SearchGuard of all client certificates allowed to administrate SearhGuard.
searchguard.authcz.admin_dn:
- CN=sgadmin,OU=APO,O=company,L=France,C=FR
######## End Search Guard Configuration ########
``
I would really appreciate if someone could help.