search-guard-2 not working with ES 2.3.1

Hello there,

I have succesfully set up a cluster of 3 nodes (elise1-elise3) with search-guard-ssl, but when it comes to search-guard-2, something fails with the certificates.

The server gives me this error at start (host names have been removed/changed in this dump):

com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,238][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,239][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Config directory is /etc/elasticsearch/, from there the key- and truststore files are resolved relatively
[setting dump removed]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,241][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Effective settings:
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,346][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,347][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,348][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,357][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP client auth mode OPTIONAL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,366][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,784][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportClientProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportServerProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslHTTPProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,106][INFO ][http ] [elise1] Using [org.elasticsearch.http.netty.NettyHttpServerTransport] as http transport, overridden by [search-guard-ssl]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,314][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.transport.SearchGuardTransportService] as transport service, overridden by [search-guard2]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,315][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport] as transport, overridden by [search-guard-ssl]
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,727][INFO ][node ] [elise1] initialized
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,731][INFO ][node ] [elise1] starting …
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,830][INFO ][com.floragunn.searchguard.transport.SearchGuardTransportService] [elise1] publish_address {10.69.192.153:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}, {10.69.192.153:9300}
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,838][INFO ][discovery ] [elise1] elise.example.com/dhIpumhbQg6i_D45-qvOUw
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,863][DEBUG][action.admin.cluster.health] [elise1] no known master node, scheduling a retry
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,530][INFO ][cluster.service ] [elise1] detected_master {elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}, added {{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300},{elise3}{HEBJdqcQTRmJNvGPiijo7g}{10.69.192.155}{10.69.192.155:9300},}, reason: zen-disco-receive(from master [{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}])
juin 14 10:50:31 elise1 elasticsearch[999]: Exception in thread “Thread-4” ElasticsearchSecurityException[No SSL client certificates found. Search Guards needs the Search Guard SSL plugin to be installed]
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.transport.SearchGuardTransportService.messageReceivedDecorate(SearchGuardTransportService.java:204)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardSSLTransportService$Interceptor.messageReceived(SearchGuardSSLTransportService.java:85)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:245)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.handleRequest(SearchGuardMessageChannelHandler.java:57)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:114)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.messageReceived(SearchGuardMessageChannelHandler.java:45)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.lang.Thread.run(Thread.java:745)
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,977][INFO ][http ] [elise1] publish_address {10.69.192.153:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}, {10.69.192.153:9200}
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,980][INFO ][node ] [elise1] started

And when trying to use the sgadmin tool, I get :

java -cp ‘/usr/share/elasticsearch/plugins/search-guard-ssl/:/usr/share/elasticsearch/plugins/search-guard-2/:/usr/share/elasticsearch/lib/*’ com.floragunn.searchguard.tools.SearchGuardAdmin -ks /etc/elasticsearch/ssl/sgadmin.jks -kspass SOMEPASS -ts /etc/elasticsearch/ssl/truststore.jks -tspass SOMEPASS -cd /etc/elasticsearch/search-guard -h elise1.lyon.cemagref.fr -p 9300 -cn MYCLUSTERNAME
Connect to elise1.example.com:9300
Exception in thread “main” NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{10.69.192.153}{elise1.example.com/10.69.192.153:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:848)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.health(AbstractClient.java:868)
at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:137)

The plugins seem to be correctly installed.

/usr/share/elasticsearch/bin/plugin list
Installed plugins in /usr/share/elasticsearch/plugins:
- search-guard-2
- head
- search-guard-ssl

All the certificates (one per node for transport, one for http, and one for the sgadmin user) have been signed by the same CA.

Any idea about these errors ?

Regards,
Guillaume Perréal.

Can you please upgrade to ES 2.3.3 and SG 2.3.3.0-rc1 and SG SSL 2.3.3.11 and try again?

···

Am 14.06.2016 um 11:24 schrieb Guillaume Perréal <guillaume.perreal@irstea.fr>:

Hello there,

I have succesfully set up a cluster of 3 nodes (elise1-elise3) with search-guard-ssl, but when it comes to search-guard-2, something fails with the certificates.

The server gives me this error at start (host names have been removed/changed in this dump):

com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,238][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,239][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Config directory is /etc/elasticsearch/, from there the key- and truststore files are resolved relatively
[setting dump removed]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,241][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Effective settings:
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,346][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,347][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,348][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,357][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP client auth mode OPTIONAL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,366][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,784][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportClientProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportServerProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslHTTPProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,106][INFO ][http ] [elise1] Using [org.elasticsearch.http.netty.NettyHttpServerTransport] as http transport, overridden by [search-guard-ssl]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,314][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.transport.SearchGuardTransportService] as transport service, overridden by [search-guard2]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,315][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport] as transport, overridden by [search-guard-ssl]
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,727][INFO ][node ] [elise1] initialized
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,731][INFO ][node ] [elise1] starting ...
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,830][INFO ][com.floragunn.searchguard.transport.SearchGuardTransportService] [elise1] publish_address {10.69.192.153:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}, {10.69.192.153:9300}
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,838][INFO ][discovery ] [elise1] elise.example.com/dhIpumhbQg6i_D45-qvOUw
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,863][DEBUG][action.admin.cluster.health] [elise1] no known master node, scheduling a retry
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,530][INFO ][cluster.service ] [elise1] detected_master {elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}, added {{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300},{elise3}{HEBJdqcQTRmJNvGPiijo7g}{10.69.192.155}{10.69.192.155:9300},}, reason: zen-disco-receive(from master [{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}])
juin 14 10:50:31 elise1 elasticsearch[999]: Exception in thread "Thread-4" ElasticsearchSecurityException[No SSL client certificates found. Search Guards needs the Search Guard SSL plugin to be installed]
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.transport.SearchGuardTransportService.messageReceivedDecorate(SearchGuardTransportService.java:204)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardSSLTransportService$Interceptor.messageReceived(SearchGuardSSLTransportService.java:85)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:245)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.handleRequest(SearchGuardMessageChannelHandler.java:57)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:114)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.messageReceived(SearchGuardMessageChannelHandler.java:45)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.lang.Thread.run(Thread.java:745)
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,977][INFO ][http ] [elise1] publish_address {10.69.192.153:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}, {10.69.192.153:9200}
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,980][INFO ][node ] [elise1] started

And when trying to use the sgadmin tool, I get :

java -cp '/usr/share/elasticsearch/plugins/search-guard-ssl/*:/usr/share/elasticsearch/plugins/search-guard-2/*:/usr/share/elasticsearch/lib/*' com.floragunn.searchguard.tools.SearchGuardAdmin -ks /etc/elasticsearch/ssl/sgadmin.jks -kspass SOMEPASS -ts /etc/elasticsearch/ssl/truststore.jks -tspass SOMEPASS -cd /etc/elasticsearch/search-guard -h elise1.lyon.cemagref.fr -p 9300 -cn MYCLUSTERNAME
Connect to elise1.example.com:9300
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{10.69.192.153}{elise1.example.com/10.69.192.153:9300}]]
    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
    at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348)
    at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:848)
    at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.health(AbstractClient.java:868)
    at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:137)

The plugins seem to be correctly installed.

/usr/share/elasticsearch/bin/plugin list
Installed plugins in /usr/share/elasticsearch/plugins:
    - search-guard-2
    - head
    - search-guard-ssl

All the certificates (one per node for transport, one for http, and one for the sgadmin user) have been signed by the same CA.

Any idea about these errors ?

Regards,
Guillaume Perréal.

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/779874c3-ce9c-4ea7-8a29-d4a061e21f08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2.3.3 gave a more precise error. It turns out it was an issue with the node certificates. This is now fixed.

···

Le mardi 14 juin 2016 11:24:43 UTC+2, Guillaume Perréal a écrit :

Hello there,

I have succesfully set up a cluster of 3 nodes (elise1-elise3) with search-guard-ssl, but when it comes to search-guard-2, something fails with the certificates.

The server gives me this error at start (host names have been removed/changed in this dump):

com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,238][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.1t 3 May 2016 available
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,239][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Config directory is /etc/elasticsearch/, from there the key- and truststore files are resolved relatively
[setting dump removed]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,241][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Effective settings:
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,346][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,347][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,348][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Transport keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,357][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP client auth mode OPTIONAL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,366][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 0 [removed for privacy]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 1 CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,367][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] HTTP keystore subject DN no. 2 CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,784][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportClientProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportServerProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:24 elise1 elasticsearch[999]: [2016-06-14 10:50:24,785][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslHTTPProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,106][INFO ][http ] [elise1] Using [org.elasticsearch.http.netty.NettyHttpServerTransport] as http transport, overridden by [search-guard-ssl]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,314][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.transport.SearchGuardTransportService] as transport service, overridden by [search-guard2]
juin 14 10:50:25 elise1 elasticsearch[999]: [2016-06-14 10:50:25,315][INFO ][transport ] [elise1] Using [com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport] as transport, overridden by [search-guard-ssl]
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,727][INFO ][node ] [elise1] initialized
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,731][INFO ][node ] [elise1] starting …
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,830][INFO ][com.floragunn.searchguard.transport.SearchGuardTransportService] [elise1] publish_address {10.69.192.153:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}, {10.69.192.153:9300}
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,838][INFO ][discovery ] [elise1] elise.example.com/dhIpumhbQg6i_D45-qvOUw
juin 14 10:50:27 elise1 elasticsearch[999]: [2016-06-14 10:50:27,863][DEBUG][action.admin.cluster.health] [elise1] no known master node, scheduling a retry
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,530][INFO ][cluster.service ] [elise1] detected_master {elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}, added {{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300},{elise3}{HEBJdqcQTRmJNvGPiijo7g}{10.69.192.155}{10.69.192.155:9300},}, reason: zen-disco-receive(from master [{elise2}{Cjv2-hc1T-qKpMZ75-Vnww}{10.69.192.154}{10.69.192.154:9300}])
juin 14 10:50:31 elise1 elasticsearch[999]: Exception in thread “Thread-4” ElasticsearchSecurityException[No SSL client certificates found. Search Guards needs the Search Guard SSL plugin to be installed]
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.transport.SearchGuardTransportService.messageReceivedDecorate(SearchGuardTransportService.java:204)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardSSLTransportService$Interceptor.messageReceived(SearchGuardSSLTransportService.java:85)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:245)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.handleRequest(SearchGuardMessageChannelHandler.java:57)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:114)
juin 14 10:50:31 elise1 elasticsearch[999]: at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.messageReceived(SearchGuardMessageChannelHandler.java:45)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
juin 14 10:50:31 elise1 elasticsearch[999]: at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
juin 14 10:50:31 elise1 elasticsearch[999]: at java.lang.Thread.run(Thread.java:745)
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,977][INFO ][http ] [elise1] publish_address {10.69.192.153:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}, {10.69.192.153:9200}
juin 14 10:50:31 elise1 elasticsearch[999]: [2016-06-14 10:50:31,980][INFO ][node ] [elise1] started

And when trying to use the sgadmin tool, I get :

java -cp ‘/usr/share/elasticsearch/plugins/search-guard-ssl/:/usr/share/elasticsearch/plugins/search-guard-2/:/usr/share/elasticsearch/lib/*’ com.floragunn.searchguard.tools.SearchGuardAdmin -ks /etc/elasticsearch/ssl/sgadmin.jks -kspass SOMEPASS -ts /etc/elasticsearch/ssl/truststore.jks -tspass SOMEPASS -cd /etc/elasticsearch/search-guard -h elise1.lyon.cemagref.fr -p 9300 -cn MYCLUSTERNAME
Connect to elise1.example.com:9300
Exception in thread “main” NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{10.69.192.153}{elise1.example.com/10.69.192.153:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:848)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.health(AbstractClient.java:868)
at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:137)

The plugins seem to be correctly installed.

/usr/share/elasticsearch/bin/plugin list
Installed plugins in /usr/share/elasticsearch/plugins:
- search-guard-2
- head
- search-guard-ssl

All the certificates (one per node for transport, one for http, and one for the sgadmin user) have been signed by the same CA.

Any idea about these errors ?

Regards,
Guillaume Perréal.