sgadmin "None of the configured nodes are available" with 6.2.2-22.1

  • ES 6.2.2 / SG 6.2.2-22.1

  • Default modules from fresh install / trial license

  • JVM 1.8.0_161 on Windows 7 Pro SP1

  • Demo certs and config files

  • No other plugins

There are various other posts here with a similar sgadmin “None of the configured nodes are available” failure, but all seem to be related to custom certificates and cert config, or conflicts between node cert and admin cert. This case is on a development cluster with a fresh Search Guard install with demo certs and the standard elasticsearch.yml config lines added by install_demo_configuration.sh.

This is a 2-node cluster which has been happily running with SG 6.2.2-21.0 until we upgraded a few days ago. This condition persists through complete wipes and reinstalls of the cluster, and shows up also if I only run a single node. I’ve not yet tried a regression test against 6.2.2-21.0.

tools\sgadmin.bat -icl -key …..\config\kirk-key.pem -cert …..\config\kirk.pem -cacert …..\config\root-ca.pem -nhnv -cd d:\project.git\sgconfig

Search Guard Admin v6

Will connect to localhost:9300 … done

Unable to check whether cluster is sane: None of the configured nodes are available: [{#transport#-1}{2oijFSfIQRWnny_P5qVk2A}{localhost}{127.0.0.1:9300}]

ERR: Cannot connect to Elasticsearch. Please refer to elasticsearch logfile for more information

Trace:

NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{2oijFSfIQRWnny_P5qVk2A}{localhost}{127.0.0.1:9300}]]

    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)

    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)

    at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)

    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:371)

    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)

    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)

    at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:444)

    at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:123)

I’ve also tried with -cn hostname instead of -icl.

Elasticsearch DEBUG log messages are too voluminous to reproduce here, but here is everything in the startup that seems relevant to certificate management; nothing at all shows at the time of running sgadmin.

[2018-05-31T17:06:01,212][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is D:\elastic\6.2.2\elasticsearch-6.2.2\config/, from there the key- and truststore files are resolved relatively

[2018-05-31T17:06:01,212][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemcert_filepath is esnode.pem

[2018-05-31T17:06:01,212][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemkey_filepath is esnode-key.pem

[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode-key.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode-key.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemtrustedcas_filepath is root-ca.pem

[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved root-ca.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\root-ca.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,271][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemtrustedcas_filepath is root-ca.pem

[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved root-ca.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\root-ca.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemcert_filepath is esnode.pem

[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,274][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemkey_filepath is esnode-key.pem

[2018-05-31T17:06:01,274][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode-key.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode-key.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config

[2018-05-31T17:06:01,276][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK

[2018-05-31T17:06:01,278][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK

[2018-05-31T17:06:01,278][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK

[2018-05-31T17:06:01,278][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportClientProvider:JDK with ciphers […lots…]

[2018-05-31T17:06:01,279][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportServerProvider:JDK with ciphers […lots…]

[2018-05-31T17:06:01,279][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslHTTPProvider:JDK with ciphers […lots…]

[2018-05-31T17:06:01,280][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]

[2018-05-31T17:06:01,280][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]

[2018-05-31T17:06:01,730][DEBUG][o.e.x.c.s.SSLService ] [devnode-1] using ssl settings [SSLConfiguration{keyConfig=[NONE], trustConfig=JDK trusted certs], cipherSuites=[[…lots…]], supportedProtocols=[[TLSv1.2, TLSv1.1, TLSv1]], sslClientAuth=[REQUIRED], verificationMode=[FULL]}]

[2018-05-31T17:06:04,344][DEBUG][c.f.s.c.AdminDNs ] CN=kirk,OU=client,O=client,L=test, C=de is registered as an admin dn

[2018-05-31T17:06:04,345][DEBUG][c.f.s.c.AdminDNs ] Loaded 1 admin DN’s [CN=kirk,OU=client,O=client,L=test, C=de]

[2018-05-31T17:06:04,346][DEBUG][c.f.s.c.AdminDNs ] Loaded 0 impersonation DN’s {}

[2018-05-31T17:06:04,346][DEBUG][c.f.s.c.AdminDNs ] Loaded 0 impersonation users for REST {}

I’ve also tried running the standalone sgadmin tool from another host (with the certs copied over) which gives the same result; Wireshark shows a single TCP payload from sgadmin to server and an immediate connection close from the server.

Our demo certificates changed recently, maybe thats the reason: https://groups.google.com/d/msgid/search-guard/cbc49abf-f378-4358-96c1-584513c6481d%40googlegroups.com

Make sure sgadmin use the same (new) certificates!

···

Am 31.05.2018 um 18:47 schrieb James Beckett <choccybikkit@gmail.com>:

* ES 6.2.2 / SG 6.2.2-22.1
* Default modules from fresh install / trial license
* JVM 1.8.0_161 on Windows 7 Pro SP1
* Demo certs and config files
* No other plugins

There are various other posts here with a similar sgadmin "None of the configured nodes are available" failure, but all seem to be related to custom certificates and cert config, or conflicts between node cert and admin cert. This case is on a development cluster with a fresh Search Guard install with demo certs and the standard elasticsearch.yml config lines added by install_demo_configuration.sh.

This is a 2-node cluster which has been happily running with SG 6.2.2-21.0 until we upgraded a few days ago. This condition persists through complete wipes and reinstalls of the cluster, and shows up also if I only run a single node. I've not yet tried a regression test against 6.2.2-21.0.

> tools\sgadmin.bat -icl -key ..\..\config\kirk-key.pem -cert ..\..\config\kirk.pem -cacert ..\..\config\root-ca.pem -nhnv -cd d:\project.git\sgconfig
Search Guard Admin v6
Will connect to localhost:9300 ... done
Unable to check whether cluster is sane: None of the configured nodes are available: [{#transport#-1}{2oijFSfIQRWnny_P5qVk2A}{localhost}{127.0.0.1:9300}]
ERR: Cannot connect to Elasticsearch. Please refer to elasticsearch logfile for more information
Trace:
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{2oijFSfIQRWnny_P5qVk2A}{localhost}{127.0.0.1:9300}]]
        at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
        at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
        at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:371)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:444)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:123)

I've also tried with -cn hostname instead of -icl.

Elasticsearch DEBUG log messages are too voluminous to reproduce here, but here is everything in the startup that seems relevant to certificate management; nothing at all shows at the time of running sgadmin.

[2018-05-31T17:06:01,212][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is D:\elastic\6.2.2\elasticsearch-6.2.2\config/, from there the key- and truststore files are resolved relatively
[2018-05-31T17:06:01,212][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemcert_filepath is esnode.pem
[2018-05-31T17:06:01,212][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemkey_filepath is esnode-key.pem
[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode-key.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode-key.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.transport.pemtrustedcas_filepath is root-ca.pem
[2018-05-31T17:06:01,213][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved root-ca.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\root-ca.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,271][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemtrustedcas_filepath is root-ca.pem
[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved root-ca.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\root-ca.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemcert_filepath is esnode.pem
[2018-05-31T17:06:01,272][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,274][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Value for searchguard.ssl.http.pemkey_filepath is esnode-key.pem
[2018-05-31T17:06:01,274][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] Resolved esnode-key.pem to D:\elastic\6.2.2\elasticsearch-6.2.2\config\esnode-key.pem against D:\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-05-31T17:06:01,276][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK
[2018-05-31T17:06:01,278][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK
[2018-05-31T17:06:01,278][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK
[2018-05-31T17:06:01,278][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportClientProvider:JDK with ciphers [...lots...]
[2018-05-31T17:06:01,279][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslTransportServerProvider:JDK with ciphers [...lots...]
[2018-05-31T17:06:01,279][DEBUG][c.f.s.s.DefaultSearchGuardKeyStore] sslHTTPProvider:JDK with ciphers [...lots...]
[2018-05-31T17:06:01,280][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
[2018-05-31T17:06:01,280][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.2, TLSv1.1]
...
[2018-05-31T17:06:01,730][DEBUG][o.e.x.c.s.SSLService ] [devnode-1] using ssl settings [SSLConfiguration{keyConfig=[NONE], trustConfig=JDK trusted certs], cipherSuites=[[...lots...]], supportedProtocols=[[TLSv1.2, TLSv1.1, TLSv1]], sslClientAuth=[REQUIRED], verificationMode=[FULL]}]
...
[2018-05-31T17:06:04,344][DEBUG][c.f.s.c.AdminDNs ] CN=kirk,OU=client,O=client,L=test, C=de is registered as an admin dn
[2018-05-31T17:06:04,345][DEBUG][c.f.s.c.AdminDNs ] Loaded 1 admin DN's [CN=kirk,OU=client,O=client,L=test, C=de]
[2018-05-31T17:06:04,346][DEBUG][c.f.s.c.AdminDNs ] Loaded 0 impersonation DN's {}
[2018-05-31T17:06:04,346][DEBUG][c.f.s.c.AdminDNs ] Loaded 0 impersonation users for REST {}

I've also tried running the standalone sgadmin tool from another host (with the certs copied over) which gives the same result; Wireshark shows a single TCP payload from sgadmin to server and an immediate connection close from the server.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/fd6d89bf-6662-466e-a5c2-138484559002%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

I know - that’s the only reason we moved to 22.1.

sgadmin is using the same certs, as it’s running from the same install location: “sgadmin … -cert …..\config\kirk.pem …”.

···

On Thursday, 31 May 2018 17:59:44 UTC+1, Search Guard wrote:

Our demo certificates changed recently, maybe thats the reason: https://groups.google.com/d/msgid/search-guard/cbc49abf-f378-4358-96c1-584513c6481d%40googlegroups.com

Make sure sgadmin use the same (new) certificates!

So just to make sure I understand correctly:

You are installing SG 6.2.2-22.1 on a fresh ES 6.2.2 instance, and then the sgadmin call already fails? You are saying you are running on Windows 7, but also write you have executed the (.sh) demo installer script.

Could you please explain the steps you performed to install and configure SG so we are able to reproduce your scenario.

Thanks!

···

On Monday, June 4, 2018 at 12:45:46 PM UTC+2, James Beckett wrote:

On Thursday, 31 May 2018 17:59:44 UTC+1, Search Guard wrote:

Our demo certificates changed recently, maybe thats the reason: https://groups.google.com/d/msgid/search-guard/cbc49abf-f378-4358-96c1-584513c6481d%40googlegroups.com

Make sure sgadmin use the same (new) certificates!

I know - that’s the only reason we moved to 22.1.

sgadmin is using the same certs, as it’s running from the same install location: “sgadmin … -cert …..\config\kirk.pem …”.

Demo installer script (.sh) seems to run fine in a git bash shell. Writes out the demo certs, edits the elasticsearch.yml with sound-looking SG config, writes an sgadmin script.

I'm out of office right now so can't paste my setup steps, but there's not much more to say: from memory, install 6.2.2 from zip, set cluster/node name, discovery etc, install x-pack (for monitoring) and disable x-pack security, start and verify, install SG, run demo installer with auto init disabled, (re)start ES, run sgadmin.

I'm going to repeat afresh on a couple of other nodes (Win and Linux) and compare the resulting configs - will update here after.

SOLVED: third-party interaction :frowning:

I’m going to repeat afresh on a couple of other nodes (Win and Linux) and compare the resulting configs - will update here after.

After successes on other nodes, I tried a parallel install on the same node that had a problem (using ports 9205 and 9305 to keep well clear of the usual). Clean install worked fine, sgadmin perfectly happy.

Recursive diff of each install directory showed nothing corrupted in install - only differences were a few elasticsearch.yml lines around zen discovery and the different paths I’d deliberately set for clustername/data/logs etc. I’ve no hair left at this point.

I tried getting a bare TLS connection a few times - even that wasn’t happy:

openssl s_client -msg -CAfile config.devhost/root-ca.pem -cert config.devhost/kirk.pem -key config.devhost/kirk-key.pem -connect devhost:9300
CONNECTED(00000003)
140537057156824:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 305 bytes

but using curl it did reply with a familiar “This is not a HTTP port” so … that’s Elasticsearch. A non-TLS-enabled Elasticsearch. Another Elasticsearch? A vague memory stirred that I’d at some point installed it as a service - but no, that’s not running. Check netstat - OK yes, ports 9300,9301 and 9505 are listening. Check Process Explorer for a some zombie Java process (could it have kept running after a CMD prompt had closed? Seems unlikely.) Check the PID from netstat against the process list and oh, here’s the culprit.

One of the components in App Dynamics, which we’ve been looking at recently, apparently contains an embedded instance of Elasticsearch - which starts up using the default 9200/9300 ports. Host has been running fine with that installed for a while - my supposition is that when I installed it, I had “my” Elasticsearch running, so it claimed ports 9201/9301. On a recent reboot, it claimed the default ports first, moving “my” instance quietly onto 9201/9301. I say “quietly” - it does of course appear in the logs, but unless you’re looking for it, it’s swamped by other output - and it’s sufficiently similar to what you’d expect that it can go un-noticed.

Forcing the ports to 9200/9300 in elasticsearch.yml provokes a failure if already claimed, and I think is a practice I’m going to promote. I’m unconvinced that the standard Elasticsearch choice to try ports in a range is a good idea, but that’s no fault of Search Guard.

What could perhaps be improved by Search Guard is the handling of such a failure case - sgadmin making a TLS connection to a service which turns out to be plaintext Elastic transport protocol. Something to handle the TLS library’s failure cases and report more like “this Elastic instance is not TLS-secured” - perhaps fall back to a non-TLS TransportClient, make a cluster status request and report some basics like cluster name / node name / version? Is it too much an edge case to be worth handling? “None of the configured nodes are available” provides no indication of a transport-level issue.

regards,

James

···

On Tuesday, 5 June 2018 17:47:49 UTC+1, James Beckett wrote:

Hi James,

thanks very much for your very good explanation, that makes perfect sense and is indeed very hard to spot. While I do think it might be an edge case, I will open an issue for improving the error message nonetheless. Because if it does happen, it creates a lot of confusion. I will also add another ticket to add this specific case to the sgadmin Troubleshooting docs, or improve the existing docs. Thanks again for your very helpful explanation!

···

On Thursday, June 7, 2018 at 6:45:25 PM UTC+2, James Beckett wrote:

SOLVED: third-party interaction :frowning:

On Tuesday, 5 June 2018 17:47:49 UTC+1, James Beckett wrote:

I’m going to repeat afresh on a couple of other nodes (Win and Linux) and compare the resulting configs - will update here after.

After successes on other nodes, I tried a parallel install on the same node that had a problem (using ports 9205 and 9305 to keep well clear of the usual). Clean install worked fine, sgadmin perfectly happy.

Recursive diff of each install directory showed nothing corrupted in install - only differences were a few elasticsearch.yml lines around zen discovery and the different paths I’d deliberately set for clustername/data/logs etc. I’ve no hair left at this point.

I tried getting a bare TLS connection a few times - even that wasn’t happy:

openssl s_client -msg -CAfile config.devhost/root-ca.pem -cert config.devhost/kirk.pem -key config.devhost/kirk-key.pem -connect devhost:9300
CONNECTED(00000003)
140537057156824:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 305 bytes

but using curl it did reply with a familiar “This is not a HTTP port” so … that’s Elasticsearch. A non-TLS-enabled Elasticsearch. Another Elasticsearch? A vague memory stirred that I’d at some point installed it as a service - but no, that’s not running. Check netstat - OK yes, ports 9300,9301 and 9505 are listening. Check Process Explorer for a some zombie Java process (could it have kept running after a CMD prompt had closed? Seems unlikely.) Check the PID from netstat against the process list and oh, here’s the culprit.

One of the components in App Dynamics, which we’ve been looking at recently, apparently contains an embedded instance of Elasticsearch - which starts up using the default 9200/9300 ports. Host has been running fine with that installed for a while - my supposition is that when I installed it, I had “my” Elasticsearch running, so it claimed ports 9201/9301. On a recent reboot, it claimed the default ports first, moving “my” instance quietly onto 9201/9301. I say “quietly” - it does of course appear in the logs, but unless you’re looking for it, it’s swamped by other output - and it’s sufficiently similar to what you’d expect that it can go un-noticed.

Forcing the ports to 9200/9300 in elasticsearch.yml provokes a failure if already claimed, and I think is a practice I’m going to promote. I’m unconvinced that the standard Elasticsearch choice to try ports in a range is a good idea, but that’s no fault of Search Guard.

What could perhaps be improved by Search Guard is the handling of such a failure case - sgadmin making a TLS connection to a service which turns out to be plaintext Elastic transport protocol. Something to handle the TLS library’s failure cases and report more like “this Elastic instance is not TLS-secured” - perhaps fall back to a non-TLS TransportClient, make a cluster status request and report some basics like cluster name / node name / version? Is it too much an edge case to be worth handling? “None of the configured nodes are available” provides no indication of a transport-level issue.

regards,

James