So, I popped the jar into the plugins folder and rebooted ES on each node and now the cluster is stuck in a red state.
Curling the cluster health states:
Search Guard not initialized (SG11)
The logs suggest the same.
[2017-10-10T09:31:52,370][INFO ][o.e.c.m.MetaDataUpdateSettingsService] [elastic-master-01.x-x.com] updating number_of_replicas to  for indices [searchguard]
[2017-10-10T09:31:52,384][INFO ][o.e.c.m.MetaDataUpdateSettingsService] [elastic-master-01.x-x.com] [searchguard/MfRS8XLmSoOGul-OHMMaDQ] auto expanded replicas to 
[2017-10-10T09:31:57,063][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)
[2017-10-10T09:32:17,684][WARN ][c.f.s.c.IndexBaseConfigurationRepository] index ‘searchguard’ not healthy yet, we try again … (Reason: timeout)
Re-running the sgadmin tool just hangs and eventually fails with a timeout. What logs can I look at to figure out why this has happened? This seems to happen almost every time something in a config is changed. Am I doing something wrong somewhere?
I am running sgadmin in the exact same was as I do it on cluster setup.
*/bin/bash /usr/share/elasticsearch/plugins/search-guard-5/tools/sgadmin.sh *
*-cd /usr/share/elasticsearch/plugins/search-guard-5/sgconfig *
*-hostname elastic-master-01.x-x.com *
*-cn x-cluster *
*-cert /etc/elasticsearch/elastic-master-01.x-x.com/elastic-admin.pem *
*-cacert /etc/elasticsearch/elastic-master-01.x-x.com/ca-bundle.pem *
*-key /etc/elasticsearch/elastic-master-01.x-x.com/elastic-admin-key.pkcs8 *