Can't init searchguard after new certificates

Tudor_Rogojanu · March 1, 2019, 3:19pm

Hello,

I have ES 6.0.0 and SG 6-6.0.0-17.beta1 running on a server for more than a year. It’s just a small demo that i play with from time to time, so nobody else has access to it. I had no problems with it for a long time.

I noticed this week that i can’t login anymore. I haven’t done any changes to ES or SG for more than a year. After some debugging it seemed to be because of the old certificates, so i downloaded the new demo certificates. But when i run
./sgadmin.sh -cd …/sgconfig/ -icl -nhnv -cacert root-ca.pem -cert kirk.pem -key kirk-key.pem

``

it says

Search Guard Admin v6
Will connect to localhost:9300 … done
Connected as CN=kirk,OU=client,O=client,L=test,C=de
Contacting elasticsearch cluster ‘searchguard_demo’ and wait for YELLOW clusterstate …
ERR: Timed out while waiting for a green or yellow cluster state.

``

Running

./sgadmin.sh -cn searchguard_demo -cd …/sgconfig/ -nhnv -cacert root-ca.pem -cert kirk.pem -key kirk-key.pem --accept-red-cluster

``

results in

Search Guard Admin v6
Will connect to localhost:9300 … done
Connected as CN=kirk,OU=client,O=client,L=test,C=de
Contacting elasticsearch cluster ‘searchguard_demo’ …
Clustername: searchguard_demo
Clusterstate: RED
Number of nodes: 1
Number of data nodes: 1
searchguard index already exists, so we do not need to create one.
ERR: searchguard index state is RED.
Populate config from /usr/share/elasticsearch/plugins/search-guard-6/sgconfig
Will update ‘config’ with …/sgconfig/sg_config.yml
FAIL: Configuration for ‘config’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][config], source[n/a, actual length: [3.2kb], max length: 2kb]}] and a refresh]]
Will update ‘roles’ with …/sgconfig/sg_roles.yml
FAIL: Configuration for ‘roles’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][roles], source[n/a, actual length: [3.6kb], max length: 2kb]}] and a refresh]]
Will update ‘rolesmapping’ with …/sgconfig/sg_roles_mapping.yml
FAIL: Configuration for ‘rolesmapping’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][rolesmapping], source[{“rolesmapping”:“eyJzZ19hbGxfYWNjZXNzIjp7InJlYWRvbmx5Ijp0cnVlLCJiYWNrZW5kcm9sZXMiOlsiYWRtaW4iXX0sInNnX2xvZ3N0YXNoIjp7ImJhY2tlbmRyb2xlcyI6WyJsb2dzdGFzaCJdfSwic2dfa2liYW5hX3NlcnZlciI6eyJyZWFkb25seSI6dHJ1ZSwidXNlcnMiOlsia2liYW5hc2VydmVyIl19LCJzZ19raWJhbmFfdXNlciI6eyJiYWNrZW5kcm9sZXMiOlsia2liYW5hdXNlciJdfSwic2dfcmVhZGFsbCI6eyJyZWFkb25seSI6dHJ1ZSwiYmFja2VuZHJvbGVzIjpbInJlYWRhbGwiXX0sInNnX21hbmFnZV9zbmFwc2hvdHMiOnsicmVhZG9ubHkiOnRydWUsImJhY2tlbmRyb2xlcyI6WyJzbmFwc2hvdHJlc3RvcmUiXX0sInNnX293bl9pbmRleCI6eyJ1c2VycyI6WyIqIl19fQ==”}]}] and a refresh]]
Will update ‘internalusers’ with …/sgconfig/sg_internal_users.yml
FAIL: Configuration for ‘internalusers’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][internalusers], source[{“internalusers”:“eyJhZG1pbiI6eyJyZWFkb25seSI6dHJ1ZSwiaGFzaCI6IiQyYSQxMiRWY0NEZ2gyTkRrMDdKR04wcmpHYk0uQWQ0MXFWUi9ZRkpjZ0hwMFVHbnM1SkR5bXYuLlRPRyIsInJvbGVzIjpbImFkbWluIl19LCJsb2dzdGFzaCI6eyJyZWFkb25seSI6dHJ1ZSwiaGFzaCI6IiQyYSQxMiR1MVNoUjRsNHVCUzNVdjU5UGEyeTUuMXVRdVpCclp0bU5mcUIzaU0vLmpMMFhvVjlzZ2hTMiIsInJvbGVzIjpbImxvZ3N0YXNoIl19LCJraWJhbmFzZXJ2ZXIiOnsicmVhZG9ubHkiOnRydWUsImhhc2giOiIkMmEkMTIkNEFjZ0F0M3h3T1dhZEE1czVibEw2ZXYzOU9YRE5obU9lc0VvbzMzZVp0cnEyTjBZclUzSC4ifSwia2liYW5hcm8iOnsicmVhZG9ubHkiOnRydWUsImhhc2giOiIkMmEkMTIkSkpTWE5mVG93ejdVdTV0dFhmZVlwZVlFMGFyQUN2Y3dsUEJTdEIxRi5NSTdmMFU5WjRER0MiLCJyb2xlcyI6WyJraWJhbmF1c2VyIiwicmVhZGFsbCJdfSwicmVhZGFsbCI6eyJoYXNoIjoiJDJhJDEyJGFlNHljd3p3dkx0Wnh3WjgyUm1pRXVuQmJJUGlBbUdaZHVCQWpLTjBUWGR3UUZ0Q3dBUnoyIiwicm9sZXMiOlsicmVhZGFsbCJdfSwic25hcHNob3RyZXN0b3JlIjp7Imhhc2giOiIkMnkkMTIkRHB3bWV0SEt3Z1lub3JiZ2R2T1JDZW52NE5BSzhjUFVnOEFJNnB4TEN1V2YvQUxjMC52N1ciLCJyb2xlcyI6WyJzbmFwc2hvdHJlc3RvcmUiXX19”}]}] and a refresh]]
Will update ‘actiongroups’ with …/sgconfig/sg_action_groups.yml
FAIL: Configuration for ‘actiongroups’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][actiongroups], source[n/a, actual length: [2.6kb], max length: 2kb]}] and a refresh]]
FAIL: Expected 1 nodes to return response, but got only 0
Done with failures

``

The yml config is

searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:

CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: [“sg_all_access”]
cluster.name: searchguard_demo
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3

``

ES logs just say:

[ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

``

Java is 1.8.0_151, OS is Debian GNU/Linux 8.7 (jessie).

Thank you

sgadmin_diag_trace_2019-Mar-01_15-34-38.txt (9.21 MB)

searchguard_google_group · March 4, 2019, 11:44pm

Probably the easiest thing you can do is to download a recent ES version (like 6.6.1), install current Search Guard (24.1) with the demo installer and copy over certificates and configuration (selectively) to your 6.0.0 cluster. Even better would be the other way around because 6.0.0 with 17.beta1 is out of date. Make sure you replace all certificates inside the config/ folder and those you use with sgadmin.

In recent versions of Search Guard there is a --accept-red-cluster command line option for sgadmin to deal with "red" clusters.

···

Am 01.03.2019 um 05:41 schrieb Tudor Rogojanu <tudro2001@gmail.com>:

Hello,

I have ES 6.0.0 and SG 6-6.0.0-17.beta1 running on a server for more than a year. It's just a small demo that i play with from time to time, so nobody else has access to it. I had no problems with it for a long time.

I noticed this week that i can't login anymore. I haven't done any changes to ES or SG for more than a year. After some debugging it seemed to be because of the old certificates, so i downloaded the new demo certificates. But when i run
./sgadmin.sh -cd ../sgconfig/ -icl -nhnv -cacert root-ca.pem -cert kirk.pem -key kirk-key.pem
it says
Search Guard Admin v6
Will connect to localhost:9300 ... done
Connected as CN=kirk,OU=client,O=client,L=test,C=de
Contacting elasticsearch cluster 'searchguard_demo' and wait for YELLOW clusterstate ...
ERR: Timed out while waiting for a green or yellow cluster state.

The yml config is
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:
- CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
cluster.name: searchguard_demo
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3

ES logs just say:
[ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

Java is 1.8.0_151, OS is Debian GNU/Linux 8.7 (jessie).

Thank you

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/561908d0-6909-4127-92cd-30a282ed5abb%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

Tudor_Rogojanu · March 5, 2019, 5:50am

Ok, thank you for the info. I hope for an easier solution than this

I’ve aleardy run it with --accept-red-cluster. It's in my first message. Unfortunately, it doesn't work:
primary shard is not active

vineri, 1 martie 2019, 15:41:26 UTC+2, Tudor Rogojanu a scris:

···