I have a proposal to implement SearchGaurd in my ElasticSearch cluster. The ES cluster has so many indices currently. After implementation of Search Gaurd I need to enable the shards allocation, but unfortunately iam not able to track the progress of shard allocation for that particular time. Since the process consumes a lot of time , it is not practical to wait till n number of hours to verify the ES cluster health.
My requirement is to track the shard allocation progress before initializing the SearchGarud. Can you help me on this regard? Thanks in advance
You always can use curl with the same certificates you use for sgadmin to allow any operation. Should then look like curl -Ss https://eshost:9200/<api> --cert /path/to/admincert.pem --key /path/to/adminkey.pem --cacert /path/to/root-ca.pem
I have problem with initialize searchgurad in red status.Below is the error when i initialze searchguard
Will update ‘sg/internalusers’ with config/sg/sg_internal_users.yml
FAIL: Configuration for ‘internalusers’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][internalusers], source[n/a, actual length: [5.4kb], max length: 2kb]}] and a refresh]]
Will update ‘sg/actiongroups’ with config/sg/sg_action_groups.yml
FAIL: Configuration for ‘actiongroups’ failed because of UnavailableShardsException[[searchguard][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[searchguard][0]] containing [index {[searchguard][sg][actiongroups], source[n/a, actual length: [2.6kb], max length: 2kb]}] and a refresh]]
FAIL: Expected 4 nodes to return response, but got only 0
Done with failures
Currently I have below error when I initilize searcgurd
Search Guard Admin v6
Will connect to localhost:9300 … done
Elasticsearch Version: 6.6.2
Search Guard Version: 6.6.2-24.3
Connected as CN=sgadmin,OU=Carrefour Operations,O=Carrefour
Contacting elasticsearch cluster ‘elasticsearch’ …
Clustername: c4_correctivo_log
Clusterstate: YELLOW
Number of nodes: 4
Number of data nodes: 3
searchguard index already exists, so we do not need to create one.
INFO: searchguard index state is YELLOW, it seems you miss some replicas
Populate config from /usr/share/elasticsearch/config/sg/
Will update ‘sg/config’ with /usr/share/elasticsearch/config/sg/sg_config.yml
FAIL: Configuration for ‘config’ failed because of ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping) within 30s]
Will update ‘sg/roles’ with /usr/share/elasticsearch/config/sg/sg_roles.yml
FAIL: Configuration for ‘roles’ failed because of ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping) within 30s]
Will update ‘sg/rolesmapping’ with /usr/share/elasticsearch/config/sg/sg_roles_mapping.yml
FAIL: Configuration for ‘rolesmapping’ failed because of ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping) within 30s]
Will update ‘sg/internalusers’ with /usr/share/elasticsearch/config/sg/sg_internal_users.yml
FAIL: Configuration for ‘internalusers’ failed because of ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping) within 30s]
Will update ‘sg/actiongroups’ with /usr/share/elasticsearch/config/sg/sg_action_groups.yml
FAIL: Configuration for ‘actiongroups’ failed because of ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping) within 30s]
Please have a look into the logfiles of your ES nodes. There must be something in there that reveal the reasons why the shards are not available. Are you really sure you have shard allocation enabled? Because the issue you are facing is typical for not having shard allocation enabled. You can run sgadmin with the --diagnose flag (which produces a textfile with additional diagnostic information) and post the diagnose file here.
Pleaso also post the output of curl -Ss "https://eshost:9200/_cluster/allocation/explain?include_disk_info=true" --cert /path/to/admincert.pem --key /path/to/adminkey.pem --cacert /path/to/root-ca.pem (see Cluster Allocation Explain API | Elasticsearch Guide [6.8] | Elastic for more options)
Please post the error message. Which operating system do you use? Linux/Mac/Win?
Normally the diagnose file is written to the current working directory.
At that time I have huge number of shards.Also I’m tried below command unfortunately, Its not working curl -Ss "https://eshost:9200/_cluster/allocation/explain?include_disk_info=true" --cert /path/to/admincert.pem --key /path/to/adminkey.pem --cacert /path/to/root-ca.pem