Failed to create new users and roles

[hossein]

Hi,
I’m using Elasticsearch 6.7 with search guard. Everything was good until the Elasticsearch hard disk reaches to 90% limitation. Elasticsearch goes to block writing in all indices. Using the following command I try to re-enable writing:

PUT _all/_settings
{
“index.blocks.read_only_allow_delete”: null
}

However, this command does not work since the following error:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “no permissions for and User [name=myuser, roles=[admin], requestedTenant=null]”
}
],
“type”: “security_exception”,
“reason”: “no permissions for and User [name=myuser, roles=[admin], requestedTenant=null]”
},
“status”: 403
}

I found that I can enable all my indices. For example, I re-enable ‘myindex’ using the following command:

PUT myindex/_settings
{
“index.blocks.read_only_allow_delete”: null
}

I thought my problem solved. However, when I wanted to add an user I got this error:

Number of nodes: 1
Number of data nodes: 1
searchguard index already exists, so we do not need to create one.
Populate config from /usr/share/elasticsearch/plugins/search-guard-6/sgconfig
Will update ‘sg/config’ with …/sgconfig/sg_config.yml
FAIL: Configuration for ‘config’ failed because of ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
Will update ‘sg/roles’ with …/sgconfig/sg_roles.yml
FAIL: Configuration for ‘roles’ failed because of ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
Will update ‘sg/rolesmapping’ with …/sgconfig/sg_roles_mapping.yml
FAIL: Configuration for ‘rolesmapping’ failed because of ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
Will update ‘sg/internalusers’ with …/sgconfig/sg_internal_users.yml
FAIL: Configuration for ‘internalusers’ failed because of ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
Will update ‘sg/actiongroups’ with …/sgconfig/sg_action_groups.yml
FAIL: Configuration for ‘actiongroups’ failed because of ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
Done with failures

All in all, I couldn’t add users or roles. I’m sure my command is correct since I’ve used it before many times.
Would you please tell me how I can set “index.blocks.read_only_allow_delete”: null for ‘_all’ indices?

Thanks

[jkressin]

The easiest way to get complete access over the cluster, including the internal searchguard configuration index is to use the admin certificate. This is the one that you use when executing sgadmin. You can for example use curl with the admin certificate like:

curl -k \
  --cert "/path/to/admin-cert.pem" \
  --key "/path/to/admin-key.pem"  \
  -XPUT "https://sgssl-0.example.com:9200/_all/_settings" \
  -H 'Content-Type: application/json' \
  -d '{ "index.blocks.read_only_allow_delete": null }'

(This is using curl compiled with OpenSSL. Depending on your curl version the command line switches for the certificate and the key might vary).

[hsaly]

It makes no sense to remove the block when disk is nearly full! You need either to delete data or increase the disk size (assuming this is a production system).