Including protected searchguard security & signal indices in snapshots when TLS on REST layer is disabled

Hi,
Continuing the discussion from How to backup and restore .signals_* indices,
it seems the easiest way to use the ES snapshot API to backup and restore protected indices (searchguard signals indices, as well as the Search Guard security index) is to use the TLS admin certificate that you would also use for sgadmin.

Describe the issue:
When searchguard.ssl.http.enabled is disabled, TLS is disabled on the REST layer. Only when TLS client authentication is enabled, REST clients can send a TLS certificate with the HTTP request to provide identity information to the security plugin.

My use-case: I use elasticsearch in k8s environment with istio mtls enabled. With that, as pod-to-pod mtls is handled by istio, and so tls on REST layer for searchguard has been disabled.
With this, I can connect to the REST api with user credentials on http (like http://localhost:9200 -u < uname>:< pwd>), but I cannot use certificate-based authentication.

Now, lets say, I want to periodically backup all data in ES (including these protected indices) using elasticsearch snapshots and restore from it in case of disaster recovery. In the current situation, I can only restore other indices while there would be data loss for these protected indices.

Is there a way to mitigate this issue? Can an admin user in internal_users.yml (having unlimited permissions like “sgs_all_access”) be used to restore .these indices instead of admin-certificates?

Any pointers would be appreciated.
Thanks!

The configuration indices are protected and protected indexes can only be accessed with a certificate. It is not possible right now to backup signals indices without a certificate. We will discuss internally whether we should lift these restrictions and allow other means of authentication.

Hi. We have an option to allow creating and restoring snapshots while REST layer TLS is disabled.

Make sure you have the following configuration options. The elasticsearch user must have RW permissions for “/path/to/snapshots”.
elasticsearch.yml

searchguard.ssl.http.enabled: false
searchguard.unsupported.restore.sgindex.enabled: true
searchguard.enable_snapshot_restore_privilege: true
path.repo: ["/path/to/snaphots"]

Register a snapshot repository.

curl -k \
 -u admin:admin \
 -X PUT "localhost:9200/_snapshot/my_backup?pretty" \
 -H 'Content-Type: application/json' \
 -d'
 {
  "type": "fs",
  "settings": {
   "location": "my_backup_location",
   "compress": true
  }
 }
 '

Create a snapshot.

curl -k \
 -u admin:admin \
 -X PUT "localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true"

Restore the snapshot.

curl -k \
 -u admin:admin \
 -X POST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore" \
 -H 'Content-Type: application/json' \
 -d '
  {
   "ignore_unavailable": true
  }
 '