curator - no permissions for cluster:admin/snapshot/restore

Search Guard
1

Hi,

We are currently evaluating Search Guard to replace Shield.
We still have one blocking point:

We are able to generate some snapshots using curator.
We are able to restore from these snapshots using a CURL request.
But, we are not able to restore using curator. It always fail with the following error:

2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, “restore”
2017-03-01 09:53:38,295 INFO Trying Action ID: 1, “restore”: Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
2017-03-01 09:53:38,515 INFO Restoring indices “[‘.kibana’, ‘searchguard’]” from snapshot: snapshot-20170301090739
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

We are using:

  • ES 2.3.3
  • search-guard-2 2.3.3.11
  • search-guard-ssl 2.3.3.20

We have used example PKI scripts to generate keys & self signed certificate

Using curl to restore is working:
cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem ‘https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty’

But curator fails to do the same:
curator --config /tmp/curator.yml /etc/curator.restore.yml
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

How can I get ride of this error?
What did I miss in curator config?

Here are the curator yml files:

curator.yml:
client:
hosts: [dev-ct-elasticsearch04]
port: 9200
url_prefix:
use_ssl: true
certificate:
client_cert: /tmp/ch.pem
client_key:
aws_key:
aws_secret_key:
aws_region:
ssl_no_validate: true
http_auth: admin:Superuser
timeout:
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator.log
logformat: default
blacklist: [‘elasticsearch’, ‘urllib3’]

curator.restore.yml:
actions:
1:
action: restore
description: >-
Restore all indices
options:
repository: backup_repository
name:
indices:
include_aliases: False
ignore_unavailable: False
include_global_state: True
partial: False
rename_pattern:
rename_replacement:
extra_settings:
wait_for_completion: True
skip_repo_fs_check: False
timeout_override:
continue_if_exception: False
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: snapshot-
exclude:
- filtertype: state
state: SUCCESS
exclude:

Any help will be appreciated.

Thanks in advance,
Patrick

2

You have two options here:

- User curator with ssl client authentication and run it with an admin certificate (like sgadmin) but we had seen problems with this approach: https://github.com/floragunncom/search-guard/issues/196
- We are working currently on this feature (pull requests 245 and 257) and here is preliminary version for ES 2.3.3. which includes PR #245 https://gist.github.com/floragunncom/fe1f5d88837b1e3d96db82a844f75bf7
This approach does not (yet) allow to restore the global state! Do you really need that?

···

Am 01.03.2017 um 11:44 schrieb Patrick Monnoire <pmonnoire@gmail.com>:

Hi,

We are currently evaluating Search Guard to replace Shield.
We still have one blocking point:

We are able to generate some snapshots using curator.
We are able to restore from these snapshots using a CURL request.
But, we are not able to restore using curator. It always fail with the following error:

2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, "restore"
2017-03-01 09:53:38,295 INFO Trying Action ID: 1, "restore": Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
2017-03-01 09:53:38,515 INFO Restoring indices "['.kibana', 'searchguard']" from snapshot: snapshot-20170301090739
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')

We are using:
- ES 2.3.3
- search-guard-2 2.3.3.11
- search-guard-ssl 2.3.3.20

We have used example PKI scripts to generate keys & self signed certificate

Using curl to restore is working:
cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem 'https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty&#39;

But curator fails to do the same:
curator --config /tmp/curator.yml /etc/curator.restore.yml
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')

How can I get ride of this error?
What did I miss in curator config?

Here are the curator yml files:

curator.yml:
client:
  hosts: [dev-ct-elasticsearch04]
  port: 9200
  url_prefix:
  use_ssl: true
  certificate:
  client_cert: /tmp/ch.pem
  client_key:
  aws_key:
  aws_secret_key:
  aws_region:
  ssl_no_validate: true
  http_auth: admin:Superuser
  timeout:
  master_only: False
logging:
  loglevel: INFO
  logfile: /var/log/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

curator.restore.yml:
actions:
  1:
    action: restore
    description: >-
      Restore all indices
    options:
      repository: backup_repository
      name:
      indices:
      include_aliases: False
      ignore_unavailable: False
      include_global_state: True
      partial: False
      rename_pattern:
      rename_replacement:
      extra_settings:
      wait_for_completion: True
      skip_repo_fs_check: False
      timeout_override:
      continue_if_exception: False
      disable_action: false
    filters:
    - filtertype: pattern
      kind: prefix
      value: snapshot-
      exclude:
    - filtertype: state
      state: SUCCESS
      exclude:

Any help will be appreciated.

Thanks in advance,
Patrick

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

3

Hi,

I’ve already tried SSL authentication without any success.
But, never mind.
The fact is that your preliminary version of SG release 12 is solving the issue. I’ve installed it and was able to restore from a snapshot.
Thanks for having shared this preliminary version.

So, one more question: Do you have yet scheduled a delivery date for SG 2 release 12? If not, just a estimate (couple of weeks or months) will be also appreciated.

Thanks a lot,
Patrick

···

Le mercredi 1 mars 2017 14:06:00 UTC+1, Search Guard a écrit :

You have two options here:

This approach does not (yet) allow to restore the global state! Do you really need that?

Am 01.03.2017 um 11:44 schrieb Patrick Monnoire pmon...@gmail.com:

Hi,

We are currently evaluating Search Guard to replace Shield.

We still have one blocking point:

We are able to generate some snapshots using curator.

We are able to restore from these snapshots using a CURL request.

But, we are not able to restore using curator. It always fail with the following error:

2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, “restore”

2017-03-01 09:53:38,295 INFO Trying Action ID: 1, “restore”: Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.

2017-03-01 09:53:38,515 INFO Restoring indices “[‘.kibana’, ‘searchguard’]” from snapshot: snapshot-20170301090739

2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

We are using:

  • ES 2.3.3
  • search-guard-2 2.3.3.11
  • search-guard-ssl 2.3.3.20

We have used example PKI scripts to generate keys & self signed certificate

Using curl to restore is working:

cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem

curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem ‘https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty’

But curator fails to do the same:

curator --config /tmp/curator.yml /etc/curator.restore.yml

2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

How can I get ride of this error?

What did I miss in curator config?

Here are the curator yml files:

curator.yml:

client:

hosts: [dev-ct-elasticsearch04]

port: 9200

url_prefix:

use_ssl: true

certificate:

client_cert: /tmp/ch.pem

client_key:

aws_key:

aws_secret_key:

aws_region:

ssl_no_validate: true

http_auth: admin:Superuser

timeout:

master_only: False

logging:

loglevel: INFO

logfile: /var/log/curator.log

logformat: default

blacklist: [‘elasticsearch’, ‘urllib3’]

curator.restore.yml:

actions:

1:

action: restore

description: >-

  Restore all indices

options:

  repository: backup_repository

  name:

  indices:

  include_aliases: False

  ignore_unavailable: False

  include_global_state: True

  partial: False

  rename_pattern:

  rename_replacement:

  extra_settings:

  wait_for_completion: True

  skip_repo_fs_check: False

  timeout_override:

  continue_if_exception: False

  disable_action: false

filters:

- filtertype: pattern

  kind: prefix

  value: snapshot-

  exclude:

- filtertype: state

  state: SUCCESS

  exclude:

Any help will be appreciated.

Thanks in advance,

Patrick

–
You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

4

2-3 weeks

···

Am 02.03.2017 um 13:43 schrieb Patrick Monnoire <pmonnoire@gmail.com>:

Hi,

I've already tried SSL authentication without any success.
But, never mind.
The fact is that your preliminary version of SG release 12 is solving the issue. I've installed it and was able to restore from a snapshot.
Thanks for having shared this preliminary version.

So, one more question: Do you have yet scheduled a delivery date for SG 2 release 12? If not, just a estimate (couple of weeks or months) will be also appreciated.

Thanks a lot,
Patrick

Le mercredi 1 mars 2017 14:06:00 UTC+1, Search Guard a écrit :
You have two options here:

- User curator with ssl client authentication and run it with an admin certificate (like sgadmin) but we had seen problems with this approach: https://github.com/floragunncom/search-guard/issues/196
- We are working currently on this feature (pull requests 245 and 257) and here is preliminary version for ES 2.3.3. which includes PR #245 https://gist.github.com/floragunncom/fe1f5d88837b1e3d96db82a844f75bf7
This approach does not (yet) allow to restore the global state! Do you really need that?

> Am 01.03.2017 um 11:44 schrieb Patrick Monnoire <pmon...@gmail.com>:
>
>
> Hi,
>
> We are currently evaluating Search Guard to replace Shield.
> We still have one blocking point:
>
> We are able to generate some snapshots using curator.
> We are able to restore from these snapshots using a CURL request.
> But, we are not able to restore using curator. It always fail with the following error:
>
>
> 2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, "restore"
> 2017-03-01 09:53:38,295 INFO Trying Action ID: 1, "restore": Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
> 2017-03-01 09:53:38,515 INFO Restoring indices "['.kibana', 'searchguard']" from snapshot: snapshot-20170301090739
> 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
>
>
> We are using:
> - ES 2.3.3
> - search-guard-2 2.3.3.11
> - search-guard-ssl 2.3.3.20
>
> We have used example PKI scripts to generate keys & self signed certificate
>
> Using curl to restore is working:
> cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
> curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem 'https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty&#39;
>
> But curator fails to do the same:
> curator --config /tmp/curator.yml /etc/curator.restore.yml
> 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
>
> How can I get ride of this error?
> What did I miss in curator config?
>
> Here are the curator yml files:
>
> curator.yml:
> client:
> hosts: [dev-ct-elasticsearch04]
> port: 9200
> url_prefix:
> use_ssl: true
> certificate:
> client_cert: /tmp/ch.pem
> client_key:
> aws_key:
> aws_secret_key:
> aws_region:
> ssl_no_validate: true
> http_auth: admin:Superuser
> timeout:
> master_only: False
> logging:
> loglevel: INFO
> logfile: /var/log/curator.log
> logformat: default
> blacklist: ['elasticsearch', 'urllib3']
>
> curator.restore.yml:
> actions:
> 1:
> action: restore
> description: >-
> Restore all indices
> options:
> repository: backup_repository
> name:
> indices:
> include_aliases: False
> ignore_unavailable: False
> include_global_state: True
> partial: False
> rename_pattern:
> rename_replacement:
> extra_settings:
> wait_for_completion: True
> skip_repo_fs_check: False
> timeout_override:
> continue_if_exception: False
> disable_action: false
> filters:
> - filtertype: pattern
> kind: prefix
> value: snapshot-
> exclude:
> - filtertype: state
> state: SUCCESS
> exclude:
>
>
> Any help will be appreciated.
>
> Thanks in advance,
> Patrick
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com\.
> For more options, visit https://groups.google.com/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9ce0fcf8-f40d-45ea-b88b-2c8cb91b6230%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

5

Release is scheduled for Tue 18th of April

···

Am 02.03.2017 um 13:43 schrieb Patrick Monnoire <pmonnoire@gmail.com>:

Hi,

I've already tried SSL authentication without any success.
But, never mind.
The fact is that your preliminary version of SG release 12 is solving the issue. I've installed it and was able to restore from a snapshot.
Thanks for having shared this preliminary version.

So, one more question: Do you have yet scheduled a delivery date for SG 2 release 12? If not, just a estimate (couple of weeks or months) will be also appreciated.

Thanks a lot,
Patrick

Le mercredi 1 mars 2017 14:06:00 UTC+1, Search Guard a écrit :
You have two options here:

- User curator with ssl client authentication and run it with an admin certificate (like sgadmin) but we had seen problems with this approach: https://github.com/floragunncom/search-guard/issues/196
- We are working currently on this feature (pull requests 245 and 257) and here is preliminary version for ES 2.3.3. which includes PR #245 https://gist.github.com/floragunncom/fe1f5d88837b1e3d96db82a844f75bf7
This approach does not (yet) allow to restore the global state! Do you really need that?

> Am 01.03.2017 um 11:44 schrieb Patrick Monnoire <pmon...@gmail.com>:
>
>
> Hi,
>
> We are currently evaluating Search Guard to replace Shield.
> We still have one blocking point:
>
> We are able to generate some snapshots using curator.
> We are able to restore from these snapshots using a CURL request.
> But, we are not able to restore using curator. It always fail with the following error:
>
>
> 2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, "restore"
> 2017-03-01 09:53:38,295 INFO Trying Action ID: 1, "restore": Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
> 2017-03-01 09:53:38,515 INFO Restoring indices "['.kibana', 'searchguard']" from snapshot: snapshot-20170301090739
> 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
>
>
> We are using:
> - ES 2.3.3
> - search-guard-2 2.3.3.11
> - search-guard-ssl 2.3.3.20
>
> We have used example PKI scripts to generate keys & self signed certificate
>
> Using curl to restore is working:
> cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
> curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem 'https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty&#39;
>
> But curator fails to do the same:
> curator --config /tmp/curator.yml /etc/curator.restore.yml
> 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
>
> How can I get ride of this error?
> What did I miss in curator config?
>
> Here are the curator yml files:
>
> curator.yml:
> client:
> hosts: [dev-ct-elasticsearch04]
> port: 9200
> url_prefix:
> use_ssl: true
> certificate:
> client_cert: /tmp/ch.pem
> client_key:
> aws_key:
> aws_secret_key:
> aws_region:
> ssl_no_validate: true
> http_auth: admin:Superuser
> timeout:
> master_only: False
> logging:
> loglevel: INFO
> logfile: /var/log/curator.log
> logformat: default
> blacklist: ['elasticsearch', 'urllib3']
>
> curator.restore.yml:
> actions:
> 1:
> action: restore
> description: >-
> Restore all indices
> options:
> repository: backup_repository
> name:
> indices:
> include_aliases: False
> ignore_unavailable: False
> include_global_state: True
> partial: False
> rename_pattern:
> rename_replacement:
> extra_settings:
> wait_for_completion: True
> skip_repo_fs_check: False
> timeout_override:
> continue_if_exception: False
> disable_action: false
> filters:
> - filtertype: pattern
> kind: prefix
> value: snapshot-
> exclude:
> - filtertype: state
> state: SUCCESS
> exclude:
>
>
> Any help will be appreciated.
>
> Thanks in advance,
> Patrick
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com\.
> For more options, visit https://groups.google.com/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9ce0fcf8-f40d-45ea-b88b-2c8cb91b6230%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

6

Hi,

Even if preliminary version of SG release 12 is solving the issue, I’m still facing the same issue using the official release.

I’ve configured my group like this:

sg_admin:
cluster:
- cluster:admin/snapshot/restore
- ‘’
indices:
'’:
‘':
- '’

Then I’ve added these 2 lines in /etc/elasticsearch/elasticsearch.yml

searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: false

My snapshot does not contain the searchguard index
and finaly, when calling currator to restore, I set

include_global_state: False

But I still get this error:

[2017-05-05 09:34:22,783][WARN ][com.floragunn.searchguard.configuration.PrivilegesEvaluator] cluster:admin/snapshot/restore is not allowed for a regular user

What am I doing wrong?

Regards,
Patrick

···

Le samedi 15 avril 2017 22:51:58 UTC+2, Search Guard a écrit :

Release is scheduled for Tue 18th of April

Am 02.03.2017 um 13:43 schrieb Patrick Monnoire pmon...@gmail.com:

Hi,

I’ve already tried SSL authentication without any success.

But, never mind.

The fact is that your preliminary version of SG release 12 is solving the issue. I’ve installed it and was able to restore from a snapshot.

Thanks for having shared this preliminary version.

So, one more question: Do you have yet scheduled a delivery date for SG 2 release 12? If not, just a estimate (couple of weeks or months) will be also appreciated.

Thanks a lot,

Patrick

Le mercredi 1 mars 2017 14:06:00 UTC+1, Search Guard a écrit :

You have two options here:

This approach does not (yet) allow to restore the global state! Do you really need that?

Am 01.03.2017 um 11:44 schrieb Patrick Monnoire pmon...@gmail.com:

Hi,

We are currently evaluating Search Guard to replace Shield.
We still have one blocking point:

We are able to generate some snapshots using curator.
We are able to restore from these snapshots using a CURL request.
But, we are not able to restore using curator. It always fail with the following error:

2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, “restore”
2017-03-01 09:53:38,295 INFO Trying Action ID: 1, “restore”: Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
2017-03-01 09:53:38,515 INFO Restoring indices “[‘.kibana’, ‘searchguard’]” from snapshot: snapshot-20170301090739
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

We are using:

  • ES 2.3.3
  • search-guard-2 2.3.3.11
  • search-guard-ssl 2.3.3.20

We have used example PKI scripts to generate keys & self signed certificate

Using curl to restore is working:
cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem ‘https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty’

But curator fails to do the same:
curator --config /tmp/curator.yml /etc/curator.restore.yml
2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class ‘curator.exceptions.FailedExecution’>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, ‘security_exception’, ‘no permissions for cluster:admin/snapshot/restore’)

How can I get ride of this error?
What did I miss in curator config?

Here are the curator yml files:

curator.yml:
client:
hosts: [dev-ct-elasticsearch04]
port: 9200
url_prefix:
use_ssl: true
certificate:
client_cert: /tmp/ch.pem
client_key:
aws_key:
aws_secret_key:
aws_region:
ssl_no_validate: true
http_auth: admin:Superuser
timeout:
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator.log
logformat: default
blacklist: [‘elasticsearch’, ‘urllib3’]

curator.restore.yml:
actions:
1:
action: restore
description: >-
Restore all indices
options:
repository: backup_repository
name:
indices:
include_aliases: False
ignore_unavailable: False
include_global_state: True
partial: False
rename_pattern:
rename_replacement:
extra_settings:
wait_for_completion: True
skip_repo_fs_check: False
timeout_override:
continue_if_exception: False
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: snapshot-
exclude:
- filtertype: state
state: SUCCESS
exclude:

Any help will be appreciated.

Thanks in advance,
Patrick

–
You received this message because you are subscribed to the Google Groups “Search Guard” group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
To post to this group, send email to search...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to the Google Groups “Search Guard” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9ce0fcf8-f40d-45ea-b88b-2c8cb91b6230%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

7

Did you set "searchguard.enable_snapshot_restore_privilege: true" in elasticsearch on *all* nodes and are you sure you running SG 12 on all nodes too?

···

Am 05.05.2017 um 11:45 schrieb Patrick Monnoire <pmonnoire@gmail.com>:

Hi,

Even if preliminary version of SG release 12 is solving the issue, I'm still facing the same issue using the official release.

I've configured my group like this:

sg_admin:
  cluster:
    - cluster:admin/snapshot/restore
    - '*'
  indices:
    '*':
      '*':
        - '*'

Then I've added these 2 lines in /etc/elasticsearch/elasticsearch.yml

searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: false

My snapshot does not contain the searchguard index
and finaly, when calling currator to restore, I set

include_global_state: False

But I still get this error:

[2017-05-05 09:34:22,783][WARN ][com.floragunn.searchguard.configuration.PrivilegesEvaluator] cluster:admin/snapshot/restore is not allowed for a regular user

What am I doing wrong?

Regards,
Patrick

Le samedi 15 avril 2017 22:51:58 UTC+2, Search Guard a écrit :
Release is scheduled for Tue 18th of April

> Am 02.03.2017 um 13:43 schrieb Patrick Monnoire <pmon...@gmail.com>:
>
> Hi,
>
> I've already tried SSL authentication without any success.
> But, never mind.
> The fact is that your preliminary version of SG release 12 is solving the issue. I've installed it and was able to restore from a snapshot.
> Thanks for having shared this preliminary version.
>
> So, one more question: Do you have yet scheduled a delivery date for SG 2 release 12? If not, just a estimate (couple of weeks or months) will be also appreciated.
>
>
> Thanks a lot,
> Patrick
>
> Le mercredi 1 mars 2017 14:06:00 UTC+1, Search Guard a écrit :
> You have two options here:
>
> - User curator with ssl client authentication and run it with an admin certificate (like sgadmin) but we had seen problems with this approach: https://github.com/floragunncom/search-guard/issues/196
> - We are working currently on this feature (pull requests 245 and 257) and here is preliminary version for ES 2.3.3. which includes PR #245 https://gist.github.com/floragunncom/fe1f5d88837b1e3d96db82a844f75bf7
> This approach does not (yet) allow to restore the global state! Do you really need that?
>
> > Am 01.03.2017 um 11:44 schrieb Patrick Monnoire <pmon...@gmail.com>:
> >
> >
> > Hi,
> >
> > We are currently evaluating Search Guard to replace Shield.
> > We still have one blocking point:
> >
> > We are able to generate some snapshots using curator.
> > We are able to restore from these snapshots using a CURL request.
> > But, we are not able to restore using curator. It always fail with the following error:
> >
> >
> > 2017-03-01 09:53:38,206 INFO Preparing Action ID: 1, "restore"
> > 2017-03-01 09:53:38,295 INFO Trying Action ID: 1, "restore": Restore all indices in the most recent snapshot-* snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore.
> > 2017-03-01 09:53:38,515 INFO Restoring indices "['.kibana', 'searchguard']" from snapshot: snapshot-20170301090739
> > 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
> >
> >
> > We are using:
> > - ES 2.3.3
> > - search-guard-2 2.3.3.11
> > - search-guard-ssl 2.3.3.20
> >
> > We have used example PKI scripts to generate keys & self signed certificate
> >
> > Using curl to restore is working:
> > cat /data/elasticsearch/cert/admin.crt.pem /data/elasticsearch/cert/ca/chain-ca.pem > /tmp/ch.pem
> > curl -Ss --insecure -XPOST -E /tmp/ch.pem --key /data/elasticsearch/cert/admin.key.pem 'https://dev-ct-elasticsearch01:9200/_snapshot/backup_repository/snapshot-20170223104445/_restore?pretty&#39;
> >
> > But curator fails to do the same:
> > curator --config /tmp/curator.yml /etc/curator.restore.yml
> > 2017-03-01 09:53:38,532 ERROR Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'security_exception', 'no permissions for cluster:admin/snapshot/restore')
> >
> > How can I get ride of this error?
> > What did I miss in curator config?
> >
> > Here are the curator yml files:
> >
> > curator.yml:
> > client:
> > hosts: [dev-ct-elasticsearch04]
> > port: 9200
> > url_prefix:
> > use_ssl: true
> > certificate:
> > client_cert: /tmp/ch.pem
> > client_key:
> > aws_key:
> > aws_secret_key:
> > aws_region:
> > ssl_no_validate: true
> > http_auth: admin:Superuser
> > timeout:
> > master_only: False
> > logging:
> > loglevel: INFO
> > logfile: /var/log/curator.log
> > logformat: default
> > blacklist: ['elasticsearch', 'urllib3']
> >
> > curator.restore.yml:
> > actions:
> > 1:
> > action: restore
> > description: >-
> > Restore all indices
> > options:
> > repository: backup_repository
> > name:
> > indices:
> > include_aliases: False
> > ignore_unavailable: False
> > include_global_state: True
> > partial: False
> > rename_pattern:
> > rename_replacement:
> > extra_settings:
> > wait_for_completion: True
> > skip_repo_fs_check: False
> > timeout_override:
> > continue_if_exception: False
> > disable_action: false
> > filters:
> > - filtertype: pattern
> > kind: prefix
> > value: snapshot-
> > exclude:
> > - filtertype: state
> > state: SUCCESS
> > exclude:
> >
> >
> > Any help will be appreciated.
> >
> > Thanks in advance,
> > Patrick
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Search Guard" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > To post to this group, send email to search...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9238b352-3c53-4e92-ad16-67f6f4f0f007%40googlegroups.com\.
> > For more options, visit https://groups.google.com/d/optout\.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/9ce0fcf8-f40d-45ea-b88b-2c8cb91b6230%40googlegroups.com\.
> For more options, visit https://groups.google.com/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/385341ea-8a60-49ed-9fd4-eb98c4c7feb4%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

8

GET _cat/plugins?v
name component version type url
ci-ct-es02-778 search-guard-2 2.4.4.12 j
ci-ct-es02-778 search-guard-ssl 2.4.4.21 j
ci-ct-es03-778 search-guard-2 2.4.4.12 j
ci-ct-es03-778 search-guard-ssl 2.4.4.21 j
ci-ct-es04-778 search-guard-2 2.4.4.12 j
ci-ct-es04-778 search-guard-ssl 2.4.4.21 j
ci-ct-es01-778 search-guard-2 2.4.4.12 j
ci-ct-es01-778 search-guard-ssl 2.4.4.21 j

And on all 4 nodes:

searchguard.ssl.transport.keystore_filepath: dev-ct-elasticsearch03-keystore.jks
searchguard.ssl.transport.keystore_password: xxxxx
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: xxxx
searchguard.ssl.transport.enforce_hostname_verification: true
searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: dev-ct-elasticsearch03-keystore.jks
searchguard.ssl.http.keystore_password: xxxx
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_password: xxxx

searchguard.authcz.admin_dn:

  • cn=admin,ou=client,o=client,l=Test,c=DE

searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: false

Regards,
Patrick

9

Ah sorry, this feature is available only for ES >= 5.0.0 (and therefore SG >= 5)

See GitHub - floragunncom/search-guard: Search Guard Plugin - Security for Elasticsearch

Any chance to upgrade?

···

Am 05.05.2017 um 13:56 schrieb Patrick Monnoire <pmonnoire@gmail.com>:

GET _cat/plugins?v
name component version type url
ci-ct-es02-778 search-guard-2 2.4.4.12 j
ci-ct-es02-778 search-guard-ssl 2.4.4.21 j
ci-ct-es03-778 search-guard-2 2.4.4.12 j
ci-ct-es03-778 search-guard-ssl 2.4.4.21 j
ci-ct-es04-778 search-guard-2 2.4.4.12 j
ci-ct-es04-778 search-guard-ssl 2.4.4.21 j
ci-ct-es01-778 search-guard-2 2.4.4.12 j
ci-ct-es01-778 search-guard-ssl 2.4.4.21 j

And on all 4 nodes:

searchguard.ssl.transport.keystore_filepath: dev-ct-elasticsearch03-keystore.jks
searchguard.ssl.transport.keystore_password: xxxxx
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: xxxx
searchguard.ssl.transport.enforce_hostname_verification: true
searchguard.ssl.transport.resolve_hostname: false

searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: dev-ct-elasticsearch03-keystore.jks
searchguard.ssl.http.keystore_password: xxxx
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_password: xxxx

searchguard.authcz.admin_dn:
  - cn=admin,ou=client,o=client,l=Test,c=DE

searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: false

Regards,
Patrick

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/150df126-c258-484c-ac07-b85f0951096b%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

10

:frowning: :frowning: :frowning:

But it was working with the 2.3.3.12 pre-release you sent me for test purpose.
Any chance to have this feature for ES < 5.0?

For sure we will migrate to ES 5.x
But it is still no yet scheduled.

11

Hi Patrick,

we just decided that we will backport the functionality for ES 2.4.5 but not for ES 2.3.x.

Can you upgrade to ES 2.4.5?

ES 2.3 has EOL in September 2017 (End-of-Life-Termine für Elastic-Produkte | Elastic) and yesterday the first

Alpha of Elasticsearch 6 was announced. So i think it would make really sense to upgrade at least to 2.4.5 (or even better to 5.4).

···

On Friday, 5 May 2017 14:37:38 UTC+2, Patrick Monnoire wrote:

:frowning: :frowning: :frowning:

But it was working with the 2.3.3.12 pre-release you sent me for test purpose.
Any chance to have this feature for ES < 5.0?

For sure we will migrate to ES 5.x
But it is still no yet scheduled.

12

Hi,

We are currently using ES 2.4.4.
I think it will not be a big deal to migrate to 2.4.5

thanks for the backport to ES 2.4.5,
Patrick

13

do you need “searchguard.check_snapshot_restore_write_privileges” functionality or is “searchguard.enable_snapshot_restore_privilege” sufficient?

···

On Wednesday, 10 May 2017 10:27:02 UTC+2, Patrick Monnoire wrote:

Hi,

We are currently using ES 2.4.4.
I think it will not be a big deal to migrate to 2.4.5

thanks for the backport to ES 2.4.5,
Patrick

14

Hi,

searchguard.enable_snapshot_restore_privilege is enough because our backups do not contain the searchguard index.

Thanks,
Patrick

15

here is the release candidate: https://oss.sonatype.org/content/repositories/snapshots/com/floragunn/search-guard-2/2.4.5.13-SNAPSHOT/search-guard-2-2.4.5.13-20170518.181116-1.zip

Release is planned for Sunday.

···

On Thursday, 18 May 2017 17:41:24 UTC+2, Patrick Monnoire wrote:

Hi,

searchguard.enable_snapshot_restore_privilege is enough because our backups do not contain the searchguard index.

Thanks,
Patrick

16

Officially released today Redirecting to Google Groups

···

Am 18.05.2017 um 20:35 schrieb Search Guard <info@search-guard.com>:

here is the release candidate: https://oss.sonatype.org/content/repositories/snapshots/com/floragunn/search-guard-2/2.4.5.13-SNAPSHOT/search-guard-2-2.4.5.13-20170518.181116-1.zip

Release is planned for Sunday.

On Thursday, 18 May 2017 17:41:24 UTC+2, Patrick Monnoire wrote:
Hi,

searchguard.enable_snapshot_restore_privilege is enough because our backups do not contain the searchguard index.

Thanks,
Patrick

--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8ed3d00b-7be7-4952-aacc-f5dde2dd5bfc%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.