Unexpected exception cluster:admin/snapshot/restore

[Adrien_Dessemond]

Hi,

I restored several indices on an blank new elasticsearch cluster (5.6.4) and everything is perfectly normal whenever we restore our indices without searchguard being enabled. However, as soon as searchguard (5.6.4-18) is enabled, any attempted index restore fails:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
}
],
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
},
“status”: 500
}

And two exceptions are logged :

[ERROR][c.f.s.f.SearchGuardFilter] Unexpected exception RepositoryMissingException[[myrepository] missing]
org.elasticsearch.repositories.RepositoryMissingException: [myrepository] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:334) ~[elasticsearch-5.6.4.jar:5.6.4]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluateSnapshotRestore(PrivilegesEvaluator.java:839) ~[?:?]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:351) ~[?:?]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:131) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
(…)

[2018-04-09T15:32:04,311][WARN ][r.suppressed ] path: /_snapshot/myrepository/mysnapshot/_restore, params: {repository=myrepository, snapshot=mysnapshot}
org.elasticsearch.ElasticsearchSecurityException: Unexpected exception cluster:admin/snapshot/restore
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:149) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

The user account I use is associated with the role “sg_all_access” which is defined as being:

sg_all_access:
cluster:
- ‘’
indices:
'’:
‘':
- '’

Also, I put this statement in elasticsearch.yml:

searchguard.enable_snapshot_restore_privilege: true

I tried to de-registrer and re-register “myrepository” once searchguard has been enabled but nothing changes but I still have the exception when a restore is attempted. . Aside of this issue, everything works as intended and I am able to see what myrepository contains.

Any clue about what could go on here or should I open a bug report?

[jkressin]

When you restore the index, do you exclude the global state? Something like:

{

“indices”: “…”,

“ignore_unavailable”: true,

“include_global_state”: false

}

``

A bit more about that here: Authorization for snapshot and restore in Search Guard | Security for Elasticsearch | Search Guard

If this does not help, yes, please file an issue, we have not had this kind of problem before and will look into it.

···

On Monday, April 9, 2018 at 6:17:52 PM UTC+2, Adrien Dessemond wrote:

Hi,

I restored several indices on an blank new elasticsearch cluster (5.6.4) and everything is perfectly normal whenever we restore our indices without searchguard being enabled. However, as soon as searchguard (5.6.4-18) is enabled, any attempted index restore fails:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
}
],
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
},
“status”: 500
}

And two exceptions are logged :

[ERROR][c.f.s.f.SearchGuardFilter] Unexpected exception RepositoryMissingException[[myrepository] missing]
org.elasticsearch.repositories.RepositoryMissingException: [myrepository] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:334) ~[elasticsearch-5.6.4.jar:5.6.4]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluateSnapshotRestore(PrivilegesEvaluator.java:839) ~[?:?]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:351) ~[?:?]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:131) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
(…)

[2018-04-09T15:32:04,311][WARN ][r.suppressed ] path: /_snapshot/myrepository/mysnapshot/_restore, params: {repository=myrepository, snapshot=mysnapshot}
org.elasticsearch.ElasticsearchSecurityException: Unexpected exception cluster:admin/snapshot/restore
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:149) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

The user account I use is associated with the role “sg_all_access” which is defined as being:

sg_all_access:
cluster:
- ‘’
indices:
'’:
‘':
- '’

Also, I put this statement in elasticsearch.yml:

searchguard.enable_snapshot_restore_privilege: true

I tried to de-registrer and re-register “myrepository” once searchguard has been enabled but nothing changes but I still have the exception when a restore is attempted. . Aside of this issue, everything works as intended and I am able to see what myrepository contains.

Any clue about what could go on here or should I open a bug report?

[Adrien_Dessemond]

Yes I exclude the global state when I try to restore. I will open an issue.

···

When you restore the index, do you exclude the global state? Something like:

{

“indices”: “…”,

“ignore_unavailable”: true,

“include_global_state”: false

}

``

A bit more about that here: https://docs.search-guard.com/latest/snapshot-restore#restoring-a-snapshot

If this does not help, yes, please file an issue, we have not had this kind of problem before and will look into it.

On Monday, April 9, 2018 at 6:17:52 PM UTC+2, Adrien Dessemond wrote:

Hi,

I restored several indices on an blank new elasticsearch cluster (5.6.4) and everything is perfectly normal whenever we restore our indices without searchguard being enabled. However, as soon as searchguard (5.6.4-18) is enabled, any attempted index restore fails:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
}
],
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
},
“status”: 500
}

And two exceptions are logged :

[ERROR][c.f.s.f.SearchGuardFilter] Unexpected exception RepositoryMissingException[[myrepository] missing]
org.elasticsearch.repositories.RepositoryMissingException: [myrepository] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:334) ~[elasticsearch-5.6.4.jar:5.6.4]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluateSnapshotRestore(PrivilegesEvaluator.java:839) ~[?:?]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:351) ~[?:?]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:131) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
(…)

[2018-04-09T15:32:04,311][WARN ][r.suppressed ] path: /_snapshot/myrepository/mysnapshot/_restore, params: {repository=myrepository, snapshot=mysnapshot}
org.elasticsearch.ElasticsearchSecurityException: Unexpected exception cluster:admin/snapshot/restore
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:149) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

The user account I use is associated with the role “sg_all_access” which is defined as being:

sg_all_access:
cluster:
- ‘’
indices:
'’:
‘':
- '’

Also, I put this statement in elasticsearch.yml:

searchguard.enable_snapshot_restore_privilege: true

I tried to de-registrer and re-register “myrepository” once searchguard has been enabled but nothing changes but I still have the exception when a restore is attempted. . Aside of this issue, everything works as intended and I am able to see what myrepository contains.

Any clue about what could go on here or should I open a bug report?

[jkressin]

Thanks! Besides your configuration files, please also add the exact steps you are doing so we can reproduce the behavior. It’s working across all integration tests, so might be something “special” you do that’s not covered yet.

···

On Wednesday, April 11, 2018 at 9:45:41 PM UTC+2, Adrien Dessemond wrote:

Yes I exclude the global state when I try to restore. I will open an issue.

When you restore the index, do you exclude the global state? Something like:

{

“indices”: “…”,

“ignore_unavailable”: true,

“include_global_state”: false

}

``

A bit more about that here: https://docs.search-guard.com/latest/snapshot-restore#restoring-a-snapshot

If this does not help, yes, please file an issue, we have not had this kind of problem before and will look into it.

On Monday, April 9, 2018 at 6:17:52 PM UTC+2, Adrien Dessemond wrote:

Hi,

I restored several indices on an blank new elasticsearch cluster (5.6.4) and everything is perfectly normal whenever we restore our indices without searchguard being enabled. However, as soon as searchguard (5.6.4-18) is enabled, any attempted index restore fails:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
}
],
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
},
“status”: 500
}

And two exceptions are logged :

[ERROR][c.f.s.f.SearchGuardFilter] Unexpected exception RepositoryMissingException[[myrepository] missing]
org.elasticsearch.repositories.RepositoryMissingException: [myrepository] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:334) ~[elasticsearch-5.6.4.jar:5.6.4]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluateSnapshotRestore(PrivilegesEvaluator.java:839) ~[?:?]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:351) ~[?:?]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:131) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
(…)

[2018-04-09T15:32:04,311][WARN ][r.suppressed ] path: /_snapshot/myrepository/mysnapshot/_restore, params: {repository=myrepository, snapshot=mysnapshot}
org.elasticsearch.ElasticsearchSecurityException: Unexpected exception cluster:admin/snapshot/restore
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:149) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

The user account I use is associated with the role “sg_all_access” which is defined as being:

sg_all_access:
cluster:
- ‘’
indices:
'’:
‘':
- '’

Also, I put this statement in elasticsearch.yml:

searchguard.enable_snapshot_restore_privilege: true

I tried to de-registrer and re-register “myrepository” once searchguard has been enabled but nothing changes but I still have the exception when a restore is attempted. . Aside of this issue, everything works as intended and I am able to see what myrepository contains.

Any clue about what could go on here or should I open a bug report?

[Adrien_Dessemond]

Reported in issue #476 => https://github.com/floragunncom/search-guard/issues/476

···

Le mercredi 11 avril 2018 16:07:19 UTC-4, Jochen Kressin a écrit :

Thanks! Besides your configuration files, please also add the exact steps you are doing so we can reproduce the behavior. It’s working across all integration tests, so might be something “special” you do that’s not covered yet.

On Wednesday, April 11, 2018 at 9:45:41 PM UTC+2, Adrien Dessemond wrote:

Yes I exclude the global state when I try to restore. I will open an issue.

When you restore the index, do you exclude the global state? Something like:

{

“indices”: “…”,

“ignore_unavailable”: true,

“include_global_state”: false

}

``

A bit more about that here: https://docs.search-guard.com/latest/snapshot-restore#restoring-a-snapshot

If this does not help, yes, please file an issue, we have not had this kind of problem before and will look into it.

On Monday, April 9, 2018 at 6:17:52 PM UTC+2, Adrien Dessemond wrote:

Hi,

I restored several indices on an blank new elasticsearch cluster (5.6.4) and everything is perfectly normal whenever we restore our indices without searchguard being enabled. However, as soon as searchguard (5.6.4-18) is enabled, any attempted index restore fails:

{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
}
],
“type”: “security_exception”,
“reason”: “Unexpected exception cluster:admin/snapshot/restore”
},
“status”: 500
}

And two exceptions are logged :

[ERROR][c.f.s.f.SearchGuardFilter] Unexpected exception RepositoryMissingException[[myrepository] missing]
org.elasticsearch.repositories.RepositoryMissingException: [myrepository] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:334) ~[elasticsearch-5.6.4.jar:5.6.4]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluateSnapshotRestore(PrivilegesEvaluator.java:839) ~[?:?]
at com.floragunn.searchguard.configuration.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:351) ~[?:?]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:131) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
(…)

[2018-04-09T15:32:04,311][WARN ][r.suppressed ] path: /_snapshot/myrepository/mysnapshot/_restore, params: {repository=myrepository, snapshot=mysnapshot}
org.elasticsearch.ElasticsearchSecurityException: Unexpected exception cluster:admin/snapshot/restore
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:149) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

The user account I use is associated with the role “sg_all_access” which is defined as being:

sg_all_access:
cluster:
- ‘’
indices:
'’:
‘':
- '’

Also, I put this statement in elasticsearch.yml:

searchguard.enable_snapshot_restore_privilege: true

I tried to de-registrer and re-register “myrepository” once searchguard has been enabled but nothing changes but I still have the exception when a restore is attempted. . Aside of this issue, everything works as intended and I am able to see what myrepository contains.

Any clue about what could go on here or should I open a bug report?