Querying missing index causes security exception

[faxmodem]

We use grafana and in some cases users get a security exception no permissions when at least one of the requested indices does not exist, e.g. when there’s a one day gap when requesting a week.

How can we avoid that?

[jkressin]

Do you know what the underlying query looks like? Is it a wildcard query, or simply a query with, say, 3 indices where one does not exist?

And what is the setting of the following property in sg_config:

searchguard:
  dynamic:
    kibana:
      do_not_fail_on_forbidden: true|false

For Kibana (and Grafana for that matter) it should be set to true:

Security for Elasticsearch | Kibana Plugin | Search Guard

[faxmodem]

Hi Jochen,

I tried that setting, wondering why it was in the kibana section: will it only affect the kibana user? Anyway, it didn’t help solving my issue which involves queries against multiple explicitly requested indices like foo-2019.06.14,foo-2019.06.15,foo-2019.06.16 when the middle one is missing for instance.

[jkressin]

The setting is effective not only for Kibana, but for the whole ES cluster. In SG7 the setting is not listed under Kibana anymore to make that more clear.

However, I was not able to reproduce the behavior, so let me explain what I am doing:

I create two indices:

auditlog-2019.06.17
auditlog-2019.06.19

And I have a user that has CRUD access to:

auditlog-*

Now I am querying the indices, and one index (auditlog-2019.06.18) does not exist:

curl -k -u auditlog:auditlog -XGET "https://<es host>:9200/auditlog-2019.06.17,auditlog-2019.06.18,auditlog-2019.06.19/_search?pretty"

The dnfof flag is set to true. I’m getting:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [auditlog-2019.06.18]",
        "resource.type" : "index_or_alias",
        "resource.id" : "auditlog-2019.06.18",
        "index_uuid" : "_na_",
        "index" : "auditlog-2019.06.18"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [auditlog-2019.06.18]",
    "resource.type" : "index_or_alias",
    "resource.id" : "auditlog-2019.06.18",
    "index_uuid" : "_na_",
    "index" : "auditlog-2019.06.18"
  },
  "status" : 404
}

Which IMHO is correct. Does that resemble your scenario? Also, which exact version of ES and SG are you using at the moment? Maybe you hit an issue that was fixed on one the latest versions?

[faxmodem]

Thanks for your answer!
I found the problem: I was hitting aliases, not indices. This uncovers an interesting situation:
The target user was issuing searches against foo-alias, which was an alias to foo-index.
However, he only had permissions on the index, not the alias:

sg_role_foo:
  cluster:
    - CLUSTER_COMPOSITE_OPS_RO
    - CLUSTER_MONITOR
  indices:
    # no explicit permissions for 'foo-alias' but only 'foo-index':
    'foo-index':
      '*':
        - INDICES_MONITOR
        - READ
        - SEARCH
        - indices:admin/mappings/fields/get*
        - indices:admin/mappings/get
        - indices:data/read/field_caps*

Now, when foo-index doesn-t exist:

GET /foo-alias/_search
 {"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [indices:data/read/search]

GET /foo-index/_search
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index"

Interesting, isn’t it?

[jkressin]

Yes, indeed, this is unexpected IMHO. An index and an alias should basically behave the same way. We treat aliases a bit different than Elastic does. Whenever we need to decide if a request is allowed or not, we would first resolve all aliases to their concrete index names. Hence, the requests to foo-alias and foo-index should yield the same result.

Thanks for letting us know, we will look into this!

[cstaley]

@jkressin will investigate …

[hsaly]

Lets assume that we have an alias “foo-alias” pointing to an non existent index “foo-index” which is the same situation as when neither the alias nor the index exists. There is no such a thing like a dangling alias in elasticsearch.

So an user has permissions for foo-index and now queries foo-index then we return 404 which is correct.
If the same user queries foo-alias (which does not exist and therefore we can not trace it back to foo-index which does also not exist) we return 403 which is correct.

[faxmodem]

You can’t have an orphan alias, this isn’t possible in Elasticsearch

EDIT: I think I understand your rationale, but I disagree: when the user asks for index-foo, we don’t know in advance if it’s an alias or an index. In any case, if it doesn’t exist, the error should be the same: index doesn’t exist. However, now we have two possible outcomes: either ‘index doesn’t exist’, or ‘unauthorized’.