Exclude Role from Access to Multiple Indices

[Doug_Renze]

TL;DR

What’s an easy way to deny users access to multiple index patterns?

Implementation Details

• Elasticsearch/Kibana: 7.8.0 (official Elastic Docker images, with Search Guard installed)
• Search Guard: 43.0.0

Question

According to the Search Guard documentation, index permissions can be granted to users based on regular expressions, which makes the following permissions grant possible, giving the ability to read all indices except for myapp-*:

# Grant read access on all indices except for those matching
# regex pattern.
READALL_EXCEPT:
  reserved: false
  hidden: false
  description: 'Grant RO access to indices for users.'
  cluster_permissions:
    - 'SGS_CLUSTER_COMPOSITE_OPS_RO'
  index_permissions:
    - index_patterns:
        - '/^(?!myapp-)\S*$/'
      fls: []
      masked_fields: []
      allowed_actions:
        - 'SGS_READ'
  tenant_permissions: []
  static: false

We primarily use our Elastic Stack for log aggregation, so our permission structure in Elastic is permissive: We grant all users the ability to view all data unless there is a specific reason to exclude them from viewing that data (i.e., certain applications). In our environment, there are multiple applications, which need to have data restricted. When using the pattern above, this can end up creating a very long regex, and can be a little tricky to create in the Kibana Search Guard UI:

# Grant read access on all indices except for those matching
# regex pattern.
READALL_EXCEPT:
  reserved: false
  hidden: false
  description: 'Grant RO access to indices for users.'
  cluster_permissions:
    - 'SGS_CLUSTER_COMPOSITE_OPS_RO'
  index_permissions:
    - index_patterns:
        - '/^(?!first-app-|second-app-|third-app-|fourth-app-)\S*$/'
      fls: []
      masked_fields: []
      allowed_actions:
        - 'SGS_READ'
  tenant_permissions: []
  static: false

Is there any way to simplify this without having to specifically add each index to the allow list? I.e., I note that index_patterns is a list. Does SG process these in order, to look for the first match, or does it look for any match? For example, would the following index_permissions specification allow or deny access to the third-app-* index pattern?

index_permissions:
  - index_patterns:
      - '/^(?!first-app-)\S*$/'
      - '/^(?!second-app-)\S*$/'
      - '/^(?!third-app-)\S*$/'
      - '/^(?!fourth-app-)\S*$/'
      - '*'  # Match all other index patterns.

From the design of the Kibana Search Guard UI, I’m guessing the answer is “no,” but I’m hoping. If that doesn’t work, is there an alternate way to accomplish the same thing without either having the super-long index pattern or having to individually add indices to the allow list?

[jkressin]

The answer to your question is: yes and no and maybe

While it is not possible to specify “includes” and “excludes” for index patterns, you can in fact use RegEx negative lookahead definition in index patterns. Search Guard would first evaluate what index / indices a user wants to access. It then evaluates all index patterns of all roles to see if there is any match. Then the actual permission(s) of the index pattern is checked.

So take for example this role definition (taken from one of our unit tests):

sg_negativelookahead:
  cluster_permissions:
    - SGS_CLUSTER_COMPOSITE_OPS_RO
  index_permissions:
    - index_patterns:
      - '/((?!humanresources)(\S|\s))*/'
      allowed_actions:
        - '*'

This would grant the bearer of this role access to all indices, but not to the index “humanresources”. This is tested and supported. Mixing negative lookahead with matching (positive) patterns is IMHO not supported, and would not work from a pure theoretical point of view: When looking for matching index patterns, Search Guard would look for any matching pattern in a role. Which means your index patterns would state something like: Apply these permissions if the index name matches this pattern OR if it does not match that pattern.

But, if your use case is: Allow access to all indices, but exclude others, this should work.

I hope I understood your use case correct!

[Doug_Renze]

@jkressin

This would grant the bearer of this role access to all indices, but not to the index “humanresources”. This is tested and supported. Mixing negative lookahead with matching (positive) patterns is IMHO not supported, and would not work from a pure theoretical point of view: When looking for matching index patterns, Search Guard would look for any matching pattern in a role. Which means your index patterns would state something like: Apply these permissions if the index name matches this pattern OR if it does not match that pattern.

OK - so the short answer to the question is, as I expected, “no.” The second pattern I provided:

'/^(?!first-app-|second-app-|third-app-|fourth-app-)\S*$/'

correctly excludes multiple index patterns. It’s just a nuisance to write and test the initial pattern.

[jkressin]

We will definitely evaluate what the efforts would be to implement your use case. While it may not be the most common one, I do think it’s valid nonetheless and helpful also for other users.

So in short this would make it possible to grant access to a broader range of indices, but then you would be able to narrow it down again by excluding one or more indices. This would solve your problem, right?

[Doug_Renze]

@jkressin

We will definitely evaluate what the efforts would be to implement your use case. While it may not be the most common one, I do think it’s valid nonetheless and helpful also for other users.

Thank you. I’m also submitting a feature request for this.

@jkressin

So in short this would make it possible to grant access to a broader range of indices, but then you would be able to narrow it down again by excluding one or more indices. This would solve your problem, right?

That is correct. A deny list is easier to manage than an allow list, unless you’re only going to be allowing a very few things.

Thanks again!

Update

Here is my feature request for this item.