# Search Guard incompatible with data streams?
**URL:** https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347
**Category:** Search Guard
**Created:** [September 2, 2022, 1:55pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347 "2022-09-02T13:55:34Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@trauta](https://forum.search-guard.com/u/trauta)
#### Post date: [September 2, 2022, 1:55pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/1 "2022-09-02T13:55:34Z")
Hi, I am running an Elasticsearch cluster with a Search Guard Scientific license.
The cluster is primarily used for log events. Up to now, daily indices were created like `SERVICE-%{+YYYY.MM.dd}` for example. Recently I converted all indices to [data streams](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/data-streams.html).
After migrating the existing indices to data streams, the existing Search Guard roles seem to be incompatible with the data streams.
On a test cluster, I have recreated following scenario for debugging purposes.
Two data streams exists:
- logstash-SERVICE1-default
- logstash-SERVICE2-default
sg\_roles.yml:
```auto
TEST:
reserved: false
hidden: false
description: ""
cluster_permissions:
- "SGS_CLUSTER_COMPOSITE_OPS_RO"
index_permissions:
- index_patterns:
- "logstash-SERVICE1-default"
dls: []
fls: []
masked_fields: []
allowed_actions:
- "SGS_GET"
- "SGS_READ"
- "SGS_SEARCH"
tenant_permissions:
- tenant_patterns:
- "test"
allowed_actions:
- "SGS_KIBANA_ALL_READ"
- "SGS_KIBANA_ALL_WRITE"
- "SGS_SIGNALS_WATCH_MANAGE"
exclude_cluster_permissions: []
exclude_index_permissions: []
static: false
```
Despite this role a user can access both data streams with the following index pattern `logstash-*`.
Even an `exclude_index_permissions` setting for the data stream `logstash-SERVICE2-default` does not prevent this.
Also, without any configured index permissions (includes and excludes) I can access both data stream in Kibana and via the Elasticsearch REST APIs.
Is Search Guard unable to handle data streams?
I have not found anything about this in the documentation pages.
I hope you can help me.
Regards,
Alex
**Elasticsearch version:**
7.17.4
**Server OS version:**
Ubuntu 20.04.5 LTS
**Kibana version:**
7.17.4
**Search Guard version:**
7.17.4-53.1.0
---
### Author:  [@trauta](https://forum.search-guard.com/u/trauta)
#### Post date: [September 2, 2022, 6:10pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/2 "2022-09-02T18:10:50Z")
Hi,
a small but disturbing update:
Even if I remove every non-default role and action group, an authenticated user can access all data stream events via the \_search API.
Requests to every other regular indices are blocked, but every data stream can be accessed via the `logstash-*` index pattern.
Direct access to a specific data stream is blocked, but once you add an \* anywhere in the index string the data is retrievable.
I’m using LDAP authentication and sg\_roles.yml and sg\_roles\_mapping.yml are empty.
This seems to be a major security flaw!
Why does Search Guard allow unrestricted access to all data streams?
Regards,
Alex
---
### Author:  [@jkressin](https://forum.search-guard.com/u/jkressin)
#### Post date: [September 3, 2022, 7:32pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/3 "2022-09-03T19:32:57Z")
---
### Author:  [@jkressin](https://forum.search-guard.com/u/jkressin)
#### Post date: [September 4, 2022, 7:25pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/4 "2022-09-04T19:25:31Z")
Hi Alex - this of course does not seem right, we are investigating and get back to you asap.
---
### Author:  [@jkressin](https://forum.search-guard.com/u/jkressin)
#### Post date: [September 4, 2022, 7:25pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/5 "2022-09-04T19:25:56Z")
---
### Author:  [@jkressin](https://forum.search-guard.com/u/jkressin)
#### Post date: [September 4, 2022, 7:26pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/6 "2022-09-04T19:26:08Z")
---
### Author:  [@trauta](https://forum.search-guard.com/u/trauta)
#### Post date: [September 13, 2022, 10:11am UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/7 "2022-09-13T10:11:53Z")
Hi, any updates on this?
---
### Author:  [@jkressin](https://forum.search-guard.com/u/jkressin)
#### Post date: [September 13, 2022, 12:18pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/8 "2022-09-13T12:18:36Z")
Hi, we have analyzed the issue. Since data streams are an X-Pack feature in Elasticsearch 7.x, we did not officially support them. However, we already changed the implementation for Search Guard FLX, so the behavior you are describing is not present anymore in FLX, which is available for 7.17.x and the upcoming FLX for 8.x ([Latest Releases | Security for Elasticsearch | Search Guard](https://docs.search-guard.com/latest/search-guard-versions)).
Having said that, we are looking into ways to mitigate the behavior also for older versions of Search Guard and will keep you posted.
---
### Author:  [@trauta](https://forum.search-guard.com/u/trauta)
#### Post date: [September 16, 2022, 8:36am UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/9 "2022-09-16T08:36:50Z")
Hi, thank you for your reply! I didn’t know that data streams are a X-Pack feature. Where did you get that information from? I don’t see any hints on this on the [documentation sites](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/data-streams.html).
I’ve updated our dev cluster to Search Guard FLX and I can happily confirm that the update fixes the security problem. Once I finished testing all features I will migrate our prod cluster, so we can safely use data streams again.
One more small hint, after upgrading to FLX the built-in admin user is unable to see the data streams in the ‚Index Management‘ → ‚Data Streams‘ view in Kibana. There is no error messages, it simply says that there are no existing data streams. Calling the `GET /_data_stream/` API also results in an empty response, but fetching a specific existing stream like `GET /_data_stream/logstash-SERVICE1-default` works. This is only a small problem, but it would simplify the handling of the data streams.
---
### Author:  [@nils](https://forum.search-guard.com/u/nils)
#### Post date: [September 16, 2022, 6:39pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/10 "2022-09-16T18:39:24Z")
Thank you for the report; I filed an issue in our Gitlab repo to check what’s the matter with `GET /_data_steam/` API: [GET /\_data\_stream/ API returns an empty response (#119) · Issues · search-guard / Search Guard Suite Enterprise · GitLab](https://git.floragunn.com/search-guard/search-guard-suite-enterprise/-/issues/119)
---
### Author:  [@nils](https://forum.search-guard.com/u/nils)
#### Post date: [September 16, 2022, 6:40pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/11 "2022-09-16T18:40:54Z")
---
### Author:  [@system](https://forum.search-guard.com/u/system)
#### Post date: [October 7, 2022, 6:41pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/12 "2022-10-07T18:41:48Z")
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.
---
### Author:  [@searchguard](https://forum.search-guard.com/u/searchguard)
#### Post date: [October 25, 2022, 1:27pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/13 "2022-10-25T13:27:30Z")
---
### Author:  [@searchguard](https://forum.search-guard.com/u/searchguard)
#### Post date: [October 25, 2022, 1:29pm UTC](https://forum.search-guard.com/t/search-guard-incompatible-with-data-streams/2347/14 "2022-10-25T13:29:13Z")
We fixed this with Search Guard 53.5.0. Search Guard FLX for Elasticsearch 7 and 8 is not affected.