Unable to Refresh Index Pattern in Kibana

Search Guard
1

Elasticsearch version:
7.9.1 and 7.10.2

Kibana version (if relevant):
7.9.1 and 7.10.2

Describe the issue:
Unable to refresh index patterns in Kibana

Steps to reproduce:

  1. enable FLS on some indices
  2. try to refresh Kibana index pattern - fails
  3. disable FLS on all indices
  4. Clear SG cache
  5. try to refresh Kibana index pattern - still fails

Expected behavior:
Index patttern gets refreshed and adds new fields

Provide logs:
Elasticsearch

PrivEvalResponse [allowed=true, missingPrivileges=[indices:data/write/index, indices:data/write/bulk[s]], allowedFlsFields={quantum_prod_v1_2021_02_23=[~operation.settingsUpdates.selectAnswer, ~operation.settingsUpdates.password, ~operation.settingsUpdates.securityCode, ~operation.settingsUpdates.selectQuestion, ~operation.settingsUpdates.confirmPassword]}, maskedFields=null, queries=null]

Kibana (if relevant)

{“type”:“response”,“@timestamp”:“2021-02-23T16:27:04Z”,“tags”:,“pid”:16,“method”:“get”,“statusCode”:200,“req”:{“url”:“/api/index_patterns/_fields_for_wildcard?pattern=quantum_prod_today&meta_fields=_source&meta_fields=_id&meta_fields=_type&meta_fields=_index&meta_fields=_score”,“method”:“get”,“headers”:{“host”:“edited”,“sec-ch-ua”:“"Chromium";v="88", "Google Chrome";v="88", ";Not A Brand";v="99"”,“sec-ch-ua-mobile”:“?0”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36”,“kbn-version”:“7.10.2”,“content-type”:“application/json”,“accept”:“/”,“sec-fetch-site”:“same-origin”,“sec-fetch-mode”:“cors”,“sec-fetch-dest”:“empty”,“referer”:“https://edited/kibana/app/management/kibana/indexPatterns/patterns/8be439c0-cd83-11e8-8da8-33f92a7b1a47",“accept-encoding”:"gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9,fr;q=0.8”,“x-opaque-id”:“mine”,“x-forwarded-for”:“1.1.1.1, 1.1.1.1”,“x-forwarded-host”:“edited”,“x-forwarded-proto”:“https”,“connection”:“close”},“remoteAddress”:“1.1.1.1”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36”,“referer”:“https://edited/kibana/app/management/kibana/indexPatterns/patterns/8be439c0-cd83-11e8-8da8-33f92a7b1a47"},“res”:{“statusCode”:200,“responseTime”:77,“contentLength”:9},“message”:"GET /api/index_patterns/_fields_for_wildcard?pattern=quantum_prod_today&meta_fields=_source&meta_fields=_id&meta_fields=_type&meta_fields=_index&meta_fields=_score 200 77ms - 9.0B”}

{“type”:“log”,“@timestamp”:“2021-02-23T16:27:04Z”,“tags”:[“error”,“elasticsearch”,“data”],“pid”:16,“message”:“[security_exception]: Update is not supported when FLS or DLS or Fieldmasking is activated”}

{“type”:“error”,“@timestamp”:“2021-02-23T16:27:04Z”,“tags”:,“pid”:16,“level”:“error”,“error”:{“message”:“Internal Server Error”,“name”:“Error”,“stack”:“Error: Internal Server Error\n at HapiResponseAdapter.toError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:132:19)\n at HapiResponseAdapter.toHapiResponse (/usr/share/kibana/src/core/server/http/router/response_adapter.js:86:19)\n at HapiResponseAdapter.handle (/usr/share/kibana/src/core/server/http/router/response_adapter.js:81:17)\n at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:164:34)\n at process._tickCallback (internal/process/next_tick.js:68:7)”},“url”:{“protocol”:null,“slashes”:null,“auth”:null,“host”:null,“port”:null,“hostname”:null,“hash”:null,“search”:null,“query”:{},“pathname”:“/api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47”,“path”:“/api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47”,“href”:“/api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47”},“message”:“Internal Server Error”}

{“type”:“response”,“@timestamp”:“2021-02-23T16:27:04Z”,“tags”:,“pid”:16,“method”:“put”,“statusCode”:500,“req”:{“url”:“/api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47”,“method”:“put”,“headers”:{“host”:“edited”,“content-length”:“89698”,“sec-ch-ua”:“"Chromium";v="88", "Google Chrome";v="88", ";Not A Brand";v="99"”,“sec-ch-ua-mobile”:“?0”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36”,“kbn-version”:“7.10.2”,“content-type”:“application/json”,“accept”:“/”,“origin”:“https://edited”,“sec-fetch-site”:“same-origin”,“sec-fetch-mode”:“cors”,“sec-fetch-dest”:“empty”,“referer”:“https://edited/kibana/app/management/kibana/indexPatterns/patterns/8be439c0-cd83-11e8-8da8-33f92a7b1a47",“accept-encoding”:"gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9,fr;q=0.8”,“x-opaque-id”:“mine”,“x-forwarded-for”:“1.1.1.1, 1.1.1.1”,“x-forwarded-host”:“edited”,“x-forwarded-proto”:“https”,“connection”:“close”},“remoteAddress”:“1.1.1.1”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36”,“referer”:“https://edited/kibana/app/management/kibana/indexPatterns/patterns/8be439c0-cd83-11e8-8da8-33f92a7b1a47"},“res”:{“statusCode”:500,“responseTime”:348,“contentLength”:9},“message”:"PUT /api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47 500 348ms - 9.0B”}

Screenshots (if relevant):

Errors in browser console (if relevant):

VM271:1 PUT https://edited/kibana/api/saved_objects/index-pattern/8be439c0-cd83-11e8-8da8-33f92a7b1a47 500 (Internal Server Error)
(anonymous) @ VM271:1
_callee3$ @ core.entry.js:6
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
fetch_asyncGeneratorStep @ core.entry.js:6
_next @ core.entry.js:6
(anonymous) @ core.entry.js:6
(anonymous) @ core.entry.js:6
fetchResponse @ core.entry.js:6
_callee$ @ core.entry.js:6
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
fetch_asyncGeneratorStep @ core.entry.js:6
_next @ core.entry.js:6
Promise.then (async)
fetch_asyncGeneratorStep @ core.entry.js:6
_next @ core.entry.js:6
(anonymous) @ core.entry.js:6
(anonymous) @ core.entry.js:6
(anonymous) @ core.entry.js:6
_callee2$ @ core.entry.js:6
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
fetch_asyncGeneratorStep @ core.entry.js:6
_next @ core.entry.js:6
(anonymous) @ core.entry.js:6
(anonymous) @ core.entry.js:6
(anonymous) @ core.entry.js:6
savedObjectsFetch @ core.entry.js:13
update @ core.entry.js:13
_callee3$ @ data.plugin.js:1
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
asyncGeneratorStep @ data.plugin.js:1
_next @ data.plugin.js:1
(anonymous) @ data.plugin.js:1
(anonymous) @ data.plugin.js:1
update @ data.plugin.js:1
_callee19$ @ data.plugin.js:1
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
index_patterns_asyncGeneratorStep @ data.plugin.js:1
_next @ data.plugin.js:1
(anonymous) @ data.plugin.js:1
(anonymous) @ data.plugin.js:1
updateSavedObject @ data.plugin.js:1
_callee$ @ indexPatternManagement.chunk.1.js:1
l @ kbn-ui-shared-deps.js:380
(anonymous) @ kbn-ui-shared-deps.js:380
forEach.e. @ kbn-ui-shared-deps.js:380
edit_index_pattern_asyncGeneratorStep @ indexPatternManagement.chunk.1.js:1
_next @ indexPatternManagement.chunk.1.js:1
Promise.then (async)
edit_index_pattern_asyncGeneratorStep @ indexPatternManagement.chunk.1.js:1
_next @ indexPatternManagement.chunk.1.js:1
(anonymous) @ indexPatternManagement.chunk.1.js:1
(anonymous) @ indexPatternManagement.chunk.1.js:1
(anonymous) @ indexPatternManagement.chunk.1.js:1
Promise.then (async)
refreshFields @ indexPatternManagement.chunk.1.js:1
g @ kbn-ui-shared-deps.js:434
S @ kbn-ui-shared-deps.js:434
(anonymous) @ kbn-ui-shared-deps.js:434
M @ kbn-ui-shared-deps.js:434
L @ kbn-ui-shared-deps.js:434
A @ kbn-ui-shared-deps.js:434
x @ kbn-ui-shared-deps.js:434
wn @ kbn-ui-shared-deps.js:434
ce @ kbn-ui-shared-deps.js:434
Ln @ kbn-ui-shared-deps.js:434
Nn @ kbn-ui-shared-deps.js:434
xn @ kbn-ui-shared-deps.js:434
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:442
Hi @ kbn-ui-shared-deps.js:434
se @ kbn-ui-shared-deps.js:434
An @ kbn-ui-shared-deps.js:434
Show 26 more frames
core.entry.js:6 Uncaught (in promise) Error: Internal Server Error
at Fetch._callee3$ (core.entry.js:6)
at l (kbn-ui-shared-deps.js:380)
at Generator._invoke (kbn-ui-shared-deps.js:380)
at Generator.forEach.e. [as next] (kbn-ui-shared-deps.js:380)
at fetch_asyncGeneratorStep (core.entry.js:6)
at _next (core.entry.js:6)

Additional data:
Have tried with my personal account and the admin account

2

We have an issue open with our search guard support vendor for at least a week, that has not yielded anything fruitful as of yet.

3

Can you please post the content your your sg_roles configuration with FLS active?

Also, the logs you are providing seem to reflect that FLS still active. You also wrote that the refresh does not work when FLS is disabled. Do you also have logs (from Elasticsearch and Kibana) in this situation?

4

I applied FLS: null changes to wrong cluster.
This needs to be documented for FLS, DLS
Ideally, using FLS, DLS, field_masking should not restrict refreshing any kibana index_pattern

5

We would need the sg_roles with the FLS configuration to be able to tell what exactly went wrong here.

6

_sg_meta:
type: “roles”
config_version: 2
kibana_dashboard_only_user:
reserved: true
hidden: false
description: “Migrated from v6 (all types mapped)”
cluster_permissions:

  • “INDICES_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “?kibana”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “?kibana-6”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/read/field_caps*”
    • “indices:data/read/search”
      tenant_permissions:
      static: false
      sg_own_index:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “${user_name}”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
      tenant_permissions:
      static: false
      venona_admin:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “indices:admin/template/put”
  • “indices:admin/template/get”
    index_permissions:
  • index_patterns:
    • “venona*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “unlimited”
      tenant_permissions:
      static: false
      nifingest:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/write/index”
    • “indices:admin/refresh*”
    • “indices:data/write/bulk*”
    • “indices:admin/create*”
    • “indices:admin/mapping/put”
    • “indices:admin/create”
    • “indices:admin/auto_create”
    • “indices:admin/mapping/auto_put”
    • “indices:admin/*”
    • “indices:admin/mappings/auto_put”
      tenant_permissions:
      static: false
      pi_kibana_user:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “INDICES_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “?kibana”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “?kibana-6”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/read/field_caps*”
    • “indices:data/read/search”
      tenant_permissions:
      static: false
      sg_xp_monitoring:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “cluster:monitor/xpack/info”
  • “cluster:monitor/main”
  • “cluster:admin/xpack/monitoring/bulk”
    index_permissions:
  • index_patterns:
    • “?monitor*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
      tenant_permissions:
      static: false
      sg_kibana:
      reserved: false
      hidden: false
      description: “Migrated from v6, was in rolemappings but no role existed”
      cluster_permissions:
      index_permissions:
      tenant_permissions:
      static: false
      sg_kibana_user:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “INDICES_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “?kibana*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
    • “DELETE”
    • “MANAGE”
    • “INDEX”
  • index_patterns:
    • “?kibana-6”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
    • “DELETE”
    • “MANAGE”
    • “INDEX”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/read/field_caps*”
      tenant_permissions:
      static: false
      sg_xp_alerting:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “indices:data/read/scroll”
  • “cluster:admin/xpack/watcher*”
  • “cluster:monitor/xpack/watcher*”
    index_permissions:
  • index_patterns:
    • “?watches*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?watcher-history-*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?triggered_watches”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
    • “indices:admin/aliases/get”
      tenant_permissions:
      static: false
      elasticview:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “spectrum_guide_rollout_*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      gis_user:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “gis_*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      read_only_venona:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
  • index_patterns:
    • “venona*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “indices:data/read/search*”
    • “indices:admin/get*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_READ”
      static: false
      elastalert:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/mapping/put*”
    • “indices:admin/refresh*”
    • “indices:admin/create*”
    • “indices:admin/mapping/put”
    • “indices:data/read/search”
    • “indices:admin/get*”
  • index_patterns:
    • “elastalert*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “indices:data/read/search”
    • “indices:admin/get*”
    • “UNLIMITED”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      sg_readonly_dlsfls:
      reserved: false
      hidden: false
      description: “Migrated from v6, was in rolemappings but no role existed”
      cluster_permissions:
      index_permissions:
      tenant_permissions:
      static: false
      sg_readall_and_monitor:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS_RO”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
      tenant_permissions:
      static: false
      sg_readall:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_COMPOSITE_OPS_RO”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
    • “indices:data/read/search”
      tenant_permissions:
      static: false
      sg_readall_except_gis:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “cluster:monitor/nodes/stats”
  • “cluster:monitor/health”
  • “CLUSTER_COMPOSITE_OPS_RO”
  • “cluster:monitor/state”
  • “cluster:monitor/nodes/info”
  • “cluster:monitor/main”
  • “indices:data/read/scroll*”
  • “indices:monitor/settings/get”
  • “indices:monitor/*”
    index_permissions:
  • index_patterns:
    • ‘/((?!gis_)(\S|\s))*/’
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
    • “indices:data/read/search”
    • “indices:data/read/scroll”
    • “indices:admin/aliases/get”
    • “indices:monitor/settings/get”
    • “indices:monitor/*”
      tenant_permissions:
      static: false
      sg_kibana_testindex:
      reserved: false
      hidden: false
      description: “Migrated from v6, was in rolemappings but no role existed”
      cluster_permissions:
      index_permissions:
      tenant_permissions:
      static: false
      sg_alerting:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “indices:data/read/scroll”
  • “cluster:admin/xpack/watcher/watch/put”
  • “cluster:admin/xpack/watcher*”
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “?kibana*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “?watches*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?watcher-history-*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?triggered_watches”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
      tenant_permissions:
      static: false
      sg_readonly_and_monitor:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS_RO”
  • “cluster:admin/xpack/monitoring/*”
  • “cluster:admin/ingest/pipeline/get”
  • “indices:admin/template/get”
  • “indices:admin/get”
  • “cluster:admin/repository/get”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
    • “indices:monitor/stats”
    • “indices:data/read/field_caps”
    • “indices:admin/get”
    • “indices:monitor/settings/get”
    • “indices:admin/aliases/get”
    • “indices:monitor/recovery”
    • “indices:monitor/segments”
      tenant_permissions:
      static: false
      quantum_stream_aggs:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “quantum_stream_aggs*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      sg_monitor:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “cluster:admin/xpack/monitoring/*”
  • “cluster:admin/ingest/pipeline/put”
  • “cluster:admin/ingest/pipeline/get”
  • “indices:admin/template/get”
  • “indices:admin/template/put”
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
    index_permissions:
  • index_patterns:
    • “?monitor*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?marvel*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?kibana*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:monitor/stats”
    • “indices:data/read/field_caps”
    • “indices:monitor/settings/get”
      tenant_permissions:
      static: false
      sg_manage_snapshots:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “MANAGE_SNAPSHOTS”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/write/index”
    • “indices:admin/create”
      tenant_permissions:
      static: false
      sg_xp_machine_learning:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “cluster:admin/persistent*”
  • “cluster:internal/xpack/ml*”
  • “indices:data/read/scroll*”
  • “cluster:admin/xpack/ml*”
  • “cluster:monitor/xpack/ml*”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “READ”
    • “indices:admin/get*”
  • index_patterns:
    • “?ml-*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “*”
      tenant_permissions:
      static: false
      customeridentity:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “customer_identity_*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      sg_kibana_server:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
  • “cluster:admin/xpack/monitoring*”
  • “indices:admin/template*”
  • “indices:admin/create*”
  • “indices:data/read/scroll*”
  • “cluster:admin/ilm/get”
  • “cluster:admin/ilm/put”
    index_permissions:
  • index_patterns:
    • “?kibana*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
    • “INDICES_ALL”
    • “DELETE”
    • “MANAGE”
    • “INDEX”
    • “indices:admin/auto_create”
    • “indices:data/write/reindex”
  • index_patterns:
    • “?tasks*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “?reporting*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
    • “indices:admin/auto_create”
    • “indices:data/read/get”
  • index_patterns:
    • “?monitoring*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “INDICES_ALL”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “INDICES_ALL”
    • “READ”
    • “DELETE”
    • “MANAGE”
    • “INDEX”
    • “indices:data/read/get”
    • “indices:admin/auto_create”
    • “indices:admin/mappings/auto_put”
      tenant_permissions:
      static: false
      sg_all_access:
      reserved: true
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
  • “MANAGE_SNAPSHOTS”
  • “cluster:admin/snapshot/restore”
  • “cluster:admin/*”
  • “*”
    index_permissions:
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/write/index”
    • “indices:data/write/update”
    • “indices:admin/mapping/put”
    • “indices:admin/create”
    • “UNLIMITED”
    • “indices:admin/get”
    • “*”
    • “cluster:admin/snapshot/restore”
      tenant_permissions:
  • tenant_patterns:
    • “admin_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      sg_public:
      reserved: false
      hidden: false
      description: “Migrated from v6, was in rolemappings but no role existed”
      cluster_permissions:
      index_permissions:
      tenant_permissions:
      static: false
      sg_logstash:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “CLUSTER_MONITOR”
  • “CLUSTER_COMPOSITE_OPS”
  • “indices:admin/template/get”
  • “indices:admin/template/put”
    index_permissions:
  • index_patterns:
    • “logstash-*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “CRUD”
    • “CREATE_INDEX”
  • index_patterns:
    • “beat”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “CRUD”
    • “CREATE_INDEX”
      tenant_permissions:
      static: false
      hsdookla:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “hsd_ookla_*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “gis_events_hsd_ookla*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      speedtest:
      reserved: false
      hidden: false
      description: “Migrated from v6 (all types mapped)”
      cluster_permissions:
  • “UNLIMITED”
    index_permissions:
  • index_patterns:
    • “speedtest*”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “UNLIMITED”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:admin/refresh*”
    • “indices:admin/create*”
      tenant_permissions:
  • tenant_patterns:
    • “test_tenant”
    • “adm_tenant”
      allowed_actions:
    • “SGS_KIBANA_ALL_WRITE”
      static: false
      sg_signals_manager:
      reserved: false
      hidden: false
      description: “Role for kibana users to be able to use watches”
      cluster_permissions:
  • “SGS_SIGNALS_ACCOUNT_MANAGE”
  • “SGS_CLUSTER_COMPOSITE”
  • “SGS_SIGNALS_ALL”
    index_permissions:
  • index_patterns:
    • “?kibana-6”
      dls: null
      fls: null
      masked_fields: null
      allowed_actions:
    • “READ”
  • index_patterns:
    • “*”
      dls: null
      fls:
      • “~operation.settingsUpdates.confirmPassword”
      • “~operation.settingsUpdates.password”
      • “~operation.settingsUpdates.securityCode”
      • “~operation.settingsUpdates.selectAnswer”
      • “~operation.settingsUpdates.selectQuestion”
        masked_fields: null
        allowed_actions:
    • “indices:data/read/field_caps*”
    • “indices:data/read/search”
      tenant_permissions:
  • tenant_patterns:
    • “SGS_GLOBAL_TENANT”
      allowed_actions:
    • “SGS_SIGNALS_ALL”
7

Thank you.

The problem you are observing results from the following factors:

Kibana stores index patterns, visualisations and other things in its own, private index (named .kibana; if you use multi-tenancy, there will be an additional suffix appended to the index name).

When configuring DLS or FLS for an index, this index will be automatically read-only. This is because otherwise a user would be allowed to manipulate data they might be not allowed to view.

The sg_roles configuration you posted above applies FLS to all indexes by using the index pattern *. This also includes the index used by Kibana. Thus, Kibana won’t be able to write to this index. This causes the errors you are observing.

The solution would be using a more specific index pattern like my_index or my_index_*.

8

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.