Kibana unable to access SGS_GLOBAL_TENANT after upgrade to FLX

I migrate working configuration from 7.16.3 cluster old plugin to 7.17.6 FLX plugin and I cant get kibana to create kibana index successfully. After various attempts I have stripped my configuration to bare minimum but no luck. Kibana has issues with access to global tenant. If I check component-state it says _main tenant is initialized.

component-state:
- type: “tenant”
name: “_main”
state: “INITIALIZED”
started_at: “2022-09-13T14:33:28.874144Z”
changed_at: “2022-09-13T14:33:29.166364Z”

kibana.log

{"ecs":{"version":"1.12.0"},"@timestamp":"2022-09-14T06:06:34.911+00:00","message":"[.kibana] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 32010ms.","log":{"level":"INFO","logger":"savedobjects-service"},"process":{"pid":7}}
{"ecs":{"version":"1.12.0"},"@timestamp":"2022-09-14T06:07:38.918+00:00","message":"[.kibana] Action failed with 'security_exception: [security_exception] Reason: Insufficient permissions'. Retrying attempt 7 in 64 seconds.","log":{"level":"ERROR","logger":"savedobjects-service"},"process":{"pid":7}}

elasticsearch.log

{"type": "server", "timestamp": "2022-09-14T08:06:34,907+02:00", "level": "WARN", "component": "c.f.s.e.f.PrivilegesInterceptorImpl", "cluster.name": "test", "node.name": "test-1", "message": "Tenant SGS_GLOBAL_TENANT is not allowed for user super_kibana_user", "cluster.uuid": "BrYWyqcdRZS3yq1MyyKxmA", "node.id": "qWxJqns_TP21tdx6W2SP-A"  }

sg_roles.yml

---
kibanaserver_additionals:   # I added this because kibana complaint it was missing permissions, maybe it should be included in SGS_KIBANA_SERVER ??
  cluster_permissions:
  - "indices:monitor/settings/get"
  index_permissions:
  - index_patterns: null
    allowed_actions: null
    fls: null
    masked_fields: []
  tenant_permissions:
  - tenant_patterns: null
    allowed_actions: null

sg_internal_users.yml

---
super_kibana_user:
  hash: "$2y$12$fSzJgMISnDBh99JeCaDTnuy4QwTxYRq0rTjaQ2M9fVulE2JkXJgXW"
  reserved: false
  description: "Demo kibanaserver user"
  search_guard_roles:
  - "kibanaserver_additionals"
  - "SGS_KIBANA_SERVER"

curl https://test:9200/_searchguard/authinfo

{
  "user": "User super_kibana_user <basic/internal_users_db> [sg_roles=[kibanaserver_additionals, SGS_KIBANA_SERVER]]",
  "user_name": "super_kibana_user",
  "user_requested_tenant": null,
  "remote_address": "x.x.x.x:64842",
  "backend_roles": [],
  "custom_attribute_names": [],
  "attribute_names": [],
  "sg_roles": [
    "kibanaserver_additionals",
    "SGS_KIBANA_SERVER"
  ],
  "sg_tenants": {
    "super_kibana_user": true
  },
  "principal": null,
  "peer_certificates": "0",
  "cluster_name": "test"
}

@peter82 Could you tell me more about the super_kibana_user? Is it a regular user or a Kibana service account?

After digging more into it, the problem was in my configuration, I was missing:
ssg_frontend_multi_tenancy.yml

---
enabled: true
index: ".kibana"
server_user: "super_kibana_user"

@peter82 Thanks for sharing your fix. The user name was confusing, and I suspected that you were creating a regular user to access Kibana UI.

When you create a custom service account for the Kibana server, you have to configure it in the server_user option in sg_frontend_multi_tenancy.yml as you did. If not set, kibanaserver is the default value.