Kibana index patterns and "do_not_fail_on_forbidden" option

SearchGuard Version: 6.2.3-22.1

SearchGuard Kibana version: 6.2.3-13

Elasticsearch Version: 6.2.3

openjdk version “1.8.0_171”

OpenJDK Runtime Environment (build 1.8.0_171-b10)

OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

I’m using the kibana multi tenancy plugin and have user access restricted in searchguard to only indexes that have the username prefix. For example, user “johndoe” can only access indexes with a prefix “johndoe-*”. This access seems to work fine except when searching for indexes in kibana to create an index pattern. Even though the user only has access to their own index, they can still see other indexes.

If the user creates an index pattern on an index they have access to, it works just fine but if they create an index pattern to an index they don’t have access to, it gives a permission error once they create the pattern. It also prevents them from deleting or being able to manage any of the index patterns because the “Index Patterns” page fails to load. The error that is show is: [object Object]: [security_exception] no permissions for [indices:data/read/field_caps]

I figured that I could get around this issue by setting the “do_not_fail_on_forbidden” option to true but that doesn’t seem to take affect at all.

I’m trying to either get the “do_not_fail_on_forbidden” option to work. A better option would be if I can only show indexes that the user has access to on the search results when creating an index pattern.

This is what my searchguard config looks like. I used the sgadmin to pull the config from the cluster. Its basically all the default settings except for the “do_not_fail_on_forbidden” option.

···

searchguard:

dynamic:

kibana:

do_not_fail_on_forbidden: true

http:

anonymous_auth_enabled: false

xff:

enabled: false

internalProxies: “192\.168\.0\.10|192\.168\.0\.11”

remoteIpHeader: “x-forwarded-for”

proxiesHeader: “x-forwarded-by”

authc:

kerberos_auth_domain:

http_enabled: false

transport_enabled: false

order: 6

http_authenticator:

type: “kerberos”

challenge: true

config:

krb_debug: false

strip_realm_from_principal: true

authentication_backend:

type: “noop”

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 4

http_authenticator:

type: “basic”

challenge: true

authentication_backend:

type: “intern”

proxy_auth_domain:

http_enabled: false

transport_enabled: false

order: 3

http_authenticator:

type: “proxy”

challenge: false

config:

user_header: “x-proxy-user”

roles_header: “x-proxy-roles”

authentication_backend:

type: “noop”

jwt_auth_domain:

http_enabled: false

transport_enabled: false

order: 0

http_authenticator:

type: “jwt”

challenge: false

config:

signing_key: “base64 encoded HMAC key or public RSA/ECDSA pem key”

jwt_header: “Authorization”

jwt_url_parameter: null

roles_key: null

subject_key: null

authentication_backend:

type: “noop”

clientcert_auth_domain:

http_enabled: false

transport_enabled: false

order: 2

http_authenticator:

type: “clientcert”

config:

username_attribute: “cn”

challenge: false

authentication_backend:

type: “noop”

ldap:

http_enabled: false

transport_enabled: false

order: 5

http_authenticator:

type: “basic”

challenge: false

authentication_backend:

type: “ldap”

config:

enable_ssl: false

enable_start_tls: false

enable_ssl_client_auth: false

verify_hostnames: true

hosts:

  • “localhost:8389”

bind_dn: null

password: null

userbase: “ou=people,dc=example,dc=com”

usersearch: “(sAMAccountName={0})”

username_attribute: null

authz:

roles_from_myldap:

http_enabled: false

transport_enabled: false

authorization_backend:

type: “ldap”

config:

enable_ssl: false

enable_start_tls: false

enable_ssl_client_auth: false

verify_hostnames: true

hosts:

  • “localhost:8389”

bind_dn: null

password: null

rolebase: “ou=groups,dc=example,dc=com”

rolesearch: “(member={0})”

userroleattribute: null

userrolename: “disabled”

rolename: “cn”

resolve_nested_roles: true

userbase: “ou=people,dc=example,dc=com”

usersearch: “(uid={0})”

roles_from_another_ldap:

enabled: false

authorization_backend:

type: “ldap”

The permissions for the role associated with the user are as follows.

Cluster Permissons: CLUSTER_MONITOR action group

Index Permissions: ${user_name)-* index with a document type of “*” and a permission of READ action group.

The action groups are the default ones setup when the searchguard index is initialized and have not been modified.

Hi,

first, the error you are seeing:

[object Object]: [security_exception] no permissions for [indices:data/read/field_caps]

This is due to incorrect permission settings for the user. Have a look at the default sg_kibana_user role we ship:

sg_kibana_user:
readonly: true
cluster:
- MONITOR
- CLUSTER_COMPOSITE_OPS
indices:
‘?kibana’:
':
- MANAGE
- INDEX
- READ
- DELETE
'
’:
'':
- indices:data/read/field_caps

``

It seems the field_caps permission is missing for your user.

The issue with users seeing indices that they do not have permission for is known. We have fixed that in the Compliance Edition code base, and once the code has been merged back it will also be available for other versions.

At the moment the workaround would be to install the Compliance Edition, and depending on what features you need:

  • Disable the commercial features and run the Community Edition
  • Use an Enterprise or Compliance license

The actual fix has nothing to do with the commercial modules so it is freely available.

The “do_not_fail_on_forbidden” ( we should definitely look for a better name :wink: ) does something different: If the user issues queries that span multiple indices, like msearch for example, it returns only the data from indices where the user has access to. If it is set to false, the query would fail altogether. Both scenarios are valid, and which one is more suitable depends on the use case. Since Kibana performs a lot of msearches and wildcard searches, we usually recommend to set it to true for this case.

···

On Friday, June 22, 2018 at 12:43:06 AM UTC+2, Gevorg Kalantaryan wrote:

SearchGuard Version: 6.2.3-22.1

SearchGuard Kibana version: 6.2.3-13

Elasticsearch Version: 6.2.3

openjdk version “1.8.0_171”

OpenJDK Runtime Environment (build 1.8.0_171-b10)

OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

I’m using the kibana multi tenancy plugin and have user access restricted in searchguard to only indexes that have the username prefix. For example, user “johndoe” can only access indexes with a prefix “johndoe-*”. This access seems to work fine except when searching for indexes in kibana to create an index pattern. Even though the user only has access to their own index, they can still see other indexes.

If the user creates an index pattern on an index they have access to, it works just fine but if they create an index pattern to an index they don’t have access to, it gives a permission error once they create the pattern. It also prevents them from deleting or being able to manage any of the index patterns because the “Index Patterns” page fails to load. The error that is show is: [object Object]: [security_exception] no permissions for [indices:data/read/field_caps]

I figured that I could get around this issue by setting the “do_not_fail_on_forbidden” option to true but that doesn’t seem to take affect at all.

I’m trying to either get the “do_not_fail_on_forbidden” option to work. A better option would be if I can only show indexes that the user has access to on the search results when creating an index pattern.

This is what my searchguard config looks like. I used the sgadmin to pull the config from the cluster. Its basically all the default settings except for the “do_not_fail_on_forbidden” option.


searchguard:

dynamic:

kibana:

do_not_fail_on_forbidden: true

http:

anonymous_auth_enabled: false

xff:

enabled: false

internalProxies: “192\.168\.0\.10|192\.168\.0\.11”

remoteIpHeader: “x-forwarded-for”

proxiesHeader: “x-forwarded-by”

authc:

kerberos_auth_domain:

http_enabled: false

transport_enabled: false

order: 6

http_authenticator:

type: “kerberos”

challenge: true

config:

krb_debug: false

strip_realm_from_principal: true

authentication_backend:

type: “noop”

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 4

http_authenticator:

type: “basic”

challenge: true

authentication_backend:

type: “intern”

proxy_auth_domain:

http_enabled: false

transport_enabled: false

order: 3

http_authenticator:

type: “proxy”

challenge: false

config:

user_header: “x-proxy-user”

roles_header: “x-proxy-roles”

authentication_backend:

type: “noop”

jwt_auth_domain:

http_enabled: false

transport_enabled: false

order: 0

http_authenticator:

type: “jwt”

challenge: false

config:

signing_key: “base64 encoded HMAC key or public RSA/ECDSA pem key”

jwt_header: “Authorization”

jwt_url_parameter: null

roles_key: null

subject_key: null

authentication_backend:

type: “noop”

clientcert_auth_domain:

http_enabled: false

transport_enabled: false

order: 2

http_authenticator:

type: “clientcert”

config:

username_attribute: “cn”

challenge: false

authentication_backend:

type: “noop”

ldap:

http_enabled: false

transport_enabled: false

order: 5

http_authenticator:

type: “basic”

challenge: false

authentication_backend:

type: “ldap”

config:

enable_ssl: false

enable_start_tls: false

enable_ssl_client_auth: false

verify_hostnames: true

hosts:

  • “localhost:8389”

bind_dn: null

password: null

userbase: “ou=people,dc=example,dc=com”

usersearch: “(sAMAccountName={0})”

username_attribute: null

authz:

roles_from_myldap:

http_enabled: false

transport_enabled: false

authorization_backend:

type: “ldap”

config:

enable_ssl: false

enable_start_tls: false

enable_ssl_client_auth: false

verify_hostnames: true

hosts:

  • “localhost:8389”

bind_dn: null

password: null

rolebase: “ou=groups,dc=example,dc=com”

rolesearch: “(member={0})”

userroleattribute: null

userrolename: “disabled”

rolename: “cn”

resolve_nested_roles: true

userbase: “ou=people,dc=example,dc=com”

usersearch: “(uid={0})”

roles_from_another_ldap:

enabled: false

authorization_backend:

type: “ldap”

The permissions for the role associated with the user are as follows.

Cluster Permissons: CLUSTER_MONITOR action group

Index Permissions: ${user_name)-* index with a document type of “*” and a permission of READ action group.

The action groups are the default ones setup when the searchguard index is initialized and have not been modified.

The cluster is meant to have many users sharing the environment and only having access to read their data without writing any data or making cluster level changes. The reason the error is shown is because the user created an index pattern on an index which the user does not have access to.

The READ action group already gives indices:data/read* which should cover the fields_caps permission if I’m not mistaken. The permission is only for the “${user_name}-*” index which should keep user to only within their own index.

In regards to the “do_not_fail_on_forbidden” option. The documentation here (https://docs.search-guard.com/latest/troubleshooting-kibana) uses the timelion example of .es(*) which still throws an error even after setting that value to true.

We’re in the trial period of our license and are currently in the process of purchasing the compliance edition. Does the trial license not provide the compliance functionality until the trial license expires?

Hi,

yes, the READ permission also includes the indices:data/read/field_caps* permissions, but it should be granted on all indices, not just the “${user_name}-*” index. Background: If Kibana fetches the index patterns, it issues a field caps wildcard query on all indices.

But what puzzles me is that you write:

“In regards to the “do_not_fail_on_forbidden” option. The documentation here (https://docs.search-guard.com/latest/troubleshooting-kibana) uses the timelion example of .es(*) which still throws an error even after setting that value to true.”

This should not be the case. Can you attach your sg_config.yml so we can check the settings here? If Timelion still throws an error it might be some kind of configuration error. If you are not comfortable with posting it publicly you can also send it (PGP encrypted if you like) to info@search-guard.com.

“We’re in the trial period of our license and are currently in the process of purchasing the compliance edition. Does the trial license not provide the compliance functionality until the trial license expires?”

Yes, the trial license includes all compliance features, but for the time being the Compliance Edition is a separate download. So instead of installing 6.2.3-22.1 you want to install 6.2.3-compliance-2. You can always find the latest versions here:

This should eliminate the problem of users seeing indices they do not have access to.

Let me know if this works!

···

On Friday, June 22, 2018 at 8:14:42 PM UTC+2, Gevorg Kalantaryan wrote:

The cluster is meant to have many users sharing the environment and only having access to read their data without writing any data or making cluster level changes. The reason the error is shown is because the user created an index pattern on an index which the user does not have access to.

The READ action group already gives indices:data/read* which should cover the fields_caps permission if I’m not mistaken. The permission is only for the “${user_name}-*” index which should keep user to only within their own index.

In regards to the “do_not_fail_on_forbidden” option. The documentation here (https://docs.search-guard.com/latest/troubleshooting-kibana) uses the timelion example of .es(*) which still throws an error even after setting that value to true.

We’re in the trial period of our license and are currently in the process of purchasing the compliance edition. Does the trial license not provide the compliance functionality until the trial license expires?

I added the field_caps permission to the * index for the role associated with the user that was having issues and that seems to have resolved the kibana UI failing issue. The user is now able to delete the index pattern they mistakenly created and create the proper index pattern on the index they have access to. thank you.

The timelion issue still exists though. I posted the sg config in my initial post. I used sgadmin to pull the config directly from the cluster.

I’ll try the compliance version next week and let you know how that goes. Is the compliance and enterprise version ever going to be the same install version in the future? If so, maybe I can wait a bit until that happens so I don’t have to change version and just apply the compliance license once we complete the purchase.

“I’ll try the compliance version next week and let you know how that goes. Is the compliance and enterprise version ever going to be the same install version in the future? If so, maybe I can wait a bit until that happens so I don’t have to change version and just apply the compliance license once we complete the purchase.”

Yes, this is definitely the plan. According to the roadmap that will happen with the release of Search Guard 6.4. However, both versions are compatible with each other, which means you do not change anything regarding your existing configuration. You can simply remove the regular 6.x-22-x version and replace it with 6.x-22-compliance-2, which will fix the index visibility problem even if you don’t need any of the compliance features at the moment.

···

On Friday, June 22, 2018 at 11:26:36 PM UTC+2, Gevorg Kalantaryan wrote:

I added the field_caps permission to the * index for the role associated with the user that was having issues and that seems to have resolved the kibana UI failing issue. The user is now able to delete the index pattern they mistakenly created and create the proper index pattern on the index they have access to. thank you.

The timelion issue still exists though. I posted the sg config in my initial post. I used sgadmin to pull the config directly from the cluster.

I’ll try the compliance version next week and let you know how that goes. Is the compliance and enterprise version ever going to be the same install version in the future? If so, maybe I can wait a bit until that happens so I don’t have to change version and just apply the compliance license once we complete the purchase.