Unable to integrate Grafana to Elasticsearch 7.12.0 using JWT

[Doug_Renze]

I’m attempting to create an auth token that will Grafana to use my Elasticsearch cluster as a data source. When I create the token and test it from the command line with curl, everything works as expected. However, when I execute the search through Grafana, I get the following response back:

response:Object

error:Object
root_cause:Array[1]
 

type:"security_exception"
 

reason:"no permissions for [indices:data/read/msearch] and User me@example.com (AuthToken grafana-token-009 [0N0MCJKOQVqPzDm5MctTLA]) [backend_roles=[SG_ADMIN, SG_USER]]"
 


status:403

My token was created as follows; based on the way I read it, SGS_SEARCH should contain the indices:data/read/msearch as a subset, but I still get the error.

POST /_searchguard/authtoken
{
  "name": "grafana-token-009",
  "requested": {
    "cluster_permissions": [
      "SGS_CLUSTER_MONITOR"
    ],
    "index_permissions": [
      {
        "index_patterns": [
          "*beat-*"
        ],
        "allowed_actions": [
          "SGS_READ",
          "SGS_SEARCH",
          "indices:data/read/msearch"
        ]
      }
    ]
  },
  "expires_after": "1y"
}

I’m using an EKS deployment of Elasticsearch 7.12.0 from the stock images released by Elastic, with the corresponding version of SG added.

A quick PS

I have a role that can perform all these actions, but I’ve never been able to successfully configure a token to use role-based authentication; I always get the following error:

Cannot create token. The resulting token would have no privileges as the specified roles do not intersect with the user’s roles. Specified: [BASIC_ACCESS] User: [admin] + [ ]

[nils]

Hello Doug!

In order that I understand correctly: You tested the access token using a query submitted by curl; then, the query was successful. But, when using Grafana, the query fails with the above mentioned error?

We need to collect some further information to better understand the problem. Can you provide the following:

• The curl command you used
• Your sg_config configuration; particularly, the state of the do_not_fail_on_forbidden setting would be of interest.
• The SG_ADMIN and SG_USER role definitions

If indeed curl succeeds and Grafana fails, one of the first things to check would be whether Grafana issues the query really to the same indices as curl or possibly to more indices. If do_not_fail_on_forbidden is false, the search would fail if one index does not match *beat-*.

Regarding this:

@Doug_Renze

I have a role that can perform all these actions, but I’ve never been able to successfully configure a token to use role-based authentication; I always get the following error:

So, are you creating a token like this then?

POST /_searchguard/authtoken
{
  "requested": {
    "roles": [
      "BASIC_ACCESS"
    ]
  }
}

If this is so, then you get the error because the logged in user does not normally have the BASIC_ACCESS role. If you use the role attribute, Search Guard just calculates the intersection of the roles the user has (in this case admin) and the requested roles. As BASIC_ACCESS is not part of the present roles of the user, the operation fails.

[Doug_Renze]

@nils

In order that I understand correctly: You tested the access token using a query submitted by curl; then, the query was successful. But, when using Grafana, the query fails with the above mentioned error?

So I was incorrect on the test - I tested a _search, not _msearch. When I try _msearch from curl, it fails as well.

@nils

We need to collect some further information to better understand the problem. Can you provide the following:

• The curl command you used
• Your sg_config configuration; particularly, the state of the do_not_fail_on_forbidden setting would be of interest.
• The SG_ADMIN and SG_USER role definitions

I’m attaching my sg_config.yml, however it is set do_not_fail_on_forbidden: true. As for SG_ADMIN and SG_USER, these are back-end roles assigned from ADFS. SG_ADMIN is mapped to the built-in SGS_ALL_ACCESS role, and SGS_USER is mapped to the BASIC_ACCESS role. I also have the same issue if I’m attempting to create the token with the built-in admin account, although it returns an empty roleset. Would manually assigning the admin account to the requested roles and then creating the token with admin resolve the issue without breaking the admin account?

@nils

So, are you creating a token like this then?

POST /_searchguard/authtoken
{
  "requested": {
    "roles": [
      "BASIC_ACCESS"
    ]
  }
}

Yes

@nils

If this is so, then you get the error because the logged in user does not normally have the BASIC_ACCESS role. If you use the role attribute, Search Guard just calculates the intersection of the roles the user has (in this case admin) and the requested roles. As BASIC_ACCESS is not part of the present roles of the user, the operation fails.

I’m creating it with my admin user, which is a member both of the SG_ADMIN backend role and the SG_USER backend role, which is assigned to the BASIC_ACCESS role.

sg_config.yml (4.4 KB)

[Doug_Renze]

I was able to get the correct permissions working for the token by updating the token request as follows:

POST /_searchguard/authtoken
{
  "name": "grafana-token-012",
  "requested": {
    "cluster_permissions": [
      "SGS_CLUSTER_MONITOR",
      "SGS_CLUSTER_COMPOSITE_OPS_RO"
    ],
    "index_permissions": [
      {
        "index_patterns": [
          "*beat-*"
        ],
        "allowed_actions": [
          "SGS_READ",
          "SGS_SEARCH"
        ]
      }
    ]
  },
  "expires_after": "1y"
}

Adding the SGS_CLUSTER_COMPOSITE_OPS_RO did it (the SGS_CLUSTER_MONITOR was added earlier by trial-and-error).

It would still be nice to be able to utilize role-based permissions, but I can handle assigning permissions by action groups, if I need to.