Roles not evaluated - in jwt

Search Guard
1

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

2

Please have a look at the docs for how the roles mapping works:

So, you just need to map backend roles to Search Guard roles:

"Depending on your configuration, you can use the following data to assign a request to one or more Search Guard roles:

…

backend roles

  • the roles fetched by the authorization backend(s), like LDAP, JWT or the internal user database."
    In Search Guard you can use any / many backend systems for authentication and authorization, so any role that comes from such a system (like JWT) needs to be mapped to a SG role.

This is explained in the doc mentioned above in Chapter “Backend roles and Search Guard roles” and also here:

So you can use something like:

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

backendroles:

  • 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

Wildcards are also supported for the roles mapping.

···

On Monday, November 13, 2017 at 3:54:10 AM UTC+1, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

3

Hi

I have read both these docs already, but there is something still missing. In the sg_roles_mapping.yml file you have roles such as sg_monitor, which then has a backendroles under of kibanarole. But the only reference i can find in the sgconfig folder for kibanarole is in sg_internal_users.yml - which is a password.

Where is kibanarole defined? You have many other backend roles in the sg_roles_mapping.yml file - but again no references can be found in other files.

Also how is sg_all_access in sg_roles_mapping.yml mapped back to permissions? Is this using sg_roles.yml? As it looks like it is, but there is no reference to any backendroles for this in either file. I can see in sg_roles.yml that it has lines under it that are in the sg_actions_group.yml file - so that ‘can’ make sense. But then there is no obvious way that the others are mapped.

Also is there a way that the backendrole can be mapped to the role in a wildcard manner so we dont need to map everyone across? Ie

*:

backendroles:

  • {rolename}

So that every role would map to the backendrole with the same name? Or is this not needed.

···

On Tuesday, November 14, 2017 at 10:52:51 PM UTC+11, Jochen Kressin wrote:

Please have a look at the docs for how the roles mapping works:

http://floragunncom.github.io/search-guard-docs/configuration_roles_mapping.html

So, you just need to map backend roles to Search Guard roles:

"Depending on your configuration, you can use the following data to assign a request to one or more Search Guard roles:

…

backend roles

  • the roles fetched by the authorization backend(s), like LDAP, JWT or the internal user database."
    In Search Guard you can use any / many backend systems for authentication and authorization, so any role that comes from such a system (like JWT) needs to be mapped to a SG role.

This is explained in the doc mentioned above in Chapter “Backend roles and Search Guard roles” and also here:

http://floragunncom.github.io/search-guard-docs/overview.html

So you can use something like:

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

backendroles:

  • 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

Wildcards are also supported for the roles mapping.

On Monday, November 13, 2017 at 3:54:10 AM UTC+1, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

4

I think the confusion here is about authentication/authorisation and the concept of backend roles in general, and what the sg_internal_users.yml file is for.

Search Guard supports many different authentication/authorisation methods. One of them is the so called “internal user database”. This is meant for customers that do not have anything like LDAP, Active Directory etc. in place, and just want to set up some users and permissions. These users are stored in your Elasticsearch clusters directly, and the file where you define them is sg_internal_users.yml. For example, if you would use JWT exclusively, the sg_internal_users.yml is totally irrelevant.

In sg_internal_users.yml you can define users, their backend role(s) and their password hashes like:

  <username>:
hash: <hashed password>
roles:
- <rolename>
- <rolename>

(taken from Redirecting…)

So regarding your question “Where is kibanarole defined?”:

In sg_internal_users.yml we have defined a user called “kibanaro” and assigned the backend role “kibanarole”:

kibanaro:

hash: $2a$12$JJSXNfTowz7Uu5ttXfeYpeYE0arACvcwlPBStB1F.MI7f0U9Z4DGC

#password is: kibanaro

roles:

  • kibanarole

When you log in with this user, its backend role “kibanarole” is mapped to the Search Guard role “sg_kibana” in the roles_mapping.yml file via this entry:

sg_kibana:

backendroles:

  • kibanarole

Then in sg_roles.yml, the permissions for the sg_kibana roles are defined:

For users which use kibana

sg_kibana:

cluster:

  • CLUSTER_COMPOSITE_OPS_RO

indices:

‘*’:

‘*’:

  • READ

‘?kibana’:

‘*’:

  • INDICES_ALL

In older versions of Search Guard we used the username for the role mapping, so in sg_internal_users you had:

kibanaro:

hash: $2a$12$JJSXNfTowz7Uu5ttXfeYpeYE0arACvcwlPBStB1F.MI7f0U9Z4DGC

#password is: kibanaro

and in roles_mapping the user is directly mapped:

sg_kibana4:

users:

  • kibanaro

Both ways are possible, but using the backend roles for mapping is more flexible. But the flow is always:

  • A user is authenticated and authorized against the configured modules in sg_config.yml

  • During this process, backend roles for the user are collected

  • These backend roles can for example come from Active Directory, JWT, or the internal user database

  • The user and its backend roles are mapped to SG roles

  • The SG roles define the permissions

“You have many other backend roles in the sg_roles_mapping.yml file - but again no references can be found in other files.” Correct, and there’s no need to add those roles to any other files, that’s the concept of having the role mapping: Say you fetch your backend roles from Active Directory, you just need to map the AD roles to SG roles in the roles mapping file, that’s it.

"Also how is sg_all_access in sg_roles_mapping.yml mapped back to permissions? "

It’s mapped by it’s SG role name. The structure of the roles mapping entries is:

  <Search Guard role name>:
users:
- <username>
- ...
backendroles:
- <rolename>
- ...
hosts:
- <hostname>
- ...

(Redirecting…)

So this entry here:

sg_all_access:

users:

  • admin

Maps the user with username “admin” (and again, the user can come from any auth module like internal user database, AD, JWT etc.) to the Search Guard role sg_all_access. The definition of this role can be found in sg_roles.yml:

Allows everything

but not changes to searchguard config/index

sg_all_access:

cluster:

  • UNLIMITED

indices:

‘*’:

‘*’:

  • UNLIMITED

tenants:

adm_tenant: RW

test_tenant_ro: RW

“So that every role would map to the backendrole with the same name? Or is this not needed.”

It’s the other way round - you map backend roles to SG roles, not SG roles to backend roles, which would not make much sense. The roles mapping is essential, it’s the connection between username/backendroles and SG roles.

What’s the exact use case here, so I can advice on a best practice?

···

On Tuesday, November 14, 2017 at 7:43:28 PM UTC+1, Paul Azad wrote:

Hi

I have read both these docs already, but there is something still missing. In the sg_roles_mapping.yml file you have roles such as sg_monitor, which then has a backendroles under of kibanarole. But the only reference i can find in the sgconfig folder for kibanarole is in sg_internal_users.yml - which is a password.

Where is kibanarole defined? You have many other backend roles in the sg_roles_mapping.yml file - but again no references can be found in other files.

Also how is sg_all_access in sg_roles_mapping.yml mapped back to permissions? Is this using sg_roles.yml? As it looks like it is, but there is no reference to any backendroles for this in either file. I can see in sg_roles.yml that it has lines under it that are in the sg_actions_group.yml file - so that ‘can’ make sense. But then there is no obvious way that the others are mapped.

Also is there a way that the backendrole can be mapped to the role in a wildcard manner so we dont need to map everyone across? Ie

*:

backendroles:

  • {rolename}

So that every role would map to the backendrole with the same name? Or is this not needed.

On Tuesday, November 14, 2017 at 10:52:51 PM UTC+11, Jochen Kressin wrote:

Please have a look at the docs for how the roles mapping works:

http://floragunncom.github.io/search-guard-docs/configuration_roles_mapping.html

So, you just need to map backend roles to Search Guard roles:

"Depending on your configuration, you can use the following data to assign a request to one or more Search Guard roles:

…

backend roles

  • the roles fetched by the authorization backend(s), like LDAP, JWT or the internal user database."
    In Search Guard you can use any / many backend systems for authentication and authorization, so any role that comes from such a system (like JWT) needs to be mapped to a SG role.

This is explained in the doc mentioned above in Chapter “Backend roles and Search Guard roles” and also here:

http://floragunncom.github.io/search-guard-docs/overview.html

So you can use something like:

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

backendroles:

  • 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

Wildcards are also supported for the roles mapping.

On Monday, November 13, 2017 at 3:54:10 AM UTC+1, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

5

Hi

Thanks, so from this i take the following takeaways:

sg_internal_users

  • not needed at all

sg_roles:

  • has the sg roles defined

  • sg roles are what is evaluated by SG to check authorization.

sg_roles_mapping

  • has the mapping between the SG role and the backend role

  • backend roles are just a grouping, and does not need to be used.

For JWT,

Do backend roles need to be used, or can we just specify SG roles?

Do SG roles have to start with sg_ ?

Can backend roles start with sg_ ?

Our use case is that we we will want to pass the username & a role from JWT, with the least configuration on the ELK host as possible.

So if i send username ‘user1’, and role ‘role1’, (but also have user 2,3, 4 , etc etc, and have role1, 2 3 etc etc) i would like to know which configuration i need to do, either via modifying the yml files, or if this PoC goes well, we will look at using the SG REST API to do it.

···

On Monday, November 13, 2017 at 1:54:10 PM UTC+11, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

6

Hi

Any chance of getting a reply?

Thanks

···

On Monday, November 20, 2017 at 10:34:31 AM UTC+11, Paul Azad wrote:

Hi

Thanks, so from this i take the following takeaways:

sg_internal_users

  • not needed at all

sg_roles:

  • has the sg roles defined
  • sg roles are what is evaluated by SG to check authorization.

sg_roles_mapping

  • has the mapping between the SG role and the backend role
  • backend roles are just a grouping, and does not need to be used.

For JWT,

Do backend roles need to be used, or can we just specify SG roles?

Do SG roles have to start with sg_ ?

Can backend roles start with sg_ ?

Our use case is that we we will want to pass the username & a role from JWT, with the least configuration on the ELK host as possible.

So if i send username ‘user1’, and role ‘role1’, (but also have user 2,3, 4 , etc etc, and have role1, 2 3 etc etc) i would like to know which configuration i need to do, either via modifying the yml files, or if this PoC goes well, we will look at using the SG REST API to do it.

On Monday, November 13, 2017 at 1:54:10 PM UTC+11, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]

7

Do backend roles need to be used, or can we just specify SG roles?

For Search Guard 5, mapping the backend roles from your JWT to Search Guard roles is mandatory. So if your token contains username and optional backend roles, you have to map username and/or the backend roles from JWT to Search Guard roles in roles_mapping.yml. Otherwise, no Search Guard role would be assigned.

For Search Guard 6, you can configure how Search Guard will map backend roles to Search Guard roles:

I guess you are looking for BACKENDROLES_ONLY or BOTH. When enabled, you can use the roles in the JWT directly as Search Guard roles, and skip the mapping completely. But, this feature is available from 6.x onwards only.

Do SG roles have to start with sg_ ?

No, you can use any name here.

Can backend roles start with sg_ ?

No, you can use any name here.

*Our use case is that we we will want to pass the username & a role from JWT, with the least configuration on the ELK host as possible. *

So if i send username ‘user1’, and role ‘role1’, (but also have user 2,3, 4 , etc etc, and have role1, 2 3 etc etc) i would like to know which configuration i need to do, either via modifying the yml files, or if this PoC goes well, we will look at using the SG REST API to do it.

To make sure I understand the use case correctly:

In your JWT, you have a field containing the username and another one containing the roles. Say, it’s jwt_user_1/jwt_role_1 and in another token it’s jwt_user_2/jwt_role_2.

You want to define different permissions for jwt_role1 and jwt_role2.

For Search Guard 5:

You need to map the roles from the JWT to Search Guard roles, that’s where the actual permissions are defined. For each backend role in your JWTs, add an entry in roles_mapping.yml like:

sg_role_1:

backendroles:

  • jwt_role_1

So jwt_role_1 from the token is mapped to a Search Guard role called sg_role_1. Again, you can use any name here, doesn’t need to start with “sg_”.

If applicable to your use case, you can also use wildcards here like:

sg_role_1:

backendroles:

  • jwt_role_*

This would map all backend roles from the JWT starting with “jwt_role_” to the Search Guard role sg_role_1.

Then, in sg_roles, defined the permissions for your Search Guard roles:

sg_role_1:

cluster:

  • …

indices:

…

For Search Guard 6:

You can use the same approach as with Search Guard 5, but you can also use the roles from your JWT directy:

In elasticsearch.yml, configure the mapping mode:

searchguard.roles_mapping_resolution: BOTH

Then, simply configure the permissions in sg_roles.yml by directly using the backend roles you have in your JWT:

jwt_role_1:

cluster:

  • …

indices:

…

···

On Wednesday, November 29, 2017 at 11:30:56 AM UTC+1, Paul Azad wrote:

Hi

Any chance of getting a reply?

Thanks

On Monday, November 20, 2017 at 10:34:31 AM UTC+11, Paul Azad wrote:

Hi

Thanks, so from this i take the following takeaways:

sg_internal_users

  • not needed at all

sg_roles:

  • has the sg roles defined
  • sg roles are what is evaluated by SG to check authorization.

sg_roles_mapping

  • has the mapping between the SG role and the backend role
  • backend roles are just a grouping, and does not need to be used.

For JWT,

Do backend roles need to be used, or can we just specify SG roles?

Do SG roles have to start with sg_ ?

Can backend roles start with sg_ ?

Our use case is that we we will want to pass the username & a role from JWT, with the least configuration on the ELK host as possible.

So if i send username ‘user1’, and role ‘role1’, (but also have user 2,3, 4 , etc etc, and have role1, 2 3 etc etc) i would like to know which configuration i need to do, either via modifying the yml files, or if this PoC goes well, we will look at using the SG REST API to do it.

On Monday, November 13, 2017 at 1:54:10 PM UTC+11, Paul Azad wrote:

HI

i am still trying to work through our PoC with SG, using JWT.

I have now been able to authenticate through JWT, but although we are passing a role through, and can see it in the ES logs, its not evaluated. If i add the username to the sg_roles_mapping.yml then it works

sg_roles_mapping.yml

00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa:

users:

  • 2f55955c816744f4b4d009a1ee774ffa

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa shows this, you can see that the role is mapped from the sg_roles_mapping file.

[2017-11-13T02:44:06,014][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? false (cache size: 2)

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] 2f55955c816744f4b4d009a1ee774ffa not cached, return from noop backend directly

[2017-11-13T02:44:06,019][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa, sg_own_index, sg_public]

[2017-11-13T02:44:06,020][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa.?kibana’, evaluate other roles

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers 00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa=>

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:44:06,021][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:44:06,022][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

If i remove the line from that mapping, although the role is still sent through the JWT token, but its not evaluated.

tail -f /mnt/logs/elasticsearch.log | grep 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘2f55955c816744f4b4d009a1ee774ffa’ is in cache? true (cache size: 4)

[2017-11-13T02:40:38,300][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]’ is authenticated

[2017-11-13T02:40:38,300][DEBUG][c.f.s.c.PrivilegesEvaluator] evaluate permissions for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for 2f55955c816744f4b4d009a1ee774ffa: [sg_own_index, sg_public]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve and match 2f55955c816744f4b4d009a1ee774ffa

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no permittedAliasesIndex ‘2f55955c816744f4b4d009a1ee774ffa’ found for ‘indices:data/read/search’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] permittedAliasesIndices ‘{2f55955c816744f4b4d009a1ee774ffa=org.elasticsearch.common.settings.Settings@37b8cba5}’ → ‘{*.0=INDICES_ALL}’

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved permitted aliases indices for 2f55955c816744f4b4d009a1ee774ffa: [2f55955c816744f4b4d009a1ee774ffa]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for 2f55955c816744f4b4d009a1ee774ffa, will check now types [*]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action indices:data/read/search against 2f55955c816744f4b4d009a1ee774ffa/: [indices:]

[2017-11-13T02:40:38,301][DEBUG][c.f.s.c.PrivilegesEvaluator] no match 2f55955c816744f4b4d009a1ee774ffa* in [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][DEBUG][c.f.s.c.PrivilegesEvaluator] For index 2f55955c816744f4b4d009a1ee774ffa remaining requested indextype: [IndexType [index=.kibana, type=*]]

[2017-11-13T02:40:38,302][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=2f55955c816744f4b4d009a1ee774ffa, roles=[00f679b4eec54b1ba8662c5895a4237a2f55955c816744f4b4d009a1ee774ffa]] [IndexType [index=.kibana, type=*]] [Action [indices:data/read/search]] [RolesChecked [sg_own_index, sg_public]]