SG user frustrations - where's the cookbook?

We are running Elasticsearch/Kibana 6.5.4 on Ubuntu 18.04 with SG 6.5.4-24.0

Java™ SE Runtime Environment (build 1.8.0_201-b09)

Java HotSpot™ 64-Bit Server VM (build 25.201-b09, mixed mode)

We received an academic enterprise license, thank you Search Guard.

My goals as far as users are:

There are a couple admins who can be trusted to do whatever needs doing.

There are a handful of operators who should be able to create indices, save searches, create visualizations, and group them into dashboards.

Each operator will have some associates who want to look at what they have done - everything from dashboards to general digging with Discover.

We very much want to provide embedded dashboards in iframes. There is already a reverse proxy host in the mix, but it’s running Apache, not Nginx.

I have read the docs on Elastic’s site, read the docs on Search Guard’s site. I slept on it. Came back and read some more. I get that there is a difference between authentication, which might be off-platform, and the authorization which is internal, but this business of roles and role mappings is maddening. I follow SG instructions to the letter … and get a read only account that can log in, look around, but not see any indexes. Read some more, figure out how to give it -READ to get into everything, but I need to get to this workgroups environment I describe above.

The documentation seems … fragmented … like there is a presumption that anyone reading it has already got some unifying world view based on Elasticsearch training, and they are now going to switch to SG. The only enterprise I have experience with is my own - a couple of systems, and a bunch of social media data, heavy on Twitter and hunting disinformation efforts.

So … if I want to crib from a working system … where would I find the config files? Is there some blog that explores how to do these things?

  • Search Guard and Elasticsearch version

  • Installed and used enterprise modules, if any

  • JVM version and operating system version

  • Search Guard configuration files

  • Elasticsearch log messages on debug level

  • Other installed Elasticsearch or Kibana plugins, if any

First, I’m sorry that you have a hard time using Search Guard. Search Guard is like a swiss army knife regarding security: We support many use cases and technologies, but this versatility also comes with some complexity. For example, configuring an OpenID IdP for Kibana SSO is fundamentally different from, say, LDAP or Kerberos. But I do admit that we can do a better job in describing the most common use cases, and we have already started to work on some cookbooks.

Regarding the roles mapping: This is necessary because depending on what technology you use, backend roles can come from anywhere: For example, LDAP groups, SAML assertions or JWT claims. That’s why we have the concept of mapping those backend roles to Search Guard roles. For example, you may want to map multiple LDAP groups to only one Search Guard role. That’s what the mapping is for.

This is also explained here in more detail:

But judging from your post I assume you do not have an external auth provider in place, and just want to store users in Elasticsearch, right?

In that case, the approach would be:

  1. Create your users in the sg_internalusers.yml file

  2. Since the users should be able to log into Kibana, you can use the built-in kibanauser backend role to assign all Kibana related privileges to the user:

myuser:
hash: …
roles:
- kibanauser

``

(note that this only gives basic Kibana access, it does not define what indices the user is able to see, we do that in a next step)

  1. Because we already have a corresponding entry in the roles mapping file Search Guard ships with, these users will all get assigned to the sg_kibanauser Search Guard role. The mapping looks like:

sg_kibana_user:
backendroles:
- kibanauser

``

This basically says: Assign all user that have the backend role “kibanauser” to the Search Guard role “sg_kibana_user”. In your case, the backend role comes from the internalusers.yml file, but it could as well be an LDAP group or a JWT claim.

  1. You can then use the same approach to assign permissions to your operators. In the internalusers file, assign a second backend role to your users, like:

myuser:
hash: …
roles:
- kibanauser
- operator

``

  1. Map this “operator” backend role to a SG role in the roles mapping file, like:

sg_operators:
backendroles:
- operator

``

  1. This maps all users that have the “operator” role to the sg_operators SG role. In the roles.yml file, you can then define access permissions to indices, and advanced stuff like document and field level security:

sg_operators:
cluster:
- CLUSTER_COMPOSITE_OPS
indices:
‘index-1’:
’:
- READ
‘index-2’:
'
’:
- CRUD
‘index-3’:
‘*’:
- …

``

For the index names, you can also use wildcards and regular expressions.

For a list of permissions you can use in the roles.yml file please have a look at:

If you run into permission problems, there are two troubleshooting guides which will help you pinpoint the issue:

Finally, it would be super helpful to understand where you struggled most, so we can work on improvinf the docs.

···

On Wednesday, March 6, 2019 at 7:06:17 AM UTC+1, Neal Rauhauser wrote:

We are running Elasticsearch/Kibana 6.5.4 on Ubuntu 18.04 with SG 6.5.4-24.0

Java™ SE Runtime Environment (build 1.8.0_201-b09)

Java HotSpot™ 64-Bit Server VM (build 25.201-b09, mixed mode)

We received an academic enterprise license, thank you Search Guard.

My goals as far as users are:

There are a couple admins who can be trusted to do whatever needs doing.

There are a handful of operators who should be able to create indices, save searches, create visualizations, and group them into dashboards.

Each operator will have some associates who want to look at what they have done - everything from dashboards to general digging with Discover.

We very much want to provide embedded dashboards in iframes. There is already a reverse proxy host in the mix, but it’s running Apache, not Nginx.

I have read the docs on Elastic’s site, read the docs on Search Guard’s site. I slept on it. Came back and read some more. I get that there is a difference between authentication, which might be off-platform, and the authorization which is internal, but this business of roles and role mappings is maddening. I follow SG instructions to the letter … and get a read only account that can log in, look around, but not see any indexes. Read some more, figure out how to give it -READ to get into everything, but I need to get to this workgroups environment I describe above.

The documentation seems … fragmented … like there is a presumption that anyone reading it has already got some unifying world view based on Elasticsearch training, and they are now going to switch to SG. The only enterprise I have experience with is my own - a couple of systems, and a bunch of social media data, heavy on Twitter and hunting disinformation efforts.

So … if I want to crib from a working system … where would I find the config files? Is there some blog that explores how to do these things?

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

Jochen,

I have many diverse duties, so I just got a chance to try this last night - the creation of the ‘operator’ piece seems to be what was missing. I don’t fully understand what I did there, but it’s running AND I got multi-tenant working just a bit ago.

I would say that if you can do just one thing to improve things, a diagram showing where the various parts fit would be the best.

And after that, maybe some sets of config files that are commented for just this one task - getting a system running on internal auth and more than just the “everybody is admin” security model. If the demo setup came with some users, action groups, etc, so we could crib from that it would be very helpful.

Neal

···

Wire:nrauhauser

sms:202-642-1717

mailto:nrauhauser@gmail.com//

Neal, thanks for your input, much appreciated. I have already created a couple of tickets which will address the points you brought up. Diagrams and commented sample configs will be helpful, agreed.

···

On Monday, March 18, 2019 at 3:10:42 AM UTC+1, Neal Rauhauser wrote:

Jochen,

I have many diverse duties, so I just got a chance to try this last night - the creation of the ‘operator’ piece seems to be what was missing. I don’t fully understand what I did there, but it’s running AND I got multi-tenant working just a bit ago.

I would say that if you can do just one thing to improve things, a diagram showing where the various parts fit would be the best.

And after that, maybe some sets of config files that are commented for just this one task - getting a system running on internal auth and more than just the “everybody is admin” security model. If the demo setup came with some users, action groups, etc, so we could crib from that it would be very helpful.

Neal

On Tue, Mar 12, 2019 at 12:40 PM Jochen Kressin jkressin@floragunn.com wrote:

First, I’m sorry that you have a hard time using Search Guard. Search Guard is like a swiss army knife regarding security: We support many use cases and technologies, but this versatility also comes with some complexity. For example, configuring an OpenID IdP for Kibana SSO is fundamentally different from, say, LDAP or Kerberos. But I do admit that we can do a better job in describing the most common use cases, and we have already started to work on some cookbooks.

Regarding the roles mapping: This is necessary because depending on what technology you use, backend roles can come from anywhere: For example, LDAP groups, SAML assertions or JWT claims. That’s why we have the concept of mapping those backend roles to Search Guard roles. For example, you may want to map multiple LDAP groups to only one Search Guard role. That’s what the mapping is for.

This is also explained here in more detail:

https://docs.search-guard.com/latest/main-concepts

But judging from your post I assume you do not have an external auth provider in place, and just want to store users in Elasticsearch, right?

In that case, the approach would be:

  1. Create your users in the sg_internalusers.yml file
  1. Since the users should be able to log into Kibana, you can use the built-in kibanauser backend role to assign all Kibana related privileges to the user:

myuser:
hash: …
roles:
- kibanauser

``

(note that this only gives basic Kibana access, it does not define what indices the user is able to see, we do that in a next step)

  1. Because we already have a corresponding entry in the roles mapping file Search Guard ships with, these users will all get assigned to the sg_kibanauser Search Guard role. The mapping looks like:

sg_kibana_user:
backendroles:
- kibanauser

``

This basically says: Assign all user that have the backend role “kibanauser” to the Search Guard role “sg_kibana_user”. In your case, the backend role comes from the internalusers.yml file, but it could as well be an LDAP group or a JWT claim.

  1. You can then use the same approach to assign permissions to your operators. In the internalusers file, assign a second backend role to your users, like:

myuser:
hash: …
roles:
- kibanauser
- operator

``

  1. Map this “operator” backend role to a SG role in the roles mapping file, like:

sg_operators:
backendroles:
- operator

``

  1. This maps all users that have the “operator” role to the sg_operators SG role. In the roles.yml file, you can then define access permissions to indices, and advanced stuff like document and field level security:

sg_operators:
cluster:
- CLUSTER_COMPOSITE_OPS
indices:
‘index-1’:
’:
- READ
‘index-2’:
'
’:
- CRUD
‘index-3’:
‘*’:
- …

``

For the index names, you can also use wildcards and regular expressions.

For a list of permissions you can use in the roles.yml file please have a look at:

https://docs.search-guard.com/latest/action-groups

If you run into permission problems, there are two troubleshooting guides which will help you pinpoint the issue:

https://docs.search-guard.com/latest/troubleshooting-search-guard-user-roles

https://docs.search-guard.com/latest/troubleshooting-search-guard-permission

Finally, it would be super helpful to understand where you struggled most, so we can work on improvinf the docs.

On Wednesday, March 6, 2019 at 7:06:17 AM UTC+1, Neal Rauhauser wrote:

We are running Elasticsearch/Kibana 6.5.4 on Ubuntu 18.04 with SG 6.5.4-24.0

Java™ SE Runtime Environment (build 1.8.0_201-b09)

Java HotSpot™ 64-Bit Server VM (build 25.201-b09, mixed mode)

We received an academic enterprise license, thank you Search Guard.

My goals as far as users are:

There are a couple admins who can be trusted to do whatever needs doing.

There are a handful of operators who should be able to create indices, save searches, create visualizations, and group them into dashboards.

Each operator will have some associates who want to look at what they have done - everything from dashboards to general digging with Discover.

We very much want to provide embedded dashboards in iframes. There is already a reverse proxy host in the mix, but it’s running Apache, not Nginx.

I have read the docs on Elastic’s site, read the docs on Search Guard’s site. I slept on it. Came back and read some more. I get that there is a difference between authentication, which might be off-platform, and the authorization which is internal, but this business of roles and role mappings is maddening. I follow SG instructions to the letter … and get a read only account that can log in, look around, but not see any indexes. Read some more, figure out how to give it -READ to get into everything, but I need to get to this workgroups environment I describe above.

The documentation seems … fragmented … like there is a presumption that anyone reading it has already got some unifying world view based on Elasticsearch training, and they are now going to switch to SG. The only enterprise I have experience with is my own - a couple of systems, and a bunch of social media data, heavy on Twitter and hunting disinformation efforts.

So … if I want to crib from a working system … where would I find the config files? Is there some blog that explores how to do these things?

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d5f9e473-c1b5-462a-b3e5-8b6b238c3ed3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Wire:nrauhauser

sms:202-642-1717

mailto:nrauhauser@gmail.com//