unable to create new indices after installing searchguard

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version 6.3.2-23.2

hi

after installing searchguard. i am not able to creted new indices in elasticsearch using logstash.

even when i list indices it only shows the indices which were created before installing searchguard plugin on elasticsearch

it gives this error

bash-4.4# curl -XGET “http://elasticsearch:9200/_cat/indices?pretty” -u logstash -p

Enter host password for user ‘logstash’:

{

“error” : {

“root_cause” : [

{

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

}

],

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

},

“status” : 403

}

my logstash.conf file looks like-

output {

if [namespace] != “kube-system” {

elasticsearch {

hosts => [“http://elasticsearch:9200/”]

index => “logstash-%{+YYYY.MM.dd}”

user => logstash

password => logstash

}

}

}

Check if this user Has create index privilage i have the same isue. Ruby codec on logstash will show you an error

···

wt., 20.11.2018, 10:41: Manjushree Gokhale gokhalemanjushree@gmail.com napisał(a):

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version 6.3.2-23.2

hi

after installing searchguard. i am not able to creted new indices in elasticsearch using logstash.

even when i list indices it only shows the indices which were created before installing searchguard plugin on elasticsearch

it gives this error

bash-4.4# curl -XGET “http://elasticsearch:9200/_cat/indices?pretty” -u logstash -p

Enter host password for user ‘logstash’:

{

“error” : {

“root_cause” : [

{

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

}

],

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

},

“status” : 403

}

my logstash.conf file looks like-

output {

if [namespace] != “kube-system” {

elasticsearch {

hosts => [“http://elasticsearch:9200/”]

index => “logstash-%{+YYYY.MM.dd}”

user => logstash

password => logstash

}

}

}

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/ee5609e5-3d12-4a94-8c7f-f4c39892fd95%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

I have given all the access to create indices in sg_roles.yml

For logstash and beats

sg_logstash:

cluster:

  • CLUSTER_MONITOR

  • CLUSTER_COMPOSITE_OPS

  • indices:admin/template/get

  • indices:admin/template/put

  • indices:data/write/bulk*

  • indices:monitor/stats*

indices:

‘logstash-*’:

‘*’:

  • CRUD

  • CREATE_INDEX

  • INDICES_MONITOR

beat’:

‘*’:

  • CRUD

  • CREATE_INDEX

  • INDICES_MONITOR

still i am getting the same error

When debugging permission problems, please always post the ES log file and your roles definition as well. The curl output alone will not tell you which exact index/permission etc. is missing, Thanks.

···

On Tuesday, November 20, 2018 at 10:41:46 AM UTC+1, Manjushree Gokhale wrote:

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version 6.3.2-23.2

hi

after installing searchguard. i am not able to creted new indices in elasticsearch using logstash.

even when i list indices it only shows the indices which were created before installing searchguard plugin on elasticsearch

it gives this error

bash-4.4# curl -XGET “http://elasticsearch:9200/_cat/indices?pretty” -u logstash -p

Enter host password for user ‘logstash’:

{

“error” : {

“root_cause” : [

{

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

}

],

“type” : “security_exception”,

“reason” : “no permissions for [indices:monitor/stats] and User [name=logstash, roles=[logstash], requestedTenant=null]”

},

“status” : 403

}

my logstash.conf file looks like-

output {

if [namespace] != “kube-system” {

elasticsearch {

hosts => [“http://elasticsearch:9200/”]

index => “logstash-%{+YYYY.MM.dd}”

user => logstash

password => logstash

}

}

}

here you can see the logs of ES

[2018-11-20T11:53:12,624][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Type: TRIAL, valid

[2018-11-20T11:53:47,541][INFO ][c.f.s.p.PrivilegesEvaluator] No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=[], indices=[], allIndices=[], types=[], isAll()=true, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_all_user, sg_logstash]]

[2018-11-20T11:53:47,542][INFO ][c.f.s.p.PrivilegesEvaluator] No permissions for [indices:monitor/stats]

[2018-11-20T11:53:53,494][INFO ][c.f.s.p.PrivilegesEvaluator] No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=[], indices=[], allIndices=[], types=[], isAll()=true, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_all_user, sg_logstash]]

[2018-11-20T11:53:53,494][INFO ][c.f.s.p.PrivilegesEvaluator] No permissions for [indices:monitor/stats]

and below mentioned is the role i have created for logstash.

For logstash and beats

sg_logstash:

cluster:

  • CLUSTER_MONITOR

  • CLUSTER_COMPOSITE_OPS

  • indices:admin/template/get

  • indices:admin/template/put

  • indices:data/write/bulk*

  • indices:monitor/stats*

indices:

‘logstash-*’:

‘*’:

  • CRUD

  • CREATE_INDEX

  • INDICES_MONITOR

beat’:

‘*’:

  • CRUD

  • CREATE_INDEX

  • INDICES_MONITOR

···

From the error message you can see that Kibana is actually using a wildcard, means all indices are queried:

2018-11-20T11:53:47,541][INFO ][c.f.s.p.PrivilegesEvaluator] No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=[], indices=[], allIndices=[], types=[], isAll()=true, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_all_user, sg_logstash]]

If you do not have it already, please try to enable the do_not_fail_on_forbidden feature in sg_config.yml and try again:

searchguard:
dynamic:
kibana:
do_not_fail_on_forbidden: true

``

You can read more about this feature here:

https://docs.search-guard.com/latest/kibana-plugin-installation#configuring-elasticsearch-enable-do-not-fail-on-forbidden

···

On Monday, November 26, 2018 at 6:53:16 AM UTC+1, Manjushree Gokhale wrote:

here you can see the logs of ES

[2018-11-20T11:53:12,624][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Type: TRIAL, valid

[2018-11-20T11:53:47,541][INFO ][c.f.s.p.PrivilegesEvaluator] No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=[], indices=[], allIndices=[], types=[], isAll()=true, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_all_user, sg_logstash]]

[2018-11-20T11:53:47,542][INFO ][c.f.s.p.PrivilegesEvaluator] No permissions for [indices:monitor/stats]

[2018-11-20T11:53:53,494][INFO ][c.f.s.p.PrivilegesEvaluator] No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=[], indices=[], allIndices=[], types=[], isAll()=true, isEmpty()=false] [Action [indices:monitor/stats]] [RolesChecked [sg_own_index, sg_all_user, sg_logstash]]

[2018-11-20T11:53:53,494][INFO ][c.f.s.p.PrivilegesEvaluator] No permissions for [indices:monitor/stats]

and below mentioned is the role i have created for logstash.

For logstash and beats

sg_logstash:

cluster:

  • CLUSTER_MONITOR
  • CLUSTER_COMPOSITE_OPS
  • indices:admin/template/get
  • indices:admin/template/put
  • indices:data/write/bulk*
  • indices:monitor/stats*

indices:

‘logstash-*’:

‘*’:

  • CRUD
  • CREATE_INDEX
  • INDICES_MONITOR

beat’:

‘*’:

  • CRUD
  • CREATE_INDEX
  • INDICES_MONITOR

Hi,

I have already made it true.

the problem is related to logstash

in my sg_logstash role logstash is able to access all the indices but it is not creating any indices from the day i have installed searchguard plugin.

···

Hi,

The problem i was facing is now solved.

thank you.

So, what was the problem and how did you solve it? Would be helpful to other users.

···

On Tuesday, November 27, 2018 at 7:45:01 AM UTC+1, Manjushree Gokhale wrote:

Hi,

The problem i was facing is now solved.

thank you.