Logstash error with "no permission for [indices:admin/create]

Hello Search Guard team,

I configure Logstash to read logs from RabbitMQ and send to Elasticsearch.

  • Search Guard and Elasticsearch version and Logstash version

24.1, 6.6.0 and 5.5

  • Installed and used enterprise modules, if any

No

  • JVM version and operating system version

1.8

  • Search Guard configuration files

In the attached files, and I configured more on Kibana UI.

sg_logstash config

{

“cluster”: [

“CLUSTER_MONITOR”,

“CLUSTER_COMPOSITE_OPS”,

“indices:admin/template/get”,

“indices:admin/template/put”,

“cluster:admin/ingest/pipeline/get”,

“cluster:admin/ingest/pipeline/put”,

“indices:admin/create”

],

“indices”: {

“logstash-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

},

“a[0-9]{6}-events-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”,

“indices:admin/create”

]

},

beat”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

}

},

“tenants”: {}

}

``

  • Logstash configuration

elasticsearch {

hosts => [“host1:9200”,“host2.81:9200”]

flush_size => 4000

index => “a%{application_id}-events-%{+YYYY.MM.dd}”

document_type => “%{type}”

document_id => “%{fingerprint}”

ssl => true

cacert => “/tmp/intermediate-ca.pem”

user => “logstash”

password => “logstash”

}

``

  • Elasticsearch log messages on debug level

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=, indices=[a200067-events-2019.03.18], allIndices=[a200067-events-2019.03.18], types=[*], isAll()=false, isEmpty()=false] [Action [indices:admin/create]] [RolesChecked [sg_logstash, sg_own_index]]

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No permissions for [indices:admin/create]

``

  • Other installed Elasticsearch or Kibana plugins, if any

No

sg_action_groups.yml (2.27 KB)

sg_config.yml (9.4 KB)

sg_internal_users.yml (1.05 KB)

sg_roles_mapping.yml (548 Bytes)

sg_roles.yml (6.96 KB)

Log stash user does not have the permission to access for indices.
Two way to slice this issue

  1. Use admin username and password instead of log stash username and password in log stash configuration

  2. Provide logstash user have all indices permission.

Thanks

Sundar

···

On 21-Mar-2019, at 9:25 AM, Worapoj Chokeanankun worapojc@gmail.com wrote:

Hello Search Guard team,

I configure Logstash to read logs from RabbitMQ and send to Elasticsearch.

  • Search Guard and Elasticsearch version and Logstash version

24.1, 6.6.0 and 5.5

  • Installed and used enterprise modules, if any

No

  • JVM version and operating system version

1.8

  • Search Guard configuration files

In the attached files, and I configured more on Kibana UI.

sg_logstash config

{

“cluster”: [

“CLUSTER_MONITOR”,

“CLUSTER_COMPOSITE_OPS”,

“indices:admin/template/get”,

“indices:admin/template/put”,

“cluster:admin/ingest/pipeline/get”,

“cluster:admin/ingest/pipeline/put”,

“indices:admin/create”

],

“indices”: {

“logstash-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

},

“a[0-9]{6}-events-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”,

“indices:admin/create”

]

},

beat”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

}

},

“tenants”: {}

}

``

  • Logstash configuration

elasticsearch {

hosts => [“host1:9200”,“host2.81:9200”]

flush_size => 4000

index => “a%{application_id}-events-%{+YYYY.MM.dd}”

document_type => “%{type}”

document_id => “%{fingerprint}”

ssl => true

cacert => “/tmp/intermediate-ca.pem”

user => “logstash”

password => “logstash”

}

``

  • Elasticsearch log messages on debug level

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=, indices=[a200067-events-2019.03.18], allIndices=[a200067-events-2019.03.18], types=[*], isAll()=false, isEmpty()=false] [Action [indices:admin/create]] [RolesChecked [sg_logstash, sg_own_index]]

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No permissions for [indices:admin/create]

``

  • Other installed Elasticsearch or Kibana plugins, if any

No

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/37afb255-aff5-4bf7-876e-a39e8a86c46d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

<sg_action_groups.yml>

<sg_config.yml>

<sg_internal_users.yml>

<sg_roles_mapping.yml>

<sg_roles.yml>

Thanks Sundar.

I see that the Elasticsearch log shows No permission for [indices:admin/create].

This indices permission already in CREATE_INDEX action group and in the indices permission level.

What’s kind of permission to access for indices? Could you give me an example?

Your trying to create index starts with a%{application_id}-events-%{+YYYY.MM.dd}"from logstash output section but logstash user doesn’t have access. Logstash user has only access for index starts with logstash and beats.

For logstash and beats

sg_logstash:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- indices:admin/template/get
- indices:admin/template/put
- cluster:admin/ingest/pipeline/put
- cluster:admin/ingest/pipeline/get
indices:
‘logstash-*’:
':
- CRUD
- CREATE_INDEX
beat’:
'
’:
- CRUD
- CREATE_INDEX

Append below highlighted content under sg_losgatsh section to in sg_roles.yaml.

``
sg_logstash:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- indices:admin/template/get
- indices:admin/template/put
- cluster:admin/ingest/pipeline/put
- cluster:admin/ingest/pipeline/get
indices:
‘logstash-':
'
’:
- CRUD
- CREATE_INDEX
beat’:
‘*’:
- CRUD
- CREATE_INDEX

-events-’:

‘*’:
- CRUD
- CREATE_INDEX

``

···

On Thu, Mar 21, 2019 at 10:00 AM Worapoj Chokeanankun worapojc@gmail.com wrote:

Thanks Sundar.

I see that the Elasticsearch log shows No permission for [indices:admin/create].

This indices permission already in CREATE_INDEX action group and in the indices permission level.

What’s kind of permission to access for indices? Could you give me an example?

On Thursday, 21 March 2019 11:09:31 UTC+7, sundar reddy wrote:

Log stash user does not have the permission to access for indices.
Two way to slice this issue

  1. Use admin username and password instead of log stash username and password in log stash configuration
  1. Provide logstash user have all indices permission.

Thanks

Sundar

On 21-Mar-2019, at 9:25 AM, Worapoj Chokeanankun wora...@gmail.com wrote:

Hello Search Guard team,

I configure Logstash to read logs from RabbitMQ and send to Elasticsearch.

  • Search Guard and Elasticsearch version and Logstash version

24.1, 6.6.0 and 5.5

  • Installed and used enterprise modules, if any

No

  • JVM version and operating system version

1.8

  • Search Guard configuration files

In the attached files, and I configured more on Kibana UI.

sg_logstash config

{

“cluster”: [

“CLUSTER_MONITOR”,

“CLUSTER_COMPOSITE_OPS”,

“indices:admin/template/get”,

“indices:admin/template/put”,

“cluster:admin/ingest/pipeline/get”,

“cluster:admin/ingest/pipeline/put”,

“indices:admin/create”

],

“indices”: {

“logstash-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

},

“a[0-9]{6}-events-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”,

“indices:admin/create”

]

},

beat”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

}

},

“tenants”: {}

}

``

  • Logstash configuration

elasticsearch {

hosts => [“host1:9200”,“host2.81:9200”]

flush_size => 4000

index => “a%{application_id}-events-%{+YYYY.MM.dd}”

document_type => “%{type}”

document_id => “%{fingerprint}”

ssl => true

cacert => “/tmp/intermediate-ca.pem”

user => “logstash”

password => “logstash”

}

``

  • Elasticsearch log messages on debug level

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No index-level perm match for User [name=logstash, roles=[logstash], requestedTenant=null] Resolved [aliases=, indices=[a200067-events-2019.03.18], allIndices=[a200067-events-2019.03.18], types=[*], isAll()=false, isEmpty()=false] [Action [indices:admin/create]] [RolesChecked [sg_logstash, sg_own_index]]

[2019-03-21T03:46:07,555][INFO ][c.f.s.p.PrivilegesEvaluator] [host1-ingest-node-0]No permissions for [indices:admin/create]

``

  • Other installed Elasticsearch or Kibana plugins, if any

No

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/37afb255-aff5-4bf7-876e-a39e8a86c46d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

<sg_action_groups.yml>

<sg_config.yml>

<sg_internal_users.yml>

<sg_roles_mapping.yml>

<sg_roles.yml>

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/c4dce817-dcc1-4f24-a1a6-be157a730b26%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

The config is just the initializing. I re-configured it in Kibana UI and I already posted it in my first post.

sg_logstash (from Kibana UI)

{

“cluster”: [

“CLUSTER_MONITOR”,

“CLUSTER_COMPOSITE_OPS”,

“indices:admin/template/get”,

“indices:admin/template/put”,

“cluster:admin/ingest/pipeline/get”,

“cluster:admin/ingest/pipeline/put”,

“indices:admin/create”

],

“indices”: {

“logstash-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

},

“a[0-9]{6}-events-*”: {

“*”: [

“CRUD”,

“CREATE_INDEX”,

“indices:admin/create”

]

},

beat”: {

“*”: [

“CRUD”,

“CREATE_INDEX”

]

}

},

“tenants”: {}

}

You can see a[0-9]{6}-events-* for sg_logstash role. For example, a%{application_id}-events-%{+YYYY.mm.dd} will be a200999-events-2019.03.21. You can see in the log.