Search guard - Issue in creating shards

Search Guard
1

Hi All,

ES Version - 6.1.2

Search Guard Version - 6.1.2-20.1

I am facing the below issue after the installation of search guard plugin in elastic search on Kubernetes

[2018-01-22T13:32:58,969][INFO ][o.e.n.Node ] [elasticsearch] started

[2018-01-22T13:32:59,004][INFO ][c.f.s.c.IndexBaseConfigurationRepository] searchguard index does not exist yet, so we create a default config

[2018-01-22T13:32:59,009][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Will create searchguard index so we can apply default config

[2018-01-22T13:32:59,011][INFO ][o.e.g.GatewayService ] [elasticsearch] recovered [0] indices into cluster_state

[2018-01-22T13:32:59,125][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch] [searchguard] creating index, cause [api], templates , shards [1]/[1], mappings

[2018-01-22T13:33:29,169][INFO ][c.f.s.s.ConfigHelper ] Will update ‘config’ with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_config.yml

[2018-01-22T13:34:29,305][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@9e900a] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard)

Below is my elastic search yaml file

···

cluster:

name: dst

node:

master: true

data: true

name: elasticsearch

ingest: true

network.host: 0.0.0.0

path:

data: /data/data

logs: /data/log

discovery:

zen:

ping.unicast.hosts: elasticsearch-discovery

minimum_master_nodes: 3

cluster.routing.allocation.disk.threshold_enabled: false

######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.pemcert_filepath: client.pem

searchguard.ssl.transport.pemkey_filepath: client-key.pem

searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

#searchguard.ssl.http.enabled: true

#searchguard.ssl.http.pemcert_filepath: esnode.pem

#searchguard.ssl.http.pemkey_filepath: esnode-key.pem

#searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

searchguard.allow_unsafe_democertificates: true

searchguard.allow_default_init_sgindex: true

searchguard.authcz.admin_dn:

  • CN=kirk,OU=IT,O=IBM,L=Dallas,ST=TX,C=US

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

searchguard.enterprise_modules_enabled: false

node.max_local_storage_nodes: 3

######## End Search Guard Demo Configuration ########

2

We need a little bit more infos here:

- Does your setup work without Search Guard?
- How many elasticsearch nodes you have?
- Can you post a more complete log file?
- Why did you set "cluster.routing.allocation.disk.threshold_enabled" to "false"?
- Does it work on non-kubernetes environment?

···

Am 22.01.2018 um 15:14 schrieb Senthil kumar.R <rskit1986@gmail.com>:

Hi All,

ES Version - 6.1.2
Search Guard Version - 6.1.2-20.1

        I am facing the below issue after the installation of search guard plugin in elastic search on Kubernetes

[2018-01-22T13:32:58,969][INFO ][o.e.n.Node ] [elasticsearch] started
[2018-01-22T13:32:59,004][INFO ][c.f.s.c.IndexBaseConfigurationRepository] searchguard index does not exist yet, so we create a default config
[2018-01-22T13:32:59,009][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Will create searchguard index so we can apply default config
[2018-01-22T13:32:59,011][INFO ][o.e.g.GatewayService ] [elasticsearch] recovered [0] indices into cluster_state
[2018-01-22T13:32:59,125][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch] [searchguard] creating index, cause [api], templates , shards [1]/[1], mappings
[2018-01-22T13:33:29,169][INFO ][c.f.s.s.ConfigHelper ] Will update 'config' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_config.yml

[2018-01-22T13:34:29,305][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@9e900a] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard)

Below is my elastic search yaml file
----------------------------------------------------------

cluster:
    name: dst

node:
    master: true
    data: true
    name: elasticsearch
    ingest: true

network.host: 0.0.0.0

path:
    data: /data/data
    logs: /data/log

discovery:
    zen:
# ping.unicast.hosts: elasticsearch-discovery
        minimum_master_nodes: 3

cluster.routing.allocation.disk.threshold_enabled: false

######## Start Search Guard Demo Configuration ########
# WARNING: revise all the lines below before you go into production
searchguard.ssl.transport.pemcert_filepath: client.pem
searchguard.ssl.transport.pemkey_filepath: client-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
#searchguard.ssl.http.enabled: true
#searchguard.ssl.http.pemcert_filepath: esnode.pem
#searchguard.ssl.http.pemkey_filepath: esnode-key.pem
#searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:
  - CN=kirk,OU=IT,O=IBM,L=Dallas,ST=TX,C=US

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
searchguard.enterprise_modules_enabled: false
node.max_local_storage_nodes: 3
######## End Search Guard Demo Configuration ########

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d9a48f59-be2e-4ed3-a31d-e8d39b0325a1%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

3

We need a little bit more infos here:

  • Does your setup work without Search Guard?

  • How many elasticsearch nodes you have?

  • Can you post a more complete log file?

  • Why did you set “cluster.routing.allocation.disk.threshold_enabled” to “false”?

  • Does it work on non-kubernetes environment?

Hi All,

ES Version - 6.1.2

Search Guard Version - 6.1.2-20.1

    I am facing the below issue after the installation of search guard plugin in elastic search on Kubernetes

[2018-01-22T13:32:58,969][INFO ][o.e.n.Node ] [elasticsearch] started

[2018-01-22T13:32:59,004][INFO ][c.f.s.c.IndexBaseConfigurationRepository] searchguard index does not exist yet, so we create a default config

[2018-01-22T13:32:59,009][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Will create searchguard index so we can apply default config

[2018-01-22T13:32:59,011][INFO ][o.e.g.GatewayService ] [elasticsearch] recovered [0] indices into cluster_state

[2018-01-22T13:32:59,125][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch] [searchguard] creating index, cause [api], templates , shards [1]/[1], mappings

[2018-01-22T13:33:29,169][INFO ][c.f.s.s.ConfigHelper ] Will update ‘config’ with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_config.yml

[2018-01-22T13:34:29,305][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@9e900a] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard)

Below is my elastic search yaml file

cluster:

name: dst

node:

master: true

data: true

name: elasticsearch

ingest: true

network.host: 0.0.0.0

path:

data: /data/data

logs: /data/log

discovery:

zen:

ping.unicast.hosts: elasticsearch-discovery

    minimum_master_nodes: 3

cluster.routing.allocation.disk.threshold_enabled: false

######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.pemcert_filepath: client.pem

searchguard.ssl.transport.pemkey_filepath: client-key.pem

searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem

searchguard.ssl.transport.enforce_hostname_verification: false

#searchguard.ssl.http.enabled: true

#searchguard.ssl.http.pemcert_filepath: esnode.pem

#searchguard.ssl.http.pemkey_filepath: esnode-key.pem

#searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem

searchguard.allow_unsafe_democertificates: true

searchguard.allow_default_init_sgindex: true

searchguard.authcz.admin_dn:

  • CN=kirk,OU=IT,O=IBM,L=Dallas,ST=TX,C=US

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

searchguard.enterprise_modules_enabled: false

node.max_local_storage_nodes: 3

######## End Search Guard Demo Configuration ########

–
You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d9a48f59-be2e-4ed3-a31d-e8d39b0325a1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

  • Does your setup work without Search Guard? - Yes
  • How many elasticsearch nodes you have? - 3 master and 2 data nodes
  • Can you post a more complete log file? - attached
  • Why did you set “cluster.routing.allocation.disk.threshold_enabled” to “false”? - setting to true also i tried and it was not working
  • Does it work on non-kubernetes environment? - Yes

es-sg.log (744 KB)

···

Am 22.01.2018 um 15:14 schrieb Senthil kumar.R rski...@gmail.com:

On Monday, 22 January 2018 21:13:50 UTC+5:30, Search Guard wrote:

4

Can you pls try giving more memory to the nodes. -Xmx256m is extrem low and i am wondering ES even starts.
Pls set at least to -Xms2G, -Xmx2G (giving 2 GB mem) and report back if that helps.

How did you setup kubernetes and how did you setup ES on kubernetes? Can you elaborate a bit more on that.
If its really a issue with kubernetes we need to reproduce it and currently we did no tests on kubernetes.
Or maybe you can reproduce the issue in a single docker container, that would be helpful because its easier for us
to set it up on our side.

···

Am 23.01.2018 um 06:27 schrieb Senthil kumar.R <rskit1986@gmail.com>:

- Does your setup work without Search Guard? - Yes
- How many elasticsearch nodes you have? - 3 master and 2 data nodes
- Can you post a more complete log file? - attached
- Why did you set "cluster.routing.allocation.disk.threshold_enabled" to "false"? - setting to true also i tried and it was not working
- Does it work on non-kubernetes environment? - Yes

On Monday, 22 January 2018 21:13:50 UTC+5:30, Search Guard wrote:
We need a little bit more infos here:

- Does your setup work without Search Guard?
- How many elasticsearch nodes you have?
- Can you post a more complete log file?
- Why did you set "cluster.routing.allocation.disk.threshold_enabled" to "false"?
- Does it work on non-kubernetes environment?

> Am 22.01.2018 um 15:14 schrieb Senthil kumar.R <rski...@gmail.com>:
>
> Hi All,
>
> ES Version - 6.1.2
> Search Guard Version - 6.1.2-20.1
>
> I am facing the below issue after the installation of search guard plugin in elastic search on Kubernetes
>
> [2018-01-22T13:32:58,969][INFO ][o.e.n.Node ] [elasticsearch] started
> [2018-01-22T13:32:59,004][INFO ][c.f.s.c.IndexBaseConfigurationRepository] searchguard index does not exist yet, so we create a default config
> [2018-01-22T13:32:59,009][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Will create searchguard index so we can apply default config
> [2018-01-22T13:32:59,011][INFO ][o.e.g.GatewayService ] [elasticsearch] recovered [0] indices into cluster_state
> [2018-01-22T13:32:59,125][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch] [searchguard] creating index, cause [api], templates , shards [1]/[1], mappings
> [2018-01-22T13:33:29,169][INFO ][c.f.s.s.ConfigHelper ] Will update 'config' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_config.yml
>
> [2018-01-22T13:34:29,305][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@9e900a] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard)
>
>
>
> Below is my elastic search yaml file
> ----------------------------------------------------------
>
>
> cluster:
> name: dst
>
> node:
> master: true
> data: true
> name: elasticsearch
> ingest: true
>
> network.host: 0.0.0.0
>
> path:
> data: /data/data
> logs: /data/log
>
> discovery:
> zen:
> # ping.unicast.hosts: elasticsearch-discovery
> minimum_master_nodes: 3
>
> cluster.routing.allocation.disk.threshold_enabled: false
>
> ######## Start Search Guard Demo Configuration ########
> # WARNING: revise all the lines below before you go into production
> searchguard.ssl.transport.pemcert_filepath: client.pem
> searchguard.ssl.transport.pemkey_filepath: client-key.pem
> searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
> searchguard.ssl.transport.enforce_hostname_verification: false
> #searchguard.ssl.http.enabled: true
> #searchguard.ssl.http.pemcert_filepath: esnode.pem
> #searchguard.ssl.http.pemkey_filepath: esnode-key.pem
> #searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
> searchguard.allow_unsafe_democertificates: true
> searchguard.allow_default_init_sgindex: true
> searchguard.authcz.admin_dn:
> - CN=kirk,OU=IT,O=IBM,L=Dallas,ST=TX,C=US
>
> searchguard.audit.type: internal_elasticsearch
> searchguard.enable_snapshot_restore_privilege: true
> searchguard.check_snapshot_restore_write_privileges: true
> searchguard.restapi.roles_enabled: ["sg_all_access"]
> searchguard.enterprise_modules_enabled: false
> node.max_local_storage_nodes: 3
> ######## End Search Guard Demo Configuration ########
>
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d9a48f59-be2e-4ed3-a31d-e8d39b0325a1%40googlegroups.com\.
> For more options, visit https://groups.google.com/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8742ba23-4755-4ec7-8db9-1f8bef888c83%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.
<es-sg.log>