Warning logs

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version

  • Used enterprise modules, if any

  • JVM version and operating system version

  • Search Guard configuration files

  • Elasticsearch log messages on debug level

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.
ES 5.4.2

S-G-5 : search-guard-5:5.4.2-12

java version “1.8.0_131”

  1. What is the proper way to setup search guard on a production instance without the use of ‘install_demo_configuration.sh’ , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.

  2. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node ] [elasticsearch-app-dev-fpjs] started

[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid

[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state

[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node ‘elasticsearch-app-dev-fpjs’ initialized

[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] …]).

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

``

Thanks,

Roman

Also wrapping my head around with the monitoring exporter:

[2017-08-01T18:35:48,341][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1724/1875992075@318ffaf2

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-*?filter_path=*.aliases: HTTP/1.1 403 Forbidden

{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354) ~[?:?]

at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343) ~[?:?]

at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[?:?]

at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[?:?]

at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[?:?]

at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[?:?]

at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]

at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[?:?]

at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[?:?]

at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]

at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]

at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

``

The following syntax seems not work

sg_monitor:

cluster:

  • “cluster:admin/xpack/monitoring/*”

  • “indices:admin/template/get”

  • “indices:admin/template/put”

  • “indices:admin/*get”

  • “cluster:admin/ingest/pipeline/get”

  • CLUSTER_MONITOR

  • CLUSTER_COMPOSITE_OPS

indices:

monitoring’:

‘*’:

  • INDICES_ALL

marvel-es’:

‘*’:

  • INDICES_ALL

``

What works for me is :

sg_monitor:

cluster:

  • “cluster:admin/xpack/monitoring/*”

  • “indices:admin/template/get”

  • “indices:admin/template/put”

  • “indices:admin/*get”

  • “cluster:admin/ingest/pipeline/get”

  • CLUSTER_MONITOR

  • CLUSTER_COMPOSITE_OPS

indices:

‘*’:

‘*’:

  • INDICES_ALL

``

···

On Tuesday, August 1, 2017 at 11:25:10 AM UTC-7, Roman Kournjaev wrote:

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.
ES 5.4.2

S-G-5 : search-guard-5:5.4.2-12

java version “1.8.0_131”

  1. What is the proper way to setup search guard on a production instance without the use of ‘install_demo_configuration.sh’ , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.
  1. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node ] [elasticsearch-app-dev-fpjs] started

[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid

[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state

[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node ‘elasticsearch-app-dev-fpjs’ initialized

[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] …]).

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

``

Thanks,

Roman

In you case you can ignore these two log messages:

- Not yet initialized (you may need to run sgadmin)
- _all does not exist in cluster metadata

The proper way to setup SSL for production is to use your own PKI. If you do not have one you maybe want to establish one (depends if you maybe need SSL certificates in the future for other services as well).
If you do not have a PKI and want not setup a company wide one then you can you our scripts also for production, but you do it on your own risk. It depends a bit wether your elasticsearch cluster is exposed to the public or if you can install root certifcates into the browsers of your users.. If its public or if you cannot install root certificates into the browsers you can of course also buy a commercial SSL certificate from verisign, thawte, ... or go with letsencrypt for free.

To make xpack monitoring work pls update to SG 14. This should work then out of the box. See also https://github.com/floragunncom/search-guard/blob/ves-5.4.3-14/sgconfig/sg_roles.yml

···

Am 01.08.2017 um 20:39 schrieb Roman Kournjaev <kournjaev@gmail.com>:

Also wrapping my head around with the monitoring exporter:

[2017-08-01T18:35:48,341][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1724/1875992075@318ffaf2
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-*?filter_path=*.aliases: HTTP/1.1 403 Forbidden
{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for indices:admin/get"}],"type":"security_exception","reason":"no permissions for indices:admin/get"},"status":403}
  at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354) ~[?:?]
  at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343) ~[?:?]
  at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[?:?]
  at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[?:?]
  at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[?:?]
  at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[?:?]
  at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]
  at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[?:?]
  at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[?:?]
  at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[?:?]
  at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]
  at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
  at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]
  at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

The following syntax seems not work

sg_monitor:
  cluster:
    - "cluster:admin/xpack/monitoring/*"
    - "indices:admin/template/get"
    - "indices:admin/template/put"
    - "indices:admin/*get"
    - "cluster:admin/ingest/pipeline/get"
    - CLUSTER_MONITOR
    - CLUSTER_COMPOSITE_OPS
  indices:
    '*monitoring*':
      '*':
        - INDICES_ALL
    '*marvel-es*':
      '*':
        - INDICES_ALL

What works for me is :

sg_monitor:
  cluster:
    - "cluster:admin/xpack/monitoring/*"
    - "indices:admin/template/get"
    - "indices:admin/template/put"
    - "indices:admin/*get"
    - "cluster:admin/ingest/pipeline/get"
    - CLUSTER_MONITOR
    - CLUSTER_COMPOSITE_OPS
  indices:
    '*':
      '*':
        - INDICES_ALL

On Tuesday, August 1, 2017 at 11:25:10 AM UTC-7, Roman Kournjaev wrote:

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.
ES 5.4.2
S-G-5 : search-guard-5:5.4.2-12
java version "1.8.0_131"

1. What is the proper way to setup search guard on a production instance without the use of 'install_demo_configuration.sh' , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.

2. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node ] [elasticsearch-app-dev-fpjs] started
[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)
[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid
[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state
[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)
[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node 'elasticsearch-app-dev-fpjs' initialized
[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] ...]).
[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata
[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

Thanks,
Roman

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/f3aaa0d2-bb30-416c-bd04-f49b513aac76%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

Thanks,
Any way I can get rid of the ‘_all does not exist in cluster metadata’ log line ? It practically writes this line every second , and i would like to keep my log lines clean.

I guess setting a specifc logger only to ‘ERROR’ level would do the job.

···

On Wednesday, August 2, 2017 at 4:36:16 AM UTC-4, Search Guard wrote:

In you case you can ignore these two log messages:

  • Not yet initialized (you may need to run sgadmin)

  • _all does not exist in cluster metadata

The proper way to setup SSL for production is to use your own PKI. If you do not have one you maybe want to establish one (depends if you maybe need SSL certificates in the future for other services as well).

If you do not have a PKI and want not setup a company wide one then you can you our scripts also for production, but you do it on your own risk. It depends a bit wether your elasticsearch cluster is exposed to the public or if you can install root certifcates into the browsers of your users… If its public or if you cannot install root certificates into the browsers you can of course also buy a commercial SSL certificate from verisign, thawte, … or go with letsencrypt for free.

To make xpack monitoring work pls update to SG 14. This should work then out of the box. See also https://github.com/floragunncom/search-guard/blob/ves-5.4.3-14/sgconfig/sg_roles.yml

Am 01.08.2017 um 20:39 schrieb Roman Kournjaev kour...@gmail.com:

Also wrapping my head around with the monitoring exporter:

[2017-08-01T18:35:48,341][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1724/1875992075@318ffaf2

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden

{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354) ~[?:?]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343) ~[?:?]
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[?:?]
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[?:?]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[?:?]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[?:?]
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[?:?]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[?:?]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

The following syntax seems not work

sg_monitor:

cluster:

- "cluster:admin/xpack/monitoring/*"
- "indices:admin/template/get"
- "indices:admin/template/put"
- "indices:admin/*get"
- "cluster:admin/ingest/pipeline/get"
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS

indices:

'*monitoring*':
  '*':
    - INDICES_ALL
'*marvel-es*':
  '*':
    - INDICES_ALL

What works for me is :

sg_monitor:

cluster:

- "cluster:admin/xpack/monitoring/*"
- "indices:admin/template/get"
- "indices:admin/template/put"
- "indices:admin/*get"
- "cluster:admin/ingest/pipeline/get"
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS

indices:

'*':
  '*':
    - INDICES_ALL

On Tuesday, August 1, 2017 at 11:25:10 AM UTC-7, Roman Kournjaev wrote:

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.

ES 5.4.2

S-G-5 : search-guard-5:5.4.2-12

java version “1.8.0_131”

  1. What is the proper way to setup search guard on a production instance without the use of ‘install_demo_configuration.sh’ , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.
  1. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node ] [elasticsearch-app-dev-fpjs] started

[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid

[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state

[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node ‘elasticsearch-app-dev-fpjs’ initialized

[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] …]).

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

Thanks,

Roman


You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/f3aaa0d2-bb30-416c-bd04-f49b513aac76%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Adding this to conf/log4j2.properties should to the trick:

logger.pe.name = com.floragunn.searchguard.configuration.PrivilegesEvaluator

logger.pe.level = error

···

On Thursday, August 3, 2017 at 8:12:47 PM UTC+2, Roman Kournjaev wrote:

Thanks,
Any way I can get rid of the ‘_all does not exist in cluster metadata’ log line ? It practically writes this line every second , and i would like to keep my log lines clean.

I guess setting a specifc logger only to ‘ERROR’ level would do the job.

On Wednesday, August 2, 2017 at 4:36:16 AM UTC-4, Search Guard wrote:

In you case you can ignore these two log messages:

  • Not yet initialized (you may need to run sgadmin)

  • _all does not exist in cluster metadata

The proper way to setup SSL for production is to use your own PKI. If you do not have one you maybe want to establish one (depends if you maybe need SSL certificates in the future for other services as well).

If you do not have a PKI and want not setup a company wide one then you can you our scripts also for production, but you do it on your own risk. It depends a bit wether your elasticsearch cluster is exposed to the public or if you can install root certifcates into the browsers of your users… If its public or if you cannot install root certificates into the browsers you can of course also buy a commercial SSL certificate from verisign, thawte, … or go with letsencrypt for free.

To make xpack monitoring work pls update to SG 14. This should work then out of the box. See also https://github.com/floragunncom/search-guard/blob/ves-5.4.3-14/sgconfig/sg_roles.yml

Am 01.08.2017 um 20:39 schrieb Roman Kournjaev kour...@gmail.com:

Also wrapping my head around with the monitoring exporter:

[2017-08-01T18:35:48,341][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1724/1875992075@318ffaf2

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden

{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354) ~[?:?]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343) ~[?:?]
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[?:?]
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[?:?]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[?:?]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[?:?]
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[?:?]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[?:?]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

The following syntax seems not work

sg_monitor:

cluster:

- "cluster:admin/xpack/monitoring/*"
- "indices:admin/template/get"
- "indices:admin/template/put"
- "indices:admin/*get"
- "cluster:admin/ingest/pipeline/get"
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS

indices:

'*monitoring*':
  '*':
    - INDICES_ALL
'*marvel-es*':
  '*':
    - INDICES_ALL

What works for me is :

sg_monitor:

cluster:

- "cluster:admin/xpack/monitoring/*"
- "indices:admin/template/get"
- "indices:admin/template/put"
- "indices:admin/*get"
- "cluster:admin/ingest/pipeline/get"
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS

indices:

'*':
  '*':
    - INDICES_ALL

On Tuesday, August 1, 2017 at 11:25:10 AM UTC-7, Roman Kournjaev wrote:

I have finally set up search-guard, though it was quite tricky todo , since we provision our ES instances with CHEF scripts.

ES 5.4.2

S-G-5 : search-guard-5:5.4.2-12

java version “1.8.0_131”

  1. What is the proper way to setup search guard on a production instance without the use of ‘install_demo_configuration.sh’ , since it explicitly says not to use it on production env. What I would do is to run it and then run some scripts to update the updated elasticsearch.yml and alter all the sg_config files , not sure thats the way the develop intended it to be, but it works.
  1. I have some strange log lines that you might help me explain:

[2017-08-01T18:13:42,984][INFO ][o.e.n.Node ] [elasticsearch-app-dev-fpjs] started

[2017-08-01T18:13:43,332][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:43,915][INFO ][o.e.l.LicenseService ] [elasticsearch-app-dev-fpjs] license [0c10aaae-9db8-40ed-9ccf-2cf7421e8a3b] mode [trial] - valid

[2017-08-01T18:13:43,917][INFO ][o.e.g.GatewayService ] [elasticsearch-app-dev-fpjs] recovered [7] indices into cluster_state

[2017-08-01T18:13:44,159][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)

[2017-08-01T18:13:44,520][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node ‘elasticsearch-app-dev-fpjs’ initialized

[2017-08-01T18:13:44,956][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-app-dev-fpjs] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[products_latest][3], [.monitoring-es-2-2017.08.01][0]] …]).

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,430][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:45,433][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:13:48,942][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:01,552][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

[2017-08-01T18:14:04,555][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

Thanks,

Roman


You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/f3aaa0d2-bb30-416c-bd04-f49b513aac76%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.