SG + ES + X-Pack Monitoring + ActiveDirectory = I need help.

[mcostantini]

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)

• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.

• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[jkressin]

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

···

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcostantini@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[mcostantini]

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

···

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[jkressin]

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

···

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcostantini@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[mcostantini]

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-*?filter_path=*.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

···

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[mcostantini]

Respectful bump. Thank you.

···

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[searchguard_google_group]

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

···

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[jkressin]

We’ve released a preliminary documentation: https://github.com/floragunncom/search-guard-docs/blob/master/monitoring.md

This issue here requires an updated version of the Kibana Plugin, we’re working on it:

https://github.com/floragunncom/search-guard/issues/345

···

On Monday, June 26, 2017 at 5:29:51 PM UTC+2, Search Guard wrote:

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[mcostantini]

Thank you very much for your hard work and that prelim document. On my end, I’ve had some success, but it’s not 100% and I think you may be interested in my observations.

So, to repeat, I am using SG 5.2.2 v12 WITH the enterprise extension.

I was not able to get things going with the sg_monitor role described in the document. I was able to make things work with the sg_monitor role described earlier in this thread, but with the following message being found repeatedly in the console output of ES:
[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=<<>>, roles=] [IndexType [index=_all, type=esqueue]] [Action [indices:data/read/search]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]

[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=esqueue]], sg_own_index=[IndexType [index=_all, type=esqueue]], sg_monitor=[IndexType [index=_all, type=esqueue]], sg_kibana_server=[IndexType [index=_all, type=esqueue]]}

``

However, monitoring in Kibana seems to be working.

Now, if I open up the permissions for the monitoring user entirely (i.e. add him to the sg_all_access role), that message goes away. In fact, I am only left with a warning:
[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

I guess what I’m saying is, with the stack I’m using, the document may not be 100% correct. Also, I’d love your help diagnosing the permission problem.

Many thanks,

Marco.

···

On Thursday, June 29, 2017 at 9:56:32 PM UTC+2, Jochen Kressin wrote:

We’ve released a preliminary documentation: https://github.com/floragunncom/search-guard-docs/blob/master/monitoring.md

This issue here requires an updated version of the Kibana Plugin, we’re working on it:

https://github.com/floragunncom/search-guard/issues/345

On Monday, June 26, 2017 at 5:29:51 PM UTC+2, Search Guard wrote:

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[mcostantini]

Respectful bump. Question distilled: What should I set the permissions to for kibana user in order to avoid the above, but not open up the permissions entirely.

Thank you,

···

On Tuesday, July 4, 2017 at 10:09:23 AM UTC+2, mcost...@np6.com wrote:

Thank you very much for your hard work and that prelim document. On my end, I’ve had some success, but it’s not 100% and I think you may be interested in my observations.

So, to repeat, I am using SG 5.2.2 v12 WITH the enterprise extension.

I was not able to get things going with the sg_monitor role described in the document. I was able to make things work with the sg_monitor role described earlier in this thread, but with the following message being found repeatedly in the console output of ES:
[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=<<>>, roles=] [IndexType [index=_all, type=esqueue]] [Action [indices:data/read/search]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]

[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=esqueue]], sg_own_index=[IndexType [index=_all, type=esqueue]], sg_monitor=[IndexType [index=_all, type=esqueue]], sg_kibana_server=[IndexType [index=_all, type=esqueue]]}

``

However, monitoring in Kibana seems to be working.

Now, if I open up the permissions for the monitoring user entirely (i.e. add him to the sg_all_access role), that message goes away. In fact, I am only left with a warning:
[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

I guess what I’m saying is, with the stack I’m using, the document may not be 100% correct. Also, I’d love your help diagnosing the permission problem.

Many thanks,

Marco.

On Thursday, June 29, 2017 at 9:56:32 PM UTC+2, Jochen Kressin wrote:

We’ve released a preliminary documentation: https://github.com/floragunncom/search-guard-docs/blob/master/monitoring.md

This issue here requires an updated version of the Kibana Plugin, we’re working on it:

https://github.com/floragunncom/search-guard/issues/345

On Monday, June 26, 2017 at 5:29:51 PM UTC+2, Search Guard wrote:

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[jkressin]

Hi Marco,

yes, the docs are geared towards ES > 5.3.0 and SG 14. Plus, we assume that your monitoring user has also the Kibana User role. I guess we should make that clearer in the docs. However, we’ve tested the setup now with:

• ES 5.2.2

• SG 5.2.2-12

• KI 5.2.2

• X-Pack 5.2.2, both on ES and KI

Here’s the role definition that works for us with the said versions:

sg_monitor:

cluster:

• “cluster:admin/xpack/monitoring/*”

• “cluster:admin/ingest/pipeline/*”

• “indices:admin/template/get”

• “indices:admin/template/put”

• “indices:admin/*get”

• CLUSTER_MONITOR

• CLUSTER_COMPOSITE_OPS

indices:

‘?monitoring*’:

‘*’:

• UNLIMITED

‘?marvel*’:

‘*’:

• UNLIMITED

‘*’:

‘*’:

• READ

• indices:admin/get

‘?kibana’:

‘*’:

• INDICES_ALL

The monitoring user needs full control over the monitoring and marvel (for backwards compatibility) indices, and also access to the .kibana index.

Since monitoring affects all indices, the monitoring user must have READ and admin/get permissions on all indices, hence the wildcard.

The last message:

[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

is fixed in later versions of ES/SG, so for the moment with your versions you either need to ignore it, or just set the log level of the PrivilegesEvaluator to ERROR. In config/log4j2.properties, set:

logger.pe.name = com.floragunn.searchguard.configuration.PrivilegesEvaluator

logger.pe.level = error

Let me know if this works for you!

···

On Wednesday, July 19, 2017 at 11:41:00 AM UTC+2, mcostantini@np6.com wrote:

Respectful bump. Question distilled: What should I set the permissions to for kibana user in order to avoid the above, but not open up the permissions entirely.

Thank you,

On Tuesday, July 4, 2017 at 10:09:23 AM UTC+2, mcost...@np6.com wrote:

Thank you very much for your hard work and that prelim document. On my end, I’ve had some success, but it’s not 100% and I think you may be interested in my observations.

So, to repeat, I am using SG 5.2.2 v12 WITH the enterprise extension.

I was not able to get things going with the sg_monitor role described in the document. I was able to make things work with the sg_monitor role described earlier in this thread, but with the following message being found repeatedly in the console output of ES:
[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=<<>>, roles=] [IndexType [index=_all, type=esqueue]] [Action [indices:data/read/search]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]

[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=esqueue]], sg_own_index=[IndexType [index=_all, type=esqueue]], sg_monitor=[IndexType [index=_all, type=esqueue]], sg_kibana_server=[IndexType [index=_all, type=esqueue]]}

``

However, monitoring in Kibana seems to be working.

Now, if I open up the permissions for the monitoring user entirely (i.e. add him to the sg_all_access role), that message goes away. In fact, I am only left with a warning:
[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

I guess what I’m saying is, with the stack I’m using, the document may not be 100% correct. Also, I’d love your help diagnosing the permission problem.

Many thanks,

Marco.

On Thursday, June 29, 2017 at 9:56:32 PM UTC+2, Jochen Kressin wrote:

We’ve released a preliminary documentation: https://github.com/floragunncom/search-guard-docs/blob/master/monitoring.md

This issue here requires an updated version of the Kibana Plugin, we’re working on it:

https://github.com/floragunncom/search-guard/issues/345

On Monday, June 26, 2017 at 5:29:51 PM UTC+2, Search Guard wrote:

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.

[Marco_Costantini]

I was just able to try this out and it works great. Thank you for helping me with this. Excellent support.

Marco.

···

On Thursday, July 20, 2017 at 11:47:45 AM UTC+2, Jochen Kressin wrote:

Hi Marco,

yes, the docs are geared towards ES > 5.3.0 and SG 14. Plus, we assume that your monitoring user has also the Kibana User role. I guess we should make that clearer in the docs. However, we’ve tested the setup now with:

• ES 5.2.2

• SG 5.2.2-12

• KI 5.2.2

• X-Pack 5.2.2, both on ES and KI

Here’s the role definition that works for us with the said versions:

sg_monitor:

cluster:

• “cluster:admin/xpack/monitoring/*”

• “cluster:admin/ingest/pipeline/*”

• “indices:admin/template/get”

• “indices:admin/template/put”

• “indices:admin/*get”

• CLUSTER_MONITOR

• CLUSTER_COMPOSITE_OPS

indices:

‘?monitoring*’:

‘*’:

• UNLIMITED

‘?marvel*’:

‘*’:

• UNLIMITED

‘*’:

‘*’:

• READ

• indices:admin/get

‘?kibana’:

‘*’:

• INDICES_ALL

The monitoring user needs full control over the monitoring and marvel (for backwards compatibility) indices, and also access to the .kibana index.

Since monitoring affects all indices, the monitoring user must have READ and admin/get permissions on all indices, hence the wildcard.

The last message:

[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

is fixed in later versions of ES/SG, so for the moment with your versions you either need to ignore it, or just set the log level of the PrivilegesEvaluator to ERROR. In config/log4j2.properties, set:

logger.pe.name = com.floragunn.searchguard.configuration.PrivilegesEvaluator

logger.pe.level = error

Let me know if this works for you!

On Wednesday, July 19, 2017 at 11:41:00 AM UTC+2, mcost...@np6.com wrote:

Respectful bump. Question distilled: What should I set the permissions to for kibana user in order to avoid the above, but not open up the permissions entirely.

Thank you,

On Tuesday, July 4, 2017 at 10:09:23 AM UTC+2, mcost...@np6.com wrote:

Thank you very much for your hard work and that prelim document. On my end, I’ve had some success, but it’s not 100% and I think you may be interested in my observations.

So, to repeat, I am using SG 5.2.2 v12 WITH the enterprise extension.

I was not able to get things going with the sg_monitor role described in the document. I was able to make things work with the sg_monitor role described earlier in this thread, but with the following message being found repeatedly in the console output of ES:
[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=<<>>, roles=] [IndexType [index=_all, type=esqueue]] [Action [indices:data/read/search]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]

[2017-07-04T10:04:27,435][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=esqueue]], sg_own_index=[IndexType [index=_all, type=esqueue]], sg_monitor=[IndexType [index=_all, type=esqueue]], sg_kibana_server=[IndexType [index=_all, type=esqueue]]}

``

However, monitoring in Kibana seems to be working.

Now, if I open up the permissions for the monitoring user entirely (i.e. add him to the sg_all_access role), that message goes away. In fact, I am only left with a warning:
[2017-07-04T10:06:36,597][WARN ][c.f.s.c.PrivilegesEvaluator] _all does not exist in cluster metadata

I guess what I’m saying is, with the stack I’m using, the document may not be 100% correct. Also, I’d love your help diagnosing the permission problem.

Many thanks,

Marco.

On Thursday, June 29, 2017 at 9:56:32 PM UTC+2, Jochen Kressin wrote:

We’ve released a preliminary documentation: https://github.com/floragunncom/search-guard-docs/blob/master/monitoring.md

This issue here requires an updated version of the Kibana Plugin, we’re working on it:

https://github.com/floragunncom/search-guard/issues/345

On Monday, June 26, 2017 at 5:29:51 PM UTC+2, Search Guard wrote:

sg_monitor:

cluster:

• cluster:admin/mappings/get

• cluster:admin/ingest/pipeline/put

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘*’:

‘*’:

• indices:admin/get

‘?monitoring*’:

‘*’:

• ‘*’

‘?marvel*’:

‘*’:

• ‘*’

On Monday, 26 June 2017 10:21:31 UTC+2, m…i@n…com wrote:

Respectful bump. Thank you.

On Thursday, June 22, 2017 at 6:27:42 PM UTC+2, mcost...@np6.com wrote:

Another very helpful reply. Thank you.

Yes, my account was not set up properly. I changed the http exporter to use an account that is known to be working. With that, I was able to get further. I ended up needing to add a few new entries under cluster as shown below:

sg_monitor:

cluster:

• cluster:admin/mappings/get <--------- added

• cluster:admin/ingest/pipeline/put <--------- added

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

‘?marvel*’: <------ added

‘*’:

• ALL

``

I added these entries because I received errors concerning permissions for them. One by one my ES progressed. Finally, it complained about being able to access the .marvel-es-1 index. That was a red flag for me. Anyway, As shown above, I added this index to the role, but this time, my environment did not progress. It still complained about permission and access to this index. The error observed is:

[2017-06-22T18:21:28,870][INFO ][c.f.s.c.PrivilegesEvaluator] No index-level perm match for User [name=CN=Kibana SearchGuard Dev,OU=Comptes de services,OU=hidden,DC=hidden,DC=hidden,DC=hidden, roles=] [IndexType [index=_all, type=]] [Action [indices:admin/get]] [RolesChecked [sg_kibana_server, sg_monitor, sg_own_index, sg_public]]
[2017-06-22T18:21:28,871][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {sg_public=[IndexType [index=_all, type=]], sg_own_index=[IndexType [index=_all, type=]], sg_monitor=[IndexType [index=_all, type=]], sg_kibana_server=[IndexType [index=_all, type=*]]}
[2017-06-22T18:21:28,874][ERROR][o.e.x.m.e.h.BackwardsCompatibilityAliasesResource] org.elasticsearch.xpack.monitoring.exporter.http.BackwardsCompatibilityAliasesResource$$Lambda$1681/1710762394@59f4ec16
org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/.marvel-es-1-?filter_path=.aliases: HTTP/1.1 403 Forbidden
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”}],“type”:“security_exception”,“reason”:“no permissions for indices:admin/get”},“status”:403}

``

To repeat, this error occurs even after I added the config (above) and ran sg_admin.sh AND restarted my ES cluster. You can see from the logs that user is correctly mapped to the sg_monitor role.

Please let me know what you think,

Marco.

On Thursday, June 22, 2017 at 3:05:34 PM UTC+2, Jochen Kressin wrote:

Where do you configure/store the monitoring user? In ActiveDirectory or in the SG internal user database?

I would first check that the user “monitoring” can log in with password “pword”, as configured in your elasticsearch.yml. If not already enabled, set the challenge flag to true:

For AD:

ldap:

enabled: true

order: …

http_authenticator:

type: basic

challenge: true

authentication_backend:

Or for the internal user DB:

basic_internal_auth_domain:

enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

Note: You can only enable it for one HTTP authenticator, so choose the auth domain where you configured the monitoring user.

And then try to log in by using a browser. Best is to access the authinfo endpoint of SG:

http://localhost:9200/_searchguard/authinfo

The browser should prompt you for username/password - use the monitoring user for that. If authentication succeeded, you will see a JSON snippet containing the roles of the user. Check that the monitoring user is assigned to the sg_monitor role.

On Thursday, June 22, 2017 at 2:54:37 PM UTC+2, mcost...@np6.com wrote:

Thank you for the help. No worries on it not being finalized as of yet. Sadly, I still have some problems with the environment.

I have the following normally set:

searchguard.ssl.http.enabled: false

``

…I get:

org.elasticsearch.client.ResponseException: GET http://127.0.0.1:9200/?filter_path=version.number: HTTP/1.1 401 Unauthorized
Authentication finally failed

``

I see someone also had this issue, and the role_mapping (with new role) seemed to fix it for him/her. Not for me sadly. Maybe I’ve done something wrong. Here are my pertinent configs:

sg_roles.yml

sg_monitor:
cluster:
- cluster:admin/ingest/pipeline/get
- cluster:admin/xpack/monitoring/bulk
- indices:admin/template/get
- indices:admin/template/put
- indices:admin/get
- cluster:monitor/nodes/info
- cluster:monitor/health
- cluster:monitor/main
- cluster:monitor/xpack/info
- indices:data/write/bulk
indices:
‘?monitoring*’:
‘*’:
- ALL

``

sg_roles_mapping.yml

sg_monitor:

users:

• monitoring

``

elasticsearch.yml
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster
network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.monitoring.exporters:
id1:
type: http
host: [“http://127.0.0.1:9200”]
auth.username: monitoring
auth.password: pword

ssl:

truststore.path: truststore.jks

truststore.password: changeit

``

As you can see, the query is sent without basic auth credentials. I had my doubts that the mentioned solution was relevant to me as I am using ActiveDirectory.

Any thoughts?

Please and thank you,

Marco.

On Thursday, June 22, 2017 at 11:50:28 AM UTC+2, Jochen Kressin wrote:

Hi,

we’re working on the monitoring integration. At the moment, you would need to use an HTTP exportert instead of local, like:

xpack.monitoring.exporters:

id1:

type: http

host: [“https://127.0.0.1:9200”]

auth.username: monitoring

auth.password: …

ssl:

truststore.path: truststore.jks

truststore.password: changeit

And then map the monitoring user to the role with the permissions explained in the GitHub issue you referenced:

sg_monitor:

cluster:

• cluster:admin/ingest/pipeline/get

• cluster:admin/xpack/monitoring/bulk

• indices:admin/template/get

• indices:admin/template/put

• indices:admin/get

• cluster:monitor/nodes/info

• cluster:monitor/health

• cluster:monitor/main

• cluster:monitor/xpack/info

• indices:data/write/bulk

indices:

‘?monitoring*’:

‘*’:

• ALL

If you also use the Kibana monitoring plugin from X-Pack, you will run into the following issue:

https://github.com/floragunncom/search-guard/issues/345

We’re working on the compatibility of Search Guard and X-Pack Monitoring. It’s high on our priority list, but it might take some time until the integration runs smoothly on both ES and KI side.

As to your other question, no admin_dn has nothing to do with this issue here. It simply configures which TLS certificate(s) can be used as admin certificate. An admin certificate has full access to the cluster, and you need it whenever you make changes to the SG index. If you use our demo certificates, it’s the “kirk” certificate, and it’s contained in the keystore you use when executing sgadmin.

On Thursday, June 22, 2017 at 10:05:01 AM UTC+2, mcost...@np6.com wrote:

Hello,
I’m trying to build a local test setup with the following components:

• ElasticSearch 5.2.2

• SearchGuard 5.2.2 (With Enterprise extension for ActiveDirectory usage)
• X-Pack 5.2.2 (Monitoring only)

Without X-Pack, the environment works fine. When I install X-Pack, I get errors in stdout. With the help of an existing Github issue, I’ve been able to work my way past a few of the errors, but at this point, I am stuck. I’ll add some details below. I’d like to mention that I intend to add Kibana to the mix, but I’d like to keep the environment minimal while sorting out these errors.

So, at the moment this is where I am stuck:

Caused by: org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:data/write/bulk for user User [name=_sg_internal, roles=]

``

As recommended by the above-mentioned Github issue (https://github.com/floragunncom/search-guard-ssl/issues/43), I’ve changed my elasticsearch/config/elasticsearch.yml to have the following settings:

searchguard.ssl.transport.keystore_filepath: keystore.jks

searchguard.ssl.transport.truststore_filepath: truststore.jks

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.http.enabled: false

searchguard.ssl.http.keystore_filepath: keystore.jks

searchguard.ssl.http.truststore_filepath: truststore.jks

searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test, C=de

cluster.name: docker-ezcore-cluster

network.host: 0.0.0.0

######## End Search Guard Demo Configuration ########

xpack.security.enabled: false

xpack.monitoring.enabled: true

xpack.graph.enabled: false

xpack.watcher.enabled: false

xpack.monitoring.exporters:

my_local:

type: local

``

I’ve tried the other changes mentioned in the issue, but they seem to have no effect for me.

Here are some things I think are remarkable:

• I’ve successfully hooked up SearchGuard with ActiveDirectory. Yet, I don’t know what role this admin_dn plays, and have not yet created an account for this admin. Also, I don’t feel like this is related to this problem I’m blocked on, so I’ve left it alone for now.
• I’m curious if Kibana is ESSENTIAL to the environment, and whether or not excluding it for now is exacerbating the problem.

• I’m constrained to working with version 5.2.2 of ES.

• I am testing with the demo versions of the keystores, although I use the sg_admin.sh tool to initialize SearchGuard. I provide the keystore file paths in the command arguments.

Any ideas or help is greatly appreciated.
Please and thank you,

Marco.