# Metricbeat and Elasticsearch module 401

**URL:** <https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584>\
**Category:** Search Guard\
**Created:** [July 6, 2019, 9:27am UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584 "2019-07-06T09:27:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rschirin](https://avatars.discourse-cdn.com/v4/letter/r/ac8455/32.png) [@rschirin](https://forum.search-guard.com/u/rschirin)\
**Post date:** [July 6, 2019, 9:27am UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/1 "2019-07-06T09:27:15Z")

</div>

Hi,  
I’m using Elasticsearch 6.6.1 and Metricbeat 6.6.1. I have already installed SG with the correct version and everything works properly (i.e. LDAP authentication). currently I have installed metricbeat to collect data using Elasticsearch module and below you can find my section:

> - module: elasticsearch  
> metricsets:
> - node
> - node\_stats
> - index
> - index\_recovery
> - index\_summary
> - shard
> - ml\_job
> - ccr
> - cluster\_stats
> - pending\_tasks
> - shard  
> period: 30s  
> enabled: true  
> hosts: [“[https://elk01ci.env.net:9200](https://elk01ci.env.net:9200)”]  
> user: admin  
> password:xxxxx  
> ssl.certificate\_authorities: [“/etc/elasticsearch/certs/chain.pem”]  
> ssl.certificate: “/etc/elasticsearch/certs/elk01ci.cer”  
> ssl.key: “/etc/elasticsearch/certs/elk01ci.key”  
> ssl.verification\_mode: “none”  
> ssl.enabled: true

unfortunately, when I start Metricbeat it is not able to collect data with the following error.

```
ERROR [elasticsearch] elastic/elastic.go:117 error determining if connected Elasticsearch node is master: HTTP error 401 in shard: 401 Unauthorized

```

For sure, that node is master since it is my only one node but I cannot fix the 401 error code. Password is correct since I tested it with cURL and user has the correct permissions.

Furthermore, if I disable SG from elasticsearch.yml, everything works correctly.

---

<div class="post-metadata">

**Author:** ![cstaley](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/cstaley/32/463_2.png) [@cstaley](https://forum.search-guard.com/u/cstaley)\
**Post date:** [July 7, 2019, 1:56pm UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/2 "2019-07-07T13:56:20Z")

</div>

Assigned @cstaley on July 7, 2019, 1:56pm UTC



---

<div class="post-metadata">

**Author:** ![cstaley](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/cstaley/32/463_2.png) [@cstaley](https://forum.search-guard.com/u/cstaley)\
**Post date:** [July 7, 2019, 2:01pm UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/3 "2019-07-07T14:01:56Z")

</div>

Please post your sg\_config.yml, elasticsearch.yml and logs from the ES node (if there are no interesting logs pls enable debug logging as explained here [Search Guard logging | Security for Elasticsearch | Search Guard](https://docs.search-guard.com/latest/troubleshooting-setting-log-level#turn-on-debug-logging-temporarily)).

And i think you should remove `ssl.certificate` and `ssl.key` properties in your metricbeat configuration because you already provided a username and a password.

---

<div class="post-metadata">

**Author:** ![rschirin](https://avatars.discourse-cdn.com/v4/letter/r/ac8455/32.png) [@rschirin](https://forum.search-guard.com/u/rschirin)\
**Post date:** [July 8, 2019, 8:46am UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/4 "2019-07-08T08:46:56Z")

</div>

here you can find my sg\_config.yml. basically, I added only the LDAP method:

> searchguard:  
> dynamic:  
> kibana:  
> multitenancy/wiki  
> http:  
> anonymous\_auth\_enabled: false  
> xff:  
> enabled: false  
> internalProxies: ‘192.168.0.10|192.168.0.11’ # regex pattern  
> remoteIpHeader: ‘x-forwarded-for’  
> proxiesHeader: ‘x-forwarded-by’  
> authc:  
> kerberos\_auth\_domain:  
> enabled: false  
> order: 6  
> http\_authenticator:  
> type: kerberos # NOT FREE FOR COMMERCIAL USE  
> challenge: true  
> config:  
> # If true a lot of kerberos/security related debugging output will be logged to standard out  
> krb\_debug: false  
> # If true then the realm will be stripped from the user name  
> strip\_realm\_from\_principal: true  
> authentication\_backend:  
> type: noop  
> basic\_internal\_auth\_domain:  
> enabled: true  
> order: 0  
> http\_authenticator:  
> type: basic  
> challenge: true  
> authentication\_backend:  
> type: intern  
> proxy\_auth\_domain:  
> enabled: false  
> order: 3  
> http\_authenticator:  
> type: proxy  
> challenge: false  
> config:  
> user\_header: “x-proxy-user”  
> roles\_header: “x-proxy-roles”  
> authentication\_backend:  
> type: noop  
> host\_auth\_domain:  
> enabled: false  
> order: 5  
> http\_authenticator:  
> type: host #DEPRECATED, will be removed in a future version  
> challenge: false  
> authentication\_backend:  
> type: noop  
> jwt\_auth\_domain:  
> enabled: false  
> order: 4  
> http\_authenticator:  
> type: jwt  
> challenge: false  
> config:  
> signing\_key: “base64 encoded key”  
> jwt\_header: “Authorization”  
> jwt\_url\_parameter: null  
> roles\_key: null  
> subject\_key: null  
> authentication\_backend:  
> type: noop  
> clientcert\_auth\_domain:  
> enabled: false  
> order: 2  
> http\_authenticator:  
> type: clientcert  
> config:  
> username\_attribute: cn #optional, if omitted DN becomes username  
> challenge: false  
> authentication\_backend:  
> type: noop  
> ldap:  
> enabled: true  
> order: 1  
> http\_authenticator:  
> type: basic  
> challenge: false  
> authentication\_backend:  
> type: ldap # NOT FREE FOR COMMERCIAL USE  
> config:  
> enable\_ssl: false  
> enable\_start\_tls: false  
> enable\_ssl\_client\_auth: false  
> verify\_hostnames: false  
> hosts:  
> - [dcoll.net:389](http://dcoll.net:389)  
> bind\_dn: CN=elksget,OU=Application Users,DC=dcoll,DC=net  
> password: ZPJdd2RiuJ\_DgzIllivH  
> userbase: ‘DC=dcoll,DC=net’  
> usersearch: ‘(sAMAccountName={0})’  
> username\_attribute: null  
> authz:  
> roles\_from\_myldap:  
> enabled: true  
> authorization\_backend:  
> type: ldap # NOT FREE FOR COMMERCIAL USE  
> config:  
> enable\_ssl: false  
> enable\_start\_tls: false  
> enable\_ssl\_client\_auth: false  
> verify\_hostnames: true  
> hosts:  
> - [dcoll.net:389](http://dcoll.net:389)  
> bind\_dn: CN=elksget,OU=Application Users,DC=dcoll,DC=net  
> password: ZPJdd2RiuJ\_DgzIllivH  
> rolebase: ‘OU=Groups,DC=dcoll,DC=net’  
> rolesearch: ‘(member={0})’  
> userroleattribute: null  
> userrolename: memberOf  
> rolename: cn  
> resolve\_nested\_roles: true  
> userbase: ‘DC=dcoll,DC=net’  
> usersearch: ‘(sAMAccountName={0})’

my elasticsearch.yml

> node.name: ${HOSTNAME}  
> node.master: true  
> node.data: true  
> node.ingest: false  
> path.data: /var/es  
> path.logs: /var/log/elasticsearch  
> network.host: 172.21.40.224  
> http.port: 9200  
> discovery.zen.ping.unicast.hosts: [“172.21.40.224”]  
> xpack.security.enabled: false  
> searchguard.disabled: false  
> searchguard.ssl.transport.pemcert\_filepath: certs/elkcrc01ci.cer  
> searchguard.ssl.transport.pemkey\_filepath: certs/elkcrc01ci.key  
> searchguard.ssl.transport.pemtrustedcas\_filepath: certs/FullChainCA.pem  
> searchguard.ssl.transport.enforce\_hostname\_verification: false  
> searchguard.ssl.transport.resolve\_hostname: false  
> searchguard.ssl.http.enabled: true  
> searchguard.ssl.http.pemcert\_filepath: certs/elkcrc01ci.cer  
> searchguard.ssl.http.pemkey\_filepath: certs/elkcrc01ci.key  
> searchguard.ssl.http.pemtrustedcas\_filepath: certs/FullChainCA.pem  
> searchguard.nodes\_dn:
> 
> - xxx  
> searchguard.authcz.admin\_dn:
> - xxx

I have enabled the debug level and this is the result:

```
[2019-07-08T17:25:37,509][DEBUG][c.f.s.a.BackendRegistry] [elkcrc01ci] Rest authentication request from 172.21.40.224:60710 [original: /172.21.40.224:60710]
[2019-07-08T17:25:39,197][DEBUG][c.f.s.a.BackendRegistry] [elkcrc01ci] Rest authentication request from 172.21.40.224:60712 [original: /172.21.40.224:60712]
[2019-07-08T17:25:39,805][DEBUG][c.f.s.a.BackendRegistry] [elkcrc01ci] Rest authentication request from 172.21.40.224:60714 [original: /172.21.40.224:60714]
[2019-07-08T17:25:40,456][DEBUG][c.f.s.a.BackendRegistry] [elkcrc01ci] Rest authentication request from 172.21.40.224:60716 [original: /172.21.40.224:60716]
[2019-07-08T17:25:40,848][DEBUG][c.f.s.a.BackendRegistry] [elkcrc01ci] Rest authentication request from 172.21.40.224:60718 [original: /172.21.40.224:60718]

```

my log file is full of these entries.

---

<div class="post-metadata">

**Author:** ![rschirin](https://avatars.discourse-cdn.com/v4/letter/r/ac8455/32.png) [@rschirin](https://forum.search-guard.com/u/rschirin)\
**Post date:** [July 9, 2019, 4:16pm UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/5 "2019-07-09T16:16:15Z")

</div>

solved.  
issue was due a wrong key in elasticsearch module.  
user → username

---

<div class="post-metadata">

**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/system/32/1870_2.png) [@system](https://forum.search-guard.com/u/system)\
**Post date:** [July 30, 2019, 4:16pm UTC](https://forum.search-guard.com/t/metricbeat-and-elasticsearch-module-401/1584/6 "2019-07-30T16:16:16Z")

</div>

Closed on July 30, 2019, 4:16pm UTC

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.