Kerberos acceptor_principal seems to be ignored

Search Guard
1

Using ES 6.8.6 and latest SG.
Using kerberos auth we noticed that the setting searchguard.kerberos.acceptor_principal seems to be ignored.
Here’s the symptom:

On node01 in elasticsearch.yml we have:

searchguard.kerberos.acceptor_keytab_filepath: sg.keytab
searchguard.kerberos.acceptor_principal: HTTP/node01.example.com@EXAMPLE.COM

sg.keytab contains:

Keytab name: FILE:/etc/elasticsearch/test/sg.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   1 08/02/2019 16:51:22 HTTP/node01.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
   1 08/02/2019 16:51:22 HTTP/node01.example.com@EXAMPLE.COM (aes128-cts-hmac-sha1-96)

This is all nice and works. However, if we simply replace the keytab with another one from node02 which is identical except for the hostname, then the requests still get accepted, even without changing the acceptor principal to ...node02....

2

Now you have 2 nodes, with the following config, right? And you have both nodes successfully authenticated and serving requests, right?

Node 1

elasticsearch.yml

searchguard.kerberos.acceptor_keytab_filepath: sg.keytab
searchguard.kerberos.acceptor_principal: HTTP/node01.example.com@EXAMPLE.COM

sg.keytab

Keytab name: FILE:/etc/elasticsearch/test/sg.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   1 08/02/2019 16:51:22 HTTP/node02.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
   1 08/02/2019 16:51:22 HTTP/node02.example.com@EXAMPLE.COM (aes128-cts-hmac-sha1-96)

Node 2

elasticsearch.yml

searchguard.kerberos.acceptor_keytab_filepath: sg.keytab
searchguard.kerberos.acceptor_principal: HTTP/node02.example.com@EXAMPLE.COM

sg.keytab

Keytab name: FILE:/etc/elasticsearch/test/sg.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   1 08/02/2019 16:51:22 HTTP/node02.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
   1 08/02/2019 16:51:22 HTTP/node02.example.com@EXAMPLE.COM (aes128-cts-hmac-sha1-96)

Do you see any errors or warnings in the Authentication Server and Elasticsearch logs?

Gather some debug data to post it here. To enable the debug:

  1. Put krb_debug: true in sg_config.yml
  2. If you see no debug put the following JVM options
-Dsun.security.krb5.debug=true
-Djava.security.debug=gssloginconfig,logincontext,configparser,configfile
-Dsun.security.spnego.debug=true
3

yes exactly. let me check the logs

4

the elasticsearch logfile claims Kerberos debug is enabled on stdout so I guess I can only see it in journalctl ?

5

I think yes, if you use systemd to run elasticsearch.

6

We are still having issues with the acceptor principal, now on 7.9.1.
The use-case is this : we have a CNAME pointing to a proxy node, which then does tcp haproxy load-balancing to the elasticsearch nodes. When using acceptor_principal: HTTP/proxy-node@OUR.REALM requests get refused. I haven’t been able to enable debug logs on the searchguard/elastic side, all instructions applied. But I can get client logs by enabling KRB5_TRACE, and they suggest the principal being requested is the DNS A record (as resolved by the CNAME), and not the searchguard.kerberos.acceptor_principal from the config.

7

But I can get client logs by enabling KRB5_TRACE

Hi. Where do you enable the KRB5_TRACE? Can you paste this log here? Or send it privately.

8

on the client side (curl). In the meantime we switched to using the A record instead of the CNAME, and it works.
I didn’t keep the full trace, sorry, but I’ll do as soon as we test this on the dev cluster.

9

Could you please paste here or send me your sg_config.yml and elasticsearch.yml?

10

Did I draw your use case correctly?

1830
1830960×720 29.5 KB

11

exactly, and the proxy-node is called from clients using its CNAME