Hi,
I followed the documentation [1] and [2] step by step, but am unable to get the kerberos plugin to load.
When starting up, there is no log message about the kerberos plugin whatsoever, even when enabling debug in sg_config.yml.
I tried adding another jar, namely the audit logging, and that seems to work as expected.
I managed to enable the plugin (PBCAK: didn’t run sgadmin.sh).
However, I still have no clue as to what’s going on. Even debug mode spits nothing on stdout (elasticsearch has been started in the foreground):
[INFO ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Kerberos debug is enabled on stdout
Then running curl -k --negotiate -u: https://localhost:9200/_searchguard/authinfo returns nothing. Also, stdout of elasticsearch stays silent.
I would expect the debug mode to tell me what’s going on, but it just stays silent when issuing SPNEGO requests.
Now when enabling dynamic.http.anonymous_auth_enabled to true, I can get the following output:
{“user”:“User [name=sg_anonymous, roles=[sg_anonymous_backendrole]]”,“remote_address”:“127.0.0.1:57934”,“sg_roles”:[“sg_public”],“principal”:null,“peer_certificates”:“0”}
I was hoping the principal to contain my principal’s name, stripped of the realm, but apparently this doesn’t happen.
Moreover, still nothing in the “debug” log.
Any help appreciated
Kerberos is tricky 
Pls post your
- elasticsearch.yml
- sg_config.yml and the output of these commands:
- krb5.conf
and the output of these commands:
- "curl -V" (not all precompiled curl binaries are working)
- "klist" (after a "kinit")
Please change the logging.yml to log on DEBUG level and post the output after a login attempt like
$ "kinit" (get a TGT)
$ "curl -k -u : --negotiate -u: https://localhost:9200/"
Am 05.10.2016 um 16:28 schrieb Fabien Wernli <swissunix@gmail.com>:
I managed to enable the plugin (PBCAK: didn't run sgadmin.sh).
However, I still have no clue as to what's going on. Even debug mode spits nothing on stdout (elasticsearch has been started in the foreground):
[INFO ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Kerberos debug is enabled on stdout
Then running `curl -k --negotiate -u: https://localhost:9200/_searchguard/authinfo\` returns nothing. Also, stdout of elasticsearch stays silent.
I would expect the debug mode to tell me what's going on, but it just stays silent when issuing SPNEGO requests.Now when enabling `dynamic.http.anonymous_auth_enabled` to true, I can get the following output:
{"user":"User [name=sg_anonymous, roles=[sg_anonymous_backendrole]]","remote_address":"127.0.0.1:57934","sg_roles":["sg_public"],"principal":null,"peer_certificates":"0"}
I was hoping the principal to contain my principal's name, stripped of the realm, but apparently this doesn't happen.
Moreover, still nothing in the "debug" log.Any help appreciated
--
You received this message because you are subscribed to the Google Groups "Search Guard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/519fc713-ec63-4ead-b0b0-f2f535c40f84%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.
Hi,
Thanks for your quick reply.
I added the config files and output of the shell session to a gist for clarity (and size restriction).
Cheers
Hi,
Some additional info: from the KDC log it seems something is wrong:
kdc Server not found in database: HTTP/localhost@OUR.TLD: no such entry found in hdb
``
Progress!
I changed the listening port of elasticsearch to the public interface, and modified the principal’s attributes (which used to be too restrictive).
Now I see the following in ES log:
[WARN ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Service login not successful due to java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
``
Maybe the keytab entries are wrong? Can you list the contents of your /etc/elasticsearch/searchguard.our.tld.keytab?
Seems to be a hostname/dns resolution problem.
You can also try adding this to krb5.conf (not recommended for production of course):
[libdefaults]
dns_lookup_realm = false
dns_lookup_kdc = false
forwardable=true
dns_canonicalize_hostname = false
rdns = false
ignore_acceptor_hostname = true
Here is a gist with a working configuration (also not recommended for production):
https://gist.github.com/floragunncom/e12d2eee5c25e2ae0165d37df12bba4a
PS: Can you also please add the output of “curl -V”?
On Thursday, 6 October 2016 10:51:55 UTC+2, Fabien Wernli wrote:
Progress!
I changed the listening port of elasticsearch to the public interface, and modified the principal’s attributes (which used to be too restrictive).
Now I see the following in ES log:
[WARN ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Service login not successful due to java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
``
I updated the gist with the info from curl -V.
I resolved the DNS issue (see earlier reply), and now I’m seeing the entry of the Service principal in klist after issuing the curl command:
Ticket cache: FILE:/tmp/krb5cc_3942_cacbU16950
Default principal: fwernli@OUR.TLD
Valid starting Expires Service principal
10/06/16 11:04:02 10/07/16 08:45:31 krbtgt/OUR.TLD@OUR.TLD
10/06/16 11:04:03 10/07/16 08:45:31 HTTP/some_node.our.tld@OUR.TLD
``
However, I’m getting 401 Unauthorized.
If I set authc.kerberos_auth_admin.http_authenticator.challenge to false, then the ES log is empty.
If I set authc.kerberos_auth_admin.http_authenticator.challenge to true,then I get the following error in ES log:
[2016-10-06 11:12:17,330][WARN ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Service login not successful due to java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
``
authc.kerberos_auth_admin.http_authenticator.challenge must be set to true (more on this here in the comments https://github.com/floragunncom/search-guard/blob/master/sgconfig/sg_config.yml)
your curl is pretty old and curl has few known kerberos bugs in old versions
i recommend to compile a recent version:
sudo apt-get -yqq install libssl-dev autoconf libtool build-essential wget openssl libkrb5-dev
EXPORT CURLDIR=“/curlk”
mkdir -p $CURLDIR
cd $CURLDIR
wget -nv https://github.com/curl/curl/archive/curl-7_49_1.tar.gz
tar -xzf curl-7_49_1.tar.gz
cd $CURLDIR/curl-curl-7_49_1
./buildconf
./configure --with-gssapi --with-ssl --prefix=“$CURLDIR”
make
make install
chmod +x $CURLDIR/bin/curl
$CURLDIR/bin/curl -V
On Thursday, 6 October 2016 11:14:24 UTC+2, Fabien Wernli wrote:
I updated the gist with the info from
curl -V.
I resolved the DNS issue (see earlier reply), and now I’m seeing the entry of the Service principal inklistafter issuing the curl command:
Ticket cache: FILE:/tmp/krb5cc_3942_cacbU16950
Default principal: fwernli@OUR.TLD
Valid starting Expires Service principal
10/06/16 11:04:02 10/07/16 08:45:31 krbtgt/OUR.TLD@OUR.TLD
10/06/16 11:04:03 10/07/16 08:45:31 HTTP/some_node.our.tld@OUR.TLD
``
However, I’m getting
401 Unauthorized.
If I set
authc.kerberos_auth_admin.http_authenticator.challengetofalse, then the ES log is empty.
If I set
authc.kerberos_auth_admin.http_authenticator.challengetotrue,then I get the following error in ES log:
[2016-10-06 11:12:17,330][WARN ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Service login not successful due to java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
java.security.PrivilegedActionException: GSSException: No credential found for: 1.2.840.113554.1.2.2 usage: Accept
``
I’ll try, but I am successfully using this curl version against another SPNEGO enabled service.
Moreover, could you comment on the GSSException?
I tested using curl 7.49.1.
It still fails, but with a different error in the ES log:
[2016-10-06 12:36:04,001][WARN ][com.floragunn.dlic.auth.http.kerberos.HTTPSpnegoAuthenticator] Service login not successful due to java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled)
java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled)
``
For the record, the newer curl still works with our other SPNEGO enabled service
I just tried using firefox, and the error message in ES is the same as for curl 7.49.1:
Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled
``
I guess this is related to java then: FME Community
On Thursday, October 6, 2016 at 12:45:52 PM UTC+2, Fabien Wernli wrote:
I just tried using firefox, and the error message in ES is the same as for curl 7.49.1:
Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled
``
yes, you need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" on every node.
If you`re fine with AES128 you can configure this with “supported_enctypes” in kdc.conf/krb5.conf - see https://gist.github.com/floragunncom/e12d2eee5c25e2ae0165d37df12bba4a
On Thursday, 6 October 2016 12:47:46 UTC+2, Fabien Wernli wrote:
I guess this is related to java then: https://knowledge.safe.com/articles/395/enabling-aes256-in-the-java-runtime-environment-fo.html
On Thursday, October 6, 2016 at 12:45:52 PM UTC+2, Fabien Wernli wrote:
I just tried using firefox, and the error message in ES is the same as for curl 7.49.1:
Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled
``
I did: replaced the two jar files from JCE, but it didn’t change anything (after restarting ES).
I am now trying to use openssl instead by following the guide at https://github.com/floragunncom/search-guard-ssl-docs/blob/master/openssl.md
On Thursday, October 6, 2016 at 12:53:16 PM UTC+2, in...@search-guard.com wrote:
yes, you need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" on every node.
If you`re fine with AES128 you can configure this with “supported_enctypes” in kdc.conf/krb5.conf - see https://gist.github.com/floragunncom/e12d2eee5c25e2ae0165d37df12bba4a
On Thursday, 6 October 2016 12:47:46 UTC+2, Fabien Wernli wrote:
I guess this is related to java then: https://knowledge.safe.com/articles/395/enabling-aes256-in-the-java-runtime-environment-fo.html
On Thursday, October 6, 2016 at 12:45:52 PM UTC+2, Fabien Wernli wrote:
I just tried using firefox, and the error message in ES is the same as for curl 7.49.1:
Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled
``
That will not help here - don’t confuse kerberos with SSL
You can verify that JCE was installed correctly if you do NOT see something like “AES-256 not supported, max key length for AES …” in elasticsearch logs.
On the other side if you have JCE not installed you see something like “AES-256 not supported, max key length for AES …”
On Thursday, 6 October 2016 13:06:39 UTC+2, Fabien Wernli wrote:
I did: replaced the two jar files from JCE, but it didn’t change anything (after restarting ES).
I am now trying to use openssl instead by following the guide at https://github.com/floragunncom/search-guard-ssl-docs/blob/master/openssl.md
On Thursday, October 6, 2016 at 12:53:16 PM UTC+2, in...@search-guard.com wrote:
yes, you need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" on every node.
If you`re fine with AES128 you can configure this with “supported_enctypes” in kdc.conf/krb5.conf - see https://gist.github.com/floragunncom/e12d2eee5c25e2ae0165d37df12bba4a
On Thursday, 6 October 2016 12:47:46 UTC+2, Fabien Wernli wrote:
I guess this is related to java then: https://knowledge.safe.com/articles/395/enabling-aes256-in-the-java-runtime-environment-fo.html
On Thursday, October 6, 2016 at 12:45:52 PM UTC+2, Fabien Wernli wrote:
I just tried using firefox, and the error message in ES is the same as for curl 7.49.1:
Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled
``
I successfully enabled OpenSSL:
[2016-10-06 13:14:05,691][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.2h 3 May 2016 available
[2016-10-06 13:14:06,022][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportClientProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
``
And the error is still the same:
java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)
``
Using a weaker key as suggested in your krb5.conf results in almost the same message (matching the key type of AES128):
rbException: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES128 CTS mode with HMAC SHA1-96
``
as i said: don’t confuse Kerberos with SSL
Make sure that you keytab and krb tickets have a the appropriate keytype (AES128 CTS mode with HMAC SHA1-96) - maybe you have created them with AES256 and will not work of course
On Thursday, 6 October 2016 13:55:13 UTC+2, Fabien Wernli wrote:
I successfully enabled OpenSSL:
[2016-10-06 13:14:05,691][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] Open SSL OpenSSL 1.0.2h 3 May 2016 available
[2016-10-06 13:14:06,022][INFO ][com.floragunn.searchguard.ssl.SearchGuardKeyStore] sslTransportClientProvider:OPENSSL with ciphers [TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA]
``
And the error is still the same:
java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)
``
Using a weaker key as suggested in your krb5.conf results in almost the same message (matching the key type of AES128):
rbException: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES128 CTS mode with HMAC SHA1-96
``
To leave no doubts, I removed all keys except AES128 CTS from the keyfile:
klist -k /etc/elasticsearch/searchguard.in2p3.fr.keytab -e
Keytab name: FILE:/etc/elasticsearch/searchguard.in2p3.fr.keytab
KVNO Principal
1 HTTP/node42.our.tld@OUR.TLD (aes128-cts-hmac-sha1-96)
``
I still get the same error message using new curl, and No credential found for: 1.2.840.113554.1.2.2 usage: Accept when using the old version
And yes, I confuse SSL and kerberos, of course I do it’s so confusing
No credential found for: 1.2.840.113554.1.2.2 usage: Accept is a curl bug with old curl → Curl: [curl:bugs] #1302 HTTP Auth Negotiate sends Kerberos token instead of SPNEGO token
Can you again post kdc.conf and krb5.conf with enctypes set?
In kdc.conf you should have something like supported_enctypes = aes128-cts-hmac-sha1-96:normal and in krb5.conf:
default_tkt_enctypes = aes128-cts-hmac-sha1-96 rc4-hmac
default_tgs_enctypes = aes128-cts-hmac-sha1-96 rc4-hmac
permitted_enctypes = aes128-cts-hmac-sha1-96 rc4-hmac
And pls check your KVNO → Problems With Key Version Numbers - Managing Kerberos and Other Authentication Services in Oracle® Solaris 11.2
On Thursday, 6 October 2016 14:32:14 UTC+2, Fabien Wernli wrote:
To leave no doubts, I removed all keys except AES128 CTS from the keyfile:
klist -k /etc/elasticsearch/searchguard.in2p3.fr.keytab -e
Keytab name: FILE:/etc/elasticsearch/searchguard.in2p3.fr.keytab
KVNO Principal
1 HTTP/node42.our.tld@OUR.TLD (aes128-cts-hmac-sha1-96)
``
I still get the same error message using new curl, and
No credential found for: 1.2.840.113554.1.2.2 usage: Acceptwhen using the old version
And yes, I confuse SSL and kerberos, of course I do it’s so confusing