Issues configuring SAML authentication

Doug_Renze · July 6, 2020, 9:46pm

Implementation Details

Elasticsearch: 7.8.0 (official Elastic Docker images, with Search Guard installed)
Search Guard: 42.0.0 (appears that 43.0.0 may be out now, but I haven’t built this image yet)

Issues

I am having issues configuring SAML for my ELK instance (Elasticsearch 7.8.0 with corresponding version of Search Guard). Please note that I have sanitized these documents by substituting example.com for my company’s domain and changeme for the exchange_key. These are not the values that are being used in my live config files.

SAML Config

Note: exchange_key and corporate domain have been sanitized, these values are not being used in live deployment.

saml:
  http_enabled: true
  # transport_enabled: true
  order: 2
  http_authenticator:
    type: 'saml'
    challenge: true
    config:
      idp:
        metadata_url: 'https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml'  # Validated with ADFS team
        entity_id: 'http://ifs.example.com/adfs/services/trust'  # Validated with ADFS team.
        enable_ssl: true
        verify_hostnames: false
        pemtrustedcas_filepath: 'certs/ca_bundle.pem'  # Works successfully for LDAP, validated same cert bundle for both.
      sp:
        entity_id: 'https://elk.dev.example.com/kibana'
      roles_key: 'Role'
      exchange_key: 'changeme'
      kibana_url: 'https://elk.dev.example.com'
  authentication_backend:
    type: 'noop'

Debug Log

I am receiving the following errors in my debug log.

Note: corporate domain has been sanitized, this value is not being used in live deployment.

{"type": "server", "timestamp": "2020-07-06T20:53:04,146Z", "level": "INFO", "component": "o.o.s.m.r.i.AbstractReloadingMetadataResolver", "cluster.name": "elk-es", "node.name": "elk-es-master-0", "message": "Metadata Resolver SamlHTTPMetadataResolver com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Next refresh cycle for metadata provider 'https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml' will occur on '2020-07-06T20:54:04.146Z' ('2020-07-06T20:54:04.146Z' local time)", "cluster.uuid": "xMmOk1dFTCeOmXHJyeeDgA", "node.id": "q6WqgtzvQt20Uedo1ge_Ug"  }
{"type": "server", "timestamp": "2020-07-06T20:53:09,387Z", "level": "ERROR", "component": "o.o.s.m.r.i.HTTPMetadataResolver", "cluster.name": "elk-es", "node.name": "elk-es-master-0", "message": "Metadata Resolver SamlHTTPMetadataResolver com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error retrieving metadata from https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml", "cluster.uuid": "xMmOk1dFTCeOmXHJyeeDgA", "node.id": "q6WqgtzvQt20Uedo1ge_Ug" , 
"stacktrace": ["javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:325) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:268) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:645) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:445) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:423) ~[?:?]",
"at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) ~[?:?]",
"at sun.security.ssl.SSLTransport.decode(SSLTransport.java:167) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1462) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1370) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:437) ~[?:?]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:287) [opensaml-saml-impl-3.3.0.jar:?]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver.access$001(SamlHTTPMetadataResolver.java:34) [dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver$1.run(SamlHTTPMetadataResolver.java:56) [dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver$1.run(SamlHTTPMetadataResolver.java:53) [dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at java.security.AccessController.doPrivileged(AccessController.java:554) [?:?]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver.fetchMetadata(SamlHTTPMetadataResolver.java:53) [dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:285) [opensaml-saml-impl-3.3.0.jar:?]",
"at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver$RefreshMetadataTask.run(AbstractReloadingMetadataResolver.java:552) [opensaml-saml-impl-3.3.0.jar:?]",
"at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]",
"at java.util.TimerThread.run(Timer.java:506) [?:?]",
"Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) ~[?:?]",
"at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]",
"at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629) ~[?:?]",
"... 32 more",
"Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:?]",
"at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:?]",
"at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) ~[?:?]",
"at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) ~[?:?]",
"at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]",
"at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629) ~[?:?]",
"... 32 more"] }
{"type": "server", "timestamp": "2020-07-06T20:53:09,388Z", "level": "ERROR", "component": "o.o.s.m.r.i.AbstractReloadingMetadataResolver", "cluster.name": "elk-es", "node.name": "elk-es-master-0", "message": "Metadata Resolver SamlHTTPMetadataResolver com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error occurred while attempting to refresh metadata from 'https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml'", "cluster.uuid": "xMmOk1dFTCeOmXHJyeeDgA", "node.id": "q6WqgtzvQt20Uedo1ge_Ug" , 
"stacktrace": ["net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml",
"at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:314) ~[opensaml-saml-impl-3.3.0.jar:?]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver.access$001(SamlHTTPMetadataResolver.java:34) ~[dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver$1.run(SamlHTTPMetadataResolver.java:56) ~[dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver$1.run(SamlHTTPMetadataResolver.java:53) ~[dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]",
"at com.floragunn.dlic.auth.http.saml.SamlHTTPMetadataResolver.fetchMetadata(SamlHTTPMetadataResolver.java:53) ~[dlic-search-guard-suite-security-7.8.0-42.0.0.jar:7.8.0-42.0.0]",
"at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:285) [opensaml-saml-impl-3.3.0.jar:?]",
"at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver$RefreshMetadataTask.run(AbstractReloadingMetadataResolver.java:552) [opensaml-saml-impl-3.3.0.jar:?]",
"at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]",
"at java.util.TimerThread.run(Timer.java:506) [?:?]",
"Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:325) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:268) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:645) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:445) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:423) ~[?:?]",
"at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) ~[?:?]",
"at sun.security.ssl.SSLTransport.decode(SSLTransport.java:167) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1462) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1370) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:437) ~[?:?]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:287) ~[opensaml-saml-impl-3.3.0.jar:?]",
"... 9 more",
"Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) ~[?:?]",
"at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]",
"at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:445) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:423) ~[?:?]",
"at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) ~[?:?]",
"at sun.security.ssl.SSLTransport.decode(SSLTransport.java:167) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1462) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1370) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:437) ~[?:?]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:287) ~[opensaml-saml-impl-3.3.0.jar:?]",
"... 9 more",
"Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:?]",
"at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:?]",
"at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) ~[?:?]",
"at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) ~[?:?]",
"at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]",
"at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]",
"at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:445) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:423) ~[?:?]",
"at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) ~[?:?]",
"at sun.security.ssl.SSLTransport.decode(SSLTransport.java:167) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1462) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1370) ~[?:?]",
"at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:437) ~[?:?]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]",
"at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:287) ~[opensaml-saml-impl-3.3.0.jar:?]",
"... 9 more"] }

Updates

2020-07-07 09:56:00-0500

The following command fails to work, when executed from the config directory:

curl --cacert certs/ca_bundle.pem https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml

However, this (which uses the host’s default CA bundle, located at /etc/ssl/certs/ca-bundle.crt) does:

curl https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml

Additionally, I’ve validated that this is the correct cert bundle, by executing the following successfully:

curl --cafile /etc/ssl/certs/ca-bundle.crt https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml

However, when I set idp.pemtrustedcas_filepath: '/etc/ssl/certs/ca-bundle.crt' in the config, I still get the same error, even though /etc/ssl/certs/ca-bundle.crt is in PEM format.

2020-07-07 08:34:00-0500

While attempting to retrieve the metadata via curl from this pod using:

curl -sk https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml

I am able to get the metadata file, as expected.

2020-07-07 08:11:00-0500

In continuing to troubleshoot with my ADFS/SAML expert, he determined that Search Guard’s SAML requests are reaching ADFS, and ADFS is attempting to return the list of claims to:

https://elk-dev.example.com/searchguard/saml/acs

However, it is getting the following:

{"statusCode":404,"error":"Not Found","message":"Not Found"}

My Kibana config (per the documentation) is:

server:
  xsrf:
    whitelist:
      # - '/searchguard/saml/acs/idpinitiated'
      - '/searchguard/saml/acs'
      - '/searchguard/saml/logout'
  # Require TLS for connections to Kibana
  ssl:
    enabled: true
    certificate: '/usr/share/kibana/config/certs/cert.pem'
    key: '/usr/share/kibana/config/certs/key.pem'

Clarifications:

ADFS/SAML expert just notified me that he receives that error when he does IDP-initiated authentication (this can coexist OK with standard SAML auth, correct?).
Determined that, with idpinitiated URL whitelisted in kibana.yml and using correct IDP-initiated endpoint, still gettting 404 error.

2020-07-07 07:50:00-0500

I have updated SG to 43.0.0, since my original post, with the same results.
If the question arises about certs - I validated that root and intermediate certs were identical myself, using shasum. Hashes were identical.

srgbnd · July 7, 2020, 5:27pm

“javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target”

Was the IDP certificate signed by certs/ca_bundle.pem that you have in idp.pemtrustedcas_filepath? It must be.

Check it

openssl verify -CAfile ca.pem idp_cert.pem

And if the intermediate certificate was used

openssl verify -CAfile root-ca.pem -untrusted signing-ca.pem idp_cert.pem

My Kibana config…

I don’t see searchguard.auth.type: "saml" in your Kibana config snippet. Do you have it?

Enable debug in Kibana, start it, try SAML authentication again, and send the log.
kibana.yml

searchguard.auth.debug: true

Doug_Renze · July 7, 2020, 6:07pm

I finally identified the issue. We’re using o365 for our authentication, and you are correct, the o365 cert is not signed by our internal CA, which is why I received that error. My resolution for this issue was to add the following lines to my Dockerfile build:

mkdir -p /usr/share/elasticsearch/config/certs; \
ln -s /etc/ssl/certs /usr/share/elasticsearch/config/certs/host; \

Next, I updated sg_config.yml as follows:

pemtrustedcas_filepath: 'certs/host/ca-bundle.crt'  # Use default host CA bundle

That eliminated the error, and now I’m seeing the following in the logs:

{"type": "server", "timestamp": "2020-07-07T17:30:53,113Z", "level": "INFO", "component": "o.o.s.m.r.i.AbstractReloadingMetadataResolver", "cluster.name": "elk-es", "node.name": "elk-es-master-1", "message": "Metadata Resolver SamlHTTPMetadataResolver com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator_1: New metadata successfully loaded for 'https://ifs.example.com/FederationMetadata/2007-06/FederationMetadata.xml'", "cluster.uuid": "xMmOk1dFTCeOmXHJyeeDgA", "node.id": "uNwNvi_DQdWz7lcBCLBQsw"  }

However, I’m still not receiving anything back from SAML, as far as I can tell. It appears that ADFS is now getting a 401/Unauthorized error.

Doug_Renze · July 7, 2020, 6:35pm

Just noticed this on a re-read of your post. No. That was not in my kibana.yml. I just added it and am redeploying as we speak.

This is now my full searchguard config in kibana.yml.

searchguard:
  # See documentation at:
  #   https://docs.search-guard.com/latest/saml-authentication#activating-saml
  allow_client_certificates: true
  auth:
    type: 'saml'
  basicauth:
    forbidden_usernames:
      - 'kibanaserver'
      - 'logstash'

Update

I’m now being directed to a SAML error page.

A quick question, while I’m at it: How do I login to Kibana using the backup authentication as LDAP or local authentication?

srgbnd · July 8, 2020, 7:46am

I don’t see searchguard.auth.debug: true in your kibana.yml snippet. It makes Search Guard produce debug messages that can be helpful to investigate the problem. Add it and try SAML auth again.

Your config should look this way:
kibana.yml

searchguard:
  # See documentation at:
  #   https://docs.search-guard.com/latest/saml-authentication#activating-saml
  allow_client_certificates: true
  auth:
    type: 'saml'
    debug: true
  basicauth:
    forbidden_usernames:
      - 'kibanaserver'
      - 'logstash'

Doug_Renze · July 8, 2020, 1:09pm

My mistake. Earlier, when I said I set searchguard.auth.debug: true, I quoted the wrong part of your reply. I actually set searchguard.auth.type: 'saml', which, along with updating the certs, resolved that part of the issue.

I’ve set searchguard.auth.debug: true now, and here’s what I got from the log:

{"type":"response","@timestamp":"2020-07-08T19:30:16Z","tags":[],"pid":6,"method":"get","statusCode":302,"req":{"url":"/auth/saml/login?nextUrl=%2F","method":"get","headers":{"host":"elk-dev.k8s.onintranet.com","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.229.80.249","userAgent":"10.229.80.249"},"res":{"statusCode":302,"responseTime":19,"contentLength":9},"message":"GET /auth/saml/login?nextUrl=%2F 302 19ms - 9.0B"}
{"type":"log","@timestamp":"2020-07-08T19:30:25Z","tags":["info","_searchguard","auth","saml"],"pid":6,"message":"Got auth header credentials, trying to authenticate"}
{"type":"log","@timestamp":"2020-07-08T19:30:25Z","tags":["info","_searchguard","auth","saml"],"pid":6,"message":"{ authHeaderCredentials:\n   { authHeaderValue:\n      'Basic a2liYW5hc2VydmVyOjAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODk=' } }"}
{"type":"log","@timestamp":"2020-07-08T19:30:25Z","tags":["info","_searchguard","auth","saml"],"pid":6,"message":"Authenticating using Basic a2liYW5hc2VydmVyOjAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODk="}
{"type":"response","@timestamp":"2020-07-08T19:30:25Z","tags":[],"pid":6,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"user-agent":"curl/7.29.0","host":"localhost:5601","accept":"*/*"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":200,"responseTime":74,"contentLength":9},"message":"GET /app/kibana 200 74ms - 9.0B"}
{"type":"log","@timestamp":"2020-07-08T19:30:35Z","tags":["info","_searchguard","auth","saml"],"pid":6,"message":"Got auth header credentials, trying to authenticate"}
{"type":"log","@timestamp":"2020-07-08T19:30:35Z","tags":["info","_searchguard","auth","saml"],"pid":6,"message":"{ authHeaderCredentials:\n   { authHeaderValue:\n      'Basic a2liYW5hc2VydmVyOjAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODk=' } }"}

Also - again, how do I login to Kibana using my backup methods of LDAP or Local User? The config instructions imply that I should be able to do this, but don’t show how to force an LDAP/Local auth. I’m currently working on configuring SAML in my test environment, but I’m not able to login to do anything else in that environment.

Update

2020-07-08 14:06:00-0500

Correcting sp.entity_id didn’t change anything, although I think that I probably had the wrong value with /kibana attached to it anyway.

2020-07-08 13:40:00-0500

Another thing that just crossed my mind…do sp.entity_id and kibana_url need to be the same? I just realized that they’re not, by virtue of the fact that I’ve got /kibana on the end of the entity_id, which I don’t use…

srgbnd · July 9, 2020, 8:08am

how do I login to Kibana using my backup methods of LDAP or Local User?

Right now you can chain multiple authentication modes in the Search Guard Elasticsearch plugin. But it is not supported in the Search Guard Kibana plugin. In Kibana you can select only one type of authentication. This is in the future queue. I can’t give you a precise time of the arrival for this feature now, maybe in the release after the next release.

Another thing that just crossed my mind…do sp.entity_id and kibana_url need to be the same?

No, they don’t need to be the same. The sp.entity_id is an identifier of the identity service provider. You need to configure it on the service provider side. Read this article: Using SAML with Kibana | Search Guard | Security for Elasticsearch

Doug_Renze · July 9, 2020, 3:14pm

Thank you! I’ve read that post multiple times, and this is the first time I’ve noticed the comment that sp.entity_id may be the same as <Audience /> in the response. I observed that there is a typo in the <Audience /> in the response. Still not successfully logging in, but now I’m getting a slightly different response. I will post further updates as I have them.

Ahh. I misread the docs. Please accept this as a +1 for this feature!!!

Updates

2020-07-09 12:01:00-0500

When I attempt to do an IDP-initiated login with searchguard.auth.debug: true in kibana.yml, I can now see the expected response content coming back from the SAML provider when I’m doing an IDP-initiated login. However, it still doesn’t actually log me in (see XML, below, which was taken and decoded from the log line immediately above it). But I’m not seeing anything when I attempt to initiate a SAML login by going to Kibana.

Log entry

{"type":"response","@timestamp":"2020-07-09T17:10:48Z","tags":[],"pid":6,"method":"get","statusCode":302,"req":{"url":"/searchguard/saml/acs?SAMLResponse=...","method":"get","headers":{"host":"elk.dev.example.com","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.229.91.149","userAgent":"10.229.91.149"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /searchguard/saml/acs?SAMLResponse=...&SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256 302 2ms - 9.0B"}

SAML Response

<samlp:Response Consent="urn:oasis:names:tc:SAML:2.0:consent:unspecified" Destination="https://elk.dev.example.com/searchguard/saml/acs" ID="_769a7a29-06ca-4e39-940a-52e91f6cd23b" IssueInstant="2020-07-09T16:21:26.210Z" Version="2.0" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol">
    <Issuer xmlns="urn:oasis:names:tc:SAML:2.0:assertion">http://ifs.example.com/adfs/services/trust</Issuer>
    <samlp:Status>
        <samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success"/>
    </samlp:Status>
    <Assertion ID="_20f3a6f5-0998-4f8c-931b-d11360f37815" IssueInstant="2020-07-09T16:21:26.210Z" Version="2.0" xmlns="urn:oasis:names:tc:SAML:2.0:assertion">
        <Issuer>http://ifs.example.com/adfs/services/trust</Issuer>
        <Subject>
            <NameID>me@example.com</NameID>
            <SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
                <SubjectConfirmationData NotOnOrAfter="2020-07-09T16:26:26.210Z" Recipient="https://elk.dev.example.com/searchguard/saml/acs"/>
            </SubjectConfirmation>
        </Subject>
        <Conditions NotBefore="2020-07-09T16:21:26.210Z" NotOnOrAfter="2020-07-09T17:21:26.210Z">
            <AudienceRestriction>
                <Audience>https://elk.dev.example.com/kibana</Audience>
            </AudienceRestriction>
        </Conditions>
        <AttributeStatement>
            <Attribute Name="http://schemas.microsoft.com/ws/2008/06/identity/claims/role">
                <AttributeValue>SG_ADMIN</AttributeValue>
                <AttributeValue>SG_USER</AttributeValue>
            </Attribute>
        </AttributeStatement>
        <AuthnStatement AuthnInstant="2020-07-09T14:34:59.445Z" SessionIndex="_20f3a6f5-0998-4f8c-931b-d11360f37815">
            <AuthnContext>
                <AuthnContextClassRef>urn:federation:authentication:windows</AuthnContextClassRef>
            </AuthnContext>
        </AuthnStatement>
    </Assertion>
</samlp:Response>

srgbnd · July 9, 2020, 5:36pm

I’ll try to reproduce this on Monday and let you know.

srgbnd · July 13, 2020, 8:05am

Today I have tried the SAML test environment. The authentication worked for me. See the configs I tried, some tips and questions below.

The configs

Keycloak

sg_config.yml

_sg_meta:
  type: "config"
  config_version: 2

sg_config:
  dynamic:
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        internalProxies: '.*' # trust all internal proxies, regex pattern
        remoteIpHeader:  'x-forwarded-for'
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      saml_auth_domain_keycloak:
         http_enabled: true
         transport_enabled: true
         order: 1
         http_authenticator:
            type: 'com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator'
            challenge: true
            config:
                idp:
                  metadata_url: http://keycloak.example.com:8080/auth/realms/master/protocol/saml/descriptor
                  entity_id: http://keycloak.example.com:8080/auth/realms/master
                sp:
                  entity_id: es-saml
                kibana_url: https://kibana.example.com:5601/
                roles_key: Role
                jwt:
                  key:
                    kty: oct
                    use: sig
                    k: 'peuvg...Sg'
                    alg: HS512
         authentication_backend:
            type: noop

Auth0

sg_config.yml

_sg_meta:
  type: "config"
  config_version: 2

sg_config:
  dynamic:
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        internalProxies: '.*' # trust all internal proxies, regex pattern
        remoteIpHeader:  'x-forwarded-for'
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      saml_auth_domain_auth0:
         http_enabled: true
         transport_enabled: true
         order: 10
         http_authenticator:
            type: 'com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator'
            challenge: true
            config:
                idp:
                    metadata_file: auth0.xml
                    entity_id: urn:wolf.eu.auth0.com
                sp:
                    entity_id: es-saml
                kibana_url: https://kibana.example.com:5601/
                subject_key: http://schemas.auth0.com/https://kibana;example;com/preferred_username
                roles_key: http://schemas.auth0.com/https://kibana;example;com/roles
                roles_seperator: ","
                exchange_key: 'peuv..KSHDg'
         authentication_backend:
            type: noop

The tips and questions

sp: entity_id: ‘https://elk.dev.example.com/kibana’

Are you sure you have https://elk.dev.example.com/kibana as the client/application id in the IDP configuration? Maybe you have another value there; please double-check this.

You probably did it already, but it doesn’t hurt to read and try things from the SAML troubleshooting guide one more time. See whether you have everything in order Kibana SAML Advanced Troubleshooting | Security for Elasticsearch | Search Guard
If you still can’t see the problem, post the log with SAML debug enabled.

What IDP provider do you use? Can you share its settings?

Doug_Renze · July 13, 2020, 1:20pm

This is specified in the <Audience /> tag of the SAML response, so it appears that it is.

Ya - I’ve been through this several times, will do so again.

PM’ing you the logs.

srgbnd · July 13, 2020, 4:10pm

What IDP provider do you use? Can you share its settings?

srgbnd · July 13, 2020, 4:12pm

I see, it looks like you use OASIS.

Doug_Renze · July 14, 2020, 12:23pm

ADFS, actually. Settings PM’d.

Doug_Renze · July 14, 2020, 1:11pm

Issue appears to be resolved. Once you got me as far as you did, we started going through IDP settings and were able to get it running by switching our claims from ADFS auto-config to a custom config, which resolved the issue with the Roles not being presented.

Topic		Replies	Views
SAML Configuration Error Search Guard	6	797	September 17, 2019
SAML for kibana Single Sign-on using Okta Search Guard	5	1111	December 11, 2018
Add SAML role to SearchGuard role Search Guard	15	1061	March 31, 2021
Basic Auth with SAML Search Guard	3	525	August 26, 2021
SAML SSO with Kibana Search Guard	20	2415	August 31, 2018

Issues configuring SAML authentication

Implementation Details

Issues

SAML Config

Debug Log

Updates

2020-07-07 09:56:00-0500

2020-07-07 08:34:00-0500

2020-07-07 08:11:00-0500

2020-07-07 07:50:00-0500

Update

Update

2020-07-08 14:06:00-0500

2020-07-08 13:40:00-0500

Updates

2020-07-09 12:01:00-0500

Log entry

SAML Response

The configs

Keycloak

Auth0

The tips and questions

Related Topics