SAML SSO with Kibana

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

That sounds like you are using curl to attempt auth. You need to specify the header type in your curl if so.

···

On Wed, Aug 29, 2018 at 10:30 AM dchan@galileo.io wrote:

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a81041fd-9462-479f-91f4-670b4b5d86e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

So, this is where I’m a little unsure on how to specify the header type. Is anybody aware on if that’s something that can be configured with Google? https://support.google.com/a/answer/6087519?hl=en

I guess I want to make sure that I rule out any potential issues in my Elasticsearch configuration before attempting to debug my Google SSO configuration around header type specification.

···

On Wednesday, August 29, 2018 at 10:32:24 AM UTC-4, erik clark wrote:

That sounds like you are using curl to attempt auth. You need to specify the header type in your curl if so.

On Wed, Aug 29, 2018 at 10:30 AM dc...@galileo.io wrote:

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a81041fd-9462-479f-91f4-670b4b5d86e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

SAML is not suited for authenticating against Elasticsearch directly. As OpenID it defines a request flow, redirects, response types etc. that cannot be used when interacting with ES directly. Therefore we recommend in the documentation that you should also configure another authentication domain, like Basic Auth + internal database in your case. That way you can still use curl for querying ES directly.

In other words - SAML is only really useful and usable in combination with Kibana.

···

On Wednesday, August 29, 2018 at 10:35:12 AM UTC-4, dchan@galileo.io wrote:

So, this is where I’m a little unsure on how to specify the header type. Is anybody aware on if that’s something that can be configured with Google? https://support.google.com/a/answer/6087519?hl=en

I guess I want to make sure that I rule out any potential issues in my Elasticsearch configuration before attempting to debug my Google SSO configuration around header type specification.

On Wednesday, August 29, 2018 at 10:32:24 AM UTC-4, erik clark wrote:

That sounds like you are using curl to attempt auth. You need to specify the header type in your curl if so.

On Wed, Aug 29, 2018 at 10:30 AM dc...@galileo.io wrote:

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a81041fd-9462-479f-91f4-670b4b5d86e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Got it! I’ll take a look at my Google configuration and post back here with some updates. Thanks for the help and responses so far.

···

On Wednesday, August 29, 2018 at 10:59:03 AM UTC-4, Jochen Kressin wrote:

SAML is not suited for authenticating against Elasticsearch directly. As OpenID it defines a request flow, redirects, response types etc. that cannot be used when interacting with ES directly. Therefore we recommend in the documentation that you should also configure another authentication domain, like Basic Auth + internal database in your case. That way you can still use curl for querying ES directly.

In other words - SAML is only really useful and usable in combination with Kibana.

On Wednesday, August 29, 2018 at 10:35:12 AM UTC-4, dc...@galileo.io wrote:

So, this is where I’m a little unsure on how to specify the header type. Is anybody aware on if that’s something that can be configured with Google? https://support.google.com/a/answer/6087519?hl=en

I guess I want to make sure that I rule out any potential issues in my Elasticsearch configuration before attempting to debug my Google SSO configuration around header type specification.

On Wednesday, August 29, 2018 at 10:32:24 AM UTC-4, erik clark wrote:

That sounds like you are using curl to attempt auth. You need to specify the header type in your curl if so.

On Wed, Aug 29, 2018 at 10:30 AM dc...@galileo.io wrote:

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a81041fd-9462-479f-91f4-670b4b5d86e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

When setting the SSO Url in Google to https://kibana.xxx/searchguard/saml/acs, I receive the following error when attempting to SSO from Google: {“statusCode”:404,“error”:“Not Found”}

Am I possibly missing something in my Kibana setup?

···

On Wed, Aug 29, 2018 at 11:00 AM, dchan@galileo.io wrote:

Got it! I’ll take a look at my Google configuration and post back here with some updates. Thanks for the help and responses so far.

On Wednesday, August 29, 2018 at 10:59:03 AM UTC-4, Jochen Kressin wrote:

SAML is not suited for authenticating against Elasticsearch directly. As OpenID it defines a request flow, redirects, response types etc. that cannot be used when interacting with ES directly. Therefore we recommend in the documentation that you should also configure another authentication domain, like Basic Auth + internal database in your case. That way you can still use curl for querying ES directly.

In other words - SAML is only really useful and usable in combination with Kibana.

On Wednesday, August 29, 2018 at 10:35:12 AM UTC-4, dc...@galileo.io wrote:

So, this is where I’m a little unsure on how to specify the header type. Is anybody aware on if that’s something that can be configured with Google? https://support.google.com/a/answer/6087519?hl=en

I guess I want to make sure that I rule out any potential issues in my Elasticsearch configuration before attempting to debug my Google SSO configuration around header type specification.

On Wednesday, August 29, 2018 at 10:32:24 AM UTC-4, erik clark wrote:

That sounds like you are using curl to attempt auth. You need to specify the header type in your curl if so.

On Wed, Aug 29, 2018 at 10:30 AM dc...@galileo.io wrote:

Using kibana 6.3.2-14 and Elasticsearch 6.3.2-23.0, attempting to setup Google SSO using SAML.

I’m currently running into an error where https://elasticsearch.xxx.com/searchguard/saml/acs is returning: {“error”:“Content-Type header [application/x-www-form-urlencoded] is not supported”,“status”:406}

Have I misconfigured something in my sg_config.yml or possibly Google SSO settings? Unsure on what I could try doing to debug the situation.

Thanks in advance,

Dan

sg_config.yml

authc:

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: basic

challenge: false

authentication_backend:

type: internal

saml:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: saml

challenge: true

config:

idp.metadata_file: GoogleIDPMetadata-galileo.io.xml

idp.entity_id: https://accounts.google.com/o/saml2/idp?idpid=xxxxxxxx

sp.entity_id: KIBANA_URL

kibana_url: KIBANA_URL

subject_key: emailAddress

roles_key: role

exchange_key: SAML_EXCHANGE_KEY

authentication_backend:

type: noop

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/a81041fd-9462-479f-91f4-670b4b5d86e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to a topic in the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/meqs61LQa9s/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/8089196f-bcb3-4f88-b174-e120147c9642%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

For additional information, my Kibana configuration looks like such:

···

server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"logger.token2.name": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

···

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

···

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dchan@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Hi,

I’ve figured out that my curl command is misconfigured.

Based on https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html and https://docs.search-guard.com/latest/troubleshooting-saml, I believe the curl command I want is:

curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

“logger.com.floragunn.dlic.auth.http.saml.Token”: “debug”

}

}

But, this seems to send debug log for much more than just com.floragunn.dlic.auth.http.saml.Token.

Even so, I’m not finding any logs for c.o.s.a.SamlResponse or c.f.d.a.h.s.Token

I am seeing logs for c.o.s.a.AuthnRequest, but nothing afterwards. Does this log indicate that something is misconfigured?

[2018-08-30T20:42:24,358][DEBUG][c.o.s.a.AuthnRequest ] AuthNRequest --> <samlp:AuthnRequest xmlns:samlp=“urn:oasis:names:tc:SAML:2.0:protocol” xmlns:saml=“urn:oasis:names:tc:SAML:2.0:assertion” ID=“ONELOGIN_bd2754fd-4c39-48d8-8109-7dd9ae2442b9” Version=“2.0” IssueInstant=“2018-08-30T20:42:24Z” ForceAuthn=“true” Destination=“https://accounts.google.com/o/saml2/idp?idpid=#####” ProtocolBinding=“urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST” AssertionConsumerServiceURL=“https://kibana.xxx:443/searchguard/saml/acs”>saml:Issuerhttps://kibana.xxx:443/</saml:Issuer><samlp:NameIDPolicy Format=“urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified” AllowCreate=“true” /></samlp:AuthnRequest>

Hoping somebody has seen something similar, and I appreciate any help or comments while trying to debug this SAML setup.

Thanks,

Dan

···

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

···

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

···

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dchan@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

···

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

···

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

···

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

Would it matter if https://kibana.xxx and https://elasticsearch.xxx are not accessible from the public internet? These are currently resources that reside within a private AWS VPC.

Trying to rule out any possible sources of failure.

···

On Friday, August 31, 2018 at 12:23:06 PM UTC-4, dc...@galileo.io wrote:

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

I think my question boils down to: What are the different reasons that Google SAML app could be redirecting me to https://kibana.xxx/customerror?type=samlAuthError#?_g=()? I’ve read the troubleshooting guide at https://docs.search-guard.com/latest/troubleshooting-saml, but have run out of ideas for how to proceed with the setup.

···

On Fri, Aug 31, 2018 at 12:29 PM dchan@galileo.io wrote:

Would it matter if https://kibana.xxx and https://elasticsearch.xxx are not accessible from the public internet? These are currently resources that reside within a private AWS VPC.

Trying to rule out any possible sources of failure.

On Friday, August 31, 2018 at 12:23:06 PM UTC-4, dc...@galileo.io wrote:

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

You received this message because you are subscribed to a topic in the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/meqs61LQa9s/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/e8e4fe9e-e5ce-4371-97b8-6a701120a1e4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

After changing my authc entry to:

saml_auth_domain_google:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: ‘saml’

challenge: true

config:

idp:

metadata_file: GoogleIDPMetadata-galileo.io.xml

entity_id: https://accounts.google.com/o/saml2?idpid=#####

sp:

entity_id: __SP_ENTITY_ID__searchguard/saml/acs

kibana_url: KIBANA_URL

roles_key: role

exchange_key: ‘SAML_EXCHANGE_KEY

authentication_backend:

type: noop

After booting up Elasticsearch, I found in the logs:

[2018-08-31T20:43:15,351][ERROR][c.f.s.a.BackendRegistry ] Unable to initialize auth domain saml_auth_domain_google due to ElasticsearchException[Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled]

org.elasticsearch.ElasticsearchException: Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled

Is HTTPSamlAuthenticator an enterprise feature of Search Guard? Do I require a Search Guard enterprise license to use this feature?

My cluster settings are set to:

{"persistent":{},"transient":{"logger":{"com":{"floragunn":{"dlic":"debug"}}}}}
···

On Fri, Aug 31, 2018 at 4:20 PM, Dan Chan dchan@galileo.io wrote:

I think my question boils down to: What are the different reasons that Google SAML app could be redirecting me to https://kibana.xxx/customerror?type=samlAuthError#?_g=()? I’ve read the troubleshooting guide at https://docs.search-guard.com/latest/troubleshooting-saml, but have run out of ideas for how to proceed with the setup.

On Fri, Aug 31, 2018 at 12:29 PM dchan@galileo.io wrote:

Would it matter if https://kibana.xxx and https://elasticsearch.xxx are not accessible from the public internet? These are currently resources that reside within a private AWS VPC.

Trying to rule out any possible sources of failure.

On Friday, August 31, 2018 at 12:23:06 PM UTC-4, dc...@galileo.io wrote:

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

You received this message because you are subscribed to a topic in the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/meqs61LQa9s/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/e8e4fe9e-e5ce-4371-97b8-6a701120a1e4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Yes, it is. For an overview about the Enterprise and non-Enterprise features you can refer to the website:

https://search-guard.com/product/

In the docs the features are also marked in the upper right corner:

That would explain why you get some strange SAML errors. However, if you install SG fresh it will automatically create a trial license with all features enabled, that’s why it actually did not came to my mind that disabled enterprise features could be the cause. Sorry about that.

That also explains why enabling the debug setting for the Token logger would not yield any effect. But in my opinion the message:

Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled

``

should appear every time you either try to change the sg_config settings or restart the cluster. Did it not appear before on your system?

To answer your question regaring other ways of debugging, you can always try to capture the HTTP traffic between Kibana/Elasticsearch/Idp manually for debugging. A bit more work, but should work in any case. The first request you should see is the post-back from the IdP to the acs endpoint:

The SAMLResponse is the base64 encoded XML from the IdP. You can just decode it and check whether the content matches your expectations. That would be the first step. If you want to also inspect the traded JWT then you need to activate the Token logger.

Hope that helps and let me know if you need further information.

···

On Friday, August 31, 2018 at 4:59:30 PM UTC-4, Dan Chan wrote:

After changing my authc entry to:

saml_auth_domain_google:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: ‘saml’

challenge: true

config:

idp:

metadata_file: GoogleIDPMetadata-galileo.io.xml

entity_id: https://accounts.google.com/o/saml2?idpid=#####

sp:

entity_id: __SP_ENTITY_ID__searchguard/saml/acs

kibana_url: KIBANA_URL

roles_key: role

exchange_key: ‘SAML_EXCHANGE_KEY

authentication_backend:

type: noop

After booting up Elasticsearch, I found in the logs:

[2018-08-31T20:43:15,351][ERROR][c.f.s.a.BackendRegistry ] Unable to initialize auth domain saml_auth_domain_google due to ElasticsearchException[Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled]

org.elasticsearch.ElasticsearchException: Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled

Is HTTPSamlAuthenticator an enterprise feature of Search Guard? Do I require a Search Guard enterprise license to use this feature?

My cluster settings are set to:

{"persistent":{},"transient":{"logger":{"com":{"floragunn":{"dlic":"debug"}}}}}

On Fri, Aug 31, 2018 at 4:20 PM, Dan Chan dchan@galileo.io wrote:

I think my question boils down to: What are the different reasons that Google SAML app could be redirecting me to https://kibana.xxx/customerror?type=samlAuthError#?_g=()? I’ve read the troubleshooting guide at https://docs.search-guard.com/latest/troubleshooting-saml, but have run out of ideas for how to proceed with the setup.

On Fri, Aug 31, 2018 at 12:29 PM dchan@galileo.io wrote:

Would it matter if https://kibana.xxx and https://elasticsearch.xxx are not accessible from the public internet? These are currently resources that reside within a private AWS VPC.

Trying to rule out any possible sources of failure.

On Friday, August 31, 2018 at 12:23:06 PM UTC-4, dc...@galileo.io wrote:

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

You received this message because you are subscribed to a topic in the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/meqs61LQa9s/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/e8e4fe9e-e5ce-4371-97b8-6a701120a1e4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

That helps immensely, thanks for the confirmation.

I did not notice the Enterprise icon in the upper right corner. I think I scanned over every other part of the page, but my eyes never noticed the Enterprise icon. That makes more sense.

The message:

Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled
did not appear because I did not have com.floragunn.dlic set to debug?

Either that, or I never noticed it.

Coincidentally, I did also upgrade from 6.3.2 to 6.4.0. It’s possible that I didn’t notice that log in 6.3.2, or the message did not appear.

It does appear, definitively, in 6.4.0, and I do have com.floragunn.dlic set to debug in the transient settings.

Capturing the HTTP traffic makes sense, but because the HTTPSamlAuthenticator was never started, I don’t think I ever saw that acs entry in the Network tab of my browser. After I enable Enterprise license, I’ll keep an eye out for the acs entry for debugging purposes. Decoding the base64 encoded XML from the IdP should yield some more information, if I run into more issues.

I’ll abandon this thread, as it stands. I think there is enough information here on how to debug, after I enable the enterprise functionality.

Thanks again,

Dan

···

On Friday, August 31, 2018 at 5:28:07 PM UTC-4, Jochen Kressin wrote:

Yes, it is. For an overview about the Enterprise and non-Enterprise features you can refer to the website:

https://search-guard.com/product/

In the docs the features are also marked in the upper right corner:

That would explain why you get some strange SAML errors. However, if you install SG fresh it will automatically create a trial license with all features enabled, that’s why it actually did not came to my mind that disabled enterprise features could be the cause. Sorry about that.

That also explains why enabling the debug setting for the Token logger would not yield any effect. But in my opinion the message:

Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled

``

should appear every time you either try to change the sg_config settings or restart the cluster. Did it not appear before on your system?

To answer your question regaring other ways of debugging, you can always try to capture the HTTP traffic between Kibana/Elasticsearch/Idp manually for debugging. A bit more work, but should work in any case. The first request you should see is the post-back from the IdP to the acs endpoint:

The SAMLResponse is the base64 encoded XML from the IdP. You can just decode it and check whether the content matches your expectations. That would be the first step. If you want to also inspect the traded JWT then you need to activate the Token logger.

Hope that helps and let me know if you need further information.

On Friday, August 31, 2018 at 4:59:30 PM UTC-4, Dan Chan wrote:

After changing my authc entry to:

saml_auth_domain_google:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: ‘saml’

challenge: true

config:

idp:

metadata_file: GoogleIDPMetadata-galileo.io.xml

entity_id: https://accounts.google.com/o/saml2?idpid=#####

sp:

entity_id: __SP_ENTITY_ID__searchguard/saml/acs

kibana_url: KIBANA_URL

roles_key: role

exchange_key: ‘SAML_EXCHANGE_KEY

authentication_backend:

type: noop

After booting up Elasticsearch, I found in the logs:

[2018-08-31T20:43:15,351][ERROR][c.f.s.a.BackendRegistry ] Unable to initialize auth domain saml_auth_domain_google due to ElasticsearchException[Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled]

org.elasticsearch.ElasticsearchException: Can not load ‘com.floragunn.dlic.auth.http.saml.HTTPSamlAuthenticator’ because enterprise modules are disabled

Is HTTPSamlAuthenticator an enterprise feature of Search Guard? Do I require a Search Guard enterprise license to use this feature?

My cluster settings are set to:

{"persistent":{},"transient":{"logger":{"com":{"floragunn":{"dlic":"debug"}}}}}

On Fri, Aug 31, 2018 at 4:20 PM, Dan Chan dc...@galileo.io wrote:

I think my question boils down to: What are the different reasons that Google SAML app could be redirecting me to https://kibana.xxx/customerror?type=samlAuthError#?_g=()? I’ve read the troubleshooting guide at https://docs.search-guard.com/latest/troubleshooting-saml, but have run out of ideas for how to proceed with the setup.

On Fri, Aug 31, 2018 at 12:29 PM dc...@galileo.io wrote:

Would it matter if https://kibana.xxx and https://elasticsearch.xxx are not accessible from the public internet? These are currently resources that reside within a private AWS VPC.

Trying to rule out any possible sources of failure.

On Friday, August 31, 2018 at 12:23:06 PM UTC-4, dc...@galileo.io wrote:

Something feels a bit off to me.

I reconfigured Elasticsearch and disabled the authc SAML configuration. I’m still receiving the same https://kibana.xxx/customerror?type=samlAuthError#?_g=() page from Kibana when attempting to SSO with Google.

Is my Kibana setup not forwarding the request to Elasticsearch properly? It feels strange to not have any logs in Elasticsearch, showing the POST request from Kibana. And receiving the same error page on the Kibana side also seems suspicious.

Thanks for the patience so far, I really appreciate the responses.

On Thursday, August 30, 2018 at 6:53:14 PM UTC-4, dc...@galileo.io wrote:

If it helps, this is the POST from Kibana.

{

“type”: “response”,

@timestamp”: “2018-08-30T22:32:18Z”,

“tags”: ,

“pid”: 56,

“method”: “post”,

“statusCode”: 302,

“req”: {

“url”: “/searchguard/saml/acs”,

“method”: “post”,

“headers”: {

“x-forwarded-for”: “172.31.3.26”,

“x-forwarded-proto”: “https”,

“x-forwarded-port”: “443”,

“host”: “kibana.xxx”,

“x-amzn-trace-id”: “Root=1-5b887072-d2b37cf4630cf00895ef73c4”,

“content-length”: “5997”,

“cache-control”: “max-age=0”,

“origin”: “https://accounts.google.com”,

“upgrade-insecure-requests”: “1”,

“content-type”: “application/x-www-form-urlencoded”,

“user-agent”: “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36”,

“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false”,

“accept-encoding”: “gzip, deflate, br”,

“accept-language”: “en-US,en;q=0.9”

},

“remoteAddress”: “x.x.x.x”,

“userAgent”: “x.x.x.x”,

“referer”: “https://accounts.google.com/o/saml2/initsso?idpid=#####&spid=#####&forceauthn=false

},

“res”: {

“statusCode”: 302,

“responseTime”: 29,

“contentLength”: 9

},

“message”: “POST /searchguard/saml/acs 302 29ms - 9.0B”

}

On Thursday, August 30, 2018 at 6:35:28 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Okay, so I don’t know why it initially failed to boot, but took your advice in setting

logger.token.name = com.floragunn.dlic.auth.http.saml.Token

logger.token.level = debug

in log4j2.properties, and Elasticsearch boots properly.

I’m still not seeing any debug logs for Token however, when attempting to login using SAML Google SSO. I receive the SAML authentication error on https://kibana.xxx/customerror?type=samlAuthError#?_g=(), but can’t seem to find any corresponding logs in Elasticsearch.

Is there another set of logs that I can look at that might provide more information?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:43:17 PM UTC-4, Jochen Kressin wrote:

Well, the error message indicates that you confused the logger name with the log level (“Unknown level constant”). Something like:

logger.token.name: error

logger.token.level: com.floragunn.dlic.auth.http.saml.Token

``

Can you check for this both in log4j2.properties and elasticsearch.yml?

On Thursday, August 30, 2018 at 5:32:01 PM UTC-4, dc...@galileo.io wrote:

Hi Jochen,

Thanks for the replies so far.

I’m seeing an error when I add the logger in log4j2 properties in my Elasticsearch cluster, along the lines of Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].

Is there a way to get around this, or is this error supposed to not be happening?

Thanks,

Dan

On Thursday, August 30, 2018 at 5:28:21 PM UTC-4, Jochen Kressin wrote:

I think you can only change log levels via the cluster API. You will not be able to add a new logger that way.

So you can add the logger in log4j2.properties and set it to error:

logger.token.name = com.floragunn.dlic.auth.http.saml.Token
logger.token.level = error

``

And then use the cluster API to dynamically switch on debug level when needed.

On Thursday, August 30, 2018 at 2:43:13 PM UTC-4, dc...@galileo.io wrote:

Update with a new question!

Figured out that my Kibana setup was not deploying properly, and now I am correctly forwarding requests from Kibana to Elasticsearch, using “/searchguard/saml/acs”. Also, removed the duplicate YAML key for “server.xsrf.whitelist”

New question:

I am trying to debug SAML authentication error from the SSO (https://docs.search-guard.com/latest/troubleshooting-saml)

I attempted to enable com.floragunn.dlic.auth.http.saml.Token via cluster settings, but I am receiving an error response of illegal_argument_exception

$ curl -X PUT “https://user:password@elasticsearch.xxx:443/_cluster/settings” -H ‘Content-Type: application/json’ -d’

{

“transient”: {

"[logger.token2.name](http://logger.token2.name)": "com.floragunn.dlic.auth.http.saml.Token",
"logger.token2.level": "debug"

}

}

{“error”:{“root_cause”:[{“type”:“remote_transport_exception”,“reason”:"[CEPj10A][1.2.3.4:9300][cluster:admin/settings/update]"}],“type”:“illegal_argument_exception”,“reason”:“Unknown level constant [COM.FLORAGUNN.DLIC.AUTH.HTTP.SAML.TOKEN].”},“status”:400}

What else can I try to debug the SAML authentication error?

Thanks for the help so far,

Dan

On Wednesday, August 29, 2018 at 3:28:49 PM UTC-4, dc...@galileo.io wrote:

For additional information, my Kibana configuration looks like such:


server.name: kibana

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.certificate: /usr/share/kibana/config/kibana.crt.pem

server.ssl.key: /usr/share/kibana/config/kibana.key.pem

elasticsearch.url: ELASTICSEARCH_URL

elasticsearch.username: ELASTICSEARCH_USERNAME

elasticsearch.password: ELASTICSEARCH_PASSWORD

searchguard.basicauth.enabled: true

searchguard.cookie.secure: true

searchguard.cookie.name: SEARCHGUARD_COOKIE_NAME

searchguard.cookie.password: SEARCHGUARD_COOKIE_PASSWORD

searchguard.cookie.ttl: 28800000

searchguard.session.ttl: 28800000

searchguard.session.keepalive: false

searchguard.auth.type: “saml”

server.xsrf.whitelist: ["/searchguard/saml/acs"]

server.xsrf.whitelist: ["/searchguard/saml/acs", “/searchguard/saml/logout”]

You received this message because you are subscribed to a topic in the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/search-guard/meqs61LQa9s/unsubscribe.

To unsubscribe from this group and all its topics, send an email to search-guard...@googlegroups.com.

To post to this group, send email to search...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/e8e4fe9e-e5ce-4371-97b8-6a701120a1e4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.