Hi there
I am configuring kibana 7.2 with Single Sign-On with OpenID and Keycloak for https and its throwing error unable to obtain IDP endpoint. q
I did the same today and it worked. Show the corresponding parts of your kibana.yml and sg_config.yml
sg_config.yml
# This is the main Search Guard configuration file where authentication
# and authorization is defined.
#
# You need to configure at least one authentication domain in the authc of this file.
# An authentication domain is responsible for extracting the user credentials from
# the request and for validating them against an authentication backend like Active Directory for example.
#
# If more than one authentication domain is configured the first one which succeeds wins.
# If all authentication domains fail then the request is unauthenticated.
# In this case an exception is thrown and/or the HTTP status is set to 401.
#
# After authentication authorization (authz) will be applied. There can be zero or more authorizers which collect
# the roles from a given backend for the authenticated user.
#
# Both, authc and auth can be enabled/disabled separately for REST and TRANSPORT layer. Default is true for both.
# http_enabled: true
# transport_enabled: true
#
# For HTTP it is possible to allow anonymous authentication. If that is the case then the HTTP authenticators try to
# find user credentials in the HTTP request. If credentials are found then the user gets regularly authenticated.
# If none can be found the user will be authenticated as an "anonymous" user. This user has always the username "sg_anonymous"
# and one role named "sg_anonymous_backendrole".
# If you enable anonymous authentication all HTTP authenticators will not challenge.
#
#
# Note: If you define more than one HTTP authenticators make sure to put non-challenging authenticators like "proxy" or "clientcert"
# first and the challenging one last.
# Because it's not possible to challenge a client with two different authentication methods (for example
# Kerberos and Basic) only one can have the challenge flag set to true. You can cope with this situation
# by using pre-authentication, e.g. sending a HTTP Basic authentication header in the request.
#
# Default value of the challenge flag is true.
#
#
# HTTP
# basic (challenging)
# proxy (not challenging, needs xff)
# kerberos (challenging) NOT FREE FOR COMMERCIAL
# clientcert (not challenging, needs https)
# jwt (not challenging) NOT FREE FOR COMMERCIAL
# host (not challenging) #DEPRECATED, will be removed in a future version.
# host based authentication is configurable in sg_roles_mapping
# Authc
# internal
# noop
# ldap NOT FREE FOR COMMERCIAL USE
# Authz
# ldap NOT FREE FOR COMMERCIAL USE
# noop
# For more details pls refer to https://docs.search-guard.com/latest/authentication-authorization
_sg_meta:
type: "config"
config_version: 2
sg_config:
dynamic:
# Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
# Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
# Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
#filtered_alias_mode: warn
#do_not_fail_on_forbidden: false
#kibana:
# Kibana multitenancy - NOT FREE FOR COMMERCIAL USE
# see https://github.com/floragunncom/search-guard-docs/blob/master/multitenancy.md
# To make this work you need to install https://github.com/floragunncom/search-guard-module-kibana-multitenancy/wiki
#multitenancy_enabled: true
#server_username: kibanaserver
#index: '.kibana'
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
#internalProxies: '.*' # trust all internal proxies, regex pattern
#remoteIpHeader: 'x-forwarded-for'
###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
###### and here https://tools.ietf.org/html/rfc7239
###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
authc:
openid_auth_domain:
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: openid
challenge: false
config:
subject_key: preferred_username
roles_key: roles
openid_connect_url: "https://domain_name/auth/realms/kibana-realm/.well-known/openid-configuration"
#enable_ssl: false
#verify_hosts: false
#enable_ssl_client_auth: false
#pemtrustedcas_filepath: "/home/smart/smart/CACert.crt"
#pemcert_filepath: "/home/smart/smart/cert_name.crt"
#pemkey_filepath: "/home/smart/smart/cert_name.crt.key"
authentication_backend:
type: noop
kerberos_auth_domain:
http_enabled: false
transport_enabled: false
order: 6
http_authenticator:
type: kerberos # NOT FREE FOR COMMERCIAL USE
challenge: true
config:
# If true a lot of kerberos/security related debugging output will be logged to standard out
krb_debug: false
# If true then the realm will be stripped from the user name
strip_realm_from_principal: true
authentication_backend:
type: noop
basic_internal_auth_domain:
description: "Authenticate via HTTP Basic against internal users database"
http_enabled: true
transport_enabled: true
order: 4
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
proxy_auth_domain:
description: "Authenticate via proxy"
http_enabled: false
transport_enabled: false
order: 3
http_authenticator:
type: proxy
challenge: false
config:
user_header: "x-proxy-user"
roles_header: "x-proxy-roles"
authentication_backend:
type: noop
jwt_auth_domain:
http_enabled: false
transport_enabled: false
order: 2
http_authenticator:
type: jwt
challenge: false
config:
signing_key: |-
-----BEGIN PUBLIC KEY-----
key_conatct
-----END PUBLIC KEY-----
jwt_header: "Authorization"
jwt_url_parameter: jwtparam
roles_key: roles
subject_key: preferred_username
authentication_backend:
type: noop
clientcert_auth_domain:
description: "Authenticate via SSL client certificates"
http_enabled: false
transport_enabled: false
order: 2
http_authenticator:
type: clientcert
config:
username_attribute: cn #optional, if omitted DN becomes username
challenge: false
authentication_backend:
type: noop
ldap:
description: "Authenticate via LDAP or Active Directory"
http_enabled: false
transport_enabled: false
order: 5
http_authenticator:
type: basic
challenge: false
authentication_backend:
# LDAP authentication backend (authenticate users against a LDAP or Active Directory)
type: ldap # NOT FREE FOR COMMERCIAL USE
config:
# enable ldaps
enable_ssl: false
# enable start tls, enable_ssl should be false
enable_start_tls: false
# send client certificate
enable_ssl_client_auth: false
# verify ldap hostname
verify_hostnames: true
hosts:
- localhost:8389
bind_dn: null
password: null
userbase: 'ou=people,dc=example,dc=com'
# Filter to search for users (currently in the whole subtree beneath userbase)
# {0} is substituted with the username
usersearch: '(sAMAccountName={0})'
# Use this attribute from the user as username (if not set then DN is used)
username_attribute: null
authz:
roles_from_myldap:
description: "Authorize via LDAP or Active Directory"
http_enabled: false
transport_enabled: false
authorization_backend:
# LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
type: ldap # NOT FREE FOR COMMERCIAL USE
config:
# enable ldaps
enable_ssl: false
# enable start tls, enable_ssl should be false
enable_start_tls: false
# send client certificate
enable_ssl_client_auth: false
# verify ldap hostname
verify_hostnames: true
hosts:
- localhost:8389
bind_dn: null
password: null
rolebase: 'ou=groups,dc=example,dc=com'
# Filter to search for roles (currently in the whole subtree beneath rolebase)
# {0} is substituted with the DN of the user
# {1} is substituted with the username
# {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
rolesearch: '(member={0})'
# Specify the name of the attribute which value should be substituted with {2} above
userroleattribute: null
# Roles as an attribute of the user entry
userrolename: disabled
#userrolename: memberOf
# The attribute in a role entry containing the name of that role, Default is "name".
# Can also be "dn" to use the full DN as rolename.
rolename: cn
# Resolve nested roles transitive (roles which are members of other roles and so on ...)
resolve_nested_roles: true
userbase: 'ou=people,dc=example,dc=com'
# Filter to search for users (currently in the whole subtree beneath userbase)
# {0} is substituted with the username
usersearch: '(uid={0})'
# Skip users matching a user name, a wildcard or a regex pattern
#skip_users:
# - 'cn=Michael Jackson,ou*people,o=TEST'
# - '/\S*/'
roles_from_another_ldap:
description: "Authorize via another Active Directory"
http_enabled: false
transport_enabled: false
authorization_backend:
type: ldap # NOT FREE FOR COMMERCIAL USE
#config goes here ...
# auth_failure_listeners:
# ip_rate_limiting:
# type: ip
# allowed_tries: 10
# time_window_seconds: 3600
# block_expiry_seconds: 600
# max_blocked_clients: 100000
# max_tracked_clients: 100000
# internal_authentication_backend_limiting:
# type: username
# authentication_backend: intern
# allowed_tries: 10
# time_window_seconds: 3600
# block_expiry_seconds: 600
# max_blocked_clients: 100000
# max_tracked_clients: 100000
kibana.yml
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""
# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false
# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576
# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://elastic_domain_name"]
# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true
# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#kibana.index: ".kibana"
# The default application to load.
#kibana.defaultAppId: "home"
# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full
# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500
# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000
# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]
# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}
# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000
# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000
# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false
# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid
# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout
# Set the value of this setting to true to suppress all logging output.
#logging.silent: false
# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false
# Set the value of this setting to true to log all events, including system usage information
# and all requests.
logging.verbose: true
# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000
# Specifies locale to be used for all localizable strings, dates and number formats.
#i18n.locale: "en"
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
elasticsearch.ssl.verificationMode: none
xpack.security.enabled: false
searchguard.multitenancy.enabled: true
searchguard.openid.connect_url: "https://keycloak_domain_name/auth/realms/kibana-realm/.well-known/openid-configuration"
searchguard.openid.client_id: "kibana-sso"
searchguard.openid.client_secret: "***********************"
searchguard.openid.base_redirect_url: "https://kibana_domain_name/"
elasticsearch.requestHeadersWhitelist: ["Authorization", "sgtenant"]
#searchguard.basicauth.enabled: false
searchguard.auth.type: "openid"
#searchguard.jwt.enabled: true
#searchguard.jwt.url_param: jwtparam
#searchguard.jwt.header: jwtheader
xpack.monitoring.enabled: false
xpack.graph.enabled: false
xpack.ml.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
xpack.reporting.enabled: false
#sentinl configuration for email
sentinl:
es:
hosts: ["https://elastic_domain_name"]
settings:
email:
active: true
user: EMAIL
password: PASSWORD
host: smtp.gmail.com
port: 465
ssl: true
tls: false
#searchguard.openid.root_ca: "/home/smart/smart/certs/CACert.crt"
#searchguard.openid.verify_hostnames: false
Error: Failed when trying to obtain the endpoints from your IdP
It indicates that the metadata endpoint of the IDP is not reachable.
First, check if the following setting has the correct URL.
kibana.yml
searchguard.openid.connect_url: "https://keycloak_domain_name/auth/realms/kibana-realm/.well-known/openid-configuration"
Second, go to the Kibana server and check if the URL is accessible from there. For example, you can use curl to do this.
curl -k https://keycloak_domain_name/auth/realms/kibana-realm/.well-known/openid-configuration
Do you get the response?
The elasticsearch nodes must have access to the IDP endpoint too.
sg_config.yml
openid_connect_url: "https://domain_name/auth/realms/kibana-realm/.well-known/openid-configuration"
Hi srgbnd
I have tried all the solutions which you mention. Url and curl request giving proper response from kibana, keycloak, and elastic search server.
Can you post the response here?
searchguard.openid.connect_url: “https://keycloak_domain_name/auth/realms/kibana-realm/.well-known/openid-configuration” response form ElasticSearch Server
{
"issuer": "https://keycloak_domain/auth/realms/kibana-realm",
"authorization_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/auth",
"token_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/token",
"token_introspection_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/token/introspect",
"userinfo_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/userinfo",
"end_session_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/logout",
"jwks_uri": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/certs",
"check_session_iframe": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/login-status-iframe.html",
"grant_types_supported": ["authorization_code", "implicit", "refresh_token", "password", "client_credentials"],
"response_types_supported": ["code", "none", "id_token", "token", "id_token token", "code id_token", "code token", "code id_token token"],
"subject_types_supported": ["public", "pairwise"],
"id_token_signing_alg_values_supported": ["ES384", "RS384", "HS256", "HS512", "ES256", "RS256", "HS384", "ES512", "RS512"],
"userinfo_signing_alg_values_supported": ["ES384", "RS384", "HS256", "HS512", "ES256", "RS256", "HS384", "ES512", "RS512", "none"],
"request_object_signing_alg_values_supported": ["none", "RS256"],
"response_modes_supported": ["query", "fragment", "form_post"],
"registration_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/clients-registrations/openid-connect",
"token_endpoint_auth_methods_supported": ["private_key_jwt", "client_secret_basic", "client_secret_post", "client_secret_jwt"],
"token_endpoint_auth_signing_alg_values_supported": ["RS256"],
"claims_supported": ["sub", "iss", "auth_time", "name", "given_name", "family_name", "preferred_username", "email"],
"claim_types_supported": ["normal"],
"claims_parameter_supported": false,
"scopes_supported": ["openid", "address", "email", "offline_access", "phone", "profile", "roles", "web-origins"],
"request_parameter_supported": true,
"request_uri_parameter_supported": true,
"code_challenge_methods_supported": ["plain", "S256"],
"tls_client_certificate_bound_access_tokens": true,
"introspection_endpoint": "https://keycloak_domain/auth/realms/kibana-realm/protocol/openid-connect/token/introspect"
}
curl -k https://keycloak_domain_name/auth/realms/kibana-realm/.well-known/openid-configuration Response From Kibana Server !!
{
"issuer": "https://keycloak_domainm/auth/realms/kibana-realm",
"authorization_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/auth",
"token_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/token",
"token_introspection_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/token/introspect",
"userinfo_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/userinfo",
"end_session_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/logout",
"jwks_uri": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/certs",
"check_session_iframe": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/login-status-iframe.html",
"grant_types_supported": ["authorization_code", "implicit", "refresh_token", "password", "client_credentials"],
"response_types_supported": ["code", "none", "id_token", "token", "id_token token", "code id_token", "code token", "code id_token token"],
"subject_types_supported": ["public", "pairwise"],
"id_token_signing_alg_values_supported": ["ES384", "RS384", "HS256", "HS512", "ES256", "RS256", "HS384", "ES512", "RS512"],
"userinfo_signing_alg_values_supported": ["ES384", "RS384", "HS256", "HS512", "ES256", "RS256", "HS384", "ES512", "RS512", "none"],
"request_object_signing_alg_values_supported": ["none", "RS256"],
"response_modes_supported": ["query", "fragment", "form_post"],
"registration_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/clients-registrations/openid-connect",
"token_endpoint_auth_methods_supported": ["private_key_jwt", "client_secret_basic", "client_secret_post", "client_secret_jwt"],
"token_endpoint_auth_signing_alg_values_supported": ["RS256"],
"claims_supported": ["sub", "iss", "auth_time", "name", "given_name", "family_name", "preferred_username", "email"],
"claim_types_supported": ["normal"],
"claims_parameter_supported": false,
"scopes_supported": ["openid", "address", "email", "offline_access", "phone", "profile", "roles", "web-origins"],
"request_parameter_supported": true,
"request_uri_parameter_supported": true,
"code_challenge_methods_supported": ["plain", "S256"],
"tls_client_certificate_bound_access_tokens": true,
"introspection_endpoint": "https://keycloak_domainm/auth/realms/kibana-realm/protocol/openid-connect/token/introspect"
}
Ok. Let’s try to debug it.
First, Go to the Kibana server, execute the following two commands, and show me the results.
nslookup keycloak_domain_name
traceroute keycloak_domain_name
Second, while on the Kibana server go to kibana/plugins/searchguard/lib/auth/types/openid/OpenId.js. Find setupRoutes method, it should look like this lib/auth/types/openid/OpenId.js · 7.2.x · search-guard / Search Guard Kibana Plugin · GitLab Comment it like this.
/*
setupRoutes() {
...
}
*/
Put the following method in the OpenId.js file
async setupRoutes() {
try {
const {response, payload} = await Wreck.get(this.config.get('searchguard.openid.connect_url'));
console.log('openid', 'setupRoutes', 'response', response);
console.log('openid', 'setupRoutes', 'payload', payload);
const parsedPayload = JSON.parse(payload.toString());
let endPoints = {
authorization_endpoint: parsedPayload.authorization_endpoint,
token_endpoint: parsedPayload.token_endpoint,
end_session_endpoint: parsedPayload.end_session_endpoint || null
};
console.log('openid', 'setupRoutes', 'endPoints', endPoints);
require('./routes')(this.pluginRoot, this.server, this.kbnServer, this.APP_ROOT, this.API_ROOT, endPoints);
console.log('openid', 'setupRoutes', 'routes registered');
} catch (error) {
console.error('openid', error);
if (error ||
error.output.statusCode < 200 ||
error.output.statusCode > 299) {
throw new Error('Failed when trying to obtain the endpoints from your IdP');
}
}
}
Save, the Kibana should restart. Try OpenID authentication, get the Kibana log and paste it here.