Please help with Kibana JWT access

Kibana and ES version => 6.2.4

Hi,

I have read and re-read the docs and have been googling this stuff forever now. I have a working JWT protected cluster and all I want to do is use Kibana on top of it. I installed Kibana and also the searchguard plugin for Kibana. Kibana is able to connect to ES since I have kibanaserver credentials defined in kibana.yml

I have been treating this discussion https://groups.google.com/forum/#!searchin/search-guard/JWT$20kibana|sort:date/search-guard/JaGlsMOxeDo/nqUZVZX3BQAJ as my holy grail to get this done and I have read this multiple times and applied things like moving JWT config on top of basic auth and so on.

I tried using the JWT url param and it fails to work too.

http://localhost:5601/?some_jwt=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiYWRtaW4iLCJpYXQiOjE1MTYyMzkwMjIsImV4cCI6MTY1NjcyMzg3Mywic3ViIjoiYWRtaW4iLCJyb2xlcyI6ImFkbWluIn0.E0p_41R2qzm7UNioF8vW3Lt_OwGoCwo187c6Lt-LNSG8cnKNZTrEEkUCTV2iO-1BpWrekUY7v5WiTm0hV55YZqg62cHl5T_KVrQvTjO1ozLAfMiZnVeTCEI7EgR1xc_nKqmknhU6ogtVOrnGWBixU2lyPEGePa31niU98lyX4cWKtr2Ti4xNWrbeSwSXjqspEwG7-tmArUorgS5rJ20KtxAOj8FAXZhVPL1aC-VUTo9caXSgEl6-ktFKoDfm-lX7Snnsx5sPdX90zIugmllVPq_mUD75m5HiOhMlG4OkFVTc8aQ2WhrJqTOV4MFPYCSyz50RcTelqwG-MMKDrXzguUApyM4yVOynKhghdaEGDJBh6FIi8AOS8mHWxXwzd6VnZeLNmAFDw5Jm-IdI2T_tVeDc24BFJCG6zODiY3DAmOwLKpKSYeEU29sgeLA7M5-Q1BU4jF4LhelaE0E5kwtOO8MRWFAIsyoFCCuFqoP6W50achMQQi95h-KWtlHNKmrJs6Z2tCk7GwkyTgFPc76wAAioIhJnfXGNE_RQq0RRmtR6Kuzd-DPr4exrX6tmuUb2B3fGYkEeFXOWmXa1bg-DDLTWpVlkH3_Zsw3XS7wNwIMNzZQsDpXv5fzFtD3sajANPUjs0zsqE4asLBjMbSM4ohOrth6IV2JkBhrpuNYIHdU

``

My main goal here is to make “Authorization” headers work. I used the Chrome extension “requestly” to insert an “Authorization” header with the Bearer JWT token and Kibana keeps taking me to search guard login page. Is there a different way to test it out when you dont have a authentication server ready so that you can insert Auth headers from the browser (hence, requestly)? The ES logs always show this which looks like its using basic all the time.

[2018-05-16T21:54:50,783][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,290][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,294][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,297][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,300][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]

``

Here’s my sg_config.yml

searchguard:
dynamic:
kibana:
multitenancy_enabled: true
server_username: kibanaserver
index: ‘.kibana’
do_not_fail_on_forbidden: true
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: ‘192.168.0.10|192.168.0.11’ # regex pattern
#internalProxies: ‘.*’ # trust all internal proxies, regex pattern
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
authc:
kerberos_auth_domain:
http_enabled: false
transport_enabled: false
order: 6
http_authenticator:
type: kerberos # NOT FREE FOR COMMERCIAL USE
challenge: true
config:
# If true a lot of kerberos/security related debugging output will be logged to standard out
krb_debug: false
# If true then the realm will be stripped from the user name
strip_realm_from_principal: true
authentication_backend:
type: noop
jwt_auth_domain:
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: jwt
challenge: false
config:
signing_key: |-
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAyzAnJJCV/ihJGhutqA11
47elFKtDQ4pNpQjhIwNPc1YkHYqanThHbyejv5Nsfn4yC9wpuEnpnsmD1yrfsFuV
nl8MymOKmL7FoLkT9vALDpPhnMtHd9b4ffI6kZTrVpQBmd5yNud6buVPTjH+N3o7
Y9yZhAMChc9X+6SyllnE55gBXR57r1e0b+s0BKqswE/iUuvSNHUN+JyuJnF+TrUI
E6pqQJpkyXQOgjGgKuE+/HZzY7XT54PnUbZRH+CIReS2nJ9TholG4UtRnxCl2hPg
e21KcjFG0bYPFJfpS/+Js+XS1p2g7ttPTLlPMdg5GZYa9cUbJvIu1CatudWvb7Ku
c/FeNI5QKIe2nOKiTh+4CL9ENWsN9e6IVXNPFwGGOOgklnL8tCy00lZTMoTkpD20
eK830ZXRMHQ6E4D/fgbttL4CJYY3BNwNRcuQEZGOuUS85kiDGVyqdrqK2oVUlfQ6
ezU6qWRyrPuZJK2Kpmaq572VFJtZ0yZYkQ/7mtLzcHhavKvnqeYV5nhqkmRHG+/q
oqMLdrzhL04HGaWmPRe2kSrZnCAXXD+qED7noDcbGdv521ioxrnBbv6LweGSoHoH
ogEU+ReQrst2nJijfxgGS5tHLct5dRkztxcLXdt5WM0xydobqhp/sf1mLaXqRhwq
hWevTP4OD4NFAqDi2IDFHZ0CAwEAAQ==
-----END PUBLIC KEY-----
jwt_header: “Authorization”
jwt_url_parameter: “some_jwt”
cookieName: “Cookie”
cookieHeaderName: “jwt_token”
roles_key: roles
subject_key: sub
authentication_backend:
type: noop
basic_internal_auth_domain:
http_enabled: true
transport_enabled: true
order: 4
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
proxy_auth_domain:
http_enabled: false
transport_enabled: false
order: 3
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
clientcert_auth_domain:
http_enabled: false
transport_enabled: false
order: 2
http_authenticator:
type: clientcert
config:
username_attribute: cn #optional, if omitted DN becomes username
challenge: false
authentication_backend:
type: noop

``

Kibana.yml

console.enabled: false
elasticsearch.requestTimeout: 600000
elasticsearch.shardTimeout: 595000
elasticsearch.ssl.verificationMode: none
elasticsearch.url: “https://localhost:9200
elasticsearch.username: “kibanaserver”
elasticsearch.password: “kibanaserver”
logging.verbose: false
server.host: “0.0.0.0”
searchguard.jwt.enabled: true
elasticsearch.requestHeadersWhitelist: [ “Authorization”, “sgtenant”, “some_jwt” ]

``

Elasticsearch.yml

node.name: “elasticsearch”
node.master: true
node.data: true
gateway.recover_after_nodes: 1
gateway.recover_after_time: 10m
gateway.expected_nodes: 1
action.auto_create_index: true

######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: [“sg_all_access”]
cluster.routing.allocation.disk.threshold_enabled: false
cluster.name: searchguard_demo
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3
######## End Search Guard Demo Configuration ########

``

Please do not cross post, this seems to be the same question as asked here:

https://github.com/floragunncom/search-guard/issues/495

···

On Wednesday, May 16, 2018 at 11:57:06 PM UTC+2, .mni wrote:

Kibana and ES version => 6.2.4

Hi,

I have read and re-read the docs and have been googling this stuff forever now. I have a working JWT protected cluster and all I want to do is use Kibana on top of it. I installed Kibana and also the searchguard plugin for Kibana. Kibana is able to connect to ES since I have kibanaserver credentials defined in kibana.yml

I have been treating this discussion https://groups.google.com/forum/#!searchin/search-guard/JWT$20kibana%7Csort:date/search-guard/JaGlsMOxeDo/nqUZVZX3BQAJ as my holy grail to get this done and I have read this multiple times and applied things like moving JWT config on top of basic auth and so on.

I tried using the JWT url param and it fails to work too.

http://localhost:5601/?some_jwt=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiYWRtaW4iLCJpYXQiOjE1MTYyMzkwMjIsImV4cCI6MTY1NjcyMzg3Mywic3ViIjoiYWRtaW4iLCJyb2xlcyI6ImFkbWluIn0.E0p_41R2qzm7UNioF8vW3Lt_OwGoCwo187c6Lt-LNSG8cnKNZTrEEkUCTV2iO-1BpWrekUY7v5WiTm0hV55YZqg62cHl5T_KVrQvTjO1ozLAfMiZnVeTCEI7EgR1xc_nKqmknhU6ogtVOrnGWBixU2lyPEGePa31niU98lyX4cWKtr2Ti4xNWrbeSwSXjqspEwG7-tmArUorgS5rJ20KtxAOj8FAXZhVPL1aC-VUTo9caXSgEl6-ktFKoDfm-lX7Snnsx5sPdX90zIugmllVPq_mUD75m5HiOhMlG4OkFVTc8aQ2WhrJqTOV4MFPYCSyz50RcTelqwG-MMKDrXzguUApyM4yVOynKhghdaEGDJBh6FIi8AOS8mHWxXwzd6VnZeLNmAFDw5Jm-IdI2T_tVeDc24BFJCG6zODiY3DAmOwLKpKSYeEU29sgeLA7M5-Q1BU4jF4LhelaE0E5kwtOO8MRWFAIsyoFCCuFqoP6W50achMQQi95h-KWtlHNKmrJs6Z2tCk7GwkyTgFPc76wAAioIhJnfXGNE_RQq0RRmtR6Kuzd-DPr4exrX6tmuUb2B3fGYkEeFXOWmXa1bg-DDLTWpVlkH3_Zsw3XS7wNwIMNzZQsDpXv5fzFtD3sajANPUjs0zsqE4asLBjMbSM4ohOrth6IV2JkBhrpuNYIHdU

``

My main goal here is to make “Authorization” headers work. I used the Chrome extension “requestly” to insert an “Authorization” header with the Bearer JWT token and Kibana keeps taking me to search guard login page. Is there a different way to test it out when you dont have a authentication server ready so that you can insert Auth headers from the browser (hence, requestly)? The ES logs always show this which looks like its using basic all the time.

[2018-05-16T21:54:50,783][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,290][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,294][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,297][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]
[2018-05-16T21:54:53,300][WARN ][c.f.d.a.h.j.HTTPJwtAuthenticator] [Content-Length=[0], Host=[localhost:9200], Authorization=[Basic a2liYW5hc2VydmVyOmtpYmFuYXNlcnZlcg==], Connection=[keep-alive]]

``

Here’s my sg_config.yml

searchguard:
dynamic:
kibana:
multitenancy_enabled: true
server_username: kibanaserver
index: ‘.kibana’
do_not_fail_on_forbidden: true
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: ‘192.168.0.10|192.168.0.11’ # regex pattern
#internalProxies: ‘.*’ # trust all internal proxies, regex pattern
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
authc:
kerberos_auth_domain:
http_enabled: false
transport_enabled: false
order: 6
http_authenticator:
type: kerberos # NOT FREE FOR COMMERCIAL USE
challenge: true
config:
# If true a lot of kerberos/security related debugging output will be logged to standard out
krb_debug: false
# If true then the realm will be stripped from the user name
strip_realm_from_principal: true
authentication_backend:
type: noop
jwt_auth_domain:
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: jwt
challenge: false
config:
signing_key: |-
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAyzAnJJCV/ihJGhutqA11
47elFKtDQ4pNpQjhIwNPc1YkHYqanThHbyejv5Nsfn4yC9wpuEnpnsmD1yrfsFuV
nl8MymOKmL7FoLkT9vALDpPhnMtHd9b4ffI6kZTrVpQBmd5yNud6buVPTjH+N3o7
Y9yZhAMChc9X+6SyllnE55gBXR57r1e0b+s0BKqswE/iUuvSNHUN+JyuJnF+TrUI
E6pqQJpkyXQOgjGgKuE+/HZzY7XT54PnUbZRH+CIReS2nJ9TholG4UtRnxCl2hPg
e21KcjFG0bYPFJfpS/+Js+XS1p2g7ttPTLlPMdg5GZYa9cUbJvIu1CatudWvb7Ku
c/FeNI5QKIe2nOKiTh+4CL9ENWsN9e6IVXNPFwGGOOgklnL8tCy00lZTMoTkpD20
eK830ZXRMHQ6E4D/fgbttL4CJYY3BNwNRcuQEZGOuUS85kiDGVyqdrqK2oVUlfQ6
ezU6qWRyrPuZJK2Kpmaq572VFJtZ0yZYkQ/7mtLzcHhavKvnqeYV5nhqkmRHG+/q
oqMLdrzhL04HGaWmPRe2kSrZnCAXXD+qED7noDcbGdv521ioxrnBbv6LweGSoHoH
ogEU+ReQrst2nJijfxgGS5tHLct5dRkztxcLXdt5WM0xydobqhp/sf1mLaXqRhwq
hWevTP4OD4NFAqDi2IDFHZ0CAwEAAQ==
-----END PUBLIC KEY-----
jwt_header: “Authorization”
jwt_url_parameter: “some_jwt”
cookieName: “Cookie”
cookieHeaderName: “jwt_token”
roles_key: roles
subject_key: sub
authentication_backend:
type: noop
basic_internal_auth_domain:
http_enabled: true
transport_enabled: true
order: 4
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
proxy_auth_domain:
http_enabled: false
transport_enabled: false
order: 3
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
clientcert_auth_domain:
http_enabled: false
transport_enabled: false
order: 2
http_authenticator:
type: clientcert
config:
username_attribute: cn #optional, if omitted DN becomes username
challenge: false
authentication_backend:
type: noop

``

Kibana.yml

console.enabled: false
elasticsearch.requestTimeout: 600000
elasticsearch.shardTimeout: 595000
elasticsearch.ssl.verificationMode: none
elasticsearch.url: “https://localhost:9200
elasticsearch.username: “kibanaserver”
elasticsearch.password: “kibanaserver”
logging.verbose: false
server.host: “0.0.0.0”
searchguard.jwt.enabled: true
elasticsearch.requestHeadersWhitelist: [ “Authorization”, “sgtenant”, “some_jwt” ]

``

Elasticsearch.yml

node.name: “elasticsearch”
node.master: true
node.data: true
gateway.recover_after_nodes: 1
gateway.recover_after_time: 10m
gateway.expected_nodes: 1
action.auto_create_index: true

######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: [“sg_all_access”]
cluster.routing.allocation.disk.threshold_enabled: false
cluster.name: searchguard_demo
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3
######## End Search Guard Demo Configuration ########

``

So i commented out the elasticsearch.username and elasticsearch.password fields from kibana.yml and also added the searchguard.basicauth.enabled: false and now when I re-start kibana I get

[15:11:36.893] [error][status][plugin:elasticsearch@6.2.4] Status changed from yellow to red - Authentication Exception

kibana.yml

console.enabled: false
elasticsearch.requestTimeout: 600000
elasticsearch.shardTimeout: 595000
elasticsearch.ssl.verificationMode: none
elasticsearch.url: "https://localhost:9200"
#elasticsearch.username: "kibanaserver"
#elasticsearch.password: "kibanaserver"
logging.verbose: false
server.host: "0.0.0.0"
searchguard.basicauth.enabled: false
searchguard.jwt.enabled: true
elasticsearch.requestHeadersWhitelist: [ "Authorization", "sgtenant", "some_jwt" ]

Quick Update: Even with the above Authentication Exception I can login into the Kibana UI if I pass in a Authorization header. Now, the next step for me is to show who is logged into the Kibana UI by showing a username extracted from the JWT token or something like that? We have an issue where our user fire long running queries that wrecks havoc on our ES cluster and we want to know who does that and when.

image


Any pointers would really be appreciated since its been a few days I have been struggling with this. We are looking forward to get this setup done so that we can subscribe to the enterprise license. Without a POC, it would be impossible for us to go that route.

OK, I think I figured this out. I had to uncomment the kibana basic creds from kibana.yml so that kibana can talk to ES internally.

Yes, this is true. There are two users involved here:

First, the so-called kibanaserver user. This is a user that Kibana uses internally when talking to Elasticsearch, for example for health checks. The kibanaserver user uses HTTP Basic authentication when talking to Elasticsearch. That’s why you need the HTTP Basic Auth domain in sg_config.yml enabled, even if you plan to use JWT.

The username/password for this internal user is specified by:

elasticsearch.username: “kibanaserver”
elasticsearch.password: “kibanaserver”

``

``

The second one is your actual user that wants to use Kibana, create dashboards, visualizations etc. In your case, you want to use JWT as the authentication method. This is a Single-Sign-On authentication method where the credentials are already fully contained in the token. Thus, you do not need any Kibana login dialogue. To disable the unwanted login dialogue, set:

searchguard.basicauth.enabled: false

``

If you now issue a request to Kibana that contains a JWT token in the HTTP header, it is passed from Kibana to Elasticsearch/Search Guard, and Search Guard can use it for authentication.

One other thing: Kibana requires you to explicitly whitelist any header that you want to pass from Kibana to Elasticsearch. So if the HTTP header is called “some_jwt”, this configuration entry here is correct:

elasticsearch.requestHeadersWhitelist: [ “Authorization”, “sgtenant”, “some_jwt” ]

``

However, I would suggest that you stick with the standard “Authorization” header, it’s less error prone. On the Search Guard side, just configure the name of the HTTP header you use. The default HTTP header name is “Authorization”, and the default for the subject_key is “sub”. So if you use these defaults, it is actually sufficient to just configure:

jwt_auth_domain:

``
http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: jwt

challenge: false

config:

signing_key: |-

-----BEGIN PUBLIC KEY-----


-----END PUBLIC KEY-----
roles_key: roles

···

On Tuesday, May 22, 2018 at 5:30:54 PM UTC+2, .mni wrote:

OK, I think I figured this out. I had to uncomment the kibana basic creds from kibana.yml so that kibana can talk to ES internally.