We went through an upgrade process and this are our current versions:
Kibana - 6.8.6-19.0
Elasticsearch - 6.8.6-25.5
OpenID authentication backend - 6.8.6-34.5
I enabled level trace as you mentioned, but I don’t see anything on that level being logged.
The ID Token lifetime has been set to 1 minute to make ti easier to replicate.
At this timestamp: “2020-02-06T22:56:36Z” the webpage gets redirected to do authentication on the ODIP-URL but with an OPTIONS request.
“sg_config.yml” :
---
searchguard:
dynamic:
kibana:
do_not_fail_on_forbidden: true
license: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
authc:
basic_internal_auth_domain:
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: "basic"
challenge: false
authentication_backend:
type: "intern"
openid_auth_f5:
http_enabled: true
transport_enabled: true
order: 3
http_authenticator:
type: "openid"
challenge: false
config:
openid_connect_url: "https://xxxxxxxxxxxxxxxxxxxxxxxx/v1/.well-known/openid-configuration"
subject_key: "email"
roles_key: "kibana_roles"
jwt_header: "authorization"
authentication_backend:
type: "noop"
authz: null
logs:
==> /var/log/elasticsearch/es-01/elasticserver_sg.log <==
[2020-02-06T16:51:05,596][DEBUG][com.floragunn.dlic.auth.http.jwt.keybyoidc.SelfRefreshingKeySet] [elasticnode] performRefresh(oauth-jwt-rsa)
[2020-02-06T16:51:05,596][INFO ][com.floragunn.dlic.auth.http.jwt.keybyoidc.SelfRefreshingKeySet] [elasticnode] Performing refresh 1
[2020-02-06T16:51:06,096][INFO ][com.floragunn.dlic.auth.http.jwt.keybyoidc.SelfRefreshingKeySet] [elasticnode] KeySetProvider finished
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:49:09,698][INFO ][c.f.s.c.ComplianceConfig ] [elasticnode] Compliance features are enabled
[2020-02-06T16:49:09,699][INFO ][c.f.s.c.IndexBaseConfigurationRepository] [elasticnode] Search Guard License Info: SearchGuardLicense [uid=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX, type=FULL, features=[COMPLIANCE], issueDate=2019-06-16, expiryDate=2020-06-16, issuedTo=XXXXXXXXXXXXXXXXXXXX, issuer=floragunn GmbH, startDate=2019-06-17, majorVersion=6, clusterName=*, allowedNodeCount=32768, msgs=[], expiresInDays=131, isExpired=false, valid=true, action=, prodUsage=Yes, one cluster with all commercial features and unlimited nodes per cluster., clusterService=org.elasticsearch.cluster.service.ClusterService@72cd01f2, getMsgs()=[], getExpiresInDays()=131, isExpired()=false, isValid()=true, getAction()=, getProdUsage()=Yes, one cluster with all commercial features and unlimited nodes per cluster.]
[2020-02-06T16:49:09,699][INFO ][c.f.s.c.IndexBaseConfigurationRepository] [elasticnode] Search Guard License Type: FULL, valid
[2020-02-06T16:49:09,699][INFO ][c.f.s.c.IndexBaseConfigurationRepository] [elasticnode] Node 'elasticnode' initialized
[2020-02-06T16:51:05,510][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:51:06,188][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:51:06,199][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:51:09,549][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:51:09,791][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:51:09,895][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:54:38Z","tags":[],"pid":1040,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT Proxy-IP0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","connection":"keep-alive","upgrade-insecure-requests":"1","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP"},"res":{"statusCode":302,"responseTime":1,"contentLength":9},"message":"GET / 302 1ms - 9.0B"}
{"type":"response","@timestamp":"2020-02-06T22:54:38Z","tags":[],"pid":1040,"method":"get","statusCode":302,"req":{"url":"/auth/openid/login?nextUrl=%2F","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT Proxy-IP0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","connection":"keep-alive","upgrade-insecure-requests":"1","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP"},"res":{"statusCode":302,"responseTime":1,"contentLength":9},"message":"GET /auth/openid/login?nextUrl=%2F 302 1ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:54:59,979][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:54:59Z","tags":[],"pid":1040,"method":"get","statusCode":302,"req":{"url":"/auth/openid/login?code=94d83b7fe3e0afbfaf4305a1abe3c11d4b2100e438ea243f118b83c3b1b225b4&state=98zD3HRUi8AgPnYtFIJcjY","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT Proxy-IP0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://OIDP-URL/my.policy","connection":"keep-alive","upgrade-insecure-requests":"1","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://OIDP-URL/my.policy"},"res":{"statusCode":302,"responseTime":28,"contentLength":9},"message":"GET /auth/openid/login?code=94d83b7fe3e0afbfaf4305a1abe3c11d4b2100e438ea243f118b83c3b1b225b4&state=98zD3HRUi8AgPnYtFIJcjY 302 28ms - 9.0B"}
{"type":"response","@timestamp":"2020-02-06T22:55:00Z","tags":[],"pid":1040,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT Proxy-IP0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://OIDP-URL/my.policy","connection":"keep-alive","upgrade-insecure-requests":"1","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://OIDP-URL/my.policy"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET / 302 2ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:00,052][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2020-02-06T16:55:00,060][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:00Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT Proxy-IP0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://OIDP-URL/my.policy","connection":"keep-alive","upgrade-insecure-requests":"1","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://OIDP-URL/my.policy"},"res":{"statusCode":200,"responseTime":32,"contentLength":9},"message":"GET /app/kibana 200 32ms - 9.0B"}
{"type":"response","@timestamp":"2020-02-06T22:55:04Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/javascript, */*; q=0.01","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","kbn-version":"6.8.6","x-requested-with":"XMLHttpRequest","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":15,"contentLength":9},"message":"GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0 200 15ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:04,809][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:04Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/v1/auth/authinfo","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","kbn-version":"6.8.6","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":14,"contentLength":9},"message":"GET /api/v1/auth/authinfo 200 14ms - 9.0B"}
{"type":"response","@timestamp":"2020-02-06T22:55:04Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/xpack/v1/info","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","kbn-version":"6.8.6","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /api/xpack/v1/info 200 3ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:04,955][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:04Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/_find?type=index-pattern&fields=title&search=*&search_fields=title&per_page=1&page=1","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://Kibana-URL/app/kibana","content-type":"application/json","kbn-version":"6.8.6","connection":"keep-alive","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":15,"contentLength":9},"message":"GET /api/saved_objects/_find?type=index-pattern&fields=title&search=*&search_fields=title&per_page=1&page=1 200 15ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:05,145][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:05Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/v1/restapiinfo","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","kbn-version":"6.8.6","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":14,"contentLength":9},"message":"GET /api/v1/restapiinfo 200 14ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:06,587][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:06Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/_find?type=index-pattern&per_page=10000&page=1","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://Kibana-URL/app/kibana","content-type":"application/json","kbn-version":"6.8.6","connection":"keep-alive","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":26,"contentLength":9},"message":"GET /api/saved_objects/_find?type=index-pattern&per_page=10000&page=1 200 26ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:06,682][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:06Z","tags":[],"pid":1040,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/_find?type=index-pattern&fields=title&per_page=10000&page=1","method":"get","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://Kibana-URL/app/kibana","content-type":"application/json","kbn-version":"6.8.6","connection":"keep-alive","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":18,"contentLength":9},"message":"GET /api/saved_objects/_find?type=index-pattern&fields=title&per_page=10000&page=1 200 18ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:06,831][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:06Z","tags":[],"pid":1040,"method":"post","statusCode":200,"req":{"url":"/api/saved_objects/_bulk_get","method":"post","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://Kibana-URL/app/kibana","content-type":"application/json","kbn-version":"6.8.6","origin":"https://Kibana-URL","content-length":"70","connection":"keep-alive","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":18,"contentLength":9},"message":"POST /api/saved_objects/_bulk_get 200 18ms - 9.0B"}
==> /var/log/elasticsearch/es-01/elasticserver.log <==
[2020-02-06T16:55:07,738][WARN ][c.f.s.h.HTTPBasicAuthenticator] [elasticnode] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
==> /var/log/kibana/kibana.log <==
{"type":"response","@timestamp":"2020-02-06T22:55:07Z","tags":[],"pid":1040,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/_msearch?rest_total_hits_as_int=true&ignore_throttled=true","method":"post","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","content-type":"application/x-ndjson","kbn-version":"6.8.6","content-length":"756","origin":"https://Kibana-URL","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":200,"responseTime":168,"contentLength":9},"message":"POST /elasticsearch/_msearch?rest_total_hits_as_int=true&ignore_throttled=true 200 168ms - 9.0B"}
{"type":"response","@timestamp":"2020-02-06T22:56:36Z","tags":[],"pid":1040,"method":"post","statusCode":302,"req":{"url":"/elasticsearch/_msearch?rest_total_hits_as_int=true&ignore_throttled=true","method":"post","headers":{"host":"Kibana-URL","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","content-type":"application/x-ndjson","kbn-version":"6.8.6","content-length":"756","origin":"https://Kibana-URL","connection":"keep-alive","referer":"https://Kibana-URL/app/kibana","x-forwarded-for":"Client-IP"},"remoteAddress":"Proxy-IP","userAgent":"Proxy-IP","referer":"https://Kibana-URL/app/kibana"},"res":{"statusCode":302,"responseTime":3,"contentLength":9},"message":"POST /elasticsearch/_msearch?rest_total_hits_as_int=true&ignore_throttled=true 302 3ms - 9.0B"}
thanks
