Search Guard 6.2.3 + ELK 6.2.3 impersonation

marcus_young · April 9, 2018, 9:29pm

Im attempting to upgrade from ELK 5.6.2 to ELK 6.2.3
Elasticsearch seems to have gone fine, all API actions are working as expected (search guard included), however Kibana keeps giving me a 403 for https://kibana.redact.com:4180/api/saved_objects/?type=index-pattern&per_page=10000 with error:
‘kibanaserver’
is not allowed to impersonate as ‘kibanaserver’: [security_exception] ‘kibanaserver’ is not allowed to impersonate as ‘kibanaserver’

This all worked fine under ELK 5.6.2, so Im not sure If i missed anything. (the only change to kibana.yml is searchguard.basicauth.enabled: false

ES plugin com.floragunn:search-guard-6:6.2.3-22.0
ES + kibana 6.2.3
Kibana plugin 6.2.3-11
JVM OpenJDK Runtime Environment (build 1.8.0_161-b14)
All oss

nginx.conf:
$ cat nginx.conf
events {
worker_connections 1024; ## Default: 1024
}

http {
server {
listen 5601;
server_name localhost;

    location / {
        proxy_pass https://localhost:5602/;
        proxy_set_header Authorization "Basic a2liYW5hc2VydmVyOmZvbwo=";
        proxy_set_header sg_impersonate_as kibanaserver;
        rewrite /login https://kibana.redact.com:4180/oauth2/sign_in redirect;
    }
}

}

kibana.yml:
$ cat /usr/share/kibana/config/kibana.yml
server.host: “0.0.0.0”
server.port: 5602
elasticsearch.url: “https://es.redact.com:9200”
elasticsearch.username: “kibanaserver”
elasticsearch.password: “foo”
elasticsearch.ssl.verificationMode: full
logging.dest: stdout
elasticsearch.requestHeadersWhitelist: [ sg_impersonate_as, authorization ]
searchguard.basicauth.enabled: false
server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/bundle.pem
server.ssl.key: /usr/share/kibana/key.pem

sg_internal_user.yml:
kibanaserver:
hash: $2a$hash

/etc/elasticsearch/elasticsearch.yml:
action.destructive_requires_name: true
cluster.name: redact
discovery:
zen.hosts_provider: ec2
http.max_content_length: 500mb
network.host: 0.0.0.0
node.data: true
node.master: true
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
searchguard.ssl.http.clientauth_mode: NONE
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.truststore_filepath: truststore.jks
xpack.graph.enabled: false
xpack.monitoring.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
discovery.ec2.tag.Environment: Operations
node.name: ip-10-0-3-49
searchguard.ssl.transport.truststore_password: “es”
searchguard.ssl.transport.keystore_password: “es”
searchguard.ssl.http.keystore_password: “es”
searchguard.ssl.http.truststore_password: “es”
searchguard.enable_snapshot_restore_privilege: true
searchguard.nodes_dn:

‘/CN=ip.*/’
searchguard.authcz.admin_dn:
CN=operations
searchguard.authcz.impersonation_dn:
“CN=kibanaserver”:
- “kibanaserver”

marcus_young · April 9, 2018, 9:38pm

It mostly loads: https://imgur.com/a/RUjww

Also GET api/v1/auth/authinfo (right after the 403 from first post) shows a 500: {“statusCode”:500,“error”:“Internal Server Error”,“message”:“An internal server error occurred”}

···

On Monday, April 9, 2018 at 4:29:32 PM UTC-5, marcus young wrote:

$ cat nginx.conf
events {
worker_connections 1024; ## Default: 1024
}

http {
server {
listen 5601;
server_name localhost;

    location / {
        proxy_pass [https://localhost:5602/](https://localhost:5602/);
        proxy_set_header Authorization "Basic a2liYW5hc2VydmVyOmZvbwo=";
        proxy_set_header sg_impersonate_as kibanaserver;
        rewrite /login [https://kibana.redact.com:4180/oauth2/sign_in](https://kibana.redact.com:4180/oauth2/sign_in) redirect;
    }
}

}

$ cat /usr/share/kibana/config/kibana.yml
server.host: “0.0.0.0”
server.port: 5602
elasticsearch.url: “https://es.redact.com:9200”
elasticsearch.username: “kibanaserver”
elasticsearch.password: “foo”
elasticsearch.ssl.verificationMode: full
logging.dest: stdout
elasticsearch.requestHeadersWhitelist: [ sg_impersonate_as, authorization ]
searchguard.basicauth.enabled: false
server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/bundle.pem
server.ssl.key: /usr/share/kibana/key.pem

kibanaserver:
hash: $2a$hash

action.destructive_requires_name: true
cluster.name: redact
discovery:
zen.hosts_provider: ec2
http.max_content_length: 500mb
network.host: 0.0.0.0
node.data: true
node.master: true
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
searchguard.ssl.http.clientauth_mode: NONE
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.truststore_filepath: truststore.jks
xpack.graph.enabled: false
xpack.monitoring.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
discovery.ec2.tag.Environment: Operations
node.name: ip-10-0-3-49
searchguard.ssl.transport.truststore_password: “es”
searchguard.ssl.transport.keystore_password: “es”
searchguard.ssl.http.keystore_password: “es”
searchguard.ssl.http.truststore_password: “es”
searchguard.enable_snapshot_restore_privilege: true
searchguard.nodes_dn:

‘/CN=ip.*/’
searchguard.authcz.admin_dn:

CN=operations
searchguard.authcz.impersonation_dn:
“CN=kibanaserver”:

“kibanaserver”

Im attempting to upgrade from ELK 5.6.2 to ELK 6.2.3
Elasticsearch seems to have gone fine, all API actions are working as expected (search guard included), however Kibana keeps giving me a 403 for https://kibana.redact.com:4180/api/saved_objects/?type=index-pattern&per_page=10000 with error:
‘kibanaserver’
is not allowed to impersonate as ‘kibanaserver’: [security_exception] ‘kibanaserver’ is not allowed to impersonate as ‘kibanaserver’

This all worked fine under ELK 5.6.2, so Im not sure If i missed anything. (the only change to kibana.yml is searchguard.basicauth.enabled: false

ES plugin com.floragunn:search-guard-6:6.2.3-22.0
ES + kibana 6.2.3
Kibana plugin 6.2.3-11
JVM OpenJDK Runtime Environment (build 1.8.0_161-b14)
All oss

nginx.conf:
kibana.yml:
sg_internal_user.yml:
/etc/elasticsearch/elasticsearch.yml:

marcus_young · April 9, 2018, 9:42pm

The 500:

{
“type”: “response”,
“@timestamp”: “2018-04-09T21:39:55Z”,
“tags”: ,
“pid”: 5,
“method”: “get”,
“statusCode”: 500,
“req”: {
“url”: “/api/v1/auth/authinfo”,
“method”: “get”,
“headers”: {
“sg_impersonate_as”: “kibanaserver”,
“host”: “localhost:5602”,
“connection”: “close”,
“user-agent”: “Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0”,
“accept”: “application/json, text/plain, /”,
“accept-encoding”: “gzip, deflate, br”,
“accept-language”: “en-US,en;q=0.5”,
“kbn-version”: “6.2.3”,
“referer”: “https://kibana.redact.com:4180/app/kibana”,
“x-amzn-trace-id”: “Root=1–”,
“x-forwarded-email”: “marc@redact.com”,
“x-forwarded-for”: “192.168.1.220, 10.0.3.113”,
“x-forwarded-port”: “4180”,
“x-forwarded-proto”: “https”,
“x-forwarded-user”: “marc”
},
“remoteAddress”: “127.0.0.1”,
“userAgent”: “127.0.0.1”,
“referer”: “https://kibana.redact.com:4180/app/kibana”
},
“res”: {
“statusCode”: 500,
“responseTime”: 22,
“contentLength”: 9
},
“message”: “GET /api/v1/auth/authinfo 500 22ms - 9.0B”
}

marcus_young · April 10, 2018, 1:01am

Bumped debug logs for searchguard, ran:
$ curl -k -s -u ‘kibanaserver:foo’ -H “sg_impersonate_as: kibanaserver” https://10.0.3.31:9200/_cat/indices | jq
{
“error”: {
“root_cause”: [
{
“type”: “security_exception”,
“reason”: “‘kibanaserver’ is not allowed to impersonate as ‘kibanaserver’”
}
],
“type”: “security_exception”,
“reason”: “‘kibanaserver’ is not allowed to impersonate as ‘kibanaserver’”
},
“status”: 403
}

only lines I see in elasticsearch logs:
[2018-04-10T00:59:40,482][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from basic http authenticator
[2018-04-10T00:59:40,497][DEBUG][c.f.s.a.BackendRegistry ] kibanaserver not cached, return from internal backend directly
[2018-04-10T00:59:40,879][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=kibanaserver, roles=, requestedTenant=null]’ is authenticated
[2018-04-10T00:59:40,880][DEBUG][c.f.s.a.BackendRegistry ] sgtenant ‘null’

marcus_young · April 10, 2018, 1:33am

Found the issue. Configuration…

was: #searchguard.authcz.impersonation_dn:

now: #searchguard.authcz.rest_impersonation_user:

Topic		Replies	Views
LDAP auth for ES no bypass or execute filters at all for action indices:monitor/status Search Guard	3	386	March 30, 2016
using certs when accessing kibana's api/saved_objects api not working? Search Guard	3	678	January 14, 2019
Unable to achieve SAML Single Sign-on Search Guard	1	754	December 11, 2018
6.2.3 Demo anonymous auth not functioning Search Guard	4	446	May 7, 2018
Configure searchguard to work with SSO Search Guard	1	514	August 22, 2017

Search Guard 6.2.3 + ELK 6.2.3 impersonation

Related Topics