Problems with proxy users/roles

Kalada · February 2, 2018, 3:18pm

Please see the attached

···

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:

pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: myuser

x-proxy-roles: myrole

x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized

< WWW-Authenticate: Basic realm=“Search Guard”

< content-type: text/plain; charset=UTF-8

< content-length: 12

<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:

Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.

searchguard_google_group · February 2, 2018, 6:16pm

should be something like this (the proxy authenticator was disabled)

searchguard:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        internalProxies: '.*'
        remoteIpHeader: 'x-forwarded-for'
        proxiesHeader: 'x-forwarded-by'
        trustedProxies: '.*'
    authc:
      proxy_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 0
        http_authenticator:
          type: proxy
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: noop
      basic_internal_auth:
        enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: internal

···

Am 02.02.2018 um 16:18 schrieb Kalada <kalada.opuiyo@t1cg.com>:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

> Am 01.02.2018 um 23:04 schrieb Kalada <kalada...@t1cg.com>:
>
>
> Thank you https wasnt enabled please see below
>
> * Rebuilt URL to: http://elasticsearch:9200/
> * Trying 172.22.0.3...
> * TCP_NODELAY set
> * Connected to elasticsearch (172.22.0.3) port 9200 (#0)
> > GET / HTTP/1.1
> > Host: elasticsearch:9200
> > User-Agent: curl/7.57.0
> > Accept: */*
> > x-proxy-user: myuser
> > x-proxy-roles: myrole
> > x-forwarded-for: 127.0.0.1
> >
> < HTTP/1.1 401 Unauthorized
> < WWW-Authenticate: Basic realm="Search Guard"
> < content-type: text/plain; charset=UTF-8
> < content-length: 12
> <
> * Connection #0 to host elasticsearch left intact
>
>
> On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
> Have you https enabled? If not its
> curl -sS http://elasticsearch:9200 -v -H "x-proxy-user: myuser" -H "x-proxy-roles: myrole" -H "x-forwarded-for: 127.0.0.1"
>
> otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
> curl --version
>
> > Am 01.02.2018 um 22:27 schrieb Kalada <kalada...@t1cg.com>:
> >
> > This is the messages I received
> >
> > * Rebuilt URL to: https://elasticsearch:9200/
> > * Trying 172.22.0.3...
> > * TCP_NODELAY set
> > * Connected to elasticsearch (172.22.0.3) port 9200 (#0)
> > * ALPN, offering http/1.1
> > * successfully set certificate verify locations:
> > * CAfile: /etc/ssl/certs/ca-certificates.crt
> > CApath: none
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> > * Closing connection 0
> > curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> >
> > On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
> > Can you test it via curl or postman (without kibana) like
> >
> > curl -sSk https://elasticsearch:9200 -v -H "x-proxy-user: myuser" -H "x-proxy-roles: myrole" -H "x-forwarded-for: 127.0.0.1"
> >
> > > Am 31.01.2018 um 22:36 schrieb Kalada <kalada...@t1cg.com>:
> > >
> > > internalProxies: '.*' is the current setting. The headers appear to be valid please see an example below
> > >
> > > kibana_1 | {"type":"response","@timestamp":"2018-01-31T20:14:37Z","tags":,"pid":21,"method":"post","statusCode":200,
> > > "req":{"url":"/elasticsearch/_msearch","method":"post","headers":{"es-security-runas-user":"USERNAME","x-proxy-user":"USENAME","host":
> > > "kibana:5601","connection":"close","content-length":"732","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36","accept":"application/json, text/plain, */*
> > > ","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","content-type":"application/x-ndjson",
> > > "kbn-version":"6.1.1","origin":"http://localhost:4180",“referer”:“http://localhost:4180/app/kibana",
> > > "x-forwarded-email":"EMAIL","x-forwarded-for":"172.19.0.1","x-forwarded-user":"USERNAME"},
> > > "remoteAddress":"172.19.0.3","userAgent":"172.19.0.3","referer":"http://localhost:4180/app/kibana"\},
> > > "res":{"statusCode":200,"responseTime":18,"contentLength":9},"message":"POST /elasticsearch/_msearch 200 18ms - 9.0B"}
> > >
> > >
> > >
> > > On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
> > > "xff not done" means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present
> > >
> > > you can test ist by allowing simply all internal proxies
> > >
> > > internalProxies: '.*'
> > >
> > > and make sure your request contains a valid X-Forwarded-For header like described here X-Forwarded-For - Wikipedia
> > >
> > > > Am 31.01.2018 um 19:35 schrieb Kalada <kalada...@t1cg.com>:
> > > >
> > > > Hi Jochen,
> > > >
> > > > Playing with the setting I noticed this message now
> > > >
> > > > [2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
> > > > [2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] 'ElasticsearchSecurityException[xff not done]' extracting credentials from proxy http authenticator
> > > > org.elasticsearch.ElasticsearchSecurityException: xff not done
> > > > at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
> > > > at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
> > > > at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
> > > > at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]
> > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
> > > > [2018-01-31T18:31:55,848][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from basic http authenticator
> > > > [2018-01-31T18:31:55,848][DEBUG][c.f.s.a.BackendRegistry ] User 'User [name=kibana, roles=[kibana_user], requestedTenant=null]' is authenticated
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.a.BackendRegistry ] sgtenant 'null'
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=kibana, roles=[kibana_user], requestedTenant=null] on NODE-1
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] requested indices:admin/mappings/get from 172.19.0.4:42354
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve [.kibana] from class org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest for action indices:admin/mappings/get
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] indicesOptions IndicesOptions[id=6, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=false, ignore_aliases=false]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] 1 raw indices [.kibana]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] 0 requestTypes
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] No date math indices found
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolved [.kibana] to [.kibana]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final indices: [.kibana]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final types:
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] final indices: [.kibana]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] final types: [_all]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=.kibana, type=*]]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for kibana: [kibana_user]
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: 'null'
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null
> > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: kibana_user
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Try wildcard match for ?kibana
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Wildcard match for ?kibana: [.kibana]
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for ?kibana, will check now types [*]
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action [indices:admin/mappings/get] against ?kibana/*: [indices:*, indices:data/read*]
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] removed .kibana*
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] For index ?kibana remaining requested indextype:
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for 'kibana_user.?kibana', evaluate other roles
> > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers kibana_user=>
> > > >
> > > > On Tuesday, January 30, 2018 at 3:54:36 PM UTC-5, Jochen Kressin wrote:
> > > > Then I guess we need the Elasticsearch logs on debug level to see what's going on behind the scenes. In order to enable it see here how to do it:
> > > >
> > > > TLS help | Security for Elasticsearch | Search Guard
> > > >
> > > > Word of warning: Since Kibana issues internal requests periodically you will see a lot of requests in the logs.
> > > >
> > > > Then please make a request via your proxy, and attach the logfiles here.
> > > >
> > > > On Tuesday, January 30, 2018 at 7:21:57 PM UTC+1, Kalada wrote:
> > > > Its still exhibiting the same behavior.
> > > >
> > > > On Tuesday, January 30, 2018 at 1:05:09 PM UTC-5, Jochen Kressin wrote:
> > > > In the xff section of sg_config, can you try to add:
> > > >
> > > > trustedProxies: '.*' # trust all external proxies, regex pattern
> > > >
> > > > And see if this solves the problem? Background:
> > > >
> > > > Search Guard Documentation 404 | Security for Elasticsearch | Search Guard
> > > >
> > > > searchguard.dynamic.http.xff. internalProxies:
> > > >
> > > > "Regular expression, defines the last proxy in the chain before the request is routed to the Elasticsearch cluster. If you have just one proxy, this is effectively the same as the trustedProxies setting. If the last proxy does not match the configured regular expression, an error is thrown and the request is denied."
> > > >
> > > > On Tuesday, January 30, 2018 at 6:43:56 PM UTC+1, Kalada wrote:
> > > > Thank you Jouchen for the quick response. Yes i have whitelisted the proxy headers see kibana.yml below
> > > >
> > > > # When this setting...s value is true Kibana uses the hostname specified in the server.host
> > > >
> > > > # setting. When the value of this setting is false, Kibana uses the hostname of the host
> > > >
> > > > # that connects to this Kibana instance.
> > > >
> > > > #elasticsearch.preserveHost: true
> > > >
> > > >
> > > > # Kibana uses an index in Elasticsearch to store saved searches, visualizations and
> > > >
> > > > # dashboards. Kibana creates a new index if the index doesn...t already exist.
> > > >
> > > > #kibana.index: ".kibana"
> > > >
> > > >
> > > > # The default application to load.
> > > >
> > > > #kibana.defaultAppId: "discover"
> > > >
> > > >
> > > > # Kibana is served by a back end server. This setting specifies the port to use.
> > > >
> > > > server.port: 5601
> > > >
> > > >
> > > > # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
> > > >
> > > > # The default is 'localhost', which usually means remote machines will not be able to connect.
> > > >
> > > > # To allow connections from remote users, set this parameter to a non-loopback address.
> > > >
> > > > server.host: 0.0.0.0
> > > >
> > > >
> > > > # Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects
> > > >
> > > > # the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests
> > > >
> > > > # to Kibana. This setting cannot end in a slash.
> > > >
> > > > server.basePath: /kibana
> > > >
> > > >
> > > > # The maximum payload size in bytes for incoming server requests.
> > > >
> > > > #server.maxPayloadBytes: 1048576
> > > >
> > > >
> > > > # The Kibana server's name. This is used for display purposes.
> > > >
> > > > #server.name: "your-hostname"
> > > >
> > > >
> > > > # The URL of the Elasticsearch instance to use for all your queries.
> > > >
> > > > elasticsearch.url: http://elasticsearch:9200
> > > >
> > > > # If your Elasticsearch is protected with basic authentication, these settings provide
> > > >
> > > > # the username and password that the Kibana server uses to perform maintenance on the Kibana
> > > >
> > > > # index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
> > > >
> > > > # is proxied through the Kibana server.
> > > >
> > > > elasticsearch.username: "kibana"
> > > >
> > > > elasticsearch.password: changeme
> > > >
> > > > searchguard.cookie.password: "4fjNPloEVvrVj28BsVh3jI8t3pmv7Qg6"
> > > >
> > > >
> > > > # Paths to the PEM-format SSL certificate and SSL key files, respectively. These
> > > >
> > > > # files enable SSL for outgoing requests from the Kibana server to the browser.
> > > >
> > > > #server.ssl.cert: /path/to/your/server.crt
> > > >
> > > > #server.ssl.key: /path/to/your/server.key
> > > >
> > > >
> > > > # Optional settings that provide the paths to the PEM-format SSL certificate and key files.
> > > >
> > > > # These files validate that your Elasticsearch backend uses the same key files.
> > > >
> > > > #elasticsearch.ssl.cert: /path/to/your/client.crt
> > > >
> > > > #elasticsearch.ssl.key: /path/to/your/client.key
> > > >
> > > >
> > > > # Optional setting that enables you to specify a path to the PEM file for the certificate
> > > >
> > > > # authority for your Elasticsearch instance.
> > > >
> > > > elasticsearch.ssl.certificateAuthorities: /etc/elasticsearch/searchguard/ssl/ca/root-ca.pem
> > > >
> > > >
> > > > # To disregard the validity of SSL certificates, change this setting...s value to false.
> > > >
> > > > elasticsearch.ssl.verificationMode: "certificate"
> > > >
> > > > # Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
> > > >
> > > > # the elasticsearch.requestTimeout setting.
> > > >
> > > > #elasticsearch.pingTimeout: 1500
> > > >
> > > >
> > > > # Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
> > > >
> > > > # must be a positive integer.
> > > >
> > > > #elasticsearch.requestTimeout: 30000
> > > >
> > > >
> > > > # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
> > > >
> > > > # headers, set this value to (an empty list).
> > > >
> > > > elasticsearch.requestHeadersWhitelist: [ "authorization", "x-forwarded-for", "x-forwarded-by", "x-proxy-user", "x-proxy-roles" ]
> > > >
> > > > searchguard.basicauth.enabled: false
> > > >
> > > >
> > > > # Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
> > > >
> > > > # by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
> > > >
> > > > #elasticsearch.customHeaders: {}
> > > >
> > > >
> > > > # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
> > > >
> > > > #elasticsearch.shardTimeout: 0
> > > >
> > > >
> > > > # Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
> > > >
> > > > #elasticsearch.startupTimeout: 5000
> > > >
> > > >
> > > > # Specifies the path where Kibana creates the process ID file.
> > > >
> > > > #pid.file: /var/run/kibana.pid
> > > >
> > > >
> > > > # Enables you specify a file where Kibana stores log output.
> > > >
> > > > #logging.dest: stdout
> > > >
> > > >
> > > > # Set the value of this setting to true to suppress all logging output.
> > > >
> > > > #logging.silent: false
> > > >
> > > >
> > > > # Set the value of this setting to true to suppress all logging output other than error messages.
> > > >
> > > > #logging.quiet: false
> > > >
> > > >
> > > > # Set the value of this setting to true to log all events, including system usage information
> > > >
> > > > # and all requests.
> > > >
> > > > #logging.verbose: false
> > > >
> > > >
> > > > # Set the interval in milliseconds to sample system and process performance
> > > >
> > > > # metrics. Minimum is 100ms. Defaults to 5000.
> > > >
> > > > #ops.interval: 5000
> > > >
> > > >
> > > > On Tuesday, January 30, 2018 at 12:31:32 PM UTC-5, Jochen Kressin wrote:
> > > > Did you whitelist all the proxy headers in kibana.yml? Kibana has an HTTP header whitelist and will only pass headers listed there to Elasticsearch. Headers not listed will be silently discarded.
> > > >
> > > > See also here for an example:
> > > >
> > > > Kibana Authentication | Security for Elasticsearch | Search Guard
> > > > ("SSO configuration: Whitelisting HTTP headers")
> > > >
> > > > On Tuesday, January 30, 2018 at 5:23:49 PM UTC+1, Kalada wrote:
> > > > When asking questions, please provide the following information:
> > > >
> > > > Hi,
> > > >
> > > >
> > > > I was able to configure a docker setup of the ELK stack behind oauth2_proxy and nginx using search-guard as a authentication medium. Once a user is authenticated they are forwarded to kibana correctly. However it seems all users are assigned the internal kibana user and its associated role. I would like to be able to assign a user as an admin role, which to my understanding is done using the sg_roles_mapping.yml file, but it does not appear to be working correctly. I can see the headers are passed to Kibana but elasticsearch does not reflect this. I’d really appreciate any help. Thanks
> > > >
> > > >
> > > >
> > > > nginx.conf
> > > >
> > > > http {
> > > >
> > > > server {
> > > >
> > > > listen ${NGINX_PORT};
> > > >
> > > > server_name localhost;
> > > >
> > > >
> > > > location ${BASE_PATH} {
> > > >
> > > >
> > > > proxy_pass http://kibana:5601/;
> > > >
> > > >
> > > > proxy_set_header Authorization "Basic ${DIGEST}";
> > > >
> > > >
> > > > proxy_set_header es-security-runas-user ${DOLLAR}http_x_forwarded_user;
> > > >
> > > > proxy_set_header x-proxy-user ${DOLLAR}http_x_forwarded_user;
> > > >
> > > > proxy_set_header x-proxy-roles user ${DOLLAR}http_x_forwarded_user;
> > > >
> > > >
> > > > rewrite /login http://localhost:4180/oauth2/sign_in redirect;
> > > >
> > > > }
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > > events {
> > > >
> > > > worker_connections 1024;
> > > >
> > > > }
> > > >
> > > >
> > > > Elastic search log entry
> > > >
> > > >
> > > > elasticsearch_1 | [2018-01-30T15:54:47,853][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User[name=kibana, roles=[kibana_user], requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [kibana_user]]
> > > >
> > > > elasticsearch_1 | [2018-01-30T15:54:47,854][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {}
> > > >
> > > >
> > > > Kibana Log Entry
> > > >
> > > > kibana_1 | {"type":"response","@timestamp":"2018-01-30T15:54:47Z","tags":,"pid":28,"method":"post","statusCode":503,"req":{"url":"/api/saved_objects/index-pattern?overwrite=false","method":"post","headers":{"es-security-runas-user":"user_name","x-proxy-user":"user_name","x-proxy-roles":"admin","host":"kibana:5601","connection":"close","content-length":"66","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","content-type":"application/json;charset=UTF-8","kbn-version":"6.1.1","origin":"http://localhost:4180”,“referer”:“http://localhost:4180/kibana/app/kibana",“x-forwarded-email”:"em ...@email.com”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“user_name”},“remoteAddress”:“172.19.0.2”,“userAgent”:“172.19.0.2”,“referer”:"http://localhost:4180/kibana/app/kibana"},“res”:{“statusCode”:503,“responseTime”:32,“contentLength”:9},“message”:"POST /api/saved_objects/index-pattern?overwrite=false 503 32ms - 9.0B"}
> > > >
> > > >
> > > > sg_config.yml
> > > >
> > > >
> > > > searchguard:
> > > >
> > > > dynamic:
> > > >
> > > > http:
> > > >
> > > > anonymous_auth_enabled: false
> > > >
> > > > xff:
> > > >
> > > > enabled: true
> > > >
> > > > internalProxies: '.*'
> > > >
> > > > remoteIpHeader: 'x-forwarded-for'
> > > >
> > > > proxiesHeader: 'x-forwarded-by'
> > > >
> > > > proxy_auth_domain:
> > > >
> > > > enabled: true
> > > >
> > > > order: 0
> > > >
> > > > http_authenticator:
> > > >
> > > > type: proxy
> > > >
> > > > challenge: false
> > > >
> > > > config:
> > > >
> > > > user_header: "x-proxy-user"
> > > >
> > > > roles_header: "x-proxy-roles"
> > > >
> > > > authentication_backend:
> > > >
> > > > type: noop
> > > >
> > > > authc:
> > > >
> > > > basic_internal_auth:
> > > >
> > > > enabled: true
> > > >
> > > > order: 1
> > > >
> > > > http_authenticator:
> > > >
> > > > type: basic
> > > >
> > > > challenge: true
> > > >
> > > > authentication_backend:
> > > >
> > > > type: internal
> > > >
> > > >
> > > >
> > > > sg_roles_mapping.yml
> > > >
> > > >
> > > > admin:
> > > >
> > > > backendroles:
> > > >
> > > > - admin
> > > >
> > > > host:
> > > >
> > > > - "*"
> > > >
> > > > users:
> > > >
> > > > - elastic
> > > >
> > > > - user_name
> > > >
> > > > kibana_user:
> > > >
> > > > backendroles:
> > > >
> > > > - kibana_user
> > > >
> > > > host:
> > > >
> > > > - "*"
> > > >
> > > > users:
> > > >
> > > > - kibana
> > > >
> > > >
> > > > logstash_user:
> > > >
> > > > backendroles:
> > > >
> > > > - logstash_user
> > > >
> > > > host:
> > > >
> > > > - "*"
> > > >
> > > > users:
> > > >
> > > > - logstash
> > > >
> > > >
> > > > beats_user:
> > > >
> > > > backendroles:
> > > >
> > > > - beats_user
> > > >
> > > > host:
> > > >
> > > > - "*"
> > > >
> > > > users:
> > > >
> > > > - beats
> > > >
> > > >
> > > >
> > > > sg_action_group.yml
> > > >
> > > >
> > > >
> > > > # INDICES
> > > >
> > > > READ:
> > > >
> > > > - "indices:data/read*"
> > > >
> > > > WRITE:
> > > >
> > > > - "indices:data/write*"
> > > >
> > > > CRUD:
> > > >
> > > > - READ
> > > >
> > > > - WRITE
> > > >
> > > > CREATE_INDEX:
> > > >
> > > > - "indices:admin/create*"
> > > >
> > > > - "indices:admin/mapping/put"
> > > >
> > > > DELETE_INDEX:
> > > >
> > > > - "indices:admin/delete*"
> > > >
> > > > INDEX_OWNER:
> > > >
> > > > - CREATE_INDEX
> > > >
> > > > - CRUD
> > > >
> > > > INDEX_ALL:
> > > >
> > > > - "indices:*"
> > > >
> > > >
> > > >
> > > > # ELASTICSEARCH ENTRYPOINT
> > > >
> > > > GET_TEMPLATE:
> > > >
> > > > - "indices:admin/template/get"
> > > >
> > > > PUT_TEMPLATE:
> > > >
> > > > - "indices:admin/template/put"
> > > >
> > > > TEMPLATE_OWNER:
> > > >
> > > > - GET_TEMPLATE
> > > >
> > > > - PUT_TEMPLATE
> > > >
> > > > ES_INPUT:
> > > >
> > > > - TEMPLATE_OWNER
> > > >
> > > > - WRITE
> > > >
> > > > - MONITOR
> > > >
> > > >
> > > >
> > > > # MONITORING
> > > >
> > > > MONITOR:
> > > >
> > > > - "cluster:monitor*"
> > > >
> > > >
> > > >
> > > > # SUPER POWERS
> > > >
> > > > ALL:
> > > >
> > > >
> > > > - "*"
> > > >
> > > >
> > > >
> > > > Search Guard: "20.1"
> > > >
> > > > ElasticSearch:6.1.1
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > > > To post to this group, send email to search...@googlegroups.com.
> > > > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d04c8a95-74cb-41c7-a24d-baf2e0fcda2d%40googlegroups.com\.
> > > > For more options, visit https://groups.google.com/d/optout\.
> > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > > To post to this group, send email to search...@googlegroups.com.
> > > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/3ba4a31b-2667-4b82-89e7-c95fcdb98caf%40googlegroups.com\.
> > > For more options, visit https://groups.google.com/d/optout\.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > To post to this group, send email to search...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/554d479e-fdd9-4357-ba3d-32a2d03d2699%40googlegroups.com\.
> > For more options, visit https://groups.google.com/d/optout\.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/0718d16a-a61e-46b2-8053-fdd1b2f25ddf%40googlegroups.com\.
> For more options, visit https://groups.google.com/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/12c6d585-3612-4b96-b009-83001c46c5fd%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.
<elasticsearch-default.log><elasticsearch.yml><sg_config.yml>

Kalada · February 5, 2018, 1:18pm

With the suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs

bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/
Trying 172.22.0.2…
TCP_NODELAY set
Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 439

<

{

“name” : “NODE-1”,

“cluster_name” : “elasticsearch-default”,

“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,

“version” : {

“number” : “6.1.1”,

“build_hash” : “bd92e7f”,

“build_date” : “2017-12-17T20:23:25.338Z”,

“build_snapshot” : false,

“lucene_version” : “7.1.0”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}

Connection #0 to host elasticsearch left intact

···

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:

should be something like this (the proxy authenticator was disabled)

searchguard:

dynamic:
http:

  anonymous_auth_enabled: false

  xff:

    enabled: true

    internalProxies: '.*'

    remoteIpHeader: 'x-forwarded-for'

    proxiesHeader: 'x-forwarded-by'

    trustedProxies: '.*'

authc:
  proxy_auth_domain:

    http_enabled: true

    transport_enabled: false
    order: 0

    http_authenticator:

      type: proxy

      challenge: false

      config:

        user_header: "x-proxy-user"

        roles_header: "x-proxy-roles"

    authentication_backend:

      type: noop

  basic_internal_auth:

    enabled: true

    order: 1

    http_authenticator:

      type: basic

      challenge: true

    authentication_backend:

      type: internal
Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:

pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.

searchguard_google_group · February 5, 2018, 2:29pm

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:

curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”

curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

···

Am 02.02.2018 um 20:16 schrieb Kalada > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 439

<

{

“name” : “NODE-1”,

“cluster_name” : “elasticsearch-default”,

“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,

“version” : {

"number" : "6.1.1",

"build_hash" : "bd92e7f",

"build_date" : "2017-12-17T20:23:25.338Z",

"build_snapshot" : false,

"lucene_version" : "7.1.0",

"minimum_wire_compatibility_version" : "5.6.0",

"minimum_index_compatibility_version" : "5.0.0"

},

“tagline” : “You Know, for Search”

}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:

should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

Kalada · February 5, 2018, 7:33pm

Using the curl command I can clearly see the test user I’ve
created is authenticating correctly, Please see below

bash-4.4# curl -sS
http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: USERNAME”
-H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Trying 172.22.0.5…
TCP_NODELAY set
Connected to elasticsearch (172.22.0.5) port 9200 (#0)

GET /_searchguard/authinfo HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 287

<

Connection #0 to host elasticsearch left intact

{“user”:“User [name= USERNAME, roles=[ ROLE],
requestedTenant=null]”,“user_name”:" USERNAME “,“user_requested_tenant”:null,“remote_address”:“127.0.0.1:34730”,“backend_roles”:[”
ROLE “],“custom_attribute_names”:,“sg_roles”:[”
ROLE “],“sg_tenants”:{” ROLE ":true},“principal”:null,“peer_certificates”:“0”}

However, each time the proxy user tries to authenticate from
kibana this message appears.

elasticsearch_1 |
[2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http
authenticator

elasticsearch_1 |
[2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not
done]’ extracting credentials from proxy http authenticator

elasticsearch_1 |
org.elasticsearch.ElasticsearchSecurityException: xff not done

elasticsearch_1

at

com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49)
~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1

at

com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380)
[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1

at

com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122)
[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1

at

com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47)
[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1

at

com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74)
[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic
http authentication using the internal kibana user. Is this
the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

···

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:

curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”

curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

Am 02.02.2018 um 20:16 schrieb Kalada > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > > >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 439

<

{

“name” : “NODE-1”,

“cluster_name” : “elasticsearch-default”,

“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,

“version” : {
"number" : "6.1.1",
"build_hash" : "bd92e7f",
"build_date" : "2017-12-17T20:23:25.338Z",
"build_snapshot" : false,
"lucene_version" : "7.1.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},

“tagline” : “You Know, for Search”

}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:

should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.

searchguard_google_group · February 5, 2018, 7:38pm

can you post yor kibana.yml?

···

Am 05.02.2018 um 20:33 schrieb Kalada <kalada.opuiyo@t1cg.com>:

Using the curl command I can clearly see the test user I’ve created is authenticating correctly, Please see below

bash-4.4# curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H "x-proxy-user: USERNAME" -H "x-proxy-roles: ROLE" -H "x-forwarded-for: 127.0.0.1"
* Trying 172.22.0.5...
* TCP_NODELAY set
* Connected to elasticsearch (172.22.0.5) port 9200 (#0)
> GET /_searchguard/authinfo HTTP/1.1
> Host: elasticsearch:9200
> User-Agent: curl/7.57.0
> Accept: */*
> x-proxy-user: USERNAME
> x-proxy-roles: ROLE
> x-forwarded-for: 127.0.0.1
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 287
<
* Connection #0 to host elasticsearch left intact
{"user":"User [name= USERNAME, roles=[ ROLE], requestedTenant=null]","user_name":" USERNAME ","user_requested_tenant":null,"remote_address":"127.0.0.1:34730","backend_roles":[" ROLE "],"custom_attribute_names":,"sg_roles":[" ROLE "],"sg_tenants":{" ROLE ":true},"principal":null,"peer_certificates":"0"}

However, each time the proxy user tries to authenticate from kibana this message appears.

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] 'ElasticsearchSecurityException[xff not done]' extracting credentials from proxy http authenticator
elasticsearch_1 | org.elasticsearch.ElasticsearchSecurityException: xff not done
elasticsearch_1 | at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
elasticsearch_1 | at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic http authentication using the internal kibana user. Is this the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:
This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H "x-proxy-user: myuser" -H "x-proxy-roles: xxx" -H "x-forwarded-for: 127.0.0.1"
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

> Am 02.02.2018 um 20:16 schrieb Kalada > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > >
> bash-4.4# curl -sS http://elasticsearch:9200 -v -H "x-proxy-user: USERNAME" -H "x-proxy-roles: ROLE" -H "x-forwarded-for: 127.0.0.1"
> * Rebuilt URL to: http://elasticsearch:9200/
> * Trying 172.22.0.2...
> * TCP_NODELAY set
> * Connected to elasticsearch (172.22.0.2) port 9200 (#0)
> > GET / HTTP/1.1
> > Host: elasticsearch:9200
> > User-Agent: curl/7.57.0
> > Accept: */*
> > x-proxy-user: USERNAME
> > x-proxy-roles: ROLE
> > x-forwarded-for: 127.0.0.1
> >
> < HTTP/1.1 200 OK
> < content-type: application/json; charset=UTF-8
> < content-length: 439
> <
> {
> "name" : "NODE-1",
> "cluster_name" : "elasticsearch-default",
> "cluster_uuid" : "JeICJqFVSXO1jxzgdKSxMg",
> "version" : {
> "number" : "6.1.1",
> "build_hash" : "bd92e7f",
> "build_date" : "2017-12-17T20:23:25.338Z",
> "build_snapshot" : false,
> "lucene_version" : "7.1.0",
> "minimum_wire_compatibility_version" : "5.6.0",
> "minimum_index_compatibility_version" : "5.0.0"
> },
> "tagline" : "You Know, for Search"
> }
> * Connection #0 to host elasticsearch left intact
>
>
>
> On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:
> should be something like this (the proxy authenticator was disabled)
>
> searchguard:
> dynamic:
> http:
> anonymous_auth_enabled: false
> xff:
> enabled: true
> internalProxies: '.*'
> remoteIpHeader: 'x-forwarded-for'
> proxiesHeader: 'x-forwarded-by'
> trustedProxies: '.*'
> authc:
> proxy_auth_domain:
> http_enabled: true
> transport_enabled: false
> order: 0
> http_authenticator:
> type: proxy
> challenge: false
> config:
> user_header: "x-proxy-user"
> roles_header: "x-proxy-roles"
> authentication_backend:
> type: noop
> basic_internal_auth:
> enabled: true
> order: 1
> http_authenticator:
> type: basic
> challenge: true
> authentication_backend:
> type: internal
>
> > Am 02.02.2018 um 16:18 schrieb Kalada <kalada...@t1cg.com>:
> >
> > Please see the attached
> >
> > On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
> > pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)
> >
> > > Am 01.02.2018 um 23:04 schrieb Kalada <kalada...@t1cg.com>:
> > >
> > >
> > > Thank you https wasnt enabled please see below
> > >
> > > * Rebuilt URL to: http://elasticsearch:9200/
> > > * Trying 172.22.0.3...
> > > * TCP_NODELAY set
> > > * Connected to elasticsearch (172.22.0.3) port 9200 (#0)
> > > > GET / HTTP/1.1
> > > > Host: elasticsearch:9200
> > > > User-Agent: curl/7.57.0
> > > > Accept: */*
> > > > x-proxy-user: myuser
> > > > x-proxy-roles: myrole
> > > > x-forwarded-for: 127.0.0.1
> > > >
> > > < HTTP/1.1 401 Unauthorized
> > > < WWW-Authenticate: Basic realm="Search Guard"
> > > < content-type: text/plain; charset=UTF-8
> > > < content-length: 12
> > > <
> > > * Connection #0 to host elasticsearch left intact
> > >
> > >
> > > On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
> > > Have you https enabled? If not its
> > > curl -sS http://elasticsearch:9200 -v -H "x-proxy-user: myuser" -H "x-proxy-roles: myrole" -H "x-forwarded-for: 127.0.0.1"
> > >
> > > otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
> > > curl --version
> > >
> > > > Am 01.02.2018 um 22:27 schrieb Kalada <kalada...@t1cg.com>:
> > > >
> > > > This is the messages I received
> > > >
> > > > * Rebuilt URL to: https://elasticsearch:9200/
> > > > * Trying 172.22.0.3...
> > > > * TCP_NODELAY set
> > > > * Connected to elasticsearch (172.22.0.3) port 9200 (#0)
> > > > * ALPN, offering http/1.1
> > > > * successfully set certificate verify locations:
> > > > * CAfile: /etc/ssl/certs/ca-certificates.crt
> > > > CApath: none
> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > > * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> > > > * Closing connection 0
> > > > curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> > > >
> > > > On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
> > > > Can you test it via curl or postman (without kibana) like
> > > >
> > > > curl -sSk https://elasticsearch:9200 -v -H "x-proxy-user: myuser" -H "x-proxy-roles: myrole" -H "x-forwarded-for: 127.0.0.1"
> > > >
> > > > > Am 31.01.2018 um 22:36 schrieb Kalada <kalada...@t1cg.com>:
> > > > >
> > > > > internalProxies: '.*' is the current setting. The headers appear to be valid please see an example below
> > > > >
> > > > > kibana_1 | {"type":"response","@timestamp":"2018-01-31T20:14:37Z","tags":,"pid":21,"method":"post","statusCode":200,
> > > > > "req":{"url":"/elasticsearch/_msearch","method":"post","headers":{"es-security-runas-user":"USERNAME","x-proxy-user":"USENAME","host":
> > > > > "kibana:5601","connection":"close","content-length":"732","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
> > > > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36","accept":"application/json, text/plain, */*
> > > > > ","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","content-type":"application/x-ndjson",
> > > > > "kbn-version":"6.1.1","origin":"http://localhost:4180",“referer”:“http://localhost:4180/app/kibana",
> > > > > "x-forwarded-email":"EMAIL","x-forwarded-for":"172.19.0.1","x-forwarded-user":"USERNAME"},
> > > > > "remoteAddress":"172.19.0.3","userAgent":"172.19.0.3","referer":"http://localhost:4180/app/kibana"\},
> > > > > "res":{"statusCode":200,"responseTime":18,"contentLength":9},"message":"POST /elasticsearch/_msearch 200 18ms - 9.0B"}
> > > > >
> > > > >
> > > > >
> > > > > On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
> > > > > "xff not done" means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present
> > > > >
> > > > > you can test ist by allowing simply all internal proxies
> > > > >
> > > > > internalProxies: '.*'
> > > > >
> > > > > and make sure your request contains a valid X-Forwarded-For header like described here X-Forwarded-For - Wikipedia
> > > > >
> > > > > > Am 31.01.2018 um 19:35 schrieb Kalada <kalada...@t1cg.com>:
> > > > > >
> > > > > > Hi Jochen,
> > > > > >
> > > > > > Playing with the setting I noticed this message now
> > > > > >
> > > > > > [2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
> > > > > > [2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] 'ElasticsearchSecurityException[xff not done]' extracting credentials from proxy http authenticator
> > > > > > org.elasticsearch.ElasticsearchSecurityException: xff not done
> > > > > > at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > > > at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > > > at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
> > > > > > at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
> > > > > > at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
> > > > > > at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
> > > > > > at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > > > at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > > > at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]
> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
> > > > > > [2018-01-31T18:31:55,848][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from basic http authenticator
> > > > > > [2018-01-31T18:31:55,848][DEBUG][c.f.s.a.BackendRegistry ] User 'User [name=kibana, roles=[kibana_user], requestedTenant=null]' is authenticated
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.a.BackendRegistry ] sgtenant 'null'
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=kibana, roles=[kibana_user], requestedTenant=null] on NODE-1
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] requested indices:admin/mappings/get from 172.19.0.4:42354
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve [.kibana] from class org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest for action indices:admin/mappings/get
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] indicesOptions IndicesOptions[id=6, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=false, ignore_aliases=false]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] 1 raw indices [.kibana]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] 0 requestTypes
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] No date math indices found
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolved [.kibana] to [.kibana]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final indices: [.kibana]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final types:
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] final indices: [.kibana]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] final types: [_all]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=.kibana, type=*]]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for kibana: [kibana_user]
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: 'null'
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null
> > > > > > [2018-01-31T18:31:55,849][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: kibana_user
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Try wildcard match for ?kibana
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Wildcard match for ?kibana: [.kibana]
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for ?kibana, will check now types [*]
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action [indices:admin/mappings/get] against ?kibana/*: [indices:*, indices:data/read*]
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] removed .kibana*
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] For index ?kibana remaining requested indextype:
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for 'kibana_user.?kibana', evaluate other roles
> > > > > > [2018-01-31T18:31:55,850][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers kibana_user=>
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 3:54:36 PM UTC-5, Jochen Kressin wrote:
> > > > > > Then I guess we need the Elasticsearch logs on debug level to see what's going on behind the scenes. In order to enable it see here how to do it:
> > > > > >
> > > > > > TLS help | Security for Elasticsearch | Search Guard
> > > > > >
> > > > > > Word of warning: Since Kibana issues internal requests periodically you will see a lot of requests in the logs.
> > > > > >
> > > > > > Then please make a request via your proxy, and attach the logfiles here.
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 7:21:57 PM UTC+1, Kalada wrote:
> > > > > > Its still exhibiting the same behavior.
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 1:05:09 PM UTC-5, Jochen Kressin wrote:
> > > > > > In the xff section of sg_config, can you try to add:
> > > > > >
> > > > > > trustedProxies: '.*' # trust all external proxies, regex pattern
> > > > > >
> > > > > > And see if this solves the problem? Background:
> > > > > >
> > > > > > Search Guard Documentation 404 | Security for Elasticsearch | Search Guard
> > > > > >
> > > > > > searchguard.dynamic.http.xff. internalProxies:
> > > > > >
> > > > > > "Regular expression, defines the last proxy in the chain before the request is routed to the Elasticsearch cluster. If you have just one proxy, this is effectively the same as the trustedProxies setting. If the last proxy does not match the configured regular expression, an error is thrown and the request is denied."
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 6:43:56 PM UTC+1, Kalada wrote:
> > > > > > Thank you Jouchen for the quick response. Yes i have whitelisted the proxy headers see kibana.yml below
> > > > > >
> > > > > > # When this setting...s value is true Kibana uses the hostname specified in the server.host
> > > > > >
> > > > > > # setting. When the value of this setting is false, Kibana uses the hostname of the host
> > > > > >
> > > > > > # that connects to this Kibana instance.
> > > > > >
> > > > > > #elasticsearch.preserveHost: true
> > > > > >
> > > > > >
> > > > > > # Kibana uses an index in Elasticsearch to store saved searches, visualizations and
> > > > > >
> > > > > > # dashboards. Kibana creates a new index if the index doesn...t already exist.
> > > > > >
> > > > > > #kibana.index: ".kibana"
> > > > > >
> > > > > >
> > > > > > # The default application to load.
> > > > > >
> > > > > > #kibana.defaultAppId: "discover"
> > > > > >
> > > > > >
> > > > > > # Kibana is served by a back end server. This setting specifies the port to use.
> > > > > >
> > > > > > server.port: 5601
> > > > > >
> > > > > >
> > > > > > # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
> > > > > >
> > > > > > # The default is 'localhost', which usually means remote machines will not be able to connect.
> > > > > >
> > > > > > # To allow connections from remote users, set this parameter to a non-loopback address.
> > > > > >
> > > > > > server.host: 0.0.0.0
> > > > > >
> > > > > >
> > > > > > # Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects
> > > > > >
> > > > > > # the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests
> > > > > >
> > > > > > # to Kibana. This setting cannot end in a slash.
> > > > > >
> > > > > > server.basePath: /kibana
> > > > > >
> > > > > >
> > > > > > # The maximum payload size in bytes for incoming server requests.
> > > > > >
> > > > > > #server.maxPayloadBytes: 1048576
> > > > > >
> > > > > >
> > > > > > # The Kibana server's name. This is used for display purposes.
> > > > > >
> > > > > > #server.name: "your-hostname"
> > > > > >
> > > > > >
> > > > > > # The URL of the Elasticsearch instance to use for all your queries.
> > > > > >
> > > > > > elasticsearch.url: http://elasticsearch:9200
> > > > > >
> > > > > > # If your Elasticsearch is protected with basic authentication, these settings provide
> > > > > >
> > > > > > # the username and password that the Kibana server uses to perform maintenance on the Kibana
> > > > > >
> > > > > > # index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
> > > > > >
> > > > > > # is proxied through the Kibana server.
> > > > > >
> > > > > > elasticsearch.username: "kibana"
> > > > > >
> > > > > > elasticsearch.password: changeme
> > > > > >
> > > > > > searchguard.cookie.password: "4fjNPloEVvrVj28BsVh3jI8t3pmv7Qg6"
> > > > > >
> > > > > >
> > > > > > # Paths to the PEM-format SSL certificate and SSL key files, respectively. These
> > > > > >
> > > > > > # files enable SSL for outgoing requests from the Kibana server to the browser.
> > > > > >
> > > > > > #server.ssl.cert: /path/to/your/server.crt
> > > > > >
> > > > > > #server.ssl.key: /path/to/your/server.key
> > > > > >
> > > > > >
> > > > > > # Optional settings that provide the paths to the PEM-format SSL certificate and key files.
> > > > > >
> > > > > > # These files validate that your Elasticsearch backend uses the same key files.
> > > > > >
> > > > > > #elasticsearch.ssl.cert: /path/to/your/client.crt
> > > > > >
> > > > > > #elasticsearch.ssl.key: /path/to/your/client.key
> > > > > >
> > > > > >
> > > > > > # Optional setting that enables you to specify a path to the PEM file for the certificate
> > > > > >
> > > > > > # authority for your Elasticsearch instance.
> > > > > >
> > > > > > elasticsearch.ssl.certificateAuthorities: /etc/elasticsearch/searchguard/ssl/ca/root-ca.pem
> > > > > >
> > > > > >
> > > > > > # To disregard the validity of SSL certificates, change this setting...s value to false.
> > > > > >
> > > > > > elasticsearch.ssl.verificationMode: "certificate"
> > > > > >
> > > > > > # Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
> > > > > >
> > > > > > # the elasticsearch.requestTimeout setting.
> > > > > >
> > > > > > #elasticsearch.pingTimeout: 1500
> > > > > >
> > > > > >
> > > > > > # Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
> > > > > >
> > > > > > # must be a positive integer.
> > > > > >
> > > > > > #elasticsearch.requestTimeout: 30000
> > > > > >
> > > > > >
> > > > > > # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
> > > > > >
> > > > > > # headers, set this value to (an empty list).
> > > > > >
> > > > > > elasticsearch.requestHeadersWhitelist: [ "authorization", "x-forwarded-for", "x-forwarded-by", "x-proxy-user", "x-proxy-roles" ]
> > > > > >
> > > > > > searchguard.basicauth.enabled: false
> > > > > >
> > > > > >
> > > > > > # Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
> > > > > >
> > > > > > # by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
> > > > > >
> > > > > > #elasticsearch.customHeaders: {}
> > > > > >
> > > > > >
> > > > > > # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
> > > > > >
> > > > > > #elasticsearch.shardTimeout: 0
> > > > > >
> > > > > >
> > > > > > # Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
> > > > > >
> > > > > > #elasticsearch.startupTimeout: 5000
> > > > > >
> > > > > >
> > > > > > # Specifies the path where Kibana creates the process ID file.
> > > > > >
> > > > > > #pid.file: /var/run/kibana.pid
> > > > > >
> > > > > >
> > > > > > # Enables you specify a file where Kibana stores log output.
> > > > > >
> > > > > > #logging.dest: stdout
> > > > > >
> > > > > >
> > > > > > # Set the value of this setting to true to suppress all logging output.
> > > > > >
> > > > > > #logging.silent: false
> > > > > >
> > > > > >
> > > > > > # Set the value of this setting to true to suppress all logging output other than error messages.
> > > > > >
> > > > > > #logging.quiet: false
> > > > > >
> > > > > >
> > > > > > # Set the value of this setting to true to log all events, including system usage information
> > > > > >
> > > > > > # and all requests.
> > > > > >
> > > > > > #logging.verbose: false
> > > > > >
> > > > > >
> > > > > > # Set the interval in milliseconds to sample system and process performance
> > > > > >
> > > > > > # metrics. Minimum is 100ms. Defaults to 5000.
> > > > > >
> > > > > > #ops.interval: 5000
> > > > > >
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 12:31:32 PM UTC-5, Jochen Kressin wrote:
> > > > > > Did you whitelist all the proxy headers in kibana.yml? Kibana has an HTTP header whitelist and will only pass headers listed there to Elasticsearch. Headers not listed will be silently discarded.
> > > > > >
> > > > > > See also here for an example:
> > > > > >
> > > > > > Kibana Authentication | Security for Elasticsearch | Search Guard
> > > > > > ("SSO configuration: Whitelisting HTTP headers")
> > > > > >
> > > > > > On Tuesday, January 30, 2018 at 5:23:49 PM UTC+1, Kalada wrote:
> > > > > > When asking questions, please provide the following information:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > >
> > > > > > I was able to configure a docker setup of the ELK stack behind oauth2_proxy and nginx using search-guard as a authentication medium. Once a user is authenticated they are forwarded to kibana correctly. However it seems all users are assigned the internal kibana user and its associated role. I would like to be able to assign a user as an admin role, which to my understanding is done using the sg_roles_mapping.yml file, but it does not appear to be working correctly. I can see the headers are passed to Kibana but elasticsearch does not reflect this. I’d really appreciate any help. Thanks
> > > > > >
> > > > > >
> > > > > >
> > > > > > nginx.conf
> > > > > >
> > > > > > http {
> > > > > >
> > > > > > server {
> > > > > >
> > > > > > listen ${NGINX_PORT};
> > > > > >
> > > > > > server_name localhost;
> > > > > >
> > > > > >
> > > > > > location ${BASE_PATH} {
> > > > > >
> > > > > >
> > > > > > proxy_pass http://kibana:5601/;
> > > > > >
> > > > > >
> > > > > > proxy_set_header Authorization "Basic ${DIGEST}";
> > > > > >
> > > > > >
> > > > > > proxy_set_header es-security-runas-user ${DOLLAR}http_x_forwarded_user;
> > > > > >
> > > > > > proxy_set_header x-proxy-user ${DOLLAR}http_x_forwarded_user;
> > > > > >
> > > > > > proxy_set_header x-proxy-roles user ${DOLLAR}http_x_forwarded_user;
> > > > > >
> > > > > >
> > > > > > rewrite /login http://localhost:4180/oauth2/sign_in redirect;
> > > > > >
> > > > > > }
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > > events {
> > > > > >
> > > > > > worker_connections 1024;
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > > Elastic search log entry
> > > > > >
> > > > > >
> > > > > > elasticsearch_1 | [2018-01-30T15:54:47,853][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User[name=kibana, roles=[kibana_user], requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [kibana_user]]
> > > > > >
> > > > > > elasticsearch_1 | [2018-01-30T15:54:47,854][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {}
> > > > > >
> > > > > >
> > > > > > Kibana Log Entry
> > > > > >
> > > > > > kibana_1 | {"type":"response","@timestamp":"2018-01-30T15:54:47Z","tags":,"pid":28,"method":"post","statusCode":503,"req":{"url":"/api/saved_objects/index-pattern?overwrite=false","method":"post","headers":{"es-security-runas-user":"user_name","x-proxy-user":"user_name","x-proxy-roles":"admin","host":"kibana:5601","connection":"close","content-length":"66","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","content-type":"application/json;charset=UTF-8","kbn-version":"6.1.1","origin":"http://localhost:4180”,“referer”:“http://localhost:4180/kibana/app/kibana",“x-forwarded-email”:"em ...@email.com”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“user_name”},“remoteAddress”:“172.19.0.2”,“userAgent”:“172.19.0.2”,“referer”:"http://localhost:4180/kibana/app/kibana"},“res”:{“statusCode”:503,“responseTime”:32,“contentLength”:9},“message”:"POST /api/saved_objects/index-pattern?overwrite=false 503 32ms - 9.0B"}
> > > > > >
> > > > > >
> > > > > > sg_config.yml
> > > > > >
> > > > > >
> > > > > > searchguard:
> > > > > >
> > > > > > dynamic:
> > > > > >
> > > > > > http:
> > > > > >
> > > > > > anonymous_auth_enabled: false
> > > > > >
> > > > > > xff:
> > > > > >
> > > > > > enabled: true
> > > > > >
> > > > > > internalProxies: '.*'
> > > > > >
> > > > > > remoteIpHeader: 'x-forwarded-for'
> > > > > >
> > > > > > proxiesHeader: 'x-forwarded-by'
> > > > > >
> > > > > > proxy_auth_domain:
> > > > > >
> > > > > > enabled: true
> > > > > >
> > > > > > order: 0
> > > > > >
> > > > > > http_authenticator:
> > > > > >
> > > > > > type: proxy
> > > > > >
> > > > > > challenge: false
> > > > > >
> > > > > > config:
> > > > > >
> > > > > > user_header: "x-proxy-user"
> > > > > >
> > > > > > roles_header: "x-proxy-roles"
> > > > > >
> > > > > > authentication_backend:
> > > > > >
> > > > > > type: noop
> > > > > >
> > > > > > authc:
> > > > > >
> > > > > > basic_internal_auth:
> > > > > >
> > > > > > enabled: true
> > > > > >
> > > > > > order: 1
> > > > > >
> > > > > > http_authenticator:
> > > > > >
> > > > > > type: basic
> > > > > >
> > > > > > challenge: true
> > > > > >
> > > > > > authentication_backend:
> > > > > >
> > > > > > type: internal
> > > > > >
> > > > > >
> > > > > >
> > > > > > sg_roles_mapping.yml
> > > > > >
> > > > > >
> > > > > > admin:
> > > > > >
> > > > > > backendroles:
> > > > > >
> > > > > > - admin
> > > > > >
> > > > > > host:
> > > > > >
> > > > > > - "*"
> > > > > >
> > > > > > users:
> > > > > >
> > > > > > - elastic
> > > > > >
> > > > > > - user_name
> > > > > >
> > > > > > kibana_user:
> > > > > >
> > > > > > backendroles:
> > > > > >
> > > > > > - kibana_user
> > > > > >
> > > > > > host:
> > > > > >
> > > > > > - "*"
> > > > > >
> > > > > > users:
> > > > > >
> > > > > > - kibana
> > > > > >
> > > > > >
> > > > > > logstash_user:
> > > > > >
> > > > > > backendroles:
> > > > > >
> > > > > > - logstash_user
> > > > > >
> > > > > > host:
> > > > > >
> > > > > > - "*"
> > > > > >
> > > > > > users:
> > > > > >
> > > > > > - logstash
> > > > > >
> > > > > >
> > > > > > beats_user:
> > > > > >
> > > > > > backendroles:
> > > > > >
> > > > > > - beats_user
> > > > > >
> > > > > > host:
> > > > > >
> > > > > > - "*"
> > > > > >
> > > > > > users:
> > > > > >
> > > > > > - beats
> > > > > >
> > > > > >
> > > > > >
> > > > > > sg_action_group.yml
> > > > > >
> > > > > >
> > > > > >
> > > > > > # INDICES
> > > > > >
> > > > > > READ:
> > > > > >
> > > > > > - "indices:data/read*"
> > > > > >
> > > > > > WRITE:
> > > > > >
> > > > > > - "indices:data/write*"
> > > > > >
> > > > > > CRUD:
> > > > > >
> > > > > > - READ
> > > > > >
> > > > > > - WRITE
> > > > > >
> > > > > > CREATE_INDEX:
> > > > > >
> > > > > > - "indices:admin/create*"
> > > > > >
> > > > > > - "indices:admin/mapping/put"
> > > > > >
> > > > > > DELETE_INDEX:
> > > > > >
> > > > > > - "indices:admin/delete*"
> > > > > >
> > > > > > INDEX_OWNER:
> > > > > >
> > > > > > - CREATE_INDEX
> > > > > >
> > > > > > - CRUD
> > > > > >
> > > > > > INDEX_ALL:
> > > > > >
> > > > > > - "indices:*"
> > > > > >
> > > > > >
> > > > > >
> > > > > > # ELASTICSEARCH ENTRYPOINT
> > > > > >
> > > > > > GET_TEMPLATE:
> > > > > >
> > > > > > - "indices:admin/template/get"
> > > > > >
> > > > > > PUT_TEMPLATE:
> > > > > >
> > > > > > - "indices:admin/template/put"
> > > > > >
> > > > > > TEMPLATE_OWNER:
> > > > > >
> > > > > > - GET_TEMPLATE
> > > > > >
> > > > > > - PUT_TEMPLATE
> > > > > >
> > > > > > ES_INPUT:
> > > > > >
> > > > > > - TEMPLATE_OWNER
> > > > > >
> > > > > > - WRITE
> > > > > >
> > > > > > - MONITOR
> > > > > >
> > > > > >
> > > > > >
> > > > > > # MONITORING
> > > > > >
> > > > > > MONITOR:
> > > > > >
> > > > > > - "cluster:monitor*"
> > > > > >
> > > > > >
> > > > > >
> > > > > > # SUPER POWERS
> > > > > >
> > > > > > ALL:
> > > > > >
> > > > > >
> > > > > > - "*"
> > > > > >
> > > > > >
> > > > > >
> > > > > > Search Guard: "20.1"
> > > > > >
> > > > > > ElasticSearch:6.1.1
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > > > > > To post to this group, send email to search...@googlegroups.com.
> > > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/d04c8a95-74cb-41c7-a24d-baf2e0fcda2d%40googlegroups.com\.
> > > > > > For more options, visit https://groups.google.com/d/optout\.
> > > > >
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > > > > To post to this group, send email to search...@googlegroups.com.
> > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/3ba4a31b-2667-4b82-89e7-c95fcdb98caf%40googlegroups.com\.
> > > > > For more options, visit https://groups.google.com/d/optout\.
> > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> > > > To post to this group, send email to search...@googlegroups.com.
> > > > To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/554d479e-
...

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/65d7cebe-d7b6-4dd7-a4ce-f2408768fbaa%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

Kalada · February 5, 2018, 8:29pm

Please see attached

kibana.yml (4.54 KB)

···

On Monday, February 5, 2018 at 2:38:24 PM UTC-5, Search Guard wrote:

can you post yor kibana.yml?

Am 05.02.2018 um 20:33 schrieb Kalada kalada...@t1cg.com:

Using the curl command I can clearly see the test user I’ve created is authenticating correctly, Please see below

bash-4.4# curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Trying 172.22.0.5…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.5) port 9200 (#0)

GET /_searchguard/authinfo HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 287

<

Connection #0 to host elasticsearch left intact

{“user”:“User [name= USERNAME, roles=[ ROLE], requestedTenant=null]”,“user_name”:" USERNAME “,“user_requested_tenant”:null,“remote_address”:“127.0.0.1:34730”,“backend_roles”:[” ROLE “],“custom_attribute_names”:,“sg_roles”:[” ROLE “],“sg_tenants”:{” ROLE ":true},“principal”:null,“peer_certificates”:“0”}

However, each time the proxy user tries to authenticate from kibana this message appears.

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator

elasticsearch_1 | org.elasticsearch.ElasticsearchSecurityException: xff not done

elasticsearch_1 | at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic http authentication using the internal kibana user. Is this the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

Am 02.02.2018 um 20:16 schrieb Kalada > > > > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > > >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: USERNAME
x-proxy-roles: ROLE
x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 439
<
{
“name” : “NODE-1”,
“cluster_name” : “elasticsearch-default”,
“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,
“version” : {
“number” : “6.1.1”,
“build_hash” : “bd92e7f”,
“build_date” : “2017-12-17T20:23:25.338Z”,
“build_snapshot” : false,
“lucene_version” : “7.1.0”,
“minimum_wire_compatibility_version” : “5.6.0”,
“minimum_index_compatibility_version” : “5.0.0”
},
“tagline” : “You Know, for Search”
}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:
should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(

jkressin · February 6, 2018, 2:45pm

Are you really sure that Kibana sends all required headers to Elasticsearch? Especially x-forwarded-for and x-forwarded-by? Are you running behind a real proxy?

What happens if you add the following to kibana.yml?

elasticsearch.customHeaders: {

"x-forwarded-for": "127.0.0.1",

"x-forwarded-by": "127.0.0.1",

}

···

On Monday, February 5, 2018 at 9:29:38 PM UTC+1, Kalada wrote:

Please see attached

On Monday, February 5, 2018 at 2:38:24 PM UTC-5, Search Guard wrote:

can you post yor kibana.yml?

Am 05.02.2018 um 20:33 schrieb Kalada kalada...@t1cg.com:

Using the curl command I can clearly see the test user I’ve created is authenticating correctly, Please see below

bash-4.4# curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Trying 172.22.0.5…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.5) port 9200 (#0)

GET /_searchguard/authinfo HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 287

<

Connection #0 to host elasticsearch left intact

{“user”:“User [name= USERNAME, roles=[ ROLE], requestedTenant=null]”,“user_name”:" USERNAME “,“user_requested_tenant”:null,“remote_address”:“127.0.0.1:34730”,“backend_roles”:[” ROLE “],“custom_attribute_names”:,“sg_roles”:[” ROLE “],“sg_tenants”:{” ROLE ":true},“principal”:null,“peer_certificates”:“0”}

However, each time the proxy user tries to authenticate from kibana this message appears.

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator

elasticsearch_1 | org.elasticsearch.ElasticsearchSecurityException: xff not done

elasticsearch_1 | at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic http authentication using the internal kibana user. Is this the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

Am 02.02.2018 um 20:16 schrieb Kalada > > > > > > > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > > > >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: USERNAME
x-proxy-roles: ROLE
x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 439
<
{
“name” : “NODE-1”,
“cluster_name” : “elasticsearch-default”,
“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,
“version” : {
“number” : “6.1.1”,
“build_hash” : “bd92e7f”,
“build_date” : “2017-12-17T20:23:25.338Z”,
“build_snapshot” : false,
“lucene_version” : “7.1.0”,
“minimum_wire_compatibility_version” : “5.6.0”,
“minimum_index_compatibility_version” : “5.0.0”
},
“tagline” : “You Know, for Search”
}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:
should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.

Kalada · February 6, 2018, 7:32pm

Hi Jochen,

I added the customHeader field to the kibana.yml file and I now see the searchguard gui and I am able to interact with associated index.

log messages as follows

elasticsearch_1 | [2018-02-06T19:09:59,683][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-06T19:09:59,683][DEBUG][c.f.s.h.HTTPProxyAuthenticator] headers {authorization=[Basic ZWxhc3RpYzpjaGFuZ2VtZQ==], x-proxy-roles=[USER_NAME], x-proxy-user=[USER_NAME], Connection=[keep-alive], Host=[elasticsearch:9200], Content-Length=[80], content-type=[application/json]}

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.h.HTTPProxyAuthenticator] userHeader x-proxy-user, value USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.h.HTTPProxyAuthenticator] rolesHeader x-proxy-roles, value USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=USER_NAME, roles=[USER_NAME], requestedTenant=null]’ is authenticated

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.a.BackendRegistry ] sgtenant ‘null’

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=USER_NAME, roles=[USER_NAME], requestedTenant=null] on NODE-1

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] requested indices:data/read/search from 172.22.0.1:55592

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve [.kibana] from class org.elasticsearch.action.search.SearchRequestfor action indices:data/read/search

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] indicesOptions IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false]

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] 1 raw indices [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] 0 requestTypes

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] No date math indices found

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolved [.kibana] to [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final indices: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final types:

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] final indices: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] final types: [_all]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=.kibana, type=*]]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for USER_NAME: [USER_NAME]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: ‘null’

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Try wildcard match for *

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Wildcard match for *: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for , will check now types []

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action [indices:data/read/search] against /: [*]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] removed .kibana*

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] For index * remaining requested indextype:

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘USER_NAME.*’, evaluate other roles

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers USER_NAME=>

Any other request fails to use any of the headers. Please see below;

elasticsearch_1 | [2018-02-06T19:10:00,962][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] headers {Content-Length=[0], Connection=[keep-alive], Host=[elasticsearch:9200], Authorization=[Basic a2liYW5hOmNoYW5nZW1l]}

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] userHeader x-proxy-user, value null

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] rolesHeader x-proxy-roles, value null

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from basic http authenticator

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=kibana, roles=[kibana_user], requestedTenant=null]’ is authenticated

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] sgtenant ‘null’

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=kibana, roles=[kibana_user], requestedTenant=null] on NODE-1

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] requested cluster:monitor/main from 127.0.0.1:54470

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] class org.elasticsearch.action.main.MainRequest is not an IndicesRequest

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=_all, type=*]]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for kibana: [kibana_user]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: ‘null’

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: kibana_user

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved cluster actions:[indices:data/read/msearch*, indices:data/read/mget*, cluster:monitor*, indices:data/write*, indices:data/read*]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘kibana_user’ and cluster:monitor/main, skip other roles

···

On Tuesday, February 6, 2018 at 9:45:20 AM UTC-5, Jochen Kressin wrote:

Are you really sure that Kibana sends all required headers to Elasticsearch? Especially x-forwarded-for and x-forwarded-by? Are you running behind a real proxy?

What happens if you add the following to kibana.yml?

elasticsearch.customHeaders: {

“x-forwarded-for”: “127.0.0.1”,

“x-forwarded-by”: “127.0.0.1”,

}

On Monday, February 5, 2018 at 9:29:38 PM UTC+1, Kalada wrote:

Please see attached

On Monday, February 5, 2018 at 2:38:24 PM UTC-5, Search Guard wrote:

can you post yor kibana.yml?

Am 05.02.2018 um 20:33 schrieb Kalada kalada...@t1cg.com:

Using the curl command I can clearly see the test user I’ve created is authenticating correctly, Please see below

bash-4.4# curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Trying 172.22.0.5…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.5) port 9200 (#0)

GET /_searchguard/authinfo HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 287

<

Connection #0 to host elasticsearch left intact

{“user”:“User [name= USERNAME, roles=[ ROLE], requestedTenant=null]”,“user_name”:" USERNAME “,“user_requested_tenant”:null,“remote_address”:“127.0.0.1:34730”,“backend_roles”:[” ROLE “],“custom_attribute_names”:,“sg_roles”:[” ROLE “],“sg_tenants”:{” ROLE ":true},“principal”:null,“peer_certificates”:“0”}

However, each time the proxy user tries to authenticate from kibana this message appears.

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator

elasticsearch_1 | org.elasticsearch.ElasticsearchSecurityException: xff not done

elasticsearch_1 | at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic http authentication using the internal kibana user. Is this the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

Am 02.02.2018 um 20:16 schrieb Kalada > > > > > > > > > > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > > > > >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: USERNAME
x-proxy-roles: ROLE
x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 439
<
{
“name” : “NODE-1”,
“cluster_name” : “elasticsearch-default”,
“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,
“version” : {
“number” : “6.1.1”,
“build_hash” : “bd92e7f”,
“build_date” : “2017-12-17T20:23:25.338Z”,
“build_snapshot” : false,
“lucene_version” : “7.1.0”,
“minimum_wire_compatibility_version” : “5.6.0”,
“minimum_index_compatibility_version” : “5.0.0”
},
“tagline” : “You Know, for Search”
}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:
should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.

jkressin · February 6, 2018, 10:08pm

Ok, that means you either do not have a real proxy in front of Kibana, or that the proxy is not adding the x-forwarded-for and/or x-forwarded-by headers, you might want to check that.

Thanks!

Jochen

···

On Tuesday, February 6, 2018 at 8:32:08 PM UTC+1, Kalada wrote:

Hi Jochen,

I added the customHeader field to the kibana.yml file and I now see the searchguard gui and I am able to interact with associated index.

log messages as follows

elasticsearch_1 | [2018-02-06T19:09:59,683][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-06T19:09:59,683][DEBUG][c.f.s.h.HTTPProxyAuthenticator] headers {authorization=[Basic ZWxhc3RpYzpjaGFuZ2VtZQ==], x-proxy-roles=[USER_NAME], x-proxy-user=[USER_NAME], Connection=[keep-alive], Host=[elasticsearch:9200], Content-Length=[80], content-type=[application/json]}

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.h.HTTPProxyAuthenticator] userHeader x-proxy-user, value USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.h.HTTPProxyAuthenticator] rolesHeader x-proxy-roles, value USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=USER_NAME, roles=[USER_NAME], requestedTenant=null]’ is authenticated

elasticsearch_1 | [2018-02-06T19:09:59,684][DEBUG][c.f.s.a.BackendRegistry ] sgtenant ‘null’

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=USER_NAME, roles=[USER_NAME], requestedTenant=null] on NODE-1

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] requested indices:data/read/search from 172.22.0.1:55592

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolve [.kibana] from class org.elasticsearch.action.search.SearchRequestfor action indices:data/read/search

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] indicesOptions IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false]

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] 1 raw indices [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] 0 requestTypes

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] No date math indices found

elasticsearch_1 | [2018-02-06T19:09:59,685][DEBUG][c.f.s.c.PrivilegesEvaluator] Resolved [.kibana] to [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final indices: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] pre final types:

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] final indices: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] final types: [_all]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=.kibana, type=*]]

elasticsearch_1 | [2018-02-06T19:09:59,686][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for USER_NAME: [USER_NAME]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: ‘null’

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: USER_NAME

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Try wildcard match for *

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Wildcard match for *: [.kibana]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] matches for , will check now types []

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] match requested action [indices:data/read/search] against /: [*]

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] removed .kibana*

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] For index * remaining requested indextype:

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘USER_NAME.*’, evaluate other roles

elasticsearch_1 | [2018-02-06T19:09:59,687][DEBUG][c.f.s.c.PrivilegesEvaluator] Added to leftovers USER_NAME=>

Any other request fails to use any of the headers. Please see below;

elasticsearch_1 | [2018-02-06T19:10:00,962][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] headers {Content-Length=[0], Connection=[keep-alive], Host=[elasticsearch:9200], Authorization=[Basic a2liYW5hOmNoYW5nZW1l]}

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] userHeader x-proxy-user, value null

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.h.HTTPProxyAuthenticator] rolesHeader x-proxy-roles, value null

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from basic http authenticator

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] User ‘User [name=kibana, roles=[kibana_user], requestedTenant=null]’ is authenticated

elasticsearch_1 | [2018-02-06T19:10:00,970][DEBUG][c.f.s.a.BackendRegistry ] sgtenant ‘null’

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] ### evaluate permissions for User [name=kibana, roles=[kibana_user], requestedTenant=null] on NODE-1

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] requested cluster:monitor/main from 127.0.0.1:54470

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] class org.elasticsearch.action.main.MainRequest is not an IndicesRequest

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] requested resolved indextypes: [IndexType [index=_all, type=*]]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] mapped roles for kibana: [kibana_user]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesInterceptorImpl] raw requestedTenant: ‘null’

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] Result from privileges interceptor: null

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] ---------- evaluate sg_role: kibana_user

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] resolved cluster actions:[indices:data/read/msearch*, indices:data/read/mget*, cluster:monitor*, indices:data/write*, indices:data/read*]

elasticsearch_1 | [2018-02-06T19:10:00,971][DEBUG][c.f.s.c.PrivilegesEvaluator] found a match for ‘kibana_user’ and cluster:monitor/main, skip other roles

On Tuesday, February 6, 2018 at 9:45:20 AM UTC-5, Jochen Kressin wrote:

Are you really sure that Kibana sends all required headers to Elasticsearch? Especially x-forwarded-for and x-forwarded-by? Are you running behind a real proxy?

What happens if you add the following to kibana.yml?

elasticsearch.customHeaders: {

"x-forwarded-for": "127.0.0.1",

"x-forwarded-by": "127.0.0.1",

}

On Monday, February 5, 2018 at 9:29:38 PM UTC+1, Kalada wrote:

Please see attached

On Monday, February 5, 2018 at 2:38:24 PM UTC-5, Search Guard wrote:

can you post yor kibana.yml?

Am 05.02.2018 um 20:33 schrieb Kalada kalada...@t1cg.com:

Using the curl command I can clearly see the test user I’ve created is authenticating correctly, Please see below

bash-4.4# curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Trying 172.22.0.5…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.5) port 9200 (#0)

GET /_searchguard/authinfo HTTP/1.1

Host: elasticsearch:9200

User-Agent: curl/7.57.0

Accept: /

x-proxy-user: USERNAME

x-proxy-roles: ROLE

x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK

< content-type: application/json; charset=UTF-8

< content-length: 287

<

Connection #0 to host elasticsearch left intact

{“user”:“User [name= USERNAME, roles=[ ROLE], requestedTenant=null]”,“user_name”:" USERNAME “,“user_requested_tenant”:null,“remote_address”:“127.0.0.1:34730”,“backend_roles”:[” ROLE “],“custom_attribute_names”:,“sg_roles”:[” ROLE “],“sg_tenants”:{” ROLE ":true},“principal”:null,“peer_certificates”:“0”}

However, each time the proxy user tries to authenticate from kibana this message appears.

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator

elasticsearch_1 | [2018-02-05T19:14:07,991][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator

elasticsearch_1 | org.elasticsearch.ElasticsearchSecurityException: xff not done

elasticsearch_1 | at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

elasticsearch_1 | at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

And then it falls back to basic http authentication using the internal kibana user. Is this the expect behavior as I can curl successfully with the same headers? Can you send an example configure for a proxied user?

On Saturday, February 3, 2018 at 4:08:58 PM UTC-5, Search Guard wrote:

This seems more like a misconfiguration of the users and theirs roles and is unrelated to the proxy problem i guess

To see which roles a user have login with that user and call the _searchguard/authinfo endpoint.

Example:
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -H “x-proxy-user: myuser” -H “x-proxy-roles: xxx” -H “x-forwarded-for: 127.0.0.1”
curl -sS http://elasticsearch:9200/_searchguard/authinfo -v -u kibana:kibana

Am 02.02.2018 um 20:16 schrieb Kalada > > > > > > > > > > > > > > > > With suggested changes the curl command works correctly and it recognizes the proxy user. I am still unable to extend this to the kibana ui. Please see logs > > > > > >
bash-4.4# curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: USERNAME” -H “x-proxy-roles: ROLE” -H “x-forwarded-for: 127.0.0.1”

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.2…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.2) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: USERNAME
x-proxy-roles: ROLE
x-forwarded-for: 127.0.0.1

< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 439
<
{
“name” : “NODE-1”,
“cluster_name” : “elasticsearch-default”,
“cluster_uuid” : “JeICJqFVSXO1jxzgdKSxMg”,
“version” : {
“number” : “6.1.1”,
“build_hash” : “bd92e7f”,
“build_date” : “2017-12-17T20:23:25.338Z”,
“build_snapshot” : false,
“lucene_version” : “7.1.0”,
“minimum_wire_compatibility_version” : “5.6.0”,
“minimum_index_compatibility_version” : “5.0.0”
},
“tagline” : “You Know, for Search”
}

Connection #0 to host elasticsearch left intact

On Friday, February 2, 2018 at 1:16:09 PM UTC-5, Search Guard wrote:
should be something like this (the proxy authenticator was disabled)

searchguard:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: true
internalProxies: ‘.’
remoteIpHeader: ‘x-forwarded-for’
proxiesHeader: ‘x-forwarded-by’
trustedProxies: '.’
authc:
proxy_auth_domain:
http_enabled: true
transport_enabled: false
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: “x-proxy-user”
roles_header: “x-proxy-roles”
authentication_backend:
type: noop
basic_internal_auth:
enabled: true
order: 1
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: internal

Am 02.02.2018 um 16:18 schrieb Kalada kalada...@t1cg.com:

Please see the attached

On Thursday, February 1, 2018 at 5:37:13 PM UTC-5, Search Guard wrote:
pls provide your sg_config.yml, elasticsearch.yml and the logs (as files/attachments)

Am 01.02.2018 um 23:04 schrieb Kalada kalada...@t1cg.com:

Thank you https wasnt enabled please see below

Rebuilt URL to: http://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

GET / HTTP/1.1
Host: elasticsearch:9200
User-Agent: curl/7.57.0
Accept: /
x-proxy-user: myuser
x-proxy-roles: myrole
x-forwarded-for: 127.0.0.1

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=“Search Guard”
< content-type: text/plain; charset=UTF-8
< content-length: 12
<

Connection #0 to host elasticsearch left intact

On Thursday, February 1, 2018 at 4:41:37 PM UTC-5, Search Guard wrote:
Have you https enabled? If not its
curl -sS http://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

otherwise you curl library seems messed up. Make sure its compiled against a recent openssl version.
curl --version

Am 01.02.2018 um 22:27 schrieb Kalada kalada...@t1cg.com:

This is the messages I received

Rebuilt URL to: https://elasticsearch:9200/

Trying 172.22.0.3…

TCP_NODELAY set

Connected to elasticsearch (172.22.0.3) port 9200 (#0)

ALPN, offering http/1.1

successfully set certificate verify locations:

CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none

TLSv1.2 (OUT), TLS handshake, Client hello (1):

error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

On Thursday, February 1, 2018 at 3:42:22 PM UTC-5, Search Guard wrote:
Can you test it via curl or postman (without kibana) like

curl -sSk https://elasticsearch:9200 -v -H “x-proxy-user: myuser” -H “x-proxy-roles: myrole” -H “x-forwarded-for: 127.0.0.1”

Am 31.01.2018 um 22:36 schrieb Kalada kalada...@t1cg.com:

internalProxies: ‘.*’ is the current setting. The headers appear to be valid please see an example below

kibana_1 | {“type”:“response”,“@timestamp”:“2018-01-31T20:14:37Z”,“tags”:,“pid”:21,“method”:“post”,“statusCode”:200,
“req”:{“url”:“/elasticsearch/_msearch”,“method”:“post”,“headers”:{“es-security-runas-user”:“USERNAME”,“x-proxy-user”:“USENAME”,“host”:
“kibana:5601”,“connection”:“close”,“content-length”:“732”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ADDRESS Safari/537.36”,“accept”:"application/json, text/plain, /
",“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”,“content-type”:“application/x-ndjson”,
“kbn-version”:“6.1.1”,“origin”:“http://localhost:4180”,“referer”:“http://localhost:4180/app/kibana”,
“x-forwarded-email”:“EMAIL”,“x-forwarded-for”:“172.19.0.1”,“x-forwarded-user”:“USERNAME”},
“remoteAddress”:“172.19.0.3”,“userAgent”:“172.19.0.3”,“referer”:“http://localhost:4180/app/kibana”},
“res”:{“statusCode”:200,“responseTime”:18,“contentLength”:9},“message”:“POST /elasticsearch/_msearch 200 18ms - 9.0B”}

On Wednesday, January 31, 2018 at 4:05:14 PM UTC-5, Search Guard wrote:
“xff not done” means typically that non of your configured internal proxies match and/or the X-Forwarded-For header is not present

you can test ist by allowing simply all internal proxies

internalProxies: ‘.*’

and make sure your request contains a valid X-Forwarded-For header like described here https://en.wikipedia.org/wiki/X-Forwarded-For

Am 31.01.2018 um 19:35 schrieb Kalada kalada...@t1cg.com:

Hi Jochen,

Playing with the setting I noticed this message now

[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] Try to extract auth creds from proxy http authenticator
[2018-01-31T18:31:55,847][DEBUG][c.f.s.a.BackendRegistry ] ‘ElasticsearchSecurityException[xff not done]’ extracting credentials from proxy http authenticator
org.elasticsearch.ElasticsearchSecurityException: xff not done
at com.floragunn.searchguard.http.HTTPProxyAuthenticator.extractCredentials(HTTPProxyAuthenticator.java:49) ~[search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:380) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.checkAndAuthenticateRequest(SearchGuardRestFilter.java:122) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter.access$000(SearchGuardRestFilter.java:47) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:74) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-client-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.

Topic		Replies	Views
SG behind a Proxy Search Guard	2	424	September 18, 2018
Use the backend roles in the correct way Search Guard	10	1667	August 18, 2020
How to create user with built-in roles? Search Guard	5	744	September 14, 2020
[Proxy Authentication] Issue in getting headers from SSO Session results in Kibana not loading correctly (bootstrap.js unauthorized error) Search Guard	8	745	February 11, 2021
How proxy authentication works in searchguard security plugin? Search Guard	6	524	August 19, 2021

Problems with proxy users/roles

Related Topics