Problems using HTTP-header/Proxy based authentication

Search Guard
1

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version

  • Installed and used enterprise modules, if any

  • JVM version and operating system version

  • Search Guard configuration files

  • Elasticsearch log messages on debug level

  • Other installed Elasticsearch or Kibana plugins, if any

Hi,

I have search-guard installed and this works using the basic_internal_auth_domain in sg_config.yml.

I want to add an SSO proxy for authentication purposes and followed the steps in

However, I cannot get this to work successful. My configuration is detailed below:

Note that

  1. When I try and authenticate using an existing user using basic_internal_auth_domain it is still successful.
    E.g. curl http://user:user_pw@localhost:9200
    {

“name” : “jCKtPWs”,

“cluster_name” : “docker-cluster”,

“cluster_uuid” : “nnhagc3hSiSaQYyz8p9Hgg”,

“version” : {

“number” : “6.1.1”,

“build_hash” : “bd92e7f”,

“build_date” : “2017-12-17T20:23:25.338Z”,

“build_snapshot” : false,

“lucene_version” : “7.1.0”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}

  1. When I try and authenticate a new user using proxy_auth_domain it fails

E.g. curl http://newuser:newuser_pw@localhost:9200
Unauthorized

  1. When I look in the logs I just get this

…

[WARN ][c.f.s.a.BackendRegistry ] Authentication finally failed for /newuser

…

  1. I can communicate directly with the proxy successfully for the newuser.

E.g.

curl -XPOST -v “http://newuser:newuser_pw@127.0.0.1”

this returns…

  • About to connect() to 127.0.0.1 port 80 (#0)

  • Trying 127.0.0.1…

  • Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)

  • Server auth using Basic with user ‘*********’

POST / HTTP/1.1

Authorization: Basic d2ZfYWRtaW46YWRtaW5fbmc=

User-Agent: curl/7.29.0

Host: 127.0.0.1

Accept: /

  • HTTP 1.0, assume close after body

< HTTP/1.0 200 OK

< Server: BaseHTTP/0.3 Python/2.7.5

< Date: Mon, 30 Apr 2018 14:42:47 GMT

< x-proxy-user: *********

<

  • Closing connection 0
  1. The proxy returns only x-proxy-user and not x-proxy-roles however I have set-up the roles for the new user in sg_roles_mapping.yml so I hope this is not a problem.

I hope someone has some idea what the problems might be? It would be useful to have an examples of a working version sg_config.yml that uses proxy_auth_domain.

Configuration Details

Configuration details are as follows

  • Search Guard and Elasticsearch version
    Search Guard 6 ( I can see this file search-guard-6-6.1.1-20.1.jar in /plugins/search-guard-6 )
    ElasticSearch version 6.1.1

  • Installed and used enterprise modules, if any
    I am using Elasticsearch, Logstash, Kibana and Searchguard but nothing else

  • JVM version and operating system version
    java -version

openjdk version “1.8.0_151”

OpenJDK Runtime Environment (build 1.8.0_151-b12)

OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

CentOS Linux 7 (Core)

  • Search Guard configuration files

see sg_config.yml

  • Elasticsearch log messages on debug level

I added the following to config/log4j2.properties

logger.searchguard.name = com.floragunn

logger.searchguard.level = debug

  • Other installed Elasticsearch or Kibana plugins, if any

no others

Thanks

Garin

sg_config.yml (747 Bytes)

2

pls also post your proxy configuration

···

On Monday, 30 April 2018 16:01:53 UTC+2, Garin Smith wrote:

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

Hi,

I have search-guard installed and this works using the basic_internal_auth_domain in sg_config.yml.

I want to add an SSO proxy for authentication purposes and followed the steps in
https://docs.search-guard.com/latest/proxy-authentication
https://docs.search-guard.com/latest/search-guard-proxy

However, I cannot get this to work successful. My configuration is detailed below:

Note that

{

“name” : “jCKtPWs”,

“cluster_name” : “docker-cluster”,

“cluster_uuid” : “nnhagc3hSiSaQYyz8p9Hgg”,

“version” : {

“number” : “6.1.1”,

“build_hash” : “bd92e7f”,

“build_date” : “2017-12-17T20:23:25.338Z”,

“build_snapshot” : false,

“lucene_version” : “7.1.0”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}

  1. When I try and authenticate using an existing user using basic_internal_auth_domain it is still successful.
    E.g. curl http://user:user_pw@localhost:9200
  1. When I try and authenticate a new user using proxy_auth_domain it fails

Unauthorized

E.g. curl http://newuser:newuser_pw@localhost:9200

  1. When I look in the logs I just get this

…

[WARN ][c.f.s.a.BackendRegistry ] Authentication finally failed for /newuser

…

  1. I can communicate directly with the proxy successfully for the newuser.

E.g.

curl -XPOST -v “http://newuser:newuser_pw@127.0.0.1”

this returns…

  • About to connect() to 127.0.0.1 port 80 (#0)
  • Trying 127.0.0.1…
  • Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
  • Server auth using Basic with user ‘*********’

POST / HTTP/1.1

Authorization: Basic d2ZfYWRtaW46YWRtaW5fbmc=

User-Agent: curl/7.29.0

Host: 127.0.0.1

Accept: /

  • HTTP 1.0, assume close after body

< HTTP/1.0 200 OK

< Server: BaseHTTP/0.3 Python/2.7.5

< Date: Mon, 30 Apr 2018 14:42:47 GMT

< x-proxy-user: *********

<

  • Closing connection 0
  1. The proxy returns only x-proxy-user and not x-proxy-roles however I have set-up the roles for the new user in sg_roles_mapping.yml so I hope this is not a problem.

I hope someone has some idea what the problems might be? It would be useful to have an examples of a working version sg_config.yml that uses proxy_auth_domain.

Configuration Details

Configuration details are as follows

  • Search Guard and Elasticsearch version
    Search Guard 6 ( I can see this file search-guard-6-6.1.1-20.1.jar in /plugins/search-guard-6 )
    ElasticSearch version 6.1.1
  • Installed and used enterprise modules, if any
    I am using Elasticsearch, Logstash, Kibana and Searchguard but nothing else
  • JVM version and operating system version
    java -version

openjdk version “1.8.0_151”

OpenJDK Runtime Environment (build 1.8.0_151-b12)

OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

CentOS Linux 7 (Core)

  • Search Guard configuration files

see sg_config.yml

  • Elasticsearch log messages on debug level

I added the following to config/log4j2.properties

logger.searchguard.name = com.floragunn

logger.searchguard.level = debug

  • Other installed Elasticsearch or Kibana plugins, if any

no others

Thanks

Garin

3

Hi,
I’m sorry, but I am not sure what proxy configuration file you mean?

I have submitted the sg_config.yml which I think defines how Search Guard is configured to use the proxy.

···

On Tuesday, 1 May 2018 00:54:34 UTC+1, Search Guard wrote:

pls also post your proxy configuration

On Monday, 30 April 2018 16:01:53 UTC+2, Garin Smith wrote:

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

Hi,

I have search-guard installed and this works using the basic_internal_auth_domain in sg_config.yml.

I want to add an SSO proxy for authentication purposes and followed the steps in
https://docs.search-guard.com/latest/proxy-authentication
https://docs.search-guard.com/latest/search-guard-proxy

However, I cannot get this to work successful. My configuration is detailed below:

Note that

{

“name” : “jCKtPWs”,

“cluster_name” : “docker-cluster”,

“cluster_uuid” : “nnhagc3hSiSaQYyz8p9Hgg”,

“version” : {

“number” : “6.1.1”,

“build_hash” : “bd92e7f”,

“build_date” : “2017-12-17T20:23:25.338Z”,

“build_snapshot” : false,

“lucene_version” : “7.1.0”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}

  1. When I try and authenticate using an existing user using basic_internal_auth_domain it is still successful.
    E.g. curl http://user:user_pw@localhost:9200
  1. When I try and authenticate a new user using proxy_auth_domain it fails

Unauthorized

E.g. curl http://newuser:newuser_pw@localhost:9200

  1. When I look in the logs I just get this

…

[WARN ][c.f.s.a.BackendRegistry ] Authentication finally failed for /newuser

…

  1. I can communicate directly with the proxy successfully for the newuser.

E.g.

curl -XPOST -v “http://newuser:newuser_pw@127.0.0.1”

this returns…

  • About to connect() to 127.0.0.1 port 80 (#0)
  • Trying 127.0.0.1…
  • Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
  • Server auth using Basic with user ‘*********’

POST / HTTP/1.1

Authorization: Basic d2ZfYWRtaW46YWRtaW5fbmc=

User-Agent: curl/7.29.0

Host: 127.0.0.1

Accept: /

  • HTTP 1.0, assume close after body

< HTTP/1.0 200 OK

< Server: BaseHTTP/0.3 Python/2.7.5

< Date: Mon, 30 Apr 2018 14:42:47 GMT

< x-proxy-user: *********

<

  • Closing connection 0
  1. The proxy returns only x-proxy-user and not x-proxy-roles however I have set-up the roles for the new user in sg_roles_mapping.yml so I hope this is not a problem.

I hope someone has some idea what the problems might be? It would be useful to have an examples of a working version sg_config.yml that uses proxy_auth_domain.

Configuration Details

Configuration details are as follows

  • Search Guard and Elasticsearch version
    Search Guard 6 ( I can see this file search-guard-6-6.1.1-20.1.jar in /plugins/search-guard-6 )
    ElasticSearch version 6.1.1
  • Installed and used enterprise modules, if any
    I am using Elasticsearch, Logstash, Kibana and Searchguard but nothing else
  • JVM version and operating system version
    java -version

openjdk version “1.8.0_151”

OpenJDK Runtime Environment (build 1.8.0_151-b12)

OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

CentOS Linux 7 (Core)

  • Search Guard configuration files

see sg_config.yml

  • Elasticsearch log messages on debug level

I added the following to config/log4j2.properties

logger.searchguard.name = com.floragunn

logger.searchguard.level = debug

  • Other installed Elasticsearch or Kibana plugins, if any

no others

Thanks

Garin

4

Here’s a working minimal example with nginx.

nginx configuration:

events {

worker_connections 1024;

}

http {

upstream elasticsearch {

server sgssl-0.example.com:9200;

server sgssl-1.example.com:9200;

server sgssl-2.example.com:9200;

keepalive 15;

}

server {

listen 8090;

server_name nginx.example.com;

location / {

proxy_pass https://elasticsearch;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header x-proxy-user admin;

proxy_set_header x-proxy-roles admin;

}

}

}

``

This is an nginx proxy in front of a 3 node cluster. For demo purposes we just set the user and the role hardcoded here, but you get the idea. You can also omit the roles and add the username only, given you mapped the user to an SG role in the roles mapping.

Corresponding sg_config:

searchguard:

dynamic:

http:

anonymous_auth_enabled: false

xff:

enabled: true

internalProxies: ‘172.16.0.203’ # nginx proxy

authc:

proxy_auth_domain:

http_enabled: true

transport_enabled: true

order: 0

http_authenticator:

type: proxy

challenge: false

config:

user_header: “x-proxy-user”

roles_header: “x-proxy-roles”

authentication_backend:

type: noop

basic_internal_auth_domain:

http_enabled: true

transport_enabled: true

order: 1

http_authenticator:

type: basic

challenge: true

authentication_backend:

type: intern

``

We use two authenticators here, proxy first, then basic auth with the internal user database. The important part is this one:

xff:

enabled: true

internalProxies: ‘172.16.0.203’ # nginx proxy

``

You need to enable the X-Forwarded-For resolution, and set the IP(s) of the trusted/internal proxies correctly. In this example, nginx.example.com is listening on 172.16.0.203. Search Guard will only accept/trust the proxy if it’s IP is listed.

For debugging, you can set the package com.floragunn.searchguard.http**to debug. This will print some logs regarding XFF. Setting the loglevel is described here:

···

On Tuesday, May 1, 2018 at 4:06:47 PM UTC+2, garin.smith@deimos-space.com wrote:

Hi,
I’m sorry, but I am not sure what proxy configuration file you mean?

I have submitted the sg_config.yml which I think defines how Search Guard is configured to use the proxy.

On Tuesday, 1 May 2018 00:54:34 UTC+1, Search Guard wrote:

pls also post your proxy configuration

On Monday, 30 April 2018 16:01:53 UTC+2, Garin Smith wrote:

When asking questions, please provide the following information:

  • Search Guard and Elasticsearch version
  • Installed and used enterprise modules, if any
  • JVM version and operating system version
  • Search Guard configuration files
  • Elasticsearch log messages on debug level
  • Other installed Elasticsearch or Kibana plugins, if any

Hi,

I have search-guard installed and this works using the basic_internal_auth_domain in sg_config.yml.

I want to add an SSO proxy for authentication purposes and followed the steps in
https://docs.search-guard.com/latest/proxy-authentication
https://docs.search-guard.com/latest/search-guard-proxy

However, I cannot get this to work successful. My configuration is detailed below:

Note that

{

“name” : “jCKtPWs”,

“cluster_name” : “docker-cluster”,

“cluster_uuid” : “nnhagc3hSiSaQYyz8p9Hgg”,

“version” : {

“number” : “6.1.1”,

“build_hash” : “bd92e7f”,

“build_date” : “2017-12-17T20:23:25.338Z”,

“build_snapshot” : false,

“lucene_version” : “7.1.0”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}

  1. When I try and authenticate using an existing user using basic_internal_auth_domain it is still successful.
    E.g. curl http://user:user_pw@localhost:9200
  1. When I try and authenticate a new user using proxy_auth_domain it fails

Unauthorized

E.g. curl http://newuser:newuser_pw@localhost:9200

  1. When I look in the logs I just get this

…

[WARN ][c.f.s.a.BackendRegistry ] Authentication finally failed for /newuser

…

  1. I can communicate directly with the proxy successfully for the newuser.

E.g.

curl -XPOST -v “http://newuser:newuser_pw@127.0.0.1”

this returns…

  • About to connect() to 127.0.0.1 port 80 (#0)
  • Trying 127.0.0.1…
  • Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
  • Server auth using Basic with user ‘*********’

POST / HTTP/1.1

Authorization: Basic d2ZfYWRtaW46YWRtaW5fbmc=

User-Agent: curl/7.29.0

Host: 127.0.0.1

Accept: /

  • HTTP 1.0, assume close after body

< HTTP/1.0 200 OK

< Server: BaseHTTP/0.3 Python/2.7.5

< Date: Mon, 30 Apr 2018 14:42:47 GMT

< x-proxy-user: *********

<

  • Closing connection 0
  1. The proxy returns only x-proxy-user and not x-proxy-roles however I have set-up the roles for the new user in sg_roles_mapping.yml so I hope this is not a problem.

I hope someone has some idea what the problems might be? It would be useful to have an examples of a working version sg_config.yml that uses proxy_auth_domain.

Configuration Details

Configuration details are as follows

  • Search Guard and Elasticsearch version
    Search Guard 6 ( I can see this file search-guard-6-6.1.1-20.1.jar in /plugins/search-guard-6 )
    ElasticSearch version 6.1.1
  • Installed and used enterprise modules, if any
    I am using Elasticsearch, Logstash, Kibana and Searchguard but nothing else
  • JVM version and operating system version
    java -version

openjdk version “1.8.0_151”

OpenJDK Runtime Environment (build 1.8.0_151-b12)

OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

CentOS Linux 7 (Core)

  • Search Guard configuration files

see sg_config.yml

  • Elasticsearch log messages on debug level

I added the following to config/log4j2.properties

logger.searchguard.name = com.floragunn

logger.searchguard.level = debug

  • Other installed Elasticsearch or Kibana plugins, if any

no others

Thanks

Garin