Proxy authentication (Apache and Kibana)

[Worapoj_Chokeanankun]

Hi Search Guard team,

I’m using Apache as a proxy to redirect to Kibana.
Apache is not installed on the same host as Kibana.

Elasticsearch version: 6.6
Search Guard version: 6.x-24 sg_config.yml (9.6 KB)
Kibana: 6.6 kibana.yml (5.5 KB)

I tried to setup in Apache but it doesn’t work.
With this Apache config, I got HTTP 500 from authinfo

<Proxy balancer://kibana_cluster>
  BalancerMember http://10.49.113.25:5601
  BalancerMember http://10.49.114.42:5601
  BalancerMember http://10.49.116.194:5601
</Proxy>

ProxyPass /api/v1/auth/authinfo balancer://kibana_cluster/api/v1/auth/authinfo
ProxyPassReverse /api/v1/auth/authinfo balancer://kibana_cluster/api/v1/auth/authinfo
ProxyPass /bundles balancer://kibana_cluster/bundles
ProxyPass /dlls balancer://kibana_cluster/dlls
ProxyPass /monitor/api/console balancer://kibana_cluster/api/console
ProxyPass /plugins balancer://kibana_cluster/plugins
ProxyPass /ui balancer://kibana_cluster/ui
ProxyPass /api/xpack balancer://kibana_cluster/api/xpack

<Location /monitor/events_aws/>
  RequestHeader set "Authorization" "Basic ODAxNTk5NjpXZWxjb20zKw=="
  RequestHeader set "X-Forwarded-For" "10.51.2.116"
  RequestHeader set "x-proxy-user" "8015996"

  ProxyPass balancer://kibana_cluster/app/kibana
  ProxyPassReverse balancer://kibana_cluster/app/kibana
</Location>

With this Apache config, I got HTTP 404.

<Proxy balancer://kibana_cluster>
  BalancerMember http://10.49.113.25:5601
  BalancerMember http://10.49.114.42:5601
  BalancerMember http://10.49.116.194:5601
</Proxy>

<Location /monitor/events_aws/>
  RequestHeader set "Authorization" "Basic ODAxNTk5NjpXZWxjb20zKw=="
  RequestHeader set "X-Forwarded-For" "10.51.2.116"
  RequestHeader set "x-proxy-user" "8015996"

  ProxyPass balancer://kibana_cluster/
  ProxyPassReverse balancer://kibana_cluster/
</Location>

Did I configure something wrong?

By the way, do we need to set Authorization header for the proxy authentication?

Regards,
Worapoj

[Worapoj_Chokeanankun]

Hi @pablo.lescotti, May I ask the update?

[jkressin]

I have a couple of questions regarding your setup, in order to validate / correct the configuration:

What is the purpose of putting a proxy in front of Kibana? Is it

• to load balance between the Kibana instances?
• to authenticate users and then load balance between the Kibana instances?

The question is where authentication should happen. In the former case you don’t need to add authentication headers, since auth is handled by Search Guard.

In the latter case you can either add a HTTP Basic Authentication header (as in your example). Or, add user name and user roles directly to the HTTP requests. Both setups vary in their respective configuration.

Next, in your kibana.yml, you define the ES hosts with HTTPS and HTTP simultaneously with different hosts. Is this intentional?

elasticsearch.hosts:
  - https://10.49.116.202:9200
  - https://10.49.112.126:9200
  - https://10.49.114.82:9200

xpack.monitoring.elasticsearch.hosts:
  - http://10.49.116.69:9200
  - http://10.49.113.148:9200
  - http://10.49.115.206:9200

Also, in the sg_config.yml you configure three auth domains:

• Basic Auth
• Proxy Auth
• Client cert Auth

Is this intentional, means do you plan to use all three?

[Worapoj_Chokeanankun]

Thank you for you reply.

The purpose of putting a proxy in front of Kibana is I already have my authentication service.
It is used to by-pass the authentication because I’m not using Enterprise License (It is in review stage to buy the license).

When user access a known URL, ex logging.mycompany.com (Apache), it will redirect to authentication service URL.
After the user logged in the authentication URL will redirect to logging.mycompany.com with the user information such as User ID, First Name, Last Name or etc.
Before this step, I configured the user name, role and permission in Search Guard.

So, for your questions.

• Is it to load balance between the Kibana instances?
  Actually, the Kibana instances are behind AWS Load Balancer and Routes 53 but there is the network connectivity design. I cannot point the DNSs directly just works only on IP. The Apache server is on-premise infrastructure.
• Is it to authenticate users and then load balance between the Kibana instances?
  Yes.

It is great if I don’t need to add authentication headers. I prefer this solution. Please suggest and correct the configuration.

For the Kibana configuration, elasticsearch.hosts is my ES data cluster and xpack.monitoring.elasticsearch.hosts is my monitoring ES data cluster. So, I can view the log data (Discover) and the monitor data (Monitoring tab).

For the three auth domains, I think I only need Basic Auth and Proxy Auth for my requirements.

[jkressin]

Ok, thanks, that clarifies a lot. So to summarize from a Kibana perspective:

If a request hits Kibana it is always via the proxy. The proxy takes care if the authentication, and before you forward the request from the proxy to Kibana (and then ultimately Elasticsearch), you are able to add the username and the user’s roles as HTTP headers.

If this is correct then you want to configure proxy authentication:

Proxy

In your case, you just need the proxy auth, not the proxycache auth. So in kibana.yml, set:

searchguard.auth.type: "proxy"

And make sure the HTTP headers the proxy sets are whitelisted. This seems to be already the case with the kibana.yml you attached:

elasticsearch.requestHeadersWhitelist:
  - authorization
  - sgtenant
  - x-proxy-user
  - x-proxy-roles

In sg_config, enable proxy auth and set the HTTP header names. This is nearly correct in your sg_config, but you set the proxy auth module to “enabled: false”:

  proxy_auth_domain:
    http_enabled: false <---
    transport_enabled: false <---
    order: 3
    http_authenticator:
      type: proxy
      challenge: false
      config:
        user_header: "x-proxy-user"
        roles_header: "x-proxy-roles"
    authentication_backend:
      type: noop

You need to enable at least the http layer.

The last part is to enable the proxy resolution in sg_config. In order to prevent users from spoofing the HTTP headers fir the username and the roles, you need to tell Search Guard which proxy IP(s) we can trust:

http:
  xff:
    enabled: true
    internalProxies: '192\.168\.0\.10|192\.168\.0\.11'

In the configuration above, proxy auth is only accepted from the two IPs configured. In your case, this is the Kibana IP.

You can also use wildcards and regular expressions in for the internalProxies. So when testing proxy auth, you can at first just use a wildcard like:

internalProxies: '.*'

If the setup works, you can then add the real Kibana IPs. (This is of course just a tip, but it makes a step-by-step setup easier).

[Worapoj_Chokeanankun]

Thanks. I will try this tomorrow.

By the way, can I put multiple roles in x-proxy-roles header?

[jkressin]

yes, just comma-separate them.

[Worapoj_Chokeanankun]

Hi, I configured with your recommendation. It does not work.

It always returns HTTP 401 and prompts for the username and the password.

<Proxy balancer://kibana_cluster>
  BalancerMember http://10.49.113.25:5601
  BalancerMember http://10.49.114.42:5601
  BalancerMember http://10.49.116.194:5601
</Proxy>

<Location /monitor/events_aws/>
  RequestHeader set "x-proxy-user" "admin"
  RequestHeader set "x-proxy-roles" "admin"

  ProxyPass balancer://kibana_cluster/app/kibana
  ProxyPassReverse balancer://kibana_cluster/app/kibana
</Location>

kibana.yml (5.4 KB)
sg_config.yaml (4.2 KB)

[jkressin]

Ok, let’s pinpoint the error. In your sg_config.yml I see that you have enabled a client cert domain as first authenticator, and you have the “challenge” flag set to true:

  clientcert_auth_domain:
    http_enabled: true
    transport_enabled: false
    order: 2
    http_authenticator:
      type: clientcert
      config:
        username_attribute: cn #optional, if omitted DN becomes username
      challenge: true <--
    authentication_backend:
      type: noop

Please try to set it to false or disable the client cert domain completely for the moment. Upload the config with sgadmin.

Next, please try if proxy auth in Elasticsearch (without Kibana or Apache) works. Please access Elasticsearch directly, simulating a proxy request:

curl -k -H "x-forwarded-for: 127.0.0.1" -H "x-proxy-user: admin" -H "x-proxy-roles: admin" -XGET "https://l<elastic host>:9200/_searchguard/authinfo?pretty"

This should authenticate the admin user and print some user information in JSON.

If this does not work, please set the log level to trace:

Search Guard logging

curl -u admin:admin --insecure -X PUT "https://<es host>:9200/_cluster/settings" -H 'Content-Type: application/json' -d '{
  "transient": {
    "logger.com.floragunn": "trace"
  }
}'

Then perform the curl proxy request again and post the output in the ES log file.

[jkressin]

Another observation: I am not an Apache proxy expert, but in one of your configs you set:

RequestHeader set "X-Forwarded-For" "10.51.2.116"

The proxy auth module expects an X-Forwarded-For to be present in the HTTP call. In the config you posted last, there is no directive regarding “X-Forwarded-For”. Does Apache set it automatically?

[Worapoj_Chokeanankun]

Thanks. I already disabled the client cert authenticator and I can get the JSON format.

I think the problem should not be Search Guard for now. It could be my Apache configuration.

By the way, after I configured searchguard.auth.type: proxy in Kibana, I cannot access Kibana through AWS Load Balancer DNS. I got HTTP 500 in /api/v1/auth/authinfo request. Is this normal behavior?

[Worapoj_Chokeanankun]

I can access Kibana through AWS LB DNS when I removed searchguard.auth.type: proxy.

If I configured the auth type configuration in Kibana, I have to access through the proxy server only right?