Integrating Search Guard /w my project - expanding demo

[NetwarSystem]

I have developed a system for monitoring social media traffic on Twitter, Telegram, etc, using Elasticsearch as the foundation. I made the code public a couple weeks ago: Installing Netwar System.

I found the Search Guard security model to be very challenging - looks great for enterprise architects who have an authentication platform they need to support, but for a small team oriented project, it’s a maze. Yesterday I wrote Installing Search Guard, which I hope makes the demo install a bit more digestible to someone with a unix admin background.

The next steps as far as articles with this should probably be:

1. Set up some users as my specific use case requires.
2. Walk through using self signed Certificate Authority behind Cloudflare CDN
3. Show how to get proxy access working for dashboard embedding.

I’m comfortable with #1 and #2, already have that in operation, but #3 has me a bit puzzled. There is no need to provide access to Elasticsearch itself, I just want to make Kibana available in iframes. Our layout is as follows:

1. Cloudflare CDN responsible for our domain, terminates SSL for us, self-signed certs are fine.

2. Apache reverse proxy, used for user/pass authentication when we had plain ELK setup, still present, just transparently shifting 80/443 to 5601 on Kibana machine.

3. Kibana machine is an eight core/eight gig VPS, all it does is Kibana.

4. Elasticsearch port 9200 arrives on the Kibana machine via an ssh tunnel. Getting my fancy back end equipment into my fancy datacenter would add another zero to monthly costs, which is not going to happen.

I’ve been looking at the Google Group, posts like this one, seems like it should be simple, but I’d feel a lot better if there were a Github repo that contained a commented Kibana.yml and a companion config file for the proxy.

I’m not married to Apache2 in this role, someone else set it up, an nginx config /w companion kibana.yml would be great.

The Apache proxy is there because it was doing htpasswd stuff, I could quarter the proxy service on the machine running Kibana, or directly connect Kibana to the CDN if that would be smoother.

So … who has an example config they can share?

[jkressin]

Hi,

first, thanks very much for writing this article. I guess it will also be helpful to other users, so do you mind if we link it from our docs?

Then I’d also be very interested to know what you struggled most with when setting up Search Guard. You are right, most of our users integrate Search Guard with authentication providers / technologies like LDAP, SAML, OpenID and the like. So how can we improve the documentation to make it easier for first time users? We’re already working on a Knowledge Base and also come cookbooks for the most common use cases.

Regarding your iFrame question, this can be a bit tricky, so let me try to explain:

When using Search Guard, all requests to Elasticsearch and Kibana need to carry some kind of authentication credentials. In other words, we do not allow any unauthenticated request, neither on Kibana nor on Elasticsearch (apart from anon access mode, I come to that in a bit).

So when using curl for example, you can add HTTP Basic Authentication header easily by adding the

-u user:pass

flag. However, when using an iFrame, you cannot add any HTTP header information to the call. You can only use the URL and query parameters.

Older browsers supported URLs in the format:

https://user:pass@example.com

But support for this has long been discontinued due to security considerations. The main problem here is that when using an iFrame with the format mentioned above, you inevitably expose username and password in your HTML document.

You basically have two options:

Use a proxy and add HTTP Basic Authentication to the requests

For that to work you do not need to configure anything specific regarding your Apache2 or nginx proxy on Search Guard side. Let’s say you want to embed a dashboard in an iFrame. Then make sure that the dashboard URL you are using in your iFrame hits your Apache2 proxy first. The proxy then forwards the request to Kibana, and adds an HTTP Basic authentication header to the call.

I would recommend to set up a dedicated SG user / role which has only read access to the indices that are underlying the dashboards.

So the flow would be:

Website → iFrame → Apache2 (adding credentials) → Kibana → Elasticsearch.

Using anonymous authentication

Search Guard and the Kibana plugin also support anonymous authentication. If this is enabled, Search Guard will automatically assign all unauthenticated requests to a specific user and role:

Anonymous authentication

Anonymous authentication

The anonymous mode is useful when you want to give users some limited access to your data, without the need to provide any credentials. You would then map the anon user to an SG role that has limited access permissions.

You can use the same approach also for embedding Dashboards in iFrames and omit the proxy completely. The requests from the iFrame hit Kibana directly. Kibana forwards the requests to Elasticsearch, and since anon auth is enabled, they are assigned to the anon user automatically.

Again, the SG role for this user should allow read access to the indices underlying the Dashboards. Here the flow would be:

Website → iFrame → Kibana → Elasticsearch (using anon auth)

[NetwarSystem]

Jochen - I’ve written for House/Senate/Gubernatorial races, and for a couple years I ghost wrote a column in Forbes magazine. If you want to use the stuff I do, that’s great, I’d be thrilled to actually have some of my stuff referenced here, but it needs critical review - either from you, or the community in general.

So that’s that - it’s open season on anything I write regarding Search Guard, and I’m doing the work to upgrade our system from 6.5.4 to 6.7.1 over the next few days, so I should be adding more content soon.