Multiple Kibana instances, "Invalid cookie value" and searchguard_storage cookie

Our cluster has multiple Kibana instances. With Elasticsearch 5 it is possible for someone to log in to multiple Kibana instances at the same time. With Elasticsearch 6 it is not possible. If someone logs in to one Kibana instance, then tries to log in to another one, instead of the login screen they get

{ "statusCode":400,"error":"Bad Request","message":"Invalid cookie value"}

[root@foo ~]# rpm -q --queryformat='%{version}\n' kibana
6.8.1
[root@foo ~]# /usr/share/kibana/bin/kibana-plugin list
searchguard@6.8.1-18.4

[root@foo ~]#

The Kibana instances are accessed via the same hostname with connections routed via HA Proxy. So https://blah.blah/foo shows the Kibana instance hosted on server foo and https://blah.blah/bar shows the Kibana instance hosted on server bar.

Each Kibana instance is configured to use it’s own basePath and index and Search Guard authentication cookie. E.g. on server foo kibana.yml has

server.host: "foo"
server.basePath: "/foo"
kibana.index: ".kibana-foo"
searchguard.auth.type: "basicauth"
searchguard.cookie.name: searchguard_authentication_foo

on server bar

server.host: "bar"
server.basePath: "/bar"
kibana.index: ".kibana-bar"
searchguard.auth.type: "basicauth"
searchguard.cookie.name: searchguard_authentication_bar

When we first deployed Search Guard with Elasticsearch 5 we used the same value of searchguard.cookie.name on every Kibana instance and if someone logged in to https://blah.blah/foo then logged in to https://blah.blah/bar they were instantly logged out of https://blah.blah/foo. That was fixed by making searchguard.cookie.name unique for each Kibana.

We’ve still got a cluster running Elasticsearch 5 and if I use the Firefox console to look at the cookies being used there’s only searchguard_authentication_foo and searchguard_authentication_bar listed. With the Kibana instances on the Elasticsearch 6 cluster there’s another cookie called searchguard_storage. Given the experience with Elasticsearch 5, and that deleting the searchguard_storage cookie makes the login screen appear but also breaks existing login sessions, I suspect the Invalid cookie value error would stop happening if the searchguard_storage cookie had a name unique on every Kibana instance.

I cannot find any information about the searchguard_storage cookie. The only two Google results for “searchguard_storage” are Proxy Authentication for Kibana requires Basic Authentication header where it is only mentioned as part of output someone posted and 302 redirect loop when following HTML links to Kibana after ES upgrade to 6.6.0, Searchguard 24.1 where it is mentioned as a cookie that should be present.

Is my suspicion that the searchguard_storage cookie is the cause of the error correct, or at least plausible?

If there a way to make the cookie currently called searchguard_storagehave a different name?

Hi @mikew,

I’m afraid you are right. The cookie called searchguard_storage is used to store information about the logged in user, which probably explains the invalid cookie value you are seeing.

I’ll take a look at this and then check with the team, we should probably make that cookie name configurable too.

Are you able to change the code in your Kibana installations? If you want to change this yourself before we have a fix, there are luckily only two places in the code base where the name is hard coded:


and

There’s also a third cookie called searchguard_preferences, which at the moment is used to remember the last visited tenant if you use multitenancy. To change that cookie would require a couple of changes more. It shouldn’t cause any errors though.

Thanks for reporting this!

Best Regards
Mike

Thanks for confirming my theory. I’ve modified the searchguard_cookie name in those two files on a Kibana instance and that has resulted in being able to be logged in to that Kibana instance and another one at the same time. Which is nice. I assume the change will be lost when Search Guard gets upgraded though. Maybe I’ll look at making our configuration management set a unique e searchguard_cookie name on each Kibana instance.

Thanks for confirming that it works!
I’ve made a note to update this thread as soon as we have something implemented.

Best regards,
Mike