Unable to login to Kibana after 7.10.1 upgrade

Search Guard
#1

TL;DR

Unable to login to Kibana via SAML after 7.10.1 upgrade.

Issue

I’m unable to login to Kibana, after deploying v7.10.1, with Search Guard installed, similar to this post. However, unlike this post, redeploying did not resolve the issue. Fortunately, this is my test environment, but I would like to move to production, ASAP.

Details are:

  • Elasticsearch version: 7.10.1, with Search Guard 48.0.0
  • Kibana version: 7.10.1, with Kibana Plugin 48.0.0
  • Using standard Elastic container images, with Search Guard installed.

I’m receiving the following error:

Screen Shot 2021-01-04 at 3.27.17 PM

I’ve attached copies of config files and logs. Note that these config files work without any issues under previous versions, including 7.9.1, which I’m currently using in production.

elasticsearch.log (21.3 KB) elasticsearch.yml (1.9 KB) kibana.log (268.2 KB) kibana.yml (1.6 KB) sg_config.yml (4.0 KB)

#2

Hi. Have you tried to clear the browser cookies? In the 7.10.0 we had to change the cookie. Thus you may have the cookies conflict.

I see SAML auth, failed to authorize: Error: The session cookie is absent. error in the Kibana log. Show me the browser cookies.

#3

That was my initial thought too, so I tried to open and authenticate from an incognito window, which I can usually do. That didn’t work. However, when I cleared all browser cookies from my main window and restarted, I’m still getting the same issue.

Here’s a screenshot of the browser cookies from the incognito window for a failed connection to the non-working instance:

cookies
cookies1207×235 31.8 KB

I realize you said that the cookies had been updated in the latest version, but here’s a screenshot of the cookies from the version that still works correctly:

cookies-2

#4

What does happen when you click the “Back to Kibana Home” button?
What was your upgrade procedure for the Search Guard Kibana plugin?
What IDP do you use?

#5

It attempts to go to the Kibana home, then ultimately ends up at the same screen.

We’re deploying via Kubernetes, so I wrote a Dockerfile, which pulls copies of the Elasticsearch and Kibana images published by Elastic, and then installs Search Guard into the image. At that point, I deploy.

I just rebuilt and redeployed, and confirmed that I’m using the correct version of SG with Elasticsearch & Kibana. No luck.

#6

I see the LDAP authz error in the Elasticsearch log.

"Caused by: org.elasticsearch.ElasticsearchSecurityException: No user admin found",
"at com.floragunn.dlic.auth.ldap.backend.LDAPAuthorizationBackend.fillRoles(LDAPAuthorizationBackend.java:747) ~[dlic-search-guard-suite-security-7.10.1-48.0.0.jar:7.10.1-48.0.0]",

Do you have the admin user on the LDAP server? Does connection to LDAP server work? Look at the LDAP server logs. Do you see anything suspicious in the LDAP logs?

Are you able to fetch SAML metadata_url and entity_id data from the Elasticsearch server?

curl $SG_SAML_METADATA_URL
curl $SG_SAML_IDP_ENTITY_ID
curl $SG_SAML_SP_ENTITY_ID

I put some extra logs to the code and built the SG Kibana plugin. Install it and try again. Record the Kibana logs and send the logs to me. The build: build - Google Drive

#7

Sorry for taking so long to respond. I’m not able to download files from Google Drive, as we have it blocked in our organization. Any other way you can get the plugin to me with the add’l logs?

Connection to the LDAP server works. When I do the following command against Elasticsearch using an LDAP user with appropriate permissions, I get the expected output.

❯ curl -ks -u $ELK_USER:$ELK_PASSWORD https://elasticsearch.example.com:9200/_cluster/health\?pretty\=true
{
  "cluster_name" : "elk-es",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 14,
  "number_of_data_nodes" : 9,
  "active_primary_shards" : 274,
  "active_shards" : 555,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

However, I’ll need to touch base with my LDAP admin to have him check those logs.

I initially was not able to hit SG_SAML_IDP_ENTITY_ID, but have that issue resolved. However, I’m still getting the same issues.

#9

I’ve uploaded the custom build to the official Search Guard Maven repo. You can download it from here https://maven.search-guard.com//search-guard-kibana-plugin-snapshot/my-test-SNAPSHOT/search-guard-kibana-plugin-7.10.1-master-SNAPSHOT-EXTRA-LOGGING-TEST-PURPOSES-ONLY-TO-BE-DELETED.zip

#10

Hi. Try to put the following in the kibana.yml and let me know

searchguard.cookie.isSameSite: None
searchguard.cookie.secure: true
1 Like
#11

I assume that searchguard.cookie.isSameSite: None is a string, not a data type, correct? IIRC, there is no None data type, just want to confirm…

#12

That did it. So my questions are:

  • What was the issue?
  • If I need to redeploy my 7.9.x instance before I can upgrade it to 7.10.1, will these additional config settings negatively impact my 7.9.x instance?