Elasticsearch/Kibana 7.6.2 - Receiving error: "javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown"

Search Guard
4

The issue is resolved. It appears that the majority of the errors were because I had Logstash running, but hadn’t yet configured it to use TLS. Once I stopped Logstash, the majority of these issues stopped.

I then updated my kibana.yml to the following:

server:
  xsrf:
    whitelist:
      - "/searchguard/saml/acs/idpinitiated"
      - "/searchguard/saml/acs"
      - "/searchguard/saml/logout"

# Xpack configuration.
xpack:
  # Configure xpack security.
  security.enabled: false

elasticsearch:
  username: "${SG_KIBANASERVER_USER}"
  password: "${SG_KIBANASERVER_PASSWD}"
  ssl:
    alwaysPresentCertificate: false
    certificate: /usr/share/kibana/config/certs/node/cert.pem
    key: /usr/share/kibana/config/certs/node/key.pem
    certificateAuthorities: 
      - /usr/share/kibana/config/certs/node/ca_bundle.pem
    verificationMode: none

At that point, I stopped receiving all errors from Elasticsearch, and it appeared that Kibana was successfully communicating with it. Please note that I am still receiving a 302 error from the Kibana readiness probe and cannot connect to it via a web browser, but I am addressing that issue in a separate post.

When I run the following command to retrieve the most recent document in the .monitoring-kibana-7-* indices…

curl -k -u $ELASTIC_USERNAME:$ELASTIC_PASSWORD -H 'Content-Type: application/json' \
  https://localhost:9200/.monitoring-kibana-7-*/_search?pretty -d '
    {
      "query": {
        "match_all": {}
      },
      "size": 1,
      "sort": [
        {
          "timestamp": {
            "order": "desc"
          }
        }
      ]
    }'

…I receive the following output, which appears to indicate that Kibana is communicating to Elasticsearch on an ongoing basis (see timestamp):

{
  "took" : 58,
  "timed_out" : false,
  "_shards" : {
    "total" : 7,
    "successful" : 7,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : ".monitoring-kibana-7-2020.04.23",
        "_type" : "_doc",
        "_id" : "tIvwp3EBjwb6QOtkyCo0",
        "_score" : null,
        "_source" : {
          "cluster_uuid" : "hvkuxjblQ3WsHjz8ON35-g",
          "timestamp" : "2020-04-23T16:48:27.185Z",
          "interval_ms" : 10000,
          "type" : "kibana_stats",
          "source_node" : {
            "uuid" : "urvAbq76SqCjnAEPsr_hyA",
            "host" : "10.229.94.58",
            "transport_address" : "10.229.94.58:9300",
            "ip" : "10.229.94.58",
            "name" : "elk-es-coord-0",
            "timestamp" : "2020-04-23T16:48:27.185Z"
          },
          "kibana_stats" : {
            "kibana" : {
              "uuid" : "5ddddc36-822d-4c5e-aa2d-2aa1c883f025",
              "name" : "elk-kibana-6b7d4d7dc6-cn9xj",
              "index" : ".kibana",
              "host" : "0.0.0.0",
              "transport_address" : "0.0.0.0:5601",
              "version" : "7.6.2",
              "snapshot" : false,
              "status" : "red"
            },
            "cloud" : {
              "name" : "aws",
              "id" : "i-000c8043efc0be372",
              "vm_type" : "r5d.4xlarge",
              "region" : "us-east-1",
              "zone" : "us-east-1c",
              "metadata" : {
                "architecture" : "x86_64",
                "marketplaceProductCodes" : null,
                "imageId" : "ami-0dc7713312a7ec987",
                "kernelId" : null,
                "pendingTime" : "2020-04-02T18:17:30Z",
                "ramdiskId" : null,
                "version" : "2017-09-30"
              }
            },
            "concurrent_connections" : 0,
            "os" : {
              "load" : {
                "1m" : 3.01611328125,
                "5m" : 2.9482421875,
                "15m" : 2.87060546875
              },
              "memory" : {
                "total_in_bytes" : 133653843968,
                "free_in_bytes" : 120397815808,
                "used_in_bytes" : 13256028160
              },
              "uptime_in_millis" : 747809000,
              "platform" : "linux",
              "platformRelease" : "linux-4.14.171-136.231.amzn2.x86_64",
              "distro" : "Centos",
              "distroRelease" : "Centos-7.7.1908"
            },
            "process" : {
              "event_loop_delay" : 1.523249626159668,
              "memory" : {
                "heap" : {
                  "total_in_bytes" : 858374144,
                  "used_in_bytes" : 599807576,
                  "size_limit" : 2217857988
                },
                "resident_set_size_in_bytes" : 1243553792
              },
              "uptime_in_millis" : 9496096
            },
            "requests" : {
              "disconnects" : 0,
              "total" : 2
            },
            "response_times" : {
              "average" : 2,
              "max" : 2
            },
            "timestamp" : "2020-04-23T16:48:24.960Z"
          }
        },
        "sort" : [
          1587660507185
        ]
      }
    ]
  }
}
1 Like