Authentication failed for null Warning

If you think it is a bug report or you have a technical issue, please answer the following questions. For general questions, you can delete these questions.

Elasticsearch version:
7.17.9

Kibana version (if relevant):
7.17.9

Search Guard version (if relevant):
FLEX 1.1.1-es-7.17.9

I recently upgraded my Elasticsearch server from Search Guard from 7.17.9-53.6.0 to Search Guard FLEX 1.1.1-es-7.17.9. Everything seems to be working fine, but i am seeing some new errors in my elasticsearch logs:

2023-05-25 21:21:33.686Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#19]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.20.173.95, originatingIpAddress=172.20.173.95, clientCertSubject=null]
2023-05-25 21:21:35.101Z INFO  [elasticsearch[esnode-aln-nbadev4][masterService#updateTask][T#1]] org.elasticsearch.xpack.ilm.action.TransportPutLifecycleAction - updating index lifecycle policy [.alerts-ilm-policy]
2023-05-25 21:21:35.205Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#57]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]
2023-05-25 21:21:35.207Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#58]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]
2023-05-25 21:26:35.174Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#58]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]
2023-05-25 21:31:36.719Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#19]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.20.173.95, originatingIpAddress=172.20.173.95, clientCertSubject=null]

Since they occur around the lifecycle policy message and deal with the .kibana_7.17.9/_search command, I suspect that this is some kind of Kibana index cleanup process. These warning are written about every five minutes. Are these warnings something that I need to worry about?

The 172.20.173.95 is the other Kibana server. The 172.17.0.1 address is the docker0 address on the local server.

My elasticsearch.yml:

[root@aln-nbadev4 elasticsearch]# more elasticsearch.yml
---
# Elasticsearch Configuration
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html

#
# Cluster and Discovery
#
cluster.name: nba_elasticsearch_cluster
discovery.seed_hosts: [ "aln-nbadev4"  ]
discovery.type: single-node
path.repo: /usr/share/elasticsearch/aln-nbadev4

#
# Node
#
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: esnode-aln-nbadev4

#
# Paths
#
path.data: /data
path.logs: /logs

#
# Internals
#
bootstrap.memory_lock: false
script.painless.regex.enabled: true
indices.query.bool.max_clause_count: 2048
signals.enabled: false
action.auto_create_index: true

#
# Network
#
network.host: 0.0.0.0
network.publish_host: aln-nbadev4.labs.server.com

http.publish_host: aln-nbadev4.labs.server.com
http.port: 9200
http.max_content_length: 100mb
http.compression: false

transport.publish_host: aln-nbadev4.labs.server.com
transport.port: 9300
transport.compress: true

#
# Security General
#
searchguard.enterprise_modules_enabled: false
searchguard.config_index_name: searchguard
searchguard.roles_mapping_resolution: MAPPING_ONLY
searchguard.nodes_dn:
  - "CN=esnode-*,OU=nBA,O=Company,L=City,ST=MA,C=US"
searchguard.authcz.admin_dn:
  - "CN=esadmin,OU=nBA,O=Company,L=City,ST=MA,C=US"
xpack.security.enabled: false
searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]

#
# Transport Layer Security
#
searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.pemcert_filepath: certs/esnode.pem
searchguard.ssl.transport.pemkey_filepath: certs/esnode.key
searchguard.ssl.transport.pemkey_password: ${ES_ADMIN_PASSWORD}
searchguard.ssl.transport.pemtrustedcas_filepath: certs/cacert.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.enabled_protocols:
  - "TLSv1.2"
  - "TLSv1.3"

#
# REST Layer Security
#
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: certs/esnode.pem
searchguard.ssl.http.pemkey_filepath: certs/esnode.key
searchguard.ssl.http.pemkey_password: ${ES_ADMIN_PASSWORD}
searchguard.ssl.http.pemtrustedcas_filepath: certs/cacert.pem
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.ssl.http.enabled_protocols:
  - "TLSv1.2"
  - "TLSv1.3"

# Disable geoip indexing
ingest.geoip.downloader.enabled: false
# Increase the grok timeout
ingest.grok.watchdog.max_execution_time: 5s

@silentfilm What type of authentication do you use?

Have you noticed any jobs/tasks running in the logs before that warning?

There is no usual task running before these errors. But they do occur regularly.

When Elasticsearch starts up, I do get these errors once for each transport worker. (My node will always be yellow because it is a single-node development lab.)

2023-05-26 19:19:39.960Z INFO  [main] org.elasticsearch.node.Node - started
2023-05-26 19:19:39.960Z INFO  [main] com.floragunn.searchguard.SearchGuardPlugin - Node started
2023-05-26 19:19:40.261Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#5]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.261Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#3]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.261Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.270Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.274Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.278Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.281Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#6]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.283Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#7]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.283Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#5]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.285Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#3]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.285Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#6]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.288Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#7]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.479Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#8]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:40.983Z INFO  [elasticsearch[esnode-aln-nbadev4][clusterApplierService#updateTask][T#1]] org.elasticsearch.license.LicenseService - license [54d962f6-4220-4136-a380-f9cf7861ec96] mode [basic] - valid
2023-05-26 19:19:40.987Z INFO  [elasticsearch[esnode-aln-nbadev4][clusterApplierService#updateTask][T#1]] com.floragunn.searchguard.configuration.ConfigurationRepository - Cluster state has been recovered. Starting config index initialization.
2023-05-26 19:19:40.987Z INFO  [elasticsearch[esnode-aln-nbadev4][clusterApplierService#updateTask][T#1]] com.floragunn.searchguard.configuration.ConfigurationRepository - Legacy searchguard index does exist. Loading configuration.
2023-05-26 19:19:40.991Z INFO  [elasticsearch[esnode-aln-nbadev4][masterService#updateTask][T#1]] org.elasticsearch.gateway.GatewayService - recovered [45] indices into cluster_state
2023-05-26 19:19:41.005Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#9]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.009Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#9]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.013Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#9]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.016Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#9]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.020Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.025Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.030Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.033Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#4]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.555Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#10]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:41.875Z INFO  [elasticsearch[esnode-aln-nbadev4][generic][T#3]] com.floragunn.searchguard.configuration.variables.ConfigVarService - Creating secrets: {sessions_signing_key=[scope=authc]}
2023-05-26 19:19:42.525Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#7]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.527Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#3]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.530Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#7]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.533Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#3]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.667Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#6]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.671Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#5]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.673Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#6]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:42.675Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#5]] com.floragunn.searchguard.authc.rest.AuthenticatingRestFilter - Not yet initialized (you may need to run sgctl)
2023-05-26 19:19:43.073Z INFO  [elasticsearch[esnode-aln-nbadev4][masterService#updateTask][T#1]] org.elasticsearch.cluster.routing.allocation.AllocationService - Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[searchguard][0]]]).
2023-05-26 19:19:43.663Z INFO  [elasticsearch[esnode-aln-nbadev4][generic][T#2]] com.floragunn.searchguard.authz.AuthorizationService - Updated authz config (legacy):
config@1.0/n:1
2023-05-26 19:19:43.679Z INFO  [elasticsearch[esnode-aln-nbadev4][generic][T#2]] com.floragunn.searchguard.authz.PrivilegesEvaluator - Updated authz config (legacy):
config@1.0/n:1
2023-05-26 19:19:43.751Z INFO  [elasticsearch[esnode-aln-nbadev4][generic][T#2]] com.floragunn.searchguard.authc.session.backend.SessionService - Updating keys for com.floragunn.searchguard.authc.session.backend.SessionService@2c4cbce8
2023-05-26 19:19:43.763Z INFO  [elasticsearch[esnode-aln-nbadev4][generic][T#2]] com.floragunn.searchguard.configuration.ConfigurationRepository - Node 'esnode-aln-nbadev4' initialized
2023-05-26 19:19:52.555Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#21]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.20.173.95, originatingIpAddress=172.20.173.95, clientCertSubject=null]
2023-05-26 19:19:52.556Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#20]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/, directIpAddress=172.20.173.95, originatingIpAddress=172.20.173.95, clientCertSubject=null]
2023-05-26 19:19:54.727Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#24]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]

I’m just using basic authenticaion.

root@elasticsearch-aln-nbadev4:/usr/share/elasticsearch/config# cat /config/sg/sg_config.yml
---
_sg_meta:
  type: "config"
  config_version: 2

sg_config:
  dynamic:
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        remoteIpHeader: "x-forwarded-for"
    kibana:
      multitenancy_enabled: false
      server_username: "kibanaserver"
      index: ".kibana"
      rbac_enabled: false
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: "basic"
          challenge: true
          config: { }
        authentication_backend:
          type: "internal"
          config: { }
        skip_users: [ ]
        enabled_only_for_ips: null
      proxy_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 1
        http_authenticator:
          type: "proxy"
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: "noop"
          config: { }
        skip_users: [ ]
        enabled_only_for_ips: null
      clientcert_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 2
        http_authenticator:
          type: "clientcert"
          config:
            username_attribute: "cn"
          challenge: false
        authentication_backend:
          type: "noop"
          config: { }
        skip_users: [ ]
        enabled_only_for_ips: null
    authz: { }

More debugging information:

root@elasticsearch-aln-nbadev4:/usr/share/elasticsearch/sgctl-1.1.0# ./sgctl.sh component-state
Successfully connected to cluster nba_elasticsearch_cluster (aln-nbadev4.labs.server.com) as user CN=esadmin,OU=nBA,O=Company,L=City,ST=MA,C=US
---
health: "GREEN"
components:
- name: "config_repository"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827680069Z"
  initialized_at: "2023-05-26T19:19:43.763560277Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "config_repository"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:36.638152590Z"
    changed_at: "2023-05-26T19:19:43.763560277Z"
    initialized_at: "2023-05-26T19:19:43.763560277Z"
    build:
      file: "search-guard-flx-security-1.1.1-es-7.17.9.jar"
      version: "1.1.1-es-7.17.9"
      time: "2023-02-19T14:41:10Z"
    config:
      effective_main_config_index: "searchguard"
    parts:
    - type: "config_type"
      name: "tenants"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639360342Z"
      changed_at: "2023-05-26T19:19:43.627817441Z"
      initialized_at: "2023-05-26T19:19:43.627817441Z"
      parts:
      - type: "config"
        name: "tenants"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.627815567Z"
        changed_at: "2023-05-26T19:19:43.627817441Z"
        initialized_at: "2023-05-26T19:19:43.627817441Z"
        config:
          version: "1"
    - type: "config_type"
      name: "rolesmapping"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639380233Z"
      changed_at: "2023-05-26T19:19:43.641924546Z"
      initialized_at: "2023-05-26T19:19:43.641924546Z"
      parts:
      - type: "config"
        name: "rolesmapping"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.641921464Z"
        changed_at: "2023-05-26T19:19:43.641924546Z"
        initialized_at: "2023-05-26T19:19:43.641924546Z"
        config:
          version: "1"
    - type: "config_type"
      name: "blocks"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639383810Z"
      changed_at: "2023-05-26T19:19:43.642128931Z"
      initialized_at: "2023-05-26T19:19:43.642128931Z"
      parts:
      - type: "config"
        name: "blocks"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.642126494Z"
        changed_at: "2023-05-26T19:19:43.642128931Z"
        initialized_at: "2023-05-26T19:19:43.642128931Z"
        config:
          version: "1"
    - type: "config_type"
      name: "authc"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639421557Z"
      changed_at: "2023-05-26T19:19:43.642147328Z"
      parts:
      - type: "config"
        name: "authc"
        state: "SUSPENDED"
        sub_state: "config_does_not_exist"
        started_at: "2023-05-26T19:19:43.642145881Z"
        changed_at: "2023-05-26T19:19:43.642147328Z"
    - type: "config_type"
      name: "roles"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639423868Z"
      changed_at: "2023-05-26T19:19:43.647539053Z"
      initialized_at: "2023-05-26T19:19:43.647539053Z"
      parts:
      - type: "config"
        name: "roles"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.647535920Z"
        changed_at: "2023-05-26T19:19:43.647539053Z"
        initialized_at: "2023-05-26T19:19:43.647539053Z"
        config:
          version: "1"
    - type: "config_type"
      name: "frontend_authc"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639426182Z"
      changed_at: "2023-05-26T19:19:43.647557664Z"
      parts:
      - type: "config"
        name: "frontend_authc"
        state: "SUSPENDED"
        sub_state: "config_does_not_exist"
        started_at: "2023-05-26T19:19:43.647555810Z"
        changed_at: "2023-05-26T19:19:43.647557664Z"
    - type: "config_type"
      name: "internalusers"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639428182Z"
      changed_at: "2023-05-26T19:19:43.649037080Z"
      initialized_at: "2023-05-26T19:19:43.649037080Z"
      parts:
      - type: "config"
        name: "internalusers"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.649034479Z"
        changed_at: "2023-05-26T19:19:43.649037080Z"
        initialized_at: "2023-05-26T19:19:43.649037080Z"
        config:
          version: "1"
    - type: "config_type"
      name: "actiongroups"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639430306Z"
      changed_at: "2023-05-26T19:19:43.649195508Z"
      initialized_at: "2023-05-26T19:19:43.649195508Z"
      parts:
      - type: "config"
        name: "actiongroups"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.649193937Z"
        changed_at: "2023-05-26T19:19:43.649195508Z"
        initialized_at: "2023-05-26T19:19:43.649195508Z"
        config:
          version: "1"
    - type: "config_type"
      name: "config"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639432614Z"
      changed_at: "2023-05-26T19:19:43.663056169Z"
      initialized_at: "2023-05-26T19:19:43.663056169Z"
      parts:
      - type: "config"
        name: "config"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.663053558Z"
        changed_at: "2023-05-26T19:19:43.663056169Z"
        initialized_at: "2023-05-26T19:19:43.663056169Z"
        config:
          version: "1"
    - type: "config_type"
      name: "license_key"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639435005Z"
      changed_at: "2023-05-26T19:19:43.663077422Z"
      parts:
      - type: "config"
        name: "license_key"
        state: "SUSPENDED"
        sub_state: "config_does_not_exist"
        started_at: "2023-05-26T19:19:43.663076138Z"
        changed_at: "2023-05-26T19:19:43.663077422Z"
    - type: "config_type"
      name: "authz"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.639437474Z"
      changed_at: "2023-05-26T19:19:43.663087713Z"
      parts:
      - type: "config"
        name: "authz"
        state: "SUSPENDED"
        sub_state: "config_does_not_exist"
        started_at: "2023-05-26T19:19:43.663086941Z"
        changed_at: "2023-05-26T19:19:43.663087713Z"
- type: "authc"
  name: "rest_filter"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827690376Z"
  initialized_at: "2023-05-26T19:19:43.663056169Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    type: "authc"
    name: "rest_filter"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:36.776712166Z"
    changed_at: "2023-05-26T19:19:43.663056169Z"
    initialized_at: "2023-05-26T19:19:43.663056169Z"
    parts:
    - type: "config"
      name: "config"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.663053558Z"
      changed_at: "2023-05-26T19:19:43.663056169Z"
      initialized_at: "2023-05-26T19:19:43.663056169Z"
      config:
        version: "1"
    - type: "rest_authentication_processor"
      name: "legacy_rest_authentication_processor"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.762031364Z"
      initialized_at: "2023-05-26T19:19:43.655473626Z"
      parts:
      - type: "auth_domain"
        name: "session"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:36.767534648Z"
        changed_at: "2023-05-26T19:19:36.767913074Z"
        initialized_at: "2023-05-26T19:19:36.767913074Z"
        metrics:
          authentication_backend:
            agg:
              count: 0
        parts:
        - type: "authentication_frontend"
          name: "session"
          state: "INITIALIZED"
          started_at: "2023-05-26T19:19:36.767910322Z"
          changed_at: "2023-05-26T19:19:36.767913074Z"
          initialized_at: "2023-05-26T19:19:36.767913074Z"
      - type: "legacy_auth_domain"
        name: "basic_internal_auth_domain"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.660628721Z"
        initialized_at: "2023-05-26T19:19:43.655473626Z"
        parts:
        - type: "authentication_frontend"
          name: "basic"
          state: "INITIALIZED"
          started_at: "2023-05-26T19:19:43.654718031Z"
          changed_at: "2023-05-26T19:19:43.655473626Z"
          initialized_at: "2023-05-26T19:19:43.655473626Z"
          build:
            file: "search-guard-flx-security-legacy-1.1.1-es-7.17.9.jar"
            version: "1.1.1-es-7.17.9"
            time: "2023-02-19T14:41:10Z"
- type: "authc"
  name: "session_service"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827466298Z"
  initialized_at: "2023-05-26T19:19:43.663056169Z"
  metrics:
    start_session_with_authentication:
      agg:
        count: 0
    start_session_with_external_authentication:
      agg:
        count: 0
    delete_session:
      agg:
        count: 0
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    type: "authc"
    name: "session_service"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:31.235015669Z"
    changed_at: "2023-05-26T19:19:43.760655664Z"
    initialized_at: "2023-05-26T19:19:43.663056169Z"
    build:
      file: "search-guard-flx-security-1.1.1-es-7.17.9.jar"
      version: "1.1.1-es-7.17.9"
      time: "2023-02-19T14:41:10Z"
    metrics:
      start_session_with_authentication:
        agg:
          count: 0
      start_session_with_external_authentication:
        agg:
          count: 0
      delete_session:
        agg:
          count: 0
    parts:
    - name: "session_activity_tracker"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.760027660Z"
      changed_at: "2023-05-26T19:19:36.760440845Z"
      initialized_at: "2023-05-26T19:19:36.760440845Z"
      build:
        file: "search-guard-flx-security-1.1.1-es-7.17.9.jar"
        version: "1.1.1-es-7.17.9"
        time: "2023-02-19T14:41:10Z"
      metrics:
        flush:
          agg:
            count: 0
    - type: "index"
      name: ".searchguard_sessions"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.762926199Z"
      changed_at: "2023-05-26T19:19:41.986050740Z"
      initialized_at: "2023-05-26T19:19:41.986050740Z"
    - name: "index_cleanup_agent_.searchguard_sessions"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.766394798Z"
      changed_at: "2023-05-26T19:19:39.941885852Z"
      metrics:
        delete_actions:
          agg:
            count: 0
    - type: "config"
      name: "config"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.663053558Z"
      changed_at: "2023-05-26T19:19:43.663056169Z"
      initialized_at: "2023-05-26T19:19:43.663056169Z"
      config:
        version: "1"
    - type: "config"
      name: "frontend_authc"
      state: "SUSPENDED"
      sub_state: "config_does_not_exist"
      started_at: "2023-05-26T19:19:43.647555810Z"
      changed_at: "2023-05-26T19:19:43.647557664Z"
    - type: "legacy_auth_domain"
      name: "basic_internal_auth_domain"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.660628721Z"
      initialized_at: "2023-05-26T19:19:43.655473626Z"
      parts:
      - type: "authentication_frontend"
        name: "basic"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.654718031Z"
        changed_at: "2023-05-26T19:19:43.655473626Z"
        initialized_at: "2023-05-26T19:19:43.655473626Z"
        build:
          file: "search-guard-flx-security-legacy-1.1.1-es-7.17.9.jar"
          version: "1.1.1-es-7.17.9"
          time: "2023-02-19T14:41:10Z"
- name: "license_repository"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827682280Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "license_repository"
    state: "SUSPENDED"
    sub_state: "enterprise_modules_disabled"
    started_at: "2023-05-26T19:19:36.641735958Z"
    changed_at: "2023-05-26T19:19:36.642082592Z"
    build:
      file: "search-guard-flx-security-1.1.1-es-7.17.9.jar"
      version: "1.1.1-es-7.17.9"
      time: "2023-02-19T14:41:10Z"
- name: "privileges_evaluator"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827684310Z"
  initialized_at: "2023-05-26T19:19:43.720422690Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "privileges_evaluator"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:36.733552160Z"
    changed_at: "2023-05-26T19:19:43.720422690Z"
    initialized_at: "2023-05-26T19:19:43.720422690Z"
    config:
      version: "{tenants=1, frontend_authc=-1, config=1, rolesmapping=1, authc=-1,\
        \ sessions=-1, license_key=-1, blocks=1, internalusers=1, actiongroups=1,\
        \ authz=-1, roles=1}"
    parts:
    - name: "role_based_action_authorization"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.710960585Z"
      changed_at: "2023-05-26T19:19:43.720422690Z"
      initialized_at: "2023-05-26T19:19:43.720422690Z"
      config:
        version: "1"
      metrics:
        index_action_types:
          agg:
            count: 14024
            parts:
              non_well_known:
                count: 0
              well_known:
                count: 14024
        tenant_action_check_results:
          agg:
            count: 0
        index_action_check_results:
          agg:
            count: 10648
            parts:
              partially_ok:
                count: 0
              insufficient:
                count: 0
              ok:
                count: 1218
        index_action_checks:
          agg:
            count: 10648
            parts:
              well_known_actions_uncheck_exclusions:
                count: 953
              well_known_action_index_pattern:
                count: 953
                parts:
                  render_index_pattern_template:
                    count: 2072
        tenant_action_checks:
          agg:
            count: 0
      parts:
      - name: "cluster_permissions"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.681930714Z"
        changed_at: "2023-05-26T19:19:43.688016291Z"
        initialized_at: "2023-05-26T19:19:43.688016291Z"
        config:
          version: "1"
        metrics:
          roles_to_action_pattern_map:
            count: 16
          action_to_roles_map:
            count: 66
          checks:
            agg:
              count: 7594
              parts:
                non_well_known_actions:
                  count: 420
                wildcard:
                  count: 0
      - name: "cluster_permission_exclusions"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.688542313Z"
        changed_at: "2023-05-26T19:19:43.688888885Z"
        initialized_at: "2023-05-26T19:19:43.688888885Z"
        config:
          version: "1"
      - name: "index_permissions"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.689476291Z"
        changed_at: "2023-05-26T19:19:43.707956439Z"
        initialized_at: "2023-05-26T19:19:43.707956439Z"
        config:
          version: "1"
      - name: "index_permission_exclusions"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.708719824Z"
        changed_at: "2023-05-26T19:19:43.709404159Z"
        initialized_at: "2023-05-26T19:19:43.709404159Z"
        config:
          version: "1"
      - name: "tenant_permissions"
        state: "INITIALIZED"
        started_at: "2023-05-26T19:19:43.710948952Z"
        changed_at: "2023-05-26T19:19:43.710956271Z"
        initialized_at: "2023-05-26T19:19:43.710956271Z"
        config:
          version: "1"
      - name: "index_permissions_stateful"
        state: "INITIALIZED"
        message: "Initialized with 45 indices"
        started_at: "2023-05-26T19:19:43.681171551Z"
        changed_at: "2023-05-26T19:19:43.720422690Z"
        initialized_at: "2023-05-26T19:19:43.720422690Z"
        config:
          version: "1"
    - name: "action_group_index"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:43.680005246Z"
      changed_at: "2023-05-26T19:19:43.680482453Z"
      initialized_at: "2023-05-26T19:19:43.680482453Z"
      metrics:
        size:
          count: 39
        init_rounds:
          count: 1
- name: "internal_users_database"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827688108Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "internal_users_database"
    state: "INITIALIZED"
    message: "10 users"
    started_at: "2023-05-26T19:19:36.738322834Z"
    changed_at: "2023-05-26T19:19:43.721190986Z"
    config:
      version: "1"
- name: "protected_config_index_service"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827655988Z"
  initialized_at: "2023-05-26T19:19:43.098210389Z"
  metrics:
    flush_pending_indices:
      agg:
        count: 1
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "protected_config_index_service"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:36.627581721Z"
    changed_at: "2023-05-26T19:19:43.098210389Z"
    initialized_at: "2023-05-26T19:19:43.098210389Z"
    metrics:
      flush_pending_indices:
        agg:
          count: 1
    parts:
    - type: "index"
      name: ".searchguard_config_vars"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.637991291Z"
      changed_at: "2023-05-26T19:19:41.876185713Z"
      initialized_at: "2023-05-26T19:19:41.876185713Z"
    - type: "index"
      name: ".searchguard_sessions"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.762926199Z"
      changed_at: "2023-05-26T19:19:41.986050740Z"
      initialized_at: "2023-05-26T19:19:41.986050740Z"
    - type: "index"
      name: ".searchguard_resource_owner"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:37.051581963Z"
      changed_at: "2023-05-26T19:19:42.218592629Z"
      initialized_at: "2023-05-26T19:19:42.218592629Z"
- name: "signals"
  state: "DISABLED"
  started_at: "2023-05-26T19:40:28.827647999Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "signals"
    state: "DISABLED"
    started_at: "2023-05-26T19:19:31.241893769Z"
    changed_at: "2023-05-26T19:19:31.242427681Z"
    build:
      file: "search-guard-flx-signals-1.1.1-es-7.17.9.jar"
      version: "1.1.1-es-7.17.9"
      time: "2023-02-19T14:41:10Z"
- name: "config_var_storage"
  state: "INITIALIZED"
  started_at: "2023-05-26T19:40:28.827676182Z"
  initialized_at: "2023-05-26T19:19:41.876185713Z"
  parts:
  - node_id: "MluE9MdNRoadFBoxtx_r3Q"
    node_name: "esnode-aln-nbadev4"
    name: "config_var_storage"
    state: "INITIALIZED"
    started_at: "2023-05-26T19:19:36.633042531Z"
    changed_at: "2023-05-26T19:19:41.876185713Z"
    initialized_at: "2023-05-26T19:19:41.876185713Z"
    build:
      file: "search-guard-flx-security-1.1.1-es-7.17.9.jar"
      version: "1.1.1-es-7.17.9"
      time: "2023-02-19T14:41:10Z"
    parts:
    - type: "index"
      name: ".searchguard_config_vars"
      state: "INITIALIZED"
      started_at: "2023-05-26T19:19:36.637991291Z"
      changed_at: "2023-05-26T19:19:41.876185713Z"
      initialized_at: "2023-05-26T19:19:41.876185713Z"

@silentfilm Do you use FLX or a classic plugin? The shared config belongs to the classic plugin.

I am using the FLX plugin, but I am upgrading from the classic plugin so my Search Guard config files are the same.

One issue that I found was that I was just doing a “sgctl.sh connect”. I now follow that up with an “sgctl.sh update-config”.

Selected cluster: aln-nbadev4.labs.server.com
Connecting to elasticsearch-aln-nbadev4.labs.server.com:9200 with certificate CN=esadmin, OU=nBA, O=Company, L=City, ST=MA, C=US
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <elasticsearch-aln-nbadev4.labs.server.com> doesn't match any of the subject alternative names: [aln-nbadev4.labs.server.com, esnode-aln-nbadev4.labs.server.com]
Selected cluster: aln-nbadev4.labs.server.com
Connecting to aln-nbadev4.labs.server.com:9200 with certificate CN=esadmin, OU=nBA, O=Company, L=City, ST=MA, C=US
Successfully connected to cluster nba_elasticsearch_cluster (aln-nbadev4.labs.server.com) as user CN=esadmin,OU=nBA,O=Company,L=City,ST=MA,C=US
Uploading config files from directory /config/sg: sg_tenants.yml, sg_blocks.yml, sg_action_groups.yml, sg_config.yml, sg_roles.yml, sg_authc.yml, sg_internal_users.yml, sg_roles_mapping.yml
Configuration has been updated

However, I still get the “Not yet initialized” errors. I don’t understand what you mean about the “shared” configuration from the classic Search Guard.

This is sg_config used in the classic SG plugin. Did you follow migration steps described in the FLX documentation?

When I ran the upgrade procedure, it only created a sg_authc.yml, which is included in my installation.

---
auth_domains:
- type: "basic/internal_users_db"

I didn’t get any other files. We are only using the elasticsearch plugin, not the kibana plugin, if that makes a difference. I will try it again.

Also, I noticed yesterday that the filebeat, metricbeat, and kibana users are working just fine. It is only the users that I have created and are not part of the default SG setup that are not working.

root@elasticsearch-aln-nbadev4:/config# /usr/share/elasticsearch/sgctl-1.1.0/sgctl.sh migrate-config /config/sg/sg_config.yml -o /config/newsg --target-platform es711
Welcome to the Search Guard config migration tool.

This tool converts legacy Search Guard configuration to configuration suitable for the next generation Search Guard release.
The tool also provides basic guidance for a seamless update process without outages.

You have not specified a kibana.yml file. Thus, we are assuming that you are not using Kibana. If you are using Kibana and want to adapt the migration, please specify the path to your kibana.yml file on the command line.


The update process consists of these steps:

- Update the Search Guard plugin for Elasticsearch on all nodes of your cluster. In this step, you do not yet need to modify the configuration.

- After having updated the Search Guard Elasticsearch plugin, please upload the new configuration files with sgctl:

$ ./sgctl.sh update-config sg_authc.yml

The files have been automatically generated from the settings in sg_config.yml.  The files are listed below and have been also put to /config/newsg.


----------------------------- sg_authc.yml --------------------------------------
---
auth_domains:
- type: "basic/internal_users_db"

---------------------------------------------------------------------------------

Since the migrate-config command wasn’t working, I looked in the new sample files for differences from my old files. I deleted sg_config.yml, but that didn’t help. After a lot of trial and error, I found that removing the following lines caused Search Guard to start working again. I had to remove them from each SG role:

  reserved: true
  hidden: true
  static: true

I guess that the functionality of these values has changed. Maybe the roles really were hidden now, but these lines worked in the pre-FLX SearchGuard. The strange thing is that they are still defined that way for all my users in the sg_internal_users.yml file.

I’m still getting the original errors from my first post though:

2023-05-31 22:03:45.961Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#31]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.20.173.95, originatingIpAddress=172.20.173.95, clientCertSubject=null]
2023-05-31 22:08:46.561Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#16]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]

This is all that I get when I have Search Guard debug messages turned on.

2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestAuthenticationProcessor - Rest authentication request from 172.17.0.1 [original: /172.17.0.1:59210]
2023-06-01 15:03:51.991Z DEBUG [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authenticating request using: [session, basic/internal_users_db[11ce6527]]
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Checking authdomain session (total: 2)
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestRequestAuthenticationProcessor - Try to extract auth creds from session http authenticator
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestRequestAuthenticationProcessor - no session credentials found in request
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Checking authdomain basic/internal_users_db[11ce6527] (total: 2)
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestRequestAuthenticationProcessor - Try to extract auth creds from basic http authenticator
2023-06-01 15:03:51.991Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestRequestAuthenticationProcessor - no basic credentials found in request
2023-06-01 15:03:51.991Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]
2023-06-01 15:03:51.992Z DEBUG [elasticsearch[esnode-aln-nbadev4][transport_worker][T#50]] com.floragunn.searchguard.authc.rest.RestRequestAuthenticationProcessor - Sending WWW-Authenticate: Basic realm="Search Guard"

So I realize that this request is failing because it is missing credentials. How can I troubleshoot this further because it is automatically generated by Kibana every five minutes. Kibana works fine otherwise. Is there a flag in Kibana that I can turn off to stop these messages from being generated?

And with Elasticsearch trace messages turned on:

2023-06-01 15:59:07.564Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#47]] org.elasticsearch.http.HttpTracer - [16193][null][POST][/.kibana_7.17.9/_search?from=0&rest_total_hits_as_int=true&size=100] received request from [Netty4HttpChannel{localAddress=/172.17.0.7:9200, remoteAddress=/172.17.0.1:45042}]
2023-06-01 15:59:07.565Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#47]] org.elasticsearch.indices.breaker.in_flight_requests - [in_flight_requests] Adding [500b][<http_request>] to used bytes [new used: [500b], limit: 8589934592 [8gb], estimate: 1000 [1000b]]
2023-06-01 15:59:07.565Z WARN  [elasticsearch[esnode-aln-nbadev4][transport_worker][T#47]] com.floragunn.searchguard.authc.base.RequestAuthenticationProcessor - Authentication failed for null from [request=/.kibana_7.17.9/_search, directIpAddress=172.17.0.1, originatingIpAddress=172.17.0.1, clientCertSubject=null]
2023-06-01 15:59:07.565Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#47]] org.elasticsearch.indices.breaker.in_flight_requests - [in_flight_requests] Adjusted breaker by [-500] bytes, now [0]
2023-06-01 15:59:07.565Z TRACE [elasticsearch[esnode-aln-nbadev4][transport_worker][T#47]] org.elasticsearch.http.HttpTracer - [16193][null][UNAUTHORIZED][text/plain; charset=UTF-8][12] sent response to [Netty4HttpChannel{localAddress=/172.17.0.7:9200, remoteAddress=/172.17.0.1:45042}] success [true]

The team was able to reproduce this error, and will be working on a possible fix. You can follow the progress of this issue here: Authentication failed for null from [request=/.kibana_7.17.10/_search] (#454) · Issues · search-guard / Search Guard Kibana Plugin · GitLab