Invalid index name [_searchguard], must not start with '_'.]

trautw · March 17, 2021, 9:43pm

Elasticsearch version:
7.10.2
Server OS version:
ubuntu-18.04
Kibana version (if relevant):
7.10.2
Browser version (if relevant):
Chrome 89.0.4389.82

SearchGuard Plugins version
49.0.0

Describe the issue:

There are exceptions in elasticsearch logs when Kibana reloading:

{"type": "server", "timestamp": "2021-03-17T21:15:52,188Z", "level": "ERROR", "component": "c.f.s.f.SearchGuardFilter", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "Unexpected exception [_searchguard] InvalidIndexNameException[Invalid index name [_searchguard], must not start with '_'.]", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.indices.InvalidIndexNameException: Invalid index name [_searchguard], must not start with '_'.",
"at 
SKIPPED SEE LOG SECTION FOR FULL STACKTRACE
{"type": "server", "timestamp": "2021-03-17T21:15:52,197Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "path: /_searchguard/authtoken/_search, params: {index=_searchguard, type=authtoken}", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.ElasticsearchSecurityException: Unexpected exception indices:data/read/search",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply0(SearchGuardFilter.java:323) [
SKIPPED SEE LOG SECTION FOR FULL STACKTRACE

There are other errors when reloading of Kibana on Dev Tools page (seen /_searchguard/api/permissionsinfo):

{"type": "server", "timestamp": "2021-03-17T21:30:34,121Z", "level": "ERROR", "component": "c.f.s.f.SearchGuardFilter", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "Unexpected exception [_searchguard] InvalidIndexNameException[Invalid index name [_searchguard], must not start with '_'.]", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.indices.InvalidIndexNameException: Invalid index name [_searchguard], must not start with '_'.",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.validateAliasOrIndex(
    SKIPPED 
{"type": "server", "timestamp": "2021-03-17T21:30:34,127Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "path: /_searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.ElasticsearchSecurityException: Unexpected exception indices:data/read/get",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply0(SearchGuardFilter.java:323) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
    SKIPPED 
{"type": "server", "timestamp": "2021-03-17T21:30:34,724Z", "level": "ERROR", "component": "c.f.s.f.SearchGuardFilter", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "Unexpected exception [_searchguard] InvalidIndexNameException[Invalid index name [_searchguard], must not start with '_'.]", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.indices.InvalidIndexNameException: Invalid index name [_searchguard], must not start with '_'.",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.validateAliasOrIndex(
    SKIPPED 
{"type": "server", "timestamp": "2021-03-17T21:30:34,727Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "path: /_searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.ElasticsearchSecurityException: Unexpected exception indices:data/read/get",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply0(SearchGuardFilter.java:323) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
    SKIPPED

Steps to reproduce:

Reload Kibana
Check Elasticsearch logs

Expected behavior:
No exceptions

Provide configuration:
elasticsearch/config/elasticsearch.yml

network.host: 0.0.0.0
cluster.name: elasticsearch
cluster.initial_master_nodes: -elasticsearch-master-0
discovery.seed_hosts: elasticsearch-discovery.project
elasticsearch-discovery.project
node.name: ${HOSTNAME}
node.processors: 2
searchguard.enterprise_modules_enabled: false
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.ssl.transport.pemcert_filepath: certificates/node.pem
searchguard.ssl.transport.pemkey_filepath: certificates/node.key
searchguard.ssl.transport.pemtrustedcas_filepath: certificates/root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: certificates/node_http.pem
searchguard.ssl.http.pemkey_filepath: certificates/node_http.key
searchguard.ssl.http.pemtrustedcas_filepath: certificates/root-ca.pem
searchguard.nodes_dn:
- CN=node,O=project
searchguard.authcz.admin_dn:
- CN=sgadmin,O=project
action.auto_create_index: true
xpack.ml.enabled: ${XPACK_ML_ENABLED:false}
xpack.monitoring.enabled: ${XPACK_MONITORING_ENABLED:true}
xpack.security.enabled: ${XPACK_SECURITY_ENABLED:false}
xpack.watcher.enabled: ${XPACK_WATCHER_ENABLED:false}

elasticsearch/plugins/search-guard-7/sgconfig/sg_config.yml

_sg_meta:
  type: "config"
  config_version: 2
sg_config:
  dynamic:
    filtered_alias_mode: "warn"
    disable_rest_auth: false
    disable_intertransport_auth: false
    respect_request_indices_options: false
    license: null
    auth_failure_listeners: {}
    do_not_fail_on_forbidden: true
    multi_rolespan_enabled: false
    hosts_resolver_mode: "ip-only"
    transport_userrname_attribute: null
    do_not_fail_on_forbidden_empty: true
    field_anonymization_salt2: null
    kibana:
      multitenancy_enabled: false
      server_username: "system.kibanaserver"
      index: ".kibana"
      rbac_enabled: false
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        internalProxies: ".*"
        remoteIpHeader: "x-forwarded-for"
    authc:
      proxy_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          challenge: false
          type: "proxy"
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: "noop"
          config: {}
        skip_users: []
      clientcert_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 0
        http_authenticator:
          challenge: false
          type: "clientcert"
          config:
            username_attribute: "cn"
        authentication_backend:
          type: "noop"
          config: {}
        skip_users: []
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 2
        http_authenticator:
          challenge: true
          type: "basic"
          config: {}
        authentication_backend:
          type: "intern"
          config: {}
        skip_users: []
    authz:
      roles_from_another_ldap:
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: "ldap"
          config: {}
        skipped_users: []
      roles_from_myldap:
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: "ldap"
          config:
            enable_ssl: false
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            hosts:
            - "localhost:8389"
            bind_dn: null
            password: null
            rolebase: "ou=groups,dc=example,dc=com"
            rolesearch: "(member={0})"
            userroleattribute: null
            userrolename: "disabled"
            rolename: "cn"
            resolve_nested_roles: true
            userbase: "ou=people,dc=example,dc=com"
            usersearch: "(uid={0})"
        skipped_users: []

Provide logs:
Elasticsearch

Summary

{"type": "server", "timestamp": "2021-03-17T21:15:52,188Z", "level": "ERROR", "component": "c.f.s.f.SearchGuardFilter", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "Unexpected exception [_searchguard] InvalidIndexNameException[Invalid index name [_searchguard], must not start with '_'.]", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.indices.InvalidIndexNameException: Invalid index name [_searchguard], must not start with '_'.",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.validateAliasOrIndex(IndexNameExpressionResolver.java:922) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.innerResolve(IndexNameExpressionResolver.java:859) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:838) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:217) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndexNames(IndexNameExpressionResolver.java:197) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndexNames(IndexNameExpressionResolver.java:126) ~[elasticsearch-7.10.2.jar:7.10.2]",
"at com.floragunn.searchguard.resolver.IndexResolverReplacer.resolveIndexPatterns(IndexResolverReplacer.java:237) ~[search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.resolver.IndexResolverReplacer$2.provide(IndexResolverReplacer.java:313) ~[search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.resolver.IndexResolverReplacer.getOrReplaceAllIndices(IndexResolverReplacer.java:783) ~[search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.resolver.IndexResolverReplacer.resolveRequest(IndexResolverReplacer.java:308) ~[search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:210) ~[search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply0(SearchGuardFilter.java:290) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardFilter.lambda$apply$0(SearchGuardFilter.java:122) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:46) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.lambda$provide$0(SpecialPrivilegesEvaluationContextProviderRegistry.java:41) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.internalauthtoken.InternalAuthTokenProvider.userAuthFromToken(InternalAuthTokenProvider.java:96) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:37) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:28) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:118) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:155) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:83) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:86) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:90) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:412) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$2(RestSearchAction.java:121) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:115) [elasticsearch-7.10.2.jar:7.10.2]",
"at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.lambda$handleRequest$0(SearchGuardRestFilter.java:93) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.lambda$checkCurrentAuthenticationDomain$1(RestAuthenticationProcessor.java:274) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.api.SyncAuthenticationBackend.authenticate(SyncAuthenticationBackend.java:53) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.callAuthczBackends(RestAuthenticationProcessor.java:323) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkCurrentAuthenticationDomain(RestAuthenticationProcessor.java:240) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkNextAuthenticationDomains(RestAuthenticationProcessor.java:116) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.authenticate(RestAuthenticationProcessor.java:108) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:403) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:86) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:258) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:340) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:191) [elasticsearch-7.10.2.jar:7.10.2]",
"at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:319) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:384) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:309) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:42) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:28) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at 
SKIPPED DUE TO DESCRIPTION LIMIT SIZE

"at java.lang.Thread.run(Thread.java:832) [?:?]"] }
{"type": "server", "timestamp": "2021-03-17T21:15:52,197Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client-7844bf6ff9-p5h8m", "message": "path: /_searchguard/authtoken/_search, params: {index=_searchguard, type=authtoken}", "cluster.uuid": "Rn77rhBRQqyiHvIqHvA0SA", "node.id": "HZnuNQpPRjOsYWW733Hamg" , 
"stacktrace": ["org.elasticsearch.ElasticsearchSecurityException: Unexpected exception indices:data/read/search",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply0(SearchGuardFilter.java:323) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardFilter.lambda$apply$0(SearchGuardFilter.java:122) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:46) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.lambda$provide$0(SpecialPrivilegesEvaluationContextProviderRegistry.java:41) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.internalauthtoken.InternalAuthTokenProvider.userAuthFromToken(InternalAuthTokenProvider.java:96) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:37) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.privileges.SpecialPrivilegesEvaluationContextProviderRegistry.provide(SpecialPrivilegesEvaluationContextProviderRegistry.java:28) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:118) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:155) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:83) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:86) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:90) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:412) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$2(RestSearchAction.java:121) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:115) [elasticsearch-7.10.2.jar:7.10.2]",
"at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.lambda$handleRequest$0(SearchGuardRestFilter.java:93) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.lambda$checkCurrentAuthenticationDomain$1(RestAuthenticationProcessor.java:274) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.api.SyncAuthenticationBackend.authenticate(SyncAuthenticationBackend.java:53) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.callAuthczBackends(RestAuthenticationProcessor.java:323) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkCurrentAuthenticationDomain(RestAuthenticationProcessor.java:240) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkNextAuthenticationDomains(RestAuthenticationProcessor.java:116) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.RestAuthenticationProcessor.authenticate(RestAuthenticationProcessor.java:108) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:403) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:86) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:258) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:340) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:191) [elasticsearch-7.10.2.jar:7.10.2]",
"at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-suite-security-7.10.2-49.0.0.jar:7.10.2-49.0.0]",
"at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:319) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:384) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:309) [elasticsearch-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:42) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:28) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.10.2.jar:7.10.2]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
SKIPPED DUE TO DESCRIPTION LIMIT SIZE

Additional data:
GET /_searchguard/authinfo

{
  "user" : "User [name=admin, backend_roles=[offline_access, admin, uma_authorization, user], requestedTenant=null]",
  "user_name" : "admin",
  "user_requested_tenant" : null,
  "remote_address" : "127.0.0.1:46550",
  "backend_roles" : [
    "offline_access",
    "admin",
    "uma_authorization",
    "user"
  ],
  "custom_attribute_names" : [ ],
  "attribute_names" : [ ],
  "sg_roles" : [
    "SGS_ALL_ACCESS",
    "SGS_KIBANA_USER",
    "sg_kibana_user"
  ],
  "sg_tenants" : {
    "admin" : true,
    "SGS_GLOBAL_TENANT" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}

nils · March 18, 2021, 8:16am

Hello and thanks a lot for the detailed report!

One further question: Do you see any functional degradation or is it “just” the log messages?

trautw · March 18, 2021, 9:08am

Hello. It is “just” the log (but huge repetitive embarrassing log)

nils · March 22, 2021, 7:47am

One update for this: We took a closer look at the problem and could find the cause. You can track the progress of the issue here:

system · April 12, 2021, 7:48am

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
In my Elastic Search server, i get this Search guard index error Search Guard	5	18	September 4, 2024
Invalid index name [_xpack], must not start with '_' Search Guard	7	3670	August 14, 2020
Elasticsearch SIEM with Search Guard Search Guard	3	667	February 25, 2022
Elasticsearch not starting after installing Searchguard version 41.0 Search Guard	3	2416	July 22, 2020
Java Lang Illegal Exception , failed to load plugin class Search Guard	5	546	October 21, 2022

Invalid index name [_searchguard], must not start with '_'.]

Related Topics