com.floragunn.searchguard.filter.SearchGuardRestFilter error

I am occastionally getting the following Elasticsearch search-guard error, but there are no other log messages around it that provide any more information. I can’t even find an application that is obviously failing when this occurs, so I’m wondering what this error means.

2023-03-02 15:15:45.760Z ERROR [elasticsearch[esnode-aln-nbadev4][transport_worker][T#26]] com.floragunn.searchguard.filter.SearchGuardRestFilter - Error in org.elasticsearch.xpack.search.RestSubmitAsyncSearchAction@7af0d6d4
java.lang.IllegalArgumentException: Required one of fields [field, script], but none were specified.
at org.elasticsearch.xcontent.ObjectParser.throwMissingRequiredFields(ObjectParser.java:349) ~[elasticsearch-x-content-7.17.9.jar:7.17.9]
at org.elasticsearch.xcontent.ObjectParser.parse(ObjectParser.java:331) ~[elasticsearch-x-content-7.17.9.jar:7.17.9]
at org.elasticsearch.xcontent.ObjectParser.parse(ObjectParser.java:260) ~[elasticsearch-x-content-7.17.9.jar:7.17.9]
at org.elasticsearch.search.SearchModule.lambda$registerAggregation$2(SearchModule.java:693) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:141) ~[elasticsearch-x-content-7.17.9.jar:7.17.9]
at org.elasticsearch.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:408) ~[elasticsearch-x-content-7.17.9.jar:7.17.9]
at org.elasticsearch.search.aggregations.AggregatorFactories.parseAggregators(AggregatorFactories.java:155) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.search.aggregations.AggregatorFactories.parseAggregators(AggregatorFactories.java:62) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.search.builder.SearchSourceBuilder.parseXContent(SearchSourceBuilder.java:1251) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.rest.action.search.RestSearchAction.parseSearchRequest(RestSearchAction.java:160) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.rest.action.search.RestSearchAction.parseSearchRequest(RestSearchAction.java:140) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.xpack.search.RestSubmitAsyncSearchAction.lambda$prepareRequest$1(RestSubmitAsyncSearchAction.java:49) ~[?:?]
at org.elasticsearch.rest.RestRequest.withContentOrSourceParamParserOrNull(RestRequest.java:513) ~[elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.xpack.search.RestSubmitAsyncSearchAction.prepareRequest(RestSubmitAsyncSearchAction.java:48) ~[?:?]
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:86) ~[elasticsearch-7.17.9.jar:7.17.9]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.lambda$handleRequest$0(SearchGuardRestFilter.java:97) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.RestAuthenticationProcessor.lambda$checkCurrentAuthenticationDomain$1(RestAuthenticationProcessor.java:275) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.RestAuthenticationProcessor.callAuthczBackends(RestAuthenticationProcessor.java:329) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkCurrentAuthenticationDomain(RestAuthenticationProcessor.java:241) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.RestAuthenticationProcessor.checkNextAuthenticationDomains(RestAuthenticationProcessor.java:117) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.RestAuthenticationProcessor.authenticate(RestAuthenticationProcessor.java:109) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.auth.BackendRegistry.authenticate(BackendRegistry.java:405) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:90) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:337) [elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:403) [elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:255) [elasticsearch-7.17.9.jar:7.17.9]
at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-suite-security-7.17.9-53.6.0.jar:7.17.9-53.6.0]
at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:382) [elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:461) [elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:357) [elasticsearch-7.17.9.jar:7.17.9]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:35) [transport-netty4-client-7.17.9.jar:7.17.9]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:19) [transport-netty4-client-7.17.9.jar:7.17.9]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:48) [transport-netty4-client-7.17.9.jar:7.17.9]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [netty-codec-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]

Elasticsearch version: 7.17.9

Server OS version: 7.17.9-53.6.0

Kibana version (if relevant): 7.17.9

Browser version (if relevant): N/A

Browser OS version (if relevant):

Hi!

I do not think that this indicates an issue in Search Guard. Search Guard just catches an error inside an Elasticsearch action and logs it - possibly Search Guard should not log it at an error level though.

The error indicates that some client sends a async search action, but an aggregation specified inside was invalid. Kibana is known to send async search actions. But possibly, it is triggered by some visualisation in Kibana?

To find our more about the source of the error, you can try to log the name of the user associated with the failing request. To do so, modify your log4j2 logging pattern to include %X{user}

This was probably an Elasticsearch configuration issue. I’m not sure how I fixed it, but these errors are no longer appearing.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.