Error executing alerting apiKey

Please does anyone know how to get rid of this WARNING messages in kibana logs ? It’s being logged every 10 minutes.

Elasticsearch version:
7.15.1
Kibana sg plugin version:
52.1.0

{"ecs":{"version":"1.9.0"},"@timestamp":"2021-11-30T09:16:50.852+00:00","message":"Error executing alerting apiKey invalidation task: Response Error","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":1219}}
{"ecs":{"version":"1.9.0"},"@timestamp":"2021-11-30T09:26:53.870+00:00","message":"Error executing alerting apiKey invalidation task: Response Error","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":1219}}
{"ecs":{"version":"1.9.0"},"@timestamp":"2021-11-30T09:36:56.933+00:00","message":"Error executing alerting apiKey invalidation task: Response Error","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":1219}}
{"ecs":{"version":"1.9.0"},"@timestamp":"2021-11-30T09:47:00.025+00:00","message":"Error executing alerting apiKey invalidation task: Response Error","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":1219}}

@peter82 Would you be able to provide your elasticsearch.yml and kibana.yml files to reproduce the issue? Feel free to redact any sensitive details.

I installed version 7.15.2 and it’s doing exactly the same.

kibana.yml

server.port: 5601
server.host: 0.0.0.0


server.publicBaseUrl: "https://kibana.cs.com/basePath"

server.basePath: /basePath
server.rewriteBasePath: false
server.name: server-1

elasticsearch.hosts:
  - "https://10.10.10.1:9200"
  - "https://10.10.10.2:9200"


elasticsearch.username: kibana_system
elasticsearch.password: xxxxxx

elasticsearch.requestTimeout: 30000
elasticsearch.shardTimeout: 30000

############################## XPACK ##############################

xpack.security.enabled: false
xpack.security.encryptionKey: something_at_least_32_characters
xpack.encryptedSavedObjects.encryptionKey: something_at_least_32_characters

xpack.ml.enabled: true

server.ssl.enabled: true

server.ssl.key: /usr/share/kibana/config/certificates/test-kibana.vs.cs.com.key
server.ssl.certificate: /usr/share/kibana/config/certificates/test-kibana.vs.cs.com.crt

elasticsearch.ssl.certificateAuthorities: [ "/usr/share/kibana/config/certificates/ca/cert1.cer", "/usr/share/kibana/config/certificates/ca/cert2.cer", "/usr/share/kibana/config/certificates/ca/cert3.cer" ]

elasticsearch.ssl.verificationMode: certificate

monitoring.enabled: true

#####################logging##########################################

logging:
  appenders:
    roll-file:
      type: rolling-file
      fileName: /usr/share/kibana/logs/kibana.log
      policy:
        type: time-interval
        interval: 24h
        modulate: true
      strategy:
        type: numeric
        pattern: '.%i'
        max: 2
      layout:
        type: json
  root:
    appenders: [roll-file,default]

#####################SSOSG############################################

searchguard.auth.type: "saml"
searchguard.auth.anonymous_auth_enabled: false

searchguard.cookie.isSameSite: None
searchguard.cookie.secure: true
searchguard.cookie.name: "cookie" 
searchguard.cookie.password: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

searchguard.session.keepalive: true

server.xsrf.whitelist: ["/searchguard/saml/acs", "/searchguard/saml/logout"]
elasticsearch.requestHeadersWhitelist: ["sgtenant", "Authorization", "X-Forwarded-For", "x-proxy-user", "x-proxy-roles", "urltoken", "jwtheader"]

searchguard.auth.debug: true

searchguard.basicauth.login.showbrandimage: false

searchguard.basicauth.forbidden_usernames: ["kibana_system"]

searchguard.multitenancy.enabled: true
searchguard.multitenancy.enable_filter: true

searchguard.multitenancy.tenants.enable_private: false

searchguard.multitenancy.tenants.enable_global: true

searchguard.multitenancy.tenants.preferred:  [ "tenant1", "Global" ]

xpack.spaces.enabled: false

searchguard.readonly_mode.roles:  []

elasticsearch.yml

cluster.name: basePath
node.name: node1

node.master: true
node.voting_only: false
node.data: true
node.ingest: true
node.ml: false
node.remote_cluster_client: false
node.attr.zone: primary
node.attr.tier: hot

path.data: /data_path/esdata01
path.logs: /log_path/eslog01


bootstrap.memory_lock: false

network.host: 10.10.10.1
network.tcp.keep_alive: true
network.tcp.keep_idle: 300

http.port: 9201
transport.port: 9301

transport.tcp.keep_alive: true
transport.compress: true

discovery.seed_hosts:
  - 10.10.10.1:9301
  - 10.10.10.2:9301
  - 10.10.10.3:9301

cluster.initial_master_nodes:
  - master1
  - master2
  - master3

http.cors.enabled: false
http.cors.allow-origin: /.*/


############################## x-pack ##############################
xpack.security.enabled: false

############################### SearchGuard ##############################

searchguard.ssl.transport.pemcert_filepath: /absolute_config_path/certificates/cert.crt.pem
searchguard.ssl.transport.pemkey_filepath: /absolute_config_path/certificates/cert.key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: /absolute_config_path/certificates/ca/ca_bundle.pem

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.authcz.admin_dn:
  - CN=admin,OU=cs,O=cs,C=COM
searchguard.nodes_dn:
  - CN=cert,OU=cs,O=cs,C=COM

searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: /absolute_config_path/certificates/cert.crt.pem
searchguard.ssl.http.pemkey_filepath: /absolute_config_path/certificates/cert.key.pem
searchguard.ssl.http.pemtrustedcas_filepath: /absolute_config_path/certificates/ca/ca_bundle.pem

searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]
searchguard.dfm_empty_overrides_all: true


@peter82
It does not seem to be possible to disable these warning as Alerting module appears to be built in to this elasticsearch version.
I assume apart from these annoying warning the rest of the functionality is working as expected?

yes it does not impact functionality as far as I know