Elasticsearch not starting after installing Searchguard

[rus]

If you think it is a bug report or you have a technical issue, please answer the following questions. For general questions, you can delete these questions.

Elasticsearch version: 7.1.42

Server OS version: Ubuntu 20.0.4

Kibana version (if relevant):

Browser version (if relevant):

Browser OS version (if relevant):

Describe the issue: after installing Searchguard for Elasticsearch 7.14.2 it doesn’t start

Steps to reproduce:
0. I stopped elasticsearch instance

1. I installed the plugin: bin/elasticsearch-plugin install -b file:///path/to/search-guard-7-<version>.zip
2. I generated demo certificates from ./install_demo_configuration.sh
3. I checked elasticsearch.yml config, the lines regarding searchguard configuration were added
4. I tried to start elasticsearch, I have following errors:

Caused by: java.lang.IllegalArgumentException: Cannot have additional setting [http.compression] in plugin [search-guard-7], already added in plugin [x-pack-security]
        at org.elasticsearch.plugins.PluginsService.updatedSettings(PluginsService.java:210) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.node.Node.<init>(Node.java:361) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.node.Node.<init>(Node.java:281) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.14.2.jar:7.14.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.14.2.jar:7.14.2]

Expected behavior: —

Provide configuration:
elasticsearch/config/elasticsearch.yml

# Transport layer
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.transport.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt

# HTTP layer

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.http.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt

# Elasticsearch authentication
#xpack.security.enabled: true

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch


######## Start Search Guard Demo Configuration ########
# WARNING: revise all the lines below before you go into production
searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]
cluster.routing.allocation.disk.threshold_enabled: false
cluster.name: searchguard_demo
node.max_local_storage_nodes: 3


searchguard.enterprise_modules_enabled: false

elasticsearch/plugins/search-guard-7/sgconfig/sg_config.yml - its default

kibana/config/kibana.yml (if relevant) - n/a

Provide logs:
Elasticsearch
Kibana (if relevant)

Screenshots (if relevant):

Errors in browser console (if relevant):

Additional data:

[pablo]

@rus You’ll have to disable xpack.security plugin and remove xpack.security.transport and xpack.security.ssl configurations. Search Guard will take over these security settings.

[rus]

Oh, I see, thanks. Two questions:

1. how to disable xpack.security plugin? I removed xpack config from elasticsearch.yml file but it still doesn’t work
2. Is it possible to deploy searchguard without these security settings in searchguard plugin? Currently I use elasticsearch with wazuh plugin for kibana, and it’s a lot of work for me to replace these settings as I have over >10 nodes in the cluster

[pablo]

@rus Set xpack.security.enabled: false in elasticserach.yml.

For question 2 the answer is no. You have to define the minimum configuration for the Search Guard plug-in
searchguard.ssl.transport is mandatory in Search Guard.
You can turn off SSL in searchguard.ssl.http. However, you’ll move from HTTP to HTTPS then and that is not recommended.

You can find more information Search Guard documentation.

Search Guard Installation

[rus]

ok, now it works, thanks what about second question? is it possible to disable all of these security settings for the plugin?

[rus]

@pablo sorry for taking up your time, just one more question: after installing of searchguard kibana plugin, kibana is starting but it stucks on “Kibana server is not ready yet”. Do you have any solution for this?

it’s my kibana.yml config:

server.host: 0.0.0.0
server.port: 443
elasticsearch.hosts: https://localhost:9200
elasticsearch.password: ...

# Elasticsearch from/to Kibana

elasticsearch.ssl.certificateAuthorities: /etc/kibana/certs/ca/ca.crt
elasticsearch.ssl.certificate: /etc/kibana/certs/kibana.crt
elasticsearch.ssl.key: /etc/kibana/certs/kibana.key

# Browser from/to Kibana
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana.crt
server.ssl.key: /etc/kibana/certs/kibana.key

# Elasticsearch authentication
xpack.security.enabled: false
elasticsearch.username: elastic
uiSettings.overrides.defaultRoute: "/app/wazuh"
elasticsearch.ssl.verificationMode: none
telemetry.banner: false

[pablo]

@rus Try changing elasticsearch credentials to “kibanaserver”.

elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"

Installing the Plugin