Empty response - elasticsearch and kibana not working after trying to set up Search Guard

Search Guard
1

Hi
I have the exact same error as in this post.

I installed Elastic & Kibana following the instructions from the ES site
As soon as I install searchguard I cannot connect to ES and get

root@es6:/etc/elasticsearch# curl -X GET "127.0.0.1:9200/?pretty"
**curl: (52) Empty reply from server**

I’m using ES version 6.5.4 (we have to stick to older version for compatability with other project) with Ubunto and Oracle VitualBox Virtal machine

I tried the instruction in the prior post and it gives me this error about certificates but I don’t know what it means ?

##############

root@es6:/usr/share/elasticsearch/plugins/search-guard-6/tools# curl -v -k 'https://localhost:9200/_searchguard/api/internalusers' -u admin:admin -1
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: DC=de; L=test; O=node; OU=node; CN=node-0.example.com
*  start date: Apr 22 03:43:47 2018 GMT
*  expire date: Apr 19 03:43:47 2028 GMT
*  issuer: DC=com; DC=example; O=Example Com Inc.; OU=Example Com Inc. Root CA; CN=Example Com Inc. Root CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using Basic with user 'admin'
> GET /_searchguard/api/internalusers HTTP/1.1
> Host: localhost:9200
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 383
<
* Connection #0 to host localhost left intact
{"admin":{"attributes":{"attribute1":"value1","attribute3":"value3","attribute2":"value2"},"readonly":"true","hash":"","roles":["admin"]},"logstash":{"hash":"","roles":["logstash"]},"kibanaserver":{"readonly":"true","hash":""},"kibanaro":{"hash":"","roles":["kibanauser","readall"]},"readall":{"hash":"","roles":["readall"]},"snapshotrestore":{"hash":"","roles":["snapshotrestore"]}}root@es6:/usr/share/elasticsearch/plugins/search-guard-6/tools# 


##############

elasticsearch.yml file:

 ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
 ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

######## Start Search Guard Demo Configuration ########
# WARNING: revise all the lines below before you go into production
searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
cluster.routing.allocation.disk.threshold_enabled: false
cluster.name: searchguard_demo
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3
xpack.security.enabled: false
######## End Search Guard Demo Configuration ########
2

Why do you think there are any errors? I don’t see any errors. I see you sent a query to list all internal users. As a result, you got the connection handshake messages and all the internal users

{"admin":{"attributes":{"attribute1":"value1","attribute3":"value3","attribute2":"value2"},"readonly":"true","hash":"","roles":["admin"]},"logstash":{"hash":"","roles":["logstash"]},"kibanaserver":{"readonly":"true","hash":""},"kibanaro":{"hash":"","roles":["kibanauser","readall"]},"readall":{"hash":"","roles":["readall"]},"snapshotrestore":{"hash":"","roles":["snapshotrestore"]}}

If you don’t need the connection handshake messages, don’t use -v for your curl command. Also, you don’t need -l, it is for FTP and EMAIL. Execute man curl to know more about curl options.

3

I confused -1 option with -l. You don’t need to force curl to use TLSv1. Just omit -1 option.

4

Hi thank you very much for replying.

I assume its not running as I can’t connect to ES or Kibana
http://127.0.0.1:5601/app/kibana#/visualize/new
127.0.0.1:9200

also the next step in the Searchguard install is
Test HTTPS encryption
Use a browser and open https://localhost:9200/_searchguard/authinfo?pretty
Accept the warning message regarding self-signed demo certificates

however I get this in Firefox

Secure Connection Failed
An error occurred during a connection to 127.0.0.1:9200. PR_CONNECT_ABORTED_ERROR

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the web site owners to inform them of this problem.

and this in IE
Can’t connect securely to this page
This might be because the site uses outdated or unsafe TLS security settings. If this keeps happening, try contacting the website’s owner.
Your TLS security settings aren’t set to the defaults, which could also be causing this error.

In Internet options I have ticked
Use SSL 3.0
Use TLS 1.0
Use TLS 1.1
Use TLS 1.2

5

Did you try Chrome? Update Firefox and try again. Do you have any security extension, for example, an antivirus? It can block some TLS connections based on its rules. More information can be found here Secure connection failed and Firefox did not connect | Firefox Help

6

Hi
Could not fathom it out so I started again from scratch I have now got ES returning a reply using
https://127.0.0.1:9200/_searchguard/authinfo?pretty

user “User [name=admin, roles=[admin], requestedTenant=null]”
user_name “admin”
user_requested_tenant null
remote_address “10.0.2.2:49223”
backend_roles
0 “admin”
custom_attribute_names
0 “attr.internal.attribute1”
1 “attr.internal.attribute2”
2 “attr.internal.attribute3”
sg_roles
0 “sg_all_access”
1 “sg_own_index”
sg_tenants
admin_tenant true
admin true
principal null
peer_certificates “0”
sso_logout_url null

I also get a reply using
https://127.0.0.1:9200/_searchguard/health

message null
mode “strict”
status “UP”

can also get a reply from ES https://127.0.0.1:9200
… |tagline|“You Know, for Search”|

however no luck accessing Kibana I get
https://127.0.0.1:5601/
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
or
http://127.0.0.1:5601/
Kibana server is not ready yet

I think it might be to do with some of the lines in the 2 yml files I have uncommented, Im unclear if they should be localhost, 127.0.0.1, or 0.0.0.0 I have tried various different combinations to get to this point.

I could access Kibana before I started fiddling with Searchguard install!

How should these lines be set in relation to each other?

kibana.yml
server.port: 5601
server.host: “0.0.0.0”
elasticsearch.url: “https://localhost:9200”
elasticsearch.hosts: [“https://localhost:9200”]

elasticsearch.yml
network.host: localhost

Set a custom port for HTTP:

http.port: 9200

Accessing via Ubunto… Thanks

7

I see no SearchGuard related configuration in kibana.yml. To make it work you need:

  1. Configure SearchGuard Elasticsearch plugin
  2. Configure SearchGuard Kibana plugin

You should have at least this config in kibana.yml

# Use HTTPS instead of HTTP
elasticsearch.hosts: "https://localhost:9200"

# Configure the Kibana internal server user
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"

# Disable SSL verification because we use self-signed demo certificates
elasticsearch.ssl.verificationMode: none

# Whitelist the Search Guard Multi Tenancy Header
elasticsearch.requestHeadersWhitelist: [ "Authorization", "sgtenant" ]

# X-Pack security needs to be disabled for Search Guard to work properly
xpack.security.enabled: false

Also, I would recommend you to read the following docs:

8

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.