Search Guard Kibana GUI cannot display on Navigation Pane

Hi I have installed the Search Guard Kibana plugin and running Search Guard on Enterprise license. The Search Guard entry however are not seen on Kibana navigation pane.

I have 2 nodes which are aws instances, the certificates are generated by offline TLS tool (self signed).
The kibana is installed on another client and point to node1.

I a able to login to the search guard login page with login “kibanaserver”

kibana version: 6.2.4
kibana search guard plugin: 6.2.4-13

Elasticsearch version on nodes: 6.2.4

Searchguard version: 6.2.4-22.1

My kibana.yml


# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#server.host: "localhost"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects
# the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests
# to Kibana. This setting cannot end in a slash.
#server.basePath: ""
# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576
# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"
# The URL of the Elasticsearch instance to use for all your queries.
elasticsearch.url: "https://126.21.33.72:9200/"
# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true
# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#kibana.index: ".kibana"
# The default application to load.
#kibana.defaultAppId: "home"
# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full
# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500
# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000
# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]
# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}
# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 0
# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000
# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid
# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout
# Set the value of this setting to true to suppress all logging output.
#logging.silent: false
# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false
# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false
# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000
# The default locale. This locale can be used in certain circumstances to substitute any missing
# translations.
#i18n.defaultLocale: "en"
tilemap.url: https://a.tile.openstreetmap.org/{z}/{x}/{y}.png
tilemap.options.minZoom: 0
tilemap.options.maxZoom: 20

My elasticsearch.yml

======================== Elasticsearch Configuration =========================

···

NOTE: Elasticsearch comes with reasonable defaults for most settings.

Before you set out to tweak and tune the configuration, make sure you

understand what are you trying to accomplish and the consequences.

The primary way of configuring a node is via this file. This template lists

the most important settings you may want to configure for a production cluster.

Please consult the documentation for further information on configuration options:

Elasticsearch Guide | Elastic

---------------------------------- Cluster -----------------------------------

Use a descriptive name for your cluster:

cluster.name: mycluster

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

node.name: node-1

Add custom attributes to the node:

#node.attr.rack: r1

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/lib/elasticsearch

Path to log files:

path.logs: /var/log/elasticsearch

----------------------------------- Memory -----------------------------------

Lock the memory on startup:

#bootstrap.memory_lock: true

Make sure that the heap size is set to about half the memory available

on the system and that the owner of the process is allowed to use this

limit.

Elasticsearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

Set the bind address to a specific IP (IPv4 or IPv6):

network.host: 0.0.0.0

Set a custom port for HTTP:

#http.port: 9200

For more information, consult the network module documentation.

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when new node is started:

The default list of hosts is [“127.0.0.1”, “[::1]”]

discovery.zen.ping.unicast.hosts: [“126.21.33.72”, “126.21.31.70”]

Prevent the “split brain” by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):

#discovery.zen.minimum_master_nodes:

For more information, consult the zen discovery module documentation.

---------------------------------- Gateway -----------------------------------

Block initial recovery after a full cluster restart until N nodes are started:

#gateway.recover_after_nodes: 3

For more information, consult the gateway module documentation.

---------------------------------- Various -----------------------------------

Require explicit names when deleting indices:

#action.destructive_requires_name: true
searchguard.ssl.transport.pemcert_filepath: node1.pem
searchguard.ssl.transport.pemkey_filepath: node1.key
searchguard.ssl.transport.pemkey_password: EW8i329pL1
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: node1_http.pem
searchguard.ssl.http.pemkey_filepath: node1_http.key
searchguard.ssl.http.pemkey_password: Rg43sij90L
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.nodes_dn:

  • CN=node1.example.com,OU=Ops,O=example Com, Inc.,DC=example,DC=com
  • CN=node2.example .com,OU=Ops,O=example Com, Inc.,DC=example,DC=com
    searchguard.authcz.admin_dn:
  • CN=kirk.example.com,OU=Ops,O=Example Com, Inc.,DC=example,DC=com
    searchguard.restapi.roles_enabled: [“sg_all_access”, …]
    searchguard.enterprise_modules_enabled: false

[https://126.21.33.72:9200/_searchguard/health](https://126.21.33.72:9200/_searchguard/health)

{“message”:null,“mode”:“strict”,“status”:“UP”}

For:
[https://126.21.33.72:9200/_searchguard/authinfo](https://126.21.33.72:9200/_searchguard/authinfo)

{“user”:“User [name=admin, roles=[admin], requestedTenant=null]”,“user_name”:“admin”,“user_requested_tenant”:null,“remote_address”:“112.207.17.41:43812”,“backend_roles”:[“admin”],“custom_attribute_names”:[“attr.internal.attribute1”,“attr.internal.attribute2”,“attr.internal.attribute3”],“sg_roles”:[“sg_all_access”,“sg_own_index”],“sg_tenants”:{“admin_tenant”:true,“admin”:true},“principal”:null,“peer_certificates”:“0”}


[<img src="https://lh3.googleusercontent.com/-368AOZG3lNE/WyCeBu-SW-I/AAAAAAAABKo/LA3zetVY0b8VwSV-8irPEsGyJ_Jg4ND8ACLcBGAs/s1600/41218150-b6b9ee58-6d8c-11e8-9b02-5a9b8db49612.png" border="0" style="">](https://lh3.googleusercontent.com/-368AOZG3lNE/WyCeBu-SW-I/AAAAAAAABKo/LA3zetVY0b8VwSV-8irPEsGyJ_Jg4ND8ACLcBGAs/s1600/41218150-b6b9ee58-6d8c-11e8-9b02-5a9b8db49612.png)

As i am using self signed certificates, am i missing out configuration in sg_roles or sg_mapping_roles? If yes, can you also show how to configure the user in the sg_roles?
I am now using login: kibanaserver and pwd: kibanaserver.
Thanks.


</details>

kibanaserver is a service account which kibana uses itself to connect to elasticsearch under the hood.
It's not for end user purposes. Login with admin user to see the Search Guard admin entry in the nav pane.
But keep in mind that the Admin UI is a enterprise feature where you need to pay for and you have to set
"searchguard.enterprise_modules_enabled: true" to make it work. Same is for multitenancy. With no enterprise modules
enabled there is no Search Guard nav entry.

See

···

Am 13.06.2018 um 06:33 schrieb YS .L <lyeesim988@gmail.com>:

Hi I have installed the Search Guard Kibana plugin and running Search Guard on Enterprise license. The Search Guard entry however are not seen on Kibana navigation pane.

I have 2 nodes which are aws instances, the certificates are generated by offline TLS tool (self signed).
The kibana is installed on another client and point to node1.

I a able to login to the search guard login page with login "kibanaserver"

kibana version: 6.2.4
kibana search guard plugin: 6.2.4-13
Elasticsearch version on nodes: 6.2.4
Searchguard version: 6.2.4-22.1

My kibana.yml

# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
#server.host: "localhost"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects
# the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests
# to Kibana. This setting cannot end in a slash.
#server.basePath: ""

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"

# The URL of the Elasticsearch instance to use for all your queries.
elasticsearch.url: "https://126.21.33.72:9200/&quot;

# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
#kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 0

# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000

# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid

# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# The default locale. This locale can be used in certain circumstances to substitute any missing
# translations.
#i18n.defaultLocale: "en"
tilemap.url: https://a.tile.openstreetmap.org/{z}/{x}/{y}.png
tilemap.options.minZoom: 0
tilemap.options.maxZoom: 20

My elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# Elasticsearch Guide | Elastic
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: mycluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["126.21.33.72", "126.21.31.70"]

#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
searchguard.ssl.transport.pemcert_filepath: node1.pem
searchguard.ssl.transport.pemkey_filepath: node1.key
searchguard.ssl.transport.pemkey_password: EW8i329pL1
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: node1_http.pem
searchguard.ssl.http.pemkey_filepath: node1_http.key
searchguard.ssl.http.pemkey_password: Rg43sij90L
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.nodes_dn:
- CN=node1.example.com,OU=Ops,O=example Com\, Inc.,DC=example,DC=com
- CN=node2.example .com,OU=Ops,O=example Com\, Inc.,DC=example,DC=com
searchguard.authcz.admin_dn:
- CN=kirk.example.com,OU=Ops,O=Example Com\, Inc.,DC=example,DC=com
searchguard.restapi.roles_enabled: ["sg_all_access", ...]
searchguard.enterprise_modules_enabled: false

https://126.21.33.72:9200/_searchguard/health

{"message":null,"mode":"strict","status":"UP"}
For:
https://126.21.33.72:9200/_searchguard/authinfo

{"user":"User [name=admin, roles=[admin], requestedTenant=null]","user_name":"admin","user_requested_tenant":null,"remote_address":"112.207.17.41:43812","backend_roles":["admin"],"custom_attribute_names":["attr.internal.attribute1","attr.internal.attribute2","attr.internal.attribute3"],"sg_roles":["sg_all_access","sg_own_index"],"sg_tenants":{"admin_tenant":true,"admin":true},"principal":null,"peer_certificates":"0"}

As i am using self signed certificates, am i missing out configuration in sg_roles or sg_mapping_roles? If yes, can you also show how to configure the user in the sg_roles?
I am now using login: kibanaserver and pwd: kibanaserver.

Thanks.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/c0858a49-1956-439d-9317-509c2e3b56dd%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

Hello,

I have included :

elasticsearch.ssl.certificateAuthorities: ["/etc/elasticsearch/root-ca.pem"]

in my kibana.yml but I still get the error:

log [07:32:23.184] [error][status][plugin:elasticsearch@6.2.4] Status changed from yellow to red - Request Timeout after 3000ms
  log [07:32:32.190] [error][admin][elasticsearch] Request error, retrying
HEAD https://52.21.217.85:9200/ => unable to get local issuer certificate
  log [07:32:33.038] [warning][admin][elasticsearch] Unable to revive connection

My kibana is running on Windows but elasticsearch is on Linux. Do I need to download a copy of the root ca.pem to Windows? now the elasticsearch.ssl.certificateAuthorities: ["/etc/elasticsearch/root-ca.pem"] is pointing to Linux. Can you advise me on this?

Thanks.

I don’t fully understand. Do you mean you are running Kibana on Windows, and want to use certificates which are on a different (Linux) machine? That, of course, can’t possibly work. The certificate has to be readable by the machine you run Kibana on.

···

On Monday, June 25, 2018 at 10:59:23 AM UTC+2, YS .L wrote:

Hello,
I have included :

elasticsearch.ssl.certificateAuthorities: [“/etc/elasticsearch/root-ca.pem”]

in my kibana.yml but I still get the error:

log [07:32:23.184] [error][status][plugin:elasticsearch@6.2.4] Status changed from yellow to red - Request Timeout after 3000ms
log [07:32:32.190] [error][admin][elasticsearch] Request error, retrying
HEAD https://52.21.217.85:9200/ => unable to get local issuer certificate
log [07:32:33.038] [warning][admin][elasticsearch] Unable to revive connection

My kibana is running on Windows but elasticsearch is on Linux. Do I need to download a copy of the root ca.pem to Windows? now the elasticsearch.ssl.certificateAuthorities: [“/etc/elasticsearch/root-ca.pem”] is pointing to Linux. Can you advise me on this?

Thanks.

Hi Jochen,

Thanks for the reply. I got what you mean. After I install the kibana on Linux server and point to the root ca that I have generated, it works perfectly. Thank you for clearing up my confusion.

···

On Monday, 25 June 2018 23:08:00 UTC+8, Jochen Kressin wrote:

I don’t fully understand. Do you mean you are running Kibana on Windows, and want to use certificates which are on a different (Linux) machine? That, of course, can’t possibly work. The certificate has to be readable by the machine you run Kibana on.

On Monday, June 25, 2018 at 10:59:23 AM UTC+2, YS .L wrote:

Hello,
I have included :

elasticsearch.ssl.certificateAuthorities: [“/etc/elasticsearch/root-ca.pem”]

in my kibana.yml but I still get the error:

log [07:32:23.184] [error][status][plugin:elasticsearch@6.2.4] Status changed from yellow to red - Request Timeout after 3000ms
log [07:32:32.190] [error][admin][elasticsearch] Request error, retrying
HEAD https://52.21.217.85:9200/ => unable to get local issuer certificate
log [07:32:33.038] [warning][admin][elasticsearch] Unable to revive connection

My kibana is running on Windows but elasticsearch is on Linux. Do I need to download a copy of the root ca.pem to Windows? now the elasticsearch.ssl.certificateAuthorities: [“/etc/elasticsearch/root-ca.pem”] is pointing to Linux. Can you advise me on this?

Thanks.