Custom certificate issue with Elasticsearch, Kibana and Searchguard

I am new here with ElasticSearch and Kibana!

I’m trying to implement search-guard in ES and Kibana

The below is the configuration for ES, Searchguard and Kibana

···
  1. I have installed java using commands:

sudo add-apt-repository -y ppa:webupd8team/java

sudo apt-get update

sudo apt-get install -y oracle-java8-installer

Version: java version “1.8.0_171”

  1. I have installed ES and Searchguard plugin using commands:

sudo apt-get install elasticsearch=6.2.2

sudo service elasticsearch start

sudo service elasticsearch status

curl localhost:9200

cd /usr/share/elasticsearch/

sudo bin/elasticsearch-plugin install -b com.floragunn:search-guard-6:6.2.2-22.1

  1. Use the update-rc.d command to configure Elasticsearch to start automatically when the system boots up:

sudo update-rc.d elasticsearch defaults 95 10

  1. changed the permission of the folder:

sudo chmod 777 /etc/elasticsearch/

  1. copied my custom certs to ES

mv /home/ubuntu/ssl.crt/* /etc/elasticsearch/

sudo openssl pkcs8 -nocrypt -topk8 -inform PEM -outform PEM -in /etc/elasticsearch/example.key -out /etc/elasticsearch/example.key.pkcs8

sudo mv /etc/elasticsearch/example.key /etc/elasticsearch/example.key.bak

sudo mv /etc/elasticsearch/example.key.pkcs8 /etc/elasticsearch/example.key

  1. changed the permission of the file:

sudo chmod 660 /etc/elasticsearch/elasticsearch.yml

  1. Search Guard Configuration for ES

`network.host: 0.0.0.0

searchguard.ssl.transport.pemcert_filepath: example.crt

searchguard.ssl.transport.pemkey_filepath: example.key

searchguard.ssl.transport.pemtrustedcas_filepath: gdroot-g2.crt

searchguard.ssl.transport.enforce_hostname_verification: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemcert_filepath: example.crt

searchguard.ssl.http.pemkey_filepath: example.key

searchguard.ssl.http.pemtrustedcas_filepath: gdroot-g2.crt

searchguard.allow_default_init_sgindex: true

searchguard.authcz.admin_dn:`

`- CN=example,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

discovery.zen.minimum_master_nodes: 1

node.max_local_storage_nodes: 3`

  1. changed the CA certs and restarted ES

cd /usr/local/share/ca-certificates

sudo cp /etc/elasticsearch/gd_bundle-g2.crt .

udo update-ca-certificates

sudo service elasticsearch status

sudo service elasticsearch restart

sudo service elasticsearch status

  1. Below is the status of ES from

/var/log/elasticsearch/elasticsearch.log:

[2018-05-03T19:21:23,994][INFO ][o.e.n.Node ] [VMSMQFX] started

10.Curl output:

`curl -u admin:admin “https://localhost:9200” --insecure

{

“name” : “VMSMQFX”,

“cluster_name” : “elasticsearch”,

“cluster_uuid” : “***************”,

“version” : {

“number” : “6.2.2”,

“build_hash” : “10b1edd”,

“build_date” : “2018-02-16T19:01:30.685723Z”,

“build_snapshot” : false,

“lucene_version” : “7.2.1”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}`

`curl --insecure -u admin:admin ‘https://localhost:9200/_searchguard/authinfo?pretty=true

{

“user” : “User [name=admin, roles=[admin], requestedTenant=null]”,

“user_name” : “admin”,

“user_requested_tenant” : null,

“remote_address” : “127.0.0.1:49845”,

“backend_roles” : [

“admin”

],

“custom_attribute_names” : [

“attr.internal.attribute1”,

“attr.internal.attribute2”,

“attr.internal.attribute3”

],

“sg_roles” : [

“sg_all_access”,

“sg_own_index”

],

“sg_tenants” : {

“admin_tenant” : true,

“admin” : true

},

“principal” : null,

“peer_certificates” : “0”

}`


kibana and searchguard installation:


  1. Created the kibana source list:

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

  1. updated the package

sudo apt-get update

  1. Installed the apt-transport-https package on Debian before proceeding:

sudo apt-get install apt-transport-https

  1. Installed Kibana with this command:

sudo apt-get install kibana=6.2.2

  1. downloaded the searchguard for kibana

sudo wget "https://search.maven.org/remotecontent?filepath=com/floragunn/search-guard-kibana-plugin/6.2.2-10/search-guard-kibana-plugin-6.2.2-10.zip"

sudo mv remotecontent\?filepath\=com%2Ffloragunn%2Fsearch-guard-kibana-plugin%2F6.2.2-10%2Fsearch-guard-kibana-plugin-6.2.2-10.zip search-guard-kibana-plugin-6.2.2-10.zip

cd /usr/share/kibana

sudo bin/kibana-plugin install file:///home/ubuntu/search-guard-kibana-plugin-6.2.2-10.zip

  1. permission for kibana

sudo chmod 660 /etc/kibana/

  1. changed the below lines in kibana.yml

`elasticsearch.url: “https://localhost:9200

server.ssl.enabled: false

server.ssl.certificate: /etc/elasticsearch/example.crt

server.ssl.key: /etc/elasticsearch/example.key`

  1. Started the kibana

sudo service kibana start

  1. I get bellow error:

a) /var/log/elasticsearch/elasticsearch.log

[2018-05-03T22:30:28,253][WARN ][c.f.s.h.SearchGuardHttpServerTransport] [VMSMQFX] Someone (/127.0.0.1:49632) speaks http plaintext instead of ssl, will close the channel

b)/var/log/kibana/kibana.stdout

`{“type”:“log”,"@timestamp":“2018-05-03T21:52:28Z”,“tags”:[“status”,“plugin:elasticsearch@6.2.2”,“error”],“pid”:12254,“state”:“red”,“message”:“Status changed from yellow to red - Unable to connect to Elasticsearch at http://localhost:9200.”,“prevState”:“yellow”,“prevMsg”:“Waiting for Elasticsearch”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:30Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“Unable to revive connection: http://localhost:9200/”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:30Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“No living connections”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:33Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“Unable to revive connection: http://localhost:9200/”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:33Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“No living connections”}`

Is there anything that I’m missing in the configuration files?

In kibana.yml the url pointing to ES should be https://localhost:9200 (and not http://localhost:9200) i guess.

···

On Friday, 4 May 2018 01:36:47 UTC+2, Brenda Martis wrote:

I am new here with ElasticSearch and Kibana!

I’m trying to implement search-guard in ES and Kibana

The below is the configuration for ES, Searchguard and Kibana


  1. I have installed java using commands:

sudo add-apt-repository -y ppa:webupd8team/java

sudo apt-get update

sudo apt-get install -y oracle-java8-installer

Version: java version “1.8.0_171”

  1. I have installed ES and Searchguard plugin using commands:

sudo apt-get install elasticsearch=6.2.2

sudo service elasticsearch start

sudo service elasticsearch status

curl localhost:9200

cd /usr/share/elasticsearch/

sudo bin/elasticsearch-plugin install -b com.floragunn:search-guard-6:6.2.2-22.1

  1. Use the update-rc.d command to configure Elasticsearch to start automatically when the system boots up:

sudo update-rc.d elasticsearch defaults 95 10

  1. changed the permission of the folder:

sudo chmod 777 /etc/elasticsearch/

  1. copied my custom certs to ES

mv /home/ubuntu/ssl.crt/* /etc/elasticsearch/

sudo openssl pkcs8 -nocrypt -topk8 -inform PEM -outform PEM -in /etc/elasticsearch/example.key -out /etc/elasticsearch/example.key.pkcs8

sudo mv /etc/elasticsearch/example.key /etc/elasticsearch/example.key.bak

sudo mv /etc/elasticsearch/example.key.pkcs8 /etc/elasticsearch/example.key

  1. changed the permission of the file:

sudo chmod 660 /etc/elasticsearch/elasticsearch.yml

  1. Search Guard Configuration for ES

`network.host: 0.0.0.0

searchguard.ssl.transport.pemcert_filepath: example.crt

searchguard.ssl.transport.pemkey_filepath: example.key

searchguard.ssl.transport.pemtrustedcas_filepath: gdroot-g2.crt

searchguard.ssl.transport.enforce_hostname_verification: true

searchguard.ssl.http.enabled: true

searchguard.ssl.http.pemcert_filepath: example.crt

searchguard.ssl.http.pemkey_filepath: example.key

searchguard.ssl.http.pemtrustedcas_filepath: gdroot-g2.crt

searchguard.allow_default_init_sgindex: true

searchguard.authcz.admin_dn:`

`- CN=example,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch

searchguard.enable_snapshot_restore_privilege: true

searchguard.check_snapshot_restore_write_privileges: true

searchguard.restapi.roles_enabled: [“sg_all_access”]

discovery.zen.minimum_master_nodes: 1

node.max_local_storage_nodes: 3`

  1. changed the CA certs and restarted ES

cd /usr/local/share/ca-certificates

sudo cp /etc/elasticsearch/gd_bundle-g2.crt .

udo update-ca-certificates

sudo service elasticsearch status

sudo service elasticsearch restart

sudo service elasticsearch status

  1. Below is the status of ES from

/var/log/elasticsearch/elasticsearch.log:

[2018-05-03T19:21:23,994][INFO ][o.e.n.Node ] [VMSMQFX] started

10.Curl output:

`curl -u admin:admin “https://localhost:9200” --insecure

{

“name” : “VMSMQFX”,

“cluster_name” : “elasticsearch”,

“cluster_uuid” : “***************”,

“version” : {

“number” : “6.2.2”,

“build_hash” : “10b1edd”,

“build_date” : “2018-02-16T19:01:30.685723Z”,

“build_snapshot” : false,

“lucene_version” : “7.2.1”,

“minimum_wire_compatibility_version” : “5.6.0”,

“minimum_index_compatibility_version” : “5.0.0”

},

“tagline” : “You Know, for Search”

}`

`curl --insecure -u admin:admin ‘https://localhost:9200/_searchguard/authinfo?pretty=true

{

“user” : “User [name=admin, roles=[admin], requestedTenant=null]”,

“user_name” : “admin”,

“user_requested_tenant” : null,

“remote_address” : “127.0.0.1:49845”,

“backend_roles” : [

“admin”

],

“custom_attribute_names” : [

“attr.internal.attribute1”,

“attr.internal.attribute2”,

“attr.internal.attribute3”

],

“sg_roles” : [

“sg_all_access”,

“sg_own_index”

],

“sg_tenants” : {

“admin_tenant” : true,

“admin” : true

},

“principal” : null,

“peer_certificates” : “0”

}`


kibana and searchguard installation:


  1. Created the kibana source list:

echo "deb [https://artifacts.elastic.co/packages/6.x/apt](https://artifacts.elastic.co/packages/6.x/apt) stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

  1. updated the package

sudo apt-get update

  1. Installed the apt-transport-https package on Debian before proceeding:

sudo apt-get install apt-transport-https

  1. Installed Kibana with this command:

sudo apt-get install kibana=6.2.2

  1. downloaded the searchguard for kibana

sudo wget "[https://search.maven.org/remotecontent?filepath=com/floragunn/search-guard-kibana-plugin/6.2.2-10/search-guard-kibana-plugin-6.2.2-10.zip](https://search.maven.org/remotecontent?filepath=com/floragunn/search-guard-kibana-plugin/6.2.2-10/search-guard-kibana-plugin-6.2.2-10.zip)"

sudo mv remotecontent\?filepath\=com%2Ffloragunn%2Fsearch-guard-kibana-plugin%2F6.2.2-10%2Fsearch-guard-kibana-plugin-6.2.2-10.zip search-guard-kibana-plugin-6.2.2-10.zip

cd /usr/share/kibana

sudo bin/kibana-plugin install file:///home/ubuntu/search-guard-kibana-plugin-6.2.2-10.zip

  1. permission for kibana

sudo chmod 660 /etc/kibana/

  1. changed the below lines in kibana.yml

`elasticsearch.url: “https://localhost:9200

server.ssl.enabled: false

server.ssl.certificate: /etc/elasticsearch/example.crt

server.ssl.key: /etc/elasticsearch/example.key`

  1. Started the kibana

sudo service kibana start

  1. I get bellow error:

a) /var/log/elasticsearch/elasticsearch.log

[2018-05-03T22:30:28,253][WARN ][c.f.s.h.SearchGuardHttpServerTransport] [VMSMQFX] Someone (/[127.0.0.1:49632](http://127.0.0.1:49632)) speaks http plaintext instead of ssl, will close the channel

b)/var/log/kibana/kibana.stdout

`{“type”:“log”,"@timestamp":“2018-05-03T21:52:28Z”,“tags”:[“status”,“plugin:elasticsearch@6.2.2”,“error”],“pid”:12254,“state”:“red”,“message”:“Status changed from yellow to red - Unable to connect to Elasticsearch at http://localhost:9200.”,“prevState”:“yellow”,“prevMsg”:“Waiting for Elasticsearch”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:30Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“Unable to revive connection: http://localhost:9200/”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:30Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“No living connections”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:33Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“Unable to revive connection: http://localhost:9200/”}

{“type”:“log”,"@timestamp":“2018-05-03T21:52:33Z”,“tags”:[“warning”,“elasticsearch”,“admin”],“pid”:12254,“message”:“No living connections”}`

Is there anything that I’m missing in the configuration files?