Docker configuration for existing elk images

If you think it is a bug report or you have a technical issue, please answer the following questions. For general questions, you can delete these questions.

**Elasticsearch version:**7.10.2

**Server OS version:**docker

Kibana version (if relevant):

Browser version (if relevant):

Browser OS version (if relevant):

**Describe the issue:**As per the documentation you are providing docker image along with kibana and elastic search, but how to download only searchgaurd image to configure with existing kibana and elasticsearch images on docker?

Steps to reproduce:
1.
2.
3.

Expected behavior:

Provide configuration:
elasticsearch/config/elasticsearch.yml
elasticsearch/plugins/search-guard-7/sgconfig/sg_config.yml
kibana/config/kibana.yml (if relevant)

Provide logs:
Elasticsearch
Kibana (if relevant)

Screenshots (if relevant):

Errors in browser console (if relevant):

Additional data:

Due to the license change of Elasticsearch, we cannot provide you any Docker images for ES 7.11.0 and later.

You can find images for earlier versions at Docker hub:

https://hub.docker.com/r/floragunncom/sg-elasticsearch/tags

Possibly, the SG helm charts are useful for you. Just announced:

https://forum.search-guard.com/t/now-available-as-public-beta-helm-charts-for-search-guard/2132/2

are we need to have same version of elastic and kibana same as searchGaurd version for docker also??

Yes, you need.

  1. When can we expect searchGaurd latest release of version 7.11.2 for docker image and manual setup??

  2. How to setup searchGuard on existing elastic and kibana images on docker??

With the docker image as you provided in official docs is up and running for elastic and kibana with basic auth,

how to connect beats image with elasticsearch image as it throwing ssl certificate error for beats connection.

can we disable SSL option in elasticsearch image??

When can we expect searchGaurd latest release of version 7.11.2

Very soon.

How to setup searchGuard on existing elastic and kibana images on docker??

Look at the Search Guard labs repository: Files · master · search-guard / labs · GitLab

… it throwing ssl certificate error for beats connection … can we disable SSL option in elasticsearch image??

It would be very bad do disable SSL in Search Guard. What errors do you see?
I guess it will work if you skip certificate verification in beats. For example, in filebeat the setting is verification_mode: none. Configure SSL | Filebeat Reference [7.11] | Elastic

Exiting: couldn’t connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://...:9200: 401 Unauthorized: Unauthorized]

we are facing below error for heartbeat when we provide elasticsearch public ip

tried with verification_mode:none
and
output.elasticsearch.hosts: [“https://192.168.1.42:9200”]
output.elasticsearch.ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
output.elasticsearch.ssl.certificate: “/etc/pki/client/cert.pem”
output.elasticsearch.ssl.key: “/etc/pki/client/cert.key”

passed these paths also both are not working

2021-03-18T14:02:00.571+0530 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://:9200)): Get "https://...:9200": x509: certificate is valid for 127.0.0.1, not ...

this is for external connection

output.elasticsearch:

Array of hosts to connect to.

hosts: ["...:9200"]

Protocol - either http (default) or https.

protocol: “https”

ssl.certificate_authorities: [C:/Users/sivak/Downloads/root-ca.pem]
ssl.certificate: “C:/Users/sivak/Downloads/kirk.pem”
ssl.key: “C:/Users/sivak/Downloads/kirk-key.pem”

Exiting: couldn’t connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://. .. :9200: 401 Unauthorized: Unauthorized]

Is it the same error or you just joined two errors?

2021-03-18T14:02:00.571+0530 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://:9200)): Get "https://... :9200": x509: certificate is valid for 127.0.0.1, not . ..

Where is this error from? It is hard to tell not seeing the entire log.

Please provide 2 logs (entire log for last 1 day):

  1. Elasticsearch log
  2. Beats log. What beats do you use?

Also, provide the configurations (entire files):

  1. elasticsearch.yml
  2. sg_config.yml
  3. beats configuration

The error couldn’t connect to any of the configured Elasticsearch hosts means that the system that produced the log can’t connect to Elasticsearch host. Are you sure the Elasticsearch host is reachable from that system? Maybe a firewall in the middle blocks the connection. Go to that system and check if the Elasticsearch 9200 port is open. You can do it with nmap or netcat.

nmap -p 9200 elasticsearch_host
nc -zv elasticsearch_host 9200

we are facing below error for heartbeat when we provide elasticsearch public ip
output.elasticsearch.hosts: [“https://192.168.1.42:9200”]

The 192.168.1.42 is not a public IPv4 address, it is from the private subnet 192.168.0.0/16.

Hi, how can we get docker image (non-OSS version) of searchgaurd along with kibana and elasticsearch

Can we manually setup seachgaurd plugin as we did for windows in docker container with install command??
elastic is up and running but kibana is not able to connect with elastic even though providing cert paths and auth creds properly

how can we get docker image (non-OSS version )

Can we manually setup seachgaurd plugin as we did for windows in docker container with install command??

I don’t how you did it. But yes, it is possible to install the plugins in the container. Did you see the dockerfiles of the search-guard/labs?

P.S.
In 7.11 we have bug though, the Kibana plugin doesn’t work properly without Multitenancy. Make sure you have it enabled:

searchguard.multitenancy.enabled: true
searchguard.multitenancy.tenants.enable_global: true
searchguard.multitenancy.tenants.preferred: [ "global" ]
elasticsearch.requestHeadersWhitelist: ["authorization", "sgtenant"]

Elasticsearch Dockerfile

ARG ES_VERSION
ARG SG_VERSION
ARG CONTAINER_NAME
ARG AUTH

FROM docker.elastic.co/elasticsearch/elasticsearch:${ES_VERSION}

# Must repeat the docker-compose args to make them available after FROM
ARG ES_VERSION
ARG SG_VERSION
ARG CONTAINER_NAME
ARG AUTH

RUN elasticsearch-plugin install --batch https://maven.search-guard.com/search-guard-suite-release/com/floragunn/search-guard-suite-plugin/${ES_VERSION}-${SG_VERSION}/search-guard-suite-plugin-${ES_VERSION}-${SG_VERSION}.zip

# Copy Elasticsearch config
COPY ./config/${AUTH}/${CONTAINER_NAME}/*.pem /usr/share/elasticsearch/config/
COPY ./config/${AUTH}/${CONTAINER_NAME}/*.key /usr/share/elasticsearch/config/
COPY ./config/${AUTH}/${CONTAINER_NAME}/*.readme /usr/share/elasticsearch/config/
COPY ./config/${AUTH}/${CONTAINER_NAME}/*.yml /usr/share/elasticsearch/config/
COPY ./config/${AUTH}/${CONTAINER_NAME}/*.yml.example /usr/share/elasticsearch/config/

# Copy Search Guard config
COPY ./config/${AUTH}/${CONTAINER_NAME}/sgconfig/* /usr/share/elasticsearch/plugins/search-guard-7/sgconfig/

Kibana Dockerfile

ARG SG_VERSION
ARG KIBANA_VERSION
ARG CONTAINER_NAME
ARG AUTH

FROM docker.elastic.co/kibana/kibana:${KIBANA_VERSION}

# Must repeat the docker-compose args to make them available after FROM
ARG SG_VERSION
ARG KIBANA_VERSION
ARG CONTAINER_NAME
ARG AUTH

RUN kibana-plugin install https://maven.search-guard.com/search-guard-kibana-plugin-release/com/floragunn/search-guard-kibana-plugin/${KIBANA_VERSION}-${SG_VERSION}/search-guard-kibana-plugin-${KIBANA_VERSION}-${SG_VERSION}.zip 
COPY ./config/${AUTH}/${CONTAINER_NAME} /usr/share/kibana/config/