TLS config for fluent-bit output plugin to elasticsearch

Search Guard
1

Hi,
I have secured Elasticsearch with SG.
I need to use TLS to securely transmit the logs from fluentbit to elasticsearch. I see that the below parameters needs to be populated at the ES output plugin
Example:
OUTPUT]
Name es
Match *
Host elasticsearch
Port 9200
Index nspos-app2-tomcat-logs
Type nspos-app2-tomcat
Logstash_Format On
Retry_Limit False
Time_Key @datetime
Logstash_Prefix nspos-app2-tomcat-logs
tls On
tls.verify Off
tls.ca_file
tls.crt_file
tls.key_file
tls.key_passwd
Shared_Key I have generated TLS/SSL certificates using searchguards offline TLS tool Offline TLS Tool | Security for Elasticsearch | Search Guard. I have run the example.sh and generated the CA and certificates. I am not able to figure out which of the generated certificates / CA files needs to be used at fluent-bit for the TLS parameters mentioned
Can anyone pls help me fill in the tls values at fluent-bit’s ES output plugin?

2

Although I am not familiar with fluentbit, the setup should be similar to the setup of any data shipper that operates on the REST layer.

If you have enabled TLS on REST in Search Guard (as you should), you need to first configure the Root CA used to sign the Elasticsearch certificates. In other words, the Root CA the TLS tool has produced.

The setting of tls.verify depends on the hostnames setting in your certificates: If the certificates contain the actual hostname of your ES node(s), you can set it to ON. If the hostname(s) do not match the ES node(s) hostname(s), set it to Off.

The other TLS settings are only necessary if you want to use client certificate authentication.

In addition to the TLS settings, you also need to configure the user and password that fluentbit should use to authenticate:

https://docs.fluentbit.io/manual/output/elasticsearch

    HTTP_User  user
    HTTP_Password  password
4

Thanks jkressin,

Tried using root-ca.pem along with http_username and passwd the below config and fluent-bit was able to commumicate with elasticsearch.
Should be OK right?

[OUTPUT]
Name es
Match *
Host {{ .Values.logSidecar.elastic.host }}
Port {{ .Values.logSidecar.elastic.port }}
Index {{ .Values.logSidecar.elastic.index }}
Type nspos-app2-tomcat
Logstash_Format On
Retry_Limit False
Time_Key @datetime
Logstash_Prefix nspos-app2-tomcat-logs

    tls               On
    tls.verify        Off
    tls.ca_file       /opt/certs/root-ca.pem
    http_user         admin
    http_passwd       admin
5

Yes, looks good to me. For production, I just recommend using a dedicated fluentbit user with limited access permissions. At the moment you are using the admin user which has full access.

6

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.