I wanted to ask the community on your opinion on some elasticsearch security considerations. We are operating elasticsearch on the public internet, our kubernetes cluster is directly in the public internet. Right now, we have the transport port just announced cluster-internally and the http port published on the internet. This makes it not difficult to administrate the cluster with sgadmin. Especially, because we are working in a highly dynamic docker environment, not with static servers where we could just logon and administrate. What are your considerations? Is it a good idea to publish also the transport-port 9300? What experiences did you make? How do you administrate searchguard? Is it also possible to install sgadmin as a CLI tool on mac?
We have big problems working with searchguard right now, as it’s so complicated to administrate, when you don’t have the “normal” setup with regular servers …
All good ideas from the community are highly appreciated
I don’t see why you need the 9300 port open. I have 3 dedicated master and 8 dedicated data nodes on AWS spot instances (managed by SpotInst). We put a load balancer in front which gives us a single point of entry into the cluster as well as load management and high availability. We use https on the LB so the 9200 port is also internal. Can’t see any reason the same approach wont work with k8s or with a single node ES.
Administering the community edition via sgadmin is a bit of a pain. Our plan is to create an endpoint on each master and add another load balancer to talk to that. The endpoint then updates the current SG config using sgadmin (i.e. fetch, update, push)
From what I’ve read the Enterprise version would let you do most things via it’s own REST API and/or in Kibana depending on your use case. Sadly our team couldn’t justify the $10k+ per year price tag for our cluster.
Hi Martin, thank you for your answer. I see it the same way as you - administering using sgadmin is a pain! I just don’t know how to practically do it without it? We use elasticsearch in a multi node cluster by the way in the same way than you are doing it. I’m really interested how you are managing your cluster. How are you creating new users etc? We have many microservices and really would like to make things more save by just creating users per microservice. Using the Elasticsearch API for searchguard is also a little bit of a pain, as there seems to be a caching of one hour … so making a change on a role is not reflected immediately in the cluster.
So how are you doing it? We are right now a little lost and also thought about some consulting hours from searchguard as a beginning, but that also doesn’t seem to be the business case for them, so they don’t seem to offer what we really need …
I don’t see why you need the 9300 port open. I have 3 dedicated master and 8 dedicated data nodes on AWS spot instances (managed by SpotInst). We put a load balancer in front which gives us a single point of entry into the cluster as well as load management and high availability. We use https on the LB so the 9200 port is also internal. Can’t see any reason the same approach wont work with k8s or with a single node ES.
Administering the community edition via sgadmin is a bit of a pain. Our plan is to create an endpoint on each master and add another load balancer to talk to that. The endpoint then updates the current SG config using sgadmin (i.e. fetch, update, push)
From what I’ve read the Enterprise version would let you do most things via it’s own REST API and/or in Kibana depending on your use case. Sadly our team couldn’t justify the $10k+ per year price tag for our cluster.
From a security point of view, it’s totally valid to open 9300 and make it internet facing. If you do that make sure you don’t use our demo certificates of course. With correct SSL setup clients without the proper certificates are not able to connect. So what’s left is a DDOS but this problem also exists with the HTTP(s) port and you can mitigate this using common DDOS prevention tools. If you do that only open the port for client nodes and not for master or data nodes. On the other side the transport client will be deprecated by ES in Version 7 and so we are currently migrating sgadmin to talk https instead of native transport. Not sure yet if we backport this to version 6. If you like you can help us testing this feature.
The REST API is our (commercial) convenient feature to make administration more smooth but from a pure feature point of view, there is nothing you cannot do using solely the community edition. The 1h default user cache time is configurable, so you can lower that value or even disable user caching. See Cache settings | Security for Elasticsearch | Search Guard
A standalone sgadmin binary exists for every version we release and its available on maven central. It runs on Mac, Linux, and Windows.
If you’re missing a feature just file a feature request on GitHub or get in touch with us via mail.
Hope this helps
···
On Wednesday, 21 November 2018 20:14:18 UTC+1, Jannik Z wrote:
Hi Martin, thank you for your answer. I see it the same way as you - administering using sgadmin is a pain! I just don’t know how to practically do it without it? We use elasticsearch in a multi node cluster by the way in the same way than you are doing it. I’m really interested how you are managing your cluster. How are you creating new users etc? We have many microservices and really would like to make things more save by just creating users per microservice. Using the Elasticsearch API for searchguard is also a little bit of a pain, as there seems to be a caching of one hour … so making a change on a role is not reflected immediately in the cluster.
So how are you doing it? We are right now a little lost and also thought about some consulting hours from searchguard as a beginning, but that also doesn’t seem to be the business case for them, so they don’t seem to offer what we really need …
I don’t see why you need the 9300 port open. I have 3 dedicated master and 8 dedicated data nodes on AWS spot instances (managed by SpotInst). We put a load balancer in front which gives us a single point of entry into the cluster as well as load management and high availability. We use https on the LB so the 9200 port is also internal. Can’t see any reason the same approach wont work with k8s or with a single node ES.
Administering the community edition via sgadmin is a bit of a pain. Our plan is to create an endpoint on each master and add another load balancer to talk to that. The endpoint then updates the current SG config using sgadmin (i.e. fetch, update, push)
From what I’ve read the Enterprise version would let you do most things via it’s own REST API and/or in Kibana depending on your use case. Sadly our team couldn’t justify the $10k+ per year price tag for our cluster.
