My miss-configuration opens kibana at all for anybody, I don't see why

Roger_Ineichen · January 13, 2018, 4:47pm

Hi all,

I installed elasticsearch and kibana, after that I installed the search guard plugins with custom certificate and configurations.

Now I can login with any random string as username and password. How is this possible? I don’t see any error int he log file.

Is there a config which makes everything public. Why do I get asked for a login in kibana and any random credentials are allowed?

setup@sta01:/xeebo/elasticsearch/plugins/search-guard-6/tools$ curl --insecure -u admin:mypassword ‘https://10.0.0.211:9200/_searchguard/health?pretty=true’

{

“message” : null,

“mode” : “strict”,

“status” : “UP”

}

setup@sta01:/xeebo/elasticsearch/plugins/search-guard-6/tools$ curl --insecure -u admin:mypassword ‘https://10.0.0.211:9200/_searchguard/authinfo?pretty’

{

“user” : “User [name=admin, roles=[admin, CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx], requestedTenant=null]”,

“user_name” : “admin”,

“user_requested_tenant” : null,

“remote_address” : “10.0.0.211:41198”,

“backend_roles” : [

“admin”,

“CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx”

],

“custom_attribute_names” : ,

“sg_roles” : [

“sg_all_access”,

“sg_own_index”

],

“sg_tenants” : {

“admin_tenant” : true,

“admin” : true

},

“principal” : null,

“peer_certificates” : “0”

}

As I understand the kibanaserver user talks to elasticsearch for execute some actions. Do I need to see the word kibanaserver in the elasticsearch log?

I don’t see any kibanaserver in the elasticsearch log (TRACE mode) is the kibanaserver user elavated in kibana and using the admin?

Is my kibana instance useing a backdoor in elasticsearch? I can’t understand what’s going on and where to start. Before I post the 12 MB TRACE log, can someone tell me how I can check if kibana talks to elasticsearch of corse it does but where I can see the authentication and authorization grant part? Or where can I see why any random username and password comination let shows me the kibana app and let me using the console and execute queries?

Versions:

elasticsearch-6.1.1.tar.gz

kibana-6.1.1-linux-x86_64.tar.gz

search-guard-6-6.1.1-20.1.zip

search-guard-kibana-plugin-6.1.1-8.zip

No other plugins installed

Regards Roger

Roger_Ineichen · January 13, 2018, 8:39pm

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

searchguard_google_group · January 13, 2018, 8:45pm

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/search-guard/AiczDz0u-Zk/FOrSDSP0CgAJ

···

Am 13.01.2018 um 21:39 schrieb Roger Ineichen <roger.ineichen@gmail.com>:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true
[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}
org.elasticsearch.index.IndexNotFoundException: no such index
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.<init>(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.<init>(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]
    at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]
    at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]
    at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
    at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
    at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]
    at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]
    at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]
    at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false
[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/00f00ad4-68ee-4be6-8af8-b9a7b817ce56%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

jkressin · January 13, 2018, 8:50pm

Of course, there is no backdoor in Elasticsearch or Search Guard that elevates roles automatically to an admin role.

The only way how this can happen is if you send an admin client TLS certificate with your HTTP request. An admin certificate, as the name implies, allows you to access all indices, including the Search Guard configuration index.

Kibana for example let’s you set a client certificate with the following keys:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

If you configure an admin certificate here, this will be sent with every HTTP request and thus give every request admin permissions.

This entry in the ES logs give a hint that this might be the case:

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

Please attach your elasticsearch.yml and the kibana.yml, I strongly suspect that you send an admin certificate with each request.

···

On Saturday, January 13, 2018 at 9:39:29 PM UTC+1, Roger Ineichen wrote:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

Roger_Ineichen · January 14, 2018, 3:10am

Thanks for the reply,

That’s a big security issue! It disables any other security concept and elevates admin permission to anybody with random login/password.

What’s happen here is that if you configure an admin certificate kibana will elevate any (random) credential (login, password) to an administrator. That’s for sure not what anybody needs. Not sure if this happens with X-Pack too because it’s a kibana issue. If you think this is not a security issue can you explain why it is possible to login with randon credentials?

Which part is responsible for granting/elevating my random login/password to the admin.pem. And why is the login form shown at all if it doesn’t prevent access?

NOTE I don’t have any client certificate installed on my machine which of corse whould grant if so!

btw, I defined;

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

and not:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

My sgconfig dosn’t enable clientcert_auth_domain

and searchguard.ssl.http.clientauth_mode: BOTH

Regards Roger

···

Am Samstag, 13. Januar 2018 21:50:34 UTC+1 schrieb Jochen Kressin:

Of course, there is no backdoor in Elasticsearch or Search Guard that elevates roles automatically to an admin role.

The only way how this can happen is if you send an admin client TLS certificate with your HTTP request. An admin certificate, as the name implies, allows you to access all indices, including the Search Guard configuration index.

Kibana for example let’s you set a client certificate with the following keys:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

If you configure an admin certificate here, this will be sent with every HTTP request and thus give every request admin permissions.

This entry in the ES logs give a hint that this might be the case:

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

Please attach your elasticsearch.yml and the kibana.yml, I strongly suspect that you send an admin certificate with each request.

On Saturday, January 13, 2018 at 9:39:29 PM UTC+1, Roger Ineichen wrote:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

jkressin · January 14, 2018, 3:43pm

Hi Roger,

thanks for your reply. Yes, an admin certificate does exactly that, it grants access to your cluster with elevated permissions. This is required for changing the SG index but is also used in production environments, e.g. to recover a cluster from red state or other maintenance tasks where you need complete admin access. So it’s kind of grants root access, and that’s why you need to keep your admin certificate(s) safe. From the documentation it is hopefully clear that you need to treat admin certificates with care, if this is not the case please let me know where we can improve on the docs.

We decided to use a TLS certificate for that purpose because the other option would be to install default users with default passwords, which is a poor replacement in my opinion.

I understand that if you configure Kibana to send this admin certificate in each request this leads to everyone having access to the cluster. My first question to improve on this situation is, why did you install the admin certificate in Kibana in the first place? What was the intended purpose / use case?

These configuration entries here are valid for Kibana 6.x:

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

And these are for 5.x:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

The admin certificate is unrelated to the client_cert_auth domain. This authenticator should be used to grant access by regular (non-admin) HTTP client certificates.

For searchguard.ssl.http.clientauth_mode you can use: NONE, OPTIONAL or REQUIRE.

In order to disable TLS client certificates for HTTP completely, including the admin certificate, you can set it to NONE. Search Guard then simply ignores any client certificate on HTTP layer. Note that in this case, access to the REST API with an admin certificate is also not possible anymore. So it depends on the use case you are trying to implement.

The functionality to send a TLS client certificate is a built-in Kibana functionality which we cannot disable. Technically there is nothing we can do to prevent a user from configuring an admin certificate here. We can however think about possibilities to check that on ES side and then issue an error message.

But like I said before we decide on how to improve the situation / prevent this misconfiguration I’d like to understand your specific use case better and like to know what you were trying to achieve.

Thanks for your input,

Jochen

···

On Sunday, January 14, 2018 at 4:10:58 AM UTC+1, Roger Ineichen wrote:

Thanks for the reply,

That’s a big security issue! It disables any other security concept and elevates admin permission to anybody with random login/password.

What’s happen here is that if you configure an admin certificate kibana will elevate any (random) credential (login, password) to an administrator. That’s for sure not what anybody needs. Not sure if this happens with X-Pack too because it’s a kibana issue. If you think this is not a security issue can you explain why it is possible to login with randon credentials?

Which part is responsible for granting/elevating my random login/password to the admin.pem. And why is the login form shown at all if it doesn’t prevent access?

NOTE I don’t have any client certificate installed on my machine which of corse whould grant if so!

btw, I defined;

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

and not:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

My sgconfig dosn’t enable clientcert_auth_domain

and searchguard.ssl.http.clientauth_mode: BOTH

Regards Roger

Am Samstag, 13. Januar 2018 21:50:34 UTC+1 schrieb Jochen Kressin:

Of course, there is no backdoor in Elasticsearch or Search Guard that elevates roles automatically to an admin role.

The only way how this can happen is if you send an admin client TLS certificate with your HTTP request. An admin certificate, as the name implies, allows you to access all indices, including the Search Guard configuration index.

Kibana for example let’s you set a client certificate with the following keys:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

If you configure an admin certificate here, this will be sent with every HTTP request and thus give every request admin permissions.

This entry in the ES logs give a hint that this might be the case:

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

Please attach your elasticsearch.yml and the kibana.yml, I strongly suspect that you send an admin certificate with each request.

On Saturday, January 13, 2018 at 9:39:29 PM UTC+1, Roger Ineichen wrote:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

Roger_Ineichen · January 14, 2018, 9:56pm

Hallo Jochen

Thanks for your reply

I don’t have a special use case, just playing arround and followed the guide and docs for setup an intial working infrastructure.

It’s hard to understand how to configure everything correct. Searchguard and Elasticsearch do a great job on documentation but to get the big picture is no an easy task.

It has too much posibilities and you have to find out what options you need for a specific setup.

During my setup, elasticsearch, searchguard and kibana I had a hard time to create the certificate because there are no exact requirements listed. You have to find them in documentation in sample scripts and review the generated certs. The searchguard certifacte generator is a great tool but I decided to generate our own certificates because it’s state we should do that.

The reason why I added the admin certificates is simple. The kibana.yml describes this:

Optional settings that provide the paths to the PEM-format SSL certificate and key files.

These files validate that your Elasticsearch backend uses the same key files.

#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

There is no word about what happens with this certificates. I was confused why the doc says the same “key” and was using the same cert and key already installed in elasticserach and that was the admin cert. There is no word about that this certificate is used for authorization it sound harmless and suggests that this cert is used for authentication.Ok, let’s say I was lazy and already had the admin cert on that machine deployed with xcx, salt, kleopatra encrypted as gpg/yaml and I dind’t take the time to deploy the client certs too

But, the admin as client cert is not my real issue. That’s simply a configuration which kibana allows and could be usefull for some usecase.

My issue is about what happens with this configuration. Why does the admin cert in kibana accept ANY login credential in the login form?

Why is it possible to see the login screen. Why is any random login/password combination valid and let me authenticate (not authorize, which is what the admin certificate is doing).

In my understanding there could be 2 concepts.

remove the login screen at all which makes it clear that anybody can use your elasticserach server (without a certificate)

or

keep the login screen and let the user login first and then validate the client certificate as a kind of 2 factor authentication.

Sorry that I’m ask again, did you understand that I didn’t use any (admin or client) certificate in my browser and the kibana UI login form was presented to me. After that I was using a random username/password and I was logged in with admin privilege?

I think using an admin certificate in kibana could be a feature and could allow to elevate any user/login to admin privilege but not yet. Yet It disables the login at all.

Currently with this (wrong) configuration it disables authentication and only provides authorization.

Summary,

The admin certificate defined as elasticsearch.ssl.certificate in kibana.yml forces to accept any login/password in the login form which is a big security risk if you deploy the kibana server to the public. You will never do this.

jochen, do you agree that this will also be a problem with the new GDPR (security by design)?

My suggestion whould be to add a bootstrap check and tear down the elasticserach clsuter at all not only show an error message if no other additional option allows to enable this explicit.

Thanks a lot, Roger

···

Am Sonntag, 14. Januar 2018 16:43:19 UTC+1 schrieb Jochen Kressin:

Hi Roger,

thanks for your reply. Yes, an admin certificate does exactly that, it grants access to your cluster with elevated permissions. This is required for changing the SG index but is also used in production environments, e.g. to recover a cluster from red state or other maintenance tasks where you need complete admin access. So it’s kind of grants root access, and that’s why you need to keep your admin certificate(s) safe. From the documentation it is hopefully clear that you need to treat admin certificates with care, if this is not the case please let me know where we can improve on the docs.

We decided to use a TLS certificate for that purpose because the other option would be to install default users with default passwords, which is a poor replacement in my opinion.

I understand that if you configure Kibana to send this admin certificate in each request this leads to everyone having access to the cluster. My first question to improve on this situation is, why did you install the admin certificate in Kibana in the first place? What was the intended purpose / use case?

These configuration entries here are valid for Kibana 6.x:

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

And these are for 5.x:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

The admin certificate is unrelated to the client_cert_auth domain. This authenticator should be used to grant access by regular (non-admin) HTTP client certificates.

For searchguard.ssl.http.clientauth_mode you can use: NONE, OPTIONAL or REQUIRE.

In order to disable TLS client certificates for HTTP completely, including the admin certificate, you can set it to NONE. Search Guard then simply ignores any client certificate on HTTP layer. Note that in this case, access to the REST API with an admin certificate is also not possible anymore. So it depends on the use case you are trying to implement.

The functionality to send a TLS client certificate is a built-in Kibana functionality which we cannot disable. Technically there is nothing we can do to prevent a user from configuring an admin certificate here. We can however think about possibilities to check that on ES side and then issue an error message.

But like I said before we decide on how to improve the situation / prevent this misconfiguration I’d like to understand your specific use case better and like to know what you were trying to achieve.

Thanks for your input,

Jochen

On Sunday, January 14, 2018 at 4:10:58 AM UTC+1, Roger Ineichen wrote:

Thanks for the reply,

That’s a big security issue! It disables any other security concept and elevates admin permission to anybody with random login/password.

What’s happen here is that if you configure an admin certificate kibana will elevate any (random) credential (login, password) to an administrator. That’s for sure not what anybody needs. Not sure if this happens with X-Pack too because it’s a kibana issue. If you think this is not a security issue can you explain why it is possible to login with randon credentials?

Which part is responsible for granting/elevating my random login/password to the admin.pem. And why is the login form shown at all if it doesn’t prevent access?

NOTE I don’t have any client certificate installed on my machine which of corse whould grant if so!

btw, I defined;

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

and not:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

My sgconfig dosn’t enable clientcert_auth_domain

and searchguard.ssl.http.clientauth_mode: BOTH

Regards Roger

Am Samstag, 13. Januar 2018 21:50:34 UTC+1 schrieb Jochen Kressin:

Of course, there is no backdoor in Elasticsearch or Search Guard that elevates roles automatically to an admin role.

The only way how this can happen is if you send an admin client TLS certificate with your HTTP request. An admin certificate, as the name implies, allows you to access all indices, including the Search Guard configuration index.

Kibana for example let’s you set a client certificate with the following keys:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

If you configure an admin certificate here, this will be sent with every HTTP request and thus give every request admin permissions.

This entry in the ES logs give a hint that this might be the case:

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

Please attach your elasticsearch.yml and the kibana.yml, I strongly suspect that you send an admin certificate with each request.

On Saturday, January 13, 2018 at 9:39:29 PM UTC+1, Roger Ineichen wrote:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

jkressin · January 14, 2018, 11:11pm

Hi Roger,

"It’s hard to understand how to configure everything correct. Searchguard and Elasticsearch do a great job on documentation but to get the big picture is no an easy task.

It has too much posibilities and you have to find out what options you need for a specific setup."

I totally agree with that. We support a lot of different authentication schemas and configurations options, and while this offers great flexibility and allows to adapt Search Guard to very specific use cases, it also means that it is probably hard to get the big picture. We have plans to publish configuration guidelines and examples that will cover the most important use cases. It’s a bit of a challenge. For example, using Basic Auth + Active Directory + Kibana requires a totally different setup than using an OpenID provider that issues JSON web tokens, or running Search Guard behind proxy based authentication.

“During my setup, elasticsearch, searchguard and kibana I had a hard time to create the certificate because there are no exact requirements listed. You have to find them in documentation in sample scripts and review the generated certs. The searchguard certifacte generator is a great tool but I decided to generate our own certificates because it’s state we should do that.”

I also agree. The reason why you can use the certificates from the online generator safely for PoCs and for demos, but not in production is: Since we generated the certificates for you means that we are in possession of your certificates potentially. The generated certificates are deleted automatically periodically, but nevertheless, it’s unsafe for production usage. That’s why we advise generating your own certificates, by tools like OpenSSL or by using the PKI infrastructure of your company. We are currently working on an offline version of the certificate generator. The certificates generated by this tool are safe for production usage. The estimated release will be mid of February.

"But, the admin as client cert is not my real issue. That’s simply a configuration which kibana allows and could be usefull for some usecase.

My issue is about what happens with this configuration. Why does the admin cert in kibana accept ANY login credential in the login form?

Why is it possible to see the login screen. Why is any random login/password combination valid and let me authenticate (not authorize, which is what the admin certificate is doing). […] Sorry that I’m ask again, did you understand that I didn’t use any (admin or client) certificate in my browser and the kibana UI login form was presented to me. After that I was using a random username/password and I was logged in with admin privilege?"

Yes, the description in the kibana.yml is quite misleading. It should state that “the certificate configure here is sent with each request that Kibana issues to Elasticsearch”. We can’t change the description in the kibana.yml, but we should make it more clear in our own docs what the effect of configuring a certificate here is.

For explaining why it does not matter if you installed a client cert locally in your browser I need to dive into the architecture of Kibana. Kibana is mainly based on Angular / React for the frontend part, and node.js / hapi for the backend part. If you perform any action in the browser, the request is never issued against your Elasticsearch cluster directly. The request is rather issued against the node.js / hapi backend first, and only then issued against Elasticsearch. So it’s

Browser → node → Elasticsearch

It’s the node.js part that adds the configured TLS certificate (in kibana.yml) to the requests automatically, so it does not matter what client certificates you have configured in your browser. This is the effect of configuring

elasticsearch.ssl.certificate

elasticsearch.ssl.key

in kibana.yml. Since an admin certificate will always grant full access to the cluster, and since this admin certificate is sent with every request (due to the node.js part, independent of your browser configuration), it overwrites any other credentials you send. I’m not an X-Pack expert, but since you can also configure PKI authentication there, you would probably have a similar issue with X-Pack.

“The admin certificate defined as elasticsearch.ssl.certificate in kibana.yml forces to accept any login/password in the login form which is a big security risk if you deploy the kibana server to the public. You will never do this.”

So we need to make it clear that you never want to configure an admin certificate in kibana.yml and probably also implement:

“1. remove the login screen at all which makes it clear that anybody can use your elasticserach server (without a certificate)”

“jochen, do you agree that this will also be a problem with the new GDPR (security by design)?”

No, I don’t think so. We have a “Security First” approach, which means that unlike competitor products we make security our first priority and usability second. For example, we never relied on default users and passwords, always made TLS mandatory on the transport layer and also take care about the nitty-gritty details like HTTPS compression or client-side TLS renegotiation. We are following GDPR closely and will release a first technical preview of our brand-new GDPR compliance features very soon. Stay tuned.

Thanks very much for your valuable input!

···

On Sunday, January 14, 2018 at 10:56:00 PM UTC+1, Roger Ineichen wrote:

Hallo Jochen

Thanks for your reply

I don’t have a special use case, just playing arround and followed the guide and docs for setup an intial working infrastructure.

It’s hard to understand how to configure everything correct. Searchguard and Elasticsearch do a great job on documentation but to get the big picture is no an easy task.

It has too much posibilities and you have to find out what options you need for a specific setup.

During my setup, elasticsearch, searchguard and kibana I had a hard time to create the certificate because there are no exact requirements listed. You have to find them in documentation in sample scripts and review the generated certs. The searchguard certifacte generator is a great tool but I decided to generate our own certificates because it’s state we should do that.

The reason why I added the admin certificates is simple. The kibana.yml describes this:

Optional settings that provide the paths to the PEM-format SSL certificate and key files.

These files validate that your Elasticsearch backend uses the same key files.

#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

There is no word about what happens with this certificates. I was confused why the doc says the same “key” and was using the same cert and key already installed in elasticserach and that was the admin cert. There is no word about that this certificate is used for authorization it sound harmless and suggests that this cert is used for authentication.Ok, let’s say I was lazy and already had the admin cert on that machine deployed with xcx, salt, kleopatra encrypted as gpg/yaml and I dind’t take the time to deploy the client certs too

But, the admin as client cert is not my real issue. That’s simply a configuration which kibana allows and could be usefull for some usecase.

My issue is about what happens with this configuration. Why does the admin cert in kibana accept ANY login credential in the login form?

Why is it possible to see the login screen. Why is any random login/password combination valid and let me authenticate (not authorize, which is what the admin certificate is doing).

In my understanding there could be 2 concepts.

remove the login screen at all which makes it clear that anybody can use your elasticserach server (without a certificate)

or

keep the login screen and let the user login first and then validate the client certificate as a kind of 2 factor authentication.

Sorry that I’m ask again, did you understand that I didn’t use any (admin or client) certificate in my browser and the kibana UI login form was presented to me. After that I was using a random username/password and I was logged in with admin privilege?

I think using an admin certificate in kibana could be a feature and could allow to elevate any user/login to admin privilege but not yet. Yet It disables the login at all.

Currently with this (wrong) configuration it disables authentication and only provides authorization.

Summary,

The admin certificate defined as elasticsearch.ssl.certificate in kibana.yml forces to accept any login/password in the login form which is a big security risk if you deploy the kibana server to the public. You will never do this.

jochen, do you agree that this will also be a problem with the new GDPR (security by design)?

My suggestion whould be to add a bootstrap check and tear down the elasticserach clsuter at all not only show an error message if no other additional option allows to enable this explicit.

Thanks a lot, Roger

Am Sonntag, 14. Januar 2018 16:43:19 UTC+1 schrieb Jochen Kressin:

Hi Roger,

thanks for your reply. Yes, an admin certificate does exactly that, it grants access to your cluster with elevated permissions. This is required for changing the SG index but is also used in production environments, e.g. to recover a cluster from red state or other maintenance tasks where you need complete admin access. So it’s kind of grants root access, and that’s why you need to keep your admin certificate(s) safe. From the documentation it is hopefully clear that you need to treat admin certificates with care, if this is not the case please let me know where we can improve on the docs.

We decided to use a TLS certificate for that purpose because the other option would be to install default users with default passwords, which is a poor replacement in my opinion.

I understand that if you configure Kibana to send this admin certificate in each request this leads to everyone having access to the cluster. My first question to improve on this situation is, why did you install the admin certificate in Kibana in the first place? What was the intended purpose / use case?

These configuration entries here are valid for Kibana 6.x:

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

And these are for 5.x:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

The admin certificate is unrelated to the client_cert_auth domain. This authenticator should be used to grant access by regular (non-admin) HTTP client certificates.

For searchguard.ssl.http.clientauth_mode you can use: NONE, OPTIONAL or REQUIRE.

In order to disable TLS client certificates for HTTP completely, including the admin certificate, you can set it to NONE. Search Guard then simply ignores any client certificate on HTTP layer. Note that in this case, access to the REST API with an admin certificate is also not possible anymore. So it depends on the use case you are trying to implement.

The functionality to send a TLS client certificate is a built-in Kibana functionality which we cannot disable. Technically there is nothing we can do to prevent a user from configuring an admin certificate here. We can however think about possibilities to check that on ES side and then issue an error message.

But like I said before we decide on how to improve the situation / prevent this misconfiguration I’d like to understand your specific use case better and like to know what you were trying to achieve.

Thanks for your input,

Jochen

On Sunday, January 14, 2018 at 4:10:58 AM UTC+1, Roger Ineichen wrote:

Thanks for the reply,

That’s a big security issue! It disables any other security concept and elevates admin permission to anybody with random login/password.

What’s happen here is that if you configure an admin certificate kibana will elevate any (random) credential (login, password) to an administrator. That’s for sure not what anybody needs. Not sure if this happens with X-Pack too because it’s a kibana issue. If you think this is not a security issue can you explain why it is possible to login with randon credentials?

Which part is responsible for granting/elevating my random login/password to the admin.pem. And why is the login form shown at all if it doesn’t prevent access?

NOTE I don’t have any client certificate installed on my machine which of corse whould grant if so!

btw, I defined;

elasticsearch.ssl.certificate: /xeebo/kibana/config/admin.pem
elasticsearch.ssl.key: /xeebo/kibana/config/admin.pk8

and not:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

My sgconfig dosn’t enable clientcert_auth_domain

and searchguard.ssl.http.clientauth_mode: BOTH

Regards Roger

Am Samstag, 13. Januar 2018 21:50:34 UTC+1 schrieb Jochen Kressin:

Of course, there is no backdoor in Elasticsearch or Search Guard that elevates roles automatically to an admin role.

The only way how this can happen is if you send an admin client TLS certificate with your HTTP request. An admin certificate, as the name implies, allows you to access all indices, including the Search Guard configuration index.

Kibana for example let’s you set a client certificate with the following keys:

server.ssl.certificate: /path/to/your/server.crt

server.ssl.key: /path/to/your/server.key

If you configure an admin certificate here, this will be sent with every HTTP request and thus give every request admin permissions.

This entry in the ES logs give a hint that this might be the case:

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

Please attach your elasticsearch.yml and the kibana.yml, I strongly suspect that you send an admin certificate with each request.

On Saturday, January 13, 2018 at 9:39:29 PM UTC+1, Roger Ineichen wrote:

It seems that the permissionsinfo index is msiing. Is this an issue?
This happens on accessing kibana with a random login and password.

Why does any random login elevate the invalid credentials to an admin?

[2018-01-13T21:29:16,304][TRACE][c.f.s.c.AdminDNs ] Is principal CN=admin,OU=client,O=xxx,L=xxx,ST=xxx,C=xxx an admin cert? true

[2018-01-13T21:29:16,304][DEBUG][r.suppressed ] path: _searchguard/api/permissionsinfo, params: {index=_searchguard, id=permissionsinfo, type=api}

org.elasticsearch.index.IndexNotFoundException: no such index

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:182) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:251) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:123) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:139) ~[?:?]

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:82) ~[elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.filter.SearchGuardRestFilter$1.handleRequest(SearchGuardRestFilter.java:75) [search-guard-6-6.1.1-20.1.jar:6.1.1-20.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]

at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]

at com.floragunn.searchguard.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [search-guard-ssl-6.1.1-25.0.jar:6.1.1-25.0]

at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-client-6.1.1.jar:6.1.1]

at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.1.1.jar:6.1.1]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]

[2018-01-13T21:29:16,419][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

[2018-01-13T21:29:16,420][TRACE][c.f.s.s.u.SSLRequestHelper] validateCrl: false

Topic		Replies	Views
Unable to login Kibana Search Guard	2	644	September 12, 2018
Kibana will let any credentials pass Search Guard	8	376	December 14, 2017
Basic realm="Authorization Required" after successful authentication Search Guard	4	1146	November 2, 2018
Configure searchguard to work with SSO Search Guard	1	554	August 22, 2017
installed search guard but can not login to elasticSearch(also for kibina) with created user account Search Guard	4	638	November 29, 2018

My miss-configuration opens kibana at all for anybody, I don't see why

Optional settings that provide the paths to the PEM-format SSL certificate and key files.

These files validate that your Elasticsearch backend uses the same key files.

Optional settings that provide the paths to the PEM-format SSL certificate and key files.

These files validate that your Elasticsearch backend uses the same key files.

Related topics