Search Guard + Watcher

Hey all,
We configured our ELK installation to use Search Guard for authentication.
We’d like to try out the alerting capabilities of X-Pack (Watcher).
After installing X-Pack, of course, everything broke.
We disabled every feature to make it up and running again.
If we enable Watcher (only Watcher), we have this error in the Elasticsearch logs:
[2017-05-26T13:35:26,346][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [triggered_watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,344][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,358][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watch_history_2]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]

What should we do, to have Watcher running?

ELK version: 5.3.0

Thanks for the help in advance!

Best,
Ádám

Hi,

running any component from X-Pack, apart from the free monitoring, in conjunction with Search Guard is currently not supported.

Our reasoning behind it is that in order to use any of the X-Pack components, you need to pay for a subscription. (This excludes monitoring, which comes for free, and is supported by Search Guard.) If you already pay for an X-Pack subscription, then you can also use the security feature bundled with it. So, in our view, it would not make very much sense to have X-Pack and Search Guard running at the same time, and pay for both.

I’m curious to know, why you think about using X-Pack (Watcher) and Search Guard at the same time?

···

On Friday, May 26, 2017 at 1:59:32 PM UTC+2, ezegyfelhasznalonev@gmail.com wrote:

Hey all,
We configured our ELK installation to use Search Guard for authentication.
We’d like to try out the alerting capabilities of X-Pack (Watcher).
After installing X-Pack, of course, everything broke.
We disabled every feature to make it up and running again.
If we enable Watcher (only Watcher), we have this error in the Elasticsearch logs:
[2017-05-26T13:35:26,346][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [triggered_watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,344][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,358][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watch_history_2]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]

What should we do, to have Watcher running?

ELK version: 5.3.0

Thanks for the help in advance!

Best,
Ádám

Hi Jochen,

Yes, I agree, it doesn’t actually make any sense.

We’re really just experimenting with the capabilities of ES, and our use-cases involve sending out alerts.
Since the data we’re playing with can contain sensitive data, we secured the instance first with Search Guard (since it’s free).

Then, for alerting, we saw that Watcher could be used, which is bundled in X-Pack with Shield.
We didn’t want to start everything over again, and we thought we can make Watcher with Search Guard work.

Right now we’re evaluating ElastAlert, and at the same time considering of buying a license.

So I totally agree with you.
In the end either we will have Search Guard + ElastAlert, or X-Pack (Shield + Watcher).

Best,
Ádám

···

On Monday, May 29, 2017 at 8:51:49 PM UTC+2, Jochen Kressin wrote:

Hi,

running any component from X-Pack, apart from the free monitoring, in conjunction with Search Guard is currently not supported.

Our reasoning behind it is that in order to use any of the X-Pack components, you need to pay for a subscription. (This excludes monitoring, which comes for free, and is supported by Search Guard.) If you already pay for an X-Pack subscription, then you can also use the security feature bundled with it. So, in our view, it would not make very much sense to have X-Pack and Search Guard running at the same time, and pay for both.

I’m curious to know, why you think about using X-Pack (Watcher) and Search Guard at the same time?

On Friday, May 26, 2017 at 1:59:32 PM UTC+2, ezegyfelh...@gmail.com wrote:

Hey all,
We configured our ELK installation to use Search Guard for authentication.
We’d like to try out the alerting capabilities of X-Pack (Watcher).
After installing X-Pack, of course, everything broke.
We disabled every feature to make it up and running again.
If we enable Watcher (only Watcher), we have this error in the Elasticsearch logs:
[2017-05-26T13:35:26,346][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [triggered_watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,344][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,358][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watch_history_2]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]

What should we do, to have Watcher running?

ELK version: 5.3.0

Thanks for the help in advance!

Best,
Ádám

Hi Adam,

I hope you made some progress in your evaluation of Search Guard. Just wanted to let you know that ElastAlert should work with Search Guard pretty much out of the box. Please let us know if you need any assistance in setting Search Guard + ElastAlert!

Thanks,

Jochen

···

On Friday, June 9, 2017 at 11:46:45 AM UTC+2, ezegyfelhasznalonev@gmail.com wrote:

Hi Jochen,

Yes, I agree, it doesn’t actually make any sense.

We’re really just experimenting with the capabilities of ES, and our use-cases involve sending out alerts.
Since the data we’re playing with can contain sensitive data, we secured the instance first with Search Guard (since it’s free).

Then, for alerting, we saw that Watcher could be used, which is bundled in X-Pack with Shield.
We didn’t want to start everything over again, and we thought we can make Watcher with Search Guard work.

Right now we’re evaluating ElastAlert, and at the same time considering of buying a license.

So I totally agree with you.
In the end either we will have Search Guard + ElastAlert, or X-Pack (Shield + Watcher).

Best,
Ádám

On Monday, May 29, 2017 at 8:51:49 PM UTC+2, Jochen Kressin wrote:

Hi,

running any component from X-Pack, apart from the free monitoring, in conjunction with Search Guard is currently not supported.

Our reasoning behind it is that in order to use any of the X-Pack components, you need to pay for a subscription. (This excludes monitoring, which comes for free, and is supported by Search Guard.) If you already pay for an X-Pack subscription, then you can also use the security feature bundled with it. So, in our view, it would not make very much sense to have X-Pack and Search Guard running at the same time, and pay for both.

I’m curious to know, why you think about using X-Pack (Watcher) and Search Guard at the same time?

On Friday, May 26, 2017 at 1:59:32 PM UTC+2, ezegyfelh...@gmail.com wrote:

Hey all,
We configured our ELK installation to use Search Guard for authentication.
We’d like to try out the alerting capabilities of X-Pack (Watcher).
After installing X-Pack, of course, everything broke.
We disabled every feature to make it up and running again.
If we enable Watcher (only Watcher), we have this error in the Elasticsearch logs:
[2017-05-26T13:35:26,346][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [triggered_watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,344][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,358][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watch_history_2]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]

What should we do, to have Watcher running?

ELK version: 5.3.0

Thanks for the help in advance!

Best,
Ádám

Just to keep you updated: We will introduce support for X-Pack Alerting in one of the next versions of Search Guard.

···

On Monday, June 26, 2017 at 9:10:39 PM UTC+2, Jochen Kressin wrote:

Hi Adam,

I hope you made some progress in your evaluation of Search Guard. Just wanted to let you know that ElastAlert should work with Search Guard pretty much out of the box. Please let us know if you need any assistance in setting Search Guard + ElastAlert!

Thanks,

Jochen

On Friday, June 9, 2017 at 11:46:45 AM UTC+2, ezegyfelhasznalonev@gmail.com wrote:

Hi Jochen,

Yes, I agree, it doesn’t actually make any sense.

We’re really just experimenting with the capabilities of ES, and our use-cases involve sending out alerts.
Since the data we’re playing with can contain sensitive data, we secured the instance first with Search Guard (since it’s free).

Then, for alerting, we saw that Watcher could be used, which is bundled in X-Pack with Shield.
We didn’t want to start everything over again, and we thought we can make Watcher with Search Guard work.

Right now we’re evaluating ElastAlert, and at the same time considering of buying a license.

So I totally agree with you.
In the end either we will have Search Guard + ElastAlert, or X-Pack (Shield + Watcher).

Best,
Ádám

On Monday, May 29, 2017 at 8:51:49 PM UTC+2, Jochen Kressin wrote:

Hi,

running any component from X-Pack, apart from the free monitoring, in conjunction with Search Guard is currently not supported.

Our reasoning behind it is that in order to use any of the X-Pack components, you need to pay for a subscription. (This excludes monitoring, which comes for free, and is supported by Search Guard.) If you already pay for an X-Pack subscription, then you can also use the security feature bundled with it. So, in our view, it would not make very much sense to have X-Pack and Search Guard running at the same time, and pay for both.

I’m curious to know, why you think about using X-Pack (Watcher) and Search Guard at the same time?

On Friday, May 26, 2017 at 1:59:32 PM UTC+2, ezegyfelh...@gmail.com wrote:

Hey all,
We configured our ELK installation to use Search Guard for authentication.
We’d like to try out the alerting capabilities of X-Pack (Watcher).
After installing X-Pack, of course, everything broke.
We disabled every feature to make it up and running again.
If we enable Watcher (only Watcher), we have this error in the Elasticsearch logs:
[2017-05-26T13:35:26,346][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [triggered_watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,344][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watches]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
[2017-05-26T13:35:26,358][ERROR][o.e.x.w.s.WatcherIndexTemplateRegistry] [sHQzypY] Error adding watcher template [watch_history_2]
org.elasticsearch.ElasticsearchSecurityException: unauthenticated request indices:admin/template/put for user User [name=_sg_internal, roles=]
at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:126) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:83) ~[?:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1237) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.putTemplate(AbstractClient.java:1658) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.xpack.watcher.support.init.proxy.WatcherClientProxy.putTemplate(WatcherClientProxy.java:123) ~[?:?]
at org.elasticsearch.xpack.watcher.support.WatcherIndexTemplateRegistry.lambda$putTemplate$1(WatcherIndexTemplateRegistry.java:199) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:544) [elasticsearch-5.3.0.jar:5.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]

What should we do, to have Watcher running?

ELK version: 5.3.0

Thanks for the help in advance!

Best,
Ádám