When asking questions, please provide the following information:
-
Search Guard and Elasticsearch version
-
Used enterprise modules, if any
-
JVM version and operating system version
-
Search Guard configuration files
-
Elasticsearch log messages on debug level
I am attempting to install the test certificates that were generated by the search guard test cert creator, and I’m getting internal errors from Elastic Search when I try to install the trust store. If I resend the truststore and keystore for the demo certificates, the system works fine.
I have installed the demo certificates and the system is up and running but with untrusted certificates. I’m wanting to put in the test certificates so that they can trusted.
I’m running on Windows Server 2012 R2, Server 2008 and Centos as a mixed OS cluster and have tried JRE 1.8.0.130 and 1.8.0.144. Elastic Search 5.5.1 and Search Guard 5.5.1-15
I’m to the point of running the sgadmin.bat in this manner (I’ve tried ignore cluster name as well)
./sgadmin.bat -h w12esnode1 -ts truststore.jks -tspass -ks CN=sgadmin-keystore.jks -kspass -cd …/sgconfig -cn HDR -nhnv -ff --accept-red-cluster -tsalias elasticsearch --diagnose
06:44:12.201 [main] ERROR com.floragunn.searchguard.ssl.util.SSLCertificateHelpe
r - Alias elasticsearch does not contain a certificate entry
Failfast is activated
Diagnostic trace written to: C:\elasticsearch\plugins\search-guard-5\tools\sgadm
in_diag_trace_2017-Aug-30_06-44-14.txt
Contacting elasticsearch cluster ‘HDR’ …
ERR: Cannot retrieve cluster state due to: None of the configured nodes are avai
lable: [{#transport#-1}{tdSm1SSBSIyVxVnIl81_qQ}{w12esnode1}{10.1.11.190:9300}].
- Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you
need to check your clustername as well as hostnames in your SSL certificates)
- Make also sure that your keystore or cert is a client certificate (not a no
de certificate) and configured properly in elasticsearch.yml
- If this is not working, try running sgadmin.sh with --diagnose and see diag
nose trace log file)
- Add --accept-red-cluster to allow sgadmin to operate on a red cluster.
I’ve imported the root ca into the cacerts file with alias elasticsearch.
C:\Program Files\Java\jre64\bin>keytool -list
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
elasticsearch, Aug 29, 2017, trustedCertEntry,
Certificate fingerprint (SHA1): 02:D9:41:BE:3D:F0:37:DA:24:45:6A:9B:FD:96:6F:0E:
7C:D3:64:9E
The log from sgadmin:
Search Guard Admin v5
Will connect to w12esnode1:9300 … done
LICENSE NOTICE Search Guard
If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See Licensing | Search Guard Community, Enterprise and Compliance Edition)
-
Kibana Multitenancy
-
LDAP authentication/authorization
-
Active Directory authentication/authorization
-
REST Management API
-
JSON Web Token (JWT) authentication/authorization
-
Kerberos authentication/authorization
-
Document- and Fieldlevel Security (DLS/FLS)
-
Auditlogging
In case of any doubt mail to sales@floragunn.com
sgadmin_diag_trace_2017-Aug-30_06-52-11.txt (14.2 KB)
elasticsearch.yml (3.71 KB)
sg_internal_users.yml (1.33 KB)
sg_roles.yml (6.15 KB)
sg_roles_mapping.yml (1013 Bytes)
sg_action_groups.yml (1.48 KB)
sg_config.yml (9.37 KB)
···
###################################
06:52:10.075 [main] ERROR com.floragunn.searchguard.ssl.util.SSLCertificateHelpe
r - Alias elasticsearch does not contain a certificate entry
Failfast is activated
Diagnostic trace written to: C:\elasticsearch\plugins\search-guard-5\tools\sgadm
in_diag_trace_2017-Aug-30_06-52-11.txt
Contacting elasticsearch cluster ‘HDR’ …
ERR: Cannot retrieve cluster state due to: None of the configured nodes are avai
lable: [{#transport#-1}{S28hFouYRoiBJgy8NiId6A}{w12esnode1}{10.1.11.190:9300}].
- Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you
need to check your clustername as well as hostnames in your SSL certificates)
- Make also sure that your keystore or cert is a client certificate (not a no
de certificate) and configured properly in elasticsearch.yml
- If this is not working, try running sgadmin.sh with --diagnose and see diag
nose trace log file)
- Add --accept-red-cluster to allow sgadmin to operate on a red cluster.
Cluster health is:
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1504101273 06:54:33 HDR green 3 3 33 16 0 0 0 0 - 100.0%
The error in Elastic Search is:
[2017-08-30T06:31:13,937][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [w12esn
ode1] SSL Problem Received fatal alert: internal_error
javax.net.ssl.SSLException: Received fatal alert: internal_error
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?
:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[
?:?]
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907)
~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_131]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.jav
a:254) ~[netty-handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1156) ~[netty-
handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1078) ~[netty-
handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProte
ction(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final
]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageD
ecoder.java:428) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessage
Decoder.java:265) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstra
ctChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final
]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(Defau
ltChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChanne
lPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(Abstra
ctNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.jav
a:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLo
op.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.ja
va:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-t
ransport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThread
EventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
[2017-08-30T06:44:14,373][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [w12esn
ode1] SSL Problem Received fatal alert: internal_error
javax.net.ssl.SSLException: Received fatal alert: internal_error
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?
:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[
?:?]
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907)
~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_131]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.jav
a:254) ~[netty-handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1156) ~[netty-
handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1078) ~[netty-
handler-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProte
ction(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final
]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageD
ecoder.java:428) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessage
Decoder.java:265) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Abstra
ctChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final
]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(Defau
ltChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Fin
al]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChanne
lPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(Abstra
ctNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.jav
a:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLo
op.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.ja
va:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-t
ransport-4.1.11.Final.jar:4.1.11.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThread
EventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
What am I doing wrong?