# Unable to execute sgadmin.sh

**URL:** <https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446>\
**Category:** Search Guard\
**Created:** [April 16, 2019, 5:11am UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446 "2019-04-16T05:11:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![jatin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jatin/32/466_2.png) [@jatin](https://forum.search-guard.com/u/jatin)\
**Post date:** [April 16, 2019, 5:11am UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/1 "2019-04-16T05:11:35Z")

</div>

Hi All,

I deployed Search Guard on non prod linux ES instance.  
I had to generate certificates using offline tls-generator tool by Search Guard because it involved adding IP address to the certificate.

After this I copied the certificates to **/config** folder.

![ES%20config](https://us1.discourse-cdn.com/flex019/uploads/search_guard/original/1X/aa18f7c865ee0a8b22d4ce21a452bb9d47b18fa7.png)

I created a new internal user and changed the basic config files fo SG.

Now I was trying to execute sgadmin to add new users to searchguard index. But unfortunately it is giving me the following error:-

```
Search Guard Admin v6
Will connect to 10.123.222.89:9300 ... done
ERR: An unexpected IllegalStateException occured: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
Trace:
java.lang.IllegalStateException: failed to load plugin class [com.floragunn.searchguard.SearchGuardPlugin]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:563)
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:104)
        at org.elasticsearch.client.transport.TransportClient.newPluginService(TransportClient.java:105)
        at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:130)
        at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:262)
        at com.floragunn.searchguard.tools.SearchGuardAdmin$TransportClientImpl.<init>(SearchGuardAdmin.java:886)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:441)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmin.java:123)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554)
        ... 7 more
Caused by: ElasticsearchException[Unable to read ../../../config/kirk-key.pem (../../../config/kirk-key.pem). Please make sure this files exists and is readable regarding to permissions. Property: searchguard.ssl.transport.pemkey_filepath]
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.checkPath(DefaultSearchGuardKeyStore.java:709)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.resolve(DefaultSearchGuardKeyStore.java:193)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfig(DefaultSearchGuardKeyStore.java:282)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.<init>(DefaultSearchGuardKeyStore.java:145)
        at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuardSSLPlugin.java:193)
        at com.floragunn.searchguard.SearchGuardPlugin.<init>(SearchGuardPlugin.java:197)
        ... 12 more

```

kirk was generated as admin certificate.  
I believe its because its key is not in kirk-key.pem format.  
I have a file as kirk.key. But if I feed in kirk.key instead, it seems to demand it in .pem format.

**Can anyone tell what should I do differently to make it work?**

My elasticsearch.yml is:

```
searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode.key
searchguard.ssl.transport.pemkey_password: aBcDeFgHi12
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode_http.pem
searchguard.ssl.http.pemkey_filepath: esnode_http.key
searchguard.ssl.http.pemkey_password: jKlMnOpQrS34
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.nodes_dn:
- CN=kirk,OU=client,O=client,L=test,C=de
searchguard.authcz.admin_dn:
- CN=kirk,OU=client,O=client,L=test,C=de

```

---

<div class="post-metadata">

**Author:** ![jkressin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jkressin/32/9_2.png) [@jkressin](https://forum.search-guard.com/u/jkressin)\
**Post date:** [April 16, 2019, 1:35pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/2 "2019-04-16T13:35:12Z")

</div>

As far as I can see there are two issues here.

First, you use the same kirk certificate as admin and also as node certificate:

```
searchguard.nodes_dn: 
  - CN=kirk,OU=client,O=client,L=test,C=de 
searchguard.authcz.admin_dn: 
  - CN=kirk,OU=client,O=client,L=test,C=de

```

Due to security considerations this is not allowed. You need an admin certificate (kirk in your case) that is separate from a node certificate. In your case the node certificate is es-node.pem.

The second issue is that when you use sgadmin the path to the kirk certificate seems to be wrong:

> [@jatin](#):
>
> Caused by: ElasticsearchException[Unable to read …/…/…/config/kirk-key.pem (…/…/…/config/kirk-key.pem). Please make sure this files exists and is readable regarding to permissions. Property: searchguard.ssl.transport.pemkey\_filepath]  
> …

So make sure that the path you use in the actual sgadmin call:

`../../../config/kirk-key.pem`

is correct.

The filename, by the way, does not really specify/tell you what format the certificate or key has. The kirk.key file already contains the correct format. In other words, you can name the file whatever you like.

---

<div class="post-metadata">

**Author:** ![jkressin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jkressin/32/9_2.png) [@jkressin](https://forum.search-guard.com/u/jkressin)\
**Post date:** [April 16, 2019, 1:35pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/3 "2019-04-16T13:35:21Z")

</div>



---

<div class="post-metadata">

**Author:** ![jatin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jatin/32/466_2.png) [@jatin](https://forum.search-guard.com/u/jatin)\
**Post date:** [April 16, 2019, 1:40pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/4 "2019-04-16T13:40:16Z")

</div>

Hi Jochen,

There is no file as **“kirk-key.pem”** produced by tlsgenerator.  
Instead I gave in **“kirk.key”** (produced by tlsgenerator) which I think is not accepted by the sgadmin.sh script.

**Do you think its because it demands _.pem_ format?**

**If yes, how can I convert kirk.key into _.pem_ format?**.

---

<div class="post-metadata">

**Author:** ![jkressin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jkressin/32/9_2.png) [@jkressin](https://forum.search-guard.com/u/jkressin)\
**Post date:** [April 17, 2019, 8:06pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/5 "2019-04-17T20:06:02Z")

</div>

You do not need to convert anything. The private key the generator produces is already in the correct PEM format. From what I can see in your posts, the private key is:

`kirk.key`

And the certificate is:

`kirk.pem`

From the error message I can see that when executing sgadmin you refer to the (non-existing) file kirk-key.pem:

`Unable to read ../../../config/kirk-key.pem`

Replacing the (non-existing) kirk-key.pem with the (existing) kirk.key file in the sgadmin call should do the trick.

Can you please post the sgadmin call you are issuing?

---

<div class="post-metadata">

**Author:** ![jatin](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/jatin/32/466_2.png) [@jatin](https://forum.search-guard.com/u/jatin)\
**Post date:** [April 18, 2019, 4:19pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/6 "2019-04-18T16:19:39Z")

</div>

HI jochen,

Thanks for your response.

At first I executed this statement,

> ./sgadmin.sh -cd …/sgconfig/ -icl -nhnv -cacert …/…/…/config/root-ca.pem -cert …/…/…/config/kirk.pem -key …/…/…/config/kirk.key -h 10.123.211.87

Which gave me the following error, which I assume is because of no keypass.

```
Search Guard Admin v6
Will connect to 10.123.211.87:9300 ... done
15:34:32.924 [main] ERROR com.floragunn.searchguard.ssl.DefaultSearchGuardKeySto re - Your keystore or PEM does not contain a key. If you specified a key passwor d, try removing it. If you did not specify a key password, perhaps you need to i f the key is in fact password-protected. Maybe you just confused keys and certif icates.
ERR: An unexpected IllegalStateException occured: failed to load plugin class [c om.floragunn.searchguard.SearchGuardPlugin]
Trace:
java.lang.IllegalStateException: failed to load plugin class [com.floragunn.sear chguard.SearchGuardPlugin]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.ja va:563)
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:1 04)
        at org.elasticsearch.client.transport.TransportClient.newPluginService(T ransportClient.java:105)
        at org.elasticsearch.client.transport.TransportClient.buildTemplate(Tran sportClient.java:130)
        at org.elasticsearch.client.transport.TransportClient.<init>(TransportCl ient.java:262)
        at com.floragunn.searchguard.tools.SearchGuardAdmin$TransportClientImpl. <init>(SearchGuardAdmin.java:886)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdm in.java:441)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main(SearchGuardAdmi n.java:123)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.ja va:554)
        ... 7 more
Caused by: ElasticsearchSecurityException[Error while initializing transport SSL layer from PEM: java.lang.IllegalArgumentException: File does not contain valid private key: ../../../config/kirk.key]; nested: IllegalArgumentException[File d oes not contain valid private key: ../../../config/kirk.key]; nested: InvalidKey SpecException[Neither RSA, DSA nor EC worked]; nested: InvalidKeySpecException[j ava.security.InvalidKeyException: IOException : DER input, Integer tag error]; n ested: InvalidKeyException[IOException : DER input, Integer tag error];
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfi g(DefaultSearchGuardKeyStore.java:292)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.<init>(Defau ltSearchGuardKeyStore.java:145)
        at com.floragunn.searchguard.ssl.SearchGuardSSLPlugin.<init>(SearchGuard SSLPlugin.java:193)
        at com.floragunn.searchguard.SearchGuardPlugin.<init>(SearchGuardPlugin. java:197)
        ... 12 more
Caused by: java.lang.IllegalArgumentException: File does not contain valid priva te key: ../../../config/kirk.key
        at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.j ava:267)
        at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.ja va:90)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.buildSSLServ erContext(DefaultSearchGuardKeyStore.java:613)
        at com.floragunn.searchguard.ssl.DefaultSearchGuardKeyStore.initSSLConfi g(DefaultSearchGuardKeyStore.java:287)
        ... 15 more
Caused by: java.security.spec.InvalidKeySpecException: Neither RSA, DSA nor EC w orked
        at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContex t.java:1045)
        at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1014)
        at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.j ava:265)
        ... 18 more
Caused by: java.security.spec.InvalidKeySpecException: java.security.InvalidKeyE xception: IOException : DER input, Integer tag error
        at sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java: 169)
        at java.security.KeyFactory.generatePrivate(KeyFactory.java:372)
        at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContex t.java:1043)
        ... 20 more
Caused by: java.security.InvalidKeyException: IOException : DER input, Integer t ag error
        at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:352)
        at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:357)
        at sun.security.ec.ECPrivateKeyImpl.<init>(ECPrivateKeyImpl.java:73)
        at sun.security.ec.ECKeyFactory.implGeneratePrivate(ECKeyFactory.java:23 7)
        at sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java: 165)
        ... 22 more

```

Then I provide another option -keypass with the password provided by tlsgenerator in **“client-certificates.readme”** file.

> ./sgadmin.sh -cd …/sgconfig/ -icl -nhnv -cacert …/…/…/config/root-ca.pem -cert …/…/…/config/kirk.pem -key …/…/…/config/kirk.key -keypass TUpeLXjD3u5J -h 10.123.211.87

But still even after adding keypass I got the following error.

```
Search Guard Admin v6
Will connect to 10.123.211.87:9300 ... done
Unable to check whether cluster is sane: None of the configured nodes are available: [{#transport#-1}{Hyl106pgT0aZ8_spuz0jxA}{10.123.211.87}{10.123.211.87:9300}]
ERR: Cannot connect to Elasticsearch. Please refer to elasticsearch logfile for more information
Trace:
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{Hyl106pgT0aZ8_spuz0jxA}{10.123.211.87}{10.123.211.87:9300}]]
        at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
        at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
        at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:371)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
        at com.floragunn.searchguard.tools.SearchGuardAdmin.main0(SearchGuardAdmin.java:450)

```

Background story:-

After installing the plugin.  
I used install\_demo\_.sh for setup…  
Only to find out that to upload data the ip address should be present in the certificate subjectAltName.

I generated certificates by adding ip address under node conf area in tlsconfig.yml in tlsgenerator.

And just copied certificates to /config and changed elasticsearch.yml with provided snippet

I think there can be 2 possible scenarios.

1. the dn for client and node certificates are same.
2. Since I added generated certificates after using install\_demo\_conf script. I need to add cluster name details somewhere for search.

**Which reason do you think is more likely or both?**

**If both, for first I need to generate certificates again with separate dn and again copyt new certs and change elasticsearch.yml, right?**

**What about the 2nd one?**

---

<div class="post-metadata">

**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.search-guard.com/system/32/1870_2.png) [@system](https://forum.search-guard.com/u/system)\
**Post date:** [May 9, 2019, 4:19pm UTC](https://forum.search-guard.com/t/unable-to-execute-sgadmin-sh/1446/7 "2019-05-09T16:19:40Z")

</div>

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.