Java high level client 401(unauthorized) error

[bmm]

HI, I am trying to add authentication to my ES cluster using Search guard, I followed the installation steps from search guard website, and downloaded dummy certificates for now and placed them all under elasticsearch-6.7.0\config.

in elasticsearch.yml I added following:
xpack.security.enabled: false
searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:

• CN=kirk,OU=client,O=client,L=test,C=de

When I tried checking http://localhost:9200/?pretty from browser, it indeed asked for user name and password and it worked fine.

And when I tried to access the ES index data via my java client it gives 401 error:
ElasticsearchStatusException[Unable to parse response body]; nested: ResponseException[method [POST], host [http://localhost:9200], URI [/acfs_index/entity/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 401 Unauthorized]
Unauthorized]; nested: ResponseException[method [POST], host [http://localhost:9200], URI [/acfs_index/entity/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 401 Unauthorized]
Unauthorized];
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2033)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1777)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1734)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696)

Here is my java client to use username/password (I am using default admin/admin):

private RestHighLevelClient buildClient() {
    try {
    	final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    	credentialsProvider.setCredentials(AuthScope.ANY,
    	        new UsernamePasswordCredentials("user", "password"));

    	restHighLevelClient = new RestHighLevelClient(
    		RestClient.builder(new HttpHost("localhost", 9200))
    	        .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
    	            @Override
    	            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
    	                httpClientBuilder.disableAuthCaching(); 
    	                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
    	            }
    	        }));
    } catch (Exception e) {
    	e.printStackTrace();
        //LOG.error(e.getMessage());
    }
    return restHighLevelClient;
}

Am I missing something in setup or something in java client configuration? Any help/suggestions much appreciated! Thanks.

[cstaley]

Omit httpClientBuilder.disableAuthCaching() when challenge is false in sg_config.yml for the http basic authentication domain.

[bmm]

tried that, still the same error.

[cstaley]

@pablo.lescotti can you verify that https://gist.github.com/floragunncom/caf3b1e7cffb9e970c92657a839b8f16 is working?

[bmm]

@cstaley, I am using similar code as on the github link above (except we don’t need to deal with SSL certificate as we are using http and not https), and the above issue is fixed now. I realized the index data it was trying to access was indeed incorrect and non-existent on the ES, but the 401 error kind of misled me to believe that my searchguard authentication is not working.
I am able to create index, insert index, query index data, check cluster health from java client, and incorrect password is failing with 401 (as expected), thanks for your help! I am going to try configuring Kibana now with the elasticsearch, as we would use Kibana most of the time to insert or query data, I will reach out to you if I have any further questions, thanks again.