An external UI app is able to login the user with Keycloak and grab an access token from Keycloak. The next step for that UI is to use that access token to make a request to elasticsearch. We have tried a few ways we thought should work and we have not been successful so far. Also, there is no mention in searchguard docs about how we can use CURL and pass an access token to make a request to ES.
A solution using CURL would be appreciated because that would serve as a POC and then developers would be able to test/implement the authentication with SG depending on the language/framework they are using on the app side.
probably the easiest way to integrate Keycloak with Search Guard is to use the OpenID specification. We have OpenID support already built-in in the Kibana plugin. Depending on the type of application you are building, you may want to consider following that path.
If I understood your use case correctly, I think the main problem is that you are trying to use the access token that you retrieved from Keycloak for authentication. However, you need an identity token, which is the “real” JSON web token carrying the user identity.
Once your application has obtained the identity token, you would then use the OpenID Connect authentication module with Search Guard:
When using curl or any other HTTP client library, you can either append the identity token as an URL parameter or as an HTTP header. Search Guard supports both. In the configuration of the OpenID connect module, you can specify the name of the HTTP header by using the jwt_header parameter, or the jwt_url_parameter configuration setting in case you are using an URL parameter.
The ES and SearchGuard side is already secured using the openID specifications and works exactly how SearchGuard mentions it in the docs. The question however is sending a request using CURL and getting authenticated. Should the jwt_header parameter still be used when using openid on the SearchGuard side?