Sgadmin.sh not initialising Elasticsearch

Search Guard
1

Hi,
This is a brand new, 2-node installation to evaluate Searchguard.

Despite a successful completion of the sgadmin.sh the cluster can’t initialise:

[root@spoonguard sgconfig-new]# /usr/share/elasticsearch/plugins/search-guard-7/tools/sgadmin.sh -h spoonguard.is.ed.ac.uk -cn spoonguard -cert /root/Enterprise_Services-crt.pem -key /root/Enterprise_Services.key -cacert /etc/elasticsearch/certs/eduni2.pem -nhnv -arc
WARNING: JAVA_HOME not set, will use /usr/bin/java
Search Guard Admin v7
Will connect to spoonguard.is.ed.ac.uk:9300 … done
Connected as CN=Enterprise Services/emailAddress,OU=Information Services,O=University of Edinburgh,L=Edinburgh,ST=Scotland,C=GB
Elasticsearch Version: 7.8.1
Search Guard Version: 7.8.1-43.0.0
Contacting elasticsearch cluster ‘spoonguard’ …
Clustername: spoonguard
Clusterstate: GREEN
Number of nodes: 2
Number of data nodes: 2
searchguard index already exists, so we do not need to create one.
Populate config from /root/sgconfig-new
./sg_action_groups.yml OK
./sg_internal_users.yml OK
./sg_roles.yml OK
./sg_roles_mapping.yml OK
./sg_config.yml OK
./sg_tenants.yml OK
./sg_blocks.yml OK
Will update ‘_doc/config’ with ./sg_config.yml
SUCC: Configuration for ‘config’ created or updated
Will update ‘_doc/roles’ with ./sg_roles.yml
SUCC: Configuration for ‘roles’ created or updated
Will update ‘_doc/rolesmapping’ with ./sg_roles_mapping.yml
SUCC: Configuration for ‘rolesmapping’ created or updated
Will update ‘_doc/internalusers’ with ./sg_internal_users.yml
SUCC: Configuration for ‘internalusers’ created or updated
Will update ‘_doc/actiongroups’ with ./sg_action_groups.yml
SUCC: Configuration for ‘actiongroups’ created or updated
Will update ‘_doc/tenants’ with ./sg_tenants.yml
SUCC: Configuration for ‘tenants’ created or updated
Will update ‘_doc/blocks’ with ./sg_blocks.yml
SUCC: Configuration for ‘blocks’ created or updated
Done with success

With the following appearing in the elasticsearch logs:
[2020-08-05T14:59:19,197][INFO ][c.f.s.c.ConfigurationRepository] [spoonguard] Search Guard License Info: SearchGuardLicense [uid=00000000-0000-0000-0000-000000000000, type=TRIAL, features=[COMPLIANCE], issueDate=2020-08-05, expiryDate=2020-10-04, issuedTo=The world, issuer=floragunn GmbH, startDate=2020-08-05, majorVersion=7, clusterName=*, allowedNodeCount=2147483647, msgs=, expiresInDays=60, isExpired=false, valid=true, action=, prodUsage=Yes, one cluster with all commercial features and unlimited nodes per cluster., clusterService=org.elasticsearch.cluster.service.ClusterService@5243a948, getMsgs()=, getExpiresInDays()=60, isExpired()=false, isValid()=true, getAction()=, getProdUsage()=Yes, one cluster with all commercial features and unlimited nodes per cluster.]
[2020-08-05T14:59:19,197][INFO ][c.f.s.c.ConfigurationRepository] [spoonguard] Search Guard License Type: TRIAL, valid
[2020-08-05T14:59:19,923][ERROR][c.f.s.a.BackendRegistry ] [spoonguard] Not yet initialized (you may need to run sgadmin)
[2020-08-05T14:59:22,425][ERROR][c.f.s.a.BackendRegistry ] [spoonguard] Not yet initialized (you may need to run sgadmin)

No matter how many times I run sgadmin.sh I get the same behaviour.
Is there any way to get the cluster into an initialised state?

searchguard-es.log (24.8 KB) searchguard-elasticsearch.yml (1.5 KB)

2

Hi.
You don’t need to run sgadmin to initialize the Search Guard. You need it to apply the configuration update.

I can’t reproduce your issue. Can you show more log? The log you send is less than 1 minute long. Can you start the Elasticsearch cluster without Search Guard? Is it green?

You can try Search Guard with our labs repo search-guard / labs · GitLab

# Run docker.
# Clone the repo, configure and run the containers:
git clone git@git.floragunn.com:search-guard/search-guard-labs.git
cd search-guard-labs
./create_config.sh
docker-compose -f docker-compose-basicauth.yml up
# Wait for containers to run and Search Guard to initialize.

Verify

$ curl -k -u admin:admin -X GET https://localhost:9200/_cluster/health?pretty
{ 
  "cluster_name" : "es-docker-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 9,
  "active_shards" : 19,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

$ curl -k -u admin:admin -X GET https://localhost:9200/_searchguard/authinfo?pretty
{
  "user" : "User [name=admin, backend_roles=[admin], requestedTenant=null]",
  "user_name" : "admin",
  "user_requested_tenant" : null,
  "remote_address" : "172.18.0.1:43336",
  "backend_roles" : [
    "admin"
  ],
  "custom_attribute_names" : [ ],
  "sg_roles" : [
    "SGS_ALL_ACCESS",
    "SGS_OWN_INDEX"
  ],
  "sg_tenants" : {
    "admin_tenant" : true,
    "admin" : true,
    "SGS_GLOBAL_TENANT" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}
3

Look at the typical use cases for sgadmin Examples | Security for Elasticsearch | Search Guard

4

Hi,
I can confirm that the cluster loads up and is “green” if I disable Searchguard:
curl http://spoonguard.is.ed.ac.uk:9200/_cluster/health
{“cluster_name”:“spoonguard”,“status”:“green”,“timed_out”:false,“number_of_nodes”:2,“number_of_data_nodes”:2,“active_primary_shards”:13,“active_shards”:26,“relocating_shards”:0,“initializing_shards”:0,“unassigned_shards”:0,“delayed_unassigned_shards”:0,“number_of_pending_tasks”:0,“number_of_in_flight_fetch”:0,“task_max_waiting_in_queue_millis”:0,“active_shards_percent_as_number”:100.0}

I’ve attached the 2 logs of our nodes starting up with Searchguard disabled.

I was following the documentation at: Installation | Security for Elasticsearch | Search Guard and got as far as the “Initializing Search Guard” section before hitting this issue.

*Update- I’ve went back to the “default” sgconfig provided and applied these and the issue goes away. I’d made some changes to the sg_config.yml file to try and enable LDAP authentication. The file did pass validation though so I’m not sure why it caused this issue. I’ve included a sanitised version of the file though in case there’s something obvious (Ideally we’d like to use SAML but thought LDAP would be a reasonable first step)sg_config.yml (7.9 KB)

spoonguard.log (21.1 KB) spoonguard-2.log (9.5 KB)

5

Hi.
I looked at your sg_config.yml and didn’t see basic_internal_auth_domain in the auth section. Add it.

For example

_sg_meta:
  type: "config"
  config_version: 2

sg_config:
  dynamic:
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        #internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        internalProxies: '.*' # trust all internal proxies, regex pattern
        remoteIpHeader:  'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      ldap:
        http_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          # LDAP authentication backend (authenticate users against a LDAP or Active Directory)
          type: ${env.TESTARG_LDAP_TYPE}
          config:
            # enable ldaps
            enable_ssl: true
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
              - ldap.example.com:636
            bind_dn: null
            password: null
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(uid={0})'
            # Use this attribute from the user as username (if not set then DN is used)
            username_attribute: uid
    authz:
      roles_from_myldap:
        http_enabled: true
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ${env.TESTARG_LDAP_TYPE}
          config:
            # enable ldaps
            enable_ssl: true
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
              - ldap.example.com:636
            bind_dn: null
            password: null
            rolebase: 'ou=groups,dc=example,dc=com'
            # Filter to search for roles (currently in the whole subtree beneath rolebase)
            # {0} is substituted with the DN of the user
            # {1} is substituted with the username
            # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
            rolesearch: '(uniqueMember={0})'
            # Specify the name of the attribute which value should be substituted with {2} above
            userroleattribute: null
            # Roles as an attribute of the user entry
            userrolename: memberOf
            # The attribute in a role entry containing the name of that role
            rolename: cn
            # Resolve nested roles transitive (roles which are members of other roles and so on ...)
            resolve_nested_roles: false
            userbase: 'ou=people,dc=example,dc=com'
            usersearch: '(uid={0})'
            username_attribute: uid
            skip_users:
              - kibanaserver
6

Thanks- that’s it working now. The next task will be setting up LDAP authentication/role mapping then (hopefully) SAML login :slight_smile:

1 Like
7

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.