Securing Elasticsearch

To improve Elasticsearch security we use the ReadonlyREST plugin to:

• enforce access control for operations

• run on https By default the Portal Dedicated and Agents communicate with Elasticsearch over http. To upgrade to https requires reconfiguring:

  • Elasticsearch to run on https
  • Portal Dedicated to use https
  • Agents to use https

Install ReadonlyREST plugin

The ReadonlyREST plugin for Elasticsearch 8.8.2 is bundled with the Portal Dedicated artefact in the elasticsearch directory.

For other versions of Elasticsearch download the free Elasticsearch plugin from https://readonlyrest.com/download/

If you download a different version of ReadonlyREST directly from their site, the downloaded file will be a zip file and can be installed using the command below, by updating the path to the downloaded file, and replacing the jar file name with the zip file name.

To install the plugin:

Update the path to the readonlyrest plugin in the command below based on where the Portal Dedicated is extracted to

sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install -b file:////opt/waratek/elasticsearch/readonlyrest-1.49.1.zip

If successfully installed the output should be similar to:

\-> Downloading file:////opt/waratek/elasticsearch/readonlyrest-1.49.1.zip
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission << ALL FILES >> read
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission getProperty.ssl.KeyManagerFactory.algorithm
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.
-> Installed readonlyrest

Patch Elasticsearch

If you are using Elasticsearch 8.0.x or newer, there is an extra post-installation step. Depending on the Elasticsearch version, this command might tweak the main Elasticsearch installation files and/or copy some jars to plugins/readonlyrest directory.

sudo /usr/share/elasticsearch/jdk/bin/java -jar /usr/share/elasticsearch/plugins/readonlyrest/ror-tools.jar patch

For Elasticsearch 8.3.x or newer, the patching operation requires root user privileges.

To verify that ReadonlyREST was successfully installed:

sudo /usr/share/elasticsearch/bin/elasticsearch-plugin list

should output a list of installed plugins:

readonlyrest

Configure ReadonlyREST access control

As this is a YAML file, it is important to preserve the structure below. Otherwise, Elasticsearch might fail to start

This sample file is provided in the Portal Dedicated distribution in the elasticsearch directory

Copy the readonlyrest.yml file which is bundled with the Portal, into /etc/elasticsearch , using the following command

sudo cp <MC-FOLDER>/elasticsearch/readonlyrest.yml /etc/elasticsearch/readonlyrest.yml

This configures Elasticsearch to restrict access from Portal Dedicated to a single user. The supplied hash configures the user:password as portal:Portal1234.

This configures Elasticsearch to restrict access from Agents to a single user, and restricts that user to creating new documents only. The supplied hash configures the user:password as instance:Testpass123.

These credentials are set as the default values in the Portal Dedicated and Agent configuration files

Create new username/passwords for Elasticsearch connectivity

It is optional, but recommended to generate new SHA-256 strings with new credentials for accessing Elasticsearch from the MC and the Agents

Although for older Waratek Agents the username value must be instance

To generate new SHA-256 strings you can simply replace username and password in the Linux command below and execute it in your terminal :

echo -n username:password | sha256sum

Example:

echo -n myNewUsername:myNewPassword | sha256sum

The output of that command is used to replace the auth_key_sha256 value in relevant section of readonlyrest.yml

For the Portal Dedicated the Elasticsearch properties in application.properties must be updated:

elasticsearch.cluster.username=myNewUsername
elasticsearch.cluster.password=myNewPassword

For the Agents the property for the Elasticsearch password in waratek.properties must be updated:

com.waratek.ElasticsearchUsername=instance
com.waratek.ElasticsearchPassword=Testpass123

This completes step 1 of securing Elasticsearch and restarting all components should enforce access control

Configure ReadonlyREST SSL

Before proceeding with the below steps, you should have installed ReadonlyREST, see above

This consists of 3 required steps:

1. Elasticsearch to run on HTTPS
2. Portal Dedicated to use HTTPS
3. Agents to use HTTPS

1. Elasticsearch to use HTTPS

First, configure Elasticsearch to restrict traffic to HTTPS, by editing/etc/elasticsearch/elasticsearch.yml as sudo, and adding this line:

http.type: ssl_netty4

Next, configure ReadonlyREST to use HTTPS, by editing /etc/elasticsearch/readonlyrest.yml as sudo, and uncommenting the SSL section, so that the file content looks similar to this:

readonlyrest:
  access_control_rules:
    - name: "Allows all methods for Portal instances"
      auth_key_sha256: 5e2040c5a456abc246f1cf143112ae4f753c73bcf7fc07aaeba8624f9436bf10
      verbosity: error
    - name: "Allows inserting records for agents"
      methods: [PUT, POST, HEAD]
      actions: ["indices:data/write/bulk", "indices:data/write/index", "cluster:monitor/main"]
      auth_key_sha256: d72b3afdabe49e0512b79221ae51e3eec6b6abdbbf0e168277a6e3f45feead64
      verbosity: error
  ssl:
    enable: true
    keystore_file: "es.keystore.p12"
    keystore_pass: password
    key_pass: password
    allowed_protocols: [TLSv1,TLSv1.1,TLSv1.2]

Next, copy the Portal Dedicated keystore file into a new file called /etc/elasticsearch/es.keystore.p12, by running the following command:

sudo cp controller.keystore.p12 /etc/elasticsearch/es.keystore.p12

2. Portal Dedicated to use HTTPS

Update the Elasticsearch location in application.properties to specify HTTPS:

elasticsearch.cluster.urls=https://localhost:9200

3. Agents to use HTTPS

Using the keystore located at /opt/waratek/controller.keystore.p12, run the following keytool command as sudo to create a certificate, and enter your password. Note, use the keytool that is bundled in the Portal Dedicated jre/bin/keytool directory.

sudo </opt/waratek/jre/bin/keytool -exportcert -keystore controller.keystore.p12 -alias PortalAlias -file PortalDedicatedCert.crt

When successfully created, the following output is logged by the keytool command:

Certificate stored in file <PortalDedicatedCert.crt>

Copy PortalDedicatedCert.crt over to the server where the agent to be on-boarded is running.

To complete the agent side configuration for SSL communication, see the Agent-onboarding - Configuring TLS communication agent documentation section.

This completes the final step of securing Elasticsearch and restarting all components should switch to SSL communication.

You have successfully reached Milestone Progress Checkpoint #3