Elasticsearch

Learn how to use Elasticsearch with Filebase.

What is Elasticsearch?

Elasticsearch is a search engine that provides a distributed full-text search engine using an HTTP web interface and JSON documents.

Elasticsearch has been certified for use with Filebase. Read below to learn how to configure Elasticsearch for use with Filebase.

Prerequisites:

• Download and install Elasticsearch.

• Sign up for a free Filebase account.

• Have your Filebase Access and Secret Keys. Learn how to view your access keys here.

• Create a Filebase Bucket. Learn how to create a bucket here.

This guide was tested and written using Ubuntu 20.04 in single-node mode. The Elasticsearch configuration should be tailored to your environment and adjusted to fit your operating system.

1. Install HTTPie HTTP client for making local HTTP requests to Elasticsearch:

sudo snap install http

2. Install the Elasticsearch repository-s3 plugin:

/PATH in this guide refers to the path that Elasticsearch has been installed to. For example, the path for this configuration was: /home/filebase/elasticsearch-7.6.1/

sudo $PATH/bin/elasticsearch-plugin install -b repository-s3

3. Restart Elasticsearch Service to enable repository-s3 plugin:

$PATH/bin/elasticsearch

4. Validate plugin has been installed:

sudo $PATH/bin/elasticsearch-plugin list

5. Set Filebase Access and Secret Keys:

sudo $PATH/bin/elasticsearch-keystore add s3.client.default.access_key

• When prompted to create a new keystore, select Y.

• When prompted for the Access Key, provide your Filebase Access Key.

sudo $PATH/bin/elasticsearch-keystore add s3.client.default.secret_key

• When prompted to create a new keystore, select Y.

• When prompted for the Secret Key, provide your Filebase Secret Key.

6. Reload Elasticsearch Secure Settings:

http POST 127.0.0.1:9200/_nodes/reload_secure_settings

7. Send JSON Request to Elasticsearch and Create a Repository

The following is an example JSON request. You need to update the bucket name to reflect your Filebase bucket name.

{
    "type": "s3",
        "settings": {
        "bucket": "filebase-sample-bucket",
        "endpoint": "s3.filebase.com”,
        “region”: ”us-east-1”
    }
}

You must specify the region, otherwise you will receive an access denied error.

8. Send the JSON request in an HTTP PUT to the Elasticsearch server running on 127.0.0.1:

echo '{ "type":"s3","settings":{"bucket":"filebase-sample-bucket","endpoint":"https://s3.filebase.com",”region”:”us-east-1”}}' | http PUT 127.0.0.1:9200/_snapshot/my_filebase_repository

You need to update the bucket name to reflect your Filebase bucket name.

9. Get a list of indices if you only want to take a snapshot of certain indices:

http 127.0.0.1:9200/_cat/indices?v=true

10. Take a snapshot of all of your indices:

http PUT 127.0.0.1:9200/_snapshot/my_filebase_repository/full_snapshot_021021?wait_for_completion=true

This command will take a snapshot of all of your indices, listed with the previous command, and push them to my_filebase_repository. This will upload them to the Filebase bucket you specified in your JSON request configured above.

11. Take a snapshot of a certain index only:

echo '{"indices":".geoip*"}' | http PUT 127.0.0.1:9200/_snapshot/my_filebase_repository/geoip-021021?wait_for_completion=true

12. Validate Snapshots:

http 127.0.0.1:9200/_snapshot/my_filebase_repository/full_snapshot_021021