5.4. Elasticsearch Connector

Note

It is highly recommended to use Elasticsearch 6.0.0 or later.

Configuration

To configure the Elasticsearch connector, create a catalog properties file with the following contents,replacing the properties as appropriate:

The following configuration properties are available:

Defines the schema that will contain all tables defined withouta qualifying schema name.

This property is optional; the default is default.

elasticsearch.table-description-directory

Specifies a path under the Presto deployment directory that containsone or more JSON files with table descriptions (must end with .json).

This property is optional; the default is etc/elasticsearch.

elasticsearch.scroll-size

This property defines the maximum number of hits that can be returned with eachElasticsearch scroll request.

This property is optional; the default is 1000.

elasticsearch.scroll-timeout

This property defines the amount of time (ms) Elasticsearch will keep the search context alive for scroll requests

This property is optional; the default is 20s.

elasticsearch.max-hits

This property defines the maximum number of hits an Elasticsearch request can fetch.

This property defines the timeout value for all Elasticsearch requests.

This property is optional; the default is 10s.

elasticsearch.max-request-retries

This property defines the maximum number of Elasticsearch request retries.

This property is optional; the default is 5.

elasticsearch.max-request-retry-time

Use exponential backoff starting at 1s up to the value specified by this configuration when retrying failed requests.

This property is optional; the default is 10s.

Search Guard Authentication

The Elasticsearch connector provides additional security options to support Elasticsearch clusters that have been configured to use Search Guard.

You can configure the certificate format by setting the config property in the Elasticsearch catalog properties file. The allowed values for this configuration are:

If you use X.509 PEM certificates and PKCS #8 keys, the following properties must be set:

If you use Keystore and Truststore files, the following properties must be set:

searchguard.ssl.transport.pemcert_filepath

The path to the X.509 node certificate chain. This file must be readable by the operating system user running Presto.

This property is optional; the default is etc/elasticsearch/esnode.pem.

searchguard.ssl.transport.pemkey_filepath

The path to the certificates key file. This file must be readable by the operating system user running Presto.

The key password for the key file specified by searchguard.ssl.transport.pemkey_filepath.

This property is optional; the default is empty string.

searchguard.ssl.transport.pemtrustedcas_filepath

The path to the root CA(s) (PEM format). This file must be readable by the operating system user running Presto.

This property is optional; the default is etc/elasticsearch/root-ca.pem.

searchguard.ssl.transport.keystore_filepath

The path to the keystore file. This file must be readable by the operating system user running Presto.

This property is optional; the default is etc/elasticsearch/keystore.jks.

searchguard.ssl.transport.keystore_password

The keystore password for the keystore file specified by searchguard.ssl.transport.keystore_filepath

This property is optional; the default is empty string.

searchguard.ssl.transport.truststore_filepath

The path to the truststore file. This file must be readable by the operating system user running Presto.

This property is optional; the default is .

The truststore password for the truststore file specified by searchguard.ssl.transport.truststore_password

This property is optional; the default is empty string.

Elasticsearch stores the data across multiple nodes and builds indices for fast retrieval.For Presto, this data must be mapped into columns to allow queries against the data.

{
    "tableName": ...,
    "schemaName": ...,
    "hostAddress": ...,
    "port": ...,
    "index": ...,
    "indexExactMatch": ...,
    "type": ...
    "columns": [
        {
            "name": ...,
            "type": ...,
            "jsonPath": ...,
            "jsonType": ...,
            "ordinalPosition": ...
        }
    ]
}

Elasticsearch Column Metadata

Optionally, column metadata can be described in the same table description JSON file with these fields: