Iceberg Connector

Note

It is recommended to use Iceberg 0.9.0 or later.

To configure the Iceberg connector, create a catalog properties file etc/catalog/iceberg.properties with the following contents, replacing the properties as appropriate:

The following configuration properties are available:

The URI(s) of the Hive metastore to connect to using the Thrift protocol. If multiple URIs are provided, the first URI is used by default and the rest of the URIs are fallback metastores. This property is required. Example: thrift://192.0.2.3:9083 or thrift://192.0.2.3:9083,thrift://192.0.2.4:9083

iceberg.file-format

The default is PARQUET.

The compression codec to use when writing files. The available values are NONE, SNAPPY, GZIP, LZ4, and ZSTD.

The default is GZIP.

iceberg.catalog.type

The catalog type for Iceberg tables. The available values are and hadoop, corresponding to the catalogs in the Iceberg.

The default is hive.

The catalog warehouse root path for Iceberg tables. Example: hdfs://nn:8020/warehouse/path.

This property is required if the iceberg.catalog.type is hadoop. Otherwise, it will be ignored.

iceberg.catalog.cached-catalog-num

The number of Iceberg catalogs to cache.

The default is 10. This property is required if the iceberg.catalog.type is hadoop. Otherwise, it will be ignored.

The path(s) for Hadoop configuration resources. Example: /etc/hadoop/conf/core-site.xml.