- Creation of repositories, relying on existing brokers or accessing cloud storage directly through AWS s3 protocol, or accessing HDFS directly.
- If it is a read-only repository, restores can only be done on the repository. If not, backup and restore operations are available.
- PROPERTIES are different according to different types of broker or S3 or hdfs, see the example for details.
- ON LOCATION : if it is S3 , here followed by the Bucket Name.
- Create a warehouse named bos_repo, rely on BOS broker “bos_broker”, and the data root directory is: bos://palo_backup
WITH BROKER `bos_broker`
ON LOCATION "bos://palo_backup"
PROPERTIES
(
"bos_endpoint" = "http://gz.bcebos.com",
"bos_accesskey" = "bos_accesskey",
"bos_secret_accesskey"="bos_secret_accesskey"
);
- Create the same repository as Example 1, but with read-only properties:
CREATE READ ONLY REPOSITORY `bos_repo`
WITH BROKER `bos_broker`
PROPERTIES
(
"bos_endpoint" = "http://gz.bcebos.com",
"bos_accesskey" = "bos_accesskey",
);
- Create a warehouse named hdfs_repo, rely on Baidu hdfs broker “hdfs_broker”, the data root directory is: hdfs://hadoop-name-node:54310/path/to/repo/
- Create a repository named s3_repo to link cloud storage directly without going through the broker.
CREATE REPOSITORY `s3_repo`
WITH S3
ON LOCATION "s3://s3-repo"
PROPERTIES
(
"AWS_ENDPOINT" = "http://s3-REGION.amazonaws.com",
"AWS_ACCESS_KEY" = "AWS_ACCESS_KEY",
"AWS_SECRET_KEY"="AWS_SECRET_KEY",
"AWS_REGION" = "REGION"
);
- Create a repository named hdfs_repo to link HDFS directly without going through the broker.
CREATE REPOSITORY `hdfs_repo`
WITH hdfs
ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
"fs.defaultFS"="hdfs://hadoop-name-node:54310",
"hadoop.username"="user"
);
### Keywords
- Create a repository named minio_repo to link minio storage directly through the s3 protocol.
- Create a repository named minio_repo via temporary security credentials.
CREATE REPOSITORY `minio_repo`
WITH S3
ON LOCATION "s3://minio_repo"
PROPERTIES
(
"AWS_ENDPOINT" = "AWS_ENDPOINT",
"AWS_ACCESS_KEY" = "AWS_TEMP_ACCESS_KEY",
"AWS_SECRET_KEY" = "AWS_TEMP_SECRET_KEY",
"AWS_TOKEN" = "AWS_TEMP_TOKEN",
"AWS_REGION" = "AWS_REGION"
CREATE, REPOSITORY
- A cluster can create multiple warehouses. Only users with ADMIN privileges can create repositories.
- Any user can view the created repositories through the command.