AWS S3 binding spec

To setup an AWS S3 binding create a component of type . This binding works with other S3-compatible services, such as Minio. See this guide on how to create and apply a binding configuration.

See for information about authentication-related attributes.

Warning

The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described .

Important

When running the Dapr sidecar (daprd) with your application on EKS (AWS Kubernetes), if you’re using a node/pod that has already been attached to an IAM policy defining access to AWS resources, you must not provide AWS access-key, secret-key, and tokens in the definition of the component spec you’re using.

is a service that exposes local storage as S3-compatible block storage, and it’s a popular alternative to S3 especially in development environments. You can use the S3 binding with Minio too, with some configuration tweaks:

  1. Set endpoint to the address of the Minio server, including protocol (http:// or https://) and the optional port at the end. For example, http://minio.local:9000 (the values depend on your environment).
  2. forcePathStyle must be set to true
  3. The value for region is not important; you can set it to us-east-1.
  4. Depending on your environment, you may need to set disableSSL to true if you’re connecting to Minio using a non-secure connection (using the http:// protocol). If you are using a secure connection (https:// protocol) but with a self-signed certificate, you may need to set insecureSSL to true.

This component supports output binding with the following operations:

To perform a create operation, invoke the AWS S3 binding with a POST method and the following JSON body:

  1. {
  2.   "operation": "create",
  3.   "data": "YOUR_CONTENT"
  4. }

Share object with a presigned URL

  1. curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\", \"metadata\": { \"presignTTL\": \"15m\" } }" \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1. curl -d '{ "operation": "create", "data": "Hello World", "metadata": { "presignTTL": "15m" } }' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
Response

The response body contains the following example JSON:

  1. {
  2.     "location":"https://<your bucket>.s3.<your region>.amazonaws.com/<key>",
  3.     "versionID":"<version ID if Bucket Versioning is enabled>",
  4.     "presignURL": "https://<your bucket>.s3.<your region>.amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host"
  5. }

Examples

Save text to a random generated UUID file

On Windows, utilize cmd prompt (PowerShell has different escaping mechanism)

  1. curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\" }" http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
Save text to a specific file
  1. curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\", \"metadata\": { \"key\": \"my-test-file.txt\" } }" \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
Save a file to a object

To upload a file, encode it as Base64 and let the Binding know to deserialize it:

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: <NAME>
  5. spec:
  6.   type: bindings.aws.s3
  7.   version: v1
  8.   metadata:
  9.   - name: bucket
  10.     value: mybucket
  11.   - name: region
  12.     value: us-west-2
  13.   - name: endpoint
  14.     value: s3.us-west-2.amazonaws.com
  15.   - name: accessKey
  16.     value: *****************
  17.   - name: secretKey
  18.     value: *****************
  19.   - name: sessionToken
  20.     value: mysession
  21.   - name: decodeBase64
  22.     value: <bool>
  23.   - name: forcePathStyle
  24.     value: <bool>

Then you can upload it as you would normally:

  1. curl -d "{ \"operation\": \"create\", \"data\": \"YOUR_BASE_64_CONTENT\", \"metadata\": { \"key\": \"my-test-file.jpg\" } }" http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1. curl -d '{ "operation": "create", "data": "YOUR_BASE_64_CONTENT", "metadata": { "key": "my-test-file.jpg" } }' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
Upload from file path

To upload a file from a supplied path (relative or absolute), use the filepath metadata key on a create request that contains empty data fields.

  1. curl -d '{ \"operation\": \"create\", \"metadata\": { \"filePath\": \"my-test-file.txt\" }}' http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1. curl -d '{ "operation": "create", "metadata": { "filePath": "my-test-file.txt" }}' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

The response body will contain the following JSON:

  1. {
  2.     "location":"https://<your bucket>.s3.<your region>.amazonaws.com/<key>",
  3.     "versionID":"<version ID if Bucket Versioning is enabled"
  4. }

Presign an existing object

To presign an existing S3 object with a specified time-to-live, use the presignTTL and key metadata keys on a presign request. Valid values for are .

  1. curl -d "{ \"operation\": \"presign\", \"metadata\": { \"presignTTL\": \"15m\", \"key\": \"my-test-file.txt\" } }" \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
Response

The response body contains the following example JSON:

  1. {
  2.     "presignURL": "https://<your bucket>.s3.<your region>.amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host"
  3. }
  1. {
  2.   "operation": "get",
  3.   "metadata": {
  4.   }
  5. }

The metadata parameters are:

  • key - the name of the object

Example

  1. curl -d '{ \"operation\": \"get\", \"metadata\": { \"key\": \"my-test-file.txt\" }}' http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1. curl -d '{ "operation": "get", "metadata": { "key": "my-test-file.txt" }}' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

The response body contains the value stored in the object.

To perform a delete object operation, invoke the AWS S3 binding with a POST method and the following JSON body:

  1. {
  2.   "operation": "delete",
  3.   "metadata": {
  4.     "key": "my-test-file.txt"
  5.   }
  6. }

The metadata parameters are:

  • key - the name of the object

Examples

Delete object
  1. curl -d '{ \"operation\": \"delete\", \"metadata\": { \"key\": \"my-test-file.txt\" }}' http://localhost:<dapr-port>/v1.0/bindings/<binding-name>
  1. curl -d '{ "operation": "delete", "metadata": { "key": "my-test-file.txt" }}' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

An HTTP 204 (No Content) and empty body will be returned if successful.

To perform a list object operation, invoke the S3 binding with a POST method and the following JSON body:

The data parameters are:

  • maxResults - (optional) sets the maximum number of keys returned in the response. By default the action returns up to 1,000 key names. The response might contain fewer keys but will never contain more.
  • prefix - (optional) limits the response to keys that begin with the specified prefix.
  • marker - (optional) marker is where you want Amazon S3 to start listing from. Amazon S3 starts listing after this specified key. Marker can be any key in the bucket. The marker value may then be used in a subsequent call to request the next set of list items.
  • delimiter - (optional) A delimiter is a character you use to group keys.

Response

The response body contains the list of found objects.

  1. {
  2.     "CommonPrefixes": null,
  3.     "Contents": [
  4.         {
  5.             "ETag": "\"7e94cc9b0f5226557b05a7c2565dd09f\"",
  6.             "Key": "hpNdFUxruNuwm",
  7.             "LastModified": "2021-08-16T06:44:14Z",
  8.             "Owner": {
  9.                 "DisplayName": "owner name",
  10.                 "ID": "owner id"
  11.             },
  12.             "Size": 6916,
  13.             "StorageClass": "STANDARD"
  14.         }
  15.     ],
  16.     "Delimiter": "",
  17.     "EncodingType": null,
  18.     "IsTruncated": true,
  19.     "Marker": "hvlcCQFSOD5TD",
  20.     "MaxKeys": 1,
  21.     "Name": "mybucketdapr",
  22.     "NextMarker": "hzaUPWjmvyi9W",
  23.     "Prefix": ""

Last modified October 4, 2022: Add AWS S3 presign url (#2832) (55d7bb32)