Amazon S3 Integration
Amazon S3 is a cloud storage service.
Instance and workspace object storage are different from using S3 resources within scripts, flows, and apps, which is free and unlimited. This is what is described in this page.
At the workspace level, what is exclusive to the Enterprise version is using the integration of Windmill with S3 that is a major convenience layer to enable users to read and write from S3 without having to have access to the credentials.
Additionally, for instance integration, the Enterprise version offers advanced features such as large-scale log management and distributed dependency caching.
Windmill provides a unique resource type for any API following the typical S3 schema.
If you're looking for a way to self-host Windmill using AWS, see Self-Host Windmill.
Add an S3 Resource
Amazon S3's API follows the same schema as any S3 compatible API.
To integrate Amazon S3 to Windmill, you need to save the following elements as a resource.
| Property | Type | Description | Default | Required | Where to Find | Additional Details |
|---|---|---|---|---|---|---|
| bucket | string | S3 bucket name | true | AWS Management Console | Name of the S3 bucket to access | |
| region | string | AWS region for the S3 bucket | true | AWS Management Console | Region where the S3 bucket is located. Can also be found by checking the endpoint URL for the bucket. In the form eu-west-3 | |
| useSSL | boolean | Use SSL for connections | true | false | AWS Management Console or Custom S3-compatible API | SSL/TLS is required for Amazon S3 |
| endPoint | string | S3 endpoint | true | AWS Management Console or Custom S3-compatible API | Endpoint URL will vary by region or custom API provider. Can be found in the AWS documentation in the form s3.eu-west-3.amazonaws.com. | |
| accessKey | string | AWS access key | true for Amazon | AWS Management Console (IAM) | Access key ID for AWS account owner. Can be found in the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated. | |
| pathStyle | boolean | Use path-style addressing | false | true | AWS Management Console or Custom S3-compatible API | Default is virtual-hosted-style URLs |
| secretKey | string | AWS secret key | true for Amazon | AWS Management Console (IAM) | Secret access key for AWS account owner. Can be found in the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated. |
Your resource can be used passed as parameters or directly fetched within scripts, flows and apps.
Example of a Supabase resource being used in two different manners from a script in Windmill.
Workspace object storage
Once you've created an S3 or Azure Blob resource in Windmill, you can use Windmill's native integration with S3 and Azure Blob, making it the recommended storage for large objects like files and binary data.
The workspace object storage is exclusive to the Enterprise edition. It is meant to be a major convenience layer to enable users to read and write from S3 without having to have access to the credentials.
Instance object storage
Under Enterprise Edition, Instance object storage offers advanced features to enhance performance and scalability at the instance level. This integration is separate from the Workspace object storage and provides solutions for large-scale log management and distributed dependency caching.
