# Console (/en/administration/console) The **RustFS Console** is the web administration interface for RustFS. Use this page to enable the Console, open the login page, and choose the appropriate sign-in method. Detailed procedures for buckets, objects, and identity management are covered in their respective documentation sections. ## Enable the Console [#enable-the-console] The Console is enabled by default and listens on port `9001`, separately from the S3 API on port `9000`. You can set the behavior explicitly with the following environment variables: ```ini title="/etc/default/rustfs" RUSTFS_ADDRESS=":9000" RUSTFS_CONSOLE_ADDRESS=":9001" RUSTFS_CONSOLE_ENABLE=true RUSTFS_OBS_LOGGER_LEVEL=error RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" ``` Restart RustFS after changing these values. Set `RUSTFS_CONSOLE_ENABLE=false` when the Console must not run. The equivalent command-line options are `--console-enable` and `--console-address`. See the [CLI reference](/reference/cli) and [environment variable reference](/reference/environment-variables) for the complete server configuration. ## Open the Console [#open-the-console] Open the following address, replacing `` with the RustFS server address: ```text http://:9001 ``` RustFS Console login page with key, STS, and OIDC sign-in options For a local deployment, use `http://localhost:9001`. Windows and macOS desktop launchers use port `7001` instead. If the login page cannot reach the target RustFS service, select **Server Configuration** or open `/config`. Enter the externally reachable RustFS service address and save it after the health check succeeds. **Reset** clears the saved address; **Skip** returns to login without changing it. ## Log in [#log-in] The login methods shown depend on the deployment configuration: * **Key Login** uses the access key and secret key configured for the RustFS deployment. This is the standard login method for a local administrator. * **STS Login** uses temporary Security Token Service (STS) credentials. Use it only when your identity workflow has issued a valid session token. * **OIDC Login** appears when an OpenID Connect (OIDC) provider is configured. Select the provider and complete authentication with the identity provider. After login, the Console opens the first page your account can access. Menus and actions vary by account policy and enabled platform capabilities; a missing menu does not necessarily indicate a Console error. If login fails, verify the selected login method, credentials, target server address, and account status before retrying. RustFS falls back to `rustfsadmin` / `rustfsadmin` when custom credentials are not configured. Use these defaults only for a throwaway local test. Configure a unique access key and a strong secret key before making the Console reachable by other users. ## Operational notes [#operational-notes] * Use [TLS](/integration/tls-configured) before exposing the Console outside a trusted network. * Restrict network access to the Console listener and configure [Console CORS](/administration/cors) only when cross-origin access is required. * The Console session inherits the permissions of the signed-in identity. Use a least-privilege account for routine work. * Signing out or an expired session returns you to the login page. Do not store administrator credentials in shared browsers. ## Management workflows [#management-workflows] * [Create and manage buckets](/administration/data/bucket/creation) * [Upload and manage objects](/administration/data/object/creation) * [Manage access keys](/security-compliance/iam/access-token) * [Configure identity and access management](/security-compliance/iam) ## Next steps [#next-steps] Review the [security checklist](/installation/requirement/checklists/security-checklists) before exposing the Console outside a trusted network. For OIDC-based login, continue with the [OIDC configuration guide](/security-compliance/oidc). # CORS Configuration (/en/administration/cors) Cross-Origin Resource Sharing (CORS) controls which browser origins can access the RustFS S3 API and Console. Configure each listener separately, then restart RustFS to apply the environment changes. ## S3 API origins [#s3-api-origins] Set `RUSTFS_CORS_ALLOWED_ORIGINS` to a comma-separated list of trusted origins: ```ini title="/etc/default/rustfs" RUSTFS_CORS_ALLOWED_ORIGINS="https://app.example.com,https://admin.example.com" ``` When this variable is unset or empty, the S3 endpoint does not add generic CORS response headers. A list of explicit origins allows credentialed browser requests from matching origins. You can set the value to `*` to allow requests from any origin. Wildcard mode does not allow browser credentials. Use a comma-separated allowlist for applications that send credentials. Reserve `*` for public resources that do not require credentialed browser requests. ## Console origins [#console-origins] The Console uses a separate variable: ```ini title="/etc/default/rustfs" RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS="https://admin.example.com" ``` Use a comma-separated list when more than one browser origin must access the Console. Set `*` only when any origin must be allowed. ## Verify the response [#verify-the-response] Send a request with an `Origin` header and inspect the CORS response headers: ```bash curl -i \ -H "Origin: https://app.example.com" \ http://localhost:9000/ ``` Confirm that `Access-Control-Allow-Origin` contains the expected origin. Repeat the check with an unlisted origin and confirm that it is not allowed. ## Next steps [#next-steps] See the [environment variable reference](/reference/environment-variables#cors) for the verified defaults and configuration formats. # RustFS Bucket Creation (/en/administration/data/bucket/creation) This guide explains how to create buckets using the RustFS UI, `rc`, or the S3 API. ## Requirements [#requirements] * A running RustFS instance (see [Installation Guide](../../../installation/index.md)). * [`rc`](/operations/rc) installed and configured with an alias for the command-line workflow. ## Using the RustFS UI [#using-the-rustfs-ui] 1. Log in to the RustFS Console. 2. On the Buckets page, in the top right corner, select **Create Bucket**. 3. Enter the bucket name and click **Create** to complete bucket creation. bucket creation ## Using `rc` [#using-rc] See the [`rc` guide](/operations/rc) for installation and alias configuration. Create a bucket: ```bash rc bucket create rustfs/my-bucket rc bucket list rustfs/ ``` ```text ✓ Bucket 'rustfs/my-bucket' created successfully. ``` ## Using the API [#using-the-api] Create a bucket via API: ```http PUT /{bucketName} HTTP/1.1 ``` S3 requests must be signed with AWS Signature V4, so use an S3 client rather than hand-crafting headers. With the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) configured for your access keys: ```bash aws s3api create-bucket \ --bucket bucket-creation-by-api \ --endpoint-url http://localhost:9000 ``` Verify the bucket creation in the RustFS Console. # RustFS Bucket Deletion (/en/administration/data/bucket/deletion) This guide explains how to delete buckets using the RustFS UI, `rc`, or the S3 API. ## Requirements [#requirements] * Install and configure [`rc`](/operations/rc) before using the command-line workflow. * Empty the target bucket before deleting it, or use `--force` only after reviewing the objects that will be removed. **Warning**: Deleting a bucket is irreversible and may break applications relying on it. Ensure you have backed up any necessary data before proceeding. ## Using the RustFS UI [#using-the-rustfs-ui] 1. Log in to the RustFS Console. 2. On the homepage, select the bucket you want to delete. 3. On the far right, select the **Delete** button. 4. In the popup dialog, click **Confirm** to complete bucket deletion. bucket deletion ## Using `rc` [#using-rc] See the [`rc` guide](/operations/rc) for installation and alias configuration. Delete a bucket: ```bash rc bucket remove rustfs/my-bucket ``` ```text ✓ Bucket 'rustfs/my-bucket' removed successfully. ``` ## Using the API [#using-the-api] Delete a bucket via API: ```http DELETE /{bucketName} HTTP/1.1 ``` S3 requests must be signed with AWS Signature V4, so use an S3 client rather than hand-crafting headers. With the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) configured for your access keys: ```bash aws s3api delete-bucket \ --bucket bucket-creation-by-api \ --endpoint-url http://localhost:9000 ``` Verify the bucket deletion in the RustFS Console. # Bucket Policy (/en/administration/data/bucket/policy) RustFS bucket policies are S3-compatible resource policies attached directly to a bucket. Use them to grant or deny access to the bucket and its objects, including controlled anonymous access for public downloads. ## Overview [#overview] A bucket policy contains one or more statements that match a principal, action, resource, and optional conditions. RustFS evaluates the policy for requests to the bucket before allowing the storage operation. Bucket policies and IAM policies serve different purposes: | Policy type | Attached to | Typical use | | ------------- | --------------- | ---------------------------------------------------------------------------------------- | | Bucket policy | A bucket | Grant public access, add resource-level restrictions, or authorize access to one bucket. | | IAM policy | A user or group | Define what an authenticated identity can do across one or more resources. | An explicit `Deny` takes precedence over an `Allow`. Bucket owners remain able to get, replace, or delete the bucket policy so that a deny statement cannot permanently lock policy administration. RustFS implements the standard S3 operations `PutBucketPolicy`, `GetBucketPolicy`, `GetBucketPolicyStatus`, and `DeleteBucketPolicy`. A statement with `"Principal": "*"` can grant access without authentication. Keep the action and resource scope as narrow as possible, and verify the result anonymously before using the policy in production. ## Configuration [#configuration] ### Requirements [#requirements] * Create the target bucket before applying a policy. * Configure the AWS CLI with a RustFS credential and region `us-east-1`. * Use a credential allowed to perform the required policy-management action. Set reusable variables: ```bash export RUSTFS_ENDPOINT=http://localhost:9000 export BUCKET_NAME=my-bucket ``` Policy management requires these actions: | Operation | Required action | | ------------------------- | -------------------------- | | Apply or replace a policy | `s3:PutBucketPolicy` | | Read a policy | `s3:GetBucketPolicy` | | Read public status | `s3:GetBucketPolicyStatus` | | Remove a policy | `s3:DeleteBucketPolicy` | ### Policy document structure [#policy-document-structure] A bucket policy uses version `2012-10-17` and a `Statement` array: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadObjects", "Effect": "Allow", "Principal": "*", "Action": ["s3:GetObject"], "Resource": ["arn:aws:s3:::my-bucket/public/*"] } ] } ``` The bucket ARN and object ARN are different: | Resource | ARN format | Example actions | | -------- | -------------------------- | ------------------------------------------------- | | Bucket | `arn:aws:s3:::my-bucket` | `s3:ListBucket`, `s3:GetBucketLocation` | | Objects | `arn:aws:s3:::my-bucket/*` | `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject` | Use a prefix in the object ARN, such as `arn:aws:s3:::my-bucket/public/*`, to limit access to part of the bucket. For the complete statement format and supported condition operators, see [Users, Groups, and Policies](/security-compliance/iam/policies#policy-document-format). ### Public Access Block [#public-access-block] If the bucket's Public Access Block configuration has `BlockPublicPolicy` enabled, RustFS rejects a new policy containing an `Allow` statement with a wildcard principal. Keep this protection enabled unless anonymous access is intentional. Public Access Block does not replace careful policy review. An existing explicit `Deny` still overrides allows, and authenticated requests continue to be evaluated against IAM and bucket policies. ## Usage [#usage] ### Create a public-read policy [#create-a-public-read-policy] Create a policy that allows anonymous downloads only from the `public/` prefix: ```bash cat > /tmp/my-bucket-policy.json <<'EOF' { "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadObjects", "Effect": "Allow", "Principal": "*", "Action": ["s3:GetObject"], "Resource": ["arn:aws:s3:::my-bucket/public/*"] } ] } EOF ``` This policy does not allow anonymous bucket listing, uploads, overwrites, or deletes. ### Apply the policy [#apply-the-policy] ```bash aws s3api put-bucket-policy \ --bucket "$BUCKET_NAME" \ --policy file:///tmp/my-bucket-policy.json \ --endpoint-url "$RUSTFS_ENDPOINT" ``` Applying another policy replaces the complete existing policy. Retrieve and review the current document before updating it; statements are not merged automatically. ### Read the policy [#read-the-policy] ```bash aws s3api get-bucket-policy \ --bucket "$BUCKET_NAME" \ --endpoint-url "$RUSTFS_ENDPOINT" \ --query Policy \ --output text ``` RustFS preserves the submitted policy JSON. If no policy exists, the API returns `NoSuchBucketPolicy`. ### Check public status [#check-public-status] ```bash aws s3api get-bucket-policy-status \ --bucket "$BUCKET_NAME" \ --endpoint-url "$RUSTFS_ENDPOINT" ``` The response contains a high-level public-access status: ```json { "PolicyStatus": { "IsPublic": false } } ``` The current status check detects anonymous bucket listing (`s3:ListBucket`) and uploads (`s3:PutObject`). A policy that exposes only object downloads, such as the `public/` example above, can still report `false`. Always test the exact anonymous action, object prefix, and conditions instead of treating this status as a complete access analysis. ### Remove the policy [#remove-the-policy] ```bash aws s3api delete-bucket-policy \ --bucket "$BUCKET_NAME" \ --endpoint-url "$RUSTFS_ENDPOINT" ``` Deleting a bucket policy removes only that resource policy. IAM policies attached to users and groups remain unchanged. ## Verification [#verification] Upload a test object with an authenticated credential: ```bash printf 'hello from RustFS\n' > /tmp/hello.txt aws s3api put-object \ --bucket "$BUCKET_NAME" \ --key public/hello.txt \ --body /tmp/hello.txt \ --endpoint-url "$RUSTFS_ENDPOINT" ``` Verify that the allowed object can be downloaded without credentials: ```bash curl --fail-with-body \ "${RUSTFS_ENDPOINT}/${BUCKET_NAME}/public/hello.txt" ``` Then verify that an object outside the allowed prefix is not public: ```bash aws s3api put-object \ --bucket "$BUCKET_NAME" \ --key private/hello.txt \ --body /tmp/hello.txt \ --endpoint-url "$RUSTFS_ENDPOINT" curl --fail-with-body \ "${RUSTFS_ENDPOINT}/${BUCKET_NAME}/private/hello.txt" ``` The second `curl` request should fail with `AccessDenied`. Also test each authenticated user role that relies on the policy, especially when the document contains conditions or explicit deny statements. ## Next steps [#next-steps] * [Manage IAM policies](/security-compliance/iam/policies) * [Manage credentials](/operations/credentials) * [Configure audit logs](/security-compliance/audit-logs) # Bucket Quota (/en/administration/data/bucket/quota) RustFS bucket quotas limit the total object data stored in an individual bucket. Use a quota to prevent one workload from consuming more than its assigned capacity while allowing other buckets to use the remaining storage. ## Overview [#overview] RustFS currently supports hard, byte-based quotas. Before accepting a write, RustFS compares the bucket's current usage plus the requested object size with the configured limit. A write that would exceed the limit is rejected with `InvalidRequest` and a `Bucket quota exceeded` message. Quota checks cover these operations: | Operation | Quota behavior | | ------------------------------ | ----------------------------------------------------------------------------- | | Upload an object | Rejects the upload when current usage plus the object size exceeds the limit. | | Complete a multipart upload | Checks the completed object size before committing the upload. | | Copy an object into the bucket | Checks the source object size against the destination bucket quota. | | Delete an object | Always allowed so that you can recover capacity. | The quota applies to object data, not the number of objects or request rate. A bucket without a configured limit is unlimited. The quota check reserves the full incoming object size against the bucket's current usage. Replacing an existing key can therefore require enough headroom for the complete replacement object. ## Configure in the Console [#configure-in-the-console] 1. Sign in to the RustFS Console and open **Browser**. 2. Find the bucket and select **Settings**. 3. Under **Capacity & Metadata**, find **Bucket Quota** and select **Edit**. 4. Enable **Bucket Quota**. 5. Enter the quota size and select **MiB**, **GiB**, **TiB**, or **PiB**. 6. Select **Save Quota**. Set Bucket Quota dialog with quota size and unit controls To remove the limit, open the dialog again, disable **Bucket Quota**, and save the change. ## Use rc [#use-rc] Configure an alias for your RustFS deployment: ```bash rc alias set rustfs http://localhost:9000 \ \ --region us-east-1 --bucket-lookup path ``` Replace `localhost` with the RustFS server address when `rc` runs on another host. ### Permissions [#permissions] Quota operations require these policy actions: | Operation | Required action | | --------------------------------------- | ---------------------- | | Set or clear a quota | `admin:SetBucketQuota` | | Read quota configuration or statistics | `s3:GetBucketQuota` | | Check whether a proposed operation fits | `s3:GetBucketQuota` | The root credential has these permissions. For delegated administration, attach only the actions needed for the workflow. ### Set a quota [#set-a-quota] Set a 1 GiB hard quota. `rc` accepts byte values or units such as `1G`, `500M`, and `10KB`: ```bash rc bucket quota set rustfs/my-bucket 1G ``` The response includes the configured limit and current usage: ```text Bucket: my-bucket Quota: 1 GiB Usage: 0 B Type: HARD ``` ### Read the quota [#read-the-quota] ```bash rc bucket quota info rustfs/my-bucket ``` Use `--json` when another tool needs to process the result: ```bash rc bucket quota info rustfs/my-bucket --json ``` When no limit is configured, the human-readable output reports `Quota: unlimited`. A query issued immediately after setting or clearing a quota can briefly return the previous state. Query the quota again and confirm the expected value before starting the verification workflow. ### Clear a quota [#clear-a-quota] Remove the limit without deleting objects: ```bash rc bucket quota clear rustfs/my-bucket rc bucket quota info rustfs/my-bucket ``` The bucket becomes unlimited after the configuration change is applied. ## Advanced quota checks [#advanced-quota-checks] `rc 0.1.29` does not expose commands for detailed usage statistics or advisory checks for proposed writes. Use the RustFS Admin API for these operations. Requests must use AWS Signature Version 4 with an active RustFS credential. The examples use these shell variables: ```bash export RUSTFS_ENDPOINT=http://localhost:9000 export RUSTFS_ACCESS_KEY= export RUSTFS_SECRET_KEY= export BUCKET_NAME=my-bucket ``` Environment variables are convenient for local testing but may be visible to processes running as the same operating-system user. Use your platform's secret manager or a restricted credentials file in production. ### Read detailed usage statistics [#read-detailed-usage-statistics] Use the statistics endpoint to retrieve the limit, current usage, remaining bytes, and percentage used: ```bash curl --fail-with-body \ --aws-sigv4 "aws:amz:us-east-1:s3" \ --user "${RUSTFS_ACCESS_KEY}:${RUSTFS_SECRET_KEY}" \ "${RUSTFS_ENDPOINT}/rustfs/admin/v3/quota-stats/${BUCKET_NAME}" ``` ```json { "bucket": "my-bucket", "quota_limit": 1073741824, "current_usage": 1048576, "remaining_quota": 1072693248, "usage_percentage": 0.09765625 } ``` Usage values come from RustFS data-usage accounting. Allow time for that view to reflect very recent changes before using the statistics response for external billing or orchestration. ### Check a proposed upload [#check-a-proposed-upload] Check whether a 64 MiB upload would fit without writing an object: ```bash curl --fail-with-body \ --aws-sigv4 "aws:amz:us-east-1:s3" \ --user "${RUSTFS_ACCESS_KEY}:${RUSTFS_SECRET_KEY}" \ --request POST \ --header "Content-Type: application/json" \ --data '{"operation_type":"PUT","operation_size":67108864}' \ "${RUSTFS_ENDPOINT}/rustfs/admin/v3/quota-check/${BUCKET_NAME}" ``` The `allowed` field reports the decision. This check is advisory: another write can consume capacity before the planned upload starts, so the actual upload remains authoritative. ## Verification [#verification] Set a small test quota, upload an object that fits, and then attempt to upload an object that exceeds the remaining capacity: ```bash rc bucket quota set rustfs/my-bucket 1M dd if=/dev/zero of=/tmp/quota-small.bin bs=1024 count=256 dd if=/dev/zero of=/tmp/quota-large.bin bs=1048576 count=2 rc object copy /tmp/quota-small.bin rustfs/my-bucket/hello.bin rc object copy /tmp/quota-large.bin rustfs/my-bucket/too-large.bin ``` The 256 KiB object succeeds. The 2 MiB object exceeds the 1 MiB bucket quota, so `rc` exits with an error and RustFS does not create `too-large.bin`. Delete the first object and clear the test quota: ```bash rc object remove rustfs/my-bucket/hello.bin --force rc bucket quota clear rustfs/my-bucket rc bucket quota info rustfs/my-bucket ``` Confirm that the final query reports `Quota: unlimited`. If quota enforcement cannot read or parse its internal configuration, RustFS logs `Bucket quota check degraded to allow` and permits the write. Monitor for this warning because it means quota enforcement is temporarily unavailable. ## Next steps [#next-steps] * [Create a bucket](./creation.md) * [Manage lifecycle rules](../lifecycle-management.md) * [Configure observability](/operations/observability) # Bucket Replication (/en/administration/data/bucket/replication) RustFS bucket replication copies selected object versions from a source bucket to a target bucket. Use it to maintain a remote copy of bucket data, distribute objects between deployments, or prepare a secondary copy for recovery workflows. ## Overview [#overview] Bucket replication has two configuration layers: 1. A **remote target** stores the target endpoint, bucket, credentials, and generated target ARN. 2. An S3 **replication configuration** attaches rules to the source bucket and references that target ARN. Both the source and target buckets must have versioning enabled. RustFS verifies the target connection and target-bucket versioning when you register the remote target. It rejects a replication configuration whose enabled rules reference an unknown or stale target ARN. Replication is asynchronous by default. A successful source upload means RustFS accepted the source object; it does not mean the target copy is already complete. Rules can select objects by prefix or object tags and control these behaviors: | Rule setting | Behavior | | --------------------------- | ----------------------------------------------------------------------- | | `Status` | Enables or disables the rule. | | `Filter` | Restricts replication by key prefix, object tags, or both. | | `ExistingObjectReplication` | Includes objects that existed before the rule when set to `Enabled`. | | `DeleteMarkerReplication` | Replicates delete markers when set to `Enabled`. | | `DeleteReplication` | Replicates deletion of a specific object version when set to `Enabled`. | | `Destination` | Identifies the registered remote target by ARN. | Bucket replication is directional. Configure a separate target and rule in the opposite direction if both buckets must accept writes and replicate them to each other. Do not confuse bucket replication with [site replication](/operations/high-availability/site-replication), which synchronizes broader site configuration and identity data. ## Configuration [#configuration] ### Requirements [#requirements] * A source RustFS deployment and a reachable target S3-compatible deployment. * A source bucket and target bucket with versioning enabled. * A dedicated target credential that can inspect the target bucket's versioning and write replicated object versions and delete markers. * A source administrator allowed to manage remote targets and replication configuration. * Network access from every source node that can run replication work to the target endpoint. * The RustFS [`rc`](/operations/rc) client installed on an administration host. Configure one `rc` alias for each deployment. Use dedicated credentials and replace the example endpoints before running the commands: ```bash rc alias set source https://source.example.com:9000 \ \ --region us-east-1 --bucket-lookup path rc alias set target https://target.example.com:9000 \ \ --region us-east-1 --bucket-lookup path rc alias list ``` The source deployment uses the target alias credentials when it registers the remote target. `rc alias list` displays endpoints but does not print secret keys. RustFS stores the target credential as part of the source bucket's remote-target configuration. Use a dedicated credential with access limited to the target bucket and do not reuse a root credential. ### Permissions [#permissions] Source-side administration uses these policy actions: | Operation | Required action | | ------------------------------------------- | -------------------------------- | | Register, update, or remove a remote target | `admin:SetBucketTarget` | | List remote targets | `admin:GetBucketTarget` | | Read replication metrics | `admin:GetReplicationMetrics` | | Apply or delete replication configuration | `s3:PutReplicationConfiguration` | | Read replication configuration | `s3:GetReplicationConfiguration` | The target credential must pass RustFS target validation, which checks bucket access, versioning, replicated-object writes, replicated delete markers, and object-version deletion. When Object Lock is enabled on the source bucket, the target must have compatible Object Lock support. ### Create and version the buckets [#create-and-version-the-buckets] Create the source bucket: ```bash rc bucket create source/my-bucket ``` Create the target bucket: ```bash rc bucket create target/my-bucket-replica ``` Enable versioning on both buckets: ```bash rc bucket version enable source/my-bucket rc bucket version enable target/my-bucket-replica rc bucket version info source/my-bucket rc bucket version info target/my-bucket-replica ``` Do not suspend source-bucket versioning while replication is configured. ## Usage [#usage] ### Configure replication in the Console [#configure-replication-in-the-console] The RustFS Console combines remote-target registration and replication-rule configuration in one form. 1. Sign in to the source deployment's Console. 2. Open **Buckets**, locate the source bucket, and select **Settings**. 3. Under **Data Protection**, enable **Versioning** if it is disabled. 4. Under **Automation**, select **Open Bucket Replication**. 5. Select **Add Replication Rule**. 6. Configure the destination and rule: | Console field | Value | | --------------------------------- | -------------------------------------------------------------------------------------- | | **Priority** | Rule evaluation priority. The initial value is `1`. | | **Mode** | Select **Asynchronous** or **Synchronous**. Asynchronous is selected by default. | | **Endpoint** | Target S3 API address as `host:port`, without a URL scheme. | | **Bucket** | Versioned destination bucket name. | | **Access Key** and **Secret Key** | Dedicated credentials authorized to replicate into the destination bucket. | | **Region** | Target region. The initial value is `us-east-1`. | | **Storage Class** | Storage class applied at the destination. The initial value is `STANDARD`. | | **Prefix** | Optional key prefix used to limit matching objects. | | **Tags** | Optional object-tag name and value filters. Select **Add Tag** for additional filters. | | **Use TLS** | Enables HTTPS for the destination connection. | | **Replicate Existing Objects** | Includes objects created before the rule. Enabled by default. | | **Replicate Delete Markers** | Copies delete markers to the destination. Enabled by default. | | **Replicate Delete** | Copies deletion of a specific object version. Enabled by default. | | **Health Check Interval** | Target health-check interval in seconds. The initial value is `60`. | | **Bandwidth Limit** | Per-target transfer limit, selectable in KiB/s, MiB/s, or GiB/s. | 7. Select **Save**. RustFS validates the source bucket, target connection, target credentials, and target-bucket versioning before accepting the rule. 8. Return to **Bucket Replication** to review the rule or select **Refresh** to update its displayed state. The source and destination buckets must both have versioning enabled before you save the rule. Sign in to the target deployment's Console and enable versioning under **Buckets** → **Settings** → **Data Protection** when needed. For repeatable automation or configuration management, use the `rc` workflow below. ### Add a replication rule with rc [#add-a-replication-rule-with-rc] Create an asynchronous rule that copies new and existing objects and propagates delete markers and explicit version deletions: ```bash rc bucket replication add source/my-bucket \ --remote-bucket target/my-bucket-replica \ --id replicate-all \ --priority 1 \ --replicate delete,delete-marker,existing-objects ``` `rc` registers the target, obtains its generated ARN, and applies the replication rule in one operation. Omit `--replicate` flags for behaviors you do not want. Add `--sync` only when writes must wait for synchronous replication. To limit replication to a prefix, add `--prefix`: ```bash rc bucket replication add source/my-bucket \ --remote-bucket target/my-bucket-replica \ --id replicate-documents \ --priority 2 \ --prefix documents/ \ --replicate delete-marker,existing-objects ``` Objects that do not match an enabled rule remain only in the source bucket. Use `--bandwidth` to set a byte-per-second limit, `--healthcheck-seconds` to change the target health-check interval, and `--storage-class` to override the destination storage class. ### List and update rules [#list-and-update-rules] List the active rules: ```bash rc bucket replication list source/my-bucket rc bucket replication list source/my-bucket --json ``` Update a rule by its ID. Only supplied settings are changed: ```bash rc bucket replication update source/my-bucket \ --id replicate-all \ --priority 2 \ --bandwidth 104857600 \ --healthcheck-seconds 60 ``` Use `--status Enabled|Disabled` to enable or disable a rule and `--sync true|false` to change its replication mode. ### Export and import configuration [#export-and-import-configuration] Export the complete replication configuration for review or backup: ```bash rc bucket replication export source/my-bucket --json > replication.json ``` The export includes remote-target metadata and the target Access Key, although the Secret Key is omitted. Protect the file as sensitive configuration. Import a previously exported configuration: ```bash rc bucket replication import source/my-bucket replication.json ``` ### Delete replication configuration [#delete-replication-configuration] ```bash rc bucket replication remove source/my-bucket --id replicate-all # Remove every replication rule from the bucket. rc bucket replication remove source/my-bucket --all ``` RustFS also removes replication remote targets referenced by the deleted configuration. It does not delete objects or versions already copied to the target bucket. Remove or retain those objects according to the target bucket's lifecycle and retention requirements. After RustFS removes the replication configuration and its targets, `rc 0.1.29` can report `Remote target not found` while attempting a second target cleanup. Run `rc bucket replication list source/my-bucket --json`; an empty `rules` array confirms that the configuration was removed. ## Verification [#verification] ### Check target readiness [#check-target-readiness] `rc bucket replication add` checks source access, target connectivity, target credentials, and bucket versioning before it creates the rule. Confirm the resulting destination and rule settings: ```bash rc bucket replication list source/my-bucket --json ``` ### Replicate an object [#replicate-an-object] Upload a test object to the source: ```bash printf 'hello from RustFS replication\n' > /tmp/hello.txt rc object copy /tmp/hello.txt source/my-bucket/hello.txt ``` Inspect the source object: ```bash rc object stat source/my-bucket/hello.txt --json ``` Because replication is asynchronous by default, the target object might not appear immediately. Repeat this command until it succeeds: ```bash rc object stat target/my-bucket-replica/hello.txt --json rc object show target/my-bucket-replica/hello.txt ``` Compare the source and target `etag` and `size_bytes` values, then confirm that `object show` returns the expected content. To verify delete-marker replication, delete the source object and list versions on both buckets: ```bash rc object remove source/my-bucket/hello.txt --force rc bucket version list source/my-bucket/hello.txt --json rc bucket version list target/my-bucket-replica/hello.txt --json ``` When delete-marker replication succeeds, the latest entry on both buckets has `is_delete_marker: true`. ### Inspect replication metrics [#inspect-replication-metrics] ```bash rc bucket replication status source/my-bucket rc bucket replication status source/my-bucket --json ``` The command returns the source node's current in-memory replication statistics. Metrics can remain zero even after an object reaches the destination, so use them together with target-object and version checks rather than as the only verification signal. ### Troubleshoot failures [#troubleshoot-failures] | Symptom | Check | | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `bucket versioning must be enabled` | Enable versioning on the source bucket. | | Target registration reports `not versioned` | Enable versioning on the target bucket. | | `replication target configuration not found` | Recheck the target alias, credentials, and destination bucket, then rerun `replication add`. | | Rule reports a stale target | Refresh the rule list and retry `replication add`; if it persists, remove the failed configuration and recreate the target. | | Target object does not appear | Check target reachability, credentials, target quota, and `rc bucket replication status`. | | Replication fails | Inspect source-node logs for the target ARN and object key. | | Target removal is disallowed | Delete or replace the replication configuration before removing a referenced target. | ## Next steps [#next-steps] * [Manage bucket lifecycle](../lifecycle-management.md) * [Configure bucket quotas](./quota.md) * [Configure observability](/operations/observability) # Lifecycle Management (/en/administration/data/lifecycle-management) RustFS lifecycle management applies expiration and transition rules to objects in a bucket. This page shows how to manage those rules with `rc`, verify a transition, and restore a temporary local copy of transitioned data. Before you begin, [install `rc`](/operations/rc), configure an alias named `local`, and create `my-bucket`. To transition objects, first register the target in [Tiered Storage](/administration/data/tiered-storage) and note its uppercase tier name. Lifecycle rules do not process every eligible object immediately. The [Object Scanner](/administration/data/object/scanner) evaluates lifecycle work in the background. ## Add an expiration rule [#add-an-expiration-rule] Expire objects under the `logs/` prefix 30 days after creation: ```bash rc bucket lifecycle rule add local/my-bucket \ --prefix logs/ \ --expiry-days 30 ``` The command creates an enabled rule and returns its generated rule ID. Record that ID when you need to edit or remove this specific rule. ## Add a transition rule [#add-a-transition-rule] Move object data to the registered `COLDTIER` tier after 90 days: ```bash rc bucket lifecycle rule add local/my-bucket \ --transition-days 90 \ --storage-class COLDTIER ``` `--storage-class` must match a registered tier name. Adding the rule does not create the tier or validate an AWS storage-class label as a RustFS tier. You can combine expiration and transition options in one rule. Use the noncurrent-version options only for a versioned bucket: ```bash rc bucket lifecycle rule add local/my-bucket \ --noncurrent-transition-days 30 \ --noncurrent-transition-storage-class COLDTIER \ --noncurrent-expiry-days 365 ``` ## Inspect and update rules [#inspect-and-update-rules] List the current rules and note the ID of the rule you want to change: ```bash rc bucket lifecycle rule list local/my-bucket ``` Change a rule's expiration period or disable it without deleting it: ```bash rc bucket lifecycle rule edit local/my-bucket \ --id \ --expiry-days 60 rc bucket lifecycle rule edit local/my-bucket \ --id \ --disable true ``` Remove one rule by ID. Use `--all` only when you intend to remove the bucket's complete lifecycle configuration: ```bash rc bucket lifecycle rule remove local/my-bucket --id rc bucket lifecycle rule remove local/my-bucket --all ``` ## Export and import rules [#export-and-import-rules] Export the bucket's lifecycle configuration before a bulk change: ```bash rc bucket lifecycle rule export local/my-bucket > lifecycle.json ``` Importing a file replaces the lifecycle configuration sent to the bucket. Review the JSON before applying it: ```bash rc bucket lifecycle rule import local/my-bucket lifecycle.json ``` ## Confirm a lifecycle transition [#confirm-a-lifecycle-transition] After an object becomes eligible under its lifecycle rule, inspect it on the source cluster: ```bash aws s3api head-object \ --bucket my-bucket \ --key hello.txt \ --endpoint-url http://localhost:9000 ``` After the transition completes, the response reports the registered tier name as the storage class: ```json { "StorageClass": "COLDTIER" } ``` The response can contain additional object metadata. Before the transition completes, `StorageClass` may be absent or may not yet report `COLDTIER`. Read the object through the source RustFS endpoint as usual: ```bash aws s3 cp \ s3://my-bucket/hello.txt \ /path/to/hello.txt \ --endpoint-url http://localhost:9000 ``` RustFS reads transitioned data through the source bucket and object key. Applications do not need to address the target bucket directly. ## Restore a local copy [#restore-a-local-copy] Request a temporary local copy of a transitioned object and retain it for seven days: ```bash rc bucket lifecycle restore local/my-bucket/hello.txt --days 7 ``` While the copy-back is running, `HEAD` reports `x-amz-restore: ongoing-request="true"`. After completion, it reports `ongoing-request="false"` with an expiry date. A second restore submitted while one is running returns `RestoreAlreadyInProgress`. When the restore period expires, RustFS removes the local restored copy and its restore metadata. The transitioned object remains available from the remote tier. ## Command compatibility [#command-compatibility] The latest `rc` also accepts `rc ilm` as a compatibility alias. We recommend the noun-first `rc bucket lifecycle` form for new commands and scripts: ```bash rc bucket lifecycle --help rc bucket lifecycle rule --help rc bucket lifecycle tier --help ``` ## Next steps [#next-steps] Review [Tiered Storage](/administration/data/tiered-storage) to monitor or maintain remote tiers, and use [object creation](/administration/data/object/creation) to create test objects for a lifecycle rule. # Object Creation (/en/administration/data/object/creation) Objects are the fundamental storage units in RustFS, containing data, metadata, and a unique key. This guide covers object creation (upload). ## Requirements [#requirements] * A running RustFS instance (see [Installation Guide](../../../installation/index.md)). * [`rc`](/operations/rc) installed and configured with an alias for the command-line workflow. * A target bucket. Create one by following [Bucket Creation](../bucket/creation.md). ## Creating Objects [#creating-objects] ### Using the RustFS UI [#using-the-rustfs-ui] 1. Log in to the RustFS Console. 2. Select the target bucket. 3. On the bucket page, in the top right corner, select **New Directory**, **New File**, or **Upload File/Folder**. 4. To upload from your local machine, click **Upload File/Folder**, select the files, and click **Start Upload**. object creation from ui Click on an object to view its details. object details info ### Using `rc` [#using-rc] See the [`rc` guide](/operations/rc) for installation and alias configuration. Upload a file: ```bash rc object copy /path/to/hello.txt rustfs/my-bucket/hello.txt rc object list rustfs/my-bucket ``` Verify the upload in the RustFS Console. ### Using the API [#using-the-api] Upload a file via API: ```http PUT /{bucketName}/{objectName} HTTP/1.1 ``` S3 requests must be signed with AWS Signature V4, so use an S3 client rather than hand-crafting headers. With the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) configured for your access keys: ```bash aws s3api put-object \ --bucket bucket-creation-by-api \ --key hello.txt \ --body /path/to/hello.txt \ --endpoint-url http://localhost:9000 ``` Verify the upload in the RustFS Console. ## Deleting Objects [#deleting-objects] See [Object Deletion](./deletion.md). Use the following API for file deletion: ```http DELETE /{bucketName}/{objectName} HTTP/1.1 ``` Request example: ```bash aws s3api delete-object \ --bucket bucket-creation-by-api \ --key hello.txt \ --endpoint-url http://localhost:9000 ``` You can confirm the file has been deleted on the RustFS UI. # Object Deletion (/en/administration/data/object/deletion) This guide covers object deletion. ## Requirements [#requirements] * Install and configure [`rc`](/operations/rc) before using the command-line workflow. * Confirm the alias, bucket, and object key before deleting an object. ## Using the RustFS UI [#using-the-rustfs-ui] 1. Log in to the RustFS Console. 2. Select the bucket containing the file to be deleted. 3. On the bucket page, select the file to be deleted. 4. Click **Delete Selected Items** in the upper right corner, then click **Confirm** in the popup dialog. object deletion from ui ## Using `rc` [#using-rc] Delete a file: ```bash rc object remove rustfs/my-bucket/hello.txt rc object list rustfs/my-bucket ``` ```text Removed: rustfs/my-bucket/hello.txt ✓ Removed 1 object(s). ``` Verify the deletion in the RustFS Console. ## Using the API [#using-the-api] Delete a file via API: ```http DELETE /{bucketName}/{objectName} HTTP/1.1 ``` S3 requests must be signed with AWS Signature V4, so use an S3 client rather than hand-crafting headers. With the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) configured for your access keys: ```bash aws s3api delete-object \ --bucket bucket-creation-by-api \ --key hello.txt \ --endpoint-url http://localhost:9000 ``` Verify the deletion in the RustFS Console. # Multipart Upload (/en/administration/data/object/multipart-upload) Multipart upload splits one object into independently uploaded parts and assembles them on the server. Use it for large objects, retryable transfers, and browser uploads that need progress and cancellation controls. ## Overview [#overview] The S3 multipart workflow has three required phases: 1. **CreateMultipartUpload** returns an upload ID. 2. **UploadPart** uploads numbered parts and returns an ETag for each part. 3. **CompleteMultipartUpload** submits the ordered part numbers and ETags to create the final object. Abort an unfinished upload with **AbortMultipartUpload** so temporary parts do not continue consuming storage. RustFS accepts part numbers from `1` through `10000`. `ListParts` and `ListMultipartUploads` return at most 1,000 entries per response and use markers for pagination. The completed object appears only after the completion request succeeds. ## Upload in the Console [#upload-in-the-console] 1. Sign in to the RustFS Console and open the destination bucket. 2. Select **Upload File/Folder**. 3. Optionally enter a **Current Prefix**. 4. Select **Select File** or **Select Folder**, then choose the content to upload. 5. Review the selected names and sizes, then select **Start Upload**. Console upload dialog with a selected large file The observed Console upload dialog supports up to 10,000 selected files and reports a 512 GB maximum for a single file. These are Console upload limits; S3 clients can have different local limits. After upload starts, **Task Management** groups tasks into Pending, Processing, Completed, Failed, and Canceled states. Each processing task shows progress and a **Cancel** action. Canceling an active multipart task aborts the current part request and marks the task as canceled. Refresh the bucket after completion and confirm the object size and modification time. ## Use rc [#use-rc] Upload and verify an object with `rc`: ```bash rc object copy /path/to/hello.txt rustfs/my-bucket/hello.txt rc object stat rustfs/my-bucket/hello.txt --json ``` `rc 0.1.29` does not expose create, upload-part, complete, list-parts, or abort multipart commands. A validated 20 MiB `rc object copy` used a single `PutObject` request rather than multipart upload. Use the Console or an S3 SDK when explicit multipart behavior is required; use `rc object stat` to verify the completed object. For a versioned bucket, list the version created by a completed multipart upload: ```bash rc bucket version list rustfs/my-bucket/hello.txt --json ``` RustFS returns a version ID when multipart completion succeeds in a versioned bucket. ## S3 multipart operations [#s3-multipart-operations] Use an S3 SDK that supports these standard operations: | Phase | S3 operation | Required values | | -------- | ------------------------- | ---------------------------------------------------------------------- | | Initiate | `CreateMultipartUpload` | Bucket, key, metadata, encryption, and optional Object Lock settings. | | Upload | `UploadPart` | Bucket, key, upload ID, part number, and body. Save the returned ETag. | | Inspect | `ListParts` | Bucket, key, and upload ID. Paginate when needed. | | Complete | `CompleteMultipartUpload` | Ordered part numbers and their exact ETags. | | Cancel | `AbortMultipartUpload` | Bucket, key, and upload ID. | | Discover | `ListMultipartUploads` | Bucket and optional prefix. Paginate when needed. | Do not reuse an upload ID for a different key. Submit completed parts in ascending part-number order and preserve each ETag exactly as returned. ## Verification [#verification] After completion: ```bash rc object stat rustfs/my-bucket/hello.txt --json rc object show rustfs/my-bucket/hello.txt > /tmp/hello-downloaded.txt cmp /path/to/hello.txt /tmp/hello-downloaded.txt ``` Confirm that `size_bytes` matches the local file and that `cmp` exits successfully. For a versioned bucket, also confirm that `rc bucket version list` returns a version ID. ## Troubleshooting [#troubleshooting] | Symptom | Check | | -------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Upload remains in Processing | Check browser connectivity and keep the Console tab open; cancel and retry when the connection is interrupted. | | A part request fails | Retry that part with the same upload ID and part number, then use the latest returned ETag. | | Completion reports an invalid part | Verify the submitted part numbers, ordering, and ETags against `ListParts`. | | Object is absent after uploading parts | Send `CompleteMultipartUpload`; uploaded parts alone do not create the object. | | Temporary storage continues growing | List incomplete uploads and abort those that are no longer needed. | | Completion fails on a locked key | Check Object Lock retention or Legal Hold on the current destination version. | ## Next steps [#next-steps] * [Create and inspect objects](./creation.md) * [Manage object versions](./versioning.md) * [Protect objects with Object Lock](./object-lock.md) # Object Lock (/en/administration/data/object/object-lock) RustFS Object Lock applies write-once, read-many protection to individual object versions. Use retention periods for time-bound protection and Legal Hold for protection without a predefined expiration date. ## Overview [#overview] Object Lock requires bucket versioning. You can enable it when creating a bucket or use the S3 `PutObjectLockConfiguration` API on an existing bucket whose versioning is enabled. Each overwrite creates a new version; retention and Legal Hold protect a specific version rather than the object key as a whole. | Protection | Behavior | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `GOVERNANCE` retention | Blocks deletion and retention shortening unless the caller has bypass permission and explicitly requests a bypass. Retention can be extended without bypass. | | `COMPLIANCE` retention | Blocks deletion even with governance bypass. The date can be extended but not shortened. | | Legal Hold | Blocks deletion until the hold is set to `OFF`. It has no expiration date and governance bypass does not override it. | Deleting a key without a version ID creates a delete marker. The protected version remains stored and can still be retrieved by version ID. COMPLIANCE retention cannot be bypassed or shortened before its retain-until date. Test policies in a non-production bucket and verify time synchronization, permissions, lifecycle rules, replication, and backup procedures before protecting production data. ## Configure in the Console [#configure-in-the-console] 1. Sign in to the RustFS Console and open **Buckets**. 2. Select **Create Bucket** and enter the bucket name. 3. Enable **Object Lock**. The Console also enables **Version** because Object Lock requires versioning. 4. To apply retention automatically to new object versions, enable **Retention**. 5. Select **COMPLIANCE** or **GOVERNANCE**, enter the validity, and select **Day** or **Year**. 6. Select **Create**. Create Bucket dialog with versioning, Object Lock, and default retention enabled The Console initially displays `180` days when default retention is enabled. Replace it with the period required by your retention policy; it is a UI initial value, not a general recommendation. To inspect a protected object, open the bucket, select the object name, and use the **Versions** and **Info** tabs. The Info tab exposes **Legal Hold** and **RetentionPolicy** fields. ## Use rc [#use-rc] Configure an alias before running these examples: ```bash rc alias set rustfs http://localhost:9000 \ \ --region us-east-1 --bucket-lookup path ``` `rc bucket create --with-lock` and `--with-versioning` appear in the `rc 0.1.29` help output but return `not implemented` when executed. Create the Object Lock bucket in the Console. The upload headers shown below are supported and were validated against RustFS. Upload an object version with GOVERNANCE retention. Supply both retention headers together and use an RFC 3339 UTC timestamp in the future: ```bash rc object copy /path/to/hello.txt rustfs/my-bucket/hello.txt \ -H "x-amz-object-lock-mode:GOVERNANCE" \ -H "x-amz-object-lock-retain-until-date:2027-01-01T00:00:00Z" ``` Upload an object version with Legal Hold enabled: ```bash rc object copy /path/to/hello.txt rustfs/my-bucket/legal-hold.txt \ -H "x-amz-object-lock-legal-hold:ON" ``` List the protected versions and inspect a specific version: ```bash rc bucket version list rustfs/my-bucket --json rc object stat rustfs/my-bucket/hello.txt \ --version-id --json ``` `rc 0.1.29` does not provide commands to change retention or switch an existing Legal Hold between `ON` and `OFF`. Use the Console or an S3 SDK for those operations. ## Verify protection [#verify-protection] 1. Upload a test object with GOVERNANCE retention or Legal Hold. 2. Record its version ID with `rc bucket version list`. 3. Attempt to delete that exact version through an S3 client without bypass. RustFS must return `AccessDenied` while protection is active. 4. Confirm the version remains visible in the Console and through `rc object stat --version-id`. For GOVERNANCE retention, test bypass only with a dedicated administrative identity that has `s3:BypassGovernanceRetention`. COMPLIANCE retention and Legal Hold remain protected from governance bypass. ## Operational considerations [#operational-considerations] * Default bucket retention is calculated when a new object version or multipart upload is initiated. * Copying an object creates a new destination version; destination retention policy applies independently. * Suspending versioning is not appropriate for an Object Lock bucket. * Lifecycle expiration cannot remove a version while retention or Legal Hold blocks deletion. * Replication targets for locked objects must support compatible Object Lock behavior. ## Next steps [#next-steps] * [Manage object versions](./versioning.md) * [Configure bucket replication](../bucket/replication.md) * [Review the security checklist](/installation/requirement/checklists/security-checklists) # Object Scanning (/en/administration/data/object/scanner) This guide covers the design and implementation of the RustFS object scanner, including its integration with Erasure Coding, Scrub & Repair mechanisms, scheduling strategies, monitoring metrics, and troubleshooting. ## Overview [#overview] The RustFS object scanner is built into the storage engine and is responsible for periodically checking object integrity and executing scheduled operations. Scanning tasks include disk usage statistics, lifecycle management rule evaluation, object replication execution, and triggering corrupted object self-healing. ## Architecture and Design Principles [#architecture-and-design-principles] ### Scanner Architecture [#scanner-architecture] The RustFS scanner uses a hash sampling mechanism, selecting one out of every 1024 objects for inspection based on object name hashing to minimize performance impact. The scanner is deeply integrated with the Erasure Coding module, utilizing redundant shards for online reconstruction when detecting lost or corrupted shards, ensuring high data availability and consistency. ## Data Verification and Recovery [#data-verification-and-recovery] The RustFS data verification mechanism checks metadata consistency and performs bit-by-bit reading and verification to discover hidden bad blocks. The object scanner detects issues like bit rot and triggers repair processes when necessary. ## Scanning Modes and Scheduling [#scanning-modes-and-scheduling] RustFS supports three scanning modes: online scanning during reads, background periodic scanning, and manual full scanning, balancing performance and reliability. Similar to the `osd_scrub_begin_hour` configuration in Ceph, administrators can set scanning start times and frequency. For example, light verification is set to once daily by default. ## Monitoring and Metrics [#monitoring-and-metrics] Scanner statistics include total task count, failure count, and time distribution, exposing metrics through the Prometheus data model such as `rustfs_scanner_jobs_total`, `rustfs_scanner_failures_total`, and `rustfs_scanner_duration_seconds`. Combined with monitoring systems, alerts can be set based on scanning failure rates and duration to promptly discover and locate potential issues at the storage or network levels. # Versioning (/en/administration/data/object/versioning) RustFS bucket versioning preserves multiple versions of the same object key. Use it to recover from accidental overwrites and deletes and to satisfy prerequisites for Object Lock and bucket replication. ## Overview [#overview] When versioning is enabled: * Each upload or copy to an existing key creates a new version ID. * A normal delete creates a delete marker instead of removing older versions. * Reading the key without a version ID returns the latest visible version, or `NotFound` when the latest entry is a delete marker. * Reading or deleting a specific version requires its version ID. Suspending versioning stops normal creation of new version IDs but preserves existing versions and delete markers. Suspension is not the same as disabling or removing version history. ## Configure in the Console [#configure-in-the-console] ### Enable at bucket creation [#enable-at-bucket-creation] 1. Open **Buckets** and select **Create Bucket**. 2. Enter the bucket name. 3. Enable **Version**. 4. Select **Create**. ### Enable or suspend an existing bucket [#enable-or-suspend-an-existing-bucket] 1. Open **Buckets**, locate the bucket, and select **Settings**. 2. Under **Data Protection**, locate **Versioning**. 3. Enable versioning or suspend it as required. The settings page reports `Enabled`, `Suspended`, or `Disabled`. It also notes that suspension preserves existing versions. ### Browse and recover versions [#browse-and-recover-versions] 1. Open the bucket and enable **Show Deleted Objects** when you need to see keys hidden by delete markers. 2. Select an object name to open **Object Details**. 3. Select **Versions** to inspect available versions and delete markers. 4. Download the required version and upload it again to make its content the latest version. ## Use rc [#use-rc] Create a bucket and enable versioning: ```bash rc bucket create rustfs/my-bucket rc bucket version enable rustfs/my-bucket rc bucket version info rustfs/my-bucket ``` Upload two versions of the same key: ```bash printf 'version one\n' > /tmp/hello.txt rc object copy /tmp/hello.txt rustfs/my-bucket/hello.txt printf 'version two\n' > /tmp/hello.txt rc object copy /tmp/hello.txt rustfs/my-bucket/hello.txt --overwrite ``` List versions and record the version ID you want to recover: ```bash rc bucket version list rustfs/my-bucket/hello.txt --json rc object stat rustfs/my-bucket/hello.txt \ --version-id --json ``` Restore an earlier version by downloading its body and uploading it as a new latest version: ```bash rc object show rustfs/my-bucket/hello.txt \ --version-id > /tmp/hello-restored.txt rc object copy /tmp/hello-restored.txt \ rustfs/my-bucket/hello.txt --overwrite ``` Delete the current key to create a delete marker, then inspect it: ```bash rc object remove rustfs/my-bucket/hello.txt --force rc bucket version list rustfs/my-bucket/hello.txt --json ``` The latest entry should show `is_delete_marker: true`. The earlier versions remain available by version ID. Suspend versioning when you no longer want normal writes to create new numbered versions: ```bash rc bucket version suspend rustfs/my-bucket rc bucket version info rustfs/my-bucket ``` `rc object remove --versions` is listed in the `rc 0.1.29` help output but returns `not implemented`. Use the Console or an S3 SDK to permanently delete selected version IDs. Do not remove an entire version history unless you have verified retention, replication, and recovery requirements. ## Verification [#verification] After enabling versioning: 1. Upload the same key twice with different content. 2. Run `rc bucket version list` and confirm that both entries have distinct version IDs. 3. Retrieve each version with `rc object show --version-id` and compare its content. 4. Delete the key without a version ID and confirm that a delete marker appears. 5. Retrieve an older version by version ID to confirm it remains recoverable. ## Operational considerations [#operational-considerations] * Versioning increases storage use because overwrites and deletes retain older data. * Configure lifecycle rules for noncurrent versions only after defining recovery and retention periods. * Bucket replication requires versioning on both source and destination buckets. * Object Lock depends on versioning and protects individual versions. * Multipart completion creates one new object version in a versioned bucket. ## Next steps [#next-steps] * [Protect versions with Object Lock](./object-lock.md) * [Upload large objects](./multipart-upload.md) * [Manage bucket lifecycle](../lifecycle-management.md) # S3 Tables (/en/administration/data/s3-tables) RustFS S3 Tables manages **Apache Iceberg** tables through a built-in REST catalog. Table data, manifests, and Iceberg metadata remain S3 objects in RustFS. This guide enables a dedicated table bucket and explains client connections, permissions, and maintenance boundaries. S3 Tables is a preview feature; client compatibility is limited to the workflows listed below. This page follows RustFS commit [`7e0c6711`](https://github.com/rustfs/rustfs/commit/7e0c67111b97703d47e23719b0264a739c8acea8), reviewed on September 8, 2026. Check the [support matrix](https://github.com/rustfs/rustfs/blob/7e0c67111b97703d47e23719b0264a739c8acea8/docs/architecture/s3-tables-support-matrix.md) and your release before adopting additional catalog operations or clients. ## How it works [#how-it-works] An Iceberg client uses the REST catalog to discover tables and commit metadata changes. It uses the S3 API to read and write table files. Both interfaces are served by RustFS on the S3 API port. | Resource | Purpose | | ------------ | ---------------------------------------------------------------------------------------- | | Table bucket | An existing S3 bucket enabled for catalog use; its name is the client `warehouse`. | | Namespace | A logical group of tables within that warehouse. | | Table | An Iceberg schema, snapshots, and a current metadata location maintained by the catalog. | Enabling a table bucket does not register existing Parquet files as Iceberg tables. Create or register tables through an Iceberg client. If no `location` is supplied, RustFS assigns one; a custom location must be in the same bucket. Clients should use the returned location. The default `object` catalog backing persists catalog state in RustFS object storage. A table commit validates its base metadata and referenced objects before conditionally updating the current metadata pointer. A conflicting writer must reload the table and resolve the conflict. The transaction boundary is one table. ## Before you begin [#before-you-begin] * Start a RustFS deployment with the S3 Tables endpoints described above. See [Installation](/installation). * Install the [AWS CLI](/developer/examples/aws-cli) and `curl` 7.76 or later, which supports `--aws-sigv4` and `--fail-with-body`. * Use a new, dedicated bucket for this walkthrough. The example uses `my-bucket`. * Use an existing administrative account with access to both catalog operations and S3 objects. The built-in `consoleAdmin` policy covers this walkthrough; configure narrower policies for applications. The examples use `http://localhost:9000`. Replace it with your server endpoint, and use [TLS](/integration/tls-configured) with certificate verification enabled outside a local test environment. Table buckets are excluded from ordinary bucket lifecycle expiration. Enabling this mode on an existing bucket changes how its expiration rules are applied. Use catalog maintenance that understands Iceberg references to expire snapshots and clean up table files.
## Create a bucket [#1-create-a-bucket] Set the endpoint and access credentials for the example clients: ```bash export RUSTFS_ENDPOINT="http://localhost:9000" export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_DEFAULT_REGION="us-east-1" ``` Create the dedicated bucket: ```bash aws --endpoint-url "$RUSTFS_ENDPOINT" s3api create-bucket --bucket my-bucket ``` These examples use an access key and secret key, without a temporary session token. Keep the same shell environment for the following requests and the PyIceberg guide.
## Enable the table bucket [#2-enable-the-table-bucket] Send an empty, SigV4-signed request to the table bucket endpoint: ```bash curl --fail-with-body --silent --show-error \ --aws-sigv4 "aws:amz:us-east-1:s3" \ --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \ --header "x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" \ --request PUT "$RUSTFS_ENDPOINT/iceberg/v1/buckets/my-bucket" ``` Read the state back with the same credentials: ```bash curl --fail-with-body --silent --show-error \ --aws-sigv4 "aws:amz:us-east-1:s3" \ --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \ --header "x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" \ "$RUSTFS_ENDPOINT/iceberg/v1/buckets/my-bucket" ``` Both requests return HTTP `200` on success. Confirm that the response includes these values: ```json { "table-bucket": "my-bucket", "enabled": true, "catalog-type": "iceberg-rest", "warehouse": "my-bucket", "catalog-entry-present": true } ``` This is a response excerpt. The returned `catalog-uri` is a bucket-specific route; use the client base URI in the next section when configuring an Iceberg REST client.
## Connect an Iceberg client [#3-connect-an-iceberg-client] Use these settings for the canonical RustFS endpoint: | Setting | Value | | -------------------- | -------------------------------------------------- | | REST catalog URI | `http://localhost:9000/iceberg` | | Warehouse and prefix | `my-bucket` | | REST authentication | AWS Signature Version 4, signing name `s3` | | Region | `us-east-1` | | S3 file endpoint | `http://localhost:9000` with path-style addressing | The client adds `/v1` to the catalog URI. The warehouse is a bucket name, not an S3 URI or an AWS S3 Tables ARN. Configure both REST request signing and S3 file access, even when they use the same account. If you already operate a separate Iceberg REST catalog, use the [Apache Iceberg integration](/developer/integration/big-data/iceberg) for the external-catalog deployment pattern.
## Permissions and credentials [#permissions-and-credentials] Table bucket enablement requires `admin:SetTableBucket`; inspecting it requires `admin:GetTableBucket`. Catalog discovery uses `admin:GetTableCatalog`. Namespace and table operations have their own RustFS admin actions, including `admin:SetTableNamespace`, `admin:CreateTable`, `admin:GetTableMetadata`, and `admin:CommitTable`. Table file reads and writes also require ordinary S3 permissions. RustFS checks table permissions on warehouse object paths: reads require the corresponding `admin:GetTableMetadata` authorization, and writes require `admin:SetTableMetadata`. A catalog commit grant alone does not authorize the S3 file writes that precede it. Configure [IAM policies](/security-compliance/iam/policies) for both interfaces. Catalog credential vending is disabled by default. When enabled, a compatible client must negotiate `X-Iceberg-Access-Delegation: vended-credentials`, and the caller must have permission to request table credentials. Initial catalog setup still requires an authorized principal. The linked PyIceberg walkthrough uses explicitly configured credentials. ## Maintenance and data protection [#maintenance-and-data-protection] Metadata deletion and background maintenance are disabled by default. RustFS exposes explicit planning, scheduler-run, and worker-run operations; it does not run a built-in periodic maintenance scheduler. Review a maintenance plan and its retained references before enabling deletion. Dropping a table removes its catalog entry while retaining its underlying objects. Complete any required table maintenance before unregistering the table; afterward, maintenance operations can no longer find it. Cleanup of retained objects needs a separate plan that accounts for all remaining references. Do not recursively delete S3 paths that snapshots or other metadata may still reference. Keep the default catalog backing for this walkthrough. Switching an existing deployment to `durable-strong` requires the [catalog cutover procedure](https://github.com/rustfs/rustfs/blob/7e0c67111b97703d47e23719b0264a739c8acea8/docs/operations/s3-tables-cutover-runbook.md), including migration preflight and coordinated writer fencing. ## Client compatibility and limits [#client-compatibility-and-limits] The source repository maintains the following validation scope: | Client | Validation scope | | -------------------- | ----------------------------------------------------------------------------------------- | | PyIceberg | Automated create, append, reload, scan, and catalog operation checks. | | DuckDB Iceberg 1.5.5 | Automated generic REST catalog checks for single-table reads, writes, and schema changes. | | Spark | An opt-in live harness; validate the exact Spark and Iceberg versions you deploy. | | Trino | A manual read-only probe; write compatibility is not claimed. | Iceberg format v1 and v2 are supported, with v2 as the default. Staged table creation, purge-on-drop, and Iceberg format v3 are unsupported. RustFS S3 Tables does not provide a SQL execution engine, multi-table atomic transactions, or independent active-active writes across regions. It does not claim full AWS S3 Tables control-plane compatibility. Consult the [support matrix](https://github.com/rustfs/rustfs/blob/7e0c67111b97703d47e23719b0264a739c8acea8/docs/architecture/s3-tables-support-matrix.md) before using another engine or vendor profile. ## Next steps [#next-steps] * Run the [PyIceberg walkthrough](/developer/integration/big-data/pyiceberg). * Review [IAM policies](/security-compliance/iam/policies) before granting application access. * Use the repository's [client conformance checks](https://github.com/rustfs/rustfs/blob/7e0c67111b97703d47e23719b0264a739c8acea8/scripts/table-catalog/README.md) to validate additional client versions. # Tiered Storage (/en/administration/data/tiered-storage) RustFS tiered storage moves objects from local storage to a configured remote backend. This page explains the supported targets and shows how to add and maintain a RustFS tier in the Console. Tiering is asynchronous. RustFS keeps the object metadata locally, transfers the object data to the remote tier, and continues to serve S3 reads through the original bucket and object key. Lifecycle rules refer to a registered tier by its uppercase name, such as `COLDTIER`. Do not substitute AWS class labels such as `INTELLIGENT_TIERING`, `GLACIER`, or `DEEP_ARCHIVE` unless you have registered a RustFS tier with that exact valid name and verified the target behavior. ## Supported backends [#supported-backends] The RustFS source defines warm-backend implementations for these target types: | Type | Configuration key | Typical target | | ----------- | ----------------- | ----------------------------------------------------- | | RustFS | `rustfs` | Another RustFS deployment | | S3 | `s3` | Amazon S3 or an S3 endpoint supported by this backend | | Wasabi | `wasabi` | Wasabi object storage | | MinIO | `minio` | A MinIO deployment | | Aliyun | `aliyun` | Alibaba Cloud Object Storage Service (OSS) | | Tencent | `tencent` | Tencent Cloud Object Storage (COS) | | Huaweicloud | `huaweicloud` | Huawei Cloud Object Storage Service (OBS) | | Azure | `azure` | Azure Blob Storage | | GCS | `gcs` | Google Cloud Storage | | R2 | `r2` | Cloudflare R2 | Provider payloads and credential requirements differ. The complete workflow below uses the RustFS backend because the RustFS source includes an end-to-end hot-cluster-to-cold-cluster test for this path. ## Before you begin [#before-you-begin] Prepare the following: * A source RustFS deployment that stores the hot data. * A separate target RustFS deployment and an existing target bucket. This example uses `my-bucket` on the target. * Target credentials with permission to put, get, list, and delete objects in that bucket. * Access to the source deployment's RustFS Console. Use TLS for both deployments in production. Restrict the target credentials to the dedicated tier bucket and prefix.
## Open Tiered Storage [#1-open-tiered-storage] Sign in to the source deployment's RustFS Console. In the left navigation, select **Tiered Storage**, then select **Add Tier** in the upper-right corner.
## Select the target [#2-select-the-target] Select the target provider. This example uses **RustFS** to connect the source deployment to another RustFS deployment.
## Enter the target details [#3-enter-the-target-details] Complete the form: | Field | Value | | --------------------- | ----------------------------------------------------------- | | **Name (A-Z,0-9,\_)** | Enter a unique uppercase tier name, such as `COLDTIER`. | | **Endpoint** | Enter the target RustFS S3 endpoint. | | **Access Key** | Enter the access key for the target deployment. | | **Secret Key** | Enter the secret key for the target deployment. | | **Bucket** | Enter the existing target bucket name, such as `my-bucket`. | | **Prefix (Optional)** | Optionally enter a prefix dedicated to tiered objects. | | **Region** | Optionally enter the target region, such as `us-east-1`. | Leave **Storage Class** at its default unless the target backend requires a different supported storage class. The form contains a secret key. Use credentials restricted to the target bucket and prefix. Do not expose the key in screenshots, tickets, or logs.
## Save the tier [#4-save-the-tier] Select **Save**. RustFS validates the backend and probes it by writing, reading, and removing a small object. Saving fails when the endpoint, credentials, bucket permissions, or backend configuration cannot complete that probe. After the tier appears in the **Tiers** list, configure a transition rule in [Lifecycle Management](/administration/data/lifecycle-management). A registered tier does not move objects until a lifecycle rule references its name.
## Manage tiers with `rc` [#manage-tiers-with-rc] Install and configure [`rc`](/operations/rc), then list the tiers registered on the source deployment: ```bash rc bucket lifecycle tier list local ``` Add a RustFS tier with the same settings described in the Console workflow: ```bash rc bucket lifecycle tier add rustfs COLDTIER local \ --endpoint \ --access-key \ --secret-key \ --bucket my-bucket \ --region us-east-1 ``` Inspect the tier configuration and available statistics: ```bash rc bucket lifecycle tier info COLDTIER local ``` The remaining tier commands update credentials or remove a tier: ```bash rc bucket lifecycle tier edit COLDTIER local \ --access-key \ --secret-key rc bucket lifecycle tier remove COLDTIER local ``` See [Lifecycle Management](/administration/data/lifecycle-management) to create transition rules, confirm transitions, and restore local copies. Run `rc bucket lifecycle tier --help` to inspect provider-specific options before changing a tier. ## Monitor tier activity [#monitor-tier-activity] Use the tier statistics endpoint with a SigV4-signed request and `admin:ListTier` permission: ```http GET /rustfs/admin/v3/tier-stats?tier=COLDTIER HTTP/1.1 Host: ``` Monitor transition failures together with source-cluster capacity and target-side availability. A configured tier adds the target service and network path to the read path for objects that do not have a restored local copy. ## Change or remove a tier [#change-or-remove-a-tier] The Admin API exposes these mutation routes, all requiring `admin:SetTier`: | Operation | Route | | --------------- | ----------------------------------------- | | Edit a tier | `POST /rustfs/admin/v3/tier/{tiername}` | | Remove a tier | `DELETE /rustfs/admin/v3/tier/{tiername}` | | Clear all tiers | `POST /rustfs/admin/v3/tier/clear` | Before editing or removing a tier: 1. Disable lifecycle rules that reference the tier. 2. Confirm that no transition jobs are still using it. 3. Confirm that no source objects depend on data stored in its target bucket or prefix. 4. Back up the tier configuration and record the target location. Normal tier mutations check backend usage and protect non-empty targets. Do not use a `force` option to bypass those checks unless you have independently proved that every transitioned object remains recoverable. Removing configuration for an active tier can make transitioned objects unreadable from the source cluster. Do not rename, overwrite, or delete generated objects in the target bucket. Manage source objects through the hot RustFS cluster so RustFS can keep local transition metadata and remote data consistent. ## Admin API reference [#admin-api-reference] The current RustFS source registers these tier routes: | Method | Route | Permission | | -------- | ---------------------------------------------------- | ---------------- | | `PUT` | `/rustfs/admin/v3/tier` | `admin:SetTier` | | `POST` | `/rustfs/admin/v3/tier/{tiername}` | `admin:SetTier` | | `DELETE` | `/rustfs/admin/v3/tier/{tiername}` | `admin:SetTier` | | `POST` | `/rustfs/admin/v3/tier/clear` | `admin:SetTier` | | `GET` | `/rustfs/admin/v3/tier` (list configurations) | `admin:ListTier` | | `GET` | `/rustfs/admin/v3/tier/{tier}` (verify connectivity) | `admin:ListTier` | | `GET` | `/rustfs/admin/v3/tier-stats` (read statistics) | `admin:ListTier` | All Admin API requests require SigV4 authentication. These routes are an administrative interface, not ordinary S3 bucket operations. ## Next steps [#next-steps] Review [lifecycle management](/administration/data/lifecycle-management) and configure [access policies](/security-compliance/iam/policies) for the administrators and service credentials used by tiered storage. # Administration (/en/administration) Use this section to manage RustFS through the Console, administer buckets and objects, and configure access protocols. ## Administration areas [#administration-areas] * [Console](./console/index.md) covers browser-based administration and sign-in methods. * [Data Management](./data/object/object-lock.md) covers buckets, objects, lifecycle behavior, and data protection features. * [S3 Tables](/administration/data/s3-tables) covers table buckets and the built-in Iceberg REST catalog. * [Protocol Support](./protocols/s3.md) covers S3, WebDAV, FTPS, and SFTP access. * [CORS Configuration](./cors/index.md) covers cross-origin access to RustFS services. * [Virtual-Host Access](/integration/virtual) covers domain-based S3 addressing. For identity, encryption, transport security, and auditing, continue with [Security & Compliance](/security-compliance). # FTP(S) (/en/administration/protocols/ftps) RustFS includes an FTP gateway that exposes buckets and objects to standard FTP clients. You can run it as unencrypted FTP for isolated local testing or as explicit FTP over TLS (FTPS) for encrypted connections. RustFS authenticates each session against Identity and Access Management (IAM) and applies the user's S3 policies to storage operations. FTP and FTPS support is compiled into the standard RustFS binary, but both listeners are disabled at runtime by default. Enable only the listener you intend to use. ## Overview [#overview] The gateway maps FTP paths to RustFS resources: | FTP path | RustFS resource | | ---------------------- | --------------------------------------------- | | `/` | All buckets visible to the authenticated user | | `/my-bucket/` | The `my-bucket` bucket | | `/my-bucket/hello.txt` | The `hello.txt` object in `my-bucket` | The following FTP operations are supported: | FTP command | Operation | | ----------- | ------------------------------------------------------------ | | `LIST` | List buckets at the root or objects and prefixes in a bucket | | `MKD` | Create a bucket | | `CWD` | Enter a bucket | | `STOR` | Upload an object | | `RETR` | Download an object | | `DELE` | Delete an object | | `RMD` | Recursively delete a bucket and its objects | RustFS does not currently support FTP rename operations or uploads that append to an existing object. Because S3 storage has no native working-directory or POSIX directory model, some client-specific filesystem operations may not behave like a traditional FTP server. Enter a RustFS access key as the FTP username and its secret key as the password. Invalid usernames and passwords both return `530 Not logged in`. After login, RustFS checks the IAM user's S3 permissions for each operation. FTP sends credentials and data without encryption. Bind plain FTP to a loopback or isolated test network only. Use FTPS for every remote or production connection. The current `RMD` implementation deletes the objects in the target bucket before deleting the bucket. Confirm the path before running `rmdir` in an FTP client. ## Configuration [#configuration] FTP and FTPS use separate listeners and environment variables. ### FTP variables [#ftp-variables] | Variable | Description | Default | | -------------------------- | ------------------------------------------------------------------------------ | -------------- | | `RUSTFS_FTP_ENABLE` | Enables the unencrypted FTP listener. | `false` | | `RUSTFS_FTP_ADDRESS` | Bind address for FTP control connections. | `0.0.0.0:8021` | | `RUSTFS_FTP_PASSIVE_PORTS` | Inclusive passive data port range in `start-end` format. | `40000-50000` | | `RUSTFS_FTP_EXTERNAL_IP` | Public IP or hostname advertised to passive clients when RustFS is behind NAT. | Not set | ### FTPS variables [#ftps-variables] | Variable | Description | Default | | --------------------------- | ------------------------------------------------------------------------------ | -------------- | | `RUSTFS_FTPS_ENABLE` | Enables the explicit FTPS listener. | `false` | | `RUSTFS_FTPS_ADDRESS` | Bind address for FTPS control connections. | `0.0.0.0:8022` | | `RUSTFS_FTPS_TLS_ENABLED` | Enables TLS on the FTPS listener. Keep this enabled for FTPS. | `true` | | `RUSTFS_FTPS_CERTS_DIR` | Directory containing the FTPS certificate and private key. Required for FTPS. | Not set | | `RUSTFS_FTPS_PASSIVE_PORTS` | Inclusive passive data port range in `start-end` format. | `40000-50000` | | `RUSTFS_FTPS_EXTERNAL_IP` | Public IP or hostname advertised to passive clients when RustFS is behind NAT. | Not set | The standard RustFS build enables the `ftps` compile-time feature, which provides both FTP and FTPS. If you build RustFS with `--no-default-features`, include the feature explicitly: ```bash cargo build --release --features ftps ``` ### Local testing with FTP [#local-testing-with-ftp] Start an unencrypted FTP listener on the loopback interface: ```bash export RUSTFS_FTP_ENABLE=true export RUSTFS_FTP_ADDRESS=127.0.0.1:8021 export RUSTFS_FTP_PASSIVE_PORTS=40000-40010 export RUSTFS_ACCESS_KEY= export RUSTFS_SECRET_KEY= rustfs /path/to/data ``` Binding to `127.0.0.1` prevents remote hosts from connecting to the unencrypted listener. ### Prepare a test certificate [#prepare-a-test-certificate] FTPS expects `rustfs_cert.pem` and `rustfs_key.pem` in the certificate directory. For local testing, create a short-lived self-signed certificate: ```bash mkdir -p /path/to/ftps-certs openssl req -x509 -newkey rsa:2048 -nodes \ -keyout /path/to/ftps-certs/rustfs_key.pem \ -out /path/to/ftps-certs/rustfs_cert.pem \ -days 7 \ -subj "/CN=localhost" \ -addext "subjectAltName=DNS:localhost,IP:127.0.0.1" ``` Use a certificate issued by a trusted certificate authority in production. Protect `rustfs_key.pem` from unauthorized access and ensure the certificate subject alternative names match the host clients use. ### Start FTPS [#start-ftps] Configure an explicit FTPS listener: ```bash export RUSTFS_FTPS_ENABLE=true export RUSTFS_FTPS_ADDRESS=0.0.0.0:8022 export RUSTFS_FTPS_TLS_ENABLED=true export RUSTFS_FTPS_CERTS_DIR=/path/to/ftps-certs export RUSTFS_FTPS_PASSIVE_PORTS=40000-50000 rustfs /path/to/data ``` RustFS requires TLS on both the FTPS control and data channels. Configure clients for **explicit FTP over TLS**, sometimes labeled **FTPES** or **Require explicit FTP over TLS**. Implicit FTPS is not the mode implemented by this listener. If the certificate directory is missing, does not exist, or does not contain a usable certificate and key, FTPS initialization fails. See [Configure TLS](/integration/tls-configured) for general certificate guidance. ### Configure passive connections [#configure-passive-connections] FTP uses the control port for commands and a separate data connection for listings and file transfers. For passive mode: 1. Allow inbound TCP traffic to the configured control port. 2. Allow inbound TCP traffic to every port in the configured passive range. 3. Set `RUSTFS_FTP_EXTERNAL_IP` or `RUSTFS_FTPS_EXTERNAL_IP` when clients connect through NAT or a load balancer. For example, an FTPS server behind NAT can advertise its public hostname: ```bash export RUSTFS_FTPS_EXTERNAL_IP=storage.example.com export RUSTFS_FTPS_PASSIVE_PORTS=40000-40100 ``` Use the corresponding FTP-prefixed variables for the plain FTP listener. RustFS supports both active and passive transfer modes, but passive mode is generally easier to operate through client firewalls. ## Usage [#usage] The examples use [`lftp`](https://lftp.yar.ru/) and the canonical `my-bucket` and `hello.txt` names. The IAM user must have the S3 permissions required for each operation. ### Connect with FTP [#connect-with-ftp] Use plain FTP only with the loopback test listener: ```bash lftp -u ftp://127.0.0.1:8021 ``` Enter the secret key when `lftp` prompts for a password. ### Connect with FTPS [#connect-with-ftps] Connect to the explicit FTPS listener and require encryption for control and data connections: ```bash lftp -u -e ' set ftp:ssl-force true; set ftp:ssl-protect-data true; open ftp://storage.example.com:8022 ' ``` Enter the secret key when prompted. Keep certificate verification enabled in production. For the self-signed local test certificate only, connect to `localhost` and disable certificate verification for that session: ```bash lftp -u -e ' set ftp:ssl-force true; set ftp:ssl-protect-data true; set ssl:verify-certificate no; open ftp://localhost:8022 ' ``` Do not disable certificate verification in production. Install the issuing CA in the client trust store instead. ### List buckets and create a bucket [#list-buckets-and-create-a-bucket] At the `lftp` prompt, list the visible buckets and create `my-bucket`: ```text lftp> cls -1 / lftp> mkdir my-bucket lftp> cd my-bucket ``` Bucket names must follow the same naming rules as buckets created through the S3 API. ### Upload and download an object [#upload-and-download-an-object] Upload `/path/to/hello.txt`, list the bucket, and download the object: ```text lftp> put /path/to/hello.txt -o hello.txt lftp> ls lftp> get hello.txt -o hello.txt ``` The upload replaces an object with the same key. Appending to an existing object is not supported. ### Delete an object and bucket [#delete-an-object-and-bucket] Delete the object, return to the root, and delete the bucket: ```text lftp> rm hello.txt lftp> cd / lftp> rmdir my-bucket ``` Remember that `rmdir` recursively removes remaining objects from the bucket before deleting it. ### Connect other clients [#connect-other-clients] Use these settings in graphical FTP clients such as FileZilla or Cyberduck: | Setting | FTP test listener | FTPS listener | | ------------- | ----------------- | ----------------------- | | Protocol | FTP | FTP over TLS (explicit) | | Host | `127.0.0.1` | Your FTPS hostname | | Port | `8021` | `8022` | | Username | RustFS access key | RustFS access key | | Password | RustFS secret key | RustFS secret key | | Transfer mode | Passive | Passive | If login succeeds but directory listings or transfers time out, check the passive port firewall rules and external IP setting first. ## Next steps [#next-steps] * [Manage credentials](/operations/credentials) * [Configure TLS](/integration/tls-configured) * [Check service status](/operations/status-check) # S3 (/en/administration/protocols/s3) RustFS exposes an S3-compatible REST API for common object-storage workloads. It supports AWS Signature Version 4 (SigV4) and works with AWS CLI, AWS SDKs, and other S3 clients when you configure the RustFS endpoint, region, credentials, and addressing style. ## Overview [#overview] S3 organizes data into buckets and objects. With the default path-style addressing, a RustFS URL maps to storage as follows: ```text http://localhost:9000/my-bucket/path/to/hello.txt | bucket | |---- object key ----| ``` RustFS covers the common S3 data plane used by applications, backup tools, and SDKs: | Area | Supported workflows | | --------------- | ------------------------------------------------------------------------------------------------ | | Buckets | Create, delete, list, inspect, and retrieve location | | Objects | Put, get, head, copy, delete, multi-delete, range reads, conditional requests, and user metadata | | Listing | `ListObjects`, `ListObjectsV2`, prefixes, delimiters, markers, and pagination | | Large objects | Create, upload, copy, list, complete, and abort multipart uploads | | Data management | Versioning, lifecycle rules, bucket and object tags, checksums, and object lock | | Access control | IAM credentials and policies, bucket policies, public access block, and presigned GET/PUT URLs | | Integration | CORS, bucket notifications, replication configuration, and server-side encryption workflows | Authenticated clients sign requests with an access key and secret key. Anonymous requests are evaluated against the applicable bucket policy. RustFS provides broad S3 API compatibility for supported features, but it is not identical to every AWS S3 or MinIO API. Review [Known compatibility differences](#known-compatibility-differences) before depending on advanced or vendor-specific behavior, and validate it against your target RustFS release. ## Configuration [#configuration] The S3 API is enabled on the main RustFS listener. It does not require a separate protocol switch. | Variable | Purpose | Default | | ----------------------- | ----------------------------------------------------------- | --------------------- | | `RUSTFS_ADDRESS` | S3 API bind address | `:9000` | | `RUSTFS_REGION` | Region used for request signing and region-aware clients | `us-east-1` | | `RUSTFS_ACCESS_KEY` | Root access key | Installation-specific | | `RUSTFS_SECRET_KEY` | Root secret key | Installation-specific | | `RUSTFS_SERVER_DOMAINS` | Comma-separated domains for virtual-hosted-style requests | Not set | | `RUSTFS_TLS_PATH` | Directory containing `rustfs_cert.pem` and `rustfs_key.pem` | Not set | Use IAM users or service accounts instead of root credentials for applications, and grant only the required S3 actions. ### Path-style addressing [#path-style-addressing] Path-style addressing is the default and requires no DNS configuration. The bucket name is the first component of the request path: ```text http://localhost:9000/my-bucket/hello.txt ``` Configure clients with: * Endpoint: `http://localhost:9000` * Region: `us-east-1` * Access key: `` * Secret key: `` * Addressing style: path-style Some clients default to virtual-hosted-style requests. Enable their path-style option, such as `force_path_style=true` in AWS SDK configuration or `s3_use_path_style = true` in Terraform. ### Virtual-hosted-style addressing [#virtual-hosted-style-addressing] Virtual-hosted-style addressing places the bucket name in the hostname. Set `RUSTFS_SERVER_DOMAINS` to each base domain that RustFS should recognize: ```bash export RUSTFS_SERVER_DOMAINS="s3.example.com" ``` The same object is then addressed as: ```text https://my-bucket.s3.example.com/hello.txt ``` Configure wildcard DNS, such as `*.s3.example.com`, to resolve to the RustFS endpoint. For HTTPS, the certificate must also cover the bucket hostnames. Bucket names containing dots may require explicit certificate names because a single-label wildcard does not span multiple labels. Do not send virtual-hosted-style requests unless `RUSTFS_SERVER_DOMAINS` is configured. RustFS otherwise treats requests as path-style and cannot derive the bucket from the hostname. ### TLS and network access [#tls-and-network-access] For production, set `RUSTFS_TLS_PATH` to a directory containing `rustfs_cert.pem` and `rustfs_key.pem`, then use an `https://` endpoint. Ensure clients trust the issuing certificate authority. Allow the configured S3 API port through host firewalls and load balancers. The default is TCP port `9000`; this listener also carries RustFS administrative and internode traffic, so do not expose it anonymously without appropriate network and IAM controls. ## Usage [#usage] The following examples use AWS CLI with the canonical local endpoint. Install and configure [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) before continuing. ### Configure AWS CLI [#configure-aws-cli] ```bash aws configure ``` Enter these values when prompted: ```text AWS Access Key ID [None]: AWS Secret Access Key [None]: Default region name [None]: us-east-1 Default output format [None]: json ``` Pass the RustFS endpoint to every command. The custom endpoint keeps these examples on RustFS instead of AWS S3. ```bash export RUSTFS_ENDPOINT="http://localhost:9000" ``` ### Create and list buckets [#create-and-list-buckets] ```bash aws --endpoint-url "$RUSTFS_ENDPOINT" s3api create-bucket \ --bucket my-bucket aws --endpoint-url "$RUSTFS_ENDPOINT" s3api list-buckets ``` ### Upload and list objects [#upload-and-list-objects] ```bash printf 'Hello from RustFS\n' > hello.txt aws --endpoint-url "$RUSTFS_ENDPOINT" s3 cp \ hello.txt s3://my-bucket/path/to/hello.txt aws --endpoint-url "$RUSTFS_ENDPOINT" s3 ls \ s3://my-bucket/path/to/ ``` AWS CLI automatically uses multipart upload when its transfer configuration selects it for a large object. RustFS supports the standard multipart create, upload-part, complete, list, and abort workflows. ### Download and inspect an object [#download-and-inspect-an-object] ```bash aws --endpoint-url "$RUSTFS_ENDPOINT" s3 cp \ s3://my-bucket/path/to/hello.txt ./downloaded-hello.txt aws --endpoint-url "$RUSTFS_ENDPOINT" s3api head-object \ --bucket my-bucket \ --key path/to/hello.txt ``` ### Create a presigned URL [#create-a-presigned-url] Generate a time-limited URL that can download an object without exposing credentials: ```bash aws --endpoint-url "$RUSTFS_ENDPOINT" s3 presign \ s3://my-bucket/path/to/hello.txt \ --expires-in 3600 ``` The hostname, scheme, port, region, and object path used by the recipient must match the values used to sign the URL. Use the externally reachable endpoint when generating URLs for another machine. ### Delete objects and buckets [#delete-objects-and-buckets] ```bash aws --endpoint-url "$RUSTFS_ENDPOINT" s3 rm \ s3://my-bucket/path/to/hello.txt aws --endpoint-url "$RUSTFS_ENDPOINT" s3api delete-bucket \ --bucket my-bucket ``` An S3 bucket must be empty before it can be deleted. ## Known compatibility differences [#known-compatibility-differences] Current RustFS compatibility tests cover the common workflows described above. The following areas are not equivalent to AWS S3 or MinIO: * Bucket access logging and bucket ownership controls are planned rather than complete. * ACL authorization is intentionally unsupported. Canned ACL headers have partial compatibility, while XML grant policies return `NotImplemented`. Prefer IAM and bucket policies. * POST Object form uploads are implemented, but checksum handling for that workflow is not complete. * Some multipart upload listing and part-lookup edge cases are outside the default compatibility gate. * S3 Select currently accepts only uncompressed input. * Access Point and Outposts copy-source forms are not implemented. * MinIO administrative APIs are a separate compatibility surface and should not be inferred from S3 data-plane support. Client behavior can also differ when a tool assumes AWS-specific services, storage classes, account ownership controls, or endpoint discovery. Always set the RustFS endpoint explicitly and test the exact operations your application uses. ## Next steps [#next-steps] * Follow the complete [AWS CLI example](/developer/examples/aws-cli). * Choose an [S3 SDK](/developer/sdk) for application integration. * Configure [credentials and access policies](/operations/credentials). * Enable [TLS](/integration/tls-configured) before exposing the endpoint outside a trusted network. # SFTP (/en/administration/protocols/sftp) RustFS includes an SSH File Transfer Protocol (SFTP) gateway that exposes buckets and objects to SFTP clients over an encrypted SSH connection. You can list and create buckets and upload, download, rename, or delete objects while RustFS enforces the permissions of the authenticated Identity and Access Management (IAM) user. SFTP is an optional compile-time feature and is not included in the default RustFS build. Build RustFS with the `sftp` or `full` feature before enabling the listener. The listener is disabled at runtime by default. ## Overview [#overview] The gateway maps SFTP paths to RustFS resources: | SFTP path | RustFS resource | | --------------------------- | --------------------------------------------- | | `/` | All buckets visible to the authenticated user | | `/my-bucket/` | The `my-bucket` bucket | | `/my-bucket/hello.txt` | The `hello.txt` object in `my-bucket` | | `/my-bucket/docs/hello.txt` | The `docs/hello.txt` object in `my-bucket` | Creating or removing a directory directly below `/` creates or removes a bucket. Directories below a bucket map to object key prefixes and do not exist as independent filesystem directories. RustFS supports the following operations through the SFTP gateway: | SFTP operation | RustFS operation | | ----------------------- | ----------------------------------------- | | List `/` | List visible buckets | | List a bucket or prefix | List objects and prefixes | | `mkdir` at `/` | Create a bucket | | `put` | Upload an object | | `get` | Download an object | | `rename` | Copy an object and then delete the source | | `rm` | Delete an object | | `rmdir` at `/` | Delete an empty bucket | Enter a RustFS access key as the SFTP username and its secret key as the password. RustFS checks the IAM user's S3 policies for every operation. The SFTP gateway does not support client public-key authentication or anonymous access. SSH host keys identify the server; they do not authenticate clients. Do not configure an `authorized_keys` file for RustFS SFTP. ## Configuration [#configuration] ### Build with SFTP support [#build-with-sftp-support] Build RustFS with the SFTP feature: ```bash cargo build --release --features sftp ``` To enable all optional RustFS features, including SFTP, use `--features full` instead. Setting `RUSTFS_SFTP_ENABLE=true` has no effect in a binary built without the `sftp` feature. ### SFTP variables [#sftp-variables] | Variable | Description | Default | | -------------------------- | -------------------------------------------------------------------------------------------------- | ------------------- | | `RUSTFS_SFTP_ENABLE` | Enables the SFTP listener. | `false` | | `RUSTFS_SFTP_ADDRESS` | Bind address for SFTP connections. | `0.0.0.0:2222` | | `RUSTFS_SFTP_HOST_KEY_DIR` | Directory containing at least one unencrypted SSH private host key. Required when SFTP is enabled. | Not set | | `RUSTFS_SFTP_READ_ONLY` | Rejects operations that modify buckets or objects. | `false` | | `RUSTFS_SFTP_IDLE_TIMEOUT` | SSH inactivity timeout in seconds. Must be greater than zero. | `600` | | `RUSTFS_SFTP_PART_SIZE` | Multipart upload part size in bytes. | `16777216` (16 MiB) | The default port `2222` avoids the privileged SSH port `22`. The address can use IPv4 or IPv6 syntax, for example `127.0.0.1:2222` or `[::]:2222`. ### Prepare a host key [#prepare-a-host-key] Create a persistent host-key directory and generate an Ed25519 host key: ```bash mkdir -p /path/to/sftp-keys ssh-keygen -t ed25519 \ -f /path/to/sftp-keys/ssh_host_ed25519_key \ -N "" chmod 600 /path/to/sftp-keys/ssh_host_ed25519_key* ``` The host key must not use a passphrase. RustFS also accepts decodable ECDSA and RSA private host keys. On Unix, every regular file in the host-key directory must be accessible only by its owner. The `ssh-keygen` command also creates a `.pub` file; either keep it with owner-only permissions as shown or remove it because RustFS does not read it. RustFS refuses to start if the directory is missing, contains no usable private key, or contains a regular file with group or other permission bits set. Keep the same host key across restarts so clients can verify the server identity. Restrict access to the account that runs RustFS and back up the key securely. A changed host key causes clients to display a possible man-in-the-middle warning. ### Start the SFTP listener [#start-the-sftp-listener] The following example starts SFTP on the loopback interface for local testing: ```bash export RUSTFS_SFTP_ENABLE=true export RUSTFS_SFTP_ADDRESS=127.0.0.1:2222 export RUSTFS_SFTP_HOST_KEY_DIR=/path/to/sftp-keys export RUSTFS_ACCESS_KEY= export RUSTFS_SECRET_KEY= rustfs /path/to/data ``` For remote connections, bind to an appropriate network interface and allow inbound TCP traffic to the configured SFTP port. This listener is separate from the S3 API and Console listeners. ## Usage [#usage] The following examples use the OpenSSH `sftp` client, the canonical `my-bucket` and `hello.txt` names, and a local listener on port `2222`. ### Connect [#connect] ```bash sftp -P 2222 @127.0.0.1 ``` Enter the RustFS secret key when prompted for the password. On the first connection, verify the displayed host-key fingerprint before accepting it. ### List and create buckets [#list-and-create-buckets] At the `sftp` prompt, list the buckets visible to the IAM user and create `my-bucket`: ```text sftp> ls / sftp> mkdir /my-bucket sftp> cd /my-bucket ``` Bucket names must follow the same naming rules as buckets created through the S3 API. Files cannot be created directly under `/`. ### Upload and download an object [#upload-and-download-an-object] Upload `/path/to/hello.txt`, list the bucket, and download the object: ```text sftp> put /path/to/hello.txt /my-bucket/hello.txt sftp> ls /my-bucket sftp> get /my-bucket/hello.txt hello.txt ``` Uploads must be sequential from the beginning of the file. Resume, append, in-place edits, and segmented multi-connection uploads of one object are not supported. ### Rename an object [#rename-an-object] ```text sftp> rename /my-bucket/hello.txt /my-bucket/greeting.txt ``` RustFS implements rename as a server-side copy followed by deletion of the source. The operation is not atomic, and renaming a bucket is not supported. The IAM user needs permission to read and delete the source and to write the destination. ### Delete an object and bucket [#delete-an-object-and-bucket] Delete the object and then remove the empty bucket: ```text sftp> rm /my-bucket/greeting.txt sftp> rmdir /my-bucket ``` RustFS does not recursively delete a non-empty bucket through SFTP. ### Connect a desktop client [#connect-a-desktop-client] Use these settings in graphical clients such as FileZilla, Cyberduck, or WinSCP: | Setting | Value | | -------------- | -------------------------------------------- | | Protocol | SFTP (SSH File Transfer Protocol) | | Host | Your RustFS SFTP hostname | | Port | `2222`, or the port in `RUSTFS_SFTP_ADDRESS` | | Username | RustFS access key | | Password | RustFS secret key | | Authentication | Password | Configure the client for whole-file, single-connection uploads. Symbolic links and POSIX ownership, permission, or timestamp changes are not supported because the gateway maps SFTP operations to object storage. ## Next steps [#next-steps] * [Manage credentials](/operations/credentials) * [Check service status](/operations/status-check) * [Configure lifecycle management](/administration/data/lifecycle-management) # OpenStack Swift (/en/administration/protocols/swift) RustFS can expose an OpenStack Swift-compatible API on the same HTTP endpoint as its S3 API. Use this guide to build the optional `swift` feature, configure Keystone token validation, and verify basic account, container, and object operations. Swift support is optional and does not cover every OpenStack Swift behavior. Account `HEAD` requests and non-JSON listing formats are not implemented. Validate your client workflow before using the API in production. ## How Swift maps to RustFS [#how-swift-maps-to-rustfs] Swift requests use `/v1/AUTH_/...` on the RustFS S3 API endpoint: | Swift resource | Request path | RustFS mapping | | -------------- | -------------------------------------------- | ---------------------------------- | | Account | `/v1/AUTH_` | The authenticated Keystone project | | Container | `/v1/AUTH_/` | A project-isolated RustFS bucket | | Object | `/v1/AUTH_//` | An object in the mapped bucket | The project ID in the URL must match the project ID in the validated Keystone token. RustFS accepts the token in either `X-Auth-Token` or `X-Storage-Token`. The confirmed core operations are: | Scope | Operations | | --------- | ------------------------------------------------------------------------ | | Account | List containers, update account metadata | | Container | Create, list, inspect, update metadata, delete | | Object | Upload, download, range download, inspect, update metadata, copy, delete | ## Build with Swift support [#build-with-swift-support] The default RustFS feature set does not include Swift. Build it explicitly from the `rustfs/rustfs` repository: ```bash cargo build --release --features swift ``` The resulting binary serves Swift paths on the configured S3 API address. There is no separate Swift listener or Swift-specific port. ## Configure Keystone [#configure-keystone] Enable Keystone and set its authentication endpoint before starting RustFS: ```bash export RUSTFS_KEYSTONE_ENABLE=true export RUSTFS_KEYSTONE_AUTH_URL=https://keystone.example.com export RUSTFS_KEYSTONE_VERSION=v3 export RUSTFS_KEYSTONE_VERIFY_SSL=true ``` | Variable | Purpose | Default | | ---------------------------- | ----------------------------------------------------------------------------- | ------- | | `RUSTFS_KEYSTONE_ENABLE` | Enables Keystone token validation. | `false` | | `RUSTFS_KEYSTONE_AUTH_URL` | Sets the Keystone authentication endpoint. Required when Keystone is enabled. | Not set | | `RUSTFS_KEYSTONE_VERSION` | Selects the Keystone API version. | `v3` | | `RUSTFS_KEYSTONE_VERIFY_SSL` | Verifies the Keystone TLS certificate. | `true` | | `RUSTFS_KEYSTONE_CACHE_SIZE` | Sets the maximum token-cache entry count. | `10000` | | `RUSTFS_KEYSTONE_CACHE_TTL` | Sets the token-cache lifetime in seconds. | `300` | | `RUSTFS_KEYSTONE_TIMEOUT` | Sets the Keystone request timeout in seconds. | `30` | We recommend keeping TLS verification enabled. RustFS returns `401 Unauthorized` when Keystone rejects a supplied token; it does not fall back to local credentials for that request. ## Verify the API [#verify-the-api] Obtain a scoped token and project ID from Keystone, then set these shell variables: ```bash export SWIFT_TOKEN='' export SWIFT_ACCOUNT='AUTH_' export SWIFT_URL="http://localhost:9000/v1/${SWIFT_ACCOUNT}" ``` List the containers visible to the project: ```bash curl --fail-with-body \ --header "X-Auth-Token: ${SWIFT_TOKEN}" \ "${SWIFT_URL}" ``` Create `my-bucket`, upload `hello.txt`, and download it: ```bash curl --fail-with-body --request PUT \ --header "X-Auth-Token: ${SWIFT_TOKEN}" \ "${SWIFT_URL}/my-bucket" curl --fail-with-body --request PUT \ --header "X-Auth-Token: ${SWIFT_TOKEN}" \ --upload-file /path/to/hello.txt \ "${SWIFT_URL}/my-bucket/hello.txt" curl --fail-with-body \ --header "X-Auth-Token: ${SWIFT_TOKEN}" \ "${SWIFT_URL}/my-bucket/hello.txt" ``` A request to an `AUTH_` account that does not match the token project returns `403 Forbidden`. ## Next steps [#next-steps] * [Review the S3 compatibility matrix](/en/reference/s3-compatibility) * [Manage RustFS credentials](/en/operations/credentials) * [Configure TLS for RustFS](/en/integration/tls-configured) # WebDAV (/en/administration/protocols/webdav) RustFS includes a Web Distributed Authoring and Versioning (WebDAV) gateway that exposes buckets and objects to WebDAV clients over HTTP or HTTPS. You can browse buckets, create collections, and upload, download, rename, or delete objects while RustFS enforces the permissions of the authenticated Identity and Access Management (IAM) user. WebDAV support is compiled into the standard RustFS binary, but the gateway is disabled at runtime by default. You must enable and configure it before connecting a client. ## Overview [#overview] The gateway maps WebDAV paths to RustFS resources: | WebDAV path | RustFS resource | | --------------------------- | --------------------------------------------- | | `/` | All buckets visible to the authenticated user | | `/my-bucket/` | The `my-bucket` bucket | | `/my-bucket/hello.txt` | The `hello.txt` object in `my-bucket` | | `/my-bucket/docs/hello.txt` | The `docs/hello.txt` object in `my-bucket` | RustFS supports the following operations through the gateway: | Method | Operation | | ---------- | ----------------------------------------- | | `PROPFIND` | List buckets or objects and read metadata | | `MKCOL` | Create a bucket or a directory prefix | | `PUT` | Upload an object | | `GET` | Download an object | | `HEAD` | Read object metadata | | `MOVE` | Rename or move an object or directory | | `DELETE` | Delete an object, directory, or bucket | Use `PROPFIND`, not `GET`, to list a collection. The current gateway returns `405 Method Not Allowed` for `GET` requests to directories. WebDAV uses HTTP Basic authentication. Enter a RustFS access key as the username and its secret key as the password. The gateway authenticates the credentials against RustFS IAM and applies the user's S3 policies to each operation. Basic authentication does not encrypt credentials. Disable TLS only for isolated local testing. Use HTTPS for every remote or production connection. ## Configuration [#configuration] Configure the gateway with environment variables before starting RustFS: | Variable | Description | Default | | ----------------------------- | ----------------------------------------------------------------------------------- | -------------------- | | `RUSTFS_WEBDAV_ENABLE` | Enables the WebDAV gateway. | `false` | | `RUSTFS_WEBDAV_ADDRESS` | Bind address for WebDAV connections. | `0.0.0.0:8080` | | `RUSTFS_WEBDAV_TLS_ENABLED` | Enables TLS for the WebDAV listener. | `true` | | `RUSTFS_WEBDAV_CERTS_DIR` | Certificate directory used by the RustFS TLS runtime. Required when TLS is enabled. | Not set | | `RUSTFS_WEBDAV_MAX_BODY_SIZE` | Maximum request body size in bytes. Must be greater than zero. | `5368709120` (5 GiB) | The standard RustFS build enables the WebDAV compile-time feature. If you build RustFS with `--no-default-features`, include the feature explicitly: ```bash cargo build --release --features webdav ``` ### Local testing without TLS [#local-testing-without-tls] The following example starts RustFS with an HTTP WebDAV listener on port `8080`: ```bash export RUSTFS_WEBDAV_ENABLE=true export RUSTFS_WEBDAV_ADDRESS=127.0.0.1:8080 export RUSTFS_WEBDAV_TLS_ENABLED=false export RUSTFS_ACCESS_KEY= export RUSTFS_SECRET_KEY= rustfs /path/to/data ``` Binding to `127.0.0.1` prevents other hosts from connecting to the unencrypted test endpoint. ### HTTPS [#https] For a remote or production connection, enable TLS and provide a certificate directory: ```bash export RUSTFS_WEBDAV_ENABLE=true export RUSTFS_WEBDAV_ADDRESS=0.0.0.0:8080 export RUSTFS_WEBDAV_TLS_ENABLED=true export RUSTFS_WEBDAV_CERTS_DIR=/path/to/certs rustfs /path/to/data ``` If TLS is enabled without `RUSTFS_WEBDAV_CERTS_DIR`, or the directory does not exist, WebDAV initialization fails. See [Configure TLS](/integration/tls-configured) for certificate preparation guidance. Allow inbound TCP traffic to the configured WebDAV port. This listener is separate from the S3 API and Console listeners. ## Usage [#usage] The following commands use an HTTP endpoint for local testing. Replace the endpoint and credentials with your HTTPS WebDAV endpoint and an IAM user that has the required bucket and object permissions. Set reusable shell variables without placing the secret directly in each command: ```bash export WEBDAV_URL=http://127.0.0.1:8080 export WEBDAV_USER= read -s WEBDAV_PASSWORD export WEBDAV_PASSWORD ``` ### List buckets [#list-buckets] Send `PROPFIND` with `Depth: 1` to list the buckets visible to the user: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request PROPFIND \ --header "Depth: 1" \ "$WEBDAV_URL/" ``` ### Create a bucket [#create-a-bucket] ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request MKCOL \ "$WEBDAV_URL/my-bucket/" ``` Bucket names must follow the same naming rules as buckets created through the S3 API. ### Upload and download an object [#upload-and-download-an-object] Upload `/path/to/hello.txt`: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --upload-file /path/to/hello.txt \ "$WEBDAV_URL/my-bucket/hello.txt" ``` Download the object: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --output hello.txt \ "$WEBDAV_URL/my-bucket/hello.txt" ``` ### Create and list a directory [#create-and-list-a-directory] WebDAV directories below a bucket map to object key prefixes: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request MKCOL \ "$WEBDAV_URL/my-bucket/docs/" curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request PROPFIND \ --header "Depth: 1" \ "$WEBDAV_URL/my-bucket/docs/" ``` ### Rename an object [#rename-an-object] Use `MOVE` with a destination path on the same WebDAV endpoint: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request MOVE \ --header "Destination: $WEBDAV_URL/my-bucket/greeting.txt" \ "$WEBDAV_URL/my-bucket/hello.txt" ``` The IAM user needs permission to read and delete the source and to write the destination. If authorization fails, RustFS leaves the source unchanged. ### Delete an object or bucket [#delete-an-object-or-bucket] Delete an object: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request DELETE \ "$WEBDAV_URL/my-bucket/greeting.txt" ``` Delete the bucket after removing its contents: ```bash curl --user "$WEBDAV_USER:$WEBDAV_PASSWORD" \ --request DELETE \ "$WEBDAV_URL/my-bucket/" ``` ### Connect a desktop client [#connect-a-desktop-client] Use the WebDAV endpoint in a client that supports Basic authentication: | Client | Connection address | | --------------------- | ---------------------------------------------------------------- | | GNOME Files | `dav://:8080/` for HTTP or `davs://:8080/` for HTTPS | | macOS Finder | `http://:8080/` or `https://:8080/` | | Windows File Explorer | `https://:8080/` | Enter the RustFS access key and secret key when the client prompts for credentials. Client behavior and supported WebDAV methods vary; use `curl` to isolate server-side errors when troubleshooting. ## Next steps [#next-steps] * [Manage credentials](/operations/credentials) * [Configure TLS](/integration/tls-configured) * [Check service status](/operations/status-check) # AWS CLI (/en/developer/examples/aws-cli) The [AWS CLI](https://docs.aws.amazon.com/cli/) is Amazon's official command-line tool for S3, and it works with RustFS via the `--endpoint-url` flag. ## Install [#install] Follow the [official install guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), or on macOS: ```bash brew install awscli ``` ## Configure [#configure] Set your [access keys](../../security-compliance/iam/access-token.md) and region: ```bash aws configure ``` ```text AWS Access Key ID [None]: AWS Secret Access Key [None]: Default region name [None]: us-east-1 Default output format [None]: json ``` Pass your RustFS address with `--endpoint-url` on every command. Replace `http://localhost:9000` with your server address. When `--endpoint-url` is set, the AWS CLI uses path-style addressing, which is what RustFS requires. If you did not set credentials at install, the local-test default is `rustfsadmin` / `rustfsadmin` — never use it beyond a throwaway local trial. ## Verify [#verify] Create a bucket: ```bash aws s3 mb s3://my-bucket --endpoint-url http://localhost:9000 ``` ```text make_bucket: my-bucket ``` Upload a file: ```bash aws s3 cp /path/to/hello.txt s3://my-bucket/ --endpoint-url http://localhost:9000 ``` ```text upload: ../path/to/hello.txt to s3://my-bucket/hello.txt ``` List the bucket: ```bash aws s3 ls s3://my-bucket --endpoint-url http://localhost:9000 ``` ```text 2026-07-15 10:30:00 12 hello.txt ``` ## Next steps [#next-steps] Build applications against RustFS with an [S3 SDK](../sdk/index.md), or manage objects with [`rc`](/operations/rc). # AWS SDK for Go (/en/developer/examples/aws-sdk-go) The [AWS SDK for Go v2](https://aws.github.io/aws-sdk-go-v2/docs/) connects to RustFS through a custom base endpoint. This is the minimal connection recipe; see the [Go SDK guide](../sdk/go.md) for a full program. ## Install [#install] ```bash go get github.com/aws/aws-sdk-go-v2/aws go get github.com/aws/aws-sdk-go-v2/credentials go get github.com/aws/aws-sdk-go-v2/service/s3 ``` ## Configure [#configure] Replace `http://localhost:9000` with your server address and use your own [access keys](../../security-compliance/iam/access-token.md). RustFS requires path-style addressing (`UsePathStyle: true`): ```go {9,11} import ( "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/s3" ) cfg := aws.Config{ Region: "us-east-1", Credentials: aws.NewCredentialsCache(credentials.NewStaticCredentialsProvider("", "", "")), } client := s3.NewFromConfig(cfg, func(o *s3.Options) { o.BaseEndpoint = aws.String("http://localhost:9000") o.UsePathStyle = true }) ``` ## Verify [#verify] ```go ctx := context.Background() client.CreateBucket(ctx, &s3.CreateBucketInput{Bucket: aws.String("my-bucket")}) out, _ := client.ListBuckets(ctx, &s3.ListBucketsInput{}) for _, b := range out.Buckets { fmt.Println(*b.Name) } ``` ```text my-bucket ``` ## Next steps [#next-steps] See the full [Go SDK guide](../sdk/go.md), or manage objects with [`rc`](/operations/rc). # AWS SDK for JavaScript (/en/developer/examples/aws-sdk-js) The [AWS SDK for JavaScript v3](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/) connects to RustFS through a custom endpoint. This is the minimal connection recipe; see the [JavaScript SDK guide](../sdk/javascript.md) for a full program. ## Install [#install] ```bash npm install @aws-sdk/client-s3 ``` ## Configure [#configure] Replace `http://localhost:9000` with your server address and use your own [access keys](../../security-compliance/iam/access-token.md). RustFS requires path-style addressing (`forcePathStyle: true`): ```javascript title="index.mjs" {8} import { S3Client, CreateBucketCommand, PutObjectCommand, ListObjectsV2Command } from "@aws-sdk/client-s3"; import { readFileSync } from "node:fs"; const s3 = new S3Client({ endpoint: "http://localhost:9000", region: "us-east-1", credentials: { accessKeyId: "", secretAccessKey: "" }, forcePathStyle: true, }); ``` ## Verify [#verify] ```javascript await s3.send(new CreateBucketCommand({ Bucket: "my-bucket" })); await s3.send(new PutObjectCommand({ Bucket: "my-bucket", Key: "hello.txt", Body: readFileSync("/path/to/hello.txt") })); const out = await s3.send(new ListObjectsV2Command({ Bucket: "my-bucket" })); for (const obj of out.Contents ?? []) console.log(obj.Key, obj.Size); ``` ```text hello.txt 12 ``` ## Next steps [#next-steps] See the full [JavaScript SDK guide](../sdk/javascript.md), or manage objects with [`rc`](/operations/rc). # boto3 (Python) (/en/developer/examples/boto3) [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) is the AWS SDK for Python and connects to RustFS through a custom endpoint. ## Install [#install] ```bash pip install boto3 ``` ## Configure [#configure] Point the client at your RustFS endpoint. Replace `http://localhost:9000` with your server address, and use your own [access keys](../../security-compliance/iam/access-token.md). RustFS requires path-style addressing, set via botocore `Config`: ```python import boto3 from botocore.config import Config s3 = boto3.client( "s3", endpoint_url="http://localhost:9000", aws_access_key_id="", aws_secret_access_key="", region_name="us-east-1", config=Config(s3={"addressing_style": "path"}), ) ``` ## Verify [#verify] Create a bucket, upload a file, and list the bucket: ```python s3.create_bucket(Bucket="my-bucket") s3.upload_file("/path/to/hello.txt", "my-bucket", "hello.txt") for obj in s3.list_objects_v2(Bucket="my-bucket").get("Contents", []): print(obj["Key"], obj["Size"]) ``` Expected output: ```text hello.txt 12 ``` ## Next steps [#next-steps] See the [S3 SDK overview](../sdk/index.md) for more languages, or manage objects with [`rc`](/operations/rc). # rclone (/en/developer/examples/rclone) [rclone](https://rclone.org/) is a command-line program for syncing files with cloud storage and speaks the S3 protocol that RustFS implements. ## Install [#install] ```bash curl https://rclone.org/install.sh | sudo bash ``` Or see the [official install guide](https://rclone.org/install/). ## Configure [#configure] Add a remote to `~/.config/rclone/rclone.conf`. Replace `http://localhost:9000` with your server address, and use your own [access keys](../../security-compliance/iam/access-token.md). `force_path_style = true` is required because RustFS uses path-style addressing: ```ini title="~/.config/rclone/rclone.conf" [rustfs] type = s3 provider = Other access_key_id = secret_access_key = endpoint = http://localhost:9000 region = us-east-1 force_path_style = true ``` ## Verify [#verify] Create a bucket: ```bash rclone mkdir rustfs:my-bucket ``` Upload a file: ```bash rclone copy /path/to/hello.txt rustfs:my-bucket ``` List buckets and contents: ```bash rclone lsd rustfs: rclone ls rustfs:my-bucket ``` ```text -1 2026-07-15 10:30:00 -1 my-bucket 12 hello.txt ``` ## Next steps [#next-steps] Build applications against RustFS with an [S3 SDK](../sdk/index.md), or manage objects with [`rc`](/operations/rc). # s3cmd (/en/developer/examples/s3cmd) [s3cmd](https://s3tools.org/s3cmd) is a command-line client for S3-compatible storage. Point it at your RustFS endpoint with a small config file. ## Install [#install] ```bash # macOS brew install s3cmd # Debian/Ubuntu sudo apt install s3cmd # or via pip pip install s3cmd ``` ## Configure [#configure] Create `~/.s3cfg`. Replace `localhost:9000` with your server address and use your own [access keys](../../security-compliance/iam/access-token.md). RustFS uses path-style addressing, so set the bucket host to the same endpoint: ```ini title="~/.s3cfg" [default] access_key = secret_key = host_base = localhost:9000 host_bucket = localhost:9000 use_https = False signature_v2 = False ``` Set `use_https = True` and point at port `9000` if you have [configured TLS](../../integration/tls-configured.md). ## Verify [#verify] Create a bucket, upload a file, and list it: ```bash s3cmd mb s3://my-bucket s3cmd put /path/to/hello.txt s3://my-bucket/hello.txt s3cmd ls s3://my-bucket ``` ```text upload: '/path/to/hello.txt' -> 's3://my-bucket/hello.txt' [1 of 1] 2026-07-16 10:00 12 s3://my-bucket/hello.txt ``` ## Next steps [#next-steps] See the [SDK overview](../sdk/index.md) to connect an application, or manage objects with [`rc`](/operations/rc). # Terraform (/en/developer/examples/terraform) The [Terraform AWS provider](https://registry.terraform.io/providers/hashicorp/aws/latest) works against RustFS when you point its S3 endpoint at your server and enable path-style addressing. ## Configure [#configure] Replace `http://localhost:9000` with your server address and use your own [access keys](../../security-compliance/iam/access-token.md). The skip flags stop the provider from calling AWS-only metadata and STS endpoints: ```hcl title="main.tf" provider "aws" { access_key = "" secret_key = "" region = "us-east-1" s3_use_path_style = true skip_credentials_validation = true skip_metadata_api_check = true skip_requesting_account_id = true endpoints { s3 = "http://localhost:9000" } } resource "aws_s3_bucket" "demo" { bucket = "my-bucket" } resource "aws_s3_object" "hello" { bucket = aws_s3_bucket.demo.id key = "hello.txt" source = "/path/to/hello.txt" } ``` ## Apply [#apply] ```bash terraform init terraform apply ``` ```text Plan: 2 to add, 0 to change, 0 to destroy. ... aws_s3_bucket.demo: Creation complete after 0s [id=my-bucket] aws_s3_object.hello: Creation complete after 0s [id=hello.txt] Apply complete! Resources: 2 added, 0 changed, 0 destroyed. ``` ## Next steps [#next-steps] See the [SDK overview](../sdk/index.md) to connect an application, or the [AWS CLI example](aws-cli.md) for ad-hoc commands. # Developer (/en/developer) Use this section to integrate applications with RustFS through temporary credentials, the Model Context Protocol (MCP), SDKs, and the S3-compatible API. * [Security Token Service (STS)](/security-compliance/iam/sts) explains how to request temporary credentials. * [MCP](/developer/mcp) connects AI tools and agents to RustFS. * [SDKs](/developer/sdk) provide language-specific integration examples. * [Integration guides](/developer/integration) cover proxies and data platforms. * [S3 API](/administration/protocols/s3) documents client configuration and compatibility considerations. # Backup (/en/developer/integration/backup) Use **RustFS** as the object storage backend for backup tools that store repository data in an S3-compatible service. ## Systems [#systems] * [Restic](./restic.md) Keep backup jobs in a dedicated bucket and prefix, and use credentials scoped to the required bucket operations. # Restic (/en/developer/integration/backup/restic) Restic is a command-line backup tool that stores snapshots in a repository. This guide shows how to point Restic at RustFS, back up a local directory, restore a snapshot, and verify the stored objects in RustFS. You need a running RustFS instance, a bucket such as `my-bucket`, Restic installed, access keys for the bucket, and a password for the Restic repository. Restic's S3 backend should use path-style access for RustFS. This guide sets `-o s3.bucket-lookup=path` and uses the bucket name in the repository URL. ## Architecture [#architecture] Restic writes repository metadata and backup snapshots into RustFS through the S3 API. The repository prefix in this guide is `restic`, and the bucket is `my-bucket`.
## Prepare the bucket [#1-prepare-the-bucket] Open the RustFS Console at `http://localhost:9001` and create `my-bucket`, or choose an existing bucket that is dedicated to backups. RustFS Console bucket list with the my-bucket backup bucket selected Use a dedicated bucket for each backup job. The example screenshot shows the Console in English and light theme.
## Initialize the Restic repository [#2-initialize-the-restic-repository] Set the credentials, region, repository password, and repository location: ```bash export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= export AWS_DEFAULT_REGION=us-east-1 export RESTIC_PASSWORD= export RESTIC_REPOSITORY=s3:http://localhost:9000/my-bucket/restic ``` Run `restic init` with path-style bucket lookup: ```bash restic -o s3.bucket-lookup=path init ``` The repository password protects the snapshot data. Keep it separate from the RustFS access keys.
## Back up data [#3-back-up-data] Create a small test directory and back it up: ```bash mkdir -p ~/Documents/restic-demo printf 'hello from RustFS\n' > ~/Documents/restic-demo/hello.txt restic -o s3.bucket-lookup=path backup ~/Documents/restic-demo ``` Restic prints a snapshot ID after the backup completes. Run the command again after changing a file to create a second snapshot.
## Restore a snapshot [#4-restore-a-snapshot] Restore the latest snapshot to a separate directory: ```bash mkdir -p ~/Documents/restic-restore restic -o s3.bucket-lookup=path restore latest --target ~/Documents/restic-restore ``` You can also restore a specific snapshot ID if you want to recover an earlier version.
## Verify the repository in RustFS [#5-verify-the-repository-in-rustfs] Open `my-bucket` in the RustFS Console and confirm that Restic created repository objects under the `restic/` prefix. You can also run a repository check: ```bash restic -o s3.bucket-lookup=path check ```
## Next steps [#next-steps] * [CLI Client (rc)](/operations/rc) to create buckets and inspect objects from the command line. * [TLS Configuration](/integration/tls-configured) if you expose RustFS outside a trusted network. # Apache Iceberg (/en/developer/integration/big-data/iceberg) This guide runs **Apache Iceberg** with Spark, an Iceberg REST catalog, and **RustFS** as the S3-compatible warehouse. You will create an Iceberg table, write rows, query them, and verify that the table files are stored in RustFS. To use the REST catalog built into RustFS, follow [S3 Tables setup](/administration/data/s3-tables) and the [PyIceberg guide](/developer/integration/big-data/pyiceberg). The deployment below runs a separate catalog service. You need Docker with the Compose plugin and enough local resources to run four containers. This deployment is intended for local integration testing, not production. Apache Iceberg [PR #14928](https://github.com/apache/iceberg/pull/14928) demonstrated the same Spark, REST catalog, `S3FileIO`, and RustFS workflow, including table creation and writes. The pull request was closed without merging, and the current [Spark quickstart](https://iceberg.apache.org/spark-quickstart/#docker-compose) still uses another S3-compatible store. The configuration below therefore documents a RustFS integration rather than an Apache Iceberg default. ## Architecture [#architecture] Spark uses the REST service for catalog operations. Both Spark and the REST catalog receive the RustFS endpoint, region, credentials, and path-style setting so they can access metadata and data files in `s3://my-bucket/warehouse`.
## Create the project files [#1-create-the-project-files] Create a working directory: ```bash mkdir rustfs-iceberg cd rustfs-iceberg ``` Create an environment file and replace both credential placeholders: ```ini title=".env" RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Use dedicated credentials for the warehouse bucket. Do not commit `.env` to source control. Create the Spark catalog configuration: ```ini title="spark-defaults.conf" spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions spark.sql.catalog.demo org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.demo.type rest spark.sql.catalog.demo.uri http://rest:8181 spark.sql.catalog.demo.io-impl org.apache.iceberg.aws.s3.S3FileIO spark.sql.catalog.demo.warehouse s3://my-bucket/warehouse spark.sql.catalog.demo.s3.endpoint http://rustfs:9000 spark.sql.catalog.demo.s3.path-style-access true spark.sql.defaultCatalog demo spark.sql.catalogImplementation in-memory ``` Path-style access is required for this container-network endpoint. The hostname `rustfs` is resolvable only inside the Compose network; clients running on the host use `http://localhost:9000` instead. Create the Compose file: ```yaml title="compose.yaml" services: rustfs: image: rustfs/rustfs:1.0.0-alpha.83 environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_VOLUMES: /data RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ volumes: - rustfs-data:/data ports: - "9000:9000" - "9001:9001" networks: - iceberg create-bucket: image: rustfs/rc:latest depends_on: - rustfs environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} entrypoint: - /bin/sh - -c - | until /usr/bin/rc alias set rustfs http://rustfs:9000 "$${RUSTFS_ACCESS_KEY}" "$${RUSTFS_SECRET_KEY}"; do echo "Waiting for RustFS..." sleep 2 done /usr/bin/rc ls rustfs/my-bucket >/dev/null 2>&1 || /usr/bin/rc mb rustfs/my-bucket networks: - iceberg rest: image: apache/iceberg-rest-fixture depends_on: create-bucket: condition: service_completed_successfully environment: AWS_ACCESS_KEY_ID: ${RUSTFS_ACCESS_KEY} AWS_SECRET_ACCESS_KEY: ${RUSTFS_SECRET_KEY} AWS_REGION: us-east-1 CATALOG_WAREHOUSE: s3://my-bucket/warehouse CATALOG_IO__IMPL: org.apache.iceberg.aws.s3.S3FileIO CATALOG_S3_ENDPOINT: http://rustfs:9000 CATALOG_S3_PATH__STYLE__ACCESS: "true" ports: - "8181:8181" networks: - iceberg spark-iceberg: image: tabulario/spark-iceberg depends_on: create-bucket: condition: service_completed_successfully rest: condition: service_started environment: AWS_ACCESS_KEY_ID: ${RUSTFS_ACCESS_KEY} AWS_SECRET_ACCESS_KEY: ${RUSTFS_SECRET_KEY} AWS_REGION: us-east-1 volumes: - ./spark-defaults.conf:/opt/spark/conf/spark-defaults.conf:ro ports: - "8888:8888" - "8080:8080" networks: - iceberg networks: iceberg: volumes: rustfs-data: ``` The [`rc` image](https://github.com/rustfs/cli) provides the official RustFS command-line client. The initializer checks for `my-bucket` before creating it, so repeated starts do not delete existing warehouse data. The RustFS volume preserves warehouse objects across container recreation. The Apache Iceberg quickstart images are published without stable version tags in the upstream example. Before using this pattern beyond local testing, pin every image to a tested tag or digest and validate the Spark, Iceberg runtime, and REST catalog versions together.
## Validate and start the deployment [#2-validate-and-start-the-deployment] Resolve the Compose file before starting containers: ```bash docker compose config ``` Start the services and wait for the bucket initializer to finish: ```bash docker compose up -d docker compose ps -a ``` The `create-bucket` service should show an exit code of `0`. Check its logs if it does not complete: ```bash docker compose logs create-bucket ``` Open the RustFS Console at `http://localhost:9001`. The REST catalog is available at `http://localhost:8181`, and the Spark notebook server is available at `http://localhost:8888`.
## Create and query an Iceberg table [#3-create-and-query-an-iceberg-table] Start Spark SQL: ```bash docker compose exec spark-iceberg spark-sql ``` Create a namespace and a partitioned table: ```sql CREATE NAMESPACE IF NOT EXISTS demo.nyc; CREATE TABLE demo.nyc.taxis ( vendor_id bigint, trip_id bigint, trip_distance float, fare_amount double, store_and_fwd_flag string ) PARTITIONED BY (vendor_id); ``` Insert and query sample rows: ```sql INSERT INTO demo.nyc.taxis VALUES (1, 1000371, 1.8, 15.32, 'N'), (2, 1000372, 2.5, 22.15, 'N'), (2, 1000373, 0.9, 9.01, 'N'), (1, 1000374, 8.4, 42.13, 'Y'); SELECT * FROM demo.nyc.taxis ORDER BY trip_id; ``` The query should return four rows: ```text 1 1000371 1.8 15.32 N 2 1000372 2.5 22.15 N 2 1000373 0.9 9.01 N 1 1000374 8.4 42.13 Y ```
## Verify objects in RustFS [#4-verify-objects-in-rustfs] List the warehouse from the bucket-initializer image: ```bash docker compose run --rm --entrypoint /bin/sh create-bucket -c \ '/usr/bin/rc alias set rustfs http://rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" >/dev/null && /usr/bin/rc find rustfs/my-bucket/warehouse' ``` The output should include Iceberg metadata and data objects below the `warehouse/nyc/taxis` prefix. You can also inspect the `my-bucket` bucket in the RustFS Console.
## Stop or reset the stack [#5-stop-or-reset-the-stack] Stop the containers while keeping the RustFS data volume: ```bash docker compose down ``` To delete the local warehouse and start from an empty RustFS volume, explicitly include `--volumes`: ```bash docker compose down --volumes ```
## Troubleshooting [#troubleshooting] ### Spark cannot reach RustFS [#spark-cannot-reach-rustfs] Use `http://rustfs:9000` inside Compose. `http://localhost:9000` refers to the Spark container itself when used in `spark-defaults.conf`. Confirm that `spark.sql.catalog.demo.s3.path-style-access` is `true`. Virtual-hosted requests require additional RustFS domain and DNS configuration. ### The catalog returns an S3 error [#the-catalog-returns-an-s3-error] Check that the credentials in `.env` match the RustFS credentials and that the `create-bucket` service completed successfully: ```bash docker compose logs create-bucket rest ``` The REST catalog property uses doubled underscores in `CATALOG_IO__IMPL` and `CATALOG_S3_PATH__STYLE__ACCESS`; the fixture converts them to the dotted and hyphenated Iceberg property names. ## Next steps [#next-steps] * Review [S3 compatibility notes](/administration/protocols/s3) before adopting additional Iceberg operations. * Create dedicated production credentials with [Access Key Management](/security-compliance/iam/access-token). * Follow the [Apache Iceberg Spark documentation](https://iceberg.apache.org/docs/latest/spark-getting-started/) to configure your existing Spark environment. # Big Data (/en/developer/integration/big-data) Use **RustFS** as the object storage layer for big data systems that support an S3-compatible endpoint. ## Systems [#systems] * [Iceberg](./iceberg.md) * [Milvus](./milvus.md) Keep application data in a dedicated bucket and prefix, and use credentials scoped to the required bucket operations. # Milvus (/en/developer/integration/big-data/milvus) This guide runs **Milvus Standalone** with **RustFS** as its S3-compatible object storage backend. You will start Milvus, etcd, RustFS, and Attu; insert sample vectors; and verify that Milvus persists objects in RustFS. You need Docker with the Compose plugin and Python 3.9 or later. This deployment is intended for local integration testing, not production. Milvus groups S3-compatible storage settings under the `minio` configuration key. The name does not require a MinIO server. In this guide, `minio.address` points to the RustFS service, and Milvus uses the S3 API exposed by RustFS. ## Architecture [#architecture] Milvus stores service metadata in etcd and persists vector data, indexes, and related objects under `s3://my-bucket/milvus` in RustFS. The local Milvus volume remains necessary for runtime data and caches.
## Create the project files [#1-create-the-project-files] Create a working directory: ```bash mkdir rustfs-milvus cd rustfs-milvus ``` Create an environment file and replace both credential placeholders: ```ini title=".env" RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Use dedicated credentials for the Milvus bucket. Do not commit `.env` to source control. Create the Milvus storage override: ```yaml title="user.yaml" common: storageType: remote minio: address: rustfs:9000 port: 9000 bucketName: my-bucket rootPath: milvus useSSL: false useIAM: false cloudProvider: aws region: us-east-1 useVirtualHost: false ``` `useVirtualHost: false` selects path-style S3 requests. The hostname `rustfs` resolves inside the Compose network; clients on the host use `http://localhost:9000`. Create the Compose file: ```yaml title="compose.yaml" services: etcd: image: quay.io/coreos/etcd:v3.5.18 environment: ETCD_AUTO_COMPACTION_MODE: revision ETCD_AUTO_COMPACTION_RETENTION: "1000" ETCD_QUOTA_BACKEND_BYTES: "4294967296" ETCD_SNAPSHOT_COUNT: "50000" command: - etcd - --advertise-client-urls=http://etcd:2379 - --listen-client-urls=http://0.0.0.0:2379 - --data-dir=/etcd volumes: - etcd-data:/etcd healthcheck: test: ["CMD", "etcdctl", "endpoint", "health"] interval: 30s timeout: 20s retries: 3 networks: - milvus rustfs: image: rustfs/rustfs:1.0.0-alpha.83 environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_VOLUMES: /data RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ volumes: - rustfs-data:/data ports: - "9000:9000" - "9001:9001" healthcheck: test: ["CMD-SHELL", "curl -fsS http://localhost:9000/health/ready"] interval: 10s timeout: 5s retries: 12 start_period: 20s networks: - milvus create-bucket: image: rustfs/rc:latest depends_on: rustfs: condition: service_healthy environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} entrypoint: - /bin/sh - -c - | /usr/bin/rc alias set rustfs http://rustfs:9000 "$${RUSTFS_ACCESS_KEY}" "$${RUSTFS_SECRET_KEY}" \ --region us-east-1 --bucket-lookup path /usr/bin/rc bucket create rustfs/my-bucket --ignore-existing networks: - milvus standalone: image: milvusdb/milvus:v2.6.0 command: ["milvus", "run", "standalone"] security_opt: - seccomp:unconfined depends_on: etcd: condition: service_healthy create-bucket: condition: service_completed_successfully environment: ETCD_ENDPOINTS: etcd:2379 MINIO_ADDRESS: rustfs:9000 MINIO_ACCESS_KEY_ID: ${RUSTFS_ACCESS_KEY} MINIO_SECRET_ACCESS_KEY: ${RUSTFS_SECRET_KEY} MINIO_REGION: us-east-1 MQ_TYPE: woodpecker volumes: - milvus-data:/var/lib/milvus - ./user.yaml:/milvus/configs/user.yaml:ro ports: - "19530:19530" - "9091:9091" healthcheck: test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"] interval: 30s timeout: 20s retries: 3 start_period: 90s networks: - milvus attu: image: zilliz/attu:v2.6.5 depends_on: standalone: condition: service_healthy environment: MILVUS_URL: standalone:19530 ports: - "8000:3000" networks: - milvus networks: milvus: volumes: etcd-data: rustfs-data: milvus-data: ``` The `create-bucket` service uses the official [`rc`](https://github.com/rustfs/cli) image and exits after ensuring that `my-bucket` exists. Named volumes preserve etcd metadata, RustFS objects, and Milvus local data when containers are recreated. The Compose file publishes the RustFS API and Console, Milvus gRPC and health ports, and Attu on the host for local testing. Do not expose these ports to an untrusted network. Production deployments require scoped credentials, TLS, authentication, resource planning, backups, and independently operated dependencies.
## Validate and start the deployment [#2-validate-and-start-the-deployment] Resolve the Compose file before starting containers: ```bash docker compose config ``` Start the services: ```bash docker compose up -d docker compose ps -a ``` The `create-bucket` service should exit with code `0`, and `etcd`, `rustfs`, and `standalone` should become healthy. Inspect logs if a service does not reach its expected state: ```bash docker compose logs create-bucket rustfs standalone ``` Open these local interfaces: * RustFS Console: `http://localhost:9001` * Attu: `http://localhost:8000` * Milvus health endpoint: `http://localhost:9091/healthz` Attu connects to `standalone:19530` through the Compose network. If Attu asks for a connection address, use that service name instead of `localhost:19530`.
## Insert and query sample vectors [#3-insert-and-query-sample-vectors] Create a Python virtual environment and install the Milvus client version that matches the server release: ```bash python3 -m venv .venv source .venv/bin/activate python -m pip install "pymilvus==2.6.0" ``` Create a test script: ```python title="verify_milvus.py" from pymilvus import MilvusClient client = MilvusClient(uri="http://localhost:19530") collection_name = "rustfs_demo" if client.has_collection(collection_name=collection_name): client.drop_collection(collection_name=collection_name) client.create_collection( collection_name=collection_name, dimension=4, ) client.insert( collection_name=collection_name, data=[ {"id": 1, "vector": [0.1, 0.2, 0.3, 0.4]}, {"id": 2, "vector": [0.2, 0.3, 0.4, 0.5]}, {"id": 3, "vector": [0.9, 0.8, 0.7, 0.6]}, ], ) client.flush(collection_name=collection_name) results = client.search( collection_name=collection_name, data=[[0.1, 0.2, 0.3, 0.4]], limit=2, output_fields=["id"], ) print(results) client.close() ``` Run the script: ```bash python verify_milvus.py ``` The result should rank the row with ID `1` first. Open Attu and confirm that the `rustfs_demo` collection contains three entities.
## Verify Milvus objects in RustFS [#4-verify-milvus-objects-in-rustfs] Use the bucket-initializer image to list objects below the configured `milvus` root path: ```bash docker compose run --rm --entrypoint /bin/sh create-bucket -c \ '/usr/bin/rc alias set rustfs http://rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" --region us-east-1 --bucket-lookup path >/dev/null && /usr/bin/rc find rustfs/my-bucket/milvus' ``` The output should contain objects created by Milvus below the `milvus/` prefix. You can also open `my-bucket` in the RustFS Console. Milvus may buffer or compact data before every expected object appears. The successful insert, flush, query, and RustFS object listing together validate the integration path.
## Stop or reset the stack [#5-stop-or-reset-the-stack] Stop the containers while retaining all named volumes: ```bash docker compose down ``` To delete the local test data, including the Milvus bucket contents and etcd metadata, explicitly remove the volumes: ```bash docker compose down --volumes ``` The `--volumes` option permanently deletes the named volumes used by this Compose project. Do not run it against data you need to retain.
## Troubleshooting [#troubleshooting] ### Milvus cannot reach RustFS [#milvus-cannot-reach-rustfs] Use `rustfs:9000` as the S3 endpoint inside Compose. `localhost:9000` inside the Milvus container refers to that container, not RustFS. Confirm that `useVirtualHost` remains `false` and that the credential values passed to Milvus match the RustFS credentials: ```bash docker compose logs standalone rustfs ``` ### The bucket initializer fails [#the-bucket-initializer-fails] Check RustFS readiness and the initializer logs: ```bash curl -fsS http://localhost:9000/health/ready docker compose logs create-bucket ``` Verify that `.env` contains non-empty credentials and that `docker compose config` resolves both variables. ### Milvus starts without existing data [#milvus-starts-without-existing-data] Do not change `minio.bucketName`, `minio.rootPath`, or the etcd root for an existing deployment. Confirm that the `rustfs-data`, `etcd-data`, and `milvus-data` volumes still exist and that the same Compose project name is in use. ### Attu cannot connect [#attu-cannot-connect] The Attu container must use `standalone:19530`. A browser or host-side client uses `localhost:19530`. Check Milvus health and Attu logs: ```bash curl -fsS http://localhost:9091/healthz docker compose logs attu standalone ``` ## Next steps [#next-steps] * Review [S3 compatibility notes](/administration/protocols/s3) before enabling additional Milvus storage features. * Create dedicated production credentials with [Access Key Management](/security-compliance/iam/access-token). * Follow the [Milvus documentation](https://milvus.io/docs) when adapting this local pattern to a managed or distributed deployment. # PyIceberg (/en/developer/integration/big-data/pyiceberg) Use **PyIceberg** to create a namespace and table in the RustFS S3 Tables catalog, append two rows, and verify the data after reloading the table. This walkthrough uses PyIceberg `0.10.0` and Python `3.12` with explicitly configured access credentials. ## Before you begin [#before-you-begin] * Complete [S3 Tables setup](/administration/data/s3-tables), including creating and enabling `my-bucket` and meeting the account and TLS requirements. * Keep `RUSTFS_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` set as in that guide.
## Install the client [#1-install-the-client] Create a directory and an isolated Python environment: ```bash mkdir rustfs-s3-tables cd rustfs-s3-tables python3.12 -m venv .venv source .venv/bin/activate python -m pip install 'pyiceberg[pyarrow]==0.10.0' boto3 ```
## Configure the catalog connection [#2-configure-the-catalog-connection] Save the following connection module. It signs both the initial catalog discovery request and subsequent REST requests using the S3 SigV4 signing behavior used by RustFS's [verified client example](https://github.com/rustfs/rustfs/blob/7e0c67111b97703d47e23719b0264a739c8acea8/scripts/table-catalog/pyiceberg_smoke.py). ```python title="rustfs_catalog.py" import hashlib import os from botocore.auth import S3SigV4Auth from botocore.awsrequest import AWSRequest from botocore.credentials import Credentials from pyiceberg.catalog.rest import RestCatalog from requests.adapters import HTTPAdapter endpoint = os.environ["RUSTFS_ENDPOINT"].rstrip("/") region = os.environ["AWS_DEFAULT_REGION"] access_key = os.environ["AWS_ACCESS_KEY_ID"] secret_key = os.environ["AWS_SECRET_ACCESS_KEY"] credentials = Credentials(access_key, secret_key) class RustFSSigV4Adapter(HTTPAdapter): def add_headers(self, request, **kwargs): body = request.body or b"" if isinstance(body, str): body = body.encode("utf-8") request.headers["x-amz-content-sha256"] = hashlib.sha256(body).hexdigest() request.headers.pop("connection", None) signed = AWSRequest( method=request.method, url=request.url, data=body, headers=dict(request.headers), ) S3SigV4Auth(credentials, "s3", region).add_auth(signed) request.headers.update(signed.headers) class RustFSRestCatalog(RestCatalog): def _init_sigv4(self, session): session.mount(self.uri, RustFSSigV4Adapter()) catalog = RustFSRestCatalog( "rustfs", **{ "uri": f"{endpoint}/iceberg", "warehouse": "my-bucket", "prefix": "my-bucket", "rest.sigv4-enabled": "true", "rest.signing-name": "s3", "rest.signing-region": region, "py-io-impl": "pyiceberg.io.pyarrow.PyArrowFileIO", "s3.endpoint": endpoint, "s3.access-key-id": access_key, "s3.secret-access-key": secret_key, "s3.region": region, "s3.force-virtual-addressing": "false", }, ) ``` `s3.force-virtual-addressing=false` selects path-style access for this custom endpoint in PyIceberg's PyArrow file implementation. The adapter overrides PyIceberg's `_init_sigv4` hook so that discovery is signed before the catalog constructor finishes. Keep the pinned PyIceberg version when using this module, and rerun the full walkthrough before changing it.
## Create and read a table [#3-create-and-read-a-table] The example creates namespace `analytics` and table `events` and stops if either already exists. Each namespace segment and table name must contain 1–64 ASCII characters: lowercase letters, digits, `_`, or `-`, with a letter or digit at each end. The full namespace, including dots, is limited to 512 characters. For other names, update `identifier` in `example.py` and the inspection and removal commands below. Save the following program in the same directory: ```python title="example.py" import json import pyarrow as pa from rustfs_catalog import catalog identifier = ("analytics", "events") schema = pa.schema( [ pa.field("id", pa.int64(), nullable=False), pa.field("payload", pa.string(), nullable=False), ] ) expected = [{"id": 1, "payload": "alpha"}, {"id": 2, "payload": "beta"}] catalog.create_namespace(identifier[0]) catalog.create_table(identifier, schema=schema) table = catalog.load_table(identifier) table.append(pa.Table.from_pylist(expected, schema=schema)) loaded = catalog.load_table(identifier) actual = sorted(loaded.scan().to_arrow().to_pylist(), key=lambda row: row["id"]) assert actual == expected, f"Unexpected table contents: {actual}" print("rows:", json.dumps(actual)) print("metadata:", loaded.metadata_location) ``` Run it: ```bash python example.py ``` The output includes the two complete rows and the current metadata object's S3 URI: ```text rows: [{"id": 1, "payload": "alpha"}, {"id": 2, "payload": "beta"}] metadata: s3://my-bucket/ ``` The generated metadata object key varies. Successful verification means the table was reloaded from the catalog and its data files were read through S3; table creation alone does not verify either result.
## Inspect or unregister the example [#4-inspect-or-unregister-the-example] List the table from a new Python process using the same connection module: ```bash python - <<'PY' from rustfs_catalog import catalog print(catalog.list_tables("analytics")) PY ``` The result should contain `("analytics", "events")`. The following commands remove this tutorial’s table entry and its now-empty namespace. The bucket and underlying objects remain. Plan any data cleanup before proceeding: after `drop_table`, table maintenance can no longer find the table. See [maintenance and data protection](/administration/data/s3-tables). ```bash python - <<'PY' from rustfs_catalog import catalog catalog.drop_table(("analytics", "events")) catalog.drop_namespace("analytics") PY ```
## Next steps [#next-steps] * Follow the [PyIceberg API documentation](https://py.iceberg.apache.org/api/) for client operations, checking each operation against RustFS's supported scope. * Use the [external Iceberg catalog integration](/developer/integration/big-data/iceberg) if you manage a separate catalog service. # Integration (/en/developer/integration) Use this section to connect **RustFS** to infrastructure and application platforms through its S3-compatible API. ## Integration categories [#integration-categories] * [Reverse Proxy](./reverse-proxy/index.md) covers Nginx, Traefik, Caddy, and HAProxy. * [Backup](./backup/index.md) covers Restic. * [Big Data](./big-data/index.md) covers Iceberg. Each guide identifies the RustFS endpoint and addressing requirements to use when configuring the integrating system. # Caddy (/en/developer/integration/reverse-proxy/caddy) Use **Caddy** to obtain and renew TLS certificates automatically and route separate hostnames to the RustFS S3 API and Console. This deployment runs Caddy and a single-node RustFS instance with Docker Compose. You need Docker Engine, Docker Compose, a public server, and two DNS records that resolve to that server. This guide uses these example hostnames: * `s3.example.com` for the S3 API * `console.example.com` for the Console Replace them with your public hostnames. Caddy's default ACME challenges require inbound access to ports `80` and `443`. Do not publish the S3 API under a path such as `/s3/`. AWS Signature Version 4 includes the request path and host, so rewriting either value can invalidate signed requests.
## Create the deployment directory [#1-create-the-deployment-directory] Create a directory for the deployment: ```bash mkdir rustfs-caddy cd rustfs-caddy ```
## Set deployment variables [#2-set-deployment-variables] Create an environment file and replace each value: ```ini title=".env" S3_HOSTNAME=s3.example.com CONSOLE_HOSTNAME=console.example.com ACME_EMAIL=admin@example.com RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Use an email address that receives certificate notices. Do not commit `.env` to source control.
## Configure Caddy [#3-configure-caddy] Create a Caddyfile with one site block for each RustFS endpoint: ```text title="Caddyfile" { email {$ACME_EMAIL} } {$S3_HOSTNAME} { reverse_proxy rustfs:9000 { health_uri /health/ready health_interval 10s health_timeout 5s health_fails 3 health_passes 2 lb_try_duration 5s } } {$CONSOLE_HOSTNAME} { reverse_proxy rustfs:9001 { health_uri /rustfs/console/health health_interval 10s health_timeout 5s health_fails 3 health_passes 2 lb_try_duration 5s } } ``` Caddy preserves the incoming `Host` header, HTTP method, and request URI by default. It also forwards client information through `X-Forwarded-*` headers and handles Console WebSocket upgrades without additional header rules.
## Create the Compose file [#4-create-the-compose-file] Create the Caddy and RustFS services: ```yaml title="compose.yaml" services: caddy: image: caddy:2.10-alpine restart: unless-stopped depends_on: rustfs: condition: service_healthy environment: S3_HOSTNAME: ${S3_HOSTNAME} CONSOLE_HOSTNAME: ${CONSOLE_HOSTNAME} ACME_EMAIL: ${ACME_EMAIL} ports: - "80:80" - "443:443" - "443:443/udp" volumes: - ./Caddyfile:/etc/caddy/Caddyfile:ro - caddy-data:/data - caddy-config:/config networks: - rustfs rustfs: image: rustfs/rustfs:latest restart: unless-stopped environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ expose: - "9000" - "9001" volumes: - rustfs-data:/data healthcheck: test: ["CMD-SHELL", "curl --fail http://127.0.0.1:9000/health/ready && curl --fail http://127.0.0.1:9001/rustfs/console/health"] interval: 10s timeout: 5s retries: 5 start_period: 30s networks: - rustfs volumes: caddy-data: caddy-config: rustfs-data: networks: rustfs: ``` The persistent `caddy-data` volume stores certificates, private keys, and ACME account state. Back up this volume and do not share its contents. Only Caddy publishes host ports; RustFS remains reachable inside the Compose network.
## Validate and start the deployment [#5-validate-and-start-the-deployment] Render the Compose configuration and start RustFS: ```bash docker compose config docker compose up -d rustfs ``` Validate the Caddyfile with the same image used by the deployment: ```bash docker compose run --rm --no-deps caddy caddy validate --config /etc/caddy/Caddyfile ``` Start Caddy and check both services: ```bash docker compose up -d caddy docker compose ps docker compose logs --follow caddy ``` Caddy obtains certificates in the background and redirects HTTP requests to HTTPS. If issuance fails, confirm that both DNS records resolve to this host, ports `80` and `443` are reachable, and the `caddy-data` volume is writable.
## Verify both endpoints [#6-verify-both-endpoints] Verify the API and Console through their public HTTPS hostnames: ```bash curl --fail https://s3.example.com/health/ready curl --fail https://console.example.com/rustfs/console/health ``` Configure S3 clients with `https://s3.example.com` as the endpoint and enable path-style addressing. Open `https://console.example.com` to sign in to the Console.
## Multi-node upstreams [#multi-node-upstreams] For a distributed RustFS deployment, list every node in the corresponding site block: ```text title="Caddyfile" {$S3_HOSTNAME} { reverse_proxy node1.example.net:9000 node2.example.net:9000 node3.example.net:9000 node4.example.net:9000 { lb_policy least_conn health_uri /health/ready health_interval 10s health_timeout 5s lb_try_duration 5s } } {$CONSOLE_HOSTNAME} { reverse_proxy node1.example.net:9001 node2.example.net:9001 node3.example.net:9001 node4.example.net:9001 { lb_policy cookie rustfs_console health_uri /rustfs/console/health health_interval 10s health_timeout 5s lb_try_duration 5s } } ``` Replace `` with a random secret shared by all Caddy instances. Console affinity keeps an in-progress OpenID Connect login on the RustFS node that created its `state`. Keep port `9000` open directly between RustFS nodes because internal node RPC uses the same listener. ## Next steps [#next-steps] * [Configure an S3 client](/developer/examples/aws-cli) * [Enable virtual-hosted-style bucket URLs](/integration/virtual) * [Review health and readiness endpoints](/operations/status-check) # HAProxy (/en/developer/integration/reverse-proxy/haproxy) Use **HAProxy** to terminate TLS and route separate hostnames to the RustFS S3 API and Console. This deployment runs HAProxy and a single-node RustFS instance with Docker Compose. You need Docker Engine, Docker Compose, two DNS records, and a TLS certificate that covers both hostnames. This guide uses these example hostnames: * `s3.example.com` for the S3 API * `console.example.com` for the Console Replace them with hostnames that resolve to the Docker host. Do not publish the S3 API under a path such as `/s3/`. AWS Signature Version 4 includes the request path and host, so rewriting either value can invalidate signed requests.
## Create the deployment directories [#1-create-the-deployment-directories] Create directories for the HAProxy configuration and TLS certificate: ```bash mkdir -p rustfs-haproxy/config rustfs-haproxy/certs cd rustfs-haproxy ``` HAProxy expects the certificate chain and private key in one PEM file. Combine them in this order: ```bash cat fullchain.pem privkey.pem > certs/rustfs.pem chmod 600 certs/rustfs.pem ``` The certificate must cover both public hostnames.
## Set RustFS credentials [#2-set-rustfs-credentials] Create an environment file and replace both credential placeholders: ```ini title=".env" RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Do not commit this file or the certificate private key to source control.
## Configure HAProxy [#3-configure-haproxy] Create the HAProxy configuration: ```text title="config/haproxy.cfg" global log stdout format raw local0 defaults log global mode http option httplog timeout connect 10s timeout client 1h timeout server 1h timeout http-request 30s timeout tunnel 1h frontend http bind :80 http-request redirect scheme https code 301 frontend https bind :443 ssl crt /usr/local/etc/haproxy/certs/rustfs.pem alpn h2,http/1.1 acl host_s3 hdr(host) -i s3.example.com acl host_console hdr(host) -i console.example.com use_backend rustfs_s3 if host_s3 use_backend rustfs_console if host_console default_backend reject_unknown_host backend reject_unknown_host http-request deny deny_status 404 backend rustfs_s3 balance leastconn option httpchk GET /health/ready http-check expect status 200 server rustfs rustfs:9000 check inter 10s fall 3 rise 2 backend rustfs_console balance leastconn cookie RUSTFS_CONSOLE insert indirect nocache secure httponly option httpchk GET /rustfs/console/health http-check expect status 200 server rustfs rustfs:9001 check inter 10s fall 3 rise 2 cookie rustfs ``` HAProxy preserves the incoming host and request path unless you explicitly rewrite them. The long client, server, and tunnel timeouts accommodate streaming S3 operations and Console WebSocket connections. The Console backend sets an affinity cookie. With one RustFS server it has no routing effect, but keeping it in the base configuration makes the behavior consistent when you add nodes.
## Create the Compose file [#4-create-the-compose-file] Create the HAProxy and RustFS services: ```yaml title="compose.yaml" services: haproxy: image: haproxy:3.2-alpine restart: unless-stopped depends_on: rustfs: condition: service_healthy ports: - "80:80" - "443:443" volumes: - ./config/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro - ./certs:/usr/local/etc/haproxy/certs:ro networks: - rustfs rustfs: image: rustfs/rustfs:latest restart: unless-stopped environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ expose: - "9000" - "9001" volumes: - rustfs-data:/data healthcheck: test: ["CMD-SHELL", "curl --fail http://127.0.0.1:9000/health/ready && curl --fail http://127.0.0.1:9001/rustfs/console/health"] interval: 10s timeout: 5s retries: 5 start_period: 30s networks: - rustfs volumes: rustfs-data: networks: rustfs: ``` Only HAProxy publishes host ports. RustFS ports `9000` and `9001` remain reachable inside the Compose network.
## Validate and start the deployment [#5-validate-and-start-the-deployment] Render the Compose configuration and start RustFS: ```bash docker compose config docker compose up -d rustfs ``` Validate the HAProxy configuration with the same image used by the deployment: ```bash docker compose run --rm --no-deps haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg ``` Start HAProxy and check both services: ```bash docker compose up -d haproxy docker compose ps ``` If a service does not become healthy, inspect its logs: ```bash docker compose logs haproxy docker compose logs rustfs ```
## Verify both endpoints [#6-verify-both-endpoints] Verify the API and Console through their public HTTPS hostnames: ```bash curl --fail https://s3.example.com/health/ready curl --fail https://console.example.com/rustfs/console/health ``` Configure S3 clients with `https://s3.example.com` as the endpoint and enable path-style addressing. Open `https://console.example.com` to sign in to the Console. When you replace a renewed `certs/rustfs.pem`, validate the configuration and recreate the HAProxy container to load it: ```bash docker compose run --rm --no-deps haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg docker compose up -d --force-recreate haproxy ```
## Multi-node backends [#multi-node-backends] For a distributed RustFS deployment, add every RustFS node to both backends: ```text title="config/haproxy.cfg" backend rustfs_s3 balance leastconn option httpchk GET /health/ready http-check expect status 200 server node1 node1.example.net:9000 check inter 10s fall 3 rise 2 server node2 node2.example.net:9000 check inter 10s fall 3 rise 2 server node3 node3.example.net:9000 check inter 10s fall 3 rise 2 server node4 node4.example.net:9000 check inter 10s fall 3 rise 2 backend rustfs_console balance leastconn cookie RUSTFS_CONSOLE insert indirect nocache secure httponly option httpchk GET /rustfs/console/health http-check expect status 200 server node1 node1.example.net:9001 check inter 10s fall 3 rise 2 cookie node1 server node2 node2.example.net:9001 check inter 10s fall 3 rise 2 cookie node2 server node3 node3.example.net:9001 check inter 10s fall 3 rise 2 cookie node3 server node4 node4.example.net:9001 check inter 10s fall 3 rise 2 cookie node4 ``` The Console cookie keeps an in-progress OpenID Connect login on the RustFS node that created its `state`. Keep port `9000` open directly between RustFS nodes because internal node RPC uses the same listener. ## Next steps [#next-steps] * [Configure an S3 client](/developer/examples/aws-cli) * [Enable virtual-hosted-style bucket URLs](/integration/virtual) * [Review health and readiness endpoints](/operations/status-check) # Reverse Proxy (/en/developer/integration/reverse-proxy) Use a reverse proxy to expose the **RustFS** S3 API and Console through managed hostnames and TLS endpoints. We recommend using separate hostnames for the S3 API on port `9000` and the Console on port `9001`. Serve the S3 API from the root of its hostname because S3 clients sign the request path. ## Supported guides [#supported-guides] * [Nginx](./nginx.md) * [Traefik](./traefik.md) * [Caddy](./caddy.md) * [HAProxy](./haproxy.md) ## Related configuration [#related-configuration] See [Virtual-Host Access](/integration/virtual) when clients access buckets through virtual-hosted-style URLs. # Nginx (/en/developer/integration/reverse-proxy/nginx) Use **Nginx** to terminate TLS and route separate hostnames to the RustFS S3 API and Console. This deployment runs Nginx and a single-node RustFS instance with Docker Compose. You need Docker Engine, Docker Compose, two DNS records, and a TLS certificate that covers both hostnames. This guide uses these example hostnames: * `s3.example.com` for the S3 API * `console.example.com` for the Console Replace them with hostnames that resolve to the Docker host. Do not publish the S3 API under a path such as `/s3/`. AWS Signature Version 4 includes the request path and host, so rewriting either value can invalidate signed requests.
## Create the deployment directories [#1-create-the-deployment-directories] Create directories for the Nginx configuration and TLS certificate: ```bash mkdir -p rustfs-nginx/sites rustfs-nginx/certs cd rustfs-nginx ``` Copy your certificate chain and private key into `certs/`: ```text rustfs-nginx/ ├── certs/ │ ├── fullchain.pem │ └── privkey.pem └── sites/ ``` Restrict access to the private key: ```bash chmod 600 certs/privkey.pem ```
## Set RustFS credentials [#2-set-rustfs-credentials] Create an environment file and replace both credential placeholders: ```ini title=".env" RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Do not commit this file to source control.
## Configure Nginx [#3-configure-nginx] Create the Nginx configuration: ```nginx title="sites/rustfs.conf" map $http_upgrade $connection_upgrade { default upgrade; '' ''; } upstream rustfs_s3 { server rustfs:9000; keepalive 32; } upstream rustfs_console { server rustfs:9001; keepalive 16; } server { listen 80; listen [::]:80; server_name s3.example.com console.example.com; return 301 https://$host$request_uri; } server { listen 443 ssl; listen [::]:443 ssl; http2 on; server_name s3.example.com; ssl_certificate /etc/nginx/certs/fullchain.pem; ssl_certificate_key /etc/nginx/certs/privkey.pem; ignore_invalid_headers off; client_max_body_size 0; proxy_buffering off; proxy_request_buffering off; location / { proxy_http_version 1.1; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Connection ''; proxy_cache_convert_head off; proxy_connect_timeout 300s; chunked_transfer_encoding off; proxy_pass http://rustfs_s3; } } server { listen 443 ssl; listen [::]:443 ssl; http2 on; server_name console.example.com; ssl_certificate /etc/nginx/certs/fullchain.pem; ssl_certificate_key /etc/nginx/certs/privkey.pem; client_max_body_size 0; proxy_buffering off; proxy_request_buffering off; location / { proxy_http_version 1.1; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_connect_timeout 300s; proxy_pass http://rustfs_console; } } ``` The S3 server preserves the original host and request path, disables request buffering for streaming uploads, and does not convert signed `HEAD` requests. The Console server also forwards WebSocket upgrade headers.
## Create the Compose file [#4-create-the-compose-file] Create the deployment definition: ```yaml title="compose.yaml" services: nginx: image: nginx:1.25-alpine restart: unless-stopped depends_on: rustfs: condition: service_healthy ports: - "80:80" - "443:443" volumes: - ./sites/rustfs.conf:/etc/nginx/conf.d/rustfs.conf:ro - ./certs:/etc/nginx/certs:ro networks: - rustfs rustfs: image: rustfs/rustfs:latest restart: unless-stopped environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ expose: - "9000" - "9001" volumes: - rustfs-data:/data healthcheck: test: ["CMD-SHELL", "curl --fail http://127.0.0.1:9000/health/ready && curl --fail http://127.0.0.1:9001/rustfs/console/health"] interval: 10s timeout: 5s retries: 5 start_period: 30s networks: - rustfs volumes: rustfs-data: networks: rustfs: ``` Only Nginx publishes host ports. RustFS ports `9000` and `9001` remain reachable inside the Compose network.
## Validate and start the deployment [#5-validate-and-start-the-deployment] Validate both configuration files before starting the services: ```bash docker compose config docker compose up -d rustfs docker compose run --rm --no-deps nginx nginx -t ``` Start Nginx and check both services: ```bash docker compose up -d nginx docker compose ps ``` If a service does not become healthy, inspect its logs: ```bash docker compose logs nginx docker compose logs rustfs ```
## Verify both endpoints [#6-verify-both-endpoints] Verify the API and Console through their public HTTPS hostnames: ```bash curl --fail https://s3.example.com/health/ready curl --fail https://console.example.com/rustfs/console/health ``` Configure S3 clients with `https://s3.example.com` as the endpoint and enable path-style addressing. Open `https://console.example.com` to sign in to the Console. When you replace a renewed certificate or key in `certs/`, validate and reload Nginx without interrupting active connections: ```bash docker compose exec nginx nginx -t docker compose exec nginx nginx -s reload ```
## Multi-node upstreams [#multi-node-upstreams] For a distributed RustFS deployment, replace the single server in each upstream with all RustFS nodes: ```nginx title="sites/rustfs.conf" upstream rustfs_s3 { least_conn; server node1.example.net:9000; server node2.example.net:9000; server node3.example.net:9000; server node4.example.net:9000; keepalive 32; } upstream rustfs_console { ip_hash; server node1.example.net:9001; server node2.example.net:9001; server node3.example.net:9001; server node4.example.net:9001; keepalive 16; } ``` The Console upstream uses client affinity because an in-progress OpenID Connect login stores its `state` on one RustFS node. Keep port `9000` open between RustFS nodes because internal node RPC uses the same listener. ## Next steps [#next-steps] * [Configure an S3 client](/developer/examples/aws-cli) * [Enable virtual-hosted-style bucket URLs](/integration/virtual) * [Review health and readiness endpoints](/operations/status-check) # Traefik (/en/developer/integration/reverse-proxy/traefik) Use **Traefik** and its Docker provider to discover RustFS, obtain TLS certificates from Let's Encrypt, and route separate hostnames to the S3 API and Console. You need Docker Engine, Docker Compose, a public server, and two DNS records that resolve to that server. This guide uses these example hostnames: * `s3.example.com` for the S3 API * `console.example.com` for the Console Replace them with your public hostnames. The ACME HTTP-01 challenge requires inbound access to ports `80` and `443`. Do not publish the S3 API under a path such as `/s3/`. AWS Signature Version 4 includes the request path and host, so rewriting either value can invalidate signed requests.
## Create the deployment directory [#1-create-the-deployment-directory] Create a directory and an empty ACME storage file. Traefik requires restrictive permissions on this file: ```bash mkdir rustfs-traefik cd rustfs-traefik touch acme.json chmod 600 acme.json ```
## Set deployment variables [#2-set-deployment-variables] Create an environment file and replace each value: ```ini title=".env" S3_HOSTNAME=s3.example.com CONSOLE_HOSTNAME=console.example.com ACME_EMAIL=admin@example.com RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= ``` Use an email address that receives certificate expiration notices. Do not commit `.env` or `acme.json` to source control.
## Create the Compose file [#3-create-the-compose-file] Create the Traefik and RustFS services: ```yaml title="compose.yaml" services: traefik: image: traefik:v3.6.5 restart: unless-stopped command: - --log.level=INFO - --accesslog=true - --providers.docker=true - --providers.docker.endpoint=unix:///var/run/docker.sock - --providers.docker.exposedbydefault=false - --providers.docker.network=rustfs - --entrypoints.web.address=:80 - --entrypoints.websecure.address=:443 - --entrypoints.web.http.redirections.entrypoint.to=websecure - --entrypoints.web.http.redirections.entrypoint.scheme=https - --certificatesresolvers.le.acme.email=${ACME_EMAIL} - --certificatesresolvers.le.acme.storage=/etc/traefik/acme.json - --certificatesresolvers.le.acme.httpchallenge=true - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web ports: - "80:80" - "443:443" volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./acme.json:/etc/traefik/acme.json networks: - rustfs rustfs: image: rustfs/rustfs:latest restart: unless-stopped environment: RUSTFS_ACCESS_KEY: ${RUSTFS_ACCESS_KEY} RUSTFS_SECRET_KEY: ${RUSTFS_SECRET_KEY} RUSTFS_CONSOLE_ENABLE: "true" RUSTFS_ADDRESS: ":9000" RUSTFS_CONSOLE_ADDRESS: ":9001" RUSTFS_OBS_LOGGER_LEVEL: error RUSTFS_OBS_LOG_DIRECTORY: /var/log/rustfs/ expose: - "9000" - "9001" volumes: - rustfs-data:/data healthcheck: test: ["CMD-SHELL", "curl --fail http://127.0.0.1:9000/health/ready && curl --fail http://127.0.0.1:9001/rustfs/console/health"] interval: 10s timeout: 5s retries: 5 start_period: 30s labels: - traefik.enable=true - traefik.docker.network=rustfs - traefik.http.routers.rustfs-s3.rule=Host(`${S3_HOSTNAME}`) - traefik.http.routers.rustfs-s3.entrypoints=websecure - traefik.http.routers.rustfs-s3.tls=true - traefik.http.routers.rustfs-s3.tls.certresolver=le - traefik.http.routers.rustfs-s3.service=rustfs-s3 - traefik.http.services.rustfs-s3.loadbalancer.server.port=9000 - traefik.http.services.rustfs-s3.loadbalancer.passhostheader=true - traefik.http.routers.rustfs-console.rule=Host(`${CONSOLE_HOSTNAME}`) - traefik.http.routers.rustfs-console.entrypoints=websecure - traefik.http.routers.rustfs-console.tls=true - traefik.http.routers.rustfs-console.tls.certresolver=le - traefik.http.routers.rustfs-console.service=rustfs-console - traefik.http.services.rustfs-console.loadbalancer.server.port=9001 - traefik.http.services.rustfs-console.loadbalancer.passhostheader=true networks: - rustfs volumes: rustfs-data: networks: rustfs: name: rustfs ``` The two routers use different host rules and backend ports. RustFS does not publish ports `9000` or `9001` on the Docker host, and the Traefik Dashboard is not exposed. Traefik reads container labels through the read-only Docker socket mount. Anyone who can modify Docker workloads can influence routes discovered by the Docker provider. Restrict Docker access on the proxy host.
## Validate and start the deployment [#4-validate-and-start-the-deployment] Render the Compose configuration and check that all variables resolve: ```bash docker compose config ``` Start both services: ```bash docker compose up -d docker compose ps ``` Follow the Traefik logs while it completes the ACME challenge and creates both certificates: ```bash docker compose logs --follow traefik ``` If certificate issuance fails, confirm that both DNS records resolve to this host and that ports `80` and `443` are reachable from the internet. Let's Encrypt rate limits apply, so correct DNS and firewall problems before repeatedly recreating the deployment.
## Verify both endpoints [#5-verify-both-endpoints] Verify the S3 API readiness endpoint and Console health endpoint through Traefik: ```bash curl --fail https://s3.example.com/health/ready curl --fail https://console.example.com/rustfs/console/health ``` Configure S3 clients with `https://s3.example.com` as the endpoint and enable path-style addressing. Open `https://console.example.com` to sign in to the Console.
## Multi-node services [#multi-node-services] For an external multi-node RustFS cluster, enable Traefik's file provider in the `traefik` service: ```yaml title="compose.yaml" services: traefik: command: - --providers.file.filename=/etc/traefik/dynamic.yaml - --providers.file.watch=true volumes: - ./dynamic.yaml:/etc/traefik/dynamic.yaml:ro ``` Create the dynamic configuration with every RustFS node: ```yaml title="dynamic.yaml" http: routers: rustfs-s3: rule: Host(`s3.example.com`) entryPoints: - websecure service: rustfs-s3 tls: certResolver: le rustfs-console: rule: Host(`console.example.com`) entryPoints: - websecure service: rustfs-console tls: certResolver: le services: rustfs-s3: loadBalancer: passHostHeader: true healthCheck: path: /health/ready interval: 10s timeout: 5s servers: - url: http://node1.example.net:9000 - url: http://node2.example.net:9000 - url: http://node3.example.net:9000 - url: http://node4.example.net:9000 rustfs-console: loadBalancer: passHostHeader: true sticky: cookie: name: rustfs_console secure: true httpOnly: true healthCheck: path: /rustfs/console/health interval: 10s timeout: 5s servers: - url: http://node1.example.net:9001 - url: http://node2.example.net:9001 - url: http://node3.example.net:9001 - url: http://node4.example.net:9001 ``` Configure sticky sessions for the Console service when you use OpenID Connect. An in-progress login stores its `state` on one RustFS node and the callback must return to that node. Keep port `9000` open directly between RustFS nodes because internal node RPC uses the same listener. ## Next steps [#next-steps] * [Configure an S3 client](/developer/examples/aws-cli) * [Enable virtual-hosted-style bucket URLs](/integration/virtual) * [Review health and readiness endpoints](/operations/status-check) # RustFS Open Source License (/en/developer/license) ## Open Source License [#open-source-license] * RustFS is released under the Apache 2.0 license. * [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) ## License Terms [#license-terms] *Version 2.0, January 2004* [http://www.apache.org/licenses/](http://www.apache.org/licenses/) ### Terms and Conditions for use, reproduction, and distribution [#terms-and-conditions-for-use-reproduction-and-distribution] #### Definitions [#definitions] "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means **(i)** the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or **(ii)** ownership of fifty percent (50%) or more of the outstanding shares, or **(iii)** beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. #### Grant of Copyright License [#grant-of-copyright-license] Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. #### Grant of Patent License [#grant-of-patent-license] Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. #### Redistribution [#redistribution] You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: * **(a)** You must give any other recipients of the Work or Derivative Works a copy of this License; and * **(b)** You must cause any modified files to carry prominent notices stating that You changed the files; and * **(c)** You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and * **(d)** If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. #### Submission of Contributions [#submission-of-contributions] Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. #### Trademarks [#trademarks] This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. #### Disclaimer of Warranty [#disclaimer-of-warranty] Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. #### Limitation of Liability [#limitation-of-liability] In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. #### Accepting Warranty or Additional Liability [#accepting-warranty-or-additional-liability] While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. *END OF TERMS AND CONDITIONS* ### APPENDIX: How to apply the Apache License to your work [#appendix-how-to-apply-the-apache-license-to-your-work] To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets `[]` replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright \[yyyy] \[name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # RustFS MCP (/en/developer/mcp) **RustFS MCP Server** is a high-performance [Model Context Protocol (MCP)](https://www.anthropic.com/news/model-context-protocol) server that provides AI/LLM tools with seamless access to S3-compatible object storage operations. Built with Rust for performance and security, it enables AI assistants like Claude Desktop to interact with cloud storage through standardized protocols. ### What is MCP? [#what-is-mcp] The Model Context Protocol is an open standard that enables AI applications to establish secure, controlled connections with external systems. This server acts as a bridge between AI tools and S3-compatible storage services, providing structured access to file operations while maintaining security and observability. ## ✨ Features [#-features] ### Supported S3 Operations [#supported-s3-operations] * **List Buckets**: Lists all accessible S3 buckets. * **List Objects**: Browses bucket contents with optional prefix filtering. * **Upload Files**: Uploads local files with automatic MIME type detection and cache control. * **Get Objects**: Retrieves objects from S3 storage, supporting read or download modes. ## 🔧 Installation [#-installation] The `rustfs-mcp` crate is no longer part of the current `rustfs/rustfs` main branch, so the build commands below may fail against a fresh clone. Check the [RustFS GitHub organization](https://github.com/rustfs) for the MCP server's current location before building from source. ### Prerequisites [#prerequisites] * Rust 1.75+ (for building from source) * Configured AWS credentials (via environment variables, AWS CLI, or IAM roles) * Access to S3-compatible storage services ### Building from Source [#building-from-source] ```bash # Clone the repository git clone https://github.com/rustfs/rustfs.git cd rustfs # Build the MCP server cargo build --release -p rustfs-mcp # Binary will be available at ./target/release/rustfs-mcp ``` ## ⚙️ Configuration [#️-configuration] ### Environment Variables [#environment-variables] ```bash # AWS credentials (required) export AWS_ACCESS_KEY_ID=your_access_key export AWS_SECRET_ACCESS_KEY=your_secret_key export AWS_REGION=us-east-1 # optional, defaults to us-east-1 # Optional: Custom S3 endpoint (for MinIO, etc.) export AWS_ENDPOINT_URL=http://localhost:9000 # Log level (optional) export RUST_LOG=info ``` ### Command Line Options [#command-line-options] ```bash rustfs-mcp --help ``` The server supports various command line options to customize behavior: * `--access-key-id`: AWS access key ID for S3 authentication * `--secret-access-key`: AWS secret key for S3 authentication * `--region`: AWS region for S3 operations (default: us-east-1) * `--endpoint-url`: Custom S3 endpoint URL (for MinIO, LocalStack, etc.) * `--log-level`: Log level configuration (default: rustfs\_mcp\_server=info) ## 🚀 Usage [#-usage] ### Starting the Server [#starting-the-server] ```bash # Start the MCP server rustfs-mcp # Or with custom options rustfs-mcp --log-level debug --region us-west-2 ``` ### Integration with Chat Clients [#integration-with-chat-clients] #### Option 1: Using Command Line Arguments [#option-1-using-command-line-arguments] ```json { "mcpServers": { "rustfs-mcp": { "command": "/path/to/rustfs-mcp", "args": [ "--access-key-id", "your_access_key", "--secret-access-key", "your_secret_key", "--region", "us-west-2", "--log-level", "info" ] } } } ``` #### Option 2: Using Environment Variables [#option-2-using-environment-variables] ```json { "mcpServers": { "rustfs-mcp": { "command": "/path/to/rustfs-mcp", "env": { "AWS_ACCESS_KEY_ID": "your_access_key", "AWS_SECRET_ACCESS_KEY": "your_secret_key", "AWS_REGION": "us-east-1" } } } } ``` ### Using with Docker [#using-with-docker] [RustFS MCP officially provides a Dockerfile](https://github.com/rustfs/rustfs/tree/main/crates/mcp) that can be used to build container images for using RustFS MCP. ```bash # Clone RustFS repository code git clone https://github.com/rustfs/rustfs.git # Build Docker image docker build -f crates/mcp/Dockerfile -t rustfs/rustfs-mcp . ``` After successful build, you can configure it in the MCP configuration of AI IDEs. #### Configuring MCP in AI IDEs [#configuring-mcp-in-ai-ides] Currently, mainstream AI IDEs such as Cursor, Windsurf, Trae, etc. all support MCP. For example, in Trae, add the following content to the MCP configuration (**MCP --> Add**): ```json { "mcpServers": { "rustfs-mcp": { "command": "docker", "args": [ "run", "--rm", "-i", "-e", "AWS_ACCESS_KEY_ID", "-e", "AWS_SECRET_ACCESS_KEY", "-e", "AWS_REGION", "-e", "AWS_ENDPOINT_URL", "rustfs/rustfs-mcp" ], "env": { "AWS_ACCESS_KEY_ID": "rustfs_access_key", "AWS_SECRET_ACCESS_KEY": "rustfs_secret_key", "AWS_REGION": "us-east-1", "AWS_ENDPOINT_URL": "rustfs_instance_url" } } } } ``` > `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are RustFS access keys. You can refer to the [Access Key Management chapter](../security-compliance/iam/access-token.md) for creation. If added successfully, you can list the [available tools](#️-available-tools) on the MCP configuration page. add rustfs mcp in trae mcp configuration successfully In Trae, you can use the corresponding tools by entering the corresponding prompts. For example, in Trae's chat dialog, enter: ```text Please help me list the buckets in the current rustfs instance, thank you! ``` Returns the following response: list rustfs bucket with rustfs mcp Trae uses **Builder with MCP** mode, calling the `list_buckets` tool to list all buckets in the configured RustFS instance. The same applies to calls to other tools. ## 🛠️ Available Tools [#️-available-tools] The MCP server exposes the following tools that AI assistants can use: ### `list_buckets` [#list_buckets] Lists all S3 buckets accessible with the configured credentials. **Parameters**: None ### `list_objects` [#list_objects] Lists objects in an S3 bucket with optional prefix filtering. **Parameters**: * `bucket_name` (string): Name of the S3 bucket * `prefix` (string, optional): Prefix for filtering objects ### `upload_file` [#upload_file] Uploads a local file to S3 with automatic MIME type detection. **Parameters**: * `local_file_path` (string): Local file path * `bucket_name` (string): Target S3 bucket * `object_key` (string): S3 object key (target path) * `content_type` (string, optional): Content type (auto-detected if not provided) * `storage_class` (string, optional): S3 storage class * `cache_control` (string, optional): Cache control header ### `get_object` [#get_object] Retrieves objects from S3 with two operation modes: direct content reading or download to file. **Parameters**: * `bucket_name` (string): Source S3 bucket * `object_key` (string): S3 object key * `version_id` (string, optional): Version ID for versioned objects * `mode` (string, optional): Operation mode - "read" (default) returns content directly, "download" saves to local file * `local_path` (string, optional): Local file path (required when mode is "download") * `max_content_size` (number, optional): Maximum content size for read mode in bytes (default: 1MB) ### `create_bucket` [#create_bucket] Creates a new RustFS bucket. **Parameters**: * `bucket_name` (string): Name of the bucket to create. ### `delete_bucket` [#delete_bucket] Deletes the specified RustFS bucket. **Parameters**: * `bucket_name` (string): Name of the bucket to delete. ## Architecture [#architecture] The MCP server is built with a modular architecture: ```text rustfs-mcp/ ├── src/ │ ├── main.rs # Entry point, CLI parsing and server initialization │ ├── server.rs # MCP server implementation and tool handlers │ ├── s3_client.rs # S3 client wrapper with async operations │ ├── config.rs # Configuration management and CLI options │ └── lib.rs # Library exports and public API └── Cargo.toml # Dependencies, metadata and binary configuration ``` # Golang SDK Guide (/en/developer/sdk/go) RustFS ships no first-party Go SDK — it is fully S3-compatible, so you use the official AWS SDK for Go v2 configured to point at your RustFS server. Through the SDK, you can operate on RustFS, including creating and deleting buckets/objects, uploading and downloading files, etc. ## Prerequisites [#prerequisites] * Go 1.21 or later * A working RustFS instance (refer to [Installation Guide](../../installation/index.md)) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (refer to [Access Key Management](../../security-compliance/iam/access-token.md)) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. Install the SDK modules: ```bash go mod init rustfs-go-demo go get github.com/aws/aws-sdk-go-v2/aws go get github.com/aws/aws-sdk-go-v2/credentials go get github.com/aws/aws-sdk-go-v2/service/s3 ``` ## Initializing the Client [#initializing-the-client] The following is a complete, runnable program. It reads its configuration from environment variables and initializes the client from an `aws.Config`: ```go title="main.go" package main import ( "context" "fmt" "log" "os" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/s3" ) func main() { region := os.Getenv("RUSTFS_REGION") accessKeyID := os.Getenv("RUSTFS_ACCESS_KEY_ID") secretAccessKey := os.Getenv("RUSTFS_SECRET_ACCESS_KEY") endpoint := os.Getenv("RUSTFS_ENDPOINT_URL") if accessKeyID == "" || secretAccessKey == "" || region == "" || endpoint == "" { log.Fatal("missing the env: RUSTFS_ACCESS_KEY_ID / RUSTFS_SECRET_ACCESS_KEY / RUSTFS_REGION / RUSTFS_ENDPOINT_URL") } // build aws.Config cfg := aws.Config{ Region: region, Credentials: aws.NewCredentialsCache(credentials.NewStaticCredentialsProvider(accessKeyID, secretAccessKey, "")), } // build S3 client client := s3.NewFromConfig(cfg, func(o *s3.Options) { o.BaseEndpoint = aws.String(endpoint) // RustFS uses path-style URLs by default; virtual-host style requires RUSTFS_SERVER_DOMAINS o.UsePathStyle = true }) ctx := context.Background() resp, err := client.ListBuckets(ctx, &s3.ListBucketsInput{}) if err != nil { log.Fatalf("list buckets failed: %v", err) } fmt.Println("Buckets:") for _, b := range resp.Buckets { fmt.Println(" -", *b.Name) } } ``` These environment variable names (`RUSTFS_ENDPOINT_URL`, `RUSTFS_REGION`, `RUSTFS_ACCESS_KEY_ID`, `RUSTFS_SECRET_ACCESS_KEY`) are just this example's client-side conventions — they are read by your Go program, not by RustFS. They are distinct from the server-side `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` variables used when installing RustFS. Run it (replace `localhost` with your server's IP address if RustFS runs on another machine): ```bash export RUSTFS_ENDPOINT_URL="http://localhost:9000" export RUSTFS_REGION="us-east-1" export RUSTFS_ACCESS_KEY_ID="" export RUSTFS_SECRET_ACCESS_KEY="" go run main.go ``` ```text Buckets: - my-bucket ``` You can now perform bucket and object operations. The snippets below run inside the `main` function above, reusing `client` and `ctx`. ## Create Bucket [#create-bucket] ```go _, err = client.CreateBucket(ctx, &s3.CreateBucketInput{ Bucket: aws.String("my-bucket"), }) if err != nil { log.Fatalf("create bucket failed: %v", err) } fmt.Println("bucket created") ``` ```text bucket created ``` ## List Buckets [#list-buckets] ```go resp, err := client.ListBuckets(ctx, &s3.ListBucketsInput{}) if err != nil { log.Fatalf("list buckets failed: %v", err) } fmt.Println("Buckets:") for _, b := range resp.Buckets { fmt.Println(" -", *b.Name) } ``` ```text Buckets: - my-bucket ``` ## Delete Bucket [#delete-bucket] ```go _, err = client.DeleteBucket(ctx, &s3.DeleteBucketInput{ Bucket: aws.String("my-bucket"), }) if err != nil { log.Fatalf("delete bucket failed: %v", err) } fmt.Println("bucket deleted") ``` ```text bucket deleted ``` ## List Objects [#list-objects] ```go resp, err := client.ListObjectsV2(ctx, &s3.ListObjectsV2Input{ Bucket: aws.String("my-bucket"), }) if err != nil { log.Fatalf("list object failed: %v", err) } for _, obj := range resp.Contents { fmt.Println(" -", *obj.Key) } ``` ```text - hello.txt ``` ## Upload Object [#upload-object] Uploading a string body (add `"strings"` to your imports): ```go _, err = client.PutObject(ctx, &s3.PutObjectInput{ Bucket: aws.String("my-bucket"), Key: aws.String("hello.txt"), Body: strings.NewReader("hello rustfs"), }) if err != nil { log.Fatalf("upload object failed: %v", err) } fmt.Println("object uploaded") ``` ```text object uploaded ``` ## Download Object [#download-object] Reading the object body (add `"io"` to your imports): ```go resp, err := client.GetObject(ctx, &s3.GetObjectInput{ Bucket: aws.String("my-bucket"), Key: aws.String("hello.txt"), }) if err != nil { log.Fatalf("download object fail: %v", err) } defer resp.Body.Close() // read object content data, err := io.ReadAll(resp.Body) if err != nil { log.Fatalf("read object content fail: %v", err) } fmt.Println("content is :", string(data)) ``` ```text content is : hello rustfs ``` For other operations (presigned URLs, multipart uploads, and more), see the [AWS SDK for Go v2 documentation](https://aws.github.io/aws-sdk-go-v2/docs/) — every S3-compatible call works against RustFS the same way. # RustFS SDK Overview (/en/developer/sdk) RustFS is a distributed object storage system fully compatible with the S3 protocol. Users can: * Manage RustFS through the Console management interface. * Manage RustFS through S3 clients. * Implement object storage operations and management on the business side through SDKs. Currently, the SDKs provided by RustFS include: * [Java SDK](./java.md) * [JavaScript SDK](./javascript.md) * [Python SDK](./python.md) * [Rust SDK](./rust.md) * [TypeScript SDK](./typescript.md) * [Golang SDK](./go.md) ## Terminology [#terminology] Amazon S3 (Simple Storage Service) was the first widely adopted object storage service. Its API has become the de facto standard for object storage. In this documentation, "S3" refers to the protocol. ## SDK Recommendations [#sdk-recommendations] We recommend using the official AWS S3 SDKs. These SDKs are mature, well-maintained, and highly optimized. If you have a familiar and trusted SDK from a vendor, you can use it. Some third-party SDKs may have non-standard implementations. We recommend avoiding SDKs that are not strictly S3-compliant. ## Compatibility with MinIO SDKs [#compatibility-with-minio-sdks] Yes, RustFS is fully compatible with MinIO SDKs. If you are using MinIO SDKs, you can modify the Endpoint, AK, and SK to be directly compatible with RustFS. ## Handling Incompatible SDKs [#handling-incompatible-sdks] If you encounter an SDK that does not support standard S3, MinIO, or RustFS: We recommend switching to a standard AWS S3 SDK. # Java SDK Guide (/en/developer/sdk/java) RustFS ships no first-party Java SDK — it is S3-compatible, so you use the official AWS SDK for Java v2 configured to point at your RustFS server.
## Prerequisites [#1-prerequisites] * Java 8 or later and Maven (or Gradle) * A running RustFS instance (see the [Installation Guide](../../installation/index.md)) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (see [Access Key Management](../../security-compliance/iam/access-token.md)) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. ### 1.1 Maven Project Setup [#11-maven-project-setup] Create a new Maven project: ```text rustfs-java-s3-demo/ ├── pom.xml └── src/ └── main/ └── java/ └── com/ └── example/ └── RustfsS3Example.java ``` ### 1.2 Add Dependencies [#12-add-dependencies] Add AWS SDK dependencies in `pom.xml`: ```xml title="pom.xml" software.amazon.awssdk s3 2.25.27 ``` > Recommend using AWS SDK v2, which has more complete features and supports async, reactive, and other patterns. ***
## Connecting to RustFS [#2-connecting-to-rustfs] ### 2.1 Complete Example [#21-complete-example] The following class compiles and runs as-is. Replace `localhost` with your server's IP address if RustFS runs on another machine, and fill in your own access keys: ```java title="RustfsS3Example.java" package com.example; import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.*; import java.net.URI; import java.nio.file.Paths; public class RustfsS3Example { public static void main(String[] args) { // 1. Initialize S3 client S3Client s3 = S3Client.builder() .endpointOverride(URI.create("http://localhost:9000")) // RustFS S3 API address .region(Region.US_EAST_1) // RustFS default region .credentialsProvider( StaticCredentialsProvider.create( AwsBasicCredentials.create("", "") ) ) // RustFS uses path-style URLs by default; virtual-host style requires RUSTFS_SERVER_DOMAINS .forcePathStyle(true) .build(); // 2. Create bucket String bucket = "my-bucket"; try { s3.createBucket(CreateBucketRequest.builder().bucket(bucket).build()); System.out.println("Bucket created: " + bucket); } catch (BucketAlreadyExistsException | BucketAlreadyOwnedByYouException e) { System.out.println("Bucket already exists."); } // 3. Upload file s3.putObject( PutObjectRequest.builder().bucket(bucket).key("hello.txt").build(), Paths.get("/path/to/hello.txt") ); System.out.println("Uploaded hello.txt"); // 4. Download file s3.getObject( GetObjectRequest.builder().bucket(bucket).key("hello.txt").build(), Paths.get("downloaded-hello.txt") ); System.out.println("Downloaded hello.txt"); // 5. List objects ListObjectsV2Response listResponse = s3.listObjectsV2(ListObjectsV2Request.builder().bucket(bucket).build()); listResponse.contents().forEach(obj -> System.out.println("Found object: " + obj.key())); // 6. Delete object s3.deleteObject(DeleteObjectRequest.builder().bucket(bucket).key("hello.txt").build()); System.out.println("Deleted hello.txt"); // 7. Delete bucket (optional) // s3.deleteBucket(DeleteBucketRequest.builder().bucket(bucket).build()); } } ``` Expected output: ```text Bucket created: my-bucket Uploaded hello.txt Downloaded hello.txt Found object: hello.txt Deleted hello.txt ``` ***
## Common Issues and Troubleshooting [#3-common-issues-and-troubleshooting] | Issue | Cause | Solution | | -------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------ | | `S3Exception: 301 Moved Permanently` | Path-style not enabled or region error | Set `.forcePathStyle(true)` and use region `us-east-1` | | `ConnectException: Connection refused` | RustFS not started or incorrect port | Check RustFS status and port | | `403 Forbidden` | AccessKey / SecretKey error | Check authentication configuration | | Upload fails with no response | SDK defaults to HTTPS, RustFS only supports HTTP (or needs certificates) | Use `http://` address and configure `endpointOverride` | ***
## Appendix [#4-appendix] ### 4.1 Maven Package and Run [#41-maven-package-and-run] Package project: ```bash mvn clean package ``` Execute: ```bash java -cp target/rustfs-java-s3-demo-1.0-SNAPSHOT.jar com.example.RustfsS3Example ``` ### 4.2 RustFS Configuration Recommendations [#42-rustfs-configuration-recommendations] * Ensure SSL validation is disabled when service uses HTTP protocol. * Enable CORS support (if used for web frontend). * Recommend setting limits like `max_object_size` and `max_part_size` to prevent large file transfer failures. *** The following advanced examples cover: * Presigned URL generation and usage * Multipart Upload complete process ***
## Java Advanced Features Examples [#5-java-advanced-features-examples] ### 5.1 Generate and Use Presigned URLs [#51-generate-and-use-presigned-urls] > Presigned URLs allow clients to temporarily access private objects without exposing credentials, widely used for browser direct upload or download file scenarios. #### 5.1.1 Generate Download Link (GET) [#511-generate-download-link-get] ```java import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Configuration; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.presigner.S3Presigner; import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest; import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest; import java.net.URI; import java.time.Duration; S3Presigner presigner = S3Presigner.builder() .endpointOverride(URI.create("http://localhost:9000")) .region(Region.US_EAST_1) .credentialsProvider( StaticCredentialsProvider.create( AwsBasicCredentials.create("", "") ) ) // The presigner must also sign path-style URLs .serviceConfiguration( S3Configuration.builder().pathStyleAccessEnabled(true).build() ) .build(); GetObjectRequest getObjectRequest = GetObjectRequest.builder() .bucket("my-bucket") .key("hello.txt") .build(); GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder() .getObjectRequest(getObjectRequest) .signatureDuration(Duration.ofMinutes(15)) // 15 minutes validity .build(); PresignedGetObjectRequest presignedRequest = presigner.presignGetObject(presignRequest); System.out.println("Presigned URL: " + presignedRequest.url()); ``` ```text Presigned URL: http://localhost:9000/my-bucket/hello.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&... ``` > 🔗 Open the link in browser to access the object. #### 5.1.2 Upload Presigned URL (PUT) [#512-upload-presigned-url-put] Similarly, you can also generate upload URLs: ```java import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest; import software.amazon.awssdk.services.s3.presigner.model.PutObjectPresignRequest; PutObjectRequest putRequest = PutObjectRequest.builder() .bucket("my-bucket") .key("upload.txt") .build(); PresignedPutObjectRequest presignedPut = presigner.presignPutObject( PutObjectPresignRequest.builder() .putObjectRequest(putRequest) .signatureDuration(Duration.ofMinutes(10)) .build() ); System.out.println("Upload URL: " + presignedPut.url()); ``` *** ### 5.2 Implement Multipart Upload [#52-implement-multipart-upload] > Multipart Upload is the recommended way for large file uploads, enabling resume from breakpoint during network fluctuations. The examples below reuse the `s3` client from section 2 and need these additional imports: ```java import software.amazon.awssdk.services.s3.model.*; import java.nio.file.Paths; import java.util.ArrayList; import java.util.List; ``` #### 5.2.1 Start Multipart Upload [#521-start-multipart-upload] ```java CreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder() .bucket("my-bucket") .key("bigfile.zip") .build(); CreateMultipartUploadResponse createResponse = s3.createMultipartUpload(createRequest); String uploadId = createResponse.uploadId(); ``` #### 5.2.2 Upload Each Part [#522-upload-each-part] ```java List completedParts = new ArrayList<>(); for (int i = 1; i <= 3; i++) { String partPath = "part" + i + ".bin"; // Assume each part is a local file UploadPartRequest uploadPartRequest = UploadPartRequest.builder() .bucket("my-bucket") .key("bigfile.zip") .uploadId(uploadId) .partNumber(i) .build(); UploadPartResponse uploadPartResponse = s3.uploadPart(uploadPartRequest, Paths.get(partPath)); completedParts.add( CompletedPart.builder() .partNumber(i) .eTag(uploadPartResponse.eTag()) .build() ); } ``` #### 5.2.3 Complete Multipart Upload [#523-complete-multipart-upload] ```java CompletedMultipartUpload completedUpload = CompletedMultipartUpload.builder() .parts(completedParts) .build(); CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder() .bucket("my-bucket") .key("bigfile.zip") .uploadId(uploadId) .multipartUpload(completedUpload) .build(); s3.completeMultipartUpload(completeRequest); System.out.println("Multipart upload completed."); ``` ```text Multipart upload completed. ``` #### 5.2.4 Abort Upload on Exception (Optional) [#524-abort-upload-on-exception-optional] ```java AbortMultipartUploadRequest abortRequest = AbortMultipartUploadRequest.builder() .bucket("my-bucket") .key("bigfile.zip") .uploadId(uploadId) .build(); s3.abortMultipartUpload(abortRequest); ``` *** For other operations (object tagging, bucket policies, and more), see the [AWS SDK for Java v2 documentation](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/) — every S3-compatible call works against RustFS the same way.
# JavaScript SDK Guide (/en/developer/sdk/javascript) ## I. Overview [#i-overview] RustFS ships no first-party JavaScript SDK — it is S3-compatible, so you use the official AWS SDK for JavaScript (v3) configured to point at your RustFS server. This guide shows how to connect to RustFS and perform common object storage operations. ## II. Prerequisites [#ii-prerequisites] * Node.js 18 or later * A running RustFS instance (see the [Installation Guide](../../installation/index.md)) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (see [Access Key Management](../../security-compliance/iam/access-token.md)) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. ### 2.1 SDK Installation [#21-sdk-installation] Install the required AWS SDK v3 modules with NPM: ```bash npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner ``` The examples below use ES modules (`import`). Set `"type": "module"` in your `package.json`, or save the files with the `.mjs` extension. *** ## III. Initializing the Client [#iii-initializing-the-client] The following is a complete, runnable script. Replace `localhost` with your server's IP address if RustFS runs on another machine, and fill in your own access keys: ```js title="main.mjs" import { S3Client, ListBucketsCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ endpoint: "http://localhost:9000", // RustFS S3 API address region: "us-east-1", // RustFS default region credentials: { accessKeyId: "", secretAccessKey: "", }, // RustFS uses path-style URLs by default; virtual-host style requires RUSTFS_SERVER_DOMAINS forcePathStyle: true, }); const { Buckets } = await s3.send(new ListBucketsCommand({})); console.log(Buckets?.map((b) => b.Name) ?? []); ``` Run it: ```bash node main.mjs ``` ```text [ 'my-bucket' ] ``` All snippets below reuse this `s3` client. *** ## IV. Basic Operations [#iv-basic-operations] ### 4.1 Create Bucket [#41-create-bucket] ```js import { CreateBucketCommand } from "@aws-sdk/client-s3"; await s3.send(new CreateBucketCommand({ Bucket: "my-bucket" })); console.log("Bucket created"); ``` ```text Bucket created ``` *** ### 4.2 Upload Object [#42-upload-object] ```js import { PutObjectCommand } from "@aws-sdk/client-s3"; import { readFileSync } from "fs"; const data = readFileSync("/path/to/hello.txt"); await s3.send( new PutObjectCommand({ Bucket: "my-bucket", Key: "hello.txt", Body: data, }) ); console.log("File uploaded"); ``` ```text File uploaded ``` *** ### 4.3 Download Object [#43-download-object] ```js import { GetObjectCommand } from "@aws-sdk/client-s3"; import { writeFile } from "fs/promises"; const response = await s3.send( new GetObjectCommand({ Bucket: "my-bucket", Key: "hello.txt" }) ); const streamToBuffer = async (stream) => { const chunks = []; for await (const chunk of stream) chunks.push(chunk); return Buffer.concat(chunks); }; const buffer = await streamToBuffer(response.Body); await writeFile("downloaded.txt", buffer); console.log("File downloaded"); ``` ```text File downloaded ``` *** ### 4.4 List Objects [#44-list-objects] ```js import { ListObjectsV2Command } from "@aws-sdk/client-s3"; const res = await s3.send(new ListObjectsV2Command({ Bucket: "my-bucket" })); res.Contents?.forEach((obj) => console.log(`${obj.Key} (${obj.Size} bytes)`)); ``` ```text hello.txt (12 bytes) ``` *** ### 4.5 Delete Object [#45-delete-object] ```js import { DeleteObjectCommand } from "@aws-sdk/client-s3"; await s3.send(new DeleteObjectCommand({ Bucket: "my-bucket", Key: "hello.txt" })); console.log("File deleted"); ``` ```text File deleted ``` *** ## V. Advanced Features [#v-advanced-features] ### 5.1 Generate Presigned URLs [#51-generate-presigned-urls] > Allows frontend or third parties to use temporary links for uploading/downloading files #### Download (GET) [#download-get] ```js import { GetObjectCommand } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; const url = await getSignedUrl( s3, new GetObjectCommand({ Bucket: "my-bucket", Key: "hello.txt" }), { expiresIn: 600 } ); console.log("Presigned GET URL:", url); ``` ```text Presigned GET URL: http://localhost:9000/my-bucket/hello.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&... ``` #### Upload (PUT) [#upload-put] ```js import { PutObjectCommand } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; const url = await getSignedUrl( s3, new PutObjectCommand({ Bucket: "my-bucket", Key: "upload.txt" }), { expiresIn: 600 } ); console.log("Presigned PUT URL:", url); ``` *** ### 5.2 Multipart Upload [#52-multipart-upload] ```js import { CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, AbortMultipartUploadCommand, } from "@aws-sdk/client-s3"; import { statSync, openSync, readSync, closeSync } from "fs"; const bucket = "my-bucket"; const key = "large-file.zip"; const filePath = "./large-file.zip"; const partSize = 5 * 1024 * 1024; // 5 MB // 1. Create upload task const createRes = await s3.send( new CreateMultipartUploadCommand({ Bucket: bucket, Key: key }) ); const uploadId = createRes.UploadId; // 2. Segmented upload const fileSize = statSync(filePath).size; const fd = openSync(filePath, "r"); const parts = []; for (let partNumber = 1, offset = 0; offset < fileSize; partNumber++) { const buffer = Buffer.alloc(Math.min(partSize, fileSize - offset)); readSync(fd, buffer, 0, buffer.length, offset); const uploadPartRes = await s3.send( new UploadPartCommand({ Bucket: bucket, Key: key, UploadId: uploadId, PartNumber: partNumber, Body: buffer, }) ); parts.push({ ETag: uploadPartRes.ETag, PartNumber: partNumber }); offset += partSize; } closeSync(fd); // 3. Complete upload await s3.send( new CompleteMultipartUploadCommand({ Bucket: bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: parts }, }) ); console.log("Multipart upload completed"); ``` ```text Multipart upload completed ``` *** ## VI. Common Issues and Notes [#vi-common-issues-and-notes] | Problem | Cause | Solution | | --------------------------- | --------------------------------------------- | ------------------------------------------------------- | | SignatureDoesNotMatch | Wrong signature version | JS SDK v3 uses v4 by default, ensure RustFS supports v4 | | EndpointConnectionError | Endpoint address misconfigured or not started | Check if RustFS address is accessible | | NoSuchKey | File does not exist | Check if `Key` is spelled correctly | | InvalidAccessKeyId / Secret | Credentials misconfigured | Check `accessKeyId` / `secretAccessKey` configuration | | Upload failure (path issue) | Path-style not enabled | Set `forcePathStyle: true` | *** ## VII. Appendix: Frontend Upload Adaptation [#vii-appendix-frontend-upload-adaptation] Using presigned URLs allows browsers to upload files directly without passing AccessKey. Frontend (HTML+JS) upload example: ```html ``` For other operations (object tagging, bucket policies, and more), see the [AWS SDK for JavaScript v3 documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/) — every S3-compatible call works against RustFS the same way. # Other SDKs (/en/developer/sdk/other) If AWS S3 doesn't officially support your language, you can adopt the following strategies to integrate with RustFS:
## Use HTTP Interface Direct Requests (Based on S3 API Protocol) [#1-use-http-interface-direct-requests-based-on-s3-api-protocol] The S3 protocol is a standard RESTful API. You can encapsulate access logic yourself using any language that supports HTTP requests (such as C, Rust, Lua, Erlang). ### Key points include: [#key-points-include] * **Signature Algorithm**: Implement AWS Signature Version 4 signature (more complex) * **Construct correct Headers and Canonical Request** * **Use HTTPS/HTTP client to send requests** 👉 Recommended to reference open-source project signature implementations, for example: * [https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html](https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html) ***
## Call CLI Tools or Middleware Services of Existing SDKs [#2-call-cli-tools-or-middleware-services-of-existing-sdks] If you don't want to implement signatures yourself, you can: ### 2.1. Use AWS CLI tools with existing language support: [#21-use-aws-cli-tools-with-existing-language-support] For example, call through Shell: ```bash aws s3 cp local.txt s3://mybucket/myfile.txt --endpoint-url http://rustfs.local:9000 ``` Or write a simple relay service using Node.js/Python SDK, and your language uploads/downloads by calling this service. ### 2.2. Set up a Proxy (such as Flask, FastAPI, Express) [#22-set-up-a-proxy-such-as-flask-fastapi-express] Let clients that don't support S3 call your encapsulated HTTP API: ```http POST /upload -> Service internally calls SDK to upload objects to RustFS GET /presigned-url -> Generate presigned URL for frontend/client use ``` ***
## Find Third-Party Community SDKs [#3-find-third-party-community-sdks] Although AWS doesn't have official SDKs, some language communities have developed unofficial S3 clients. For example: * Haskell: `amazonka-s3` * Rust: `rusoto` (deprecated) or `aws-sdk-rust` * OCaml: May implement through `cohttp` yourself * Delphi: Has commercial libraries supporting S3 protocol Community SDKs vary greatly in stability, so you should evaluate activity, documentation, and compatibility before use. ***
## Delegate Core Upload Logic to Platform Hosting [#4-delegate-core-upload-logic-to-platform-hosting] For example: * Delegate frontend (Web/Mobile) upload tasks to browser or App side execution (using presigned URLs) * Backend uses Node.js/Python/Go and other proxies to implement upload logic ***
## Summary Recommendations [#summary-recommendations] | Scenario | Recommended Solution | | ------------------------------------------ | --------------------------------------- | | Need complete control/embedded environment | Implement Signature V4 self-signing | | Weak language support but has Shell | Call upload through AWS CLI | | Can deploy relay service | Use Python/Node to build S3 API gateway | | Frontend upload | Use presigned URLs | # Python SDK Guide (/en/developer/sdk/python)
## Overview [#1-overview] RustFS ships no first-party SDKs — it is S3-compatible, so you use the official AWS SDK for Python, [Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html), configured to point at your RustFS server. This guide covers: * Bucket creation/deletion * Object upload/download/deletion * Listing objects * Generating presigned URLs * Multipart upload for large files ***
## Prerequisites [#2-prerequisites] * Python 3.8 or later * A running RustFS instance (see the [Installation Guide](../../installation/index.md)) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (see [Access Key Management](../../security-compliance/iam/access-token.md)) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. ### 2.1 Install Boto3 [#21-install-boto3] We recommend using a virtual environment: ```bash python3 -m venv venv source venv/bin/activate pip install boto3 ``` > Boto3 depends on `botocore`, which will be installed automatically. ***
## Connecting to RustFS [#3-connecting-to-rustfs] The following is a complete, runnable script. Replace `localhost` with your server's IP address if RustFS runs on another machine, and fill in your own access keys: ```python title="main.py" import boto3 from botocore.client import Config s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='', aws_secret_access_key='', region_name='us-east-1', config=Config( signature_version='s3v4', s3={'addressing_style': 'path'}, ), ) response = s3.list_buckets() for bucket in response['Buckets']: print(bucket['Name']) ``` Run it: ```bash python main.py ``` * `endpoint_url` — points to your RustFS S3 API (port `9000`, not the Console port `9001`) * `signature_version='s3v4'` — RustFS supports v4 signatures * `region_name='us-east-1'` — RustFS's default region * `addressing_style='path'` — RustFS uses path-style URLs by default; virtual-host style requires `RUSTFS_SERVER_DOMAINS` ***
## Basic Operations [#4-basic-operations] ### 4.1 Create Bucket [#41-create-bucket] ```python bucket_name = 'my-bucket' try: s3.create_bucket(Bucket=bucket_name) print(f'Bucket {bucket_name} created.') except s3.exceptions.BucketAlreadyOwnedByYou: print(f'Bucket {bucket_name} already exists.') ``` ```text Bucket my-bucket created. ``` *** ### 4.2 Upload File [#42-upload-file] ```python s3.upload_file('/path/to/hello.txt', bucket_name, 'hello.txt') print('File uploaded.') ``` ```text File uploaded. ``` *** ### 4.3 Download File [#43-download-file] ```python s3.download_file(bucket_name, 'hello.txt', 'hello-downloaded.txt') print('File downloaded.') ``` ```text File downloaded. ``` *** ### 4.4 List Objects [#44-list-objects] ```python response = s3.list_objects_v2(Bucket=bucket_name) for obj in response.get('Contents', []): print(f"- {obj['Key']} ({obj['Size']} bytes)") ``` ```text - hello.txt (12 bytes) ``` *** ### 4.5 Delete Object and Bucket [#45-delete-object-and-bucket] ```python s3.delete_object(Bucket=bucket_name, Key='hello.txt') print('Object deleted.') s3.delete_bucket(Bucket=bucket_name) print('Bucket deleted.') ``` ```text Object deleted. Bucket deleted. ``` ***
## Advanced Features [#5-advanced-features] ### 5.1 Generate Presigned URLs [#51-generate-presigned-urls] #### 5.1.1 Download Link (GET) [#511-download-link-get] ```python url = s3.generate_presigned_url( ClientMethod='get_object', Params={'Bucket': bucket_name, 'Key': 'hello.txt'}, ExpiresIn=600, # 10 minutes validity ) print('Presigned GET URL:', url) ``` ```text Presigned GET URL: http://localhost:9000/my-bucket/hello.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&... ``` #### 5.1.2 Upload Link (PUT) [#512-upload-link-put] ```python url = s3.generate_presigned_url( ClientMethod='put_object', Params={'Bucket': bucket_name, 'Key': 'upload-by-url.txt'}, ExpiresIn=600, ) print('Presigned PUT URL:', url) ``` You can use the `curl` tool to upload: ```bash curl -X PUT --upload-file /path/to/hello.txt "http://localhost:9000/my-bucket/upload-by-url.txt?X-Amz-Algorithm=..." ``` *** ### 5.2 Multipart Upload [#52-multipart-upload] Suitable for files larger than 10 MB, allows manual control of each part. ```python file_path = 'largefile.bin' key = 'largefile.bin' part_size = 5 * 1024 * 1024 # 5 MB # 1. Start upload response = s3.create_multipart_upload(Bucket=bucket_name, Key=key) upload_id = response['UploadId'] parts = [] try: with open(file_path, 'rb') as f: part_number = 1 while True: data = f.read(part_size) if not data: break part = s3.upload_part( Bucket=bucket_name, Key=key, PartNumber=part_number, UploadId=upload_id, Body=data, ) parts.append({'ETag': part['ETag'], 'PartNumber': part_number}) print(f'Uploaded part {part_number}') part_number += 1 # 2. Complete upload s3.complete_multipart_upload( Bucket=bucket_name, Key=key, UploadId=upload_id, MultipartUpload={'Parts': parts}, ) print('Multipart upload complete.') except Exception as e: # Abort upload s3.abort_multipart_upload(Bucket=bucket_name, Key=key, UploadId=upload_id) print('Multipart upload aborted due to error:', e) ``` ```text Uploaded part 1 Uploaded part 2 Uploaded part 3 Multipart upload complete. ``` ***
## Common Issue Troubleshooting [#6-common-issue-troubleshooting] | Issue | Cause | Solution | | -------------------------------------- | --------------------------------------------- | ----------------------------------------------------- | | `SignatureDoesNotMatch` | Not using v4 signature | Set `signature_version='s3v4'` | | `EndpointConnectionError` | Wrong RustFS address or service not started | Check endpoint and RustFS service status | | `AccessDenied` | Wrong credentials or insufficient permissions | Check AccessKey/SecretKey or bucket policies | | `PermanentRedirect` / wrong bucket URL | Path-style not enabled | Set `s3={'addressing_style': 'path'}` in the `Config` | ***
## Appendix: Quick Upload/Download Script Template [#7-appendix-quick-uploaddownload-script-template] ```python def upload_file(local_path, bucket, object_key): s3.upload_file(local_path, bucket, object_key) print(f"Uploaded {local_path} to s3://{bucket}/{object_key}") def download_file(bucket, object_key, local_path): s3.download_file(bucket, object_key, local_path) print(f"Downloaded s3://{bucket}/{object_key} to {local_path}") ``` For other operations (object tagging, bucket policies, and more), see the [Boto3 S3 documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html) — every S3-compatible call works against RustFS the same way.
# Rust SDK Guide (/en/developer/sdk/rust) RustFS ships no first-party Rust client crate — it is fully S3-compatible, so you use the official AWS SDK for Rust (`aws-sdk-s3`) configured to point at your RustFS server. Through the SDK, you can operate RustFS, including creation and deletion of buckets/objects, file upload and download, etc. ## Prerequisites [#prerequisites] * Rust 1.78 or later (install via [rustup](https://rustup.rs/)) * An available RustFS instance (refer to [Installation Guide](../../installation/index.md)) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (refer to [Access Key Management](../../security-compliance/iam/access-token.md)) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. Create a project and add the dependencies: ```bash cargo new rustfs-rust-demo && cd rustfs-rust-demo cargo add aws-config aws-sdk-s3 anyhow cargo add tokio --features full ``` Your `Cargo.toml` should contain: ```toml title="Cargo.toml" [dependencies] anyhow = "1" aws-config = "1" aws-sdk-s3 = "1" tokio = { version = "1", features = ["full"] } ``` ## Initializing the Client [#initializing-the-client] The following is a complete, runnable program. It loads the connection settings from environment variables, initializes the S3 client, and lists your buckets: ```rust title="src/main.rs" use anyhow::Result; use aws_config::BehaviorVersion; use aws_sdk_s3::config::{Credentials, Region}; use aws_sdk_s3::Client; use std::env; pub struct Config { pub region: String, pub access_key_id: String, pub secret_access_key: String, pub endpoint_url: String, } impl Config { pub fn from_env() -> Result { let region = env::var("RUSTFS_REGION")?; let access_key_id = env::var("RUSTFS_ACCESS_KEY_ID")?; let secret_access_key = env::var("RUSTFS_SECRET_ACCESS_KEY")?; let endpoint_url = env::var("RUSTFS_ENDPOINT_URL")?; Ok(Config { region, access_key_id, secret_access_key, endpoint_url, }) } } #[tokio::main] async fn main() -> Result<()> { let config = Config::from_env()?; let credentials = Credentials::new( config.access_key_id, config.secret_access_key, None, None, "rustfs", ); let region = Region::new(config.region); let shared_config = aws_config::defaults(BehaviorVersion::latest()) .region(region) .credentials_provider(credentials) .endpoint_url(config.endpoint_url) .load() .await; // RustFS uses path-style URLs by default; virtual-host style requires RUSTFS_SERVER_DOMAINS let s3_config = aws_sdk_s3::config::Builder::from(&shared_config) .force_path_style(true) .build(); let rustfs_client = Client::from_conf(s3_config); let res = rustfs_client.list_buckets().send().await?; for bucket in res.buckets() { println!("Bucket: {:?}", bucket.name()); } Ok(()) } ``` These environment variable names (`RUSTFS_ENDPOINT_URL`, `RUSTFS_REGION`, `RUSTFS_ACCESS_KEY_ID`, `RUSTFS_SECRET_ACCESS_KEY`) are just this example's client-side conventions — they are read by your program, not by RustFS. They are distinct from the server-side `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` variables used when installing RustFS. Run it (replace `localhost` with your server's IP address if RustFS runs on another machine): ```bash export RUSTFS_ENDPOINT_URL="http://localhost:9000" export RUSTFS_REGION="us-east-1" export RUSTFS_ACCESS_KEY_ID="" export RUSTFS_SECRET_ACCESS_KEY="" cargo run ``` ```text Bucket: Some("my-bucket") ``` You can now use the client for the operations below. Each snippet runs inside `main`, reusing `rustfs_client`. ## Create Bucket [#create-bucket] ```rust match rustfs_client .create_bucket() .bucket("my-bucket") .send() .await { Ok(_) => { println!("Bucket created successfully"); } Err(e) => { println!("Error creating bucket: {:?}", e); return Err(e.into()); } } ``` ```text Bucket created successfully ``` ## Delete Bucket [#delete-bucket] ```rust match rustfs_client .delete_bucket() .bucket("my-bucket") .send() .await { Ok(_) => { println!("Bucket deleted successfully"); } Err(e) => { println!("Error deleting bucket: {:?}", e); return Err(e.into()); } } ``` ```text Bucket deleted successfully ``` ## List Buckets [#list-buckets] ```rust match rustfs_client.list_buckets().send().await { Ok(res) => { println!("Total buckets number is {:?}", res.buckets().len()); for bucket in res.buckets() { println!("Bucket: {:?}", bucket.name()); } } Err(e) => { println!("Error listing buckets: {:?}", e); return Err(e.into()); } } ``` ```text Total buckets number is 1 Bucket: Some("my-bucket") ``` ## List Objects [#list-objects] ```rust match rustfs_client .list_objects_v2() .bucket("my-bucket") .send() .await { Ok(res) => { println!("Total objects number is {:?}", res.contents().len()); for object in res.contents() { println!("Object: {:?}", object.key()); } } Err(e) => { println!("Error listing objects: {:?}", e); return Err(e.into()); } } ``` ```text Total objects number is 1 Object: Some("hello.txt") ``` ## Upload File [#upload-file] Add these imports at the top of `src/main.rs`: ```rust use aws_sdk_s3::primitives::ByteStream; use tokio::fs; ``` Then upload a local file: ```rust let data = fs::read("/path/to/hello.txt").await.expect("can not open the file"); match rustfs_client .put_object() .bucket("my-bucket") .key("hello.txt") .body(ByteStream::from(data)) .send() .await { Ok(res) => { println!("Object uploaded successfully, res: {:?}", res); } Err(e) => { println!("Error uploading object: {:?}", e); return Err(e.into()); } } ``` ```text Object uploaded successfully, res: PutObjectOutput { e_tag: Some("\"...\""), ... } ``` ## Download Object [#download-object] ```rust match rustfs_client .get_object() .bucket("my-bucket") .key("hello.txt") .send() .await { Ok(res) => { let data = res.body.collect().await?.into_bytes(); println!("Object content: {}", String::from_utf8_lossy(&data)); } Err(e) => { println!("Error downloading object: {:?}", e); return Err(e.into()); } } ``` ```text Object content: hello rustfs ``` For other operations (presigned URLs, multipart uploads, and more), see the [AWS SDK for Rust documentation](https://docs.aws.amazon.com/sdk-for-rust/latest/dg/) — every S3-compatible call works against RustFS the same way. # RustFS TypeScript SDK Usage Guide (/en/developer/sdk/typescript) RustFS ships no first-party TypeScript SDK — it is fully S3-compatible, so you use the official AWS SDK for JavaScript v3 (which ships its own TypeScript type definitions) configured to point at your RustFS server. Through the SDK, you can operate RustFS, including creation and deletion of buckets/objects, file upload and download, etc. ## Prerequisites [#prerequisites] * Node.js 18 or later (the examples use ES modules — set `"type": "module"` in your `package.json`) * An available RustFS instance (refer to [Installation Guide](../../installation/index.md) for installation) — the S3 API listens on port `9000`, the Console on port `9001` * Access keys, set at install time via the `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` environment variables (refer to [Access Key Management](../../security-compliance/iam/access-token.md) for creation) If you did not set credentials at install time, the server defaults to `rustfsadmin` / `rustfsadmin` — fine for a throwaway local trial, never for anything reachable by others. Install the dependencies: ```bash npm install @aws-sdk/client-s3 npm install --save-dev typescript tsx @types/node ``` ## Initializing the Client [#initializing-the-client] The following is a complete, runnable example. Replace `localhost` with your server's IP address if RustFS runs on another machine, and fill in your own access keys: ```typescript title="main.ts" import { S3Client, CreateBucketCommand, DeleteBucketCommand, ListBucketsCommand, ListObjectsV2Command, PutObjectCommand, GetObjectCommand, } from "@aws-sdk/client-s3"; import * as fs from "fs"; const rustfs_client = new S3Client({ region: "us-east-1", // RustFS default region endpoint: "http://localhost:9000", // RustFS S3 API address credentials: { accessKeyId: "", secretAccessKey: "", }, // RustFS uses path-style URLs by default; virtual-host style requires RUSTFS_SERVER_DOMAINS forcePathStyle: true, }); const response = await rustfs_client.send(new ListBucketsCommand({})); console.log(response.Buckets?.map((bucket) => bucket.Name) ?? []); ``` Run it: ```bash npx tsx main.ts ``` ```text [ 'my-bucket' ] ``` Then use the constructed `rustfs_client` for the operations below. ## Create Bucket [#create-bucket] ```typescript async function createBucket() { try { const response = await rustfs_client.send( new CreateBucketCommand({ Bucket: "my-bucket", }) ); console.log("Bucket created:", response.Location); } catch (error) { console.log(error); } } ``` ```text Bucket created: /my-bucket ``` ## Delete Bucket [#delete-bucket] ```typescript async function deleteBucket() { try { await rustfs_client.send( new DeleteBucketCommand({ Bucket: "my-bucket", }) ); console.log("Bucket deleted"); } catch (error) { console.log(error); } } ``` ```text Bucket deleted ``` ## List Buckets [#list-buckets] ```typescript async function listBuckets() { try { const response = await rustfs_client.send(new ListBucketsCommand({})); response.Buckets?.forEach((bucket) => console.log(bucket.Name)); } catch (error) { console.log(error); } } ``` ```text my-bucket ``` ## List Objects [#list-objects] ```typescript async function listObjects() { try { const response = await rustfs_client.send( new ListObjectsV2Command({ Bucket: "my-bucket", }) ); response.Contents?.forEach((obj) => console.log(`${obj.Key} (${obj.Size} bytes)`)); } catch (error) { console.log(error); } } ``` ```text test/hello.txt (12 bytes) ``` ## Upload File [#upload-file] ```typescript async function uploadFile() { try { await rustfs_client.send( new PutObjectCommand({ Bucket: "my-bucket", Key: "test/hello.txt", Body: fs.createReadStream("/path/to/hello.txt"), }) ); console.log("Object uploaded"); } catch (error) { console.log(error); } } ``` ```text Object uploaded ``` ## Download Object [#download-object] ```typescript async function getObject() { try { const response = await rustfs_client.send( new GetObjectCommand({ Bucket: "my-bucket", Key: "test/hello.txt", }) ); // get object content if (response.Body) { const chunks: Buffer[] = []; for await (const chunk of response.Body as any) { chunks.push(chunk as Buffer); } const data = Buffer.concat(chunks).toString("utf-8"); console.log("Object content:", data); } } catch (error) { console.log(error); } } ``` ```text Object content: hello rustfs ``` For other operations (presigned URLs, multipart uploads, and more), see the [AWS SDK for JavaScript v3 documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/) — every S3-compatible call works against RustFS the same way. # RustFS Documentation (/en)
RustFS is a distributed object storage system written in Rust and built for S3-compatible workloads. This documentation covers installation, administration, security, and operations for RustFS clusters.
Review installation requirements
# cert-manager (/en/installation/cloud-native/helm-chart/cert-manager) Use **cert-manager** to issue the certificate referenced by the RustFS Ingress. This provides HTTPS for the public S3 API and Console endpoint and allows cert-manager to renew the certificate. ## Requirements [#requirements] * cert-manager is installed and its controller Pods are ready. * An `Issuer` or `ClusterIssuer` is ready. * The RustFS hostname resolves to the Ingress controller. Verify the issuer before installing RustFS: ```bash kubectl get clusterissuer letsencrypt-prod kubectl -n cert-manager get pods ```
## Configure Ingress TLS [#1-configure-ingress-tls] Add the Ingress and certificate settings to your standalone or distributed values file: ```yaml title="values.yaml" ingress: enabled: true className: nginx annotations: cert-manager.io/cluster-issuer: letsencrypt-prod hosts: - host: s3.example.com paths: - path: / pathType: Prefix tls: enabled: true certManager: enabled: true existingSecret: enabled: false name: "" ``` Replace the Ingress class, issuer, and hostname for your cluster. For a namespace-scoped Issuer, use the `cert-manager.io/issuer` annotation instead.
## Apply the configuration [#2-apply-the-configuration] ```bash helm upgrade rustfs ./helm/rustfs \ --namespace rustfs \ -f values.yaml ``` The Ingress references the `rustfs-tls` Secret for a release named `rustfs`. cert-manager ingress-shim reads the issuer annotation and creates a Certificate that writes to this Secret.
## Verify the certificate [#3-verify-the-certificate] ```bash kubectl -n rustfs get ingress,certificate,certificaterequest kubectl -n rustfs describe certificate rustfs-tls kubectl -n rustfs get secret rustfs-tls ``` Wait for the Certificate to report `Ready=True`, then open `https://s3.example.com`.
## Use cert-manager with mTLS [#use-cert-manager-with-mtls] The chart also uses cert-manager to issue server and client certificates when `mtls.enabled=true`. See [mTLS](./mtls.md) to use the chart-managed CA or reference an existing issuer. # Overview (/en/installation/cloud-native/helm-chart) The official **RustFS Helm chart** deploys one RustFS cluster directly into Kubernetes. Helm renders the workload, Services, credentials, configuration, PersistentVolumeClaims (PVCs), Ingress, and optional certificate resources from a single values file. The chart supports two deployment modes: * **Standalone** creates one Pod with one data PVC. Use it for evaluation and development. * **Distributed** creates a StatefulSet with multiple Pods and data PVCs. `replicaCount` controls the number of Pods and `drivesPerNode` controls the number of data PVCs mounted by each Pod. Distributed mode is enabled by default. The chart also supports multiple append-only server pools, but a single explicit topology is easier to operate for an initial deployment. Credentials must be supplied through chart values or an existing Secret; the chart rejects empty and well-known default credentials unless insecure development defaults are explicitly enabled. Use the Helm chart when you want Helm to manage one RustFS cluster. Use the [RustFS Operator](../operator/index.md) when you need Kubernetes custom resources, multiple Tenants, or Operator-driven pool management. ## Helm chart workflows [#helm-chart-workflows] * [Install](./installation.mdx) covers requirements and standalone or distributed deployment. * [mTLS](./mtls.md) encrypts and authenticates traffic between RustFS Pods. * [cert-manager](./cert-manager.md) issues and renews certificates for RustFS Ingress and mTLS. # Install (/en/installation/cloud-native/helm-chart/installation) This guide gets the official RustFS chart from source or the Helm repository, installs it, and verifies access to the S3 API and Console. ## Requirements [#requirements] | Component | Requirement | | ------------ | ---------------------------------------- | | Helm | Version 3 | | Kubernetes | A cluster reachable with `kubectl` | | StorageClass | Dynamic PVC provisioning for RustFS data | | RustFS | `1.0.0-alpha.69` or later | An Ingress controller is optional. If you enable Ingress, set `ingress.className` to the controller in your cluster, such as `nginx` or `traefik`. Check the active cluster and available StorageClasses: ```bash kubectl config current-context kubectl get storageclass helm version --short ```
## Get the chart [#1-get-the-chart] Choose how Helm should access the chart. Both options set `RUSTFS_CHART`, which the installation commands below use. Clone the RustFS source repository to use the chart at `helm/rustfs`: ```bash git clone https://github.com/rustfs/rustfs.git cd rustfs export RUSTFS_CHART=./helm/rustfs ``` Add the RustFS repository listed on [Artifact Hub](https://artifacthub.io/packages/helm/rustfs/rustfs), then update the local repository index: ```bash helm repo add rustfs https://charts.rustfs.com helm repo update export RUSTFS_CHART=rustfs/rustfs ```
## Install standalone mode [#2-install-standalone-mode] Create a values file for a one-Pod development deployment: ```yaml title="standalone-values.yaml" mode: standalone: enabled: true distributed: enabled: false secret: rustfs: access_key: "" secret_key: "" storageclass: name: standard dataStorageSize: 10Gi logStorageSize: 1Gi ingress: enabled: false ``` Replace `standard` with a StorageClass in your cluster, then install: ```bash helm upgrade --install rustfs "$RUSTFS_CHART" \ --namespace rustfs \ --create-namespace \ -f standalone-values.yaml ```
## Install distributed mode [#3-install-distributed-mode] For a distributed cluster, set the Pod and drive counts explicitly. Total data drives equal `replicaCount * drivesPerNode`. ```yaml title="distributed-values.yaml" mode: standalone: enabled: false distributed: enabled: true replicaCount: 4 drivesPerNode: 2 secret: rustfs: access_key: "" secret_key: "" storageclass: name: standard dataStorageSize: 100Gi logStorageSize: 1Gi ingress: enabled: false ``` The example creates four Pods and eight data PVCs. Ensure the cluster can schedule all Pods and provision all PVCs, then install: ```bash helm upgrade --install rustfs "$RUSTFS_CHART" \ --namespace rustfs \ --create-namespace \ -f distributed-values.yaml ``` Kubernetes does not allow updates to StatefulSet `volumeClaimTemplates`. Changing `drivesPerNode` later requires StatefulSet recreation or a new installation.
## Verify and access RustFS [#4-verify-and-access-rustfs] ```bash kubectl -n rustfs get pods,pvc,services kubectl -n rustfs rollout status statefulset/rustfs ``` Standalone mode creates a Deployment instead of a StatefulSet. Check it with: ```bash kubectl -n rustfs rollout status deployment/rustfs ``` Forward the S3 API and Console to your workstation: ```bash kubectl -n rustfs port-forward svc/rustfs 9000:9000 9001:9001 ``` Use `http://localhost:9000` as the S3 endpoint and open `http://localhost:9001` for the Console.
## Key values [#key-values] | Value | Purpose | Chart default | | ------------------------------ | ------------------------------ | ---------------------------- | | `mode.standalone.enabled` | Enable one-Pod standalone mode | `false` | | `mode.distributed.enabled` | Enable distributed mode | `true` | | `replicaCount` | Distributed Pod count | `4` | | `drivesPerNode` | Data PVCs per Pod | Inferred from `replicaCount` | | `storageclass.name` | StorageClass for PVCs | `local-path` | | `storageclass.dataStorageSize` | Size of each data PVC | `256Mi` | | `storageclass.logStorageSize` | Size of each log PVC | `256Mi` | | `service.endpoint.port` | S3 API port | `9000` | | `service.console.port` | Console port | `9001` | We recommend setting storage sizes explicitly; the chart defaults are intended only for basic evaluation. # mTLS (/en/installation/cloud-native/helm-chart/mtls) The chart can enable mutual Transport Layer Security (mTLS) for RustFS Pod communication. When enabled, RustFS requires client certificates, uses HTTPS for generated peer URLs, and mounts server, client, and CA material into every Pod. ## Requirements [#requirements] mTLS uses cert-manager `Issuer` and `Certificate` resources. Install cert-manager before enabling it and confirm that its CRDs are available: ```bash kubectl get crd certificates.cert-manager.io issuers.cert-manager.io ```
## Use the chart-managed CA [#1-use-the-chart-managed-ca] Add the following setting to your existing values file: ```yaml title="values.yaml" mtls: enabled: true ``` Upgrade the release: ```bash helm upgrade rustfs ./helm/rustfs \ --namespace rustfs \ -f values.yaml ``` The chart creates a self-signed root CA, a namespace Issuer, and server and client Certificates. It mounts the resulting Secrets and configures RustFS with `RUSTFS_SERVER_MTLS_ENABLE=1` and `RUSTFS_TLS_PATH=/opt/tls`. Health probes also use the generated client certificate.
## Use an existing Issuer [#2-use-an-existing-issuer] To use an Issuer or ClusterIssuer already managed by your platform, configure its reference: ```yaml title="values.yaml" mtls: enabled: true existingIssuerRef: enabled: true name: internal-ca kind: ClusterIssuer group: cert-manager.io ``` The issuer must be ready and able to issue both server and client certificates in the `rustfs` namespace. Use `kind: Issuer` for a namespace-scoped issuer.
## Verify mTLS [#3-verify-mtls] ```bash kubectl -n rustfs get issuer,certificate,secret kubectl -n rustfs describe certificate rustfs-server-tls kubectl -n rustfs describe certificate rustfs-client-tls kubectl -n rustfs get pods ``` For a release named `rustfs`, the generated certificate Secrets are `rustfs-server-tls` and `rustfs-client-tls`. mTLS requires clients to present a trusted certificate. Validate how your Ingress controller or other external client presents that certificate before enabling mTLS on an existing deployment.
# Kubernetes Installation (Helm) (/en/installation/cloud-native) RustFS ships an official Helm chart that deploys either a single-node instance (a `Deployment` with one PVC) or a distributed cluster (a `StatefulSet` with multiple pods and PVCs). This guide walks through installing the chart, choosing a deployment mode, sizing storage correctly, and the production options the chart provides. **Prerequisites** * A Kubernetes cluster and `kubectl` access * Helm 3 * RustFS image version `>= 1.0.0-alpha.69` (the chart requirement) * A StorageClass with a working provisioner — the chart defaults to [`local-path`](https://github.com/rancher/local-path-provisioner); set `storageclass.name` to use your own * [`rc`](/operations/rc) installed on the administration host before using the server-pool commands in this guide The chart lives in the RustFS source repository under `helm/rustfs`: ```bash git clone https://github.com/rustfs/rustfs.git cd rustfs/helm/rustfs ```
## Quick install [#1-quick-install] Install into a dedicated namespace, setting your own credentials and a realistic data size: ```bash helm install rustfs . \ --namespace rustfs --create-namespace \ --set secret.rustfs.access_key= \ --set secret.rustfs.secret_key= \ --set storageclass.dataStorageSize=100Gi \ --set storageclass.logStorageSize=1Gi ``` Rendering fails by default unless one of the following is true: 1. `secret.existingSecret` names a Kubernetes Secret you control, or 2. `secret.rustfs.access_key` and `secret.rustfs.secret_key` are **both** set to non-empty, non-default values, or 3. `secret.allowInsecureDefaults: true` is set (only for local development). This prevents accidental deployment with the well-known default `rustfsadmin`/`rustfsadmin` credentials. Setting only one of the two keys is also rejected, so the chart never silently falls back to a default for the missing key. Watch the pods come up: ```bash kubectl -n rustfs get pods -w ``` ```text NAME READY STATUS RESTARTS AGE rustfs-0 1/1 Running 0 2m27s rustfs-1 1/1 Running 0 2m27s rustfs-2 1/1 Running 0 2m27s rustfs-3 1/1 Running 0 2m27s ```
## Choose a deployment mode [#2-choose-a-deployment-mode] The chart supports two modes, selected via the `mode` values: | Mode | Values | Workload | Layout | | ------------------------- | ---------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------- | | Distributed (**default**) | `mode.distributed.enabled=true` | StatefulSet | `replicaCount: 4` pods, 4 data PVCs each (16 drives total) — or `replicaCount: 16` for 16 pods with 1 data PVC each | | Standalone | `mode.standalone.enabled=true`, `mode.distributed.enabled=false` | Deployment | 1 pod, 1 data PVC (single node single disk) | * **Standalone** matches single-node single-disk: no erasure-coding redundancy across nodes. Use it for development, testing, or small setups where the underlying storage provides its own durability. It can reuse existing PVCs via `mode.standalone.existingClaim.dataClaim` / `mode.standalone.existingClaim.logsClaim`. * **Distributed** behaves like [multiple node multiple disk](../linux/multiple-node-multiple-disk.md): objects are erasure-coded across pods and PVCs. `replicaCount` must be `4` (each pod gets 4 PVCs) or `16` (each pod gets 1 PVC); pick based on how many nodes your cluster can spread pods across. ```bash # Standalone mode helm install rustfs . -n rustfs --create-namespace \ --set mode.standalone.enabled=true \ --set mode.distributed.enabled=false \ --set secret.rustfs.access_key= \ --set secret.rustfs.secret_key= ```
## Storage sizing [#3-storage-sizing] PVC sizes come from the `storageclass` block: ```yaml title="values-prod.yaml" storageclass: name: local-path # your StorageClass dataStorageSize: 256Mi # per data PVC logStorageSize: 256Mi # per logs PVC ``` The chart's default size for the data and logs volumes is **256Mi**, which is only enough to verify the chart works. For any real workload set `storageclass.dataStorageSize` (for example `1Ti`) and `storageclass.logStorageSize` (for example `1Gi`) at install time. In distributed mode the data size applies to **each** data PVC (16 PVCs by default). Setting `config.rustfs.obs_log_directory` to `""` disables the log PVCs and mounts entirely. Custom PVC annotations go under `storageclass.pvcAnnotations.data` / `storageclass.pvcAnnotations.logs`.
## Health probes [#4-health-probes] The chart templates HTTP probes on the S3 port (9000) out of the box, matching the server's health endpoints: * **Liveness**: `GET /health` (`livenessProbe.httpGet.path`), initial delay 30s, period 5s * **Readiness**: `GET /health/ready` (`readinessProbe.httpGet.path`), initial delay 10s, period 5s A pod is only added to the Service endpoints once `/health/ready` returns `200`, which in distributed mode requires the storage quorum to be met. Thresholds and timings are tunable via the `livenessProbe.*` and `readinessProbe.*` values.
## Production hardening [#5-production-hardening] ### Pod Disruption Budget [#pod-disruption-budget] Disabled by default. Enable it so voluntary evictions (node drains, cluster upgrades) never take more than one pod down at a time: ```bash --set pdb.create=true # pdb.maxUnavailable defaults to 1 ``` ### Anti-affinity and topology spread [#anti-affinity-and-topology-spread] `affinity.podAntiAffinity.enabled` defaults to `true` with `topologyKey: kubernetes.io/hostname`, spreading pods across distinct nodes. For zone-level spreading, enable `topologySpreadConstraints.enabled` and supply raw constraint entries under `topologySpreadConstraints.constraints` (applied to the distributed StatefulSet). ### Inter-pod mTLS (cert-manager) [#inter-pod-mtls-cert-manager] Set `mtls.enabled=true` to encrypt traffic between pods; the chart renders cert-manager `Issuer`/`Certificate` resources for a CA, server, and client certificates. To use an issuer you already operate, set `mtls.existingIssuerRef.enabled=true` with its `name`, `kind` (`Issuer` or `ClusterIssuer`), and `group`. ### Ingress and Gateway API [#ingress-and-gateway-api] Ingress is enabled by default (`ingress.enabled=true`) with `ingress.className: nginx`; set it to `traefik` if that is your controller — the chart applies the matching session-stickiness annotations for each. Set your domain via `ingress.hosts[0].host` (default `example.rustfs.com`). For HTTPS, enable `ingress.tls.enabled` and either pass the certificate with `--set-file ingress.tls.crt=./tls.crt --set-file ingress.tls.key=./tls.key`, point at an existing secret (`ingress.tls.existingSecret`), or let cert-manager issue one (`ingress.tls.certManager.enabled=true`). The chart also has alpha [Gateway API](https://gateway-api.sigs.k8s.io/) support (`gatewayApi.enabled=true` together with `ingress.enabled=false`, Traefik gateway class), rendering `Gateway` and `HTTPRoute` resources.
## Access RustFS [#6-access-rustfs] Without an ingress, port-forward the Service: ```bash kubectl -n rustfs port-forward svc/rustfs 9000:9000 9001:9001 ``` * S3 API: `http://localhost:9000` * Console: `http://localhost:9001` Log in to the Console with the access key and secret key you set at install time. With ingress enabled, use your configured host instead (check `kubectl -n rustfs get ing`). The Service defaults to `ClusterIP`; `service.type` can be switched to `NodePort` (S3 on `service.endpoint.nodePort: 32000`, Console on `service.console.nodePort: 32001`) or `LoadBalancer`.
## Scaling out with server pools [#7-scaling-out-with-server-pools] In distributed mode the chart can run multiple **server pools** — independent StatefulSets whose drives together form one cluster. This is the chart-level equivalent of adding a Server Pool as described in [Pool Expansion](../../operations/scaling/storage-pool-expansion.md). To expand an existing deployment, enable pools and describe the current layout as pool 0 plus your new capacity: ```yaml title="values-prod.yaml (pools)" pools: enabled: true list: - {} # pool 0: inherits top-level values and keeps the # existing StatefulSet/pod/PVC names and data - replicaCount: 4 # pool 1: new capacity (4 or 16) storageclass: dataStorageSize: 10Gi ``` Then apply with `helm upgrade`. Each entry may set `replicaCount` (4 or 16) and/or a `storageclass` block; omitted fields inherit the top-level values. Additional pools render as `-pool` StatefulSets; all pools share the headless service, the main service, the configuration, and the credentials. The list index determines the StatefulSet name — never remove or reorder entries. Retire a pool with `rc admin decommission` before removing it from the list. What to expect during the rollout, per the chart's documentation: * **Crash/restart cycles are normal.** Pods restart until every pod of every pool is resolvable — the server refuses to start with unresolvable peers, so expect a few crash loops before the cluster converges. This is harmless. * **Rebalance afterwards.** After the cluster converges, run `rc admin rebalance start ` to spread existing objects across the new pool. * The PodDisruptionBudget spans all pools: with the default `pdb.maxUnavailable: 1`, at most one pod of the whole cluster may be evicted at a time. `rc` is the RustFS command-line client. Use `rc admin pool list`, `expand`, `rebalance`, and `decommission` for the server-pool workflows described by the chart.
## Uninstall [#8-uninstall] ```bash helm uninstall rustfs -n rustfs ``` Helm does not delete PVCs created by StatefulSet volume claim templates. If you intend to discard the data, remove the PVCs explicitly (`kubectl -n rustfs delete pvc -l app.kubernetes.io/name=rustfs`) — otherwise a later reinstall with the same release name reattaches them.
## Next steps [#next-steps] * [Pool Expansion](../../operations/scaling/storage-pool-expansion.md) — how Server Pool expansion works at the cluster level * [Kubernetes Upgrade](../../operations/upgrade/kubernetes/index.md) — upgrade Helm- and Operator-managed deployments * [TLS configuration](../../integration/tls-configured.md) — end-to-end TLS options # Overview (/en/installation/cloud-native/operator) **RustFS Operator** applies the Kubernetes Operator pattern to RustFS clusters. Instead of creating StatefulSets, Services, PersistentVolumeClaims (PVCs), and configuration by hand, you declare the required storage cluster as a Kubernetes custom resource. The controller watches that resource and continuously reconciles the running cluster with the declared state. The Operator installs two Custom Resource Definitions (CRDs): * `Tenant` (`rustfs.com/v1alpha1`) represents one RustFS cluster. It defines storage pools, credentials, scheduling, Transport Layer Security (TLS), and Key Management Service (KMS) settings. * `PolicyBinding` (`sts.rustfs.com/v1alpha1`) maps a Kubernetes ServiceAccount to RustFS policies when workloads request temporary credentials from the Operator Security Token Service (STS). One Operator can manage multiple Tenants across namespaces. Each Tenant has independent storage, credentials, S3 and Console services, and lifecycle. The Operator creates one StatefulSet for each pool, so you can add capacity by appending a pool without rebuilding the cluster. It also reports `Ready`, `Progressing`, or `Degraded` conditions and Kubernetes Events, and exposes health and metrics endpoints for cluster monitoring. The same API covers small test clusters and distributed deployments. Sensitive credentials and KMS material stay in Kubernetes Secrets, while version-controlled Tenant manifests hold only Secret references. This makes deployments repeatable, supports GitOps workflows, and keeps routine operations such as multi-tenant management, pool expansion, TLS, and encryption within Kubernetes-native tools. ## Operator workflows [#operator-workflows] * [Install](./installation.md) covers requirements, Helm installation, Console access, and TLS configuration. * [Multi-Tenant](./tenant.md) creates isolated RustFS clusters for different teams or workloads. * [Pool Expansion](./pool-expansion.md) adds storage capacity by appending a pool to an existing Tenant. * [KMS Integration](./kms.md) configures local or HashiCorp Vault key management for encrypted data. RustFS Operator is currently `v0.1.0` pre-release software under active development. Validate upgrades and Tenant changes in a non-production cluster first. # Install (/en/installation/cloud-native/operator/installation) This guide installs the Operator with Helm, verifies the deployment, and exposes the Operator Console locally or through HTTPS. ## Requirements [#requirements] | Component | Requirement | | ------------ | ------------------------------------------- | | Kubernetes | `v1.30` or later | | Helm | `v3.0` or later | | kubectl | Compatible with the Kubernetes cluster | | StorageClass | Dynamic PVC provisioning for Tenant storage | Your account must be able to create CRDs, cluster RBAC, Deployments, and Services. Confirm the target cluster before installation: ```bash kubectl config current-context kubectl get storageclass ```
## Install the Operator [#1-install-the-operator] The Helm chart is stored in the Operator repository: ```bash git clone https://github.com/rustfs/operator.git cd operator helm upgrade --install rustfs-operator deploy/rustfs-operator/ \ --namespace rustfs-system \ --create-namespace ``` Common settings belong in a values file: ```yaml title="values.yaml" operator: replicas: 1 metrics: enabled: true tenantMonitor: enabled: true intervalSeconds: 300 console: enabled: true service: type: ClusterIP ``` Apply the file with `-f values.yaml`. The chart generates `OPERATOR_*` variables from these values; do not duplicate them under `operator.env`.
## Verify the installation [#2-verify-the-installation] ```bash kubectl -n rustfs-system get pods,services kubectl get crd tenants.rustfs.com kubectl -n rustfs-system rollout status deployment/rustfs-operator kubectl -n rustfs-system rollout status deployment/rustfs-operator-console ```
## Access the Operator Console [#3-access-the-operator-console] The Console listens on port `9090`. Generate a short-lived login token: ```bash kubectl -n rustfs-system create token rustfs-operator-console --duration=24h ``` Forward the Console service to your workstation: ```bash kubectl -n rustfs-system port-forward \ svc/rustfs-operator-console 19090:9090 ``` Open `http://127.0.0.1:19090` and paste the token into the login form. The Helm installation notes print the exact ServiceAccount and Service names when release names or namespaces differ. If your browser does not retain the login over HTTP, set `CONSOLE_COOKIE_SECURE=false` under `console.env` for local testing only. Keep secure cookies enabled for HTTPS.
## Configure Console TLS [#4-configure-console-tls] Use one HTTPS hostname for both the Console UI and `/api/v1`. Create a TLS Secret, or let cert-manager create it, then enable Ingress: ```yaml title="values.yaml" console: ingress: enabled: true className: nginx annotations: cert-manager.io/cluster-issuer: letsencrypt-prod hosts: - host: console.example.com paths: - path: / pathType: Prefix tls: - secretName: console-tls hosts: - console.example.com ``` Upgrade the release with the values file: ```bash helm upgrade rustfs-operator deploy/rustfs-operator/ \ --namespace rustfs-system \ -f values.yaml ``` Replace the Ingress class, issuer, and hostname for your environment. If cert-manager is not installed, create the `console-tls` Secret with your certificate and private key before the upgrade. Next, [create a Tenant](./tenant.md).
# KMS Integration (/en/installation/cloud-native/operator/kms) Configure Key Management Service (KMS) integration through `spec.encryption`. Do not add `RUSTFS_KMS_*` variables to `spec.env`; the Operator generates them from the structured Tenant configuration and Secret references. ## Choose a backend [#choose-a-backend] Use `local` only for a single-server Tenant. Use `vault` for distributed deployments where every Tenant Pod can reach HashiCorp Vault. ## Local KMS [#local-kms] Create a master key Secret: ```yaml title="local-kms-secret.yaml" apiVersion: v1 kind: Secret metadata: name: rustfs-local-kms namespace: storage-a type: Opaque stringData: local-master-key: "replace-with-a-random-master-key" ``` Add the encryption block to the existing Tenant manifest: ```yaml title="tenant.yaml" spec: encryption: enabled: true backend: local local: keyDirectory: /data/rustfs0/.kms-keys masterKeySecretRef: name: rustfs-local-kms key: local-master-key defaultKeyId: tenant-default ``` The key directory must be inside a mounted data path so it survives Pod replacement. ## HashiCorp Vault KMS [#hashicorp-vault-kms] Create a Secret containing a Vault token: ```yaml title="vault-kms-secret.yaml" apiVersion: v1 kind: Secret metadata: name: rustfs-kms namespace: storage-a type: Opaque stringData: vault-token: "replace-with-vault-token" ``` Add the Vault configuration to the existing Tenant manifest: ```yaml title="tenant.yaml" spec: encryption: enabled: true backend: vault vault: endpoint: https://vault.example.com:8200 kmsSecret: name: rustfs-kms defaultKeyId: tenant-default ``` Every Tenant Pod must be able to resolve and connect to the Vault endpoint and trust its certificate. ## Apply the configuration [#apply-the-configuration] ```bash kubectl apply -f local-kms-secret.yaml kubectl apply -f tenant.yaml kubectl -n storage-a describe tenant tenant-a ``` For Vault, apply `vault-kms-secret.yaml` before `tenant.yaml`. Changing encryption settings rolls the affected StatefulSets. Back up key material and test recovery before storing production data. Losing the local master key or Vault keys can make encrypted objects unrecoverable. # Pool Expansion (/en/installation/cloud-native/operator/pool-expansion) All pools in a Tenant form one RustFS cluster. Add capacity by appending a new pool to `spec.pools`; do not change the shape of an existing pool. Do not change `servers` or `persistence.volumesPerServer` on an existing pool. The Operator creates an immutable StatefulSet for each pool.
## Check the Tenant [#1-check-the-tenant] ```bash kubectl -n storage-a get tenant tenant-a kubectl -n storage-a get pods,pvc -l rustfs.tenant=tenant-a ``` Confirm that the Tenant is `Ready` and the cluster has enough compute and storage capacity.
## Add a pool [#2-add-a-pool] Append the following entry to the existing `spec.pools` list in `tenant.yaml`. Keep all existing entries unchanged. ```yaml title="tenant.yaml" - name: pool-1 servers: 2 persistence: volumesPerServer: 2 volumeClaimTemplate: storageClassName: standard accessModes: - ReadWriteOnce resources: requests: storage: 100Gi ``` This pool creates four PVCs: two servers multiplied by two volumes per server. Apply the complete Tenant manifest: ```bash kubectl apply -f tenant.yaml ```
## Watch the expansion [#3-watch-the-expansion] ```bash kubectl -n storage-a get tenant tenant-a -w kubectl -n storage-a get pods,pvc \ -l rustfs.pool=pool-1 ``` Wait for the Tenant to return to `Ready` before making another topology change. Increasing an existing PVC size is a separate Kubernetes storage operation and depends on the StorageClass.
# Multi-Tenant (/en/installation/cloud-native/operator/tenant) A `Tenant` represents one independent RustFS cluster. Use a separate namespace, credentials Secret, and Tenant resource for each team or workload.
## Create a namespace and credentials [#1-create-a-namespace-and-credentials] Create the Secret directly so credentials are not stored in a manifest: ```bash kubectl create namespace storage-a kubectl -n storage-a create secret generic rustfs-tenant-creds \ --from-literal=accesskey='' \ --from-literal=secretkey='' ```
## Define the Tenant [#2-define-the-tenant] This development example creates one RustFS Pod and one `10Gi` PVC. Replace `standard` with a StorageClass in your cluster. ```yaml title="tenant.yaml" apiVersion: rustfs.com/v1alpha1 kind: Tenant metadata: name: tenant-a namespace: storage-a spec: image: rustfs/rustfs:1.0.0-beta.10 credsSecret: name: rustfs-tenant-creds pools: - name: pool-0 servers: 1 persistence: volumesPerServer: 1 volumeClaimTemplate: storageClassName: standard accessModes: - ReadWriteOnce resources: requests: storage: 10Gi ```
## Apply and verify [#3-apply-and-verify] ```bash kubectl apply -f tenant.yaml kubectl -n storage-a get tenant,pods,pvc,svc kubectl -n storage-a describe tenant tenant-a ```
## Access RustFS [#4-access-rustfs] ```bash kubectl -n storage-a port-forward svc/tenant-a-io 9000:9000 kubectl -n storage-a port-forward svc/tenant-a-console 9001:9001 ``` Run the commands in separate terminals. Use `http://localhost:9000` as the S3 endpoint and open `http://localhost:9001` for the Tenant Console. To add another Tenant, repeat the process with a different namespace, Secret, and Tenant name. List all managed Tenants with: ```bash kubectl get tenants --all-namespaces ``` The one-server example is for evaluation. Production Tenants need a distributed pool layout, resource requests, scheduling constraints, and an immutable image reference.
# Docker (/en/installation/container/docker) This page covers running the official RustFS image with Docker: a single-node instance with persistent storage, host-directory permissions for the non-root container user, Docker Compose with optional observability services, TLS, and a multi-node deployment. You need a working Docker Engine and permission to run containers.
## Prerequisites [#1-prerequisites] * Docker Engine (≥ 20.10) installed and able to pull images and run containers normally * Host ports 9000 (S3 API) and 9001 (Console) available, or consistent with your custom ports * If you bind-mount a host directory, the directory owner must match the container user — see [Bind-mount a host directory](#bind-mount-a-host-directory)
## Pull the image [#2-pull-the-image] ```bash docker pull rustfs/rustfs:latest ```
## Create persistent storage [#3-create-persistent-storage] Create a named volume so object data remains available when you replace the container: ```bash docker volume create rustfs-data ```
## Start RustFS [#4-start-rustfs] Replace the credential placeholders before running the container: ```bash docker run -d \ --name rustfs \ --restart unless-stopped \ -p 9000:9000 \ -p 9001:9001 \ -v rustfs-data:/data \ -e RUSTFS_ACCESS_KEY="" \ -e RUSTFS_SECRET_KEY="" \ -e RUSTFS_ADDRESS=":9000" \ -e RUSTFS_CONSOLE_ADDRESS=":9001" \ -e RUSTFS_CONSOLE_ENABLE=true \ -e RUSTFS_OBS_LOGGER_LEVEL=error \ -e RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" \ rustfs/rustfs:latest \ /data ``` Set unique `RUSTFS_ACCESS_KEY` and `RUSTFS_SECRET_KEY` environment variables before exposing RustFS to a network. Do not use the well-known `rustfsadmin` value for either credential. If the container was started without custom credentials, stop and recreate it with both `-e` options shown above; the `rustfs-data` volume remains intact. ### Environment variables or command-line flags [#environment-variables-or-command-line-flags] The example above configures RustFS with environment variables. You can pass the same settings as command-line flags instead; when both are present, command-line flags win: ```bash docker run -d \ --name rustfs \ -p 9000:9000 \ -p 9001:9001 \ -v rustfs-data:/data \ rustfs/rustfs:latest \ --access-key "" \ --secret-key "" \ --address :9000 \ --console-enable \ /data ```
## Bind-mount a host directory [#5-bind-mount-a-host-directory] The named volume above needs no extra setup. If you mount a host directory instead (`-v /path/on/host:/data`), keep in mind that the container runs as non-root user `rustfs` with id `10001`. Make the host directory owned by `10001`, otherwise you will encounter permission denied errors: ```bash chown -R 10001:10001 /path/to/host_directory ```
## Verify the deployment [#6-verify-the-deployment] Check the container and the S3 API health endpoint: ```bash docker ps --filter name=rustfs curl --fail http://localhost:9000/health ``` The S3 API is available at `http://localhost:9000`, and the Console is available at `http://localhost:9001`.
## Docker Compose [#docker-compose] The RustFS repository ships a [`docker-compose.yml`](https://github.com/rustfs/rustfs/blob/main/docker-compose.yml) that includes `grafana`, `prometheus`, `otel-collector`, and `jaeger` services, mainly for observability. To deploy RustFS together with these services, clone the [RustFS code repository](https://github.com/rustfs/rustfs) locally: ```bash git clone https://github.com/rustfs/rustfs.git ``` Run the command from the repository root: ```bash docker compose --profile observability up -d ``` The compose file uses an initialization container to grant the correct access rights to `rustfs`: the `rustfs_perms` service below changes the ownership of the mounted volumes to `10001` before `rustfs` starts, using `depends_on` to wait for it to complete. To keep logs persistent and accessible, the host log directory is mapped to the container's `/var/log/rustfs/` path: ```yaml title="docker-compose.yml" services: # grant the necessary permissions to RUSTFS volumes path rustfs_perms: image: alpine user: root volumes: - /path/to/host_directory/volumes:/fix_path command: chown -R 10001:10001 /fix_path rustfs: image: rustfs/rustfs:latest depends_on: rustfs_perms: condition: service_completed_successfully volumes: - /path/to_host_directory/volumes/data:/data - /path/to_host_directory/volumes/logs:/var/log/rustfs/ environment: - RUSTFS_ADDRESS=":9000" - RUSTFS_CONSOLE_ADDRESS=":9001" - RUSTFS_CONSOLE_ENABLE=true - RUSTFS_OBS_LOGGER_LEVEL=error - RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" # ... other configurations ``` If you only want RustFS without Grafana, Prometheus, and the other observability services, start just the `rustfs` service (the compose file marks the collector dependency as optional): ```bash docker compose -f docker-compose.yml up -d rustfs ``` This starts only the `rustfs-server` container. Whether you start only `rustfs-server` or the full stack, the S3 API is served at `http://localhost:9000`, and the RustFS Console is at `http://localhost:9001`. Open the Console in a browser and log in with the access key and secret key you configured above. Generate a strong secret with, for example, `openssl rand -base64 24`, and never ship the placeholder values to production. For Docker Compose, define unique `RUSTFS_ACCESS_KEY` and `RUSTFS_SECRET_KEY` values in the `rustfs` service environment or in the environment file used for variable substitution, then recreate the service with `docker compose up -d rustfs`. ## Multi-node deployment [#multi-node-deployment] Docker's default bridge networking does not support multi-node deployments. Use `--network host` so each container can communicate directly with other nodes. Run the following on **each node**: ```bash docker run -d \ --name rustfs \ --network host \ -v /mnt/rustfs/data:/data \ -e RUSTFS_ACCESS_KEY="" \ -e RUSTFS_SECRET_KEY="" \ -e RUSTFS_ADDRESS=":9000" \ -e RUSTFS_CONSOLE_ADDRESS=":9001" \ -e RUSTFS_CONSOLE_ENABLE=true \ -e RUSTFS_OBS_LOGGER_LEVEL=error \ -e RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" \ -e RUSTFS_VOLUMES="http://node{1...4}:9000/data/rustfs{0...3}" \ rustfs/rustfs:latest ``` Add the entries to `/etc/hosts` on **every** node: ```ini title="/etc/hosts" 192.168.1.1 node1 192.168.1.2 node2 192.168.1.3 node3 192.168.1.4 node4 ``` ## TLS configuration [#tls-configuration] If [using TLS](../../integration/tls-configured.md), mount the certificate directory and point RustFS at it: ```bash -v /path/to/certs:/certs \ -e RUSTFS_TLS_PATH=/certs \ ``` ## Before production [#before-production] Work through the [Pre-Installation Checklists](../requirement/checklists/index.md) — hardware, network, software, and security — before deploying to production. Use a multi-node deployment architecture, [enable TLS encrypted communication](../../integration/tls-configured.md), configure a log rotation strategy, and set up a regular backup strategy. ## Next steps [#next-steps] * [RustFS Console](/administration/console) * [Configure an S3 client](../../developer/examples/aws-cli.md) * [TLS configuration](../../integration/tls-configured.md) # Container (/en/installation/container) RustFS is a high-performance, S3-compatible open-source distributed object storage system. In single-node single-disk (SNSD) deployment mode, the backend uses zero erasure coding without additional data redundancy, which makes it suitable for local testing and small-scale scenarios. The official RustFS image packages the RustFS binary and its runtime environment into a container, so you can start a service with a single command and persistent storage. The container runs as non-root user `rustfs` with id `10001`, so a bind-mounted host directory must be owned by `10001` to avoid permission denied errors. ## Container runtimes [#container-runtimes] * [Docker](./docker.md): run a single-node instance with persistent storage, then extend it with Docker Compose, TLS, and multi-node networking. * [Podman](./podman.md): run the same image in a daemonless workflow. If you mount a host directory into the container with `-v`, make sure the owner of the host directory is `10001`: ```bash chown -R 10001:10001 /path/to/host_directory ``` For Kubernetes deployments, see the [Helm chart](/installation/cloud-native/helm-chart) or the [Operator](/installation/cloud-native/operator) instead. # Podman (/en/installation/container/podman) Podman can run the official RustFS Open Container Initiative (OCI) image without a daemon. You need a working Podman installation and permission to create containers.
## Pull the image [#1-pull-the-image] ```bash podman pull docker.io/rustfs/rustfs:latest ```
## Create persistent storage [#2-create-persistent-storage] Create a named volume so object data remains available when you replace the container: ```bash podman volume create rustfs-data ``` The named volume above needs no extra setup. If you bind-mount a host directory instead, keep in mind that the container runs as non-root user `rustfs` with id `10001`. Make the host directory owned by `10001`, otherwise you will encounter permission denied errors: ```bash chown -R 10001:10001 /path/to/host_directory ```
## Start RustFS [#3-start-rustfs] Replace the credential placeholders before running the container: ```bash podman run -d \ --name rustfs \ -p 9000:9000 \ -p 9001:9001 \ -v rustfs-data:/data \ -e RUSTFS_ACCESS_KEY="" \ -e RUSTFS_SECRET_KEY="" \ -e RUSTFS_ADDRESS=":9000" \ -e RUSTFS_CONSOLE_ADDRESS=":9001" \ -e RUSTFS_CONSOLE_ENABLE=true \ -e RUSTFS_OBS_LOGGER_LEVEL=error \ -e RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" \ docker.io/rustfs/rustfs:latest \ /data ``` Set unique `RUSTFS_ACCESS_KEY` and `RUSTFS_SECRET_KEY` environment variables before exposing RustFS to a network. Do not use the well-known `rustfsadmin` value for either credential. If the container was started without custom credentials, stop and recreate it with both `-e` options shown above; the `rustfs-data` volume remains intact.
## Verify the deployment [#4-verify-the-deployment] Check the container and the S3 API health endpoint: ```bash podman ps --filter name=rustfs curl --fail http://localhost:9000/health ``` The S3 API is available at `http://localhost:9000`, and the Console is available at `http://localhost:9001`.
## Next steps [#next-steps] * [RustFS Console](/administration/console) * [Configure an S3 client](../../developer/examples/aws-cli.md) * [TLS configuration](../../integration/tls-configured.md) # Installation (/en/installation) RustFS is a distributed object storage system written in Rust and released under the Apache 2.0 license. It runs on Linux, Windows, macOS, FreeBSD, and containers, across x86, ARM, RISC-V, and other CPU architectures. After installation, set unique values for `RUSTFS_ACCESS_KEY` and `RUSTFS_SECRET_KEY` before exposing RustFS to a network. Do not use the well-known `rustfsadmin` value for either credential. * For a Linux binary or systemd installation, set both values in `/etc/default/rustfs`, then restart the `rustfs` service. * For Docker, Podman, or Docker Compose, pass both values as container environment variables when creating the container. Recreate an existing container if it was started without them. ## Choose Your Path [#choose-your-path] | Your goal | Recommended path | Guide | | ------------------------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | | Try RustFS on a local machine | One-command install script, or a container | [Linux Quick Start](./linux/quick-start.md) · [Container](./container/index.md) | | Single-server production | SNSD (one disk) or SNMD (multiple disks) | [Installing RustFS on Linux](./linux/index.md) | | Multi-server production cluster | MNMD, after completing the production checklists | [Multiple Node Multiple Disk](./linux/multiple-node-multiple-disk.md) · [Checklists](./requirement/checklists/index.md) | | Kubernetes / cloud-native | Container orchestration deployment | [Cloud Native](./cloud-native/index.md) | | Windows or macOS host | Native installation | [Windows](./windows/index.md) · [macOS](./macos/index.md) | ## Deployment Mode Comparison [#deployment-mode-comparison] | Mode | Nodes | Disks | Fault Tolerance | Typical Use | | ---------------------------------------------- | ----- | ----------------- | ------------------------------------------------------ | ------------------------------------------------------- | | [SNSD](./linux/single-node-single-disk.md) | 1 | 1 | None — rely on backups | Development, testing, low-density non-critical business | | [SNMD](./linux/single-node-multiple-disk.md) | 1 | Multiple | Up to M parity disks within the node | Medium, non-critical business on a single server | | [MNMD](./linux/multiple-node-multiple-disk.md) | 4+ | Multiple per node | Disk- and node-level via erasure coding across servers | Production workloads | ## Checklist [#checklist] Before any production deployment, work through the [Pre-Installation Checklists](./requirement/checklists/index.md) — hardware, network, software, and security — to make sure your environment meets production guidance. # Installing RustFS on Linux (/en/installation/linux) This section covers installing RustFS on Linux servers. For a one-command trial installation, use the [Quick Start](./quick-start.md). For a manual installation, pick one of the three deployment modes below — all three share the same [prerequisites and service setup](./prerequisites-and-service.md), and differ only in topology and volume configuration. After installation, set unique `RUSTFS_ACCESS_KEY` and `RUSTFS_SECRET_KEY` values in `/etc/default/rustfs`. Do not use the well-known `rustfsadmin` value for either credential. Restart the service with `sudo systemctl restart rustfs` after changing the file. ## Single Node Single Disk (SNSD) [#single-node-single-disk-snsd] One server, one data disk. The simplest mode, with no redundancy — a disk failure means data loss, so rely on backups. Suitable for development, testing, and low-density non-critical business. → [Single Node Single Disk installation](./single-node-single-disk.md) ## Single Node Multiple Disk (SNMD) [#single-node-multiple-disk-snmd] One server, multiple data disks. Erasure coding shards data across the disks, so the node tolerates a limited number of disk failures, but a whole-server failure still means data loss. Suitable for medium, non-critical business on a single machine. → [Single Node Multiple Disk installation](./single-node-multiple-disk.md) ## Multiple Node Multiple Disk (MNMD) [#multiple-node-multiple-disk-mnmd] Four or more servers, each with one or more disks. Erasure coding spans servers, providing disk- and node-level fault tolerance plus horizontal scalability. This is the mode for production workloads. → [Multiple Node Multiple Disk installation](./multiple-node-multiple-disk.md) ## Before Production [#before-production] Work through the [Pre-Installation Checklists](../requirement/checklists/index.md) — hardware, network, software, and security — before deploying to production. If you don't need production standards, you can skip them. # RustFS Multiple Node Multiple Disk Installation (/en/installation/linux/multiple-node-multiple-disk) Multiple Node Multiple Disk (MNMD) mode is the deployment mode for production workloads, providing enterprise-grade performance, security, and scalability. A minimum of **4 servers** is required, each with at least 1 disk, to safely start a distributed object storage cluster. ## Topology and Planning [#topology-and-planning] In the following architecture, requests are distributed to the servers through load balancing. With the default 12 + 4 erasure coding layout, each object is split into 12 data shards and 4 parity shards stored on different disks across different servers: * Any single server failure or maintenance does not affect data security. * Corruption of up to 4 disks does not affect data security. Before installation, review the [Pre-Installation Checklists](../requirement/checklists/index.md) and ensure all items meet production guidance. ## Hostnames [#hostnames] Creating a RustFS cluster requires **identical, sequential** hostnames. There are two ways to achieve sequential hostnames: **1. DNS Configuration:** Configure your DNS resolution server to ensure name continuity. **2. HOSTS Configuration:** Modify the local alias settings in `/etc/hosts` as follows: ```bash title="/etc/hosts" vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.1 node1 192.168.1.2 node2 192.168.1.3 node3 192.168.1.4 node4 ``` ## Prerequisites and Service Setup [#prerequisites-and-service-setup] On **every node**, complete the [common prerequisites and service setup](./prerequisites-and-service.md) — operating system, firewall, time synchronization, disk formatting, service user, binary download, and systemd unit — then continue below. Remember that all nodes must use the same listening port and must have synchronized clocks. ## Configure Environment Variables [#configure-environment-variables] 1. Create the same configuration file on every node. `RUSTFS_VOLUMES` uses brace expansion to enumerate all nodes and all disk mount points (this example: 4 nodes × 4 disks): ```ini title="/etc/default/rustfs" # Use a unique access key and a strong, random secret (e.g. openssl rand -base64 24) RUSTFS_ACCESS_KEY= RUSTFS_SECRET_KEY= RUSTFS_VOLUMES="http://node{1...4}:9000/data/rustfs{0...3}" RUSTFS_ADDRESS=":9000" RUSTFS_CONSOLE_ENABLE=true RUSTFS_CONSOLE_ADDRESS=":9001" RUSTFS_OBS_LOGGER_LEVEL=error RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/" ``` The access key, secret key, and `RUSTFS_VOLUMES` value must be identical on all nodes. The hostnames (`node1` – `node4`) must match the DNS or `/etc/hosts` configuration above. 2. Create the storage and log directories on every node: ```bash sudo mkdir -p /data/rustfs{0..3} /var/log/rustfs /opt/tls sudo chmod -R 750 /data/rustfs* /var/log/rustfs ``` ## Start Service and Verification [#start-service-and-verification] 1. Start the service on every node and enable auto-start on boot: ```bash sudo systemctl enable --now rustfs ``` 2. Verify the service status: ```bash systemctl status rustfs ``` 3. Check the service port: ```bash netstat -ntpl ``` 4. View log files: ```bash tail -f /var/log/rustfs/rustfs*.log ``` 5. Access the console: enter any node's IP address (or the load balancer address) and the console port (default 9001) in a browser. You should see: Console ## Next Steps [#next-steps] * Put a load balancer in front of the cluster — see the [Nginx integration guide](/developer/integration/reverse-proxy/nginx). * Enable TLS for production traffic — see [TLS configuration](../../integration/tls-configured.md). * Review [Pool Expansion](../../operations/scaling/storage-pool-expansion.md) before scaling. # Linux Prerequisites and Service Setup (/en/installation/linux/prerequisites-and-service) This page contains the prerequisites and service setup steps shared by all three Linux deployment modes — [SNSD](./single-node-single-disk.md), [SNMD](./single-node-multiple-disk.md), and [MNMD](./multiple-node-multiple-disk.md). Complete these steps first, then return to your mode page to configure the environment file and start the service. ## Operating System Version [#operating-system-version] We recommend Linux kernel version 4.x or later; versions 5.x/6.x achieve better I/O throughput and network performance. Ubuntu 22.04 and RHEL 8.x are both suitable for installing RustFS. ## Firewall [#firewall] Linux systems have firewalls enabled by default. Check the firewall status with: ```bash systemctl status firewalld ``` If your firewall status is "active", you can disable the firewall: ```bash systemctl stop firewalld systemctl disable firewalld ``` Or allow the RustFS S3 port (9000) and console port (9001): ```bash firewall-cmd --zone=public --add-port=9000/tcp --permanent firewall-cmd --zone=public --add-port=9001/tcp --permanent firewall-cmd --reload ``` All RustFS servers in a deployment **must** use the same listening port. If you use port 9000, every other server must also use port 9000. ## Memory Requirements [#memory-requirements] RustFS requires at least 2 GB of memory for a test environment; production environments require a minimum of 128 GB of memory. ## Time Synchronization [#time-synchronization] All nodes in a RustFS distributed deployment **must** maintain synchronized clocks. RustFS relies on timestamps for request signing, object versioning, distributed locking, and replication. Significant clock drift between nodes can cause: * **Request signing failures** — S3 signature verification depends on accurate timestamps. * **Replication and consistency issues** — Clock skew can lead to stale or conflicting object versions. * **Lock contention problems** — Distributed locks use timestamps for lease expiration. * **Service startup failures** — RustFS refuses to start if clock skew between nodes exceeds safe thresholds. Clock drift between any two nodes should not exceed **15 minutes**. For production environments, we recommend keeping drift under **1 second**. ### Recommended NTP Tools [#recommended-ntp-tools] Use any of the following time synchronization services on **every** node. Choose one and configure it consistently across the deployment. #### chrony (Recommended) [#chrony-recommended] `chrony` is the preferred NTP implementation for modern Linux distributions. It synchronizes faster and handles intermittent network connectivity better than legacy `ntpd`. Install chrony: ```bash # RHEL / CentOS / Rocky Linux sudo dnf install chrony -y # Ubuntu / Debian sudo apt install chrony -y ``` Edit the configuration file `/etc/chrony.conf` (RHEL) or `/etc/chrony/chrony.conf` (Debian/Ubuntu) to point to your preferred NTP servers: ```ini server time1.google.com iburst server time2.google.com iburst server time3.google.com iburst server time4.google.com iburst ``` > Replace the server addresses with your organization's internal NTP servers if available. Using `iburst` speeds up initial synchronization. Enable and start the service: ```bash sudo systemctl enable chronyd sudo systemctl start chronyd ``` #### systemd-timesyncd [#systemd-timesyncd] `systemd-timesyncd` is a lightweight SNTP client built into systemd-based distributions. It is suitable for environments where a full NTP daemon is not required. Edit `/etc/systemd/timesyncd.conf` to configure NTP servers: ```ini [Time] NTP=time1.google.com time2.google.com time3.google.com time4.google.com FallbackNTP=0.pool.ntp.org 1.pool.ntp.org ``` Enable and start the service: ```bash sudo timedatectl set-ntp true sudo systemctl enable systemd-timesyncd sudo systemctl start systemd-timesyncd ``` #### ntpd (Legacy) [#ntpd-legacy] The classic `ntpd` from the NTP reference implementation is still widely available. Use `chrony` instead unless your environment specifically requires `ntpd`. ```bash # RHEL / CentOS / Rocky Linux sudo dnf install ntp -y # Ubuntu / Debian sudo apt install ntp -y ``` Edit `/etc/ntp.conf` to set your NTP servers, then enable and start: ```bash sudo systemctl enable ntpd sudo systemctl start ntpd ``` ### Verifying Time Synchronization [#verifying-time-synchronization] After configuring your NTP service, verify synchronization on each node. Check the system clock status: ```bash timedatectl status ``` The output should show `System clock synchronized: yes` and `NTP service: active`. For `chrony`, use the following command to check detailed synchronization status: ```bash chronyc tracking ``` Key fields to verify: * **Leap status** — Should be `Normal` (not `Not synchronised`). * **System time** — The offset from the reference server. Should be close to `0.000000000 seconds`. * **Root delay** — Round-trip time to the reference server. To list the current NTP sources and their status: ```bash chronyc sources -v ``` Columns to watch: * **`*`** — The currently selected synchronization source. * **`+`** — Other acceptable sources. * **`-`** — Sources rejected by the selection algorithm. * **`?`** — Sources whose connectivity is in question. For `ntpd`, use: ```bash ntpq -p ``` ### Verifying Cross-Node Clock Consistency [#verifying-cross-node-clock-consistency] After all nodes are synchronized, verify that clocks are consistent across the cluster. On each node, compare timestamps: ```bash # Run on each node and compare the output date -u '+%Y-%m-%d %H:%M:%S' ``` For a more precise comparison, install `sshpass` and run: ```bash for host in node1 node2 node3 node4; do echo -n "$host: " ssh "$host" date -u '+%Y-%m-%d %H:%M:%S.%N' done ``` The difference between any two nodes should be negligible (under 1 millisecond in a well-configured environment). ## Capacity Planning [#capacity-planning] When planning object storage capacity, we recommend considering: * Initial data volume: How much data do you plan to migrate or store at once? (e.g., 500 TB) * Data growth volume: Daily/weekly/monthly data growth capacity * Planning cycle: How long should this hardware planning last? (recommended: 3 years) * Your company's hardware iteration and update cycles. Review [EC Configuration](../requirement/ec-configuration.md) to calculate usable capacity, understand the automatic parity defaults, and validate any explicit parity or erasure-set width before deployment. ## Disk Planning [#disk-planning] Because NFS generates phantom writes and lock issues under high I/O, **NFS is prohibited** as the underlying storage medium for RustFS. We strongly recommend **JBOD (Just a Bunch of Disks)** mode: expose physical disks directly and independently to the operating system, and let the RustFS software layer handle data redundancy and protection. The reasons are as follows: * **Better Performance:** RustFS's Erasure Coding engine is highly optimized and reads/writes multiple disks concurrently, achieving higher throughput than hardware RAID controllers. Hardware RAID becomes a performance bottleneck. * **Lower Cost:** No expensive RAID cards needed, reducing hardware procurement costs. * **Simpler Management:** RustFS manages disks uniformly, simplifying storage layer operations and maintenance. * **Faster Fault Recovery:** The RustFS healing process is faster than a traditional RAID rebuild and has less impact on cluster performance. We recommend NVMe SSDs as the storage medium for higher performance and throughput. ## File System Selection [#file-system-selection] RustFS strongly recommends formatting all storage disks with the XFS file system. RustFS development and testing are based on XFS, ensuring optimal performance and stability. Avoid other file systems such as ext4, BTRFS, or ZFS, as they may cause performance degradation or unpredictable issues. XFS suits RustFS's workload for three reasons: * **High-concurrency I/O:** XFS was designed for high performance and scalability. Its internal journaling and data structures (such as B+ trees) efficiently handle large numbers of parallel read/write requests, matching how RustFS shards large objects and reads/writes multiple disks in an erasure set in parallel. * **Massive files and large file sizes:** XFS is a 64-bit file system supporting extremely large files (up to 8 EB). Its metadata management stays efficient even with millions of files in a single directory — important because RustFS stores each object (or object version) as an independent file. * **Space reservation:** XFS provides an efficient `fallocate` API. RustFS uses it to reserve contiguous disk space before writing objects, avoiding the overhead of dynamic expansion and metadata updates during writes and minimizing file fragmentation. For better disk discovery, we recommend using **Label** tags when formatting XFS file systems. First, check the disk layout: ```bash sudo lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 465.7G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi └─sda2 8:2 0 465.2G 0 part / nvme0n1 8:16 0 3.7T 0 disk <-- if this is our format new disk nvme1n1 8:32 0 3.7T 0 disk <-- if this is our format new disk nvme2n1 8:48 0 3.7T 0 disk ``` Format each data disk: ```bash sudo mkfs.xfs -i size=512 -n ftype=1 -L RUSTFS0 /dev/sdb ``` Formatting options: * `-L