Bucket replication

Server-side, rule-driven replication of a bucket's objects to another bucket — on the same server or on a remote S3 endpoint. Asynchronous by default (or synchronous per rule — see Replication mode) and durable: work is enqueued to a persistent, cluster-safe queue drained by a background service, retried with exponential backoff, and re-queued by a scanner (and on GET/HEAD access) until it lands.

Requirements

• Versioning must be enabled on the source bucket. PutBucketReplication returns InvalidRequest otherwise.
• For a remote destination, a saved endpoint (URL + credentials) created via the admin API /endpoints resource or the console. Endpoints are persisted across restarts with encrypted credentials.

Rules

Configuration is the standard S3 ReplicationConfiguration XML via PutBucketReplication / GetBucketReplication / DeleteBucketReplication:

• Status — Enabled / Disabled per rule.
• Priority — when several enabled rules match a key, the single highest-priority rule wins (S3 semantics; no fan-out).
• Filter/Prefix — key-prefix scoping.
• DeleteMarkerReplication — opt in/out of delete-marker propagation.
• Destination — the target bucket:
  • Local: a bucket name on the same server, owned by the same account.
  • Remote: a saved endpoint referenced as arn:tinys3:replication:<endpointId>:<bucket> (the legacy <Destination><Endpoint>ENDPOINT_ID</Endpoint> form still works).

<ReplicationConfiguration>
  <Rule>
    <ID>to-dr</ID>
    <Status>Enabled</Status>
    <Priority>1</Priority>
    <Filter><Prefix>logs/</Prefix></Filter>
    <DeleteMarkerReplication><Status>Enabled</Status></DeleteMarkerReplication>
    <Destination><Bucket>arn:tinys3:replication:ep-123:backup</Bucket></Destination>
  </Rule>
</ReplicationConfiguration>

What replicates

• Object creates — PUT, COPY and multipart uploads.
• Metadata mutations — tag, retention, legal-hold and ACL changes.
• Delete markers — when DeleteMarkerReplication is enabled. Version-specific deletes are not propagated (matching AWS): deleting a version at the source leaves the destination's versions intact.
• Existing objects — when a rule is created, existing versioned objects that match it are enqueued. Objects without a version id are skipped.

Per-object status is exposed on GET/HEAD as x-amz-replication-status: PENDING, COMPLETED, FAILED, or REPLICA (the object is a copy written by replication).

Two-way (active-active)

Replicas are marked REPLICA and are not replicated onward, so a pair of buckets can replicate to each other without loops. The replica marker is only honored when the writing principal is explicitly granted s3:ReplicateObject / s3:ReplicateDelete — a regular client cannot forge it.

Encryption

• SSE-S3 / SSE-KMS objects are decrypted at the source and re-encrypted at the destination according to its configuration. Sending one to a remote destination requires an https endpoint — the server refuses to put plaintext on a cleartext link.
• SSE-C objects are not replicated (the server never has the key).
• Unencrypted objects travel as-is; use an https endpoint unless the link is trusted.

Resync (disaster recovery)

Re-queue a bucket's source objects, skipping those already matching the destination:

curl -X POST -H "X-API-Key: admin" -H "X-API-Secret: admin-secret" \
  http://localhost:8000/rest/api/v1/buckets/mybucket/replication/resync

Root-only admin operation; returns the number of objects enqueued.

IAM

s3:PutReplicationConfiguration and s3:GetReplicationConfiguration gate rule management. Ready-made policies ship in the distribution:

• app/examples/replication-admin-policy.json — manage replication on a bucket.
• app/examples/replication-remote-user-policy.json — the identity a remote source uses to write replicas in (grants s3:ReplicateObject/s3:ReplicateDelete and friends).

Console

The bucket-settings Replication tab manages both local (same-server) and remote S3 targets (using saved endpoints and ARNs), and allows triggering an existing-objects DR resync.

Replication mode (async / sync)

A rule replicates asynchronously by default — the write returns immediately and replication happens in the background. Add <Mode>SYNC</Mode> to a rule to replicate synchronously: the write attempts replication inline, before the client is acknowledged, giving a tighter window for a client that needs the remote copy to exist on return.

<Rule>
  <Status>Enabled</Status>
  <Mode>SYNC</Mode>
  <Filter><Prefix>critical/</Prefix></Filter>
  <Destination><Bucket>dest</Bucket></Destination>
</Rule>

Sync is store-and-fall-back, never fail-the-write: the object is always durably stored locally first, then replication is attempted inline up to a timeout (default 10s, override with -Dtinys3.replication.syncTimeoutMs). If it succeeds, the write returns with the object already at the destination and status COMPLETED. If it fails or exceeds the timeout, the write still returns success and the object is left on the durable queue for the async workers to retry — so a slow or down destination never blocks or fails writes to the source. Mode applies to object creates; deletes and metadata changes always replicate asynchronously.

Limitations

• Delete markers only — version-specific deletes are not propagated (as in AWS).
• SSE-C objects are not replicated.
• One destination per key (highest-priority rule); multi-site fan-out and edge replication are not implemented.