Configuration

The standalone server is configured with a YAML file passed as --config /path/to/config.yaml, plus a small set of environment variables for secrets and development toggles.

Zero-config quick start

For a single-node server you don't need a YAML file at all:

tinys3 server /data          # or `tinys3 server` for ./data
tinys3 server --address :9000 /data

This runs simple (single-filesystem) storage on the given directory. Root credentials come from TINYS3_ROOT_USER / TINYS3_ROOT_PASSWORD; if unset, a random secret is generated on first run, printed in the startup banner, and saved to <dir>/.tinys3-root so it survives restarts. Everything below — clusters, server pools, erasure, TLS — requires the YAML path (tinys3 server --config <file>), because a single directory can't express that topology.

YAML reference

server:
  port: 8000                       # listen port
  baseUrl: "http://localhost:8000" # advertised endpoint (presigned URLs, console)

auth:
  accessKey: "admin"               # root credential
  secretKey: "admin-secret"
  region: "us-east-1"

storage:
  mode: simple                     # simple | erasure
  path: /data                      # simple mode: data directory
  fsync: data                      # write durability: none | data (default) | full
  fullSync: false                  # optional: use the platform's strongest barrier (macOS F_FULLFSYNC) for the data flush

  # erasure mode instead of path:
  erasure:
    dataShards: 4                  # data shards per stripe
    parityShards: 2                # parity shards — tolerates this many disk losses
    chunkSize: 16777216            # optional, bytes (default 16 MB)
    disks:
      - /disk1
      - /disk2
      - /disk3
      - /disk4
      - /disk5
      - /disk6

  # OR standalone multi-pool (takes precedence over mode/erasure/path): several LOCAL pools on one
  # server, no cluster. A bucket pins to a pool by name via LocationConstraint (see Bucket pool
  # affinity). Each pool is independent — erasure over its own disks, or plain files on one path.
  pools:
    - name: fast
      mode: erasure
      erasure: { dataShards: 4, parityShards: 2, disks: [/disk1, /disk2, /disk3, /disk4, /disk5, /disk6] }
    - name: bulk
      mode: disk                   # plain files on a single path (capacity, not redundancy)
      path: /mnt/bulk

healing:                           # erasure mode only
  enabled: true                    # background healing (default true)
  scanIntervalMinutes: 60          # scan cadence (default 60)
  maxConcurrentHeals: 4            # parallel heals (default 4)
  healOnStartup: false             # scan immediately on boot (default false)

logging:
  file: /var/log/tinys3/server.log # one rotating log file; omit for console-only
  level: info                      # trace|debug|info|warn|error (default info)
  console: true                    # also log to stdout (default true)

audit:
  enabled: true                    # off by default
  file: /var/log/tinys3/audit.jsonl        # JSON-lines audit file; omit for no file target

telemetry:
  tracing:
    enabled: false                 # OpenTelemetry (OTLP) tracing; off by default
    endpoint: http://localhost:4317 # OTLP collector endpoint
    protocol: grpc                 # grpc | http/protobuf
    serviceName: tinys3            # service.name resource attribute
    sampleRatio: 1.0               # 0.0–1.0 trace sampling
    captureObjectKeys: false       # include the object key on spans (off: keys can be sensitive)

Tracing (telemetry.tracing)

When enabled, TinyS3 exports one OpenTelemetry SERVER span per S3 request — operation (REST.GET.OBJECT, …), bucket, key, HTTP status, and latency — to an OTLP collector. It is off by default: with enabled: false no exporter is created and no connection is opened, so there is no cost. Point endpoint at a collector (e.g. the OpenTelemetry Collector on :4317 for gRPC) and set enabled: true. Environment overrides for containers: TINY_S3_OTEL_ENABLED and TINY_S3_OTEL_ENDPOINT take precedence over the YAML.

Object keys are not exported by default: they are unbounded-cardinality and can embed sensitive data. Set captureObjectKeys: true only if you accept that object keys will be sent to the collector; the bucket, operation, status, and latency are always included.

Logging

Server logging is configured entirely from the logging: block (or its environment overrides) — no logback.xml to edit. Three settings:

Key	Default	Effect
logging.file	(unset)	Path to the log file. When set, logs go to this rotating file. When omitted, TinyS3 logs to the console only — the zero-config and Docker default, so docker logs / systemd-journald capture everything.
logging.level	info	Root level: trace, debug, info, warn, or error.
logging.console	true	When a file is configured, also mirror logs to stdout. Set false for file-only.

Environment overrides (take precedence over the YAML — handy for containers) — TINY_S3_LOG_FILE overrides logging.file, TINY_S3_LOG_LEVEL overrides logging.level:

TINY_S3_LOG_LEVEL=debug tinys3 server /data          # verbose logs for troubleshooting
TINY_S3_LOG_FILE=/var/log/tinys3/server.log tinys3 server /data

Format. Each line is 2026-08-20 14:03:11 - INFO - dev.totis.tinys3.S3Handler - <message> (timestamp - level - logger - message).

Rotation. The file appender rolls when it reaches 100 MB or at the day boundary, whichever comes first; archives are gzip'd as <file>.<date>.<n>.gz, 14 days of history are kept, and the total archive size is capped at 2 GB (oldest deleted first). The appender is asynchronous — a log write only enqueues onto a bounded buffer, so logging never blocks a request thread and adds no measurable throughput cost.

Audit logging (audit)

One structured JSON record per S3 request, for security/compliance retention — separate from both the server log (logging:) and S3 access logs (?logging). Off unless audit.enabled and at least one target is set.

Key	Default	Effect
audit.enabled	false	Master switch. When off there is zero per-request cost.
audit.file	(unset)	Path to a JSON-lines file; created owner-only (0600) where the FS supports it. One entry per line.
audit.bucket	(unset)	A local bucket to ingest the trail into as a tamper-evident WORM archive. It must already have versioning + object lock enabled or the config is rejected.
audit.bucketPrefix	audit/	Key prefix under which archive objects and manifest seals are written.
audit.retentionDays	30	GOVERNANCE object-lock retention stamped on each archived object. Clamped to [1, 36500].
audit.deliveryIntervalSeconds	60	How often the spill is flushed into a new rotated archive object.

Environment overrides: TINY_S3_AUDIT_ENABLED, TINY_S3_AUDIT_FILE.

WORM archive bucket. When audit.bucket is set, the trail is ingested into a versioned, object-lock (GOVERNANCE) bucket as immutable, time-partitioned .jsonl objects — a tamper-evident archive. Each object opens with a hash-chain header ({prevKey, prevObjectSha256}) and every flush also writes a small retained manifest seal object, so an offline chain walk detects a missing or altered object. Delivery is at-least-once from a locally fsynced spill (a crash re-delivers the tail; duplicates carry requestId); the spill is size-capped and self-limiting — a genuine loss is dropped-and-counted as a sealed AUDIT_GAP record, never a silent gap. The bucket is written through the internal facade, so ingesting the archive is not itself audited (no self-amplifying loop). GOVERNANCE (not COMPLIANCE) retention means a s3:BypassGovernanceRetention principal can still rewrite history — the archive is tamper-evident against outside/at-rest tampering, not tamper-proof against a root operator (see docs/audit-to-locked-bucket-plan.md).

Live tail & archive browser. The console Audit page adds a Live tail tab (a polling view of the most recent audit entries from an in-memory ring, via GET /audit/tail?since=) and an Archive tab (a paginated browser of the WORM bucket via GET /audit/archive, with a record-level object viewer via GET /audit/archive/object?key=). Both are root-only.

Runtime management. Beyond the audit: block (the boot seed), auditing can be turned on/off and its targets changed without a restart from the console (Audit page) or the root-only admin REST GET/PUT /rest/api/v1/audit. Runtime changes are persisted and reload on boot, overriding the YAML/env seed. In a cluster the change applies to the node that received it — configure each node (or rely on each node's persisted config after restart).

Record fields: version, time, requestId, operation, bucket, object, statusCode, errorCode, timeToResponseMs, bytesReceived, bytesSent, remoteHost, userAgent, principal, tls. Credentials, signatures, and encryption keys are never recorded; principal is the caller's identity (or anonymous).

Scope & guarantees. Audit covers the S3 request surface (object/bucket operations), including anonymous and failed requests; the ops namespace (/tinys3/health, /tinys3/metrics) is excluded as noise. Delivery is best-effort and off the request thread — under sustained backpressure entries are dropped rather than slowing a request, and a dropped run is surfaced in the file as a synthetic AUDIT_GAP record so a gap is never silent. The file target flushes each line (a process crash loses at most the in-flight line; a host/power crash may lose unsynced lines).

Security & IP access control (security)

Front-door controls applied on the resolved client IP before routing or auth, for both S3 and admin requests.

security:
  trustedProxies: ["10.0.0.0/8"]   # CIDRs whose forwarding headers are trusted
  blocklistDisabled: false         # kill-switch for the IP blocklist
  allowlistDisabled: false         # kill-switch for the IP allowlist

Key	Default	Effect
security.trustedProxies	(empty)	CIDRs (or bare IPs) of reverse proxies whose X-Forwarded-For / X-Real-IP / X-Forwarded-Proto headers are trusted. Empty trusts nothing and always uses the socket peer — a direct client can never spoof its source. Env: TINY_S3_TRUSTED_PROXIES (comma-separated).
security.blocklistDisabled	false	When true the IP blocklist is not constructed or enforced. Recovery lever. Env: TINY_S3_BLOCKLIST_DISABLED.
security.allowlistDisabled	false	When true the IP allowlist is not constructed or enforced. Recovery lever. Env: TINY_S3_ALLOWLIST_DISABLED.

IP blocklist (default-allow) bans listed IPs/CIDRs; IP allowlist (default-deny, only when its persisted enabled flag is on) admits only listed IPs/CIDRs. A block always wins over an allow, and loopback always passes so a local operator is never locked out. Both are managed from the console (Settings → IP Access Control) and the admin API, persist cluster-wide, and take effect within seconds — no restart.

Behind a reverse proxy

Set security.trustedProxies to your proxy's CIDRs before enforcing an allowlist (or relying on the blocklist). Without it, every request resolves to the proxy's IP, so the whole world is treated as one address — an allowlist would admit or deny everyone at once. Only forwarding headers from a trusted-proxy peer are honored.

Self-lockout recovery (loopback bypass, kill-switches, on-disk reset, the allowlist enable-guard) is detailed in Operations.

Durability (storage.fsync)

How durably a write is flushed before its success is acknowledged:

Value	Behaviour
none	No flush; page-cache durable only. Fastest. A power loss can lose a just-acked write; a cluster's erasure + self-healing backstops a single node's loss.
data	Default. fdatasync the object bytes before it becomes visible. A visible object always has durable data; a lost rename is rebuilt by erasure healing. The default durability guarantee.
full	Also fsync the directory after the rename, so the rename is crash-durable. Strictest, slowest on small objects.

The default is data — durable by default: object bytes are fdatasync'd before the object becomes visible, so an acknowledged write survives a power loss. Set none to opt into the fastest, page-cache-only behavior (an acknowledged write can then be lost on power loss) where you don't need crash durability. full additionally makes the rename crash-durable. In a cluster, erasure + self-healing further backstops a single node's power loss (see Self-healing & data integrity).

Stronger barrier (storage.fullSync)

data/full flush the object bytes with fdatasync (Linux) / fsync (macOS) — drive-level durability, matching the reference implementations. On macOS the two syscalls diverge: the plain fsync does not force the drive's own write cache to platter, while F_FULLFSYNC does (at a large throughput cost). Set storage.fullSync: true (or TINY_S3_FSYNC_FULLSYNC=true) to make the data flush use the platform's strongest barrier — F_FULLFSYNC on macOS. Default false. On Linux fdatasync is already the strongest data barrier, so the flag changes nothing there.

TLS is configured with a certificate and key — see the config-tls.yaml example in app/examples/.

External KMS (kms)

Server-side-encrypted objects (SSE-S3 / SSE-KMS) can be protected with envelope encryption: each object gets a random data key, encrypted with the object, and that data key is wrapped by a KMS provider and stored (wrapped) in the object's metadata. The provider is pluggable and opt-in — declare a kms: block to enable it:

kms:
  provider: local          # local | bitwarden
  options: {}              # provider-specific settings

Envelope encryption is off unless a kms: block is present. With no block, SSE uses the legacy master-key-derived scheme unchanged — so existing deployments are unaffected, and (critically) a node must not start writing envelope objects while peers still run older code. Enable KMS only after every node in a cluster is upgraded (a mid-upgrade old node would rewrite an envelope object's metadata without the envelope and lose its data key).

Provider	Root key	Notes
local	TINY_S3_SSE_MASTER_KEY	Wraps data keys under the SSE master key. Keep the master key configured even after switching to another provider, so existing local-wrapped objects stay readable.
bitwarden	Bitwarden Secrets Manager	The 32-byte root key is stored as a Secrets Manager secret and fetched once at startup with a machine-account access token, then data keys are wrapped locally.

Bitwarden options:

kms:
  provider: bitwarden
  options:
    secretId: "<uuid of the secret holding the base64 root key>"
    apiUrl: "https://api.bitwarden.com"        # optional (self-hosted)
    identityUrl: "https://identity.bitwarden.com"  # optional (self-hosted)

The machine-account access token is a secret — supply it via the BWS_ACCESS_TOKEN environment variable (preferred) rather than the config file. Objects written before envelope encryption (no wrapped key in their metadata) still decrypt via the legacy master-key path, and an object always records which provider wrote it, so changing the default later never orphans existing data.

Multi-rack clusters

A cluster that spans racks (or power feeds, switches, availability zones) declares each node's failure domain with cluster.racks — one entry per node, in the same order as cluster.nodes:

cluster:
  name: "prod"
  nodes: ["r1n1:9101", "r2n1:9102", "r3n1:9103", "r1n2:9104", "r2n2:9105", "r3n2:9106"]
  self: "r1n1:9101"
  racks: [0, 1, 2, 0, 1, 2]   # nodes 0,3 in rack 0; 1,4 in rack 1; 2,5 in rack 2

Placement then composes each erasure set so a single rack holds at most the parity budget of any object's shards — so losing a whole rack loses no data, and reads keep serving from the surviving racks. The console's Cluster page shows the resulting rack fault tolerance (how many whole racks can go down safely); the boot log prints it too. With no racks, every node is its own failure domain (node-level placement, unchanged).

For the guarantee to hold you need enough racks that ceil(totalShards / rackCount) ≤ parityShards — e.g. a 4+2 scheme needs ≥ 3 racks to survive one rack loss. Writes additionally require the surviving racks to retain a write quorum of each set.

Server pools (expansion and decommission)

Capacity changes online through server pools (available in every deployment — single edition, no feature gating; a cluster of more than 2 nodes must set license.enterprise: true). Each pool is an independent placement domain with its own nodes and racks; declare them with cluster.pools instead of cluster.nodes — every node of every pool uses the same config and finds its own pool by its address:

cluster:
  name: "prod"
  self: "p1n1:9101"
  pools:
    - name: rack-a             # optional; a bucket can pin to this pool by name (see below)
      nodes: ["p0n1:9101", "p0n2:9102", "p0n3:9103"]
      racks: [0, 1, 2]          # optional, per pool
    - name: rack-b
      nodes: ["p1n1:9101", "p1n2:9102", "p1n3:9103"]
      disksPerNode: 4           # optional; defaults to this node's local disk count

• name is optional. It gives the pool an operator-facing name a bucket can be pinned to at creation time (see Bucket pool affinity). A pool without a name is unnamed and can never be a pin target; older pool configs without any name boot unchanged.
• Expansion: add a pool to the list and restart the nodes. Existing objects never move. Every bucket lives in exactly one pool — a bucket's objects are never striped across pools — so an added pool receives only buckets explicitly pinned to it (unpinned buckets stay on the default pool).
• An overwrite follows its object to the pool that holds it — an object's versions stay together.
• Decommission (retire a pool): POST /rest/api/v1/pools/{name}/decommission (root-only, keyed by pool name), or the pool card in the console. The pool stops taking new writes but keeps serving reads while a background job migrates every bucket resident on it — pinned or not — to the emptiest surviving pool (source deleted only after the copy is size- and CRC-verified), then rewrites each migrated bucket's pool pin to the target. Progress at GET /rest/api/v1/pools/{name}/decommission. There is no cancel operation, and a drain is not auto-resumed on boot: if it is interrupted (pool left DECOMMISSIONING), re-issue the same POST to finish it — the migration is idempotent. Per-pool state (ACTIVE/DECOMMISSIONING/DECOMMISSIONED) is persisted, so a decommissioned pool does not revert to active across a restart while still in config. When the pool reports DECOMMISSIONED, remove it from cluster.pools.
• Starting a drain is refused unless the surviving pools have the capacity (plus headroom) and keep their rack fault tolerance.

Bucket pool affinity

A named pool is a placement domain — either a set of cluster nodes (cluster.pools) or, on a single server, a local disk group (storage.pools, above). You can pin a bucket to one pool at creation time so all of that bucket's data lives entirely within that pool. Pinning uses S3's LocationConstraint: if the CreateBucketConfiguration value exactly matches a configured pool name, the bucket is pinned to that pool.

aws --endpoint-url http://localhost:8000 s3api create-bucket \
    --bucket photos \
    --create-bucket-configuration LocationConstraint=rack-a

• Every bucket lives in exactly one pool. A pinned bucket's objects all land in its pool; an unpinned bucket's objects all land in the default pool (the first configured/active pool). A bucket's data is never striped across pools — there is no "auto"/spread placement.

• GetBucketLocation echoes the pinned pool name back.

• Pinning is permission-gated: because a pin concentrates a bucket's data in one fault domain, only a configured root/operator credential or a principal explicitly granted s3:PutBucketPoolAffinity may pin. Any other caller sending a pool name in LocationConstraint is refused with 403 AccessDenied at create. (Real region values never match a pool name, so region-configured clients are unaffected.) Example grant:

  {"Version":"2012-10-17","Statement":[{"Effect":"Allow",
   "Action":["s3:PutBucketPoolAffinity"],"Resource":["arn:aws:s3:::*"]}]}

• Accept-and-ignore for non-matches: a LocationConstraint that is not a configured pool name (a real AWS region such as eu-west-1, or a typo) is not rejected — the bucket is created unpinned and the value is still stored as its location. This keeps region-configured S3 clients working unchanged.

• Fixed by normal operation: a bucket's pool is set at creation and there is no client-facing reassignment operation — you cannot re-pin a bucket via the S3 API. The one exception is decommission: draining a pool migrates every bucket on it (pinned or not) to a surviving pool and rewrites the pin to the target. That is the only path by which a pinned bucket changes pools.

Commercial license flag (clusters larger than 2 nodes)

TinyS3 is a single edition with the full feature set; the only boundary is cluster size. A cluster of 2 nodes or fewer needs nothing. A cluster of more than 2 nodes must acknowledge the commercial terms in LICENSE.enterprise:

license:
  enterprise: true
  organization: "Acme Corp"   # optional; surfaced in the boot notice

This is an honor-system config flag, not a cryptographic license. Without it, a >2-node cluster refuses to boot. Setting it flips the reported edition label to enterprise (visible on GET /server/info and in the console) and logs a notice that the cluster operates under LICENSE.enterprise. The environment variable TINY_S3_LICENSE_ENTERPRISE=true sets the same flag.

Environment variables

Variable	Purpose
TINY_S3_SSE_MASTER_KEY	Base64 master key enabling SSE-S3 and SSE-KMS. Kept out of the config file so ciphertext and key are never written down together. Without it, requests demanding server-side encryption are refused with 501 rather than stored in cleartext.
TINY_S3_DISK_ENCRYPTION_KEY	Base64 key for at-rest encryption of erasure shards (AES-256-CTR + keyed integrity MAC). A lost disk reveals nothing and cannot be tampered with undetected.
BWS_ACCESS_TOKEN	Bitwarden Secrets Manager machine-account access token, used when kms.provider: bitwarden. Kept in the environment so the token is never written to the config file.
TINY_S3_CORS_ALLOW_ORIGINS	Comma-separated origins for whole-server CORS. Development only — used when the console runs on its own port (npm run dev). The production console is same-origin and needs none of this.
TINY_S3_ERASURE_ENABLED	true enables erasure coding when configuring via environment instead of YAML.
TINY_S3_ERASURE_DISKS	Comma-separated disk paths (env-based erasure config).
TINY_S3_ERASURE_DATA_SHARDS	Data shard count (default 4).
TINY_S3_ERASURE_PARITY_SHARDS	Parity shard count (default 2).
TINY_S3_OTEL_ENABLED	true enables OpenTelemetry tracing; overrides telemetry.tracing.enabled.
TINY_S3_OTEL_ENDPOINT	OTLP collector endpoint; overrides telemetry.tracing.endpoint.
TINY_S3_LOG_FILE	Log file path; overrides logging.file.
TINY_S3_LOG_LEVEL	Log level; overrides logging.level.
TINY_S3_AUDIT_ENABLED	true turns on audit logging; overrides audit.enabled.
TINY_S3_AUDIT_FILE	JSON-lines audit file path; overrides audit.file.
TINY_S3_LICENSE_ENTERPRISE	true acknowledges the commercial terms for a cluster larger than 2 nodes; overrides license.enterprise. Honor-system flag, not a cryptographic license.
TINY_S3_FSYNC	Durability policy (`none
TINY_S3_FSYNC_FULLSYNC	true makes the data/full flush use the platform's strongest barrier (macOS F_FULLFSYNC); overrides storage.fullSync. Default false.

IAM state

IAM identities (users, groups, roles, policies, access keys) are persisted to a state file inside the storage path (.tinys3-iam.json) so they survive restarts. The root credential from auth: is the account root; everything else is created through the IAM API or the console.

Precedence

Environment variables override YAML where both configure the same thing. Secrets (TINY_S3_SSE_MASTER_KEY, TINY_S3_DISK_ENCRYPTION_KEY) are environment-only by design.