Favicon of SeaweedFS

SeaweedFS

SeaweedFS is an open-source distributed store built on Facebook's Haystack design, keeping per-file metadata overhead to about 40 bytes so it can serve billions of small files with a single disk seek.

LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hosted
PricingFree
Operational complexityMedium
LanguageGo

Use it when

  • The workload is billions of small files (images, thumbnails, documents), where per-file metadata defeats general stores; the Haystack design keeps it to about 40 bytes and one disk seek per read.
  • Many access paths are needed over one store: S3, POSIX FUSE mounts, WebDAV, and an HDFS interface.
  • A central index must not become the bottleneck; the master tracks volumes, not files.
  • Deployment should start small: one Go binary runs master, volume, filer, and S3 roles on a laptop and grows to a cluster.

Think twice when

  • The workload is large Parquet scans; a lakehouse's access pattern is not what the design optimizes, even though the S3 API is complete enough to serve one.
  • Strict multi-site consistency is needed; filer.sync replication is asynchronous, one-way or two-way.
  • You want hyperscaler-grade attestation of durability rather than replication codes and erasure coding you configure.
  • The S3 path must be minimal: it runs through the filer, which is an extra component to size.

How it runs

Self-hosted Go processes: a master for volume placement, volume servers holding blob data with local metadata, and an optional filer adding directories, POSIX semantics, and the S3 API. Durability comes from replication codes spanning servers, racks, and datacenters, or RS(10,4) erasure coding for warm data; tiering offloads to S3, GCS, or Azure.

Details

Compare

How SeaweedFS answers the questions Object Storage turns on.

Object Storage
How it works
S3 APICompatible, via the filer
Durability modelReplication codes for server, rack and DC copies, or RS(10,4) for warm data
Event notificationsFiler events to Kafka, SQS, Pub/Sub, NATS or RabbitMQ
Running it
Minimum durationNone
Multi-regionfiler.sync: asynchronous one-way or two-way between clusters
Minimum deploymentOne Go binary: master, volume, filer and S3 roles on a laptop or a cluster
Connections
InterfacesObject, POSIX FUSE, WebDAV, HDFS
Access
ImmutabilityS3 Object Lock: governance or compliance mode, plus legal holds
Cost
EgressNone: self-hosted
Storage tiersDisk types you tag (hdd, ssd, nvme); tiering to S3, GCS or Azure

Share:

Alternatives to SeaweedFS

Favicon

 

  
  
Favicon

 

  
  
Favicon