gitsyncd

gitsyncd is a Go sidecar daemon that replicates git repositories across Forgejo pods via git bundle streaming over gRPC. It eliminates the need for NFS/shared storage in multi-replica Forgejo deployments, giving each pod a local-path PVC with real-time replication between peers.

Works with any git server that fires post-receive hooks. Designed for and tested with Forgejo on Kubernetes. Currently is used on src.krea.to.

• Multi-writer: all pods accept pushes. Git's non-fast-forward rejection handles conflicts. No leader election.
• Live-peer quorum: push returns only after all currently-live peers ACK. Down peers are excluded from quorum.
• Async S3 WAL (optional): durable checkpoint intents are queued after peer replication; full bundles are built and uploaded in the background, so S3 adds no bundle-build latency to pushes.
• Background reconciler: 30s interval safety net catches missed replications and bootstraps new nodes.

How it works

User pushes → Forgejo updates refs → post-receive hook fires
  → shell script sends ref data to gitsyncd via Unix socket
  → gitsyncd creates an incremental git bundle (objects since the old ref)
  → streams bundle via gRPC to all live peers concurrently
  → waits for all live peers to ACK
  → returns OK to hook → user sees successful push

Async (background): a full checkpoint bundle is built and copied to S3 for down-node recovery

Quick start

Build:

docker build -t gitsyncd .

Deploy alongside Forgejo as a StatefulSet sidecar. [TBD]

Architecture

            Forgejo Pod
  +-----------------------------+
  |  Forgejo     gitsyncd       |
  |  reads git     (Go daemon)  |
  |  from local                 |
  |       \            /        |
  |        local-path PVC       |
  +-----------------------------+
       | gRPC :9999    | socket
       v               v
   headless service   async S3
   (peer discovery)   WAL upload

gRPC protocol

service GitSync {
  rpc ReplicatePush(stream PushChunk)       returns (Ack);
  rpc ReplicateRepoCreate(stream Bundle)    returns (Ack);
  rpc ReplicateRepoDelete(RepoID)           returns (Ack);
  rpc GetRepoHashes(Empty)                  returns (RepoHashes);
  rpc FetchRepo(RepoID)                     returns (stream BundleChunk);
  rpc HealthCheck(Empty)                    returns (HealthStatus);
}

Configuration

All via environment variables:

Variable	Default	Description
GITSYNCD_REPOS_DIR	/repos	Path to git repos root
GITSYNCD_SOCKET	/shared/gitsyncd.sock	Unix socket shared with Forgejo
GITSYNCD_GRPC_PORT	9999	gRPC listen port
GITSYNCD_METRICS_ADDR	:9100	Prometheus HTTP listen address; empty, 0, or :0 disables
GITSYNCD_PEER_TIMEOUT	5s	Per-peer ACK timeout
GITSYNCD_PEER_HEALTH_TIMEOUT	2s	Health check timeout
GITSYNCD_RECONCILE_INTERVAL	30s	Reconciler loop interval
POD_IP	(required)	Pod IP for self-identification
GITSYNCD_S3_*	(unset)	S3 WAL config (endpoint, bucket, region, key, secret)

Prometheus metrics

When enabled, Prometheus can scrape http://<pod>:9100/metrics. Metrics include RPC and hook request rates/latencies, replication outcomes and bytes, peer health, reconciliation activity, and S3 WAL queue/upload status. Labels are kept low-cardinality and do not include repository names, ref names, peer IPs, or error strings. Configure a Kubernetes ServiceMonitor or equivalent scrape target for the metrics address.

Project structure

cmd/gitsyncd/         daemon entrypoint
cmd/gitsyncd-hook/    minimal hook binary
internal/
  server/             gRPC server, metrics endpoint, Unix socket listener, health
  metrics/            Prometheus collectors and application instrumentation
  replicate/          push, create, apply, quorum
  s3wal/              async WAL queue, uploader, S3 replay
  reconcile/          background reconciler (hash compare + sync)
  discovery/          headless service peer discovery
  git/                bundle, refs, hook helpers
  config/             env var config
proto/gitsync.proto   gRPC service definition
docs/design.md        full architecture spec

License

MIT