repo-sync/README.md

repo-sync

A universal Linux package repository synchronization tool. It mirrors remote repositories into a local directory tree, preserving the upstream layout so the result can be served directly to package managers.

Supported repository types:

• rpm — yum/dnf repositories (repodata/repomd.xml), including plain-text mirrorlist and metalink URLs with failover between mirrors.
• deb — apt repositories, both standard (dists/<suite> with a shared pool/) and flat layouts, including Acquire-By-Hash population.
• arch — pacman repositories (<name>.db), including detached package signatures and companion metadata (.files, .db.tar.gz, .links.tar.gz).
• apk — Alpine Linux repositories (APKINDEX.tar.gz with packages beside it).

Usage

repo-sync <type> [flags] <url> [<url> ...] <destination-directory>

The last argument is always the destination directory; every argument before it is a repository or mirrorlist URL, synchronized one after the other.

# Mirror one repository. The URL path is copied below the destination, so
# this produces ./mirror/repos/CentOS/7/EA4/.
repo-sync rpm https://example.com/repos/CentOS/7/EA4/ ./mirror

# Trim the first two path components: ./mirror/7/EA4/.
repo-sync rpm --trim 2 https://example.com/repos/CentOS/7/EA4/ ./mirror

# No path copying at all: the repository lands directly in ./mirror.
repo-sync rpm --flat https://example.com/repos/CentOS/7/EA4/ ./mirror

# Multiple repositories in one run.
repo-sync rpm https://example.com/repos/a/ https://example.com/repos/b/ ./mirror

# Crawl directory listings for repositories, at most 4 levels deep.
repo-sync rpm --discover --depth 4 https://example.com/repos/ ./mirror

# A pacman repository. The database name is discovered from the directory
# listing, or by probing URL path segments when listings are disabled.
repo-sync arch https://example.com/archlinux/core/os/x86_64/ ./mirror

# An Alpine repository is one arch directory; discovery syncs every arch
# of a release/repo tree in one run.
repo-sync apk https://dl-cdn.alpinelinux.org/alpine/v3.24/community/x86_64/ ./mirror
repo-sync apk --discover --depth 1 https://dl-cdn.alpinelinux.org/alpine/v3.24/community/ ./mirror

# A Fedora-style metalink URL; mirrors are used in preference order. The
# mirrors' paths differ, so --flat keeps the destination stable.
repo-sync rpm --flat 'https://mirrors.fedoraproject.org/metalink?repo=epel-9&arch=x86_64' ./mirror/epel9

# An apt suite. Pool files resolve against the archive root, producing
# ./mirror/debian/dists/bookworm/ and ./mirror/debian/pool/.
repo-sync deb https://deb.example.com/debian/dists/bookworm ./mirror

# Limit an apt mirror to specific components and architectures. Include
# "source" as an architecture to keep source indexes.
repo-sync deb --component main --arch amd64,source https://deb.example.com/debian/dists/bookworm ./mirror

Building

make

The build stamps the binary with the contents of the VERSION file plus the git commit and build date, shown by repo-sync --version. A plain go build also works but reports the version as dev.

Testing

make test

The suite is hermetic: fixture repositories for every supported format (rpm, deb, arch, apk) are generated in temp directories and served over local HTTP, so no network access is required.