repo-sync

A universal Linux package repository synchronization tool. It mirrors remote repositories into a local directory tree, preserving the upstream layout so the result can be served directly to package managers.

Supported repository types:

• rpm — yum/dnf repositories (repodata/repomd.xml), including plain-text mirrorlist and metalink URLs with failover between mirrors.
• deb — apt repositories, both standard (dists/<suite> with a shared pool/) and flat layouts, including Acquire-By-Hash population.
• arch — pacman repositories (<name>.db), including detached package signatures and companion metadata (.files, .db.tar.gz, .links.tar.gz).
• apk — Alpine Linux repositories (APKINDEX.tar.gz with packages beside it).

Usage

repo-sync <type> [flags] <url> [<url> ...] <destination-directory>

The last argument is always the destination directory; every argument before it is a repository or mirrorlist URL, synchronized one after the other.

A configuration file is optional: without one the built-in defaults apply. When one is present its crawler section supplies the defaults for every command, so workers, prune_grace, request_timeout, and user_agent are shared by the sync commands and the server. The global flags --config-path, --log-level, --user-agent, and --request-timeout override the configuration for any command.

# Mirror one repository. The URL path is copied below the destination, so
# this produces ./mirror/repos/CentOS/7/EA4/.
repo-sync rpm https://example.com/repos/CentOS/7/EA4/ ./mirror

# Trim the first two path components: ./mirror/7/EA4/.
repo-sync rpm --trim 2 https://example.com/repos/CentOS/7/EA4/ ./mirror

# No path copying at all: the repository lands directly in ./mirror.
repo-sync rpm --flat https://example.com/repos/CentOS/7/EA4/ ./mirror

# Multiple repositories in one run.
repo-sync rpm https://example.com/repos/a/ https://example.com/repos/b/ ./mirror

# Crawl directory listings for repositories, at most 4 levels deep.
repo-sync rpm --discover --depth 4 https://example.com/repos/ ./mirror

# A pacman repository. The database name is discovered from the directory
# listing, or by probing URL path segments when listings are disabled.
repo-sync arch https://example.com/archlinux/core/os/x86_64/ ./mirror

# An Alpine repository is one arch directory; discovery syncs every arch
# of a release/repo tree in one run.
repo-sync apk https://dl-cdn.alpinelinux.org/alpine/v3.24/community/x86_64/ ./mirror
repo-sync apk --discover --depth 1 https://dl-cdn.alpinelinux.org/alpine/v3.24/community/ ./mirror

# A Fedora-style metalink URL; mirrors are used in preference order. The
# mirrors' paths differ, so --flat keeps the destination stable.
repo-sync rpm --flat 'https://mirrors.fedoraproject.org/metalink?repo=epel-9&arch=x86_64' ./mirror/epel9

# An apt suite. Pool files resolve against the archive root, producing
# ./mirror/debian/dists/bookworm/ and ./mirror/debian/pool/.
repo-sync deb https://deb.example.com/debian/dists/bookworm ./mirror

# Limit an apt mirror to specific components and architectures. Include
# "source" as an architecture to keep source indexes.
repo-sync deb --component main --arch amd64,source https://deb.example.com/debian/dists/bookworm ./mirror

Mirror server

repo-sync server runs a caching mirror in front of upstream repositories. Repositories are discovered from client requests and then kept synchronized in the background, so package managers can be pointed straight at it.

repo-sync server --config-path ./config.yaml

Without --config-path the config is read from ./config.yaml, ~/.config/repo-sync/config.yaml, or /etc/repo-sync/config.yaml. See config.example.yaml for a documented configuration covering the listener, domains, mounts, and crawler tuning. The server needs at least one domain and one mount; the sync commands do not.

To run it as a systemd service, install /etc/repo-sync/config.yaml and:

repo-sync service install
repo-sync service start

service also accepts stop, restart, status, and uninstall. The installed unit runs repo-sync server as a notify service, restarts on failure, and reloads its configuration on systemctl reload repo-sync.

Building

make

The build stamps the binary with the contents of the VERSION file plus the git commit and build date, shown by repo-sync --version. A plain go build also works but reports the version as dev.

Testing

make test

The suite is hermetic: fixture repositories for every supported format (rpm, deb, arch, apk) are generated in temp directories and served over local HTTP, so no network access is required.