118 lines
4.7 KiB
Markdown
118 lines
4.7 KiB
Markdown
# repo-sync
|
|
|
|
A universal Linux package repository synchronization tool. It mirrors remote
|
|
repositories into a local directory tree, preserving the upstream layout so
|
|
the result can be served directly to package managers.
|
|
|
|
Supported repository types:
|
|
|
|
- **rpm** — yum/dnf repositories (`repodata/repomd.xml`), including
|
|
plain-text mirrorlist and metalink URLs with failover between mirrors.
|
|
- **deb** — apt repositories, both standard (`dists/<suite>` with a shared
|
|
`pool/`) and flat layouts, including `Acquire-By-Hash` population.
|
|
- **arch** — pacman repositories (`<name>.db`), including detached package
|
|
signatures and companion metadata (`.files`, `.db.tar.gz`, `.links.tar.gz`).
|
|
- **apk** — Alpine Linux repositories (`APKINDEX.tar.gz` with packages
|
|
beside it).
|
|
|
|
## Usage
|
|
|
|
```
|
|
repo-sync <type> [flags] <url> [<url> ...] <destination-directory>
|
|
```
|
|
|
|
The last argument is always the destination directory; every argument before
|
|
it is a repository or mirrorlist URL, synchronized one after the other.
|
|
|
|
A configuration file is optional: without one the built-in defaults apply.
|
|
When one is present its `crawler` section supplies the defaults for every
|
|
command, so `workers`, `prune_grace`, `request_timeout`, and `user_agent`
|
|
are shared by the sync commands and the server. The global flags
|
|
`--config-path`, `--log-level`, `--user-agent`, and `--request-timeout`
|
|
override the configuration for any command.
|
|
|
|
```sh
|
|
# Mirror one repository. The URL path is copied below the destination, so
|
|
# this produces ./mirror/repos/CentOS/7/EA4/.
|
|
repo-sync rpm https://example.com/repos/CentOS/7/EA4/ ./mirror
|
|
|
|
# Trim the first two path components: ./mirror/7/EA4/.
|
|
repo-sync rpm --trim 2 https://example.com/repos/CentOS/7/EA4/ ./mirror
|
|
|
|
# No path copying at all: the repository lands directly in ./mirror.
|
|
repo-sync rpm --flat https://example.com/repos/CentOS/7/EA4/ ./mirror
|
|
|
|
# Multiple repositories in one run.
|
|
repo-sync rpm https://example.com/repos/a/ https://example.com/repos/b/ ./mirror
|
|
|
|
# Crawl directory listings for repositories, at most 4 levels deep.
|
|
repo-sync rpm --discover --depth 4 https://example.com/repos/ ./mirror
|
|
|
|
# A pacman repository. The database name is discovered from the directory
|
|
# listing, or by probing URL path segments when listings are disabled.
|
|
repo-sync arch https://example.com/archlinux/core/os/x86_64/ ./mirror
|
|
|
|
# An Alpine repository is one arch directory; discovery syncs every arch
|
|
# of a release/repo tree in one run.
|
|
repo-sync apk https://dl-cdn.alpinelinux.org/alpine/v3.24/community/x86_64/ ./mirror
|
|
repo-sync apk --discover --depth 1 https://dl-cdn.alpinelinux.org/alpine/v3.24/community/ ./mirror
|
|
|
|
# A Fedora-style metalink URL; mirrors are used in preference order. The
|
|
# mirrors' paths differ, so --flat keeps the destination stable.
|
|
repo-sync rpm --flat 'https://mirrors.fedoraproject.org/metalink?repo=epel-9&arch=x86_64' ./mirror/epel9
|
|
|
|
# An apt suite. Pool files resolve against the archive root, producing
|
|
# ./mirror/debian/dists/bookworm/ and ./mirror/debian/pool/.
|
|
repo-sync deb https://deb.example.com/debian/dists/bookworm ./mirror
|
|
|
|
# Limit an apt mirror to specific components and architectures. Include
|
|
# "source" as an architecture to keep source indexes.
|
|
repo-sync deb --component main --arch amd64,source https://deb.example.com/debian/dists/bookworm ./mirror
|
|
```
|
|
|
|
## Mirror server
|
|
|
|
`repo-sync server` runs a caching mirror in front of upstream repositories.
|
|
Repositories are discovered from client requests and then kept synchronized
|
|
in the background, so package managers can be pointed straight at it.
|
|
|
|
```sh
|
|
repo-sync server --config-path ./config.yaml
|
|
```
|
|
|
|
Without `--config-path` the config is read from `./config.yaml`,
|
|
`~/.config/repo-sync/config.yaml`, or `/etc/repo-sync/config.yaml`. See
|
|
[config.example.yaml](config.example.yaml) for a documented configuration
|
|
covering the listener, domains, mounts, and crawler tuning. The server
|
|
needs at least one domain and one mount; the sync commands do not.
|
|
|
|
To run it as a systemd service, install `/etc/repo-sync/config.yaml` and:
|
|
|
|
```sh
|
|
repo-sync service install
|
|
repo-sync service start
|
|
```
|
|
|
|
`service` also accepts `stop`, `restart`, `status`, and `uninstall`. The
|
|
installed unit runs `repo-sync server` as a notify service, restarts on
|
|
failure, and reloads its configuration on `systemctl reload repo-sync`.
|
|
|
|
## Building
|
|
|
|
```sh
|
|
make
|
|
```
|
|
|
|
The build stamps the binary with the contents of the `VERSION` file plus the
|
|
git commit and build date, shown by `repo-sync --version`. A plain
|
|
`go build` also works but reports the version as `dev`.
|
|
|
|
## Testing
|
|
|
|
```sh
|
|
make test
|
|
```
|
|
|
|
The suite is hermetic: fixture repositories for every supported format
|
|
(rpm, deb, arch, apk) are generated in temp directories and served over
|
|
local HTTP, so no network access is required.
|