Commit graph

35 commits

Author SHA1 Message Date
68cb51bb6f Add repo-sync method, sync change counts, and configurable trace saving
mirror-sync:
- Add a repo-sync sync method for rpm/deb/arch/apk repositories, with
  installer, prune setting, an optional type filter, and trace fields passed
  through to it.
- Count what each sync changed and publish SYNC_FILES_* / SYNC_CHANGED to
  the post successful sync hook, so a hook can stay idle on a no-op run.
- Make trace saving configurable with save_trace (global and per module)
  and publish traces from every method, not just rsync. Exclude the local
  trace from deletes, and stop hierarchy entries stacking up per run.
- Add parse_bool so boolean settings reject values that would otherwise be
  read as their opposite; keep an explicit bare=false from being
  auto-detected away, and require hide_remote on the clone path too.
- MIRROR_SYNC_CONF names an alternate configuration file.
- Fix docroot joining when repo has a trailing slash, unparsable
  last-modified headers skipping every sync, s3cmd --host taking a full
  URL, and the average rate line for methods with no byte count.
- Exit non-zero when help is printed because of a mistake.

mirror-file-generator:
- Match du summary paths exactly instead of by substring, so sibling repos
  sharing a prefix no longer return several sizes.
- Fall back to the default icon when no image is found, and stop the
  fallback recursing into itself.
- Validate sections against SECTIONS, count repos written to decide whether
  to publish the index, and create the log directory if missing.
- MIRROR_SYNC_CONF support, and exit non-zero on an unknown mirror.

Docs: rewrite README around quick start and sync methods, and move the
file generator reference to docs/mirror-file-generator.md.
2026-07-28 09:13:19 -05:00
ccb938e300 Add post_successful_sync_hook to each sync method
Runs a configurable hook at the very end of every sync method, after all
cleanup has finished and the sync is confirmed successful. Unlike the
rsync/qfm post_hook, it is available for all sync methods and runs after
trace files, log cleanup, and mirror reporting.
2026-07-22 13:39:24 -05:00
a92476734e Add git update-server-info for bare repos and hide_remote option
Bare mirrors served over dumb HTTP need info/refs and objects/info/packs
kept current, so run git update-server-info after every successful bare
sync and clone. The post-update hook cannot be used since it only fires
on push, not on the fetch a mirror performs.

Add a hide_remote option (requires bare + source) that keeps the upstream
URL out of the served files: it removes the stored remote from config,
deletes FETCH_HEAD after each fetch, and fetches directly from source with
a +refs/*:refs/* mirror refspec on every run.
2026-07-22 10:57:26 -05:00
da5ea76bd9 Stop in-progress sync and clean up on Ctrl-C (SIGINT/SIGTERM)
Add a SIGINT/SIGTERM trap that cleans up after an interrupted sync
instead of leaving stale state behind. On a terminal Ctrl-C the signal
reaches the running rsync/aws/git/etc. process directly (it is delivered
to the whole process group), so it stops on its own; the handler then
removes the Archive-Update-in-Progress markers, temporary stage logs and
the module lock file, and exits 130 without counting the interruption as
a sync failure (so it does not inflate the error count or send error
mail).

The rsync and quick-fedora-mirror paths register their in-progress
markers and temp logs for cleanup as they create them.

Bump version to 20260721.
2026-07-21 18:44:42 -05:00
b8c3c55574 Fix shellcheck SC2013 warnings in byte-counting loops 2026-06-16 13:27:29 -05:00
4d800e21b4 Add bare git repository support with configurable source, destination, and bare flag 2026-06-16 11:09:03 -05:00
3fc2257751 Fix dark mode link color in mirror-description and other non-repo-list areas
Add a general a:link, a:visited dark mode rule so links anywhere in the
page (e.g. the description block) use the correct color instead of the
browser's default purple.
2026-06-05 13:16:37 -05:00
bb9ff3b4ba Fix dark mode visited link color to prevent browser default purple
Add a:visited alongside a:link so visited links inherit the correct color
in both light and dark mode, instead of falling back to the browser's
default dark purple.
2026-06-05 13:05:00 -05:00
6a0fb0fe4d Update dashboard-icons repo from walkxcode to homarr-labs
The dashboard-icons repository has moved to github.com/homarr-labs/dashboard-icons.
2026-06-05 12:45:54 -05:00
782c24e9f5 Add dark mode support with logo_dark and repo_icon_dark variants
Introduces logo_dark and repo_icon_dark config options that serve alternate
images to dark-OS browsers via HTML <picture> elements, and adds a CSS
prefers-color-scheme dark block to the default header template.
2026-06-05 12:30:09 -05:00
81be7ddd2e Fix bugs, improve robustness, and add local dashboard-icons support
- mirror-sync.sh: fix upstream_check/time_file_check to handle missing
  timestamp file and empty Last-Modified header; fix rebuild_dusum_totals
  variable shadowing; propagate return codes from install helpers; refactor
  git_sync out of subshell; scope LOGFILE_STAGE2 to stage-2 block; clean up
  mirror_update_file and QFM in-progress files on failure; fix total_time
  initialization and wget_sync exit propagation
- mirror-file-generator.sh: add icons_local_repo/icons_repo_url/icons_repo_refresh
  for offline icon serving; move PIDFILE to /tmp; enhance image_copy to
  refresh stale cached files; fix temp index file cleanup on empty output;
  safer ${var:?} → [[ -n $var ]] guards throughout
- README.md: fix typos, heading levels, variable names, and document new
  icon repo config options
- Bump VERSION to 20260602 in both scripts
2026-06-04 08:54:36 -05:00
921ba5a73b Support promoted CUSTOM_MODULES timestamp/size sidecars and fix curl error handling
Move PIDFILE to /var/run, fix curl to fail fast on HTTP errors and clean partial
files before fallback, break after read_config in module loop, read per-tier
timestamp/dusum sidecars for promoted CUSTOM_MODULES, and fall back to
.last-synced mtime when no other timestamp source is configured.
2026-05-16 09:20:56 -05:00
28ac53292b For mirrors with calc disabled, don't get prior calculated values 2024-02-20 11:21:44 -06:00
9c2f56ea63 Avoid double repo size updating 2024-02-20 11:09:48 -06:00
795c295d38 Fix stat time 2024-02-20 09:26:39 -06:00
ab1939834d Fix module config reading 2024-02-20 08:38:43 -06:00
4ab8a9c8ec Better template default path. 2024-02-19 19:26:55 -06:00
5208458090 Restructure mirror-sync to improve readability 2024-02-19 19:01:21 -06:00
9eb4d55ff4 Fix comments 2024-02-19 18:01:55 -06:00
cbd6c904f9 Add a way to skip repos in generated files and add a few unset variables. 2024-02-18 21:28:55 -06:00
9aea1cbf26 Add mirror-file-generator tool 2024-02-17 10:44:32 -06:00
bddefc4778 Add version print options 2024-01-24 10:19:03 -06:00
61dae13b55 Add aging support to the time file check. 2024-01-24 09:49:51 -06:00
7a87be349f Fix dusum math to save file after sum is complete. 2024-01-02 08:35:01 -06:00
20ba5577d6 Fix dusum file check 2023-12-29 18:06:09 -06:00
be6f63048b Correct bytes measurements 2023-12-29 16:00:40 -06:00
879f02f857 Add ability to store disk usage summaries and create summary text files, also allow forcing override of time file checks. 2023-12-29 15:20:26 -06:00
b1c386dd42 Add support for checking an upstream file via rsync before updating all files. 2023-12-13 11:17:39 -06:00
3e5829918a Fix s5cmd options 2023-11-22 17:22:06 -06:00
01dae58670 Add s5cmd support. 2023-11-22 17:18:58 -06:00
ef19ebadf1 Add delete removed as delete after doesn't assume it 2023-11-22 16:23:16 -06:00
d2053a82ed Adding skip-existing by default to s3cmd 2023-11-22 16:18:30 -06:00
21ab9fb1be Add s3cmd support. 2023-11-22 15:57:22 -06:00
e1a3a83f53 Fix QFM repo configuration logic. 2023-11-14 22:22:05 -06:00
d99f3d310d Initital commit 2023-11-11 15:21:18 -06:00