GRMrGecko/nginx-cache-purge
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Add service management, harden the purge server, and modernize the build.

Browse source 
Server:
- Add a --cache-path allowlist so a server can be limited to the caches it
  is meant to purge, defaulting to any path as before.
- Set socket permissions explicitly (--socket-mode, default 0660) instead of
  inheriting the service manager's umask, which left the socket unreachable.
- Refuse to remove a socket another instance is still serving.
- Read keys both raw and decoded, so keys nginx stored with escapes and keys
  a caller escaped by hand both purge.
- Add an exact= parameter for literal keys containing glob punctuation.
- Report purge failures as 500 rather than 502, and send error bodies through
  http.Error so a failure is not reported as a successful purge.
- Graceful shutdown with systemd readiness notification.

Purge:
- Group purge arguments into PurgeRequest and report the number of entries
  removed.
- Compile exclude globs once, and fail the purge when one is invalid rather
  than purging the keys it was meant to keep.
- Cap header scanning and tolerate entries nginx evicts mid-walk.
- Switch to filepath.WalkDir to avoid an Lstat per cache file.

New:
- service command to install, start, stop, and remove the system service.
- service install takes --cache-path, writing the allowlist into the unit it
  installs, so an installed service is restricted from its first start.
- Makefile, VERSION, and build identifiers stamped via ldflags.
- Tests for the server handler and the service command.

Build:
- Update to Go 1.25, kong v1, GoReleaser v2, and current GitHub Actions.
- Add vet and test steps to CI.
- Rename purgeCmd.go/serverCmd.go to Go's file naming convention.

Bump version to 0.2.0.
This commit is contained in:
James Coleman 2026-08-12 14:36:03 -05:00
parent 44cd195109
commit 8d9b1c9302
19 changed files with 1769 additions and 259 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
.github/workflows/release.yaml vendored
View file

@ -12,18 +12,20 @@ jobs:
steps: steps:
- -
name: Checkout name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v7
with: with:
fetch-depth: 0 fetch-depth: 0
- -
name: Set up Go name: Set up Go
uses: actions/setup-go@v4 uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
- -
name: Run GoReleaser name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5 uses: goreleaser/goreleaser-action@v7
with: with:
distribution: goreleaser distribution: goreleaser
version: latest version: '~> v2'
args: release --clean args: release --clean
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

12
.github/workflows/test_golang.yaml vendored
View file

@ -7,12 +7,18 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v7
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v4 uses: actions/setup-go@v7
with: with:
go-version: '1.21' go-version-file: 'go.mod'
- name: Build - name: Build
run: go build -v ./... run: go build -v ./...
- name: Vet
run: go vet ./...
- name: Test
run: go test -v ./...

76
.goreleaser.yaml
View file

@ -1,30 +1,74 @@
# This is an example .goreleaser.yml file with some sensible defaults. # GoReleaser config for nginx-cache-purge.
# Make sure to check the documentation at https://goreleaser.com # https://goreleaser.com
#
# CGO is disabled so the binary is fully static (no glibc dependency) and runs
# unmodified across modern Linux distributions.
version: 2
# The lines below are called `modelines`. See `:help modeline` project_name: nginx-cache-purge
# Feel free to remove those if you don't want/need to use them.
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj
version: 1
before: before:
hooks: hooks:
# You may remove this if you don't use go modules.
- go mod tidy - go mod tidy
# you may remove this if you don't need go generate - go test ./...
- go generate ./...
builds: builds:
- env: - id: nginx-cache-purge
main: .
binary: nginx-cache-purge
env:
- CGO_ENABLED=0 - CGO_ENABLED=0
flags:
- -trimpath
# The build identifiers live in package main, so they are stamped there
# rather than in an imported configuration package.
ldflags:
- -s -w
- -X main.Version={{ .Version }}
- -X main.Commit={{ .ShortCommit }}
- -X main.Date={{ .Date }}
- -X main.Mode=release
goos: goos:
- linux - linux
- darwin - darwin
goarch:
- "386"
- amd64
- arm
- arm64
- ppc64le
goarm:
- "6"
ignore:
- goos: darwin
goarch: "386"
- goos: darwin
goarch: arm
- goos: darwin
goarch: ppc64le
archives: archives:
- format: tar.gz - id: default
# this name template makes the OS and Arch compatible with the results of `uname`. formats: [tar.gz]
name_template: "{{ .ProjectName }}-{{ .Version }}.{{ .Os }}-{{ .Arch }}" # Kept compatible with the results of uname, matching the naming earlier
# releases were published under.
name_template: "{{ .ProjectName }}-{{ .Version }}.{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
wrap_in_directory: true wrap_in_directory: true
strip_parent_binary_folder: false files:
- README.md
- LICENSE.txt
checksum:
name_template: "checksums.txt"
snapshot:
version_template: "{{ incpatch .Version }}-snapshot"
changelog:
use: git
sort: asc
filters:
exclude:
- "^docs:"
- "^test:"
- "^chore:"

2
LICENSE.txt
View file

@ -1,4 +1,4 @@
Copyright (c) 2023 Mr. Gecko's Media (James Coleman). http://mrgeckosmedia.com/ Copyright (c) 2023-2026 Mr. Gecko's Media (James Coleman). http://mrgeckosmedia.com/
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal

39
Makefile Normal file
View file

@ -0,0 +1,39 @@
BINARY := nginx-cache-purge
# The build identifiers live in package main, so ldflags target it directly
# rather than an imported configuration package.
PACKAGE := main
# VERSION is the single source of truth for the version string. COMMIT and DATE
# are derived from git and the build clock.
VERSION ?= $(shell cat VERSION 2>/dev/null || echo dev)
COMMIT := $(shell git rev-parse --short HEAD 2>/dev/null)
DATE := $(shell date -u '+%Y-%m-%dT%H:%M:%SZ')
LDFLAGS := -s -w -X $(PACKAGE).Version=$(VERSION) -X $(PACKAGE).Commit=$(COMMIT) -X $(PACKAGE).Date=$(DATE)
.PHONY: all build test vet fmt snapshot release clean
all: build
## build: native static binary into dist/
build:
CGO_ENABLED=0 go build -trimpath -ldflags '$(LDFLAGS)' -o dist/$(BINARY) .
## test: run the unit tests
test:
go test ./...
vet:
go vet ./...
fmt:
gofmt -w .
## snapshot: local GoReleaser build without publishing (artifacts in dist/)
snapshot:
goreleaser release --snapshot --clean
## release: full GoReleaser release (CI runs this on a tag)
release:
goreleaser release --clean
clean:
rm -rf dist

162
README.md
View file

@ -6,8 +6,9 @@ You can install either by downloading the latest binary release, or by building.
## Building ## Building
Building should be as simple as running: Building should be as simple as running:
``` ```
go build make
``` ```
## Usage ## Usage
@ -33,34 +34,100 @@ $ nginx-cache-purge purge /var/nginx/proxy_temp/cache 'example.com/*.{jpg,jpeg,p
$ nginx-cache-purge purge /var/nginx/proxy_temp/cache '*' $ nginx-cache-purge purge /var/nginx/proxy_temp/cache '*'
``` ```
## Running as a service ### Purge a key that contains wildcard characters
If you want to run as a service to allow purge requests via http requests, you'll need to create a systemd service file and place it in `/etc/systemd/system/nginx-cache-purge.service`. Whether a key is a wildcard is otherwise guessed from the punctuation in it, and
real cache keys carry that punctuation: a request URI with a query string puts
`?` in the key, and PHP-style array parameters put `[` and `]`. `--exact` says
the key is a literal, for both the key and any excludes.
``` ```
[Unit] $ nginx-cache-purge purge --exact /var/nginx/proxy_temp/cache 'example.com/list.php?f[]=x'
Description=Nginx Cache Purge
After=network.target
[Service]
User=nginx
Group=nginx
RuntimeDirectory=nginx-cache-purge
ExecStart=/usr/local/bin/nginx-cache-purge server
Restart=always
RestartSec=3s
[Install]
WantedBy=multi-user.target
``` ```
You can then run the following to start the service: ## Running as a service
If you want to run as a service to allow purge requests via http requests, the
service can install itself:
``` ```
systemctl daemon-reload nginx-cache-purge service install --cache-path /var/nginx/proxy_temp/cache
systemctl start nginx-cache-purge.service nginx-cache-purge service start
``` ```
`service` also accepts `stop`, `restart`, `status`, and `uninstall`. The
installed unit runs `nginx-cache-purge server` as a notify service, creates the
runtime directory the socket lives in, and restarts on failure. `--cache-path`
restricts which caches that server will purge, and is covered below.
Connecting to a UNIX socket needs write permission on it, so the socket is
created mode `0660`: the purge server and Nginx have to run as the same user, or
share a group. The installed unit runs as root, so either add a
`User=`/`Group=` drop-in for it, or widen the socket with
`--socket-mode`.
### Restricting which caches may be purged
The directory to purge comes from the request, and the purge deletes what it
finds under it, so a server left unrestricted will purge any path its user can
reach. Pass `--cache-path` to name the caches it may serve, repeating it for
more than one:
```
nginx-cache-purge server --cache-path /var/nginx/proxy_temp/cache
```
A request naming any other directory is refused with `403`. Directories inside a
named cache are allowed, and symlinks are resolved, so a link to a named cache is
recognised as that cache. Naming no path leaves every path purgeable, which is
what a server without the flag has always done.
`service install` takes the same flag and writes it into the unit's `ExecStart`,
so an installed service is restricted from its first start:
```
nginx-cache-purge service install --cache-path /var/nginx/proxy_temp/cache
```
Repeat the flag for more than one cache. Paths are made absolute at install, as
the unit runs from a working directory of the service manager's choosing;
symlinks are left as written and resolved per request. The allowlist lives in the
unit, so changing it means installing again: `service uninstall` then
`service install` with the paths you want.
## Nginx config ## Nginx config
If you want to purge via Nginx http requests, you'll need to add configuration to your Nginx config file. If you want to purge via Nginx http requests, you'll need to add configuration to your Nginx config file.
The server reads four query parameters:
| Parameter | Description |
| --- | --- |
| `path` | Path to the cache directory, the same one given to `proxy_cache_path`. |
| `key` | Cache key or wildcard match, the same one built by `proxy_cache_key`. |
| `exclude` | Key to keep, can be a wildcard. Repeat it to exclude more than one. |
| `exact` | Read the key and excludes as literals rather than wildcards. |
Nginx substitutes `$request_uri` into the rewrite without escaping it, so a key
containing `%` or `+` arrives as the literal bytes Nginx stored it under, and
that is what is purged. A key escaped by a caller writing the request itself is
purged too, so either convention works.
### Literal keys and wildcards
A key built from `$request_uri` is a literal, and a request URI routinely holds
the punctuation that would otherwise mark the key as a wildcard: `?` from a query
string, `[` and `]` from PHP-style array parameters. Read as wildcards those keys
purge the wrong entries, fail outright, or match nothing while still answering
`PURGED` and leaving the stale entry served. The examples below therefore pass
`exact=1`, which is what you want when a PURGE request names one URL.
Leave `exact` off where the purge is meant to take a wildcard, such as the
`/purge(/.*)` location further down, where a request for `/purge/images/*`
clears everything under `/images/`.
Because `$request_uri` carries the client's own query string into the purge
parameters, put `exact=1` **before** `key=` in the rewrite. Parameters are taken
first-wins, so an `exact=0` a client appends to its request URI arrives second
and is ignored. The same ordering already protects `path`. Note that `exclude` is
collected rather than taken first-wins, so a client can append an `exclude` that
holds back its own purge.
### Map PURGE requests ### Map PURGE requests
``` ```
http { http {
@ -77,7 +144,7 @@ http {
proxy_cache_bypass $is_purge; proxy_cache_bypass $is_purge;
if ($is_purge) { if ($is_purge) {
proxy_pass http://unix:/run/nginx-cache-purge/http.sock; proxy_pass http://unix:/run/nginx-cache-purge/http.sock;
rewrite ^ /?path=/var/nginx/proxy_temp/cache&key=$server_name$request_uri break; rewrite ^ /?path=/var/nginx/proxy_temp/cache&exact=1&key=$server_name$request_uri break;
} }
proxy_cache my_cache; proxy_cache my_cache;
@ -103,7 +170,7 @@ http {
proxy_cache_bypass $is_purge; proxy_cache_bypass $is_purge;
if ($is_purge) { if ($is_purge) {
proxy_pass http://unix:/run/nginx-cache-purge/http.sock; proxy_pass http://unix:/run/nginx-cache-purge/http.sock;
rewrite ^ /?path=/var/nginx/proxy_temp/cache&key=$server_name$request_uri break; rewrite ^ /?path=/var/nginx/proxy_temp/cache&exact=1&key=$server_name$request_uri break;
} }
proxy_cache my_cache; proxy_cache my_cache;
@ -129,7 +196,7 @@ http {
proxy_cache_bypass $is_purge; proxy_cache_bypass $is_purge;
if ($is_purge) { if ($is_purge) {
proxy_pass http://unix:/run/nginx-cache-purge/http.sock; proxy_pass http://unix:/run/nginx-cache-purge/http.sock;
rewrite ^ /?path=/var/nginx/proxy_temp/cache&key=$server_name$request_uri break; rewrite ^ /?path=/var/nginx/proxy_temp/cache&exact=1&key=$server_name$request_uri break;
} }
proxy_cache my_cache; proxy_cache my_cache;
@ -165,7 +232,7 @@ http {
proxy_cache_bypass $should_purge; proxy_cache_bypass $should_purge;
if ($should_purge) { if ($should_purge) {
proxy_pass http://unix:/run/nginx-cache-purge/http.sock; proxy_pass http://unix:/run/nginx-cache-purge/http.sock;
rewrite ^ /?path=/var/nginx/proxy_temp/cache&key=$server_name$request_uri break; rewrite ^ /?path=/var/nginx/proxy_temp/cache&exact=1&key=$server_name$request_uri break;
} }
proxy_cache my_cache; proxy_cache my_cache;
@ -176,6 +243,9 @@ http {
``` ```
### Using IP whitelists ### Using IP whitelists
This location takes a wildcard, so it leaves `exact` off: a request for
`/purge/images/*` clears everything under `/images/`.
``` ```
http { http {
proxy_cache_path /var/nginx/proxy_temp/cache levels=1:2 keys_zone=my_cache:10m; proxy_cache_path /var/nginx/proxy_temp/cache levels=1:2 keys_zone=my_cache:10m;
@ -195,47 +265,3 @@ http {
} }
} }
``` ```
## Help
```
$ nginx-cache-purge --help
Usage: nginx-cache-purge <command> [flags]
Tool to help purge cache from Nginx
Flags:
-h, --help Show context-sensitive help.
--version Print version information and quit
Commands:
server (s) Run the server
purge (p) Purge cache now
Run "nginx-cache-purge <command> --help" for more information on a command.
$ nginx-cache-purge p --help
Usage: nginx-cache-purge purge (p) <cache-path> <key> [flags]
Purge cache now
Arguments:
<cache-path> Path to cache directory.
<key> Cache key or wildcard match.
Flags:
-h, --help Show context-sensitive help.
--version Print version information and quit
--exclude-key=EXCLUDE-KEY,... Key to exclude, can be wild card and can add multiple excludes.
$ nginx-cache-purge s --help
Usage: nginx-cache-purge server (s) [flags]
Run the server
Flags:
-h, --help Show context-sensitive help.
--version Print version information and quit
--socket=STRING Socket path for HTTP communication.
``

1
VERSION Normal file
View file

@ -0,0 +1 @@
0.2.0

58
flags.go
View file

@ -2,17 +2,45 @@ package main
import ( import (
"fmt" "fmt"
"runtime/debug"
"strings"
"github.com/alecthomas/kong" "github.com/alecthomas/kong"
) )
// When version is requested, print the version. // VersionFlag prints build information and exits.
type VersionFlag bool type VersionFlag bool
// Decode satisfies kong.MapperValue. The flag is treated as a boolean toggle.
func (v VersionFlag) Decode(ctx *kong.DecodeContext) error { return nil } func (v VersionFlag) Decode(ctx *kong.DecodeContext) error { return nil }
func (v VersionFlag) IsBool() bool { return true }
// IsBool reports the flag as a boolean for kong's parser.
func (v VersionFlag) IsBool() bool { return true }
// BeforeApply emits version information then exits before the rest of the
// command is executed.
func (v VersionFlag) BeforeApply(app *kong.Kong, vars kong.Vars) error { func (v VersionFlag) BeforeApply(app *kong.Kong, vars kong.Vars) error {
fmt.Println(serviceName + ": " + serviceVersion) fmt.Printf("%s: %s (%s)\n", Name, Version, Mode)
if Commit != "" {
fmt.Printf(" commit: %s\n", Commit)
}
if Date != "" {
fmt.Printf(" built: %s\n", Date)
}
// Without build stamps, a module-aware build still records the revision
// it was built from, so fall back to what the toolchain embedded.
if Commit == "" {
if bi, ok := debug.ReadBuildInfo(); ok {
for _, s := range bi.Settings {
switch s.Key {
case "vcs.revision":
fmt.Printf(" commit: %s\n", s.Value)
case "vcs.time":
fmt.Printf(" built: %s\n", s.Value)
}
}
}
}
app.Exit(0) app.Exit(0)
return nil return nil
} }
@ -21,21 +49,33 @@ func (v VersionFlag) BeforeApply(app *kong.Kong, vars kong.Vars) error {
type Flags struct { type Flags struct {
Version VersionFlag `name:"version" help:"Print version information and quit"` Version VersionFlag `name:"version" help:"Print version information and quit"`
Server ServerCmd `cmd:"" aliases:"s" default:"1" help:"Run the server"` Server ServerCmd `cmd:"" aliases:"s" default:"1" help:"Run the server"`
Purge PurgeCmd `cmd:"" aliases:"p" help:"Purge cache now"` Purge PurgeCmd `cmd:"" aliases:"p" help:"Purge cache now"`
Service ServiceCmd `cmd:"" help:"Manage the purge server system service."`
}
// kongVars are the values the command tags interpolate. They live in one place
// so that a parser built anywhere describes the same command line.
func kongVars() kong.Vars {
return kong.Vars{
"serviceActions": strings.Join(ServiceAction, ","),
"defaultSocket": defaultSocketPath,
"defaultMode": defaultSocketMode,
}
} }
// Parse the supplied flags and commands. // Parse the supplied flags and commands.
func (a *App) ParseFlags() *kong.Context { func (a *App) ParseFlags() *kong.Context {
app.flags = &Flags{} a.flags = &Flags{}
ctx := kong.Parse(app.flags, ctx := kong.Parse(a.flags,
kong.Name(serviceName), kong.Name(Name),
kong.Description(serviceDescription), kong.Description(Description),
kong.UsageOnError(), kong.UsageOnError(),
kong.ConfigureHelp(kong.HelpOptions{ kong.ConfigureHelp(kong.HelpOptions{
Compact: true, Compact: true,
}), }),
kongVars(),
) )
return ctx return ctx
} }

15
go.mod
View file

@ -1,11 +1,18 @@
module github.com/grmrgecko/nginx-cache-purge module github.com/grmrgecko/nginx-cache-purge
go 1.22.5 go 1.25.0
require ( require (
github.com/alecthomas/kong v0.9.0 github.com/alecthomas/kong v1.16.1
github.com/coreos/go-systemd/v22 v22.7.0
github.com/gobwas/glob v0.2.3 github.com/gobwas/glob v0.2.3
github.com/portmapping/go-reuse v0.0.3 github.com/kardianos/service v1.3.0
github.com/stretchr/testify v1.11.1
) )
require golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3 // indirect require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
golang.org/x/sys v0.47.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

26
go.sum
View file

@ -1,14 +1,32 @@
github.com/alecthomas/assert/v2 v2.6.0 h1:o3WJwILtexrEUk3cUVal3oiQY2tfgr/FHWiz/v2n4FU= github.com/alecthomas/assert/v2 v2.6.0 h1:o3WJwILtexrEUk3cUVal3oiQY2tfgr/FHWiz/v2n4FU=
github.com/alecthomas/assert/v2 v2.6.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k= github.com/alecthomas/assert/v2 v2.6.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/kong v0.9.0 h1:G5diXxc85KvoV2f0ZRVuMsi45IrBgx9zDNGNj165aPA= github.com/alecthomas/kong v0.9.0 h1:G5diXxc85KvoV2f0ZRVuMsi45IrBgx9zDNGNj165aPA=
github.com/alecthomas/kong v0.9.0/go.mod h1:Y47y5gKfHp1hDc7CH7OeXgLIpp+Q2m1Ni0L5s3bI8Os= github.com/alecthomas/kong v0.9.0/go.mod h1:Y47y5gKfHp1hDc7CH7OeXgLIpp+Q2m1Ni0L5s3bI8Os=
github.com/alecthomas/kong v1.16.1 h1:ixhCt93XkJ98kGposQ54+bl0IK6XwqB40AsMynU7Z8E=
github.com/alecthomas/kong v1.16.1/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc= github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4= github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA=
github.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y= github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8= github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg= github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/portmapping/go-reuse v0.0.3 h1:iY0JDxTTUaYopewHL0CLN5BqJ0BvDP48VzC2osPpkBQ= github.com/kardianos/service v1.3.0 h1:/LGy+xPP2TM+GLTiCZ2di7cy0Jd/qrawlTUfqKYFdTI=
github.com/portmapping/go-reuse v0.0.3/go.mod h1:xKeiOLrJpAUOineqiMEm1bpy6cq0vTdpoiebdRD45mo= github.com/kardianos/service v1.3.0/go.mod h1:E4V9ufUuY82F7Ztlu1eN9VXWIQxg8NoLQlmFe0MtrXc=
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3 h1:5B6i6EAiSYyejWfvc5Rc9BbI3rzIsrrXfAQBWnYfn+w= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

17
info.go Normal file
View file

@ -0,0 +1,17 @@
package main
// Build identifiers populated at build time via -ldflags. A plain go build
// leaves them at their defaults, so the binary reports itself as a dev build.
var (
Version = "dev"
Commit = ""
Date = ""
Mode = "dev"
)
// Application identifiers used by the CLI and the system service.
const (
Name = "nginx-cache-purge"
DisplayName = "Nginx Cache Purge"
Description = "Tool to help purge Nginx cache"
)

216
main.go
View file

@ -4,7 +4,10 @@ import (
"bufio" "bufio"
"crypto/md5" "crypto/md5"
"encoding/hex" "encoding/hex"
"errors"
"fmt" "fmt"
"io"
"io/fs"
"log" "log"
"os" "os"
"path/filepath" "path/filepath"
@ -14,12 +17,9 @@ import (
"github.com/gobwas/glob" "github.com/gobwas/glob"
) )
// Basic application info. // The KEY line lives in the cache entry header, within the first few hundred
const ( // bytes. This bounds how far we read looking for it.
serviceName = "nginx-cache-purge" const maxHeaderScan = 64 * 1024
serviceDescription = "Tool to help purge Nginx cache "
serviceVersion = "0.1.4"
)
// App structure to access global app variables. // App structure to access global app variables.
type App struct { type App struct {
@ -28,26 +28,69 @@ type App struct {
var app *App var app *App
// Function to purge nginx cache keys. // Regex to determine if a key is a glob pattern. Compiled once, as the server
func (a *App) PurgeCache(CachePath string, Key string, ExcludeKeys []string) error { // purges on every request.
var globRegex = regexp.MustCompile(`[\*?\[{]+`)
// PurgeRequest describes one purge. The fields travel together through the CLI
// and the server, so they are grouped rather than passed as a widening list of
// arguments.
type PurgeRequest struct {
// CachePath is the directory to purge from, the same one given to
// proxy_cache_path.
CachePath string
// Key is the cache key to purge, read as a wildcard pattern unless Exact
// says otherwise.
Key string
// ExcludeKeys name keys to keep, read the same way as Key.
ExcludeKeys []string
// Exact turns pattern matching off, for both the key and the excludes.
// Whether a key is a pattern is otherwise guessed from the punctuation in
// it, and real cache keys carry that punctuation: a request URI with a
// query string puts ? in the key, PHP-style array parameters put [ and ],
// and either one is read as a pattern that was never meant. Exact is how a
// caller that knows it holds a literal key says so.
Exact bool
}
// Function to purge nginx cache keys. It reports how many entries were removed,
// which is what tells a purge that cleared the cache from one that matched
// nothing at all.
func (a *App) PurgeCache(req PurgeRequest) (int, error) {
// Key must be provided. // Key must be provided.
if len(Key) == 0 { if len(req.Key) == 0 {
return fmt.Errorf("no key provided") return 0, fmt.Errorf("no key provided")
} }
// Regex to determine if key is a glob pattern. // Compile the exclude patterns up front. Doing it per key would repeat
globRegex := regexp.MustCompile(`[\*?\[{]+`) // the work for every file in the cache, and a pattern that fails to
// compile has to be fatal: ignoring it would purge the very keys the
// caller asked to keep. An exact purge has no patterns to compile, so the
// excludes stay literal and one holding glob punctuation keeps its key
// rather than failing the purge it appeared in.
var excludeGlobs []glob.Glob
if !req.Exact {
for _, exclude := range req.ExcludeKeys {
if !globRegex.MatchString(exclude) {
continue
}
g, err := glob.Compile(exclude)
if err != nil {
return 0, fmt.Errorf("error while compiling exclude glob %q: %s", exclude, err)
}
excludeGlobs = append(excludeGlobs, g)
}
}
// Inline function to check if excludes contains a key. // Inline function to check if excludes contains a key.
keyIsExcluded := func(Key string) bool { keyIsExcluded := func(key string) bool {
for _, exclude := range ExcludeKeys { for _, g := range excludeGlobs {
if globRegex.MatchString(exclude) { if g.Match(key) {
g, err := glob.Compile(exclude) return true
if err == nil && g != nil && g.Match(Key) {
return true
}
} }
if exclude == Key { }
for _, exclude := range req.ExcludeKeys {
if exclude == key {
return true return true
} }
} }
@ -55,101 +98,150 @@ func (a *App) PurgeCache(CachePath string, Key string, ExcludeKeys []string) err
} }
// Confirm that the cache path exists. // Confirm that the cache path exists.
if _, err := os.Stat(CachePath); err != nil { if _, err := os.Stat(req.CachePath); err != nil {
return fmt.Errorf("cache directory error: %s", err) return 0, fmt.Errorf("cache directory error: %s", err)
} }
// Check if the key is a wildcard. If its not, we should purge the key by hash. // Count of entries actually removed, reported to the caller.
if !globRegex.MatchString(Key) { purged := 0
// Check if the key is a wildcard. If its not, we should purge the key by
// hash, which is also the only thing an exact purge does.
if req.Exact || !globRegex.MatchString(req.Key) {
// If excluded, skip the key. // If excluded, skip the key.
if keyIsExcluded(Key) { if keyIsExcluded(req.Key) {
log.Println("Key", Key, "is excluded, will not purge.") log.Println("Key", req.Key, "is excluded, will not purge.")
return nil return 0, nil
} }
// Get the hash of the key. // Get the hash of the key.
hash := md5.Sum([]byte(Key)) hash := md5.Sum([]byte(req.Key))
keyHash := hex.EncodeToString(hash[:]) keyHash := hex.EncodeToString(hash[:])
// Find key in cache directory. // Find key in cache directory. The walk reads directory entries rather
err := filepath.Walk(CachePath, func(filePath string, info os.FileInfo, err error) error { // than calling Lstat on each one, as the name is all this branch
// Do not tolerate errors. // compares against and a cache holds a great many files.
err := filepath.WalkDir(req.CachePath, func(filePath string, entry fs.DirEntry, err error) error {
// Do not tolerate errors, other than an entry going away while
// we walk. Nginx maintains the cache as we read it, so entries
// disappearing mid-walk is expected rather than a failure.
if err != nil { if err != nil {
if os.IsNotExist(err) {
return nil
}
return err return err
} }
// We only care to look at files. // We only care to look at files.
if info.IsDir() { if entry.IsDir() {
return nil return nil
} }
// If this file matches our key hash then delete. // If this file matches our key hash then delete.
if info.Name() == keyHash { if entry.Name() == keyHash {
log.Printf("Purging %s as it matches the key %s requested to be purged.\n", filePath, Key) log.Printf("Purging %s as it matches the key %s requested to be purged.\n", filePath, req.Key)
err := os.Remove(filePath) err := os.Remove(filePath)
if err != nil { if err != nil && !os.IsNotExist(err) {
return err return err
} }
if err == nil {
purged++
}
// We're done, so lets stop the walk. // We're done, so lets stop the walk.
return filepath.SkipAll return filepath.SkipAll
} }
return nil return nil
}) })
if err != nil { if err != nil {
return fmt.Errorf("error while scanning for file to purge: %s", err) return purged, fmt.Errorf("error while scanning for file to purge: %s", err)
} }
} else { } else {
// This is a wildcard, so we need to find all files that match it and delete them. // This is a wildcard, so we need to find all files that match it and delete them.
g, err := glob.Compile(Key) g, err := glob.Compile(req.Key)
if err != nil { if err != nil {
return fmt.Errorf("error while compiling glob: %s", err) return 0, fmt.Errorf("error while compiling glob: %s", err)
} }
err = filepath.Walk(CachePath, func(filePath string, info os.FileInfo, err error) error { err = filepath.WalkDir(req.CachePath, func(filePath string, entry fs.DirEntry, err error) error {
// Do not tolerate errors. // Do not tolerate errors, other than an entry going away while
// we walk. Nginx maintains the cache as we read it, so entries
// disappearing mid-walk is expected rather than a failure.
if err != nil { if err != nil {
if os.IsNotExist(err) {
return nil
}
return err return err
} }
// We only care to look at files. // We only care to look at files.
if info.IsDir() { if entry.IsDir() {
return nil return nil
} }
// Read the file to extract the key. // Read the file to extract the key.
file, err := os.Open(filePath) file, err := os.Open(filePath)
if err != nil { if err != nil {
if os.IsNotExist(err) {
return nil
}
return err return err
} }
defer file.Close() keyRead := ""
scanner := bufio.NewScanner(file) keyFound := false
// Scan file for the key. // Scan file for the key. There is exactly one KEY line per cache
// entry, in the header, so stop at the first one found. Reading on
// would scan the cached body for a line that cannot exist, which
// is why the reader is capped at the header size rather than left
// to run through gigabytes of cached response body.
scanner := bufio.NewScanner(io.LimitReader(file, maxHeaderScan))
for scanner.Scan() { for scanner.Scan() {
line := scanner.Text() line := scanner.Text()
// If line is the key, check if it matches our glob pattern and delete.
if strings.HasPrefix(line, "KEY: ") { if strings.HasPrefix(line, "KEY: ") {
keyRead := line[5:] keyRead = line[5:]
if g.Match(keyRead) { keyFound = true
// If excluded, skip the key. break
if keyIsExcluded(keyRead) { }
log.Println("Key", keyRead, "is excluded, will not purge.") }
return nil scanErr := scanner.Err()
} file.Close()
// Delete the file. // A line longer than the scan limit means no cache header here,
log.Printf("Purging %s as it matches the key %s requested to be purged.\n", filePath, Key) // only binary, so this is not an entry we can match against. The
err := os.Remove(filePath) // temporary files nginx writes alongside the cache look exactly
if err != nil { // like this. Anything else is a real read error, which we surface
return err // rather than silently leaving a matching key in the cache.
} if scanErr != nil && !errors.Is(scanErr, bufio.ErrTooLong) && !os.IsNotExist(scanErr) {
break return fmt.Errorf("error while reading %s: %s", filePath, scanErr)
} }
// Without a key, there is nothing to match against.
if !keyFound {
return nil
}
// If the key matches our glob pattern, delete it.
if g.Match(keyRead) {
// If excluded, skip the key.
if keyIsExcluded(keyRead) {
log.Println("Key", keyRead, "is excluded, will not purge.")
return nil
}
// Delete the file. An entry nginx already evicted between the
// walk and here is one less file to purge, not a failure.
log.Printf("Purging %s with key %s as it matches %s requested to be purged.\n", filePath, keyRead, req.Key)
err := os.Remove(filePath)
if err != nil && !os.IsNotExist(err) {
return err
}
if err == nil {
purged++
} }
} }
return nil return nil
}) })
if err != nil { if err != nil {
return fmt.Errorf("error while scanning for file to purge: %s", err) return purged, fmt.Errorf("error while scanning for file to purge: %s", err)
} }
} }
return nil return purged, nil
} }
// Main function to start the app. // Main function to start the app.

13
purgeCmd.go
View file

@ -1,13 +0,0 @@
package main
// Purge command for CLI to purge cache keys.
type PurgeCmd struct {
CachePath string `arg:"" name:"cache-path" help:"Path to cache directory." type:"existingdir"`
Key string `arg:"" name:"key" help:"Cache key or wildcard match."`
ExcludeKeys []string `optional:"" name:"exclude-key" help:"Key to exclude, can be wild card and can add multiple excludes."`
}
// The purge command execution just runs the apps purge cache function.
func (a *PurgeCmd) Run() error {
return app.PurgeCache(a.CachePath, a.Key, a.ExcludeKeys)
}

28
purge_cmd.go Normal file
View file

@ -0,0 +1,28 @@
package main
import "log"
// Purge command for CLI to purge cache keys.
type PurgeCmd struct {
CachePath string `arg:"" name:"cache-path" help:"Path to cache directory." type:"existingdir"`
Key string `arg:"" name:"key" help:"Cache key or wildcard match."`
ExcludeKeys []string `optional:"" name:"exclude-key" help:"Key to exclude, can be wild card and can add multiple excludes."`
Exact bool `optional:"" name:"exact" help:"Treat the key and excludes as literal keys rather than wildcard matches."`
}
// The purge command execution just runs the apps purge cache function, then
// says how much it removed. A key that matched nothing is not an error, so the
// count is the only thing that distinguishes it from a purge that worked.
func (a *PurgeCmd) Run() error {
purged, err := app.PurgeCache(PurgeRequest{
CachePath: a.CachePath,
Key: a.Key,
ExcludeKeys: a.ExcludeKeys,
Exact: a.Exact,
})
if err != nil {
return err
}
log.Printf("Purged %d cache entries matching %s.\n", purged, a.Key)
return nil
}

75
serverCmd.go
View file

@ -1,75 +0,0 @@
package main
import (
"fmt"
"io"
"log"
"net"
"net/http"
"os"
)
// The server command for the CLI to run the HTTP server.
type ServerCmd struct {
Socket string `help:"Socket path for HTTP communication." type:"path"`
}
// Handle request.
func (a *ServerCmd) ServeHTTP(w http.ResponseWriter, req *http.Request) {
// Parse query parameters.
query := req.URL.Query()
cachePath := query.Get("path")
if cachePath == "" {
io.WriteString(w, "Need path parameter.")
http.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest)
return
}
key := query.Get("key")
if key == "" {
io.WriteString(w, "Need key parameter.")
http.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest)
return
}
excludes := query["exclude"]
// Purge cache.
err := app.PurgeCache(cachePath, key, excludes)
// If error, return error.
if err != nil {
fmt.Println("Error purging cache:", err)
io.WriteString(w, "Error occurred while processing purge.")
http.Error(w, http.StatusText(http.StatusBadGateway), http.StatusBadGateway)
return
}
// Successful purge.
w.Write([]byte("PURGED"))
}
// Start the FastCGI server.
func (a *ServerCmd) Run() error {
// Determine UNIX socket path.
unixSocket := a.Socket
if unixSocket == "" {
unixSocket = "/run/nginx-cache-purge/http.sock"
}
// If socket exists, remove it.
if _, err := os.Stat(unixSocket); !os.IsNotExist(err) {
os.Remove(unixSocket)
}
// Open the socket for FCGI communication.
listener, err := net.Listen("unix", unixSocket)
if err != nil {
return err
}
defer listener.Close()
// Start the FastCGI server.
log.Println("Starting server at", unixSocket)
http.HandleFunc("/", a.ServeHTTP)
err = http.Serve(listener, nil)
return err
}

360
server_cmd.go Normal file
View file

@ -0,0 +1,360 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"net"
"net/http"
"net/url"
"os"
"os/signal"
"path/filepath"
"slices"
"strconv"
"strings"
"syscall"
"time"
"github.com/coreos/go-systemd/v22/daemon"
"github.com/kardianos/service"
)
// Where the socket goes when the command line does not say. Matches the
// RuntimeDirectory the packaged systemd unit creates.
const defaultSocketPath = "/run/nginx-cache-purge/http.sock"
// What the socket's permissions are set to when the command line does not say.
// Connecting to a UNIX socket requires write permission on it, so this decides
// whether nginx can purge at all. Left to the umask a service manager starts us
// with, the socket comes out 0755, which no other user can connect to however
// the deployment is arranged.
const defaultSocketMode = "0660"
// stopChan carries a shutdown request from the service manager. It is
// buffered so a stop delivered after the signal loop exits cannot block the
// service supervisor.
var stopChan = make(chan struct{}, 1)
// The server command for the CLI to run the HTTP server.
type ServerCmd struct {
Socket string `help:"Socket path for HTTP communication (default ${defaultSocket})." type:"path"`
SocketMode string `help:"Octal permissions to give the socket." default:"${defaultMode}"`
CachePaths []string `name:"cache-path" help:"Cache directory that may be purged, can be repeated. Any path is purgeable when none is given."`
}
// allows reports whether a request may purge the given cache path. The path
// arrives from the caller, and the purge deletes what it finds there, so a
// server started with a list of cache directories will serve no other. Naming
// none keeps every path purgeable, which is what a server without the flag has
// always done.
func (a *ServerCmd) allows(cachePath string) bool {
if len(a.CachePaths) == 0 {
return true
}
target := resolvePath(cachePath)
for _, allowed := range a.CachePaths {
root := resolvePath(allowed)
// Compare by path element rather than by string prefix, so that
// /var/cache-other is not taken for a directory inside /var/cache.
relative, err := filepath.Rel(root, target)
if err != nil {
continue
}
if relative == "." {
return true
}
if relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
return true
}
}
return false
}
// resolvePath canonicalises a path so two spellings of one directory compare
// equal, following symlinks so that a link to an allowed cache is recognised as
// that cache. A path that cannot be resolved is only cleaned: it is one the
// purge is about to fail on anyway, and inventing a resolution for it could let
// it match an allowed directory it does not name.
func resolvePath(path string) string {
if absolute, err := filepath.Abs(path); err == nil {
path = absolute
}
if resolved, err := filepath.EvalSymlinks(path); err == nil {
return resolved
}
return filepath.Clean(path)
}
// socketMode is the permissions to give the socket, falling back to the default
// when the command line does not say.
func (a *ServerCmd) socketMode() (os.FileMode, error) {
mode := a.SocketMode
if mode == "" {
mode = defaultSocketMode
}
parsed, err := strconv.ParseUint(mode, 8, 32)
if err != nil || parsed > 0o777 {
return 0, fmt.Errorf("invalid socket mode %q, expected octal permissions such as %s", mode, defaultSocketMode)
}
return os.FileMode(parsed), nil
}
// rawQuery splits a query string into its parameters without percent-decoding
// them, which url.Values.Get would do.
func rawQuery(query string) url.Values {
values := make(url.Values)
for query != "" {
var parameter string
parameter, query, _ = strings.Cut(query, "&")
if parameter == "" {
continue
}
name, value, _ := strings.Cut(parameter, "=")
values[name] = append(values[name], value)
}
return values
}
// firstValue returns the first value given for a parameter, and whether it was
// present at all. url.Values.Get cannot tell a parameter that was left empty
// from one that was never sent.
func firstValue(values url.Values, name string) (string, bool) {
given, ok := values[name]
if !ok || len(given) == 0 {
return "", false
}
return given[0], true
}
// exactRequested reports whether the request asked for the key to be read as a
// literal. Like the other parameters it is taken first-wins, so a client whose
// request URI carries its own exact= cannot override the one the Nginx rewrite
// set ahead of the key. A value that is not a boolean is refused rather than
// assumed false: silently falling back to pattern matching is how a key holding
// ? or [ stops being purged while the response still reports that it was.
func exactRequested(raw, query url.Values) (bool, error) {
value, ok := firstValue(raw, "exact")
if !ok {
value, ok = firstValue(query, "exact")
}
if !ok {
return false, nil
}
// A bare exact, with no value at all, asks for it.
if value == "" {
return true, nil
}
exact, err := strconv.ParseBool(value)
if err != nil {
return false, fmt.Errorf("invalid exact parameter %q, expected a boolean such as 1", value)
}
return exact, nil
}
// Handle request.
func (a *ServerCmd) ServeHTTP(w http.ResponseWriter, req *http.Request) {
// Parse query parameters. The key is read twice: nginx substitutes
// $request_uri into the query string as-is, so the cache key it stored is
// the escapes and plus signs exactly as they arrive here, while a caller
// purging by hand is more likely to have escaped the key properly. Decoding
// is therefore a guess either way, so the purge tries both.
query := req.URL.Query()
raw := rawQuery(req.URL.RawQuery)
// The cache path names a directory rather than a cache key, so the decoded
// form is the one that matches what is on disk. A value the parser could
// not decode is dropped rather than reported, so fall back to what was sent
// instead of answering that no path was given.
cachePath := query.Get("path")
if cachePath == "" {
cachePath = raw.Get("path")
}
if cachePath == "" {
// http.Error must send the message itself. Writing the body first
// commits a 200 status, leaving the failure indistinguishable from
// a successful purge.
http.Error(w, "Need path parameter.", http.StatusBadRequest)
return
}
if !a.allows(cachePath) {
log.Println("Refusing to purge", cachePath, "as it is not an allowed cache path.")
http.Error(w, "Cache path is not allowed.", http.StatusForbidden)
return
}
key := raw.Get("key")
if key == "" {
http.Error(w, "Need key parameter.", http.StatusBadRequest)
return
}
exact, err := exactRequested(raw, query)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// An exclude asks for a key to be kept, so both readings of one are
// honoured rather than picking a side and purging what it named.
excludes := make([]string, 0, len(raw["exclude"])+len(query["exclude"]))
excludes = append(excludes, raw["exclude"]...)
for _, exclude := range query["exclude"] {
if !slices.Contains(excludes, exclude) {
excludes = append(excludes, exclude)
}
}
// Purge cache.
purge := PurgeRequest{
CachePath: cachePath,
Key: key,
ExcludeKeys: excludes,
Exact: exact,
}
purged, err := app.PurgeCache(purge)
// Nothing matched the key as it was sent, so try it decoded before giving
// up: that is the same key for all but the callers that escaped it.
if err == nil && purged == 0 {
if decoded := query.Get("key"); decoded != "" && decoded != key {
purge.Key = decoded
var decodedPurged int
decodedPurged, err = app.PurgeCache(purge)
if decodedPurged > 0 {
key = decoded
purged = decodedPurged
}
}
}
// If error, return error. The purge is this server's own work, so a
// failure is ours to report rather than a bad gateway upstream.
if err != nil {
log.Println("Error purging cache:", err)
http.Error(w, "Error occurred while processing purge.", http.StatusInternalServerError)
return
}
// Successful purge.
log.Printf("Purged %d cache entries matching %s.\n", purged, key)
w.Write([]byte("PURGED"))
}
// Bind the UNIX socket, replacing a socket left behind by an earlier run.
func (a *ServerCmd) listen(unixSocket string) (net.Listener, error) {
// Read the mode before binding, so an unusable one is reported without
// having taken the socket path over first.
mode, err := a.socketMode()
if err != nil {
return nil, err
}
// The socket directory may not exist yet, and net.Listen will not create
// it. Under systemd RuntimeDirectory this is already there.
if err := os.MkdirAll(filepath.Dir(unixSocket), 0o755); err != nil {
return nil, fmt.Errorf("unable to create socket directory: %s", err)
}
// A socket file from a previous run has to go before we can bind, but
// removing one that another instance is still serving would silently take
// over its requests. Connecting tells the two apart: a refused connection
// means nothing is listening.
info, err := os.Lstat(unixSocket)
switch {
case err == nil && info.Mode()&os.ModeSocket == 0:
return nil, fmt.Errorf("%s exists and is not a socket", unixSocket)
case err == nil:
conn, dialErr := net.DialTimeout("unix", unixSocket, time.Second)
if dialErr == nil {
conn.Close()
return nil, fmt.Errorf("%s is already in use by another instance", unixSocket)
}
if err := os.Remove(unixSocket); err != nil && !os.IsNotExist(err) {
return nil, fmt.Errorf("unable to remove stale socket: %s", err)
}
case !os.IsNotExist(err):
return nil, fmt.Errorf("unable to check socket path: %s", err)
}
listener, err := net.Listen("unix", unixSocket)
if err != nil {
return nil, err
}
// Bind leaves the socket at whatever the umask allows, so set the mode
// rather than let the environment we were started from decide who can
// reach the purge.
if err := os.Chmod(unixSocket, mode); err != nil {
listener.Close()
return nil, fmt.Errorf("unable to set socket permissions: %s", err)
}
return listener, nil
}
// Start the HTTP server.
func (a *ServerCmd) Run() error {
// Determine UNIX socket path.
unixSocket := a.Socket
if unixSocket == "" {
unixSocket = defaultSocketPath
}
listener, err := a.listen(unixSocket)
if err != nil {
return err
}
defer listener.Close()
// Start the HTTP server. Use our own mux rather than the global default
// one so the handler registration is scoped to this server.
log.Println("Starting server at", unixSocket)
mux := http.NewServeMux()
mux.HandleFunc("/", a.ServeHTTP)
server := &http.Server{
Handler: mux,
ReadHeaderTimeout: 10 * time.Second,
}
// Shut down on signal instead of dying where we stand, so the listener
// gets closed and the socket file is unlinked for the next run. No write
// timeout is set on the server: purging a large cache walks every entry,
// and a deadline would cut the response off mid-purge.
stop := make(chan os.Signal, 1)
signal.Notify(stop, os.Interrupt, syscall.SIGTERM)
defer signal.Stop(stop)
serveErr := make(chan error, 1)
go func() {
serveErr <- server.Serve(listener)
}()
// Attach to the service manager when not run interactively, so a stop
// request arrives on stopChan, then report readiness now that the socket
// is bound and requests can be served.
if !service.Interactive() {
svc, err := new(ServiceCmd).service()
if err != nil {
return err
}
go svc.Run()
}
_, _ = daemon.SdNotify(false, daemon.SdNotifyReady)
select {
case err := <-serveErr:
return err
case <-stop:
case <-stopChan:
}
log.Println("Shutting down server.")
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
return err
}
// Serve always ends with an error; a closed server is the expected one.
if err := <-serveErr; !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}

628
server_cmd_test.go Normal file
View file

@ -0,0 +1,628 @@
package main
import (
"bytes"
"context"
"crypto/md5"
"encoding/hex"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"syscall"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// Build a file that looks like an entry nginx wrote: a binary header, then the
// "\nKEY: <key>\n" line the purge relies on, then the cached response.
func cacheEntry(key string) []byte {
var buf bytes.Buffer
// Stand in for ngx_http_file_cache_header_t. Any byte but LF works, as the
// header is opaque to us and only the KEY line is parsed.
for i := 0; i < 56; i++ {
buf.WriteByte(byte(i%9) + 1)
}
buf.WriteString("\nKEY: " + key + "\n")
buf.WriteString("HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nhello")
return buf.Bytes()
}
// Populate a cache with a known set of keys, writing each entry where nginx
// would put it: hashed name under levels=1:2 directories. Returns the cache
// path and the file each key was written to.
func newCache(t *testing.T, keys ...string) (string, map[string]string) {
t.Helper()
cachePath := t.TempDir()
paths := make(map[string]string, len(keys))
for _, key := range keys {
sum := md5.Sum([]byte(key))
name := hex.EncodeToString(sum[:])
dir := filepath.Join(cachePath, name[31:], name[29:31])
require.NoError(t, os.MkdirAll(dir, 0o755))
paths[key] = filepath.Join(dir, name)
require.NoError(t, os.WriteFile(paths[key], cacheEntry(key), 0o644))
}
return cachePath, paths
}
func exists(t *testing.T, path string) bool {
t.Helper()
_, err := os.Lstat(path)
return err == nil
}
// The handler reaches the purge through the package global, as the CLI wires it.
func withApp(t *testing.T) {
t.Helper()
previous := app
app = new(App)
t.Cleanup(func() { app = previous })
}
// Send a purge to a given server, for the tests that need one configured.
func purgeRawTo(t *testing.T, cmd *ServerCmd, query string) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest(http.MethodGet, "/?"+query, nil)
w := httptest.NewRecorder()
cmd.ServeHTTP(w, req)
return w
}
// Send a purge with the query string exactly as given, so a test can pass a key
// the way nginx does rather than the way url.Values would escape it.
func purgeRaw(t *testing.T, query string) *httptest.ResponseRecorder {
t.Helper()
return purgeRawTo(t, new(ServerCmd), query)
}
func purgeRequest(t *testing.T, query url.Values) *httptest.ResponseRecorder {
t.Helper()
return purgeRaw(t, query.Encode())
}
func TestPurgeByKey(t *testing.T) {
tests := []struct {
name string
key string
purged []string
kept []string
}{
{
name: "exact key",
key: "example.com/index.html",
purged: []string{"example.com/index.html"},
kept: []string{"example.com/img/logo.png", "other.com/index.html"},
},
{
// The glob has no separator, so it spans path segments in the key.
name: "wildcard",
key: "example.com/*",
purged: []string{"example.com/index.html", "example.com/img/logo.png"},
kept: []string{"other.com/index.html"},
},
{
// A key that is not in the cache is not a failure; there is simply
// nothing to remove, the common case when purges race each other.
name: "key not in cache",
key: "example.com/absent.html",
kept: []string{"example.com/index.html", "example.com/img/logo.png", "other.com/index.html"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t,
"example.com/index.html",
"example.com/img/logo.png",
"other.com/index.html",
)
w := purgeRequest(t, url.Values{"path": {cachePath}, "key": {test.key}})
require.Equal(t, http.StatusOK, w.Code)
require.Equal(t, "PURGED", w.Body.String())
for _, key := range test.purged {
require.False(t, exists(t, paths[key]), "%s was not purged", key)
}
for _, key := range test.kept {
require.True(t, exists(t, paths[key]), "%s was purged", key)
}
})
}
}
func TestPurgeExcludes(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t,
"example.com/index.html",
"example.com/keep.html",
"example.com/img/logo.png",
)
// Nginx repeats the parameter to pass more than one exclude.
w := purgeRequest(t, url.Values{
"path": {cachePath},
"key": {"example.com/*"},
"exclude": {"example.com/keep.html", "example.com/img/*"},
})
require.Equal(t, http.StatusOK, w.Code)
require.False(t, exists(t, paths["example.com/index.html"]), "matching key was not purged")
require.True(t, exists(t, paths["example.com/keep.html"]), "literal exclude was purged")
require.True(t, exists(t, paths["example.com/img/logo.png"]), "glob exclude was purged")
}
// Nginx substitutes $request_uri into the query string without escaping it, so
// the cache key it stored is the one on the wire byte for byte. Decoding it
// would look for a key nginx never wrote, and answer PURGED having done
// nothing. A caller escaping the key properly still has its purge land, so both
// conventions work over the same socket.
func TestPurgeKeyEncoding(t *testing.T) {
tests := []struct {
name string
key string
sent string
}{
{"percent escape as sent", "example.com/caf%C3%A9.html", "example.com/caf%C3%A9.html"},
{"plus sign as sent", "example.com/a+b.html", "example.com/a+b.html"},
{"percent sign as sent", "example.com/100%.html", "example.com/100%.html"},
{"escaped by the caller", "example.com/café.html", "example.com/caf%C3%A9.html"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, test.key, "other.com/index.html")
w := purgeRaw(t, "path="+url.QueryEscape(cachePath)+"&key="+test.sent)
require.Equal(t, http.StatusOK, w.Code)
require.False(t, exists(t, paths[test.key]), "key was not purged")
require.True(t, exists(t, paths["other.com/index.html"]), "unrelated key was purged")
})
}
}
// Excludes carry the same ambiguity as the key, and an exclude asks for a key
// to be kept, so either reading of one has to be enough to keep it.
func TestPurgeExcludeEncoding(t *testing.T) {
for _, exclude := range []string{"example.com/a+b.html", "example.com/a%2Bb.html"} {
t.Run(exclude, func(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/a+b.html", "example.com/index.html")
w := purgeRaw(t, "path="+url.QueryEscape(cachePath)+"&key=example.com/*&exclude="+exclude)
require.Equal(t, http.StatusOK, w.Code)
require.True(t, exists(t, paths["example.com/a+b.html"]), "excluded key was purged")
require.False(t, exists(t, paths["example.com/index.html"]), "matching key was not purged")
})
}
}
// Nginx writes partial responses to temporary files in the same tree. They have
// no cache header, so they must be left alone rather than read as entries, and
// a KEY line past the header scan belongs to a cached body rather than a header.
func TestPurgeLeavesNonEntriesAlone(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/index.html")
// Longer than the scanner's buffer with no line break: the shape of a
// body-only temporary file.
temporary := filepath.Join(cachePath, "0000000123")
require.NoError(t, os.WriteFile(temporary, bytes.Repeat([]byte{'a'}, maxHeaderScan*2), 0o644))
// An empty file is the other thing a half-written entry looks like.
empty := filepath.Join(cachePath, "0000000124")
require.NoError(t, os.WriteFile(empty, nil, 0o644))
// Short lines, so the scanner keeps reading until the limit cuts it off.
var body bytes.Buffer
for body.Len() < maxHeaderScan {
body.WriteString("filler line\n")
}
body.WriteString("KEY: example.com/deep.html\n")
deep := filepath.Join(cachePath, "0000000125")
require.NoError(t, os.WriteFile(deep, body.Bytes(), 0o644))
w := purgeRequest(t, url.Values{"path": {cachePath}, "key": {"*"}})
require.Equal(t, http.StatusOK, w.Code)
require.False(t, exists(t, paths["example.com/index.html"]), "matching key was not purged")
require.True(t, exists(t, temporary), "temporary file was purged")
require.True(t, exists(t, empty), "empty file was purged")
require.True(t, exists(t, deep), "file whose KEY line is past the header scan was purged")
}
// A cache key built from a request URI carries the punctuation that decides
// whether a key is read as a pattern: a query string puts ? in the key, and
// PHP-style array parameters put [ and ]. Neither was meant as a wildcard, and
// read as one they purge the wrong thing, fail to compile, or match nothing
// while still answering PURGED. Exact says the key is a literal, which is the
// only way those entries can be purged at all.
func TestPurgeExactKey(t *testing.T) {
keys := []string{
"example.com/index.html",
"example.com/page.html?id=5",
"example.com/list.php?f[]=x",
"example.com/{weird}",
// A key that holds what would otherwise be a wildcard purging the lot.
"example.com/*",
}
for _, key := range keys {
t.Run(key, func(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, key, "other.com/index.html")
// Sent unescaped, the way the Nginx rewrite substitutes it.
w := purgeRaw(t, "path="+url.QueryEscape(cachePath)+"&exact=1&key="+key)
require.Equal(t, http.StatusOK, w.Code)
require.Equal(t, "PURGED", w.Body.String())
require.False(t, exists(t, paths[key]), "key was not purged")
require.True(t, exists(t, paths["other.com/index.html"]), "unrelated key was purged")
})
}
}
// Without exact those same keys are patterns, which is what the parameter
// exists to turn off. The wildcard purging everything under it is the case that
// makes reading a literal key as a pattern dangerous rather than merely wrong.
func TestPurgeWithoutExactReadsKeyAsPattern(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/*", "other.com/index.html")
w := purgeRaw(t, "path="+url.QueryEscape(cachePath)+"&key=example.com/*")
require.Equal(t, http.StatusOK, w.Code)
require.False(t, exists(t, paths["example.com/*"]), "the pattern matched its own key")
require.True(t, exists(t, paths["other.com/index.html"]), "unrelated key was purged")
}
// An exclude carries the same punctuation as a key, so an exact purge reads it
// literally too. Compiled as a pattern, an exclude holding [ fails the whole
// purge it appears in.
func TestPurgeExactExcludes(t *testing.T) {
const excluded = "example.com/list.php?f[]=x"
withApp(t)
cachePath, paths := newCache(t, excluded, "example.com/other.php")
query := "path=" + url.QueryEscape(cachePath) + "&key=example.com/*&exclude=" + excluded
// As a pattern the exclude will not compile, and a purge that cannot honour
// an exclude must not run.
w := purgeRawTo(t, new(ServerCmd), query)
require.Equal(t, http.StatusInternalServerError, w.Code)
require.True(t, exists(t, paths[excluded]), "purged despite the failure")
// Exact reads both the key and the exclude literally, so the key named by
// the exclude is the one kept.
w = purgeRawTo(t, new(ServerCmd), query+"&exact=1")
require.Equal(t, http.StatusOK, w.Code)
require.True(t, exists(t, paths[excluded]), "excluded key was purged")
// The key is a literal too, so it purges only itself and not the sibling.
require.True(t, exists(t, paths["example.com/other.php"]), "unrelated key was purged")
}
// A value that is not a boolean has to be refused. Falling back to pattern
// matching on a typo is how a key stops being purged while the response still
// reports that it was.
func TestPurgeExactValues(t *testing.T) {
tests := []struct {
value string
exact bool
bad bool
}{
{value: "1", exact: true},
{value: "true", exact: true},
{value: "0", exact: false},
{value: "false", exact: false},
// Present with no value at all still asks for it.
{value: "", exact: true},
{value: "yes", bad: true},
{value: "2", bad: true},
}
for _, test := range tests {
t.Run("exact="+test.value, func(t *testing.T) {
withApp(t)
// Read as a pattern this key is an alternation that matches
// nothing on disk, so only an exact purge removes it.
const key = "example.com/{weird}"
cachePath, paths := newCache(t, key)
parameter := "&exact"
if test.value != "" {
parameter += "=" + test.value
}
w := purgeRaw(t, "path="+url.QueryEscape(cachePath)+parameter+"&key="+key)
if test.bad {
require.Equal(t, http.StatusBadRequest, w.Code)
require.True(t, exists(t, paths[key]), "purged despite the bad parameter")
return
}
require.Equal(t, http.StatusOK, w.Code)
require.Equal(t, test.exact, !exists(t, paths[key]), "key purged with exact=%v", test.exact)
})
}
}
// A missing parameter has to come back as a failure status. Answering 200 with
// an error in the body reads to nginx as a successful purge.
func TestPurgeRejectsMissingParameters(t *testing.T) {
withApp(t)
cachePath, _ := newCache(t)
tests := []struct {
name string
query url.Values
}{
{"no path", url.Values{"key": {"example.com/*"}}},
{"no key", url.Values{"path": {cachePath}}},
{"empty path", url.Values{"path": {""}, "key": {"example.com/*"}}},
{"empty key", url.Values{"path": {cachePath}, "key": {""}}},
{"neither", url.Values{}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
w := purgeRequest(t, test.query)
require.Equal(t, http.StatusBadRequest, w.Code)
require.NotEqual(t, "PURGED", w.Body.String())
})
}
}
// A purge that could not be carried out has to say so, rather than answer
// PURGED and leave the caller believing the keys are gone. An exclude that
// cannot compile is the sharpest case: carrying on would delete the very keys
// the caller asked to keep.
func TestPurgeReportsFailure(t *testing.T) {
tests := []struct {
name string
absentPath bool
key string
exclude string
}{
{name: "cache path does not exist", absentPath: true, key: "example.com/*"},
{name: "key glob will not compile", key: "example.com/[a-"},
{name: "exclude glob will not compile", key: "example.com/*", exclude: "example.com/[a-"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/index.html")
query := url.Values{"path": {cachePath}, "key": {test.key}}
if test.absentPath {
query.Set("path", filepath.Join(cachePath, "absent"))
}
if test.exclude != "" {
query.Set("exclude", test.exclude)
}
w := purgeRequest(t, query)
require.Equal(t, http.StatusInternalServerError, w.Code)
require.NotEqual(t, "PURGED", w.Body.String())
require.True(t, exists(t, paths["example.com/index.html"]), "purged despite the failure")
})
}
}
// The path to purge comes from the caller, and the purge deletes what it finds
// under it, so a server given cache directories has to serve no others.
func TestServerAllowsCachePath(t *testing.T) {
base := t.TempDir()
allowed := filepath.Join(base, "cache")
// A sibling whose name starts with the allowed one, which a plain string
// prefix would take for a directory inside it.
sibling := filepath.Join(base, "cache-other")
require.NoError(t, os.MkdirAll(allowed, 0o755))
require.NoError(t, os.MkdirAll(sibling, 0o755))
link := filepath.Join(base, "link")
require.NoError(t, os.Symlink(allowed, link))
tests := []struct {
name string
roots []string
path string
want bool
}{
// A server without the flag purges wherever it is told, as it always has.
{"no allowlist", nil, sibling, true},
{"the named directory", []string{allowed}, allowed, true},
{"a directory inside it", []string{allowed}, filepath.Join(allowed, "inner"), true},
{"an unclean spelling of it", []string{allowed}, filepath.Join(allowed, "inner", ".."), true},
{"one of several", []string{sibling, allowed}, allowed, true},
// Symlinks are resolved, so a link to the cache is the cache whichever
// side of the comparison it is written on.
{"a symlink to it", []string{allowed}, link, true},
{"named by a symlink to it", []string{link}, allowed, true},
{"a sibling sharing its name", []string{allowed}, sibling, false},
{"the directory above it", []string{allowed}, base, false},
{"an unrelated directory", []string{allowed}, t.TempDir(), false},
{"the root", []string{allowed}, "/", false},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cmd := &ServerCmd{CachePaths: test.roots}
require.Equal(t, test.want, cmd.allows(test.path))
})
}
}
// A path the server will not purge has to be refused outright, rather than
// walked and reported on.
func TestPurgeRefusesCachePathOutsideAllowlist(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/index.html")
cmd := &ServerCmd{CachePaths: []string{t.TempDir()}}
w := purgeRawTo(t, cmd, "path="+url.QueryEscape(cachePath)+"&key=*")
require.Equal(t, http.StatusForbidden, w.Code)
require.NotEqual(t, "PURGED", w.Body.String())
require.True(t, exists(t, paths["example.com/index.html"]), "purged from a path that is not allowed")
}
// Connecting to a UNIX socket needs write permission on it, so the mode has to
// be set rather than left to the umask the service manager started us with, and
// a mode that cannot be used has to stop the server before it takes the socket
// path over.
func TestListenSocketMode(t *testing.T) {
tests := []struct {
name string
mode string
want os.FileMode
}{
{"default", "", 0o660},
{"from the command line", "0666", 0o666},
{"not a number", "junk", 0},
{"not octal", "0999", 0},
{"out of range", "1777", 0},
{"negative", "-1", 0},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cmd := &ServerCmd{SocketMode: test.mode}
socket := filepath.Join(t.TempDir(), "http.sock")
listener, err := cmd.listen(socket)
if test.want == 0 {
require.Error(t, err)
require.False(t, exists(t, socket), "socket was bound despite the mode being rejected")
return
}
require.NoError(t, err)
defer listener.Close()
info, err := os.Lstat(socket)
require.NoError(t, err)
require.Equal(t, test.want, info.Mode().Perm())
})
}
}
func TestListenSocketPath(t *testing.T) {
// The socket directory may not exist yet, and net.Listen will not create it.
t.Run("creates the socket directory", func(t *testing.T) {
socket := filepath.Join(t.TempDir(), "run", "http.sock")
listener, err := new(ServerCmd).listen(socket)
require.NoError(t, err)
defer listener.Close()
info, err := os.Lstat(socket)
require.NoError(t, err, "socket was not created")
require.NotZero(t, info.Mode()&os.ModeSocket, "path exists but is not a socket")
})
// A socket left behind by a killed run must not stop the next one binding.
t.Run("replaces a stale socket", func(t *testing.T) {
socket := filepath.Join(t.TempDir(), "http.sock")
// Closing normally unlinks the file, so keep it to look like a crash.
stale, err := net.Listen("unix", socket)
require.NoError(t, err)
stale.(*net.UnixListener).SetUnlinkOnClose(false)
require.NoError(t, stale.Close())
require.True(t, exists(t, socket), "stale socket was not left behind")
listener, err := new(ServerCmd).listen(socket)
require.NoError(t, err, "listen over stale socket")
require.NoError(t, listener.Close())
})
// Taking over a socket another instance is serving would silently steal its
// requests, so binding has to fail instead.
t.Run("refuses a live socket", func(t *testing.T) {
socket := filepath.Join(t.TempDir(), "http.sock")
live, err := net.Listen("unix", socket)
require.NoError(t, err)
defer live.Close()
_, err = new(ServerCmd).listen(socket)
require.Error(t, err)
require.True(t, exists(t, socket), "live socket was removed")
})
// A path holding something other than a socket is not ours to unlink.
t.Run("refuses a path that is not a socket", func(t *testing.T) {
socket := filepath.Join(t.TempDir(), "http.sock")
require.NoError(t, os.WriteFile(socket, nil, 0o644))
_, err := new(ServerCmd).listen(socket)
require.Error(t, err)
require.True(t, exists(t, socket), "the existing file was removed")
})
}
// The whole command: bind the socket given on the command line, serve a purge
// over it, and shut down on the signal systemd sends to stop the service.
func TestRunServesAndShutsDown(t *testing.T) {
withApp(t)
cachePath, paths := newCache(t, "example.com/index.html", "other.com/index.html")
socket := filepath.Join(t.TempDir(), "http.sock")
cmd := &ServerCmd{Socket: socket}
runErr := make(chan error, 1)
go func() { runErr <- cmd.Run() }()
client := &http.Client{
Transport: &http.Transport{
DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, "unix", socket)
},
},
}
// Run binds asynchronously, so wait for the socket to answer.
var resp *http.Response
query := url.Values{"path": {cachePath}, "key": {"example.com/*"}}
deadline := time.Now().Add(5 * time.Second)
for {
var err error
resp, err = client.Get("http://socket/purge?" + query.Encode())
if err == nil {
break
}
require.False(t, time.Now().After(deadline), "server never accepted a connection: %s", err)
time.Sleep(10 * time.Millisecond)
}
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "PURGED", string(body))
require.False(t, exists(t, paths["example.com/index.html"]), "matching key was not purged")
require.True(t, exists(t, paths["other.com/index.html"]), "unrelated key was purged")
// The command handles SIGTERM itself, so this stops the server rather than
// the test binary.
require.NoError(t, syscall.Kill(syscall.Getpid(), syscall.SIGTERM))
select {
case err := <-runErr:
require.NoError(t, err, "run returned an error on shutdown")
case <-time.After(15 * time.Second):
t.Fatal("server did not shut down on SIGTERM")
}
// Shutting down closes the listener, which unlinks the socket for the run
// after this one.
require.False(t, exists(t, socket), "socket file was left behind")
}
// A socket that cannot be bound is the command failing, not the server running
// with nowhere to listen.
func TestRunReportsListenFailure(t *testing.T) {
socket := filepath.Join(t.TempDir(), "http.sock")
require.NoError(t, os.WriteFile(socket, nil, 0o644))
require.Error(t, (&ServerCmd{Socket: socket}).Run())
}

169
service_cmd.go Normal file
View file

@ -0,0 +1,169 @@
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/kardianos/service"
)
// systemdScript is the unit template used when installing the service. It
// replaces the library default to run as a notify service with automatic
// restart, so systemd only considers the server started once the socket is
// bound. RuntimeDirectory gives the socket a directory systemd creates on
// start and removes on stop.
const systemdScript = `[Unit]
Description={{Description}}
ConditionFileIsExecutable={{Path | cmdEscape}}
{{range Dependencies}}{{.}}
{{end}}StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
Type=notify
ExecStart={{Path | cmdEscape}}{{range Arguments}} {{. | cmd}}{{end}}
{{if ChRoot}}RootDirectory={{ChRoot | cmd}}
{{end}}{{if WorkingDirectory}}WorkingDirectory={{WorkingDirectory | cmdEscape}}
{{end}}{{if UserName}}User={{UserName}}
{{end}}{{if ReloadSignal}}ExecReload=/bin/kill -{{ReloadSignal}} "$MAINPID"
{{end}}{{if PIDFile}}PIDFile={{PIDFile | cmd}}
{{end}}{{if OutputFileSupport}}StandardOutput=file:{{LogDirectory}}/{{Name}}.out
StandardError=file:{{LogDirectory}}/{{Name}}.err
{{end}}{{if LimitNOFILE}}LimitNOFILE={{LimitNOFILE}}
{{end}}{{if Restart}}Restart={{Restart}}
{{end}}{{if SuccessExitStatus}}SuccessExitStatus={{SuccessExitStatus}}
{{end}}RuntimeDirectory={{Name}}
RestartSec=5
EnvironmentFile=-/etc/sysconfig/{{Name}}
{{range EnvVars}}{{.}}
{{end}}[Install]
WantedBy=multi-user.target
`
// ServiceAction lists the accepted service actions, in the order the Run
// switch handles them.
var ServiceAction = []string{"start", "stop", "status", "restart", "install", "uninstall"}
// ServiceCmd manages the purge server as a system service.
type ServiceCmd struct {
Action string `arg:"" enum:"${serviceActions}" help:"${serviceActions}" required:""`
CachePaths []string `name:"cache-path" type:"path" help:"Cache directory the installed service may purge, can be repeated. Any path is purgeable when none is given."`
}
// action returns the requested service action.
func (s *ServiceCmd) action() string {
return s.Action
}
// arguments builds the command line the installed unit runs. The allowlist
// belongs in the unit rather than in a drop-in written afterwards: a service
// installed with cache paths then serves no others from its first start.
func (s *ServiceCmd) arguments() []string {
arguments := make([]string, 0, 1+2*len(s.CachePaths))
arguments = append(arguments, "server")
for _, cachePath := range s.CachePaths {
// The unit runs from a working directory of the service manager's
// choosing, so a relative path here would name a different directory
// than the one the install was typed against. Symlinks are left alone:
// the server resolves them per request, and baking the target into the
// unit would pin the allowlist to wherever the link pointed at install.
if absolute, err := filepath.Abs(cachePath); err == nil {
cachePath = absolute
}
arguments = append(arguments, "--cache-path", cachePath)
}
return arguments
}
// Run performs the requested action against the installed service.
func (s *ServiceCmd) Run() (err error) {
// The allowlist is written into the unit, so it only takes effect at
// install. Accepting it on the other actions would read as having changed
// the allowlist of a service that carries on with the one it was installed
// with, which is the sort of misreading that leaves a cache purgeable.
if len(s.CachePaths) != 0 && s.action() != ServiceAction[4] {
return fmt.Errorf("--cache-path only applies to %s; reinstall the service to change it", ServiceAction[4])
}
svc, err := s.service()
if err != nil {
return err
}
switch s.action() {
case ServiceAction[0]:
err = svc.Start()
case ServiceAction[1]:
err = svc.Stop()
case ServiceAction[2]:
var status service.Status
status, err = svc.Status()
if err == nil {
switch status {
case service.StatusRunning:
fmt.Println("Service is running.")
case service.StatusStopped:
fmt.Println("Service is stopped.")
default:
fmt.Println("Service is in an unknown state.")
}
}
case ServiceAction[3]:
err = svc.Restart()
case ServiceAction[4]:
// A mistyped cache path installs cleanly and then refuses every purge
// of the cache it was meant to name, so say so now rather than leave it
// to be found by a 403. A cache directory nginx has not created yet is
// the same shape, which is why this is a warning and not a failure.
for _, cachePath := range s.CachePaths {
if _, statErr := os.Stat(cachePath); statErr != nil {
fmt.Printf("Warning: cache path %s cannot be read: %s\n", cachePath, statErr)
}
}
err = svc.Install()
case ServiceAction[5]:
err = svc.Uninstall()
}
if err != nil {
return err
}
// Status already printed its own result.
if s.action() != ServiceAction[2] {
fmt.Println("Command executed successfully.")
}
return
}
// service builds the service definition shared by the management actions and
// by the server when it is started by the service manager.
func (s *ServiceCmd) service() (service.Service, error) {
svcConfig := &service.Config{
Name: Name,
DisplayName: DisplayName,
Description: Description,
Arguments: s.arguments(),
Dependencies: []string{"After=network.target"},
Option: service.KeyValue{
"SystemdScript": systemdScript,
"Restart": "always",
},
}
return service.New(s, svcConfig)
}
// Start satisfies service.Interface. The server is already running in the
// foreground by the time the supervisor attaches.
func (s *ServiceCmd) Start(svc service.Service) error {
return nil
}
// Stop satisfies service.Interface, signalling the server's shutdown. The send
// cannot block: a shutdown already under way leaves nothing reading the
// channel, and holding the supervisor here would stall the stop it asked for.
func (s *ServiceCmd) Stop(svc service.Service) error {
select {
case stopChan <- struct{}{}:
default:
}
return nil
}

121
service_cmd_test.go Normal file
View file

@ -0,0 +1,121 @@
package main
import (
"os"
"path/filepath"
"testing"
"github.com/alecthomas/kong"
"github.com/stretchr/testify/require"
)
// The allowlist has to reach the installed unit's ExecStart, as that is the
// only place the server it starts reads it from.
func TestServiceArguments(t *testing.T) {
tests := []struct {
name string
cachePaths []string
want []string
}{
{
// What the command has always installed, so a service installed
// without the flag runs exactly as it did before.
name: "no allowlist",
want: []string{"server"},
},
{
name: "one cache path",
cachePaths: []string{"/var/nginx/proxy_temp/cache"},
want: []string{"server", "--cache-path", "/var/nginx/proxy_temp/cache"},
},
{
// The flag repeats, and each one has to arrive as its own argument
// rather than joined into a value the server reads as one path.
name: "several cache paths",
cachePaths: []string{"/var/cache/one", "/var/cache/two"},
want: []string{
"server",
"--cache-path", "/var/cache/one",
"--cache-path", "/var/cache/two",
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cmd := &ServiceCmd{Action: "install", CachePaths: test.cachePaths}
require.Equal(t, test.want, cmd.arguments())
// The same arguments have to be what the service definition is
// built with, or the unit is written without them.
svc, err := cmd.service()
require.NoError(t, err)
require.NotNil(t, svc)
})
}
}
// The unit runs from a working directory of the service manager's choosing, so
// a relative path would name a different directory there than the one the
// install was typed against.
func TestServiceArgumentsAbsolutePaths(t *testing.T) {
cmd := &ServiceCmd{Action: "install", CachePaths: []string{"cache"}}
arguments := cmd.arguments()
require.Len(t, arguments, 3)
require.True(t, filepath.IsAbs(arguments[2]), "%s is not absolute", arguments[2])
require.Equal(t, "cache", filepath.Base(arguments[2]))
}
// A symlinked cache is resolved per request by the server, so the install must
// leave the link alone rather than pin the allowlist to today's target.
func TestServiceArgumentsKeepsSymlinks(t *testing.T) {
base := t.TempDir()
target := filepath.Join(base, "cache")
link := filepath.Join(base, "link")
require.NoError(t, os.MkdirAll(target, 0o755))
require.NoError(t, os.Symlink(target, link))
cmd := &ServiceCmd{Action: "install", CachePaths: []string{link}}
require.Equal(t, []string{"server", "--cache-path", link}, cmd.arguments())
}
// The allowlist is written into the unit at install, so accepting it on an
// action that cannot act on it would read as having changed the allowlist of a
// service that carries on with the one it was installed with.
func TestServiceRejectsCachePathsOutsideInstall(t *testing.T) {
for _, action := range ServiceAction {
if action == "install" {
continue
}
t.Run(action, func(t *testing.T) {
cmd := &ServiceCmd{Action: action, CachePaths: []string{t.TempDir()}}
err := cmd.Run()
require.Error(t, err)
require.Contains(t, err.Error(), "--cache-path")
})
}
}
// The flag has to be reachable from the command line it is documented on, and
// kong's path type has to expand each value rather than only the first.
func TestServiceCachePathFlagParses(t *testing.T) {
flags := &Flags{}
parser, err := kong.New(flags, kong.Name(Name), kongVars())
require.NoError(t, err)
_, err = parser.Parse([]string{
"service", "install",
"--cache-path", "/var/cache/one",
"--cache-path", "relative/cache",
})
require.NoError(t, err)
require.Equal(t, "install", flags.Service.Action)
require.Len(t, flags.Service.CachePaths, 2)
require.Equal(t, "/var/cache/one", flags.Service.CachePaths[0])
require.True(t, filepath.IsAbs(flags.Service.CachePaths[1]), "relative path was not expanded")
}