diff options
190 files changed, 10424 insertions, 3484 deletions
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 76a1bc6..666ddef 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -1,7 +1,7 @@ Contributing to Caddy ===================== -Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be great without your involvement! +Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be nearly as excellent without your involvement! For starters, we invite you to join [the Caddy forum](https://caddy.community) where you can hang out with other Caddy users and developers. @@ -35,19 +35,29 @@ Here are some of the expectations we have of contributors: - **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other. -- **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass. +- **Write tests.** Good, automated tests are very valuable! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass. -- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling. +- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks and profiling. - **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft <diverging-commit>` then `git commit -s`. -- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged. +- **Be responsible for and maintain your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged. - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious. -- **Pull requests may still get closed.** The longer a PR stays open and idle, the more likely it is to be closed. If we haven't reviewed it in a while, it probably means the change is not a priority. Please don't take this personally, we're trying to balance a lot of tasks! If nobody else has commented or reacted to the PR, it likely means your change is useful only to you. The reality is this happens quite a bit. We don't tend to accept PRs that aren't generally helpful. For these reasons or others, the PR may get closed even after a review. We are not obligated to accept all proposed changes, even if the best justification we can give is something vague like, "It doesn't sit right." Sometimes PRs are just the wrong thing or the wrong time. Because it is open source, you can always build your own modified version of Caddy with a change you need, even if we reject it in the official repo. +- **Pull requests may still get closed.** The longer a PR stays open and idle, the more likely it is to be closed. If we haven't reviewed it in a while, it probably means the change is not a priority. Please don't take this personally, we're trying to balance a lot of tasks! If nobody else has commented or reacted to the PR, it likely means your change is useful only to you. The reality is this happens quite a lot. We don't tend to accept PRs that aren't generally helpful. For these reasons or others, the PR may get closed even after a review. We are not obligated to accept all proposed changes, even if the best justification we can give is something vague like, "It doesn't sit right." Sometimes PRs are just the wrong thing or the wrong time. Because it is open source, you can always build your own modified version of Caddy with a change you need, even if we reject it in the official repo. Plus, because Caddy is extensible, it's possible your feature could make a great plugin instead! -We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base! +- **You certify that you wrote and comprehend the code you submit.** The Caddy project welcomes original contributions that comply with [our CLA](https://cla-assistant.io/caddyserver/caddy), meaning that authors must be able to certify that they created or have rights to the code they are contributing. In addition, we require that code is not simply copy-pasted from Q/A sites or AI language models without full comprehension and rigorous testing. In other words: contributors are allowed to refer to communities for assistance and use AI tools such as language models for inspiration, but code which originates from or is assisted by these resources MUST be: + + - Licensed for you to freely share + - Fully comprehended by you (be able to explain every line of code) + - Verified by automated tests when feasible, or thorough manual tests otherwise + + We have found that current language models (LLMs, like ChatGPT) may understand code syntax and even problem spaces to an extent, but often fail in subtle ways to convey true knowledge and produce correct algorithms. Integrated tools such as GitHub Copilot and Sourcegraph Cody may be used for inspiration, but code generated by these tools still needs to meet our criteria for licensing, human comprehension, and testing. These tools may be used to help write code comments and tests as long as you can certify they are accurate and correct. Note that it is often more trouble than it's worth to certify that Copilot (for example) is not giving you code that is possibly plagiarised, unlicensed, or licensed with incompatible terms -- as the Caddy project cannot accept such contributions. If that's too difficult for you (or impossible), then we recommend using these resources only for inspiration and write your own code. Ultimately, you (the contributor) are responsible for the code you're submitting. + + As a courtesy to reviewers, we kindly ask that you disclose when contributing code that was generated by an AI tool or copied from another website so we can be aware of what to look for in code review. + +We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base. #### HOW TO MAKE A PULL REQUEST TO CADDY diff --git a/.github/SECURITY.md b/.github/SECURITY.md index 9d1b313..44cc5b7 100644 --- a/.github/SECURITY.md +++ b/.github/SECURITY.md @@ -7,7 +7,7 @@ The Caddy project would like to make sure that it stays on top of all practicall | Version | Supported | | ------- | ------------------ | -| 2.x | :white_check_mark: | +| 2.x | ✔️ | | 1.x | :x: | | < 1.x | :x: | @@ -24,7 +24,7 @@ We do not accept reports if the steps imply or require a compromised system or t Client-side exploits are out of scope. In other words, it is not a bug in Caddy if the web browser does something unsafe, even if the downloaded content was served by Caddy. (Those kinds of exploits can generally be mitigated by proper configuration of HTTP headers.) As a general rule, the content served by Caddy is not considered in scope because content is configurable by the site owner or the associated web application. -Security bugs in code dependencies are out of scope. Instead, if a dependency has patched a relevant security bug, please feel free to open a public issue or pull request to update that dependency in our code. +Security bugs in code dependencies (including Go's standard library) are out of scope. Instead, if a dependency has patched a relevant security bug, please feel free to open a public issue or pull request to update that dependency in our code. ## Reporting a Vulnerability @@ -42,7 +42,7 @@ We'll need enough information to verify the bug and make a patch. To speed thing - Specific minimal steps to reproduce the issue from scratch - A working patch -Please DO NOT use containers, VMs, cloud instances or services, or any other complex infrastructure in your steps. Always prefer `curl` instead of web browsers. +Please DO NOT use containers, VMs, cloud instances or services, or any other complex infrastructure in your steps. Always prefer `curl -v` instead of web browsers. We consider publicly-registered domain names to be public information. This necessary in order to maintain the integrity of certificate transparency, public DNS, and other public trust systems. Do not redact domain names from your reports. The actual content of your domain name affects Caddy's behavior, so we need the exact domain name(s) to reproduce with, or your report will be ignored. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ba07419..ed83744 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,17 +18,22 @@ jobs: # Default is true, cancels jobs for other platforms in the matrix if one fails fail-fast: false matrix: - os: [ ubuntu-latest, macos-latest, windows-latest ] - go: [ '1.18', '1.20' ] + os: + - ubuntu-latest + - macos-latest + - windows-latest + go: + - '1.20' + - '1.21' include: # Set the minimum Go patch version for the given Go minor # Usable via ${{ matrix.GO_SEMVER }} - - go: '1.18' - GO_SEMVER: '~1.18.4' - - go: '1.20' - GO_SEMVER: '~1.20.0' + GO_SEMVER: '~1.20.6' + + - go: '1.21' + GO_SEMVER: '~1.21.0' # Set some variables per OS, usable via ${{ matrix.VAR }} # CADDY_BIN_PATH: the path to the compiled Caddy binary, for artifact publishing @@ -48,15 +53,15 @@ jobs: runs-on: ${{ matrix.os }} steps: + - name: Checkout code + uses: actions/checkout@v4 + - name: Install Go - uses: actions/setup-go@v3 + uses: actions/setup-go@v4 with: go-version: ${{ matrix.GO_SEMVER }} check-latest: true - - name: Checkout code - uses: actions/checkout@v3 - # These tools would be useful if we later decide to reinvestigate # publishing test/coverage reports to some tool for easier consumption # - name: Install test and coverage analysis tools @@ -68,6 +73,7 @@ jobs: - name: Print Go version and environment id: vars + shell: bash run: | printf "Using go at: $(which go)\n" printf "Go version: $(go version)\n" @@ -79,23 +85,6 @@ jobs: # Calculate the short SHA1 hash of the git commit echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT - - name: Cache the build cache - uses: actions/cache@v3 - with: - # In order: - # * Module download cache - # * Build cache (Linux) - # * Build cache (Mac) - # * Build cache (Windows) - path: | - ~/go/pkg/mod - ~/.cache/go-build - ~/Library/Caches/go-build - ~\AppData\Local\go-build - key: ${{ runner.os }}-${{ matrix.go }}-go-ci-${{ hashFiles('**/go.sum') }} - restore-keys: | - ${{ runner.os }}-${{ matrix.go }}-go-ci - - name: Get dependencies run: | go get -v -t -d ./... @@ -146,8 +135,8 @@ jobs: if: github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]' continue-on-error: true # August 2020: s390x VM is down due to weather and power issues steps: - - name: Checkout code into the Go module directory - uses: actions/checkout@v3 + - name: Checkout code + uses: actions/checkout@v4 - name: Run Tests run: | mkdir -p ~/.ssh && echo -e "${SSH_KEY//_/\\n}" > ~/.ssh/id_ecdsa && chmod og-rwx ~/.ssh/id_ecdsa @@ -172,10 +161,10 @@ jobs: goreleaser-check: runs-on: ubuntu-latest steps: - - name: checkout - uses: actions/checkout@v3 + - name: Checkout code + uses: actions/checkout@v4 - - uses: goreleaser/goreleaser-action@v4 + - uses: goreleaser/goreleaser-action@v5 with: version: latest args: check diff --git a/.github/workflows/cross-build.yml b/.github/workflows/cross-build.yml index 8b5e429..497f39c 100644 --- a/.github/workflows/cross-build.yml +++ b/.github/workflows/cross-build.yml @@ -15,20 +15,35 @@ jobs: strategy: fail-fast: false matrix: - goos: ['android', 'linux', 'solaris', 'illumos', 'dragonfly', 'freebsd', 'openbsd', 'plan9', 'windows', 'darwin', 'netbsd'] - go: [ '1.20' ] + goos: + - 'android' + - 'linux' + - 'solaris' + - 'illumos' + - 'dragonfly' + - 'freebsd' + - 'openbsd' + - 'plan9' + - 'windows' + - 'darwin' + - 'netbsd' + go: + - '1.21' include: # Set the minimum Go patch version for the given Go minor # Usable via ${{ matrix.GO_SEMVER }} - - go: '1.20' - GO_SEMVER: '~1.20.0' + - go: '1.21' + GO_SEMVER: '~1.21.0' runs-on: ubuntu-latest continue-on-error: true steps: + - name: Checkout code + uses: actions/checkout@v4 + - name: Install Go - uses: actions/setup-go@v3 + uses: actions/setup-go@v4 with: go-version: ${{ matrix.GO_SEMVER }} check-latest: true @@ -43,22 +58,6 @@ jobs: printf "\n\nSystem environment:\n\n" env - - name: Cache the build cache - uses: actions/cache@v3 - with: - # In order: - # * Module download cache - # * Build cache (Linux) - path: | - ~/go/pkg/mod - ~/.cache/go-build - key: cross-build-go${{ matrix.go }}-${{ matrix.goos }}-${{ hashFiles('**/go.sum') }} - restore-keys: | - cross-build-go${{ matrix.go }}-${{ matrix.goos }} - - - name: Checkout code into the Go module directory - uses: actions/checkout@v3 - - name: Run Build env: CGO_ENABLED: 0 diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 7e56afc..e636e07 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -17,25 +17,45 @@ jobs: # From https://github.com/golangci/golangci-lint-action golangci: permissions: - contents: read # for actions/checkout to fetch code - pull-requests: read # for golangci/golangci-lint-action to fetch pull requests + contents: read # for actions/checkout to fetch code + pull-requests: read # for golangci/golangci-lint-action to fetch pull requests name: lint strategy: matrix: - os: [ubuntu-latest, macos-latest, windows-latest] + os: + - ubuntu-latest + - macos-latest + - windows-latest runs-on: ${{ matrix.os }} steps: - - uses: actions/checkout@v3 - - uses: actions/setup-go@v3 + - uses: actions/checkout@v4 + - uses: actions/setup-go@v4 with: - go-version: '~1.18.4' + go-version: '~1.21.0' check-latest: true + # Workaround for https://github.com/golangci/golangci-lint-action/issues/135 + skip-pkg-cache: true + - name: golangci-lint uses: golangci/golangci-lint-action@v3 with: - version: v1.50 + version: v1.54 + + # Workaround for https://github.com/golangci/golangci-lint-action/issues/135 + skip-pkg-cache: true + # Windows times out frequently after about 5m50s if we don't set a longer timeout. args: --timeout 10m + # Optional: show only new issues if it's a pull request. The default value is `false`. # only-new-issues: true + + govulncheck: + runs-on: ubuntu-latest + steps: + - name: govulncheck + uses: golang/govulncheck-action@v1 + with: + go-version-input: '~1.21.0' + check-latest: true diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d8ea96b..184662f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -10,14 +10,16 @@ jobs: name: Release strategy: matrix: - os: [ ubuntu-latest ] - go: [ '1.20' ] + os: + - ubuntu-latest + go: + - '1.21' include: # Set the minimum Go patch version for the given Go minor # Usable via ${{ matrix.GO_SEMVER }} - - go: '1.20' - GO_SEMVER: '~1.20.0' + - go: '1.21' + GO_SEMVER: '~1.21.0' runs-on: ${{ matrix.os }} # https://github.com/sigstore/cosign/issues/1258#issuecomment-1002251233 @@ -29,19 +31,19 @@ jobs: contents: write steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 0 + - name: Install Go - uses: actions/setup-go@v3 + uses: actions/setup-go@v4 with: go-version: ${{ matrix.GO_SEMVER }} check-latest: true - - name: Checkout code - uses: actions/checkout@v3 - with: - fetch-depth: 0 - # Force fetch upstream tags -- because 65 minutes - # tl;dr: actions/checkout@v3 runs this line: + # tl;dr: actions/checkout@v4 runs this line: # git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +ebc278ec98bb24f2852b61fde2a9bf2e3d83818b:refs/tags/ # which makes its own local lightweight tag, losing all the annotations in the process. Our earlier script ran: # git fetch --prune --unshallow @@ -94,18 +96,6 @@ jobs: # tags are only accepted if signed by Matt's key git verify-tag "${{ steps.vars.outputs.version_tag }}" || exit 1 - - name: Cache the build cache - uses: actions/cache@v3 - with: - # In order: - # * Module download cache - # * Build cache (Linux) - path: | - ~/go/pkg/mod - ~/.cache/go-build - key: ${{ runner.os }}-go${{ matrix.go }}-release-${{ hashFiles('**/go.sum') }} - restore-keys: | - ${{ runner.os }}-go${{ matrix.go }}-release - name: Install Cosign uses: sigstore/cosign-installer@main - name: Cosign version @@ -116,10 +106,10 @@ jobs: run: syft version # GoReleaser will take care of publishing those artifacts into the release - name: Run GoReleaser - uses: goreleaser/goreleaser-action@v4 + uses: goreleaser/goreleaser-action@v5 with: version: latest - args: release --rm-dist --timeout 60m + args: release --clean --timeout 60m env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} TAG: ${{ steps.vars.outputs.version_tag }} diff --git a/.github/workflows/release_published.yml b/.github/workflows/release_published.yml index 7736e85..f304888 100644 --- a/.github/workflows/release_published.yml +++ b/.github/workflows/release_published.yml @@ -10,7 +10,8 @@ jobs: name: Release Published strategy: matrix: - os: [ ubuntu-latest ] + os: + - ubuntu-latest runs-on: ${{ matrix.os }} steps: @@ -11,6 +11,8 @@ Caddyfile.* # build artifacts and helpers cmd/caddy/caddy cmd/caddy/caddy.exe +cmd/caddy/tmp/*.exe +cmd/caddy/.env # mac specific .DS_Store diff --git a/.golangci.yml b/.golangci.yml index c36821c..5f01897 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -2,15 +2,27 @@ linters-settings: errcheck: ignore: fmt:.*,go.uber.org/zap/zapcore:^Add.* ignoretests: true + gci: + sections: + - standard # Standard section: captures all standard packages. + - default # Default section: contains all imports that could not be matched to another section type. + - prefix(github.com/caddyserver/caddy/v2/cmd) # ensure that this is always at the top and always has a line break. + - prefix(github.com/caddyserver/caddy) # Custom section: groups all imports with the specified Prefix. + # Skip generated files. + # Default: true + skip-generated: true + # Enable custom order of sections. + # If `true`, make the section order the same as the order of `sections`. + # Default: false + custom-order: true linters: disable-all: true enable: - bodyclose - - deadcode - errcheck - - gofmt - - goimports + - gci + - gofumpt - gosec - gosimple - govet @@ -18,11 +30,9 @@ linters: - misspell - prealloc - staticcheck - - structcheck - typecheck - unconvert - unused - - varcheck # these are implicitly disabled: # - asciicheck # - depguard @@ -80,23 +90,23 @@ output: issues: exclude-rules: # we aren't calling unknown URL - - text: "G107" # G107: Url provided to HTTP request as taint input + - text: 'G107' # G107: Url provided to HTTP request as taint input linters: - gosec # as a web server that's expected to handle any template, this is totally in the hands of the user. - - text: "G203" # G203: Use of unescaped data in HTML templates + - text: 'G203' # G203: Use of unescaped data in HTML templates linters: - gosec # we're shelling out to known commands, not relying on user-defined input. - - text: "G204" # G204: Audit use of command execution + - text: 'G204' # G204: Audit use of command execution linters: - gosec # the choice of weakrand is deliberate, hence the named import "weakrand" - path: modules/caddyhttp/reverseproxy/selectionpolicies.go - text: "G404" # G404: Insecure random number source (rand) + text: 'G404' # G404: Insecure random number source (rand) linters: - gosec - path: modules/caddyhttp/reverseproxy/streaming.go - text: "G404" # G404: Insecure random number source (rand) + text: 'G404' # G404: Insecure random number source (rand) linters: - gosec diff --git a/.goreleaser.yml b/.goreleaser.yml index 3f7f40d..dfd6589 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -4,7 +4,9 @@ before: # This is so we can run goreleaser on tag without Git complaining of being dirty. The main.go in cmd/caddy directory # cannot be built within that directory due to changes necessary for the build causing Git to be dirty, which # subsequently causes gorleaser to refuse running. - - rm -rf caddy-build caddy-dist + - rm -rf caddy-build caddy-dist vendor + # vendor Caddy deps + - go mod vendor - mkdir -p caddy-build - cp cmd/caddy/main.go caddy-build/main.go - /bin/sh -c 'cd ./caddy-build && go mod init caddy' @@ -14,6 +16,8 @@ before: # as of Go 1.16, `go` commands no longer automatically change go.{mod,sum}. We now have to explicitly # run `go mod tidy`. The `/bin/sh -c '...'` is because goreleaser can't find cd in PATH without shell invocation. - /bin/sh -c 'cd ./caddy-build && go mod tidy' + # vendor the deps of the prepared to-build module + - /bin/sh -c 'cd ./caddy-build && go mod vendor' - git clone --depth 1 https://github.com/caddyserver/dist caddy-dist - mkdir -p caddy-dist/man - go mod download @@ -39,6 +43,7 @@ builds: - arm64 - s390x - ppc64le + - riscv64 goarm: - "5" - "6" @@ -50,15 +55,21 @@ builds: goarch: ppc64le - goos: darwin goarch: s390x + - goos: darwin + goarch: riscv64 - goos: windows goarch: ppc64le - goos: windows goarch: s390x + - goos: windows + goarch: riscv64 - goos: freebsd goarch: ppc64le - goos: freebsd goarch: s390x - goos: freebsd + goarch: riscv64 + - goos: freebsd goarch: arm goarm: "5" flags: @@ -71,7 +82,7 @@ signs: - cmd: cosign signature: "${artifact}.sig" certificate: '{{ trimsuffix (trimsuffix .Env.artifact ".zip") ".tar.gz" }}.pem' - args: ["sign-blob", "--output-signature=${signature}", "--output-certificate", "${certificate}", "${artifact}"] + args: ["sign-blob", "--yes", "--output-signature=${signature}", "--output-certificate", "${certificate}", "${artifact}"] artifacts: all sboms: @@ -89,7 +100,8 @@ sboms: args: ["$artifact", "--file", "${document}", "--output", "cyclonedx-json"] archives: - - format_overrides: + - id: default + format_overrides: - goos: windows format: zip name_template: >- @@ -100,6 +112,33 @@ archives: {{- with .Arm }}v{{ . }}{{ end }} {{- with .Mips }}_{{ . }}{{ end }} {{- if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }} + + # package the 'caddy-build' directory into a tarball, + # allowing users to build the exact same set of files as ours. + - id: source + meta: true + name_template: "{{ .ProjectName }}_{{ .Version }}_buildable-artifact" + files: + - src: LICENSE + dst: ./LICENSE + - src: README.md + dst: ./README.md + - src: AUTHORS + dst: ./AUTHORS + - src: ./caddy-build + dst: ./ + +source: + enabled: true + name_template: '{{ .ProjectName }}_{{ .Version }}_src' + format: 'tar.gz' + + # Additional files/template/globs you want to add to the source archive. + # + # Default: empty. + files: + - vendor + checksum: algorithm: sha512 @@ -70,7 +70,7 @@ - **Stays up when other servers go down** due to TLS/OCSP/certificate-related issues - **Production-ready** after serving trillions of requests and managing millions of TLS certificates - **Scales to hundreds of thousands of sites** as proven in production -- **HTTP/1.1, HTTP/2, and HTTP/3** supported all by default +- **HTTP/1.1, HTTP/2, and HTTP/3** all supported by default - **Highly extensible** [modular architecture](https://caddyserver.com/docs/architecture) lets Caddy do anything without bloat - **Runs anywhere** with **no external dependencies** (not even libc) - Written in Go, a language with higher **memory safety guarantees** than other servers @@ -87,7 +87,7 @@ See [our online documentation](https://caddyserver.com/docs/install) for other i Requirements: -- [Go 1.18 or newer](https://golang.org/dl/) +- [Go 1.20 or newer](https://golang.org/dl/) ### For development @@ -71,6 +71,11 @@ type AdminConfig struct { // parsed by Caddy. Accepts placeholders. // Default: the value of the `CADDY_ADMIN` environment variable, // or `localhost:2019` otherwise. + // + // Remember: When changing this value through a config reload, + // be sure to use the `--address` CLI flag to specify the current + // admin address if the currently-running admin endpoint is not + // the default address. Listen string `json:"listen,omitempty"` // If true, CORS headers will be emitted, and requests to the @@ -313,7 +318,32 @@ func (admin AdminConfig) allowedOrigins(addr NetworkAddress) []*url.URL { // messages. If the requested URI does not include an Internet host // name for the service being requested, then the Host header field MUST // be given with an empty value." + // + // UPDATE July 2023: Go broke this by patching a minor security bug in 1.20.6. + // Understandable, but frustrating. See: + // https://github.com/golang/go/issues/60374 + // See also the discussion here: + // https://github.com/golang/go/issues/61431 + // + // We can no longer conform to RFC 2616 Section 14.26 from either Go or curl + // in purity. (Curl allowed no host between 7.40 and 7.50, but now requires a + // bogus host; see https://superuser.com/a/925610.) If we disable Host/Origin + // security checks, the infosec community assures me that it is secure to do + // so, because: + // 1) Browsers do not allow access to unix sockets + // 2) DNS is irrelevant to unix sockets + // + // I am not quite ready to trust either of those external factors, so instead + // of disabling Host/Origin checks, we now allow specific Host values when + // accessing the admin endpoint over unix sockets. I definitely don't trust + // DNS (e.g. I don't trust 'localhost' to always resolve to the local host), + // and IP shouldn't even be used, but if it is for some reason, I think we can + // at least be reasonably assured that 127.0.0.1 and ::1 route to the local + // machine, meaning that a hypothetical browser origin would have to be on the + // local machine as well. uniqueOrigins[""] = struct{}{} + uniqueOrigins["127.0.0.1"] = struct{}{} + uniqueOrigins["::1"] = struct{}{} } else { uniqueOrigins[net.JoinHostPort("localhost", addr.port())] = struct{}{} uniqueOrigins[net.JoinHostPort("::1", addr.port())] = struct{}{} @@ -1011,9 +1041,9 @@ func handleConfigID(w http.ResponseWriter, r *http.Request) error { id := parts[2] // map the ID to the expanded path - currentCtxMu.RLock() + rawCfgMu.RLock() expanded, ok := rawCfgIndex[id] - defer currentCtxMu.RUnlock() + rawCfgMu.RUnlock() if !ok { return APIError{ HTTPStatus: http.StatusNotFound, @@ -1166,15 +1196,27 @@ traverseLoop: } case http.MethodPut: if _, ok := v[part]; ok { - return fmt.Errorf("[%s] key already exists: %s", path, part) + return APIError{ + HTTPStatus: http.StatusConflict, + Err: fmt.Errorf("[%s] key already exists: %s", path, part), + } } v[part] = val case http.MethodPatch: if _, ok := v[part]; !ok { - return fmt.Errorf("[%s] key does not exist: %s", path, part) + return APIError{ + HTTPStatus: http.StatusNotFound, + Err: fmt.Errorf("[%s] key does not exist: %s", path, part), + } } v[part] = val case http.MethodDelete: + if _, ok := v[part]; !ok { + return APIError{ + HTTPStatus: http.StatusNotFound, + Err: fmt.Errorf("[%s] key does not exist: %s", path, part), + } + } delete(v, part) default: return fmt.Errorf("unrecognized method %s", method) @@ -1316,7 +1358,7 @@ var ( // will get deleted before the process gracefully exits. func PIDFile(filename string) error { pid := []byte(strconv.Itoa(os.Getpid()) + "\n") - err := os.WriteFile(filename, pid, 0600) + err := os.WriteFile(filename, pid, 0o600) if err != nil { return err } diff --git a/admin_test.go b/admin_test.go index 04aa886..9137a88 100644 --- a/admin_test.go +++ b/admin_test.go @@ -76,6 +76,12 @@ func TestUnsyncedConfigAccess(t *testing.T) { expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`, }, { + method: "DELETE", + path: "/bar/qq", + expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`, + shouldErr: true, + }, + { method: "POST", path: "/list", payload: `"e"`, diff --git a/caddy-cgi.nix b/caddy-cgi.nix index e8805e2..c0a7ff9 100644 --- a/caddy-cgi.nix +++ b/caddy-cgi.nix @@ -7,12 +7,12 @@ testers, installShellFiles, }: let - version = "2.6.4"; + version = "2.7.5"; dist = fetchFromGitHub { owner = "caddyserver"; repo = "dist"; rev = "v${version}"; - hash = "sha256-SJO1q4g9uyyky9ZYSiqXJgNIvyxT5RjrpYd20YDx8ec="; + hash = "sha256-aZ7hdAZJH1PvrX9GQLzLquzzZG3LZSKOvt7sWQhTiR8="; }; in buildGoModule { @@ -21,7 +21,7 @@ in src = ./.; - vendorHash = "sha256-SJI0GImRqpe7ksFc3XuhHQvVyuB6JqbQDAClUtaEoTI="; + vendorHash = "sha256-cLJEXmFHR3Xs4Hv1Y5pTgx5NapXsYWkvUDgjUFNyJpE="; subPackages = ["cmd/caddy"]; @@ -34,10 +34,11 @@ import ( "sync/atomic" "time" - "github.com/caddyserver/caddy/v2/notify" "github.com/caddyserver/certmagic" "github.com/google/uuid" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2/notify" ) // Config is the top (or beginning) of the Caddy configuration structure. @@ -156,8 +157,8 @@ func changeConfig(method, path string, input []byte, ifMatchHeader string, force return fmt.Errorf("method not allowed") } - currentCtxMu.Lock() - defer currentCtxMu.Unlock() + rawCfgMu.Lock() + defer rawCfgMu.Unlock() if ifMatchHeader != "" { // expect the first and last character to be quotes @@ -257,8 +258,8 @@ func changeConfig(method, path string, input []byte, ifMatchHeader string, force // readConfig traverses the current config to path // and writes its JSON encoding to out. func readConfig(path string, out io.Writer) error { - currentCtxMu.RLock() - defer currentCtxMu.RUnlock() + rawCfgMu.RLock() + defer rawCfgMu.RUnlock() return unsyncedConfigAccess(http.MethodGet, path, nil, out) } @@ -305,7 +306,7 @@ func indexConfigObjects(ptr any, configPath string, index map[string]string) err // it as the new config, replacing any other current config. // It does NOT update the raw config state, as this is a // lower-level function; most callers will want to use Load -// instead. A write lock on currentCtxMu is required! If +// instead. A write lock on rawCfgMu is required! If // allowPersist is false, it will not be persisted to disk, // even if it is configured to. func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error { @@ -314,7 +315,7 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error { strippedCfgJSON := RemoveMetaFields(cfgJSON) var newCfg *Config - err := strictUnmarshalJSON(strippedCfgJSON, &newCfg) + err := StrictUnmarshalJSON(strippedCfgJSON, &newCfg) if err != nil { return err } @@ -340,8 +341,10 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error { } // swap old context (including its config) with the new one + currentCtxMu.Lock() oldCtx := currentCtx currentCtx = ctx + currentCtxMu.Unlock() // Stop, Cleanup each old app unsyncedStop(oldCtx) @@ -354,13 +357,13 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error { newCfg.Admin.Config.Persist == nil || *newCfg.Admin.Config.Persist) { dir := filepath.Dir(ConfigAutosavePath) - err := os.MkdirAll(dir, 0700) + err := os.MkdirAll(dir, 0o700) if err != nil { Log().Error("unable to create folder for config autosave", zap.String("dir", dir), zap.Error(err)) } else { - err := os.WriteFile(ConfigAutosavePath, cfgJSON, 0600) + err := os.WriteFile(ConfigAutosavePath, cfgJSON, 0o600) if err == nil { Log().Info("autosaved config (load with --resume flag)", zap.String("file", ConfigAutosavePath)) } else { @@ -627,22 +630,35 @@ type ConfigLoader interface { // stop the others. Stop should only be called // if not replacing with a new config. func Stop() error { + currentCtxMu.RLock() + ctx := currentCtx + currentCtxMu.RUnlock() + + rawCfgMu.Lock() + unsyncedStop(ctx) + currentCtxMu.Lock() - defer currentCtxMu.Unlock() - unsyncedStop(currentCtx) currentCtx = Context{} + currentCtxMu.Unlock() + rawCfgJSON = nil rawCfgIndex = nil rawCfg[rawConfigKey] = nil + rawCfgMu.Unlock() + return nil } -// unsyncedStop stops cfg from running, but has -// no locking around cfg. It is a no-op if cfg is -// nil. If any app returns an error when stopping, +// unsyncedStop stops ctx from running, but has +// no locking around ctx. It is a no-op if ctx has a +// nil cfg. If any app returns an error when stopping, // it is logged and the function continues stopping // the next app. This function assumes all apps in -// cfg were successfully started first. +// ctx were successfully started first. +// +// A lock on rawCfgMu is required, even though this +// function does not access rawCfg, that lock +// synchronizes the stop/start of apps. func unsyncedStop(ctx Context) { if ctx.cfg == nil { return @@ -816,7 +832,7 @@ func InstanceID() (uuid.UUID, error) { if err != nil { return uuid, err } - err = os.WriteFile(uuidFilePath, []byte(uuid.String()), 0600) + err = os.WriteFile(uuidFilePath, []byte(uuid.String()), 0o600) return uuid, err } else if err != nil { return [16]byte{}, err @@ -969,14 +985,12 @@ type CtxKey string // This group of variables pertains to the current configuration. var ( - // currentCtxMu protects everything in this var block. - currentCtxMu sync.RWMutex - // currentCtx is the root context for the currently-running // configuration, which can be accessed through this value. // If the Config contained in this value is not nil, then // a config is currently active/running. - currentCtx Context + currentCtx Context + currentCtxMu sync.RWMutex // rawCfg is the current, generic-decoded configuration; // we initialize it as a map with one field ("config") @@ -994,6 +1008,10 @@ var ( // rawCfgIndex is the map of user-assigned ID to expanded // path, for converting /id/ paths to /config/ paths. rawCfgIndex map[string]string + + // rawCfgMu protects all the rawCfg fields and also + // essentially synchronizes config changes/reloads. + rawCfgMu sync.RWMutex ) // errSameConfig is returned if the new config is the same diff --git a/caddyconfig/caddyfile/adapter.go b/caddyconfig/caddyfile/adapter.go index b924325..d6ef602 100644 --- a/caddyconfig/caddyfile/adapter.go +++ b/caddyconfig/caddyfile/adapter.go @@ -88,7 +88,7 @@ func FormattingDifference(filename string, body []byte) (caddyconfig.Warning, bo return caddyconfig.Warning{ File: filename, Line: line, - Message: "Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies", + Message: "Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies", }, true } diff --git a/caddyconfig/caddyfile/dispenser.go b/caddyconfig/caddyfile/dispenser.go index 91bd9a5..215a164 100644 --- a/caddyconfig/caddyfile/dispenser.go +++ b/caddyconfig/caddyfile/dispenser.go @@ -101,12 +101,12 @@ func (d *Dispenser) nextOnSameLine() bool { d.cursor++ return true } - if d.cursor >= len(d.tokens) { + if d.cursor >= len(d.tokens)-1 { return false } - if d.cursor < len(d.tokens)-1 && - d.tokens[d.cursor].File == d.tokens[d.cursor+1].File && - d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].Line { + curr := d.tokens[d.cursor] + next := d.tokens[d.cursor+1] + if !isNextOnNewLine(curr, next) { d.cursor++ return true } @@ -122,12 +122,12 @@ func (d *Dispenser) NextLine() bool { d.cursor++ return true } - if d.cursor >= len(d.tokens) { + if d.cursor >= len(d.tokens)-1 { return false } - if d.cursor < len(d.tokens)-1 && - (d.tokens[d.cursor].File != d.tokens[d.cursor+1].File || - d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].Line) { + curr := d.tokens[d.cursor] + next := d.tokens[d.cursor+1] + if isNextOnNewLine(curr, next) { d.cursor++ return true } @@ -203,14 +203,17 @@ func (d *Dispenser) Val() string { } // ValRaw gets the raw text of the current token (including quotes). +// If the token was a heredoc, then the delimiter is not included, +// because that is not relevant to any unmarshaling logic at this time. // If there is no token loaded, it returns empty string. func (d *Dispenser) ValRaw() string { if d.cursor < 0 || d.cursor >= len(d.tokens) { return "" } quote := d.tokens[d.cursor].wasQuoted - if quote > 0 { - return string(quote) + d.tokens[d.cursor].Text + string(quote) // string literal + if quote > 0 && quote != '<' { + // string literal + return string(quote) + d.tokens[d.cursor].Text + string(quote) } return d.tokens[d.cursor].Text } @@ -388,22 +391,22 @@ func (d *Dispenser) Reset() { // an argument. func (d *Dispenser) ArgErr() error { if d.Val() == "{" { - return d.Err("Unexpected token '{', expecting argument") + return d.Err("unexpected token '{', expecting argument") } - return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val()) + return d.Errf("wrong argument count or unexpected line ending after '%s'", d.Val()) } // SyntaxErr creates a generic syntax error which explains what was // found and what was expected. func (d *Dispenser) SyntaxErr(expected string) error { - msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.File(), d.Line(), d.Val(), expected) + msg := fmt.Sprintf("syntax error: unexpected token '%s', expecting '%s', at %s:%d import chain: ['%s']", d.Val(), expected, d.File(), d.Line(), strings.Join(d.Token().imports, "','")) return errors.New(msg) } // EOFErr returns an error indicating that the dispenser reached // the end of the input when searching for the next token. func (d *Dispenser) EOFErr() error { - return d.Errf("Unexpected EOF") + return d.Errf("unexpected EOF") } // Err generates a custom parse-time error with a message of msg. @@ -418,7 +421,10 @@ func (d *Dispenser) Errf(format string, args ...any) error { // WrapErr takes an existing error and adds the Caddyfile file and line number. func (d *Dispenser) WrapErr(err error) error { - return fmt.Errorf("%s:%d - Error during parsing: %w", d.File(), d.Line(), err) + if len(d.Token().imports) > 0 { + return fmt.Errorf("%w, at %s:%d import chain ['%s']", err, d.File(), d.Line(), strings.Join(d.Token().imports, "','")) + } + return fmt.Errorf("%w, at %s:%d", err, d.File(), d.Line()) } // Delete deletes the current token and returns the updated slice @@ -438,14 +444,14 @@ func (d *Dispenser) Delete() []Token { return d.tokens } -// numLineBreaks counts how many line breaks are in the token -// value given by the token index tknIdx. It returns 0 if the -// token does not exist or there are no line breaks. -func (d *Dispenser) numLineBreaks(tknIdx int) int { - if tknIdx < 0 || tknIdx >= len(d.tokens) { - return 0 +// DeleteN is the same as Delete, but can delete many tokens at once. +// If there aren't N tokens available to delete, none are deleted. +func (d *Dispenser) DeleteN(amount int) []Token { + if amount > 0 && d.cursor >= (amount-1) && d.cursor <= len(d.tokens)-1 { + d.tokens = append(d.tokens[:d.cursor-(amount-1)], d.tokens[d.cursor+1:]...) + d.cursor -= amount } - return strings.Count(d.tokens[tknIdx].Text, "\n") + return d.tokens } // isNewLine determines whether the current token is on a different @@ -461,25 +467,7 @@ func (d *Dispenser) isNewLine() bool { prev := d.tokens[d.cursor-1] curr := d.tokens[d.cursor] - - // If the previous token is from a different file, - // we can assume it's from a different line - if prev.File != curr.File { - return true - } - - // The previous token may contain line breaks if - // it was quoted and spanned multiple lines. e.g: - // - // dir "foo - // bar - // baz" - prevLineBreaks := d.numLineBreaks(d.cursor - 1) - - // If the previous token (incl line breaks) ends - // on a line earlier than the current token, - // then the current token is on a new line - return prev.Line+prevLineBreaks < curr.Line + return isNextOnNewLine(prev, curr) } // isNextOnNewLine determines whether the current token is on a different @@ -495,23 +483,5 @@ func (d *Dispenser) isNextOnNewLine() bool { curr := d.tokens[d.cursor] next := d.tokens[d.cursor+1] - - // If the next token is from a different file, - // we can assume it's from a different line - if curr.File != next.File { - return true - } - - // The current token may contain line breaks if - // it was quoted and spanned multiple lines. e.g: - // - // dir "foo - // bar - // baz" - currLineBreaks := d.numLineBreaks(d.cursor) - - // If the current token (incl line breaks) ends - // on a line earlier than the next token, - // then the next token is on a new line - return curr.Line+currLineBreaks < next.Line + return isNextOnNewLine(curr, next) } diff --git a/caddyconfig/caddyfile/importargs.go b/caddyconfig/caddyfile/importargs.go new file mode 100644 index 0000000..2e21a36 --- /dev/null +++ b/caddyconfig/caddyfile/importargs.go @@ -0,0 +1,153 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package caddyfile + +import ( + "regexp" + "strconv" + "strings" + + "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" +) + +// parseVariadic determines if the token is a variadic placeholder, +// and if so, determines the index range (start/end) of args to use. +// Returns a boolean signaling whether a variadic placeholder was found, +// and the start and end indices. +func parseVariadic(token Token, argCount int) (bool, int, int) { + if !strings.HasPrefix(token.Text, "{args[") { + return false, 0, 0 + } + if !strings.HasSuffix(token.Text, "]}") { + return false, 0, 0 + } + + argRange := strings.TrimSuffix(strings.TrimPrefix(token.Text, "{args["), "]}") + if argRange == "" { + caddy.Log().Named("caddyfile").Warn( + "Placeholder "+token.Text+" cannot have an empty index", + zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports)) + return false, 0, 0 + } + + start, end, found := strings.Cut(argRange, ":") + + // If no ":" delimiter is found, this is not a variadic. + // The replacer will pick this up. + if !found { + return false, 0, 0 + } + + var ( + startIndex = 0 + endIndex = argCount + err error + ) + if start != "" { + startIndex, err = strconv.Atoi(start) + if err != nil { + caddy.Log().Named("caddyfile").Warn( + "Variadic placeholder "+token.Text+" has an invalid start index", + zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports)) + return false, 0, 0 + } + } + if end != "" { + endIndex, err = strconv.Atoi(end) + if err != nil { + caddy.Log().Named("caddyfile").Warn( + "Variadic placeholder "+token.Text+" has an invalid end index", + zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports)) + return false, 0, 0 + } + } + + // bound check + if startIndex < 0 || startIndex > endIndex || endIndex > argCount { + caddy.Log().Named("caddyfile").Warn( + "Variadic placeholder "+token.Text+" indices are out of bounds, only "+strconv.Itoa(argCount)+" argument(s) exist", + zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports)) + return false, 0, 0 + } + return true, startIndex, endIndex +} + +// makeArgsReplacer prepares a Replacer which can replace +// non-variadic args placeholders in imported tokens. +func makeArgsReplacer(args []string) *caddy.Replacer { + repl := caddy.NewEmptyReplacer() + repl.Map(func(key string) (any, bool) { + // TODO: Remove the deprecated {args.*} placeholder + // support at some point in the future + if matches := argsRegexpIndexDeprecated.FindStringSubmatch(key); len(matches) > 0 { + // What's matched may be a substring of the key + if matches[0] != key { + return nil, false + } + + value, err := strconv.Atoi(matches[1]) + if err != nil { + caddy.Log().Named("caddyfile").Warn( + "Placeholder {args." + matches[1] + "} has an invalid index") + return nil, false + } + if value >= len(args) { + caddy.Log().Named("caddyfile").Warn( + "Placeholder {args." + matches[1] + "} index is out of bounds, only " + strconv.Itoa(len(args)) + " argument(s) exist") + return nil, false + } + caddy.Log().Named("caddyfile").Warn( + "Placeholder {args." + matches[1] + "} deprecated, use {args[" + matches[1] + "]} instead") + return args[value], true + } + + // Handle args[*] form + if matches := argsRegexpIndex.FindStringSubmatch(key); len(matches) > 0 { + // What's matched may be a substring of the key + if matches[0] != key { + return nil, false + } + + if strings.Contains(matches[1], ":") { + caddy.Log().Named("caddyfile").Warn( + "Variadic placeholder {args[" + matches[1] + "]} must be a token on its own") + return nil, false + } + value, err := strconv.Atoi(matches[1]) + if err != nil { + caddy.Log().Named("caddyfile").Warn( + "Placeholder {args[" + matches[1] + "]} has an invalid index") + return nil, false + } + if value >= len(args) { + caddy.Log().Named("caddyfile").Warn( + "Placeholder {args[" + matches[1] + "]} index is out of bounds, only " + strconv.Itoa(len(args)) + " argument(s) exist") + return nil, false + } + return args[value], true + } + + // Not an args placeholder, ignore + return nil, false + }) + return repl +} + +var ( + argsRegexpIndexDeprecated = regexp.MustCompile(`args\.(.+)`) + argsRegexpIndex = regexp.MustCompile(`args\[(.+)]`) +) diff --git a/caddyconfig/caddyfile/importgraph.go b/caddyconfig/caddyfile/importgraph.go index 659c368..d27f471 100644 --- a/caddyconfig/caddyfile/importgraph.go +++ b/caddyconfig/caddyfile/importgraph.go @@ -34,6 +34,7 @@ func (i *importGraph) addNode(name string) { } i.nodes[name] = true } + func (i *importGraph) addNodes(names []string) { for _, name := range names { i.addNode(name) @@ -43,6 +44,7 @@ func (i *importGraph) addNodes(names []string) { func (i *importGraph) removeNode(name string) { delete(i.nodes, name) } + func (i *importGraph) removeNodes(names []string) { for _, name := range names { i.removeNode(name) @@ -73,6 +75,7 @@ func (i *importGraph) addEdge(from, to string) error { i.edges[from] = append(i.edges[from], to) return nil } + func (i *importGraph) addEdges(from string, tos []string) error { for _, to := range tos { err := i.addEdge(from, to) diff --git a/caddyconfig/caddyfile/lexer.go b/caddyconfig/caddyfile/lexer.go index 5605a6a..bfd6c0f 100644 --- a/caddyconfig/caddyfile/lexer.go +++ b/caddyconfig/caddyfile/lexer.go @@ -17,7 +17,10 @@ package caddyfile import ( "bufio" "bytes" + "fmt" "io" + "regexp" + "strings" "unicode" ) @@ -35,15 +38,41 @@ type ( // Token represents a single parsable unit. Token struct { - File string - Line int - Text string - wasQuoted rune // enclosing quote character, if any - inSnippet bool - snippetName string + File string + imports []string + Line int + Text string + wasQuoted rune // enclosing quote character, if any + heredocMarker string + snippetName string } ) +// Tokenize takes bytes as input and lexes it into +// a list of tokens that can be parsed as a Caddyfile. +// Also takes a filename to fill the token's File as +// the source of the tokens, which is important to +// determine relative paths for `import` directives. +func Tokenize(input []byte, filename string) ([]Token, error) { + l := lexer{} + if err := l.load(bytes.NewReader(input)); err != nil { + return nil, err + } + var tokens []Token + for { + found, err := l.next() + if err != nil { + return nil, err + } + if !found { + break + } + l.token.File = filename + tokens = append(tokens, l.token) + } + return tokens, nil +} + // load prepares the lexer to scan an input for tokens. // It discards any leading byte order mark. func (l *lexer) load(input io.Reader) error { @@ -75,28 +104,107 @@ func (l *lexer) load(input io.Reader) error { // may be escaped. The rest of the line is skipped // if a "#" character is read in. Returns true if // a token was loaded; false otherwise. -func (l *lexer) next() bool { +func (l *lexer) next() (bool, error) { var val []rune - var comment, quoted, btQuoted, escaped bool + var comment, quoted, btQuoted, inHeredoc, heredocEscaped, escaped bool + var heredocMarker string makeToken := func(quoted rune) bool { l.token.Text = string(val) l.token.wasQuoted = quoted + l.token.heredocMarker = heredocMarker return true } for { + // Read a character in; if err then if we had + // read some characters, make a token. If we + // reached EOF, then no more tokens to read. + // If no EOF, then we had a problem. ch, _, err := l.reader.ReadRune() if err != nil { if len(val) > 0 { - return makeToken(0) + if inHeredoc { + return false, fmt.Errorf("incomplete heredoc <<%s on line #%d, expected ending marker %s", heredocMarker, l.line+l.skippedLines, heredocMarker) + } + + return makeToken(0), nil } if err == io.EOF { - return false + return false, nil + } + return false, err + } + + // detect whether we have the start of a heredoc + if !(quoted || btQuoted) && !(inHeredoc || heredocEscaped) && + len(val) > 1 && string(val[:2]) == "<<" { + // a space means it's just a regular token and not a heredoc + if ch == ' ' { + return makeToken(0), nil + } + + // skip CR, we only care about LF + if ch == '\r' { + continue + } + + // after hitting a newline, we know that the heredoc marker + // is the characters after the two << and the newline. + // we reset the val because the heredoc is syntax we don't + // want to keep. + if ch == '\n' { + if len(val) == 2 { + return false, fmt.Errorf("missing opening heredoc marker on line #%d; must contain only alpha-numeric characters, dashes and underscores; got empty string", l.line) + } + + // check if there's too many < + if string(val[:3]) == "<<<" { + return false, fmt.Errorf("too many '<' for heredoc on line #%d; only use two, for example <<END", l.line) + } + + heredocMarker = string(val[2:]) + if !heredocMarkerRegexp.Match([]byte(heredocMarker)) { + return false, fmt.Errorf("heredoc marker on line #%d must contain only alpha-numeric characters, dashes and underscores; got '%s'", l.line, heredocMarker) + } + + inHeredoc = true + l.skippedLines++ + val = nil + continue + } + val = append(val, ch) + continue + } + + // if we're in a heredoc, all characters are read as-is + if inHeredoc { + val = append(val, ch) + + if ch == '\n' { + l.skippedLines++ + } + + // check if we're done, i.e. that the last few characters are the marker + if len(val) > len(heredocMarker) && heredocMarker == string(val[len(val)-len(heredocMarker):]) { + // set the final value + val, err = l.finalizeHeredoc(val, heredocMarker) + if err != nil { + return false, err + } + + // set the line counter, and make the token + l.line += l.skippedLines + l.skippedLines = 0 + return makeToken('<'), nil } - panic(err) + + // stay in the heredoc until we find the ending marker + continue } + // track whether we found an escape '\' for the next + // iteration to be contextually aware if !escaped && !btQuoted && ch == '\\' { escaped = true continue @@ -111,26 +219,29 @@ func (l *lexer) next() bool { } escaped = false } else { - if quoted && ch == '"' { - return makeToken('"') - } - if btQuoted && ch == '`' { - return makeToken('`') + if (quoted && ch == '"') || (btQuoted && ch == '`') { + return makeToken(ch), nil } } + // allow quoted text to wrap continue on multiple lines if ch == '\n' { l.line += 1 + l.skippedLines l.skippedLines = 0 } + // collect this character as part of the quoted token val = append(val, ch) continue } if unicode.IsSpace(ch) { + // ignore CR altogether, we only actually care about LF (\n) if ch == '\r' { continue } + // end of the line if ch == '\n' { + // newlines can be escaped to chain arguments + // onto multiple lines; else, increment the line count if escaped { l.skippedLines++ escaped = false @@ -138,14 +249,18 @@ func (l *lexer) next() bool { l.line += 1 + l.skippedLines l.skippedLines = 0 } + // comments (#) are single-line only comment = false } + // any kind of space means we're at the end of this token if len(val) > 0 { - return makeToken(0) + return makeToken(0), nil } continue } + // comments must be at the start of a token, + // in other words, preceded by space or newline if ch == '#' && len(val) == 0 { comment = true } @@ -166,7 +281,12 @@ func (l *lexer) next() bool { } if escaped { - val = append(val, '\\') + // allow escaping the first < to skip the heredoc syntax + if ch == '<' { + heredocEscaped = true + } else { + val = append(val, '\\') + } escaped = false } @@ -174,24 +294,86 @@ func (l *lexer) next() bool { } } -// Tokenize takes bytes as input and lexes it into -// a list of tokens that can be parsed as a Caddyfile. -// Also takes a filename to fill the token's File as -// the source of the tokens, which is important to -// determine relative paths for `import` directives. -func Tokenize(input []byte, filename string) ([]Token, error) { - l := lexer{} - if err := l.load(bytes.NewReader(input)); err != nil { - return nil, err +// finalizeHeredoc takes the runes read as the heredoc text and the marker, +// and processes the text to strip leading whitespace, returning the final +// value without the leading whitespace. +func (l *lexer) finalizeHeredoc(val []rune, marker string) ([]rune, error) { + stringVal := string(val) + + // find the last newline of the heredoc, which is where the contents end + lastNewline := strings.LastIndex(stringVal, "\n") + + // collapse the content, then split into separate lines + lines := strings.Split(stringVal[:lastNewline+1], "\n") + + // figure out how much whitespace we need to strip from the front of every line + // by getting the string that precedes the marker, on the last line + paddingToStrip := stringVal[lastNewline+1 : len(stringVal)-len(marker)] + + // iterate over each line and strip the whitespace from the front + var out string + for lineNum, lineText := range lines[:len(lines)-1] { + // find an exact match for the padding + index := strings.Index(lineText, paddingToStrip) + + // if the padding doesn't match exactly at the start then we can't safely strip + if index != 0 { + return nil, fmt.Errorf("mismatched leading whitespace in heredoc <<%s on line #%d [%s], expected whitespace [%s] to match the closing marker", marker, l.line+lineNum+1, lineText, paddingToStrip) + } + + // strip, then append the line, with the newline, to the output. + // also removes all "\r" because Windows. + out += strings.ReplaceAll(lineText[len(paddingToStrip):]+"\n", "\r", "") } - var tokens []Token - for l.next() { - l.token.File = filename - tokens = append(tokens, l.token) + + // Remove the trailing newline from the loop + if len(out) > 0 && out[len(out)-1] == '\n' { + out = out[:len(out)-1] } - return tokens, nil + + // return the final value + return []rune(out), nil } func (t Token) Quoted() bool { return t.wasQuoted > 0 } + +// NumLineBreaks counts how many line breaks are in the token text. +func (t Token) NumLineBreaks() int { + lineBreaks := strings.Count(t.Text, "\n") + if t.wasQuoted == '<' { + // heredocs have an extra linebreak because the opening + // delimiter is on its own line and is not included in the + // token Text itself, and the trailing newline is removed. + lineBreaks += 2 + } + return lineBreaks +} + +var heredocMarkerRegexp = regexp.MustCompile("^[A-Za-z0-9_-]+$") + +// isNextOnNewLine tests whether t2 is on a different line from t1 +func isNextOnNewLine(t1, t2 Token) bool { + // If the second token is from a different file, + // we can assume it's from a different line + if t1.File != t2.File { + return true + } + + // If the second token is from a different import chain, + // we can assume it's from a different line + if len(t1.imports) != len(t2.imports) { + return true + } + for i, im := range t1.imports { + if im != t2.imports[i] { + return true + } + } + + // If the first token (incl line breaks) ends + // on a line earlier than the next token, + // then the second token is on a new line + return t1.Line+t1.NumLineBreaks() < t2.Line +} diff --git a/caddyconfig/caddyfile/lexer_test.go b/caddyconfig/caddyfile/lexer_test.go index 30ee0f6..92acc4d 100644 --- a/caddyconfig/caddyfile/lexer_test.go +++ b/caddyconfig/caddyfile/lexer_test.go @@ -18,13 +18,13 @@ import ( "testing" ) -type lexerTestCase struct { - input []byte - expected []Token -} - func TestLexer(t *testing.T) { - testCases := []lexerTestCase{ + testCases := []struct { + input []byte + expected []Token + expectErr bool + errorMessage string + }{ { input: []byte(`host:123`), expected: []Token{ @@ -249,12 +249,219 @@ func TestLexer(t *testing.T) { {Line: 1, Text: `quotes`}, }, }, + { + input: []byte(`heredoc <<EOF +content +EOF same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: "content"}, + {Line: 3, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`heredoc <<VERY-LONG-MARKER +content +VERY-LONG-MARKER same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: "content"}, + {Line: 3, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`heredoc <<EOF +extra-newline + +EOF same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: "extra-newline\n"}, + {Line: 4, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`heredoc <<EOF + EOF same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: ""}, + {Line: 2, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`heredoc <<EOF + content + EOF same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: "content"}, + {Line: 3, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`prev-line + heredoc <<EOF + multi + line + content + EOF same-line-arg + next-line + `), + expected: []Token{ + {Line: 1, Text: `prev-line`}, + {Line: 2, Text: `heredoc`}, + {Line: 2, Text: "\tmulti\n\tline\n\tcontent"}, + {Line: 6, Text: `same-line-arg`}, + {Line: 7, Text: `next-line`}, + }, + }, + { + input: []byte(`escaped-heredoc \<< >>`), + expected: []Token{ + {Line: 1, Text: `escaped-heredoc`}, + {Line: 1, Text: `<<`}, + {Line: 1, Text: `>>`}, + }, + }, + { + input: []byte(`not-a-heredoc <EOF + content + `), + expected: []Token{ + {Line: 1, Text: `not-a-heredoc`}, + {Line: 1, Text: `<EOF`}, + {Line: 2, Text: `content`}, + }, + }, + { + input: []byte(`not-a-heredoc <<<EOF content`), + expected: []Token{ + {Line: 1, Text: `not-a-heredoc`}, + {Line: 1, Text: `<<<EOF`}, + {Line: 1, Text: `content`}, + }, + }, + { + input: []byte(`not-a-heredoc "<<" ">>"`), + expected: []Token{ + {Line: 1, Text: `not-a-heredoc`}, + {Line: 1, Text: `<<`}, + {Line: 1, Text: `>>`}, + }, + }, + { + input: []byte(`not-a-heredoc << >>`), + expected: []Token{ + {Line: 1, Text: `not-a-heredoc`}, + {Line: 1, Text: `<<`}, + {Line: 1, Text: `>>`}, + }, + }, + { + input: []byte(`not-a-heredoc <<HERE SAME LINE + content + HERE same-line-arg + `), + expected: []Token{ + {Line: 1, Text: `not-a-heredoc`}, + {Line: 1, Text: `<<HERE`}, + {Line: 1, Text: `SAME`}, + {Line: 1, Text: `LINE`}, + {Line: 2, Text: `content`}, + {Line: 3, Text: `HERE`}, + {Line: 3, Text: `same-line-arg`}, + }, + }, + { + input: []byte(`heredoc <<s + � + s + `), + expected: []Token{ + {Line: 1, Text: `heredoc`}, + {Line: 1, Text: "�"}, + }, + }, + { + input: []byte("\u000Aheredoc \u003C\u003C\u0073\u0073\u000A\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F\u000A\u0073\u0073\u000A\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F\u000A\u00BF\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F"), + expected: []Token{ + { + Line: 2, + Text: "heredoc", + }, + { + Line: 2, + Text: "\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F", + }, + { + Line: 5, + Text: "\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F", + }, + { + Line: 6, + Text: "\u00BF\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F", + }, + }, + }, + { + input: []byte("not-a-heredoc <<\n"), + expectErr: true, + errorMessage: "missing opening heredoc marker on line #1; must contain only alpha-numeric characters, dashes and underscores; got empty string", + }, + { + input: []byte(`heredoc <<<EOF + content + EOF same-line-arg + `), + expectErr: true, + errorMessage: "too many '<' for heredoc on line #1; only use two, for example <<END", + }, + { + input: []byte(`heredoc <<EOF + content + `), + expectErr: true, + errorMessage: "incomplete heredoc <<EOF on line #3, expected ending marker EOF", + }, + { + input: []byte(`heredoc <<EOF + content + EOF + `), + expectErr: true, + errorMessage: "mismatched leading whitespace in heredoc <<EOF on line #2 [\tcontent], expected whitespace [\t\t] to match the closing marker", + }, + { + input: []byte(`heredoc <<EOF + content + EOF + `), + expectErr: true, + errorMessage: "mismatched leading whitespace in heredoc <<EOF on line #2 [ content], expected whitespace [\t\t] to match the closing marker", + }, } for i, testCase := range testCases { actual, err := Tokenize(testCase.input, "") + if testCase.expectErr { + if err == nil { + t.Fatalf("expected error, got actual: %v", actual) + continue + } + if err.Error() != testCase.errorMessage { + t.Fatalf("expected error '%v', got: %v", testCase.errorMessage, err) + } + continue + } + if err != nil { - t.Errorf("%v", err) + t.Fatalf("%v", err) } lexerCompare(t, i, testCase.expected, actual) } @@ -262,17 +469,17 @@ func TestLexer(t *testing.T) { func lexerCompare(t *testing.T, n int, expected, actual []Token) { if len(expected) != len(actual) { - t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual)) + t.Fatalf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual)) } for i := 0; i < len(actual) && i < len(expected); i++ { if actual[i].Line != expected[i].Line { - t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d", + t.Fatalf("Test case %d token %d ('%s'): expected line %d but was line %d", n, i, expected[i].Text, expected[i].Line, actual[i].Line) break } if actual[i].Text != expected[i].Text { - t.Errorf("Test case %d token %d: expected text '%s' but was '%s'", + t.Fatalf("Test case %d token %d: expected text '%s' but was '%s'", n, i, expected[i].Text, actual[i].Text) break } diff --git a/caddyconfig/caddyfile/parse.go b/caddyconfig/caddyfile/parse.go index edc86f2..65d6ee9 100644 --- a/caddyconfig/caddyfile/parse.go +++ b/caddyconfig/caddyfile/parse.go @@ -20,11 +20,11 @@ import ( "io" "os" "path/filepath" - "strconv" "strings" - "github.com/caddyserver/caddy/v2" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) // Parse parses the input just enough to group tokens, in @@ -149,7 +149,6 @@ func (p *parser) begin() error { } err := p.addresses() - if err != nil { return err } @@ -160,6 +159,25 @@ func (p *parser) begin() error { return nil } + if ok, name := p.isNamedRoute(); ok { + // named routes only have one key, the route name + p.block.Keys = []string{name} + p.block.IsNamedRoute = true + + // we just need a dummy leading token to ease parsing later + nameToken := p.Token() + nameToken.Text = name + + // get all the tokens from the block, including the braces + tokens, err := p.blockTokens(true) + if err != nil { + return err + } + tokens = append([]Token{nameToken}, tokens...) + p.block.Segments = []Segment{tokens} + return nil + } + if ok, name := p.isSnippet(); ok { if p.definedSnippets == nil { p.definedSnippets = map[string][]Token{} @@ -168,16 +186,15 @@ func (p *parser) begin() error { return p.Errf("redeclaration of previously declared snippet %s", name) } // consume all tokens til matched close brace - tokens, err := p.snippetTokens() + tokens, err := p.blockTokens(false) if err != nil { return err } // Just as we need to track which file the token comes from, we need to - // keep track of which snippets do the tokens come from. This is helpful - // in tracking import cycles across files/snippets by namespacing them. Without - // this we end up with false-positives in cycle-detection. + // keep track of which snippet the token comes from. This is helpful + // in tracking import cycles across files/snippets by namespacing them. + // Without this, we end up with false-positives in cycle-detection. for k, v := range tokens { - v.inSnippet = true v.snippetName = name tokens[k] = v } @@ -198,7 +215,7 @@ func (p *parser) addresses() error { // special case: import directive replaces tokens during parse-time if tkn == "import" && p.isNewLine() { - err := p.doImport() + err := p.doImport(0) if err != nil { return err } @@ -298,7 +315,7 @@ func (p *parser) directives() error { // special case: import directive replaces tokens during parse-time if p.Val() == "import" { - err := p.doImport() + err := p.doImport(1) if err != nil { return err } @@ -324,7 +341,7 @@ func (p *parser) directives() error { // is on the token before where the import directive was. In // other words, call Next() to access the first token that was // imported. -func (p *parser) doImport() error { +func (p *parser) doImport(nesting int) error { // syntax checks if !p.NextArg() { return p.ArgErr() @@ -337,11 +354,8 @@ func (p *parser) doImport() error { // grab remaining args as placeholder replacements args := p.RemainingArgs() - // add args to the replacer - repl := caddy.NewEmptyReplacer() - for index, arg := range args { - repl.Set("args."+strconv.Itoa(index), arg) - } + // set up a replacer for non-variadic args replacement + repl := makeArgsReplacer(args) // splice out the import directive and its arguments // (2 tokens, plus the length of args) @@ -417,7 +431,7 @@ func (p *parser) doImport() error { } nodeName := p.File() - if p.Token().inSnippet { + if p.Token().snippetName != "" { nodeName += fmt.Sprintf(":%s", p.Token().snippetName) } p.importGraph.addNode(nodeName) @@ -428,13 +442,69 @@ func (p *parser) doImport() error { } // copy the tokens so we don't overwrite p.definedSnippets - tokensCopy := make([]Token, len(importedTokens)) - copy(tokensCopy, importedTokens) + tokensCopy := make([]Token, 0, len(importedTokens)) + + var ( + maybeSnippet bool + maybeSnippetId bool + index int + ) // run the argument replacer on the tokens - for index, token := range tokensCopy { - token.Text = repl.ReplaceKnown(token.Text, "") - tokensCopy[index] = token + // golang for range slice return a copy of value + // similarly, append also copy value + for i, token := range importedTokens { + // update the token's imports to refer to import directive filename, line number and snippet name if there is one + if token.snippetName != "" { + token.imports = append(token.imports, fmt.Sprintf("%s:%d (import %s)", p.File(), p.Line(), token.snippetName)) + } else { + token.imports = append(token.imports, fmt.Sprintf("%s:%d (import)", p.File(), p.Line())) + } + + // naive way of determine snippets, as snippets definition can only follow name + block + // format, won't check for nesting correctness or any other error, that's what parser does. + if !maybeSnippet && nesting == 0 { + // first of the line + if i == 0 || isNextOnNewLine(tokensCopy[i-1], token) { + index = 0 + } else { + index++ + } + + if index == 0 && len(token.Text) >= 3 && strings.HasPrefix(token.Text, "(") && strings.HasSuffix(token.Text, ")") { + maybeSnippetId = true + } + } + + switch token.Text { + case "{": + nesting++ + if index == 1 && maybeSnippetId && nesting == 1 { + maybeSnippet = true + maybeSnippetId = false + } + case "}": + nesting-- + if nesting == 0 && maybeSnippet { + maybeSnippet = false + } + } + + if maybeSnippet { + tokensCopy = append(tokensCopy, token) + continue + } + + foundVariadic, startIndex, endIndex := parseVariadic(token, len(args)) + if foundVariadic { + for _, arg := range args[startIndex:endIndex] { + token.Text = arg + tokensCopy = append(tokensCopy, token) + } + } else { + token.Text = repl.ReplaceKnown(token.Text, "") + tokensCopy = append(tokensCopy, token) + } } // splice the imported tokens in the place of the import statement @@ -496,7 +566,6 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) { // are loaded into the current server block for later use // by directive setup functions. func (p *parser) directive() error { - // a segment is a list of tokens associated with this directive var segment Segment @@ -509,6 +578,9 @@ func (p *parser) directive() error { if !p.isNextOnNewLine() && p.Token().wasQuoted == 0 { return p.Err("Unexpected next token after '{' on same line") } + if p.isNewLine() { + return p.Err("Unexpected '{' on a new line; did you mean to place the '{' on the previous line?") + } } else if p.Val() == "{}" { if p.isNextOnNewLine() && p.Token().wasQuoted == 0 { return p.Err("Unexpected '{}' at end of line") @@ -521,7 +593,7 @@ func (p *parser) directive() error { } else if p.Val() == "}" && p.nesting == 0 { return p.Err("Unexpected '}' because no matching opening brace") } else if p.Val() == "import" && p.isNewLine() { - if err := p.doImport(); err != nil { + if err := p.doImport(1); err != nil { return err } p.cursor-- // cursor is advanced when we continue, so roll back one more @@ -562,6 +634,15 @@ func (p *parser) closeCurlyBrace() error { return nil } +func (p *parser) isNamedRoute() (bool, string) { + keys := p.block.Keys + // A named route block is a single key with parens, prefixed with &. + if len(keys) == 1 && strings.HasPrefix(keys[0], "&(") && strings.HasSuffix(keys[0], ")") { + return true, strings.TrimSuffix(keys[0][2:], ")") + } + return false, "" +} + func (p *parser) isSnippet() (bool, string) { keys := p.block.Keys // A snippet block is a single key with parens. Nothing else qualifies. @@ -572,18 +653,24 @@ func (p *parser) isSnippet() (bool, string) { } // read and store everything in a block for later replay. -func (p *parser) snippetTokens() ([]Token, error) { - // snippet must have curlies. +func (p *parser) blockTokens(retainCurlies bool) ([]Token, error) { + // block must have curlies. err := p.openCurlyBrace() if err != nil { return nil, err } - nesting := 1 // count our own nesting in snippets + nesting := 1 // count our own nesting tokens := []Token{} + if retainCurlies { + tokens = append(tokens, p.Token()) + } for p.Next() { if p.Val() == "}" { nesting-- if nesting == 0 { + if retainCurlies { + tokens = append(tokens, p.Token()) + } break } } @@ -603,9 +690,10 @@ func (p *parser) snippetTokens() ([]Token, error) { // head of the server block with tokens, which are // grouped by segments. type ServerBlock struct { - HasBraces bool - Keys []string - Segments []Segment + HasBraces bool + Keys []string + Segments []Segment + IsNamedRoute bool } // DispenseDirective returns a dispenser that contains diff --git a/caddyconfig/caddyfile/parse_test.go b/caddyconfig/caddyfile/parse_test.go index 4d18cc4..b1104ed 100644 --- a/caddyconfig/caddyfile/parse_test.go +++ b/caddyconfig/caddyfile/parse_test.go @@ -21,6 +21,88 @@ import ( "testing" ) +func TestParseVariadic(t *testing.T) { + var args = make([]string, 10) + for i, tc := range []struct { + input string + result bool + }{ + { + input: "", + result: false, + }, + { + input: "{args[1", + result: false, + }, + { + input: "1]}", + result: false, + }, + { + input: "{args[:]}aaaaa", + result: false, + }, + { + input: "aaaaa{args[:]}", + result: false, + }, + { + input: "{args.}", + result: false, + }, + { + input: "{args.1}", + result: false, + }, + { + input: "{args[]}", + result: false, + }, + { + input: "{args[:]}", + result: true, + }, + { + input: "{args[:]}", + result: true, + }, + { + input: "{args[0:]}", + result: true, + }, + { + input: "{args[:0]}", + result: true, + }, + { + input: "{args[-1:]}", + result: false, + }, + { + input: "{args[:11]}", + result: false, + }, + { + input: "{args[10:0]}", + result: false, + }, + { + input: "{args[0:10]}", + result: true, + }, + } { + token := Token{ + File: "test", + Line: 1, + Text: tc.input, + } + if v, _, _ := parseVariadic(token, len(args)); v != tc.result { + t.Errorf("Test %d error expectation failed Expected: %t, got %t", i, tc.result, v) + } + } +} + func TestAllTokens(t *testing.T) { input := []byte("a b c\nd e") expected := []string{"a", "b", "c", "d", "e"} @@ -211,6 +293,14 @@ func TestParseOneAndImport(t *testing.T) { // Unexpected next token after '{' on same line {`localhost dir1 { a b }`, true, []string{"localhost"}, []int{}}, + + // Unexpected '{' on a new line + {`localhost + dir1 + { + a b + }`, true, []string{"localhost"}, []int{}}, + // Workaround with quotes {`localhost dir1 "{" a b "}"`, false, []string{"localhost"}, []int{5}}, @@ -628,6 +718,36 @@ func TestEnvironmentReplacement(t *testing.T) { } } +func TestImportReplacementInJSONWithBrace(t *testing.T) { + for i, test := range []struct { + args []string + input string + expect string + }{ + { + args: []string{"123"}, + input: "{args[0]}", + expect: "123", + }, + { + args: []string{"123"}, + input: `{"key":"{args[0]}"}`, + expect: `{"key":"123"}`, + }, + { + args: []string{"123", "123"}, + input: `{"key":[{args[0]},{args[1]}]}`, + expect: `{"key":[123,123]}`, + }, + } { + repl := makeArgsReplacer(test.args) + actual := repl.ReplaceKnown(test.input, "") + if actual != test.expect { + t.Errorf("Test %d: Expected: '%s' but got '%s'", i, test.expect, actual) + } + } +} + func TestSnippets(t *testing.T) { p := testParser(` (common) { diff --git a/caddyconfig/caddyfile/testdata/import_args0.txt b/caddyconfig/caddyfile/testdata/import_args0.txt index af946fe..add211e 100644 --- a/caddyconfig/caddyfile/testdata/import_args0.txt +++ b/caddyconfig/caddyfile/testdata/import_args0.txt @@ -1 +1 @@ -{args.0}
\ No newline at end of file +{args[0]}
\ No newline at end of file diff --git a/caddyconfig/caddyfile/testdata/import_args1.txt b/caddyconfig/caddyfile/testdata/import_args1.txt index 519a92d..422692a 100644 --- a/caddyconfig/caddyfile/testdata/import_args1.txt +++ b/caddyconfig/caddyfile/testdata/import_args1.txt @@ -1 +1 @@ -{args.0} {args.1}
\ No newline at end of file +{args[0]} {args[1]}
\ No newline at end of file diff --git a/caddyconfig/httpcaddyfile/addresses.go b/caddyconfig/httpcaddyfile/addresses.go index 93bad27..658da48 100644 --- a/caddyconfig/httpcaddyfile/addresses.go +++ b/caddyconfig/httpcaddyfile/addresses.go @@ -24,10 +24,11 @@ import ( "strings" "unicode" + "github.com/caddyserver/certmagic" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "github.com/caddyserver/certmagic" ) // mapAddressToServerBlocks returns a map of listener address to list of server @@ -77,7 +78,8 @@ import ( // multiple addresses to the same lists of server blocks (a many:many mapping). // (Doing this is essentially a map-reduce technique.) func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBlock, - options map[string]any) (map[string][]serverBlock, error) { + options map[string]any, +) (map[string][]serverBlock, error) { sbmap := make(map[string][]serverBlock) for i, sblock := range originalServerBlocks { @@ -187,13 +189,25 @@ func (st *ServerType) consolidateAddrMappings(addrToServerBlocks map[string][]se // listenerAddrsForServerBlockKey essentially converts the Caddyfile // site addresses to Caddy listener addresses for each server block. func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key string, - options map[string]any) ([]string, error) { + options map[string]any, +) ([]string, error) { addr, err := ParseAddress(key) if err != nil { return nil, fmt.Errorf("parsing key: %v", err) } addr = addr.Normalize() + switch addr.Scheme { + case "wss": + return nil, fmt.Errorf("the scheme wss:// is only supported in browsers; use https:// instead") + case "ws": + return nil, fmt.Errorf("the scheme ws:// is only supported in browsers; use http:// instead") + case "https", "http", "": + // Do nothing or handle the valid schemes + default: + return nil, fmt.Errorf("unsupported URL scheme %s://", addr.Scheme) + } + // figure out the HTTP and HTTPS ports; either // use defaults, or override with user config httpPort, httpsPort := strconv.Itoa(caddyhttp.DefaultHTTPPort), strconv.Itoa(caddyhttp.DefaultHTTPSPort) diff --git a/caddyconfig/httpcaddyfile/builtins.go b/caddyconfig/httpcaddyfile/builtins.go index 45da4a8..94ca007 100644 --- a/caddyconfig/httpcaddyfile/builtins.go +++ b/caddyconfig/httpcaddyfile/builtins.go @@ -26,14 +26,15 @@ import ( "strings" "time" + "github.com/caddyserver/certmagic" + "github.com/mholt/acmez/acme" + "go.uber.org/zap/zapcore" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddytls" - "github.com/caddyserver/certmagic" - "github.com/mholt/acmez/acme" - "go.uber.org/zap/zapcore" ) func init() { @@ -48,6 +49,7 @@ func init() { RegisterHandlerDirective("route", parseRoute) RegisterHandlerDirective("handle", parseHandle) RegisterDirective("handle_errors", parseHandleErrors) + RegisterHandlerDirective("invoke", parseInvoke) RegisterDirective("log", parseLog) RegisterHandlerDirective("skip_log", parseSkipLog) } @@ -179,17 +181,17 @@ func parseTLS(h Helper) ([]ConfigValue, error) { case "protocols": args := h.RemainingArgs() if len(args) == 0 { - return nil, h.SyntaxErr("one or two protocols") + return nil, h.Errf("protocols requires one or two arguments") } if len(args) > 0 { if _, ok := caddytls.SupportedProtocols[args[0]]; !ok { - return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[0]) + return nil, h.Errf("wrong protocol name or protocol not supported: '%s'", args[0]) } cp.ProtocolMin = args[0] } if len(args) > 1 { if _, ok := caddytls.SupportedProtocols[args[1]]; !ok { - return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[1]) + return nil, h.Errf("wrong protocol name or protocol not supported: '%s'", args[1]) } cp.ProtocolMax = args[1] } @@ -197,7 +199,7 @@ func parseTLS(h Helper) ([]ConfigValue, error) { case "ciphers": for h.NextArg() { if !caddytls.CipherSuiteNameSupported(h.Val()) { - return nil, h.Errf("Wrong cipher suite name or cipher suite not supported: '%s'", h.Val()) + return nil, h.Errf("wrong cipher suite name or cipher suite not supported: '%s'", h.Val()) } cp.CipherSuites = append(cp.CipherSuites, h.Val()) } @@ -764,9 +766,31 @@ func parseHandleErrors(h Helper) ([]ConfigValue, error) { }, nil } +// parseInvoke parses the invoke directive. +func parseInvoke(h Helper) (caddyhttp.MiddlewareHandler, error) { + h.Next() // consume directive + if !h.NextArg() { + return nil, h.ArgErr() + } + for h.Next() || h.NextBlock(0) { + return nil, h.ArgErr() + } + + // remember that we're invoking this name + // to populate the server with these named routes + if h.State[namedRouteKey] == nil { + h.State[namedRouteKey] = map[string]struct{}{} + } + h.State[namedRouteKey].(map[string]struct{})[h.Val()] = struct{}{} + + // return the handler + return &caddyhttp.Invoke{Name: h.Val()}, nil +} + // parseLog parses the log directive. Syntax: // -// log { +// log <logger_name> { +// hostnames <hostnames...> // output <writer_module> ... // format <encoder_module> ... // level <level> @@ -787,11 +811,13 @@ func parseLogHelper(h Helper, globalLogNames map[string]struct{}) ([]ConfigValue var configValues []ConfigValue for h.Next() { // Logic below expects that a name is always present when a - // global option is being parsed. - var globalLogName string + // global option is being parsed; or an optional override + // is supported for access logs. + var logName string + if parseAsGlobalOption { if h.NextArg() { - globalLogName = h.Val() + logName = h.Val() // Only a single argument is supported. if h.NextArg() { @@ -802,26 +828,47 @@ func parseLogHelper(h Helper, globalLogNames map[string]struct{}) ([]ConfigValue // reference the default logger. See the // setupNewDefault function in the logging // package for where this is configured. - globalLogName = caddy.DefaultLoggerName + logName = caddy.DefaultLoggerName } // Verify this name is unused. - _, used := globalLogNames[globalLogName] + _, used := globalLogNames[logName] if used { - return nil, h.Err("duplicate global log option for: " + globalLogName) + return nil, h.Err("duplicate global log option for: " + logName) } - globalLogNames[globalLogName] = struct{}{} + globalLogNames[logName] = struct{}{} } else { - // No arguments are supported for the server block log directive + // An optional override of the logger name can be provided; + // otherwise a default will be used, like "log0", "log1", etc. if h.NextArg() { - return nil, h.ArgErr() + logName = h.Val() + + // Only a single argument is supported. + if h.NextArg() { + return nil, h.ArgErr() + } } } cl := new(caddy.CustomLog) + // allow overriding the current site block's hostnames for this logger; + // this is useful for setting up loggers per subdomain in a site block + // with a wildcard domain + customHostnames := []string{} + for h.NextBlock(0) { switch h.Val() { + case "hostnames": + if parseAsGlobalOption { + return nil, h.Err("hostnames is not allowed in the log global options") + } + args := h.RemainingArgs() + if len(args) == 0 { + return nil, h.ArgErr() + } + customHostnames = append(customHostnames, args...) + case "output": if !h.NextArg() { return nil, h.ArgErr() @@ -880,18 +927,16 @@ func parseLogHelper(h Helper, globalLogNames map[string]struct{}) ([]ConfigValue } case "include": - // This configuration is only allowed in the global options if !parseAsGlobalOption { - return nil, h.ArgErr() + return nil, h.Err("include is not allowed in the log directive") } for h.NextArg() { cl.Include = append(cl.Include, h.Val()) } case "exclude": - // This configuration is only allowed in the global options if !parseAsGlobalOption { - return nil, h.ArgErr() + return nil, h.Err("exclude is not allowed in the log directive") } for h.NextArg() { cl.Exclude = append(cl.Exclude, h.Val()) @@ -903,24 +948,34 @@ func parseLogHelper(h Helper, globalLogNames map[string]struct{}) ([]ConfigValue } var val namedCustomLog + val.hostnames = customHostnames + + isEmptyConfig := reflect.DeepEqual(cl, new(caddy.CustomLog)) + // Skip handling of empty logging configs - if !reflect.DeepEqual(cl, new(caddy.CustomLog)) { - if parseAsGlobalOption { - // Use indicated name for global log options - val.name = globalLogName - val.log = cl - } else { + + if parseAsGlobalOption { + // Use indicated name for global log options + val.name = logName + } else { + if logName != "" { + val.name = logName + } else if !isEmptyConfig { // Construct a log name for server log streams logCounter, ok := h.State["logCounter"].(int) if !ok { logCounter = 0 } val.name = fmt.Sprintf("log%d", logCounter) - cl.Include = []string{"http.log.access." + val.name} - val.log = cl logCounter++ h.State["logCounter"] = logCounter } + if val.name != "" { + cl.Include = []string{"http.log.access." + val.name} + } + } + if !isEmptyConfig { + val.log = cl } configValues = append(configValues, ConfigValue{ Class: "custom_log", diff --git a/caddyconfig/httpcaddyfile/builtins_test.go b/caddyconfig/httpcaddyfile/builtins_test.go index bd5e116..70f347d 100644 --- a/caddyconfig/httpcaddyfile/builtins_test.go +++ b/caddyconfig/httpcaddyfile/builtins_test.go @@ -1,6 +1,7 @@ package httpcaddyfile import ( + "strings" "testing" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" @@ -51,12 +52,13 @@ func TestLogDirectiveSyntax(t *testing.T) { }, { input: `:8080 { - log invalid { + log name-override { output file foo.log } } `, - expectError: true, + output: `{"logging":{"logs":{"default":{"exclude":["http.log.access.name-override"]},"name-override":{"writer":{"filename":"foo.log","output":"file"},"include":["http.log.access.name-override"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"name-override"}}}}}}`, + expectError: false, }, } { @@ -213,3 +215,139 @@ func TestRedirDirectiveSyntax(t *testing.T) { } } } + +func TestImportErrorLine(t *testing.T) { + for i, tc := range []struct { + input string + errorFunc func(err error) bool + }{ + { + input: `(t1) { + abort {args[:]} + } + :8080 { + import t1 + import t1 true + }`, + errorFunc: func(err error) bool { + return err != nil && strings.Contains(err.Error(), "Caddyfile:6 (import t1)") + }, + }, + { + input: `(t1) { + abort {args[:]} + } + :8080 { + import t1 true + }`, + errorFunc: func(err error) bool { + return err != nil && strings.Contains(err.Error(), "Caddyfile:5 (import t1)") + }, + }, + { + input: ` + import testdata/import_variadic_snippet.txt + :8080 { + import t1 true + }`, + errorFunc: func(err error) bool { + return err == nil + }, + }, + { + input: ` + import testdata/import_variadic_with_import.txt + :8080 { + import t1 true + import t2 true + }`, + errorFunc: func(err error) bool { + return err == nil + }, + }, + } { + adapter := caddyfile.Adapter{ + ServerType: ServerType{}, + } + + _, _, err := adapter.Adapt([]byte(tc.input), nil) + + if !tc.errorFunc(err) { + t.Errorf("Test %d error expectation failed, got %s", i, err) + continue + } + } +} + +func TestNestedImport(t *testing.T) { + for i, tc := range []struct { + input string + errorFunc func(err error) bool + }{ + { + input: `(t1) { + respond {args[0]} {args[1]} + } + + (t2) { + import t1 {args[0]} 202 + } + + :8080 { + handle { + import t2 "foobar" + } + }`, + errorFunc: func(err error) bool { + return err == nil + }, + }, + { + input: `(t1) { + respond {args[:]} + } + + (t2) { + import t1 {args[0]} {args[1]} + } + + :8080 { + handle { + import t2 "foobar" 202 + } + }`, + errorFunc: func(err error) bool { + return err == nil + }, + }, + { + input: `(t1) { + respond {args[0]} {args[1]} + } + + (t2) { + import t1 {args[:]} + } + + :8080 { + handle { + import t2 "foobar" 202 + } + }`, + errorFunc: func(err error) bool { + return err == nil + }, + }, + } { + adapter := caddyfile.Adapter{ + ServerType: ServerType{}, + } + + _, _, err := adapter.Adapt([]byte(tc.input), nil) + + if !tc.errorFunc(err) { + t.Errorf("Test %d error expectation failed, got %s", i, err) + continue + } + } +} diff --git a/caddyconfig/httpcaddyfile/directives.go b/caddyconfig/httpcaddyfile/directives.go index a772dba..13229ed 100644 --- a/caddyconfig/httpcaddyfile/directives.go +++ b/caddyconfig/httpcaddyfile/directives.go @@ -65,6 +65,7 @@ var directiveOrder = []string{ "templates", // special routing & dispatching directives + "invoke", "handle", "handle_path", "route", @@ -172,6 +173,7 @@ func (h Helper) Caddyfiles() []string { for file := range files { filesSlice = append(filesSlice, file) } + sort.Strings(filesSlice) return filesSlice } @@ -215,7 +217,8 @@ func (h Helper) ExtractMatcherSet() (caddy.ModuleMap, error) { // NewRoute returns config values relevant to creating a new HTTP route. func (h Helper) NewRoute(matcherSet caddy.ModuleMap, - handler caddyhttp.MiddlewareHandler) []ConfigValue { + handler caddyhttp.MiddlewareHandler, +) []ConfigValue { mod, err := caddy.GetModule(caddy.GetModuleID(handler)) if err != nil { *h.warnings = append(*h.warnings, caddyconfig.Warning{ @@ -427,26 +430,16 @@ func sortRoutes(routes []ConfigValue) { jPathLen = len(jPM[0]) } - // some directives involve setting values which can overwrite - // each other, so it makes most sense to reverse the order so - // that the lease specific matcher is first; everything else - // has most-specific matcher first - if iDir == "vars" { + sortByPath := func() bool { // we can only confidently compare path lengths if both // directives have a single path to match (issue #5037) if iPathLen > 0 && jPathLen > 0 { - // sort least-specific (shortest) path first - return iPathLen < jPathLen - } + // if both paths are the same except for a trailing wildcard, + // sort by the shorter path first (which is more specific) + if strings.TrimSuffix(iPM[0], "*") == strings.TrimSuffix(jPM[0], "*") { + return iPathLen < jPathLen + } - // if both directives don't have a single path to compare, - // sort whichever one has no matcher first; if both have - // no matcher, sort equally (stable sort preserves order) - return len(iRoute.MatcherSetsRaw) == 0 && len(jRoute.MatcherSetsRaw) > 0 - } else { - // we can only confidently compare path lengths if both - // directives have a single path to match (issue #5037) - if iPathLen > 0 && jPathLen > 0 { // sort most-specific (longest) path first return iPathLen > jPathLen } @@ -455,7 +448,18 @@ func sortRoutes(routes []ConfigValue) { // sort whichever one has a matcher first; if both have // a matcher, sort equally (stable sort preserves order) return len(iRoute.MatcherSetsRaw) > 0 && len(jRoute.MatcherSetsRaw) == 0 + }() + + // some directives involve setting values which can overwrite + // each other, so it makes most sense to reverse the order so + // that the least-specific matcher is first, allowing the last + // matching one to win + if iDir == "vars" { + return !sortByPath } + + // everything else is most-specific matcher first + return sortByPath }) } diff --git a/caddyconfig/httpcaddyfile/httptype.go b/caddyconfig/httpcaddyfile/httptype.go index 50e98ac..3e8fdca 100644 --- a/caddyconfig/httpcaddyfile/httptype.go +++ b/caddyconfig/httpcaddyfile/httptype.go @@ -18,18 +18,19 @@ import ( "encoding/json" "fmt" "reflect" - "regexp" "sort" "strconv" "strings" + "go.uber.org/zap" + "golang.org/x/exp/slices" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddypki" "github.com/caddyserver/caddy/v2/modules/caddytls" - "go.uber.org/zap" ) func init() { @@ -48,12 +49,13 @@ type App struct { } // ServerType can set up a config from an HTTP Caddyfile. -type ServerType struct { -} +type ServerType struct{} // Setup makes a config from the tokens. -func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, - options map[string]any) (*caddy.Config, []caddyconfig.Warning, error) { +func (st ServerType) Setup( + inputServerBlocks []caddyfile.ServerBlock, + options map[string]any, +) (*caddy.Config, []caddyconfig.Warning, error) { var warnings []caddyconfig.Warning gc := counter{new(int)} state := make(map[string]any) @@ -79,41 +81,18 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, return nil, warnings, err } - // replace shorthand placeholders (which are convenient - // when writing a Caddyfile) with their actual placeholder - // identifiers or variable names - replacer := strings.NewReplacer(placeholderShorthands()...) - - // these are placeholders that allow a user-defined final - // parameters, but we still want to provide a shorthand - // for those, so we use a regexp to replace - regexpReplacements := []struct { - search *regexp.Regexp - replace string - }{ - {regexp.MustCompile(`{header\.([\w-]*)}`), "{http.request.header.$1}"}, - {regexp.MustCompile(`{cookie\.([\w-]*)}`), "{http.request.cookie.$1}"}, - {regexp.MustCompile(`{labels\.([\w-]*)}`), "{http.request.host.labels.$1}"}, - {regexp.MustCompile(`{path\.([\w-]*)}`), "{http.request.uri.path.$1}"}, - {regexp.MustCompile(`{file\.([\w-]*)}`), "{http.request.uri.path.file.$1}"}, - {regexp.MustCompile(`{query\.([\w-]*)}`), "{http.request.uri.query.$1}"}, - {regexp.MustCompile(`{re\.([\w-]*)\.([\w-]*)}`), "{http.regexp.$1.$2}"}, - {regexp.MustCompile(`{vars\.([\w-]*)}`), "{http.vars.$1}"}, - {regexp.MustCompile(`{rp\.([\w-\.]*)}`), "{http.reverse_proxy.$1}"}, - {regexp.MustCompile(`{err\.([\w-\.]*)}`), "{http.error.$1}"}, - {regexp.MustCompile(`{file_match\.([\w-]*)}`), "{http.matchers.file.$1}"}, + // this will replace both static and user-defined placeholder shorthands + // with actual identifiers used by Caddy + replacer := NewShorthandReplacer() + + originalServerBlocks, err = st.extractNamedRoutes(originalServerBlocks, options, &warnings, replacer) + if err != nil { + return nil, warnings, err } for _, sb := range originalServerBlocks { - for _, segment := range sb.block.Segments { - for i := 0; i < len(segment); i++ { - // simple string replacements - segment[i].Text = replacer.Replace(segment[i].Text) - // complex regexp replacements - for _, r := range regexpReplacements { - segment[i].Text = r.search.ReplaceAllString(segment[i].Text, r.replace) - } - } + for i := range sb.block.Segments { + replacer.ApplyToSegment(&sb.block.Segments[i]) } if len(sb.block.Keys) == 0 { @@ -172,6 +151,18 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, result.directive = dir sb.pile[result.Class] = append(sb.pile[result.Class], result) } + + // specially handle named routes that were pulled out from + // the invoke directive, which could be nested anywhere within + // some subroutes in this directive; we add them to the pile + // for this server block + if state[namedRouteKey] != nil { + for name := range state[namedRouteKey].(map[string]struct{}) { + result := ConfigValue{Class: namedRouteKey, Value: name} + sb.pile[result.Class] = append(sb.pile[result.Class], result) + } + state[namedRouteKey] = nil + } } } @@ -222,7 +213,7 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, if ncl.name == caddy.DefaultLoggerName { hasDefaultLog = true } - if _, ok := options["debug"]; ok && ncl.log.Level == "" { + if _, ok := options["debug"]; ok && ncl.log != nil && ncl.log.Level == "" { ncl.log.Level = zap.DebugLevel.CapitalString() } customLogs = append(customLogs, ncl) @@ -241,7 +232,9 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, if _, ok := options["debug"]; ok { customLogs = append(customLogs, namedCustomLog{ name: caddy.DefaultLoggerName, - log: &caddy.CustomLog{Level: zap.DebugLevel.CapitalString()}, + log: &caddy.CustomLog{ + BaseLog: caddy.BaseLog{Level: zap.DebugLevel.CapitalString()}, + }, }) } } @@ -303,7 +296,21 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, Logs: make(map[string]*caddy.CustomLog), } } + + // Add the default log first if defined, so that it doesn't + // accidentally get re-created below due to the Exclude logic for _, ncl := range customLogs { + if ncl.name == caddy.DefaultLoggerName && ncl.log != nil { + cfg.Logging.Logs[caddy.DefaultLoggerName] = ncl.log + break + } + } + + // Add the rest of the custom logs + for _, ncl := range customLogs { + if ncl.log == nil || ncl.name == caddy.DefaultLoggerName { + continue + } if ncl.name != "" { cfg.Logging.Logs[ncl.name] = ncl.log } @@ -317,8 +324,16 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock, cfg.Logging.Logs[caddy.DefaultLoggerName] = defaultLog } defaultLog.Exclude = append(defaultLog.Exclude, ncl.log.Include...) + + // avoid duplicates by sorting + compacting + sort.Strings(defaultLog.Exclude) + defaultLog.Exclude = slices.Compact[[]string, string](defaultLog.Exclude) } } + // we may have not actually added anything, so remove if empty + if len(cfg.Logging.Logs) == 0 { + cfg.Logging = nil + } } return cfg, warnings, nil @@ -401,6 +416,81 @@ func (ServerType) evaluateGlobalOptionsBlock(serverBlocks []serverBlock, options return serverBlocks[1:], nil } +// extractNamedRoutes pulls out any named route server blocks +// so they don't get parsed as sites, and stores them in options +// for later. +func (ServerType) extractNamedRoutes( + serverBlocks []serverBlock, + options map[string]any, + warnings *[]caddyconfig.Warning, + replacer ShorthandReplacer, +) ([]serverBlock, error) { + namedRoutes := map[string]*caddyhttp.Route{} + + gc := counter{new(int)} + state := make(map[string]any) + + // copy the server blocks so we can + // splice out the named route ones + filtered := append([]serverBlock{}, serverBlocks...) + index := -1 + + for _, sb := range serverBlocks { + index++ + if !sb.block.IsNamedRoute { + continue + } + + // splice out this block, because we know it's not a real server + filtered = append(filtered[:index], filtered[index+1:]...) + index-- + + if len(sb.block.Segments) == 0 { + continue + } + + wholeSegment := caddyfile.Segment{} + for i := range sb.block.Segments { + // replace user-defined placeholder shorthands in extracted named routes + replacer.ApplyToSegment(&sb.block.Segments[i]) + + // zip up all the segments since ParseSegmentAsSubroute + // was designed to take a directive+ + wholeSegment = append(wholeSegment, sb.block.Segments[i]...) + } + + h := Helper{ + Dispenser: caddyfile.NewDispenser(wholeSegment), + options: options, + warnings: warnings, + matcherDefs: nil, + parentBlock: sb.block, + groupCounter: gc, + State: state, + } + + handler, err := ParseSegmentAsSubroute(h) + if err != nil { + return nil, err + } + subroute := handler.(*caddyhttp.Subroute) + route := caddyhttp.Route{} + + if len(subroute.Routes) == 1 && len(subroute.Routes[0].MatcherSetsRaw) == 0 { + // if there's only one route with no matcher, then we can simplify + route.HandlersRaw = append(route.HandlersRaw, subroute.Routes[0].HandlersRaw[0]) + } else { + // otherwise we need the whole subroute + route.HandlersRaw = []json.RawMessage{caddyconfig.JSONModuleObject(handler, "handler", subroute.CaddyModule().ID.Name(), h.warnings)} + } + + namedRoutes[sb.block.Keys[0]] = &route + } + options["named_routes"] = namedRoutes + + return filtered, nil +} + // serversFromPairings creates the servers for each pairing of addresses // to server blocks. Each pairing is essentially a server definition. func (st *ServerType) serversFromPairings( @@ -411,6 +501,7 @@ func (st *ServerType) serversFromPairings( ) (map[string]*caddyhttp.Server, error) { servers := make(map[string]*caddyhttp.Server) defaultSNI := tryString(options["default_sni"], warnings) + fallbackSNI := tryString(options["fallback_sni"], warnings) httpPort := strconv.Itoa(caddyhttp.DefaultHTTPPort) if hp, ok := options["http_port"].(int); ok { @@ -539,6 +630,24 @@ func (st *ServerType) serversFromPairings( } } + // add named routes to the server if 'invoke' was used inside of it + configuredNamedRoutes := options["named_routes"].(map[string]*caddyhttp.Route) + for _, sblock := range p.serverBlocks { + if len(sblock.pile[namedRouteKey]) == 0 { + continue + } + for _, value := range sblock.pile[namedRouteKey] { + if srv.NamedRoutes == nil { + srv.NamedRoutes = map[string]*caddyhttp.Route{} + } + name := value.Value.(string) + if configuredNamedRoutes[name] == nil { + return nil, fmt.Errorf("cannot invoke named route '%s', which was not defined", name) + } + srv.NamedRoutes[name] = configuredNamedRoutes[name] + } + } + // create a subroute for each site in the server block for _, sblock := range p.serverBlocks { matcherSetsEnc, err := st.compileEncodedMatcherSets(sblock) @@ -568,14 +677,21 @@ func (st *ServerType) serversFromPairings( cp.DefaultSNI = defaultSNI break } + if h == fallbackSNI { + hosts = append(hosts, "") + cp.FallbackSNI = fallbackSNI + break + } } if len(hosts) > 0 { + slices.Sort(hosts) // for deterministic JSON output cp.MatchersRaw = caddy.ModuleMap{ "sni": caddyconfig.JSON(hosts, warnings), // make sure to match all hosts, not just auto-HTTPS-qualified ones } } else { cp.DefaultSNI = defaultSNI + cp.FallbackSNI = fallbackSNI } // only append this policy if it actually changes something @@ -601,10 +717,20 @@ func (st *ServerType) serversFromPairings( } } + // If TLS is specified as directive, it will also result in 1 or more connection policy being created + // Thus, catch-all address with non-standard port, e.g. :8443, can have TLS enabled without + // specifying prefix "https://" + // Second part of the condition is to allow creating TLS conn policy even though `auto_https` has been disabled + // ensuring compatibility with behavior described in below link + // https://caddy.community/t/making-sense-of-auto-https-and-why-disabling-it-still-serves-https-instead-of-http/9761 + createdTLSConnPolicies, ok := sblock.pile["tls.connection_policy"] + hasTLSEnabled := (ok && len(createdTLSConnPolicies) > 0) || + (addr.Host != "" && srv.AutoHTTPS != nil && !sliceContains(srv.AutoHTTPS.Skip, addr.Host)) + // we'll need to remember if the address qualifies for auto-HTTPS, so we // can add a TLS conn policy if necessary if addr.Scheme == "https" || - (addr.Scheme != "http" && addr.Host != "" && addr.Port != httpPort) { + (addr.Scheme != "http" && addr.Port != httpPort && hasTLSEnabled) { addressQualifiesForTLS = true } // predict whether auto-HTTPS will add the conn policy for us; if so, we @@ -653,12 +779,20 @@ func (st *ServerType) serversFromPairings( sblockLogHosts := sblock.hostsFromKeys(true) for _, cval := range sblock.pile["custom_log"] { ncl := cval.Value.(namedCustomLog) - if sblock.hasHostCatchAllKey() { + if sblock.hasHostCatchAllKey() && len(ncl.hostnames) == 0 { // all requests for hosts not able to be listed should use // this log because it's a catch-all-hosts server block srv.Logs.DefaultLoggerName = ncl.name + } else if len(ncl.hostnames) > 0 { + // if the logger overrides the hostnames, map that to the logger name + for _, h := range ncl.hostnames { + if srv.Logs.LoggerNames == nil { + srv.Logs.LoggerNames = make(map[string]string) + } + srv.Logs.LoggerNames[h] = ncl.name + } } else { - // map each host to the user's desired logger name + // otherwise, map each host to the logger name for _, h := range sblockLogHosts { if srv.Logs.LoggerNames == nil { srv.Logs.LoggerNames = make(map[string]string) @@ -701,8 +835,8 @@ func (st *ServerType) serversFromPairings( // policy missing for any HTTPS-enabled hosts, if so, add it... maybe? if addressQualifiesForTLS && !hasCatchAllTLSConnPolicy && - (len(srv.TLSConnPolicies) > 0 || !autoHTTPSWillAddConnPolicy || defaultSNI != "") { - srv.TLSConnPolicies = append(srv.TLSConnPolicies, &caddytls.ConnectionPolicy{DefaultSNI: defaultSNI}) + (len(srv.TLSConnPolicies) > 0 || !autoHTTPSWillAddConnPolicy || defaultSNI != "" || fallbackSNI != "") { + srv.TLSConnPolicies = append(srv.TLSConnPolicies, &caddytls.ConnectionPolicy{DefaultSNI: defaultSNI, FallbackSNI: fallbackSNI}) } // tidy things up a bit @@ -911,8 +1045,8 @@ func appendSubrouteToRouteList(routeList caddyhttp.RouteList, subroute *caddyhttp.Subroute, matcherSetsEnc []caddy.ModuleMap, p sbAddrAssociation, - warnings *[]caddyconfig.Warning) caddyhttp.RouteList { - + warnings *[]caddyconfig.Warning, +) caddyhttp.RouteList { // nothing to do if... there's nothing to do if len(matcherSetsEnc) == 0 && len(subroute.Routes) == 0 && subroute.Errors == nil { return routeList @@ -974,7 +1108,7 @@ func buildSubroute(routes []ConfigValue, groupCounter counter, needsSorting bool if needsSorting { for _, val := range routes { if !directiveIsOrdered(val.directive) { - return nil, fmt.Errorf("directive '%s' is not an ordered HTTP handler, so it cannot be used here", val.directive) + return nil, fmt.Errorf("directive '%s' is not an ordered HTTP handler, so it cannot be used here - try placing within a route block or using the order global option", val.directive) } } @@ -1301,36 +1435,6 @@ func encodeMatcherSet(matchers map[string]caddyhttp.RequestMatcher) (caddy.Modul return msEncoded, nil } -// placeholderShorthands returns a slice of old-new string pairs, -// where the left of the pair is a placeholder shorthand that may -// be used in the Caddyfile, and the right is the replacement. -func placeholderShorthands() []string { - return []string{ - "{dir}", "{http.request.uri.path.dir}", - "{file}", "{http.request.uri.path.file}", - "{host}", "{http.request.host}", - "{hostport}", "{http.request.hostport}", - "{port}", "{http.request.port}", - "{method}", "{http.request.method}", - "{path}", "{http.request.uri.path}", - "{query}", "{http.request.uri.query}", - "{remote}", "{http.request.remote}", - "{remote_host}", "{http.request.remote.host}", - "{remote_port}", "{http.request.remote.port}", - "{scheme}", "{http.request.scheme}", - "{uri}", "{http.request.uri}", - "{tls_cipher}", "{http.request.tls.cipher_suite}", - "{tls_version}", "{http.request.tls.version}", - "{tls_client_fingerprint}", "{http.request.tls.client.fingerprint}", - "{tls_client_issuer}", "{http.request.tls.client.issuer}", - "{tls_client_serial}", "{http.request.tls.client.serial}", - "{tls_client_subject}", "{http.request.tls.client.subject}", - "{tls_client_certificate_pem}", "{http.request.tls.client.certificate_pem}", - "{tls_client_certificate_der_base64}", "{http.request.tls.client.certificate_der_base64}", - "{upstream_hostport}", "{http.reverse_proxy.upstream.hostport}", - } -} - // WasReplacedPlaceholderShorthand checks if a token string was // likely a replaced shorthand of the known Caddyfile placeholder // replacement outputs. Useful to prevent some user-defined map @@ -1446,8 +1550,9 @@ func (c counter) nextGroup() string { } type namedCustomLog struct { - name string - log *caddy.CustomLog + name string + hostnames []string + log *caddy.CustomLog } // sbAddrAssociation is a mapping from a list of @@ -1458,7 +1563,10 @@ type sbAddrAssociation struct { serverBlocks []serverBlock } -const matcherPrefix = "@" +const ( + matcherPrefix = "@" + namedRouteKey = "named_route" +) // Interface guard var _ caddyfile.ServerType = (*ServerType)(nil) diff --git a/caddyconfig/httpcaddyfile/options.go b/caddyconfig/httpcaddyfile/options.go index 4e5212b..ba1896b 100644 --- a/caddyconfig/httpcaddyfile/options.go +++ b/caddyconfig/httpcaddyfile/options.go @@ -17,12 +17,13 @@ package httpcaddyfile import ( "strconv" + "github.com/caddyserver/certmagic" + "github.com/mholt/acmez/acme" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddytls" - "github.com/caddyserver/certmagic" - "github.com/mholt/acmez/acme" ) func init() { @@ -33,6 +34,7 @@ func init() { RegisterGlobalOption("grace_period", parseOptDuration) RegisterGlobalOption("shutdown_delay", parseOptDuration) RegisterGlobalOption("default_sni", parseOptSingleString) + RegisterGlobalOption("fallback_sni", parseOptSingleString) RegisterGlobalOption("order", parseOptOrder) RegisterGlobalOption("storage", parseOptStorage) RegisterGlobalOption("storage_clean_interval", parseOptDuration) diff --git a/caddyconfig/httpcaddyfile/pkiapp.go b/caddyconfig/httpcaddyfile/pkiapp.go index 3414636..b5c6821 100644 --- a/caddyconfig/httpcaddyfile/pkiapp.go +++ b/caddyconfig/httpcaddyfile/pkiapp.go @@ -174,7 +174,6 @@ func (st ServerType) buildPKIApp( options map[string]any, warnings []caddyconfig.Warning, ) (*caddypki.PKI, []caddyconfig.Warning, error) { - skipInstallTrust := false if _, ok := options["skip_install_trust"]; ok { skipInstallTrust = true diff --git a/caddyconfig/httpcaddyfile/serveroptions.go b/caddyconfig/httpcaddyfile/serveroptions.go index eb57c58..6d7c678 100644 --- a/caddyconfig/httpcaddyfile/serveroptions.go +++ b/caddyconfig/httpcaddyfile/serveroptions.go @@ -18,11 +18,12 @@ import ( "encoding/json" "fmt" + "github.com/dustin/go-humanize" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "github.com/dustin/go-humanize" ) // serverOptions collects server config overrides parsed from Caddyfile global options @@ -41,9 +42,11 @@ type serverOptions struct { IdleTimeout caddy.Duration KeepAliveInterval caddy.Duration MaxHeaderBytes int + EnableFullDuplex bool Protocols []string StrictSNIHost *bool TrustedProxiesRaw json.RawMessage + ClientIPHeaders []string ShouldLogCredentials bool Metrics *caddyhttp.Metrics } @@ -156,6 +159,12 @@ func unmarshalCaddyfileServerOptions(d *caddyfile.Dispenser) (any, error) { } serverOpts.MaxHeaderBytes = int(size) + case "enable_full_duplex": + if d.NextArg() { + return nil, d.ArgErr() + } + serverOpts.EnableFullDuplex = true + case "log_credentials": if d.NextArg() { return nil, d.ArgErr() @@ -208,6 +217,18 @@ func unmarshalCaddyfileServerOptions(d *caddyfile.Dispenser) (any, error) { ) serverOpts.TrustedProxiesRaw = jsonSource + case "client_ip_headers": + headers := d.RemainingArgs() + for _, header := range headers { + if sliceContains(serverOpts.ClientIPHeaders, header) { + return nil, d.Errf("client IP header %s specified more than once", header) + } + serverOpts.ClientIPHeaders = append(serverOpts.ClientIPHeaders, header) + } + if nesting := d.Nesting(); d.NextBlock(nesting) { + return nil, d.ArgErr() + } + case "metrics": if d.NextArg() { return nil, d.ArgErr() @@ -314,9 +335,11 @@ func applyServerOptions( server.IdleTimeout = opts.IdleTimeout server.KeepAliveInterval = opts.KeepAliveInterval server.MaxHeaderBytes = opts.MaxHeaderBytes + server.EnableFullDuplex = opts.EnableFullDuplex server.Protocols = opts.Protocols server.StrictSNIHost = opts.StrictSNIHost server.TrustedProxiesRaw = opts.TrustedProxiesRaw + server.ClientIPHeaders = opts.ClientIPHeaders server.Metrics = opts.Metrics if opts.ShouldLogCredentials { if server.Logs == nil { diff --git a/caddyconfig/httpcaddyfile/shorthands.go b/caddyconfig/httpcaddyfile/shorthands.go new file mode 100644 index 0000000..102bc36 --- /dev/null +++ b/caddyconfig/httpcaddyfile/shorthands.go @@ -0,0 +1,92 @@ +package httpcaddyfile + +import ( + "regexp" + "strings" + + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" +) + +type ComplexShorthandReplacer struct { + search *regexp.Regexp + replace string +} + +type ShorthandReplacer struct { + complex []ComplexShorthandReplacer + simple *strings.Replacer +} + +func NewShorthandReplacer() ShorthandReplacer { + // replace shorthand placeholders (which are convenient + // when writing a Caddyfile) with their actual placeholder + // identifiers or variable names + replacer := strings.NewReplacer(placeholderShorthands()...) + + // these are placeholders that allow a user-defined final + // parameters, but we still want to provide a shorthand + // for those, so we use a regexp to replace + regexpReplacements := []ComplexShorthandReplacer{ + {regexp.MustCompile(`{header\.([\w-]*)}`), "{http.request.header.$1}"}, + {regexp.MustCompile(`{cookie\.([\w-]*)}`), "{http.request.cookie.$1}"}, + {regexp.MustCompile(`{labels\.([\w-]*)}`), "{http.request.host.labels.$1}"}, + {regexp.MustCompile(`{path\.([\w-]*)}`), "{http.request.uri.path.$1}"}, + {regexp.MustCompile(`{file\.([\w-]*)}`), "{http.request.uri.path.file.$1}"}, + {regexp.MustCompile(`{query\.([\w-]*)}`), "{http.request.uri.query.$1}"}, + {regexp.MustCompile(`{re\.([\w-]*)\.([\w-]*)}`), "{http.regexp.$1.$2}"}, + {regexp.MustCompile(`{vars\.([\w-]*)}`), "{http.vars.$1}"}, + {regexp.MustCompile(`{rp\.([\w-\.]*)}`), "{http.reverse_proxy.$1}"}, + {regexp.MustCompile(`{err\.([\w-\.]*)}`), "{http.error.$1}"}, + {regexp.MustCompile(`{file_match\.([\w-]*)}`), "{http.matchers.file.$1}"}, + } + + return ShorthandReplacer{ + complex: regexpReplacements, + simple: replacer, + } +} + +// placeholderShorthands returns a slice of old-new string pairs, +// where the left of the pair is a placeholder shorthand that may +// be used in the Caddyfile, and the right is the replacement. +func placeholderShorthands() []string { + return []string{ + "{dir}", "{http.request.uri.path.dir}", + "{file}", "{http.request.uri.path.file}", + "{host}", "{http.request.host}", + "{hostport}", "{http.request.hostport}", + "{port}", "{http.request.port}", + "{method}", "{http.request.method}", + "{path}", "{http.request.uri.path}", + "{query}", "{http.request.uri.query}", + "{remote}", "{http.request.remote}", + "{remote_host}", "{http.request.remote.host}", + "{remote_port}", "{http.request.remote.port}", + "{scheme}", "{http.request.scheme}", + "{uri}", "{http.request.uri}", + "{tls_cipher}", "{http.request.tls.cipher_suite}", + "{tls_version}", "{http.request.tls.version}", + "{tls_client_fingerprint}", "{http.request.tls.client.fingerprint}", + "{tls_client_issuer}", "{http.request.tls.client.issuer}", + "{tls_client_serial}", "{http.request.tls.client.serial}", + "{tls_client_subject}", "{http.request.tls.client.subject}", + "{tls_client_certificate_pem}", "{http.request.tls.client.certificate_pem}", + "{tls_client_certificate_der_base64}", "{http.request.tls.client.certificate_der_base64}", + "{upstream_hostport}", "{http.reverse_proxy.upstream.hostport}", + "{client_ip}", "{http.vars.client_ip}", + } +} + +// ApplyToSegment replaces shorthand placeholder to its full placeholder, understandable by Caddy. +func (s ShorthandReplacer) ApplyToSegment(segment *caddyfile.Segment) { + if segment != nil { + for i := 0; i < len(*segment); i++ { + // simple string replacements + (*segment)[i].Text = s.simple.Replace((*segment)[i].Text) + // complex regexp replacements + for _, r := range s.complex { + (*segment)[i].Text = r.search.ReplaceAllString((*segment)[i].Text, r.replace) + } + } + } +} diff --git a/caddyconfig/httpcaddyfile/testdata/import_variadic.txt b/caddyconfig/httpcaddyfile/testdata/import_variadic.txt new file mode 100644 index 0000000..f1e50e0 --- /dev/null +++ b/caddyconfig/httpcaddyfile/testdata/import_variadic.txt @@ -0,0 +1,9 @@ +(t2) { + respond 200 { + body {args[:]} + } +} + +:8082 { + import t2 false +}
\ No newline at end of file diff --git a/caddyconfig/httpcaddyfile/testdata/import_variadic_snippet.txt b/caddyconfig/httpcaddyfile/testdata/import_variadic_snippet.txt new file mode 100644 index 0000000..a02fcf9 --- /dev/null +++ b/caddyconfig/httpcaddyfile/testdata/import_variadic_snippet.txt @@ -0,0 +1,9 @@ +(t1) { + respond 200 { + body {args[:]} + } +} + +:8081 { + import t1 false +}
\ No newline at end of file diff --git a/caddyconfig/httpcaddyfile/testdata/import_variadic_with_import.txt b/caddyconfig/httpcaddyfile/testdata/import_variadic_with_import.txt new file mode 100644 index 0000000..ab1b32d --- /dev/null +++ b/caddyconfig/httpcaddyfile/testdata/import_variadic_with_import.txt @@ -0,0 +1,15 @@ +(t1) { + respond 200 { + body {args[:]} + } +} + +:8081 { + import t1 false +} + +import import_variadic.txt + +:8083 { + import t2 true +}
\ No newline at end of file diff --git a/caddyconfig/httpcaddyfile/tlsapp.go b/caddyconfig/httpcaddyfile/tlsapp.go index 3d32b4f..927f225 100644 --- a/caddyconfig/httpcaddyfile/tlsapp.go +++ b/caddyconfig/httpcaddyfile/tlsapp.go @@ -23,12 +23,13 @@ import ( "strconv" "strings" + "github.com/caddyserver/certmagic" + "github.com/mholt/acmez/acme" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddytls" - "github.com/caddyserver/certmagic" - "github.com/mholt/acmez/acme" ) func (st ServerType) buildTLSApp( @@ -36,7 +37,6 @@ func (st ServerType) buildTLSApp( options map[string]any, warnings []caddyconfig.Warning, ) (*caddytls.TLS, []caddyconfig.Warning, error) { - tlsApp := &caddytls.TLS{CertificatesRaw: make(caddy.ModuleMap)} var certLoaders []caddytls.CertificateLoader @@ -206,8 +206,8 @@ func (st ServerType) buildTLSApp( } // associate our new automation policy with this server block's hosts - ap.Subjects = sblock.hostsFromKeysNotHTTP(httpPort) - sort.Strings(ap.Subjects) // solely for deterministic test results + ap.SubjectsRaw = sblock.hostsFromKeysNotHTTP(httpPort) + sort.Strings(ap.SubjectsRaw) // solely for deterministic test results // if a combination of public and internal names were given // for this same server block and no issuer was specified, we @@ -217,7 +217,11 @@ func (st ServerType) buildTLSApp( var ap2 *caddytls.AutomationPolicy if len(ap.Issuers) == 0 { var internal, external []string - for _, s := range ap.Subjects { + for _, s := range ap.SubjectsRaw { + // do not create Issuers for Tailscale domains; they will be given a Manager instead + if strings.HasSuffix(strings.ToLower(s), ".ts.net") { + continue + } if !certmagic.SubjectQualifiesForCert(s) { return nil, warnings, fmt.Errorf("subject does not qualify for certificate: '%s'", s) } @@ -235,10 +239,10 @@ func (st ServerType) buildTLSApp( } } if len(external) > 0 && len(internal) > 0 { - ap.Subjects = external + ap.SubjectsRaw = external apCopy := *ap ap2 = &apCopy - ap2.Subjects = internal + ap2.SubjectsRaw = internal ap2.IssuersRaw = []json.RawMessage{caddyconfig.JSONModuleObject(caddytls.InternalIssuer{}, "module", "internal", &warnings)} } } @@ -339,14 +343,14 @@ func (st ServerType) buildTLSApp( for h := range httpsHostsSharedWithHostlessKey { al = append(al, h) if !certmagic.SubjectQualifiesForPublicCert(h) { - internalAP.Subjects = append(internalAP.Subjects, h) + internalAP.SubjectsRaw = append(internalAP.SubjectsRaw, h) } } } if len(al) > 0 { tlsApp.CertificatesRaw["automate"] = caddyconfig.JSON(al, &warnings) } - if len(internalAP.Subjects) > 0 { + if len(internalAP.SubjectsRaw) > 0 { if tlsApp.Automation == nil { tlsApp.Automation = new(caddytls.AutomationConfig) } @@ -412,7 +416,7 @@ func (st ServerType) buildTLSApp( // for convenience) automationHostSet := make(map[string]struct{}) for _, ap := range tlsApp.Automation.Policies { - for _, s := range ap.Subjects { + for _, s := range ap.SubjectsRaw { if _, ok := automationHostSet[s]; ok { return nil, warnings, fmt.Errorf("hostname appears in more than one automation policy, making certificate management ambiguous: %s", s) } @@ -533,7 +537,7 @@ func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls if automationPolicyIsSubset(aps[j], aps[i]) { return false } - return len(aps[i].Subjects) > len(aps[j].Subjects) + return len(aps[i].SubjectsRaw) > len(aps[j].SubjectsRaw) }) emptyAPCount := 0 @@ -541,7 +545,7 @@ func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls // compute the number of empty policies (disregarding subjects) - see #4128 emptyAP := new(caddytls.AutomationPolicy) for i := 0; i < len(aps); i++ { - emptyAP.Subjects = aps[i].Subjects + emptyAP.SubjectsRaw = aps[i].SubjectsRaw if reflect.DeepEqual(aps[i], emptyAP) { emptyAPCount++ if !automationPolicyHasAllPublicNames(aps[i]) { @@ -583,7 +587,7 @@ outer: aps[i].KeyType == aps[j].KeyType && aps[i].OnDemand == aps[j].OnDemand && aps[i].RenewalWindowRatio == aps[j].RenewalWindowRatio { - if len(aps[i].Subjects) > 0 && len(aps[j].Subjects) == 0 { + if len(aps[i].SubjectsRaw) > 0 && len(aps[j].SubjectsRaw) == 0 { // later policy (at j) has no subjects ("catch-all"), so we can // remove the identical-but-more-specific policy that comes first // AS LONG AS it is not shadowed by another policy before it; e.g. @@ -598,9 +602,9 @@ outer: } } else { // avoid repeated subjects - for _, subj := range aps[j].Subjects { - if !sliceContains(aps[i].Subjects, subj) { - aps[i].Subjects = append(aps[i].Subjects, subj) + for _, subj := range aps[j].SubjectsRaw { + if !sliceContains(aps[i].SubjectsRaw, subj) { + aps[i].SubjectsRaw = append(aps[i].SubjectsRaw, subj) } } aps = append(aps[:j], aps[j+1:]...) @@ -616,15 +620,15 @@ outer: // automationPolicyIsSubset returns true if a's subjects are a subset // of b's subjects. func automationPolicyIsSubset(a, b *caddytls.AutomationPolicy) bool { - if len(b.Subjects) == 0 { + if len(b.SubjectsRaw) == 0 { return true } - if len(a.Subjects) == 0 { + if len(a.SubjectsRaw) == 0 { return false } - for _, aSubj := range a.Subjects { + for _, aSubj := range a.SubjectsRaw { var inSuperset bool - for _, bSubj := range b.Subjects { + for _, bSubj := range b.SubjectsRaw { if certmagic.MatchWildcard(aSubj, bSubj) { inSuperset = true break @@ -662,7 +666,7 @@ func subjectQualifiesForPublicCert(ap *caddytls.AutomationPolicy, subj string) b } func automationPolicyHasAllPublicNames(ap *caddytls.AutomationPolicy) bool { - for _, subj := range ap.Subjects { + for _, subj := range ap.SubjectsRaw { if !subjectQualifiesForPublicCert(ap, subj) { return false } diff --git a/caddyconfig/httpcaddyfile/tlsapp_test.go b/caddyconfig/httpcaddyfile/tlsapp_test.go index 1925e02..d8edbdf 100644 --- a/caddyconfig/httpcaddyfile/tlsapp_test.go +++ b/caddyconfig/httpcaddyfile/tlsapp_test.go @@ -47,8 +47,8 @@ func TestAutomationPolicyIsSubset(t *testing.T) { expect: false, }, } { - apA := &caddytls.AutomationPolicy{Subjects: test.a} - apB := &caddytls.AutomationPolicy{Subjects: test.b} + apA := &caddytls.AutomationPolicy{SubjectsRaw: test.a} + apB := &caddytls.AutomationPolicy{SubjectsRaw: test.b} if actual := automationPolicyIsSubset(apA, apB); actual != test.expect { t.Errorf("Test %d: Expected %t but got %t (A: %v B: %v)", i, test.expect, actual, test.a, test.b) } diff --git a/caddyconfig/httploader.go b/caddyconfig/httploader.go index 7c4dc23..e0ce4eb 100644 --- a/caddyconfig/httploader.go +++ b/caddyconfig/httploader.go @@ -30,8 +30,14 @@ func init() { caddy.RegisterModule(HTTPLoader{}) } -// HTTPLoader can load Caddy configs over HTTP(S). It can adapt the config -// based on the Content-Type header of the HTTP response. +// HTTPLoader can load Caddy configs over HTTP(S). +// +// If the response is not a JSON config, a config adapter must be specified +// either in the loader config (`adapter`), or in the Content-Type HTTP header +// returned in the HTTP response from the server. The Content-Type header is +// read just like the admin API's `/load` endpoint. Uf you don't have control +// over the HTTP server (but can still trust its response), you can override +// the Content-Type header by setting the `adapter` property in this config. type HTTPLoader struct { // The method for the request. Default: GET Method string `json:"method,omitempty"` @@ -45,6 +51,11 @@ type HTTPLoader struct { // Maximum time allowed for a complete connection and request. Timeout caddy.Duration `json:"timeout,omitempty"` + // The name of the config adapter to use, if any. Only needed + // if the HTTP response is not a JSON config and if the server's + // Content-Type header is missing or incorrect. + Adapter string `json:"adapter,omitempty"` + TLS *struct { // Present this instance's managed remote identity credentials to the server. UseServerIdentity bool `json:"use_server_identity,omitempty"` @@ -108,7 +119,12 @@ func (hl HTTPLoader) LoadConfig(ctx caddy.Context) ([]byte, error) { return nil, err } - result, warnings, err := adaptByContentType(resp.Header.Get("Content-Type"), body) + // adapt the config based on either manually-configured adapter or server's response header + ct := resp.Header.Get("Content-Type") + if hl.Adapter != "" { + ct = "text/" + hl.Adapter + } + result, warnings, err := adaptByContentType(ct, body) if err != nil { return nil, err } diff --git a/caddytest/caddytest.go b/caddytest/caddytest.go index 0958e6c..39aca23 100644 --- a/caddytest/caddytest.go +++ b/caddytest/caddytest.go @@ -22,9 +22,10 @@ import ( "time" "github.com/aryann/difflib" - "github.com/caddyserver/caddy/v2/caddyconfig" + caddycmd "github.com/caddyserver/caddy/v2/cmd" + "github.com/caddyserver/caddy/v2/caddyconfig" // plug in Caddy modules here _ "github.com/caddyserver/caddy/v2/modules/standard" ) @@ -63,7 +64,6 @@ type Tester struct { // NewTester will create a new testing client with an attached cookie jar func NewTester(t *testing.T) *Tester { - jar, err := cookiejar.New(nil) if err != nil { t.Fatalf("failed to create cookiejar: %s", err) @@ -94,7 +94,6 @@ func timeElapsed(start time.Time, name string) { // InitServer this will configure the server with a configurion of a specific // type. The configType must be either "json" or the adapter type. func (tc *Tester) InitServer(rawConfig string, configType string) { - if err := tc.initServer(rawConfig, configType); err != nil { tc.t.Logf("failed to load config: %s", err) tc.t.Fail() @@ -108,7 +107,6 @@ func (tc *Tester) InitServer(rawConfig string, configType string) { // InitServer this will configure the server with a configurion of a specific // type. The configType must be either "json" or the adapter type. func (tc *Tester) initServer(rawConfig string, configType string) error { - if testing.Short() { tc.t.SkipNow() return nil @@ -232,7 +230,6 @@ const initConfig = `{ // validateTestPrerequisites ensures the certificates are available in the // designated path and Caddy sub-process is running. func validateTestPrerequisites(t *testing.T) error { - // check certificates are found for _, certName := range Default.Certifcates { if _, err := os.Stat(getIntegrationDir() + certName); os.IsNotExist(err) { @@ -284,7 +281,6 @@ func isCaddyAdminRunning() error { } func getIntegrationDir() string { - _, filename, _, ok := runtime.Caller(1) if !ok { panic("unable to determine the current file path") @@ -304,7 +300,6 @@ func prependCaddyFilePath(rawConfig string) string { // CreateTestingTransport creates a testing transport that forces call dialing connections to happen locally func CreateTestingTransport() *http.Transport { - dialer := net.Dialer{ Timeout: 5 * time.Second, KeepAlive: 5 * time.Second, @@ -332,7 +327,6 @@ func CreateTestingTransport() *http.Transport { // AssertLoadError will load a config and expect an error func AssertLoadError(t *testing.T, rawConfig string, configType string, expectedError string) { - tc := NewTester(t) err := tc.initServer(rawConfig, configType) @@ -343,7 +337,6 @@ func AssertLoadError(t *testing.T, rawConfig string, configType string, expected // AssertRedirect makes a request and asserts the redirection happens func (tc *Tester) AssertRedirect(requestURI string, expectedToLocation string, expectedStatusCode int) *http.Response { - redirectPolicyFunc := func(req *http.Request, via []*http.Request) error { return http.ErrUseLastResponse } @@ -381,7 +374,6 @@ func (tc *Tester) AssertRedirect(requestURI string, expectedToLocation string, e // CompareAdapt adapts a config and then compares it against an expected result func CompareAdapt(t *testing.T, filename, rawConfig string, adapterName string, expectedResponse string) bool { - cfgAdapter := caddyconfig.GetAdapter(adapterName) if cfgAdapter == nil { t.Logf("unrecognized config adapter '%s'", adapterName) @@ -469,14 +461,13 @@ func applyHeaders(t *testing.T, req *http.Request, requestHeaders []string) { // AssertResponseCode will execute the request and verify the status code, returns a response for additional assertions func (tc *Tester) AssertResponseCode(req *http.Request, expectedStatusCode int) *http.Response { - resp, err := tc.Client.Do(req) if err != nil { tc.t.Fatalf("failed to call server %s", err) } if expectedStatusCode != resp.StatusCode { - tc.t.Errorf("requesting \"%s\" expected status code: %d but got %d", req.RequestURI, expectedStatusCode, resp.StatusCode) + tc.t.Errorf("requesting \"%s\" expected status code: %d but got %d", req.URL.RequestURI(), expectedStatusCode, resp.StatusCode) } return resp @@ -484,7 +475,6 @@ func (tc *Tester) AssertResponseCode(req *http.Request, expectedStatusCode int) // AssertResponse request a URI and assert the status code and the body contains a string func (tc *Tester) AssertResponse(req *http.Request, expectedStatusCode int, expectedBody string) (*http.Response, string) { - resp := tc.AssertResponseCode(req, expectedStatusCode) defer resp.Body.Close() @@ -506,7 +496,6 @@ func (tc *Tester) AssertResponse(req *http.Request, expectedStatusCode int, expe // AssertGetResponse GET a URI and expect a statusCode and body text func (tc *Tester) AssertGetResponse(requestURI string, expectedStatusCode int, expectedBody string) (*http.Response, string) { - req, err := http.NewRequest("GET", requestURI, nil) if err != nil { tc.t.Fatalf("unable to create request %s", err) @@ -517,7 +506,6 @@ func (tc *Tester) AssertGetResponse(requestURI string, expectedStatusCode int, e // AssertDeleteResponse request a URI and expect a statusCode and body text func (tc *Tester) AssertDeleteResponse(requestURI string, expectedStatusCode int, expectedBody string) (*http.Response, string) { - req, err := http.NewRequest("DELETE", requestURI, nil) if err != nil { tc.t.Fatalf("unable to create request %s", err) @@ -528,7 +516,6 @@ func (tc *Tester) AssertDeleteResponse(requestURI string, expectedStatusCode int // AssertPostResponseBody POST to a URI and assert the response code and body func (tc *Tester) AssertPostResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) { - req, err := http.NewRequest("POST", requestURI, requestBody) if err != nil { tc.t.Errorf("failed to create request %s", err) @@ -542,7 +529,6 @@ func (tc *Tester) AssertPostResponseBody(requestURI string, requestHeaders []str // AssertPutResponseBody PUT to a URI and assert the response code and body func (tc *Tester) AssertPutResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) { - req, err := http.NewRequest("PUT", requestURI, requestBody) if err != nil { tc.t.Errorf("failed to create request %s", err) @@ -556,7 +542,6 @@ func (tc *Tester) AssertPutResponseBody(requestURI string, requestHeaders []stri // AssertPatchResponseBody PATCH to a URI and assert the response code and body func (tc *Tester) AssertPatchResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) { - req, err := http.NewRequest("PATCH", requestURI, requestBody) if err != nil { tc.t.Errorf("failed to create request %s", err) diff --git a/caddytest/integration/caddyfile_adapt/enable_tls_for_catch_all_site.txt b/caddytest/integration/caddyfile_adapt/enable_tls_for_catch_all_site.txt new file mode 100644 index 0000000..b37b40c --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/enable_tls_for_catch_all_site.txt @@ -0,0 +1,37 @@ +:8443 { + tls internal { + on_demand + } +} +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8443" + ], + "tls_connection_policies": [ + {} + ] + } + } + }, + "tls": { + "automation": { + "policies": [ + { + "issuers": [ + { + "module": "internal" + } + ], + "on_demand": true + } + ] + } + } + } +} + diff --git a/caddytest/integration/caddyfile_adapt/encode_options.txt b/caddytest/integration/caddyfile_adapt/encode_options.txt index 6f811ab..181bc22 100644 --- a/caddytest/integration/caddyfile_adapt/encode_options.txt +++ b/caddytest/integration/caddyfile_adapt/encode_options.txt @@ -11,6 +11,7 @@ encode gzip zstd { header Content-Type application/xhtml+xml* header Content-Type application/atom+xml* header Content-Type application/rss+xml* + header Content-Type application/wasm* header Content-Type image/svg+xml* } } @@ -47,6 +48,7 @@ encode { "application/xhtml+xml*", "application/atom+xml*", "application/rss+xml*", + "application/wasm*", "image/svg+xml*" ] }, diff --git a/caddytest/integration/caddyfile_adapt/global_options.txt b/caddytest/integration/caddyfile_adapt/global_options.txt index 57831a4..6032098 100644 --- a/caddytest/integration/caddyfile_adapt/global_options.txt +++ b/caddytest/integration/caddyfile_adapt/global_options.txt @@ -69,11 +69,11 @@ } ], "on_demand": { + "ask": "https://example.com", "rate_limit": { "interval": 30000000000, "burst": 20 - }, - "ask": "https://example.com" + } } }, "disable_ocsp_stapling": true diff --git a/caddytest/integration/caddyfile_adapt/global_options_acme.txt b/caddytest/integration/caddyfile_adapt/global_options_acme.txt index 1949d17..03aee2c 100644 --- a/caddytest/integration/caddyfile_adapt/global_options_acme.txt +++ b/caddytest/integration/caddyfile_adapt/global_options_acme.txt @@ -78,11 +78,11 @@ } ], "on_demand": { + "ask": "https://example.com", "rate_limit": { "interval": 30000000000, "burst": 20 - }, - "ask": "https://example.com" + } }, "ocsp_interval": 172800000000000, "renew_interval": 86400000000000, diff --git a/caddytest/integration/caddyfile_adapt/global_options_admin.txt b/caddytest/integration/caddyfile_adapt/global_options_admin.txt index 67cf5ad..2b90d6d 100644 --- a/caddytest/integration/caddyfile_adapt/global_options_admin.txt +++ b/caddytest/integration/caddyfile_adapt/global_options_admin.txt @@ -71,11 +71,11 @@ } ], "on_demand": { + "ask": "https://example.com", "rate_limit": { "interval": 30000000000, "burst": 20 - }, - "ask": "https://example.com" + } } } } diff --git a/caddytest/integration/caddyfile_adapt/global_server_options_single.txt b/caddytest/integration/caddyfile_adapt/global_server_options_single.txt index d963604..2f3306f 100644 --- a/caddytest/integration/caddyfile_adapt/global_server_options_single.txt +++ b/caddytest/integration/caddyfile_adapt/global_server_options_single.txt @@ -11,10 +11,13 @@ idle 30s } max_header_size 100MB + enable_full_duplex log_credentials protocols h1 h2 h2c h3 strict_sni_host trusted_proxies static private_ranges + client_ip_headers Custom-Real-Client-IP X-Forwarded-For + client_ip_headers A-Third-One } } @@ -43,6 +46,7 @@ foo.com { "write_timeout": 30000000000, "idle_timeout": 30000000000, "max_header_bytes": 100000000, + "enable_full_duplex": true, "routes": [ { "match": [ @@ -67,6 +71,11 @@ foo.com { ], "source": "static" }, + "client_ip_headers": [ + "Custom-Real-Client-IP", + "X-Forwarded-For", + "A-Third-One" + ], "logs": { "should_log_credentials": true }, diff --git a/caddytest/integration/caddyfile_adapt/header.txt b/caddytest/integration/caddyfile_adapt/header.txt index 34a044d..ec2a842 100644 --- a/caddytest/integration/caddyfile_adapt/header.txt +++ b/caddytest/integration/caddyfile_adapt/header.txt @@ -17,6 +17,8 @@ +Link "Foo" +Link "Bar" } + header >Set Defer + header >Replace Deferred Replacement } ---------- { @@ -136,6 +138,31 @@ ] } } + }, + { + "handler": "headers", + "response": { + "deferred": true, + "set": { + "Set": [ + "Defer" + ] + } + } + }, + { + "handler": "headers", + "response": { + "deferred": true, + "replace": { + "Replace": [ + { + "replace": "Replacement", + "search_regexp": "Deferred" + } + ] + } + } } ] } diff --git a/caddytest/integration/caddyfile_adapt/heredoc.txt b/caddytest/integration/caddyfile_adapt/heredoc.txt new file mode 100644 index 0000000..cc1174d --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/heredoc.txt @@ -0,0 +1,50 @@ +example.com { + respond <<EOF + <html> + <head><title>Foo</title> + <body>Foo</body> + </html> + EOF 200 +} +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "example.com" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "body": "\u003chtml\u003e\n \u003chead\u003e\u003ctitle\u003eFoo\u003c/title\u003e\n \u003cbody\u003eFoo\u003c/body\u003e\n\u003c/html\u003e", + "handler": "static_response", + "status_code": 200 + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/import_args_snippet.txt b/caddytest/integration/caddyfile_adapt/import_args_snippet.txt index 9fce9ab..10d9f4c 100644 --- a/caddytest/integration/caddyfile_adapt/import_args_snippet.txt +++ b/caddytest/integration/caddyfile_adapt/import_args_snippet.txt @@ -1,6 +1,6 @@ (logging) { log { - output file /var/log/caddy/{args.0}.access.log + output file /var/log/caddy/{args[0]}.access.log } } diff --git a/caddytest/integration/caddyfile_adapt/invoke_named_routes.txt b/caddytest/integration/caddyfile_adapt/invoke_named_routes.txt new file mode 100644 index 0000000..83d9859 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/invoke_named_routes.txt @@ -0,0 +1,154 @@ +&(first) { + @first path /first + vars @first first 1 + respond "first" +} + +&(second) { + respond "second" +} + +:8881 { + invoke first + route { + invoke second + } +} + +:8882 { + handle { + invoke second + } +} + +:8883 { + respond "no invoke" +} +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8881" + ], + "routes": [ + { + "handle": [ + { + "handler": "invoke", + "name": "first" + }, + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "invoke", + "name": "second" + } + ] + } + ] + } + ] + } + ], + "named_routes": { + "first": { + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "first": 1, + "handler": "vars" + } + ], + "match": [ + { + "path": [ + "/first" + ] + } + ] + }, + { + "handle": [ + { + "body": "first", + "handler": "static_response" + } + ] + } + ] + } + ] + }, + "second": { + "handle": [ + { + "body": "second", + "handler": "static_response" + } + ] + } + } + }, + "srv1": { + "listen": [ + ":8882" + ], + "routes": [ + { + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "invoke", + "name": "second" + } + ] + } + ] + } + ] + } + ], + "named_routes": { + "second": { + "handle": [ + { + "body": "second", + "handler": "static_response" + } + ] + } + } + }, + "srv2": { + "listen": [ + ":8883" + ], + "routes": [ + { + "handle": [ + { + "body": "no invoke", + "handler": "static_response" + } + ] + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/log_override_hostname.txt b/caddytest/integration/caddyfile_adapt/log_override_hostname.txt new file mode 100644 index 0000000..4511fd4 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/log_override_hostname.txt @@ -0,0 +1,71 @@ +*.example.com { + log { + hostnames foo.example.com bar.example.com + output file /foo-bar.txt + } + log { + hostnames baz.example.com + output file /baz.txt + } +} +---------- +{ + "logging": { + "logs": { + "default": { + "exclude": [ + "http.log.access.log0", + "http.log.access.log1" + ] + }, + "log0": { + "writer": { + "filename": "/foo-bar.txt", + "output": "file" + }, + "include": [ + "http.log.access.log0" + ] + }, + "log1": { + "writer": { + "filename": "/baz.txt", + "output": "file" + }, + "include": [ + "http.log.access.log1" + ] + } + } + }, + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "*.example.com" + ] + } + ], + "terminal": true + } + ], + "logs": { + "logger_names": { + "bar.example.com": "log0", + "baz.example.com": "log1", + "foo.example.com": "log0" + } + } + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess.txt b/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess.txt new file mode 100644 index 0000000..a3b0cec --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess.txt @@ -0,0 +1,86 @@ +{ + log access-console { + include http.log.access.foo + output file access-localhost.log + format console + } + + log access-json { + include http.log.access.foo + output file access-localhost.json + format json + } +} + +http://localhost:8881 { + log foo +} +---------- +{ + "logging": { + "logs": { + "access-console": { + "writer": { + "filename": "access-localhost.log", + "output": "file" + }, + "encoder": { + "format": "console" + }, + "include": [ + "http.log.access.foo" + ] + }, + "access-json": { + "writer": { + "filename": "access-localhost.json", + "output": "file" + }, + "encoder": { + "format": "json" + }, + "include": [ + "http.log.access.foo" + ] + }, + "default": { + "exclude": [ + "http.log.access.foo" + ] + } + } + }, + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8881" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "terminal": true + } + ], + "automatic_https": { + "skip": [ + "localhost" + ] + }, + "logs": { + "logger_names": { + "localhost:8881": "foo" + } + } + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess_debug.txt b/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess_debug.txt new file mode 100644 index 0000000..e6698e4 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/log_override_name_multiaccess_debug.txt @@ -0,0 +1,91 @@ +{ + debug + + log access-console { + include http.log.access.foo + output file access-localhost.log + format console + } + + log access-json { + include http.log.access.foo + output file access-localhost.json + format json + } +} + +http://localhost:8881 { + log foo +} +---------- +{ + "logging": { + "logs": { + "access-console": { + "writer": { + "filename": "access-localhost.log", + "output": "file" + }, + "encoder": { + "format": "console" + }, + "level": "DEBUG", + "include": [ + "http.log.access.foo" + ] + }, + "access-json": { + "writer": { + "filename": "access-localhost.json", + "output": "file" + }, + "encoder": { + "format": "json" + }, + "level": "DEBUG", + "include": [ + "http.log.access.foo" + ] + }, + "default": { + "level": "DEBUG", + "exclude": [ + "http.log.access.foo" + ] + } + } + }, + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8881" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "terminal": true + } + ], + "automatic_https": { + "skip": [ + "localhost" + ] + }, + "logs": { + "logger_names": { + "localhost:8881": "foo" + } + } + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/map_and_vars_with_raw_types.txt b/caddytest/integration/caddyfile_adapt/map_and_vars_with_raw_types.txt index af9faf4..cc75630 100644 --- a/caddytest/integration/caddyfile_adapt/map_and_vars_with_raw_types.txt +++ b/caddytest/integration/caddyfile_adapt/map_and_vars_with_raw_types.txt @@ -101,15 +101,15 @@ vars { "source": "{http.request.host}" }, { - "foo": "bar", - "handler": "vars" - }, - { "abc": true, "def": 1, "ghi": 2.3, "handler": "vars", "jkl": "mn op" + }, + { + "foo": "bar", + "handler": "vars" } ] } diff --git a/caddytest/integration/caddyfile_adapt/matcher_syntax.txt b/caddytest/integration/caddyfile_adapt/matcher_syntax.txt index fb3dfb6..ffab2c7 100644 --- a/caddytest/integration/caddyfile_adapt/matcher_syntax.txt +++ b/caddytest/integration/caddyfile_adapt/matcher_syntax.txt @@ -43,6 +43,9 @@ @matcher11 remote_ip private_ranges respond @matcher11 "remote_ip matcher with private ranges" + + @matcher12 client_ip private_ranges + respond @matcher12 "client_ip matcher with private ranges" } ---------- { @@ -250,6 +253,28 @@ "handler": "static_response" } ] + }, + { + "match": [ + { + "client_ip": { + "ranges": [ + "192.168.0.0/16", + "172.16.0.0/12", + "10.0.0.0/8", + "127.0.0.1/8", + "fd00::/8", + "::1" + ] + } + } + ], + "handle": [ + { + "body": "client_ip matcher with private ranges", + "handler": "static_response" + } + ] } ] } diff --git a/caddytest/integration/caddyfile_adapt/replaceable_upstream.txt b/caddytest/integration/caddyfile_adapt/replaceable_upstream.txt new file mode 100644 index 0000000..202e330 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/replaceable_upstream.txt @@ -0,0 +1,100 @@ +*.sandbox.localhost { + @sandboxPort { + header_regexp first_label Host ^([0-9]{3})\.sandbox\. + } + handle @sandboxPort { + reverse_proxy {re.first_label.1} + } + handle { + redir {scheme}://application.localhost + } +} + +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "*.sandbox.localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "{http.regexp.first_label.1}" + } + ] + } + ] + } + ] + } + ], + "match": [ + { + "header_regexp": { + "Host": { + "name": "first_label", + "pattern": "^([0-9]{3})\\.sandbox\\." + } + } + } + ] + }, + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "static_response", + "headers": { + "Location": [ + "{http.request.scheme}://application.localhost" + ] + }, + "status_code": 302 + } + ] + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/replaceable_upstream_partial_port.txt b/caddytest/integration/caddyfile_adapt/replaceable_upstream_partial_port.txt new file mode 100644 index 0000000..7fbcb5c --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/replaceable_upstream_partial_port.txt @@ -0,0 +1,100 @@ +*.sandbox.localhost { + @sandboxPort { + header_regexp port Host ^([0-9]{3})\.sandbox\. + } + handle @sandboxPort { + reverse_proxy app:6{re.port.1} + } + handle { + redir {scheme}://application.localhost + } +} + +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "*.sandbox.localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "app:6{http.regexp.port.1}" + } + ] + } + ] + } + ] + } + ], + "match": [ + { + "header_regexp": { + "Host": { + "name": "port", + "pattern": "^([0-9]{3})\\.sandbox\\." + } + } + } + ] + }, + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "static_response", + "headers": { + "Location": [ + "{http.request.scheme}://application.localhost" + ] + }, + "status_code": 302 + } + ] + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/replaceable_upstream_port.txt b/caddytest/integration/caddyfile_adapt/replaceable_upstream_port.txt new file mode 100644 index 0000000..8f75c5b --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/replaceable_upstream_port.txt @@ -0,0 +1,100 @@ +*.sandbox.localhost { + @sandboxPort { + header_regexp port Host ^([0-9]{3})\.sandbox\. + } + handle @sandboxPort { + reverse_proxy app:{re.port.1} + } + handle { + redir {scheme}://application.localhost + } +} + +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "*.sandbox.localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "app:{http.regexp.port.1}" + } + ] + } + ] + } + ] + } + ], + "match": [ + { + "header_regexp": { + "Host": { + "name": "port", + "pattern": "^([0-9]{3})\\.sandbox\\." + } + } + } + ] + }, + { + "group": "group2", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "static_response", + "headers": { + "Location": [ + "{http.request.scheme}://application.localhost" + ] + }, + "status_code": 302 + } + ] + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_buffers.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_buffers.txt new file mode 100644 index 0000000..3178994 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_buffers.txt @@ -0,0 +1,58 @@ +https://example.com { + reverse_proxy https://localhost:54321 { + request_buffers unlimited + response_buffers unlimited + } +} + +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "example.com" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "request_buffers": -1, + "response_buffers": -1, + "transport": { + "protocol": "http", + "tls": {} + }, + "upstreams": [ + { + "dial": "localhost:54321" + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +} diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_dynamic_upstreams.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_dynamic_upstreams.txt index 2f2cbcd..384cc05 100644 --- a/caddytest/integration/caddyfile_adapt/reverse_proxy_dynamic_upstreams.txt +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_dynamic_upstreams.txt @@ -11,6 +11,7 @@ resolvers 8.8.8.8 8.8.4.4 dial_timeout 2s dial_fallback_delay 300ms + versions ipv6 } } } @@ -66,7 +67,10 @@ "8.8.4.4" ] }, - "source": "a" + "source": "a", + "versions": { + "ipv6": true + } }, "handler": "reverse_proxy" } @@ -113,4 +117,4 @@ } } } -}
\ No newline at end of file +} diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_health_headers.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_health_headers.txt index 17adcaa..800c11f 100644 --- a/caddytest/integration/caddyfile_adapt/reverse_proxy_health_headers.txt +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_health_headers.txt @@ -6,6 +6,9 @@ reverse_proxy 127.0.0.1:65535 { X-Header-Key 95ca39e3cbe7 X-Header-Keys VbG4NZwWnipo 335Q9/MhqcNU3s2TO X-Empty-Value + Same-Key 1 + Same-Key 2 + X-System-Hostname {system.hostname} } health_uri /health } @@ -29,6 +32,10 @@ reverse_proxy 127.0.0.1:65535 { "Host": [ "example.com" ], + "Same-Key": [ + "1", + "2" + ], "X-Empty-Value": [ "" ], @@ -38,6 +45,9 @@ reverse_proxy 127.0.0.1:65535 { "X-Header-Keys": [ "VbG4NZwWnipo", "335Q9/MhqcNU3s2TO" + ], + "X-System-Hostname": [ + "{system.hostname}" ] }, "uri": "/health" diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_load_balance_wrr.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_load_balance_wrr.txt new file mode 100644 index 0000000..d41c4b8 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_load_balance_wrr.txt @@ -0,0 +1,71 @@ +:8884 + +reverse_proxy 127.0.0.1:65535 127.0.0.1:35535 { + lb_policy weighted_round_robin 10 1 + lb_retries 5 + lb_try_duration 10s + lb_try_interval 500ms + lb_retry_match { + path /foo* + method POST + } + lb_retry_match path /bar* +} +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8884" + ], + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "load_balancing": { + "retries": 5, + "retry_match": [ + { + "method": [ + "POST" + ], + "path": [ + "/foo*" + ] + }, + { + "path": [ + "/bar*" + ] + } + ], + "selection_policy": { + "policy": "weighted_round_robin", + "weights": [ + 10, + 1 + ] + }, + "try_duration": 10000000000, + "try_interval": 500000000 + }, + "upstreams": [ + { + "dial": "127.0.0.1:65535" + }, + { + "dial": "127.0.0.1:35535" + } + ] + } + ] + } + ] + } + } + } + } +} diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_options.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_options.txt index b22333a..f6420ca 100644 --- a/caddytest/integration/caddyfile_adapt/reverse_proxy_options.txt +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_options.txt @@ -6,7 +6,7 @@ https://example.com { method GET rewrite /rewritten?uri={uri} - buffer_requests + request_buffers 4KB transport http { read_buffer 10MB @@ -54,7 +54,6 @@ https://example.com { { "handle": [ { - "buffer_requests": true, "handler": "reverse_proxy", "headers": { "request": { @@ -68,6 +67,7 @@ https://example.com { } } }, + "request_buffers": 4000, "rewrite": { "method": "GET", "uri": "/rewritten?uri={http.request.uri}" diff --git a/caddytest/integration/caddyfile_adapt/reverse_proxy_port_range.txt b/caddytest/integration/caddyfile_adapt/reverse_proxy_port_range.txt new file mode 100644 index 0000000..978d8c9 --- /dev/null +++ b/caddytest/integration/caddyfile_adapt/reverse_proxy_port_range.txt @@ -0,0 +1,67 @@ +:8884 { + # Port range + reverse_proxy localhost:8001-8002 + + # Port range with placeholder + reverse_proxy {host}:8001-8002 + + # Port range with scheme + reverse_proxy https://localhost:8001-8002 +} +---------- +{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":8884" + ], + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "localhost:8001" + }, + { + "dial": "localhost:8002" + } + ] + }, + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "{http.request.host}:8001" + }, + { + "dial": "{http.request.host}:8002" + } + ] + }, + { + "handler": "reverse_proxy", + "transport": { + "protocol": "http", + "tls": {} + }, + "upstreams": [ + { + "dial": "localhost:8001" + }, + { + "dial": "localhost:8002" + } + ] + } + ] + } + ] + } + } + } + } +}
\ No newline at end of file diff --git a/caddytest/integration/caddyfile_adapt/sort_directives_within_handle.txt b/caddytest/integration/caddyfile_adapt/sort_directives_within_handle.txt index 0cf9d88..ac0d53c 100644 --- a/caddytest/integration/caddyfile_adapt/sort_directives_within_handle.txt +++ b/caddytest/integration/caddyfile_adapt/sort_directives_within_handle.txt @@ -1,12 +1,15 @@ *.example.com { @foo host foo.example.com handle @foo { - handle_path /strip* { + handle_path /strip { respond "this should be first" } - handle { + handle_path /strip* { respond "this should be second" } + handle { + respond "this should be last" + } } handle { respond "this should be last" @@ -35,13 +38,13 @@ "handler": "subroute", "routes": [ { - "group": "group5", + "group": "group6", "handle": [ { "handler": "subroute", "routes": [ { - "group": "group2", + "group": "group3", "handle": [ { "handler": "subroute", @@ -68,13 +71,13 @@ "match": [ { "path": [ - "/strip*" + "/strip" ] } ] }, { - "group": "group2", + "group": "group3", "handle": [ { "handler": "subroute", @@ -82,6 +85,14 @@ { "handle": [ { + "handler": "rewrite", + "strip_path_prefix": "/strip" + } + ] + }, + { + "handle": [ + { "body": "this should be second", "handler": "static_response" } @@ -89,6 +100,31 @@ } ] } + ], + "match": [ + { + "path": [ + "/strip*" + ] + } + ] + }, + { + "group": "group3", + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "body": "this should be last", + "handler": "static_response" + } + ] + } + ] + } ] } ] @@ -103,7 +139,7 @@ ] }, { - "group": "group5", + "group": "group6", "handle": [ { "handler": "subroute", diff --git a/caddytest/integration/caddyfile_adapt/sort_vars_in_reverse.txt b/caddytest/integration/caddyfile_adapt/sort_vars_in_reverse.txt index dff75e1..38a912f 100644 --- a/caddytest/integration/caddyfile_adapt/sort_vars_in_reverse.txt +++ b/caddytest/integration/caddyfile_adapt/sort_vars_in_reverse.txt @@ -1,7 +1,8 @@ :80 vars /foobar foo last -vars /foo foo middle +vars /foo foo middle-last +vars /foo* foo middle-first vars * foo first ---------- { @@ -25,13 +26,28 @@ vars * foo first "match": [ { "path": [ + "/foo*" + ] + } + ], + "handle": [ + { + "foo": "middle-first", + "handler": "vars" + } + ] + }, + { + "match": [ + { + "path": [ "/foo" ] } ], "handle": [ { - "foo": "middle", + "foo": "middle-last", "handler": "vars" } ] diff --git a/caddytest/integration/caddyfile_test.go b/caddytest/integration/caddyfile_test.go index 3f3ba0a..205bc5b 100644 --- a/caddytest/integration/caddyfile_test.go +++ b/caddytest/integration/caddyfile_test.go @@ -135,3 +135,352 @@ func TestReplIndex(t *testing.T) { // act and assert tester.AssertGetResponse("http://localhost:9080/", 200, "") } + +func TestInvalidPrefix(t *testing.T) { + type testCase struct { + config, expectedError string + } + + failureCases := []testCase{ + { + config: `wss://localhost`, + expectedError: `the scheme wss:// is only supported in browsers; use https:// instead`, + }, + { + config: `ws://localhost`, + expectedError: `the scheme ws:// is only supported in browsers; use http:// instead`, + }, + { + config: `someInvalidPrefix://localhost`, + expectedError: "unsupported URL scheme someinvalidprefix://", + }, + { + config: `h2c://localhost`, + expectedError: `unsupported URL scheme h2c://`, + }, + { + config: `localhost, wss://localhost`, + expectedError: `the scheme wss:// is only supported in browsers; use https:// instead`, + }, + { + config: `localhost { + reverse_proxy ws://localhost" + }`, + expectedError: `the scheme ws:// is only supported in browsers; use http:// instead`, + }, + { + config: `localhost { + reverse_proxy someInvalidPrefix://localhost" + }`, + expectedError: `unsupported URL scheme someinvalidprefix://`, + }, + } + + for _, failureCase := range failureCases { + caddytest.AssertLoadError(t, failureCase.config, "caddyfile", failureCase.expectedError) + } +} + +func TestValidPrefix(t *testing.T) { + type testCase struct { + rawConfig, expectedResponse string + } + + successCases := []testCase{ + { + "localhost", + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + "https://localhost", + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + "http://localhost", + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":80" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + `localhost { + reverse_proxy http://localhost:3000 + }`, + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "localhost:3000" + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + `localhost { + reverse_proxy https://localhost:3000 + }`, + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "transport": { + "protocol": "http", + "tls": {} + }, + "upstreams": [ + { + "dial": "localhost:3000" + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + `localhost { + reverse_proxy h2c://localhost:3000 + }`, + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "transport": { + "protocol": "http", + "versions": [ + "h2c", + "2" + ] + }, + "upstreams": [ + { + "dial": "localhost:3000" + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + { + `localhost { + reverse_proxy localhost:3000 + }`, + `{ + "apps": { + "http": { + "servers": { + "srv0": { + "listen": [ + ":443" + ], + "routes": [ + { + "match": [ + { + "host": [ + "localhost" + ] + } + ], + "handle": [ + { + "handler": "subroute", + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "localhost:3000" + } + ] + } + ] + } + ] + } + ], + "terminal": true + } + ] + } + } + } + } +}`, + }, + } + + for _, successCase := range successCases { + caddytest.AssertAdapt(t, successCase.rawConfig, "caddyfile", successCase.expectedResponse) + } +} diff --git a/caddytest/integration/reverseproxy_test.go b/caddytest/integration/reverseproxy_test.go index f7b1967..4f4261b 100644 --- a/caddytest/integration/reverseproxy_test.go +++ b/caddytest/integration/reverseproxy_test.go @@ -22,80 +22,38 @@ func TestSRVReverseProxy(t *testing.T) { }, "apps": { "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false + "certificate_authorities": { + "local": { + "install_trust": false } } }, - "http": { - "grace_period": 1, - "servers": { - "srv0": { - "listen": [ - ":8080" - ], - "routes": [ - { - "handle": [ - { - "handler": "reverse_proxy", - "upstreams": [ - { - "lookup_srv": "srv.host.service.consul" - } + "http": { + "grace_period": 1, + "servers": { + "srv0": { + "listen": [ + ":18080" + ], + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "dynamic_upstreams": { + "source": "srv", + "name": "srv.host.service.consul" + } + } + ] + } ] - } - ] - } - ] - } - } - } - } - } - `, "json") -} - -func TestSRVWithDial(t *testing.T) { - caddytest.AssertLoadError(t, ` - { - "apps": { - "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false - } + } } - }, - "http": { - "grace_period": 1, - "servers": { - "srv0": { - "listen": [ - ":8080" - ], - "routes": [ - { - "handle": [ - { - "handler": "reverse_proxy", - "upstreams": [ - { - "dial": "tcp/address.to.upstream:80", - "lookup_srv": "srv.host.service.consul" - } - ] - } - ] - } - ] - } } - } } - } - `, "json", `upstream: specifying dial address is incompatible with lookup_srv: 0: {\"dial\": \"tcp/address.to.upstream:80\", \"lookup_srv\": \"srv.host.service.consul\"}`) + } + `, "json") } func TestDialWithPlaceholderUnix(t *testing.T) { @@ -138,41 +96,41 @@ func TestDialWithPlaceholderUnix(t *testing.T) { }, "apps": { "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false - } + "certificate_authorities": { + "local": { + "install_trust": false + } } - }, - "http": { - "grace_period": 1, - "servers": { - "srv0": { - "listen": [ - ":8080" - ], - "routes": [ - { - "handle": [ - { - "handler": "reverse_proxy", - "upstreams": [ - { - "dial": "unix/{http.request.header.X-Caddy-Upstream-Dial}" - } + }, + "http": { + "grace_period": 1, + "servers": { + "srv0": { + "listen": [ + ":18080" + ], + "routes": [ + { + "handle": [ + { + "handler": "reverse_proxy", + "upstreams": [ + { + "dial": "unix/{http.request.header.X-Caddy-Upstream-Dial}" + } + ] + } + ] + } ] - } - ] - } - ] - } - } - } + } + } + } } - } + } `, "json") - req, err := http.NewRequest(http.MethodGet, "http://localhost:8080", nil) + req, err := http.NewRequest(http.MethodGet, "http://localhost:18080", nil) if err != nil { t.Fail() return @@ -190,18 +148,18 @@ func TestReverseProxyWithPlaceholderDialAddress(t *testing.T) { }, "apps": { "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false - } + "certificate_authorities": { + "local": { + "install_trust": false + } } - }, + }, "http": { "grace_period": 1, "servers": { "srv0": { "listen": [ - ":8080" + ":18080" ], "routes": [ { @@ -264,14 +222,14 @@ func TestReverseProxyWithPlaceholderDialAddress(t *testing.T) { } } } - `, "json") + `, "json") req, err := http.NewRequest(http.MethodGet, "http://localhost:9080", nil) if err != nil { t.Fail() return } - req.Header.Set("X-Caddy-Upstream-Dial", "localhost:8080") + req.Header.Set("X-Caddy-Upstream-Dial", "localhost:18080") tester.AssertResponse(req, 200, "Hello, World!") } @@ -284,18 +242,18 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) { }, "apps": { "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false - } + "certificate_authorities": { + "local": { + "install_trust": false + } } - }, + }, "http": { "grace_period": 1, "servers": { "srv0": { "listen": [ - ":8080" + ":18080" ], "routes": [ { @@ -340,7 +298,7 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) { "handler": "reverse_proxy", "upstreams": [ { - "dial": "tcp/{http.request.header.X-Caddy-Upstream-Dial}:8080" + "dial": "tcp/{http.request.header.X-Caddy-Upstream-Dial}:18080" } ] } @@ -358,7 +316,7 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) { } } } - `, "json") + `, "json") req, err := http.NewRequest(http.MethodGet, "http://localhost:9080", nil) if err != nil { @@ -369,51 +327,6 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) { tester.AssertResponse(req, 200, "Hello, World!") } -func TestSRVWithActiveHealthcheck(t *testing.T) { - caddytest.AssertLoadError(t, ` - { - "apps": { - "pki": { - "certificate_authorities" : { - "local" : { - "install_trust": false - } - } - }, - "http": { - "grace_period": 1, - "servers": { - "srv0": { - "listen": [ - ":8080" - ], - "routes": [ - { - "handle": [ - { - "handler": "reverse_proxy", - "health_checks": { - "active": { - "path": "/ok" - } - }, - "upstreams": [ - { - "lookup_srv": "srv.host.service.consul" - } - ] - } - ] - } - ] - } - } - } - } - } - `, "json", `upstream: lookup_srv is incompatible with active health checks: 0: {\"dial\": \"\", \"lookup_srv\": \"srv.host.service.consul\"}`) -} - func TestReverseProxyHealthCheck(t *testing.T) { tester := caddytest.NewTester(t) tester.InitServer(` @@ -440,7 +353,7 @@ func TestReverseProxyHealthCheck(t *testing.T) { health_timeout 100ms } } - `, "caddyfile") + `, "caddyfile") time.Sleep(100 * time.Millisecond) // TODO: for some reason this test seems particularly flaky, getting 503 when it should be 200, unless we wait tester.AssertGetResponse("http://localhost:9080/", 200, "Hello, World!") diff --git a/caddytest/integration/stream_test.go b/caddytest/integration/stream_test.go index 09d4f64..6bc612d 100644 --- a/caddytest/integration/stream_test.go +++ b/caddytest/integration/stream_test.go @@ -176,9 +176,7 @@ func testH2ToH2CStreamServeH2C(t *testing.T) *http.Server { w.Header().Set("Cache-Control", "no-store") w.WriteHeader(200) - if f, ok := w.(http.Flusher); ok { - f.Flush() - } + http.NewResponseController(w).Flush() buf := make([]byte, 4*1024) diff --git a/caddytest/integration/testdata/import_respond.txt b/caddytest/integration/testdata/import_respond.txt index 0907fbe..4513088 100644 --- a/caddytest/integration/testdata/import_respond.txt +++ b/caddytest/integration/testdata/import_respond.txt @@ -1 +1 @@ -respond "'I am {args.0}', hears {args.1}"
\ No newline at end of file +respond "'I am {args[0]}', hears {args[1]}"
\ No newline at end of file diff --git a/cmd/cobra.go b/cmd/cobra.go index 203b7bd..c071f4a 100644 --- a/cmd/cobra.go +++ b/cmd/cobra.go @@ -1,7 +1,11 @@ package caddycmd import ( + "fmt" + "github.com/spf13/cobra" + + "github.com/caddyserver/caddy/v2" ) var rootCmd = &cobra.Command{ @@ -95,26 +99,59 @@ https://caddyserver.com/docs/running // kind of annoying to have all the help text printed out if // caddy has an error provisioning its modules, for instance... SilenceUsage: true, + Version: onlyVersionText(), } const fullDocsFooter = `Full documentation is available at: https://caddyserver.com/docs/command-line` func init() { + rootCmd.SetVersionTemplate("{{.Version}}") rootCmd.SetHelpTemplate(rootCmd.HelpTemplate() + "\n" + fullDocsFooter + "\n") } +func onlyVersionText() string { + _, f := caddy.Version() + return f +} + func caddyCmdToCobra(caddyCmd Command) *cobra.Command { cmd := &cobra.Command{ Use: caddyCmd.Name, Short: caddyCmd.Short, Long: caddyCmd.Long, - RunE: func(cmd *cobra.Command, _ []string) error { - fls := cmd.Flags() - _, err := caddyCmd.Func(Flags{fls}) - return err - }, } - cmd.Flags().AddGoFlagSet(caddyCmd.Flags) + if caddyCmd.CobraFunc != nil { + caddyCmd.CobraFunc(cmd) + } else { + cmd.RunE = WrapCommandFuncForCobra(caddyCmd.Func) + cmd.Flags().AddGoFlagSet(caddyCmd.Flags) + } return cmd } + +// WrapCommandFuncForCobra wraps a Caddy CommandFunc for use +// in a cobra command's RunE field. +func WrapCommandFuncForCobra(f CommandFunc) func(cmd *cobra.Command, _ []string) error { + return func(cmd *cobra.Command, _ []string) error { + status, err := f(Flags{cmd.Flags()}) + if status > 1 { + cmd.SilenceErrors = true + return &exitError{ExitCode: status, Err: err} + } + return err + } +} + +// exitError carries the exit code from CommandFunc to Main() +type exitError struct { + ExitCode int + Err error +} + +func (e *exitError) Error() string { + if e.Err == nil { + return fmt.Sprintf("exiting with status %d", e.ExitCode) + } + return e.Err.Error() +} diff --git a/cmd/commandfuncs.go b/cmd/commandfuncs.go index 09accd0..b0c576a 100644 --- a/cmd/commandfuncs.go +++ b/cmd/commandfuncs.go @@ -22,6 +22,7 @@ import ( "errors" "fmt" "io" + "io/fs" "log" "net" "net/http" @@ -32,18 +33,27 @@ import ( "strings" "github.com/aryann/difflib" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" - "go.uber.org/zap" + "github.com/caddyserver/caddy/v2/internal" ) func cmdStart(fl Flags) (int, error) { - startCmdConfigFlag := fl.String("config") - startCmdConfigAdapterFlag := fl.String("adapter") - startCmdPidfileFlag := fl.String("pidfile") - startCmdWatchFlag := fl.Bool("watch") - startCmdEnvfileFlag := fl.String("envfile") + configFlag := fl.String("config") + configAdapterFlag := fl.String("adapter") + pidfileFlag := fl.String("pidfile") + watchFlag := fl.Bool("watch") + + var err error + var envfileFlag []string + envfileFlag, err = fl.GetStringSlice("envfile") + if err != nil { + return caddy.ExitCodeFailedStartup, + fmt.Errorf("reading envfile flag: %v", err) + } // open a listener to which the child process will connect when // it is ready to confirm that it has successfully started @@ -64,22 +74,23 @@ func cmdStart(fl Flags) (int, error) { // sure by giving it some random bytes and having it echo // them back to us) cmd := exec.Command(os.Args[0], "run", "--pingback", ln.Addr().String()) - if startCmdConfigFlag != "" { - cmd.Args = append(cmd.Args, "--config", startCmdConfigFlag) + if configFlag != "" { + cmd.Args = append(cmd.Args, "--config", configFlag) } - if startCmdEnvfileFlag != "" { - cmd.Args = append(cmd.Args, "--envfile", startCmdEnvfileFlag) + + for _, envfile := range envfileFlag { + cmd.Args = append(cmd.Args, "--envfile", envfile) } - if startCmdConfigAdapterFlag != "" { - cmd.Args = append(cmd.Args, "--adapter", startCmdConfigAdapterFlag) + if configAdapterFlag != "" { + cmd.Args = append(cmd.Args, "--adapter", configAdapterFlag) } - if startCmdWatchFlag { + if watchFlag { cmd.Args = append(cmd.Args, "--watch") } - if startCmdPidfileFlag != "" { - cmd.Args = append(cmd.Args, "--pidfile", startCmdPidfileFlag) + if pidfileFlag != "" { + cmd.Args = append(cmd.Args, "--pidfile", pidfileFlag) } - stdinpipe, err := cmd.StdinPipe() + stdinPipe, err := cmd.StdinPipe() if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("creating stdin pipe: %v", err) @@ -91,7 +102,8 @@ func cmdStart(fl Flags) (int, error) { expect := make([]byte, 32) _, err = rand.Read(expect) if err != nil { - return caddy.ExitCodeFailedStartup, fmt.Errorf("generating random confirmation bytes: %v", err) + return caddy.ExitCodeFailedStartup, + fmt.Errorf("generating random confirmation bytes: %v", err) } // begin writing the confirmation bytes to the child's @@ -99,14 +111,15 @@ func cmdStart(fl Flags) (int, error) { // started yet, and writing synchronously would result // in a deadlock go func() { - _, _ = stdinpipe.Write(expect) - stdinpipe.Close() + _, _ = stdinPipe.Write(expect) + stdinPipe.Close() }() // start the process err = cmd.Start() if err != nil { - return caddy.ExitCodeFailedStartup, fmt.Errorf("starting caddy process: %v", err) + return caddy.ExitCodeFailedStartup, + fmt.Errorf("starting caddy process: %v", err) } // there are two ways we know we're done: either @@ -154,41 +167,37 @@ func cmdStart(fl Flags) (int, error) { func cmdRun(fl Flags) (int, error) { caddy.TrapSignals() - runCmdConfigFlag := fl.String("config") - runCmdConfigAdapterFlag := fl.String("adapter") - runCmdResumeFlag := fl.Bool("resume") - runCmdLoadEnvfileFlag := fl.String("envfile") - runCmdPrintEnvFlag := fl.Bool("environ") - runCmdWatchFlag := fl.Bool("watch") - runCmdPidfileFlag := fl.String("pidfile") - runCmdPingbackFlag := fl.String("pingback") + configFlag := fl.String("config") + configAdapterFlag := fl.String("adapter") + resumeFlag := fl.Bool("resume") + printEnvFlag := fl.Bool("environ") + watchFlag := fl.Bool("watch") + pidfileFlag := fl.String("pidfile") + pingbackFlag := fl.String("pingback") // load all additional envs as soon as possible - if runCmdLoadEnvfileFlag != "" { - if err := loadEnvFromFile(runCmdLoadEnvfileFlag); err != nil { - return caddy.ExitCodeFailedStartup, - fmt.Errorf("loading additional environment variables: %v", err) - } + err := handleEnvFileFlag(fl) + if err != nil { + return caddy.ExitCodeFailedStartup, err } // if we are supposed to print the environment, do that first - if runCmdPrintEnvFlag { + if printEnvFlag { printEnvironment() } // load the config, depending on flags var config []byte - var err error - if runCmdResumeFlag { + if resumeFlag { config, err = os.ReadFile(caddy.ConfigAutosavePath) if os.IsNotExist(err) { // not a bad error; just can't resume if autosave file doesn't exist caddy.Log().Info("no autosave file exists", zap.String("autosave_file", caddy.ConfigAutosavePath)) - runCmdResumeFlag = false + resumeFlag = false } else if err != nil { return caddy.ExitCodeFailedStartup, err } else { - if runCmdConfigFlag == "" { + if configFlag == "" { caddy.Log().Info("resuming from last configuration", zap.String("autosave_file", caddy.ConfigAutosavePath)) } else { @@ -201,13 +210,23 @@ func cmdRun(fl Flags) (int, error) { } // we don't use 'else' here since this value might have been changed in 'if' block; i.e. not mutually exclusive var configFile string - if !runCmdResumeFlag { - config, configFile, err = LoadConfig(runCmdConfigFlag, runCmdConfigAdapterFlag) + if !resumeFlag { + config, configFile, err = LoadConfig(configFlag, configAdapterFlag) if err != nil { return caddy.ExitCodeFailedStartup, err } } + // create pidfile now, in case loading config takes a while (issue #5477) + if pidfileFlag != "" { + err := caddy.PIDFile(pidfileFlag) + if err != nil { + caddy.Log().Error("unable to write PID file", + zap.String("pidfile", pidfileFlag), + zap.Error(err)) + } + } + // run the initial config err = caddy.Load(config, true) if err != nil { @@ -217,13 +236,13 @@ func cmdRun(fl Flags) (int, error) { // if we are to report to another process the successful start // of the server, do so now by echoing back contents of stdin - if runCmdPingbackFlag != "" { + if pingbackFlag != "" { confirmationBytes, err := io.ReadAll(os.Stdin) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("reading confirmation bytes from stdin: %v", err) } - conn, err := net.Dial("tcp", runCmdPingbackFlag) + conn, err := net.Dial("tcp", pingbackFlag) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("dialing confirmation address: %v", err) @@ -232,24 +251,14 @@ func cmdRun(fl Flags) (int, error) { _, err = conn.Write(confirmationBytes) if err != nil { return caddy.ExitCodeFailedStartup, - fmt.Errorf("writing confirmation bytes to %s: %v", runCmdPingbackFlag, err) + fmt.Errorf("writing confirmation bytes to %s: %v", pingbackFlag, err) } } // if enabled, reload config file automatically on changes // (this better only be used in dev!) - if runCmdWatchFlag { - go watchConfigFile(configFile, runCmdConfigAdapterFlag) - } - - // create pidfile - if runCmdPidfileFlag != "" { - err := caddy.PIDFile(runCmdPidfileFlag) - if err != nil { - caddy.Log().Error("unable to write PID file", - zap.String("pidfile", runCmdPidfileFlag), - zap.Error(err)) - } + if watchFlag { + go watchConfigFile(configFile, configAdapterFlag) } // warn if the environment does not provide enough information about the disk @@ -275,11 +284,11 @@ func cmdRun(fl Flags) (int, error) { } func cmdStop(fl Flags) (int, error) { - addrFlag := fl.String("address") + addressFlag := fl.String("address") configFlag := fl.String("config") configAdapterFlag := fl.String("adapter") - adminAddr, err := DetermineAdminAPIAddress(addrFlag, nil, configFlag, configAdapterFlag) + adminAddr, err := DetermineAdminAPIAddress(addressFlag, nil, configFlag, configAdapterFlag) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("couldn't determine admin API address: %v", err) } @@ -297,7 +306,7 @@ func cmdStop(fl Flags) (int, error) { func cmdReload(fl Flags) (int, error) { configFlag := fl.String("config") configAdapterFlag := fl.String("adapter") - addrFlag := fl.String("address") + addressFlag := fl.String("address") forceFlag := fl.Bool("force") // get the config in caddy's native format @@ -309,7 +318,7 @@ func cmdReload(fl Flags) (int, error) { return caddy.ExitCodeFailedStartup, fmt.Errorf("no config file to load") } - adminAddr, err := DetermineAdminAPIAddress(addrFlag, config, configFlag, configAdapterFlag) + adminAddr, err := DetermineAdminAPIAddress(addressFlag, config, configFlag, configAdapterFlag) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("couldn't determine admin API address: %v", err) } @@ -411,60 +420,60 @@ func cmdListModules(fl Flags) (int, error) { return caddy.ExitCodeSuccess, nil } -func cmdEnviron(_ Flags) (int, error) { +func cmdEnviron(fl Flags) (int, error) { + // load all additional envs as soon as possible + err := handleEnvFileFlag(fl) + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + printEnvironment() return caddy.ExitCodeSuccess, nil } func cmdAdaptConfig(fl Flags) (int, error) { - adaptCmdInputFlag := fl.String("config") - adaptCmdAdapterFlag := fl.String("adapter") - adaptCmdPrettyFlag := fl.Bool("pretty") - adaptCmdValidateFlag := fl.Bool("validate") - - // if no input file was specified, try a default - // Caddyfile if the Caddyfile adapter is plugged in - if adaptCmdInputFlag == "" && caddyconfig.GetAdapter("caddyfile") != nil { - _, err := os.Stat("Caddyfile") - if err == nil { - // default Caddyfile exists - adaptCmdInputFlag = "Caddyfile" - caddy.Log().Info("using adjacent Caddyfile") - } else if !os.IsNotExist(err) { - // default Caddyfile exists, but error accessing it - return caddy.ExitCodeFailedStartup, fmt.Errorf("accessing default Caddyfile: %v", err) - } + inputFlag := fl.String("config") + adapterFlag := fl.String("adapter") + prettyFlag := fl.Bool("pretty") + validateFlag := fl.Bool("validate") + + var err error + inputFlag, err = configFileWithRespectToDefault(caddy.Log(), inputFlag) + if err != nil { + return caddy.ExitCodeFailedStartup, err } - if adaptCmdInputFlag == "" { - return caddy.ExitCodeFailedStartup, - fmt.Errorf("input file required when there is no Caddyfile in current directory (use --config flag)") + // load all additional envs as soon as possible + err = handleEnvFileFlag(fl) + if err != nil { + return caddy.ExitCodeFailedStartup, err } - if adaptCmdAdapterFlag == "" { + + if adapterFlag == "" { return caddy.ExitCodeFailedStartup, fmt.Errorf("adapter name is required (use --adapt flag or leave unspecified for default)") } - cfgAdapter := caddyconfig.GetAdapter(adaptCmdAdapterFlag) + cfgAdapter := caddyconfig.GetAdapter(adapterFlag) if cfgAdapter == nil { return caddy.ExitCodeFailedStartup, - fmt.Errorf("unrecognized config adapter: %s", adaptCmdAdapterFlag) + fmt.Errorf("unrecognized config adapter: %s", adapterFlag) } - input, err := os.ReadFile(adaptCmdInputFlag) + input, err := os.ReadFile(inputFlag) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("reading input file: %v", err) } - opts := map[string]any{"filename": adaptCmdInputFlag} + opts := map[string]any{"filename": inputFlag} adaptedConfig, warnings, err := cfgAdapter.Adapt(input, opts) if err != nil { return caddy.ExitCodeFailedStartup, err } - if adaptCmdPrettyFlag { + if prettyFlag { var prettyBuf bytes.Buffer err = json.Indent(&prettyBuf, adaptedConfig, "", "\t") if err != nil { @@ -482,15 +491,15 @@ func cmdAdaptConfig(fl Flags) (int, error) { if warn.Directive != "" { msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message) } - caddy.Log().Named(adaptCmdAdapterFlag).Warn(msg, + caddy.Log().Named(adapterFlag).Warn(msg, zap.String("file", warn.File), zap.Int("line", warn.Line)) } // validate output if requested - if adaptCmdValidateFlag { + if validateFlag { var cfg *caddy.Config - err = json.Unmarshal(adaptedConfig, &cfg) + err = caddy.StrictUnmarshalJSON(adaptedConfig, &cfg) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err) } @@ -504,26 +513,33 @@ func cmdAdaptConfig(fl Flags) (int, error) { } func cmdValidateConfig(fl Flags) (int, error) { - validateCmdConfigFlag := fl.String("config") - validateCmdAdapterFlag := fl.String("adapter") - runCmdLoadEnvfileFlag := fl.String("envfile") + configFlag := fl.String("config") + adapterFlag := fl.String("adapter") // load all additional envs as soon as possible - if runCmdLoadEnvfileFlag != "" { - if err := loadEnvFromFile(runCmdLoadEnvfileFlag); err != nil { - return caddy.ExitCodeFailedStartup, - fmt.Errorf("loading additional environment variables: %v", err) - } + err := handleEnvFileFlag(fl) + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + + // use default config and ensure a config file is specified + configFlag, err = configFileWithRespectToDefault(caddy.Log(), configFlag) + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + if configFlag == "" { + return caddy.ExitCodeFailedStartup, + fmt.Errorf("input file required when there is no Caddyfile in current directory (use --config flag)") } - input, _, err := LoadConfig(validateCmdConfigFlag, validateCmdAdapterFlag) + input, _, err := LoadConfig(configFlag, adapterFlag) if err != nil { return caddy.ExitCodeFailedStartup, err } input = caddy.RemoveMetaFields(input) var cfg *caddy.Config - err = json.Unmarshal(input, &cfg) + err = caddy.StrictUnmarshalJSON(input, &cfg) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err) } @@ -539,13 +555,13 @@ func cmdValidateConfig(fl Flags) (int, error) { } func cmdFmt(fl Flags) (int, error) { - formatCmdConfigFile := fl.Arg(0) - if formatCmdConfigFile == "" { - formatCmdConfigFile = "Caddyfile" + configFile := fl.Arg(0) + if configFile == "" { + configFile = "Caddyfile" } // as a special case, read from stdin if the file name is "-" - if formatCmdConfigFile == "-" { + if configFile == "-" { input, err := io.ReadAll(os.Stdin) if err != nil { return caddy.ExitCodeFailedStartup, @@ -555,7 +571,7 @@ func cmdFmt(fl Flags) (int, error) { return caddy.ExitCodeSuccess, nil } - input, err := os.ReadFile(formatCmdConfigFile) + input, err := os.ReadFile(configFile) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("reading input file: %v", err) @@ -564,7 +580,7 @@ func cmdFmt(fl Flags) (int, error) { output := caddyfile.Format(input) if fl.Bool("overwrite") { - if err := os.WriteFile(formatCmdConfigFile, output, 0600); err != nil { + if err := os.WriteFile(configFile, output, 0o600); err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("overwriting formatted file: %v", err) } return caddy.ExitCodeSuccess, nil @@ -588,13 +604,35 @@ func cmdFmt(fl Flags) (int, error) { fmt.Print(string(output)) } - if warning, diff := caddyfile.FormattingDifference(formatCmdConfigFile, input); diff { - return caddy.ExitCodeFailedStartup, fmt.Errorf("%s:%d: Caddyfile input is not formatted", warning.File, warning.Line) + if warning, diff := caddyfile.FormattingDifference(configFile, input); diff { + return caddy.ExitCodeFailedStartup, fmt.Errorf(`%s:%d: Caddyfile input is not formatted; Tip: use '--overwrite' to update your Caddyfile in-place instead of previewing it. Consult '--help' for more options`, + warning.File, + warning.Line, + ) } return caddy.ExitCodeSuccess, nil } +// handleEnvFileFlag loads the environment variables from the given --envfile +// flag if specified. This should be called as early in the command function. +func handleEnvFileFlag(fl Flags) error { + var err error + var envfileFlag []string + envfileFlag, err = fl.GetStringSlice("envfile") + if err != nil { + return fmt.Errorf("reading envfile flag: %v", err) + } + + for _, envfile := range envfileFlag { + if err := loadEnvFromFile(envfile); err != nil { + return fmt.Errorf("loading additional environment variables: %v", err) + } + } + + return nil +} + // AdminAPIRequest makes an API request according to the CLI flags given, // with the given HTTP method and request URI. If body is non-nil, it will // be assumed to be Content-Type application/json. The caller should close @@ -607,7 +645,17 @@ func AdminAPIRequest(adminAddr, method, uri string, headers http.Header, body io } origin := "http://" + parsedAddr.JoinHostPort(0) if parsedAddr.IsUnixNetwork() { - origin = "http://unixsocket" // hack so that http.NewRequest() is happy + origin = "http://127.0.0.1" // bogus host is a hack so that http.NewRequest() is happy + + // the unix address at this point might still contain the optional + // unix socket permissions, which are part of the address/host. + // those need to be removed first, as they aren't part of the + // resulting unix file path + addr, _, err := internal.SplitUnixSocketPermissionsBits(parsedAddr.Host) + if err != nil { + return nil, err + } + parsedAddr.Host = addr } // form the request @@ -616,20 +664,24 @@ func AdminAPIRequest(adminAddr, method, uri string, headers http.Header, body io return nil, fmt.Errorf("making request: %v", err) } if parsedAddr.IsUnixNetwork() { - // When listening on a unix socket, the admin endpoint doesn't - // accept any Host header because there is no host:port for - // a unix socket's address. The server's host check is fairly - // strict for security reasons, so we don't allow just any - // Host header. For unix sockets, the Host header must be - // empty. Unfortunately, Go makes it impossible to make HTTP - // requests with an empty Host header... except with this one - // weird trick. (Hopefully they don't fix it. It's already - // hard enough to use HTTP over unix sockets.) + // We used to conform to RFC 2616 Section 14.26 which requires + // an empty host header when there is no host, as is the case + // with unix sockets. However, Go required a Host value so we + // used a hack of a space character as the host (it would see + // the Host was non-empty, then trim the space later). As of + // Go 1.20.6 (July 2023), this hack no longer works. See: + // https://github.com/golang/go/issues/60374 + // See also the discussion here: + // https://github.com/golang/go/issues/61431 // - // An equivalent curl command would be something like: - // $ curl --unix-socket caddy.sock http:/:$REQUEST_URI - req.URL.Host = " " - req.Host = "" + // After that, we now require a Host value of either 127.0.0.1 + // or ::1 if one is set. Above I choose to use 127.0.0.1. Even + // though the value should be completely irrelevant (it could be + // "srldkjfsd"), if for some reason the Host *is* used, at least + // we can have some reasonable assurance it will stay on the local + // machine and that browsers, if they ever allow access to unix + // sockets, can still enforce CORS, ensuring it is still coming + // from the local machine. } else { req.Header.Set("Origin", origin) } @@ -718,6 +770,31 @@ func DetermineAdminAPIAddress(address string, config []byte, configFile, configA return caddy.DefaultAdminListen, nil } +// configFileWithRespectToDefault returns the filename to use for loading the config, based +// on whether a config file is already specified and a supported default config file exists. +func configFileWithRespectToDefault(logger *zap.Logger, configFile string) (string, error) { + const defaultCaddyfile = "Caddyfile" + + // if no input file was specified, try a default Caddyfile if the Caddyfile adapter is plugged in + if configFile == "" && caddyconfig.GetAdapter("caddyfile") != nil { + _, err := os.Stat(defaultCaddyfile) + if err == nil { + // default Caddyfile exists + if logger != nil { + logger.Info("using adjacent Caddyfile") + } + return defaultCaddyfile, nil + } + if !errors.Is(err, fs.ErrNotExist) { + // problem checking + return configFile, fmt.Errorf("checking if default Caddyfile exists: %v", err) + } + } + + // default config file does not exist or is irrelevant + return configFile, nil +} + type moduleInfo struct { caddyModuleID string goModule *debug.Module diff --git a/cmd/commands.go b/cmd/commands.go index 9216b89..e5e1265 100644 --- a/cmd/commands.go +++ b/cmd/commands.go @@ -21,9 +21,10 @@ import ( "regexp" "strings" - "github.com/caddyserver/caddy/v2" "github.com/spf13/cobra" "github.com/spf13/cobra/doc" + + "github.com/caddyserver/caddy/v2" ) // Command represents a subcommand. Name, Func, @@ -34,12 +35,6 @@ type Command struct { // Required. Name string - // Func is a function that executes a subcommand using - // the parsed flags. It returns an exit code and any - // associated error. - // Required. - Func CommandFunc - // Usage is a brief message describing the syntax of // the subcommand's flags and args. Use [] to indicate // optional parameters and <> to enclose literal values @@ -60,7 +55,21 @@ type Command struct { Long string // Flags is the flagset for command. + // This is ignored if CobraFunc is set. Flags *flag.FlagSet + + // Func is a function that executes a subcommand using + // the parsed flags. It returns an exit code and any + // associated error. + // Required if CobraFunc is not set. + Func CommandFunc + + // CobraFunc allows further configuration of the command + // via cobra's APIs. If this is set, then Func and Flags + // are ignored, with the assumption that they are set in + // this function. A caddycmd.WrapCommandFuncForCobra helper + // exists to simplify porting CommandFunc to Cobra's RunE. + CobraFunc func(*cobra.Command) } // CommandFunc is a command's function. It runs the @@ -79,34 +88,32 @@ var commands = make(map[string]Command) func init() { RegisterCommand(Command{ Name: "start", - Func: cmdStart, Usage: "[--config <path> [--adapter <name>]] [--envfile <path>] [--watch] [--pidfile <file>]", Short: "Starts the Caddy process in the background and then returns", Long: ` Starts the Caddy process, optionally bootstrapped with an initial config file. This command unblocks after the server starts running or fails to run. -If --envfile is specified, an environment file with environment variables in -the KEY=VALUE format will be loaded into the Caddy process. +If --envfile is specified, an environment file with environment variables +in the KEY=VALUE format will be loaded into the Caddy process. On Windows, the spawned child process will remain attached to the terminal, so closing the window will forcefully stop Caddy; to avoid forgetting this, try -using 'caddy run' instead to keep it in the foreground.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("start", flag.ExitOnError) - fs.String("config", "", "Configuration file") - fs.String("envfile", "", "Environment file to load") - fs.String("adapter", "", "Name of config adapter to apply") - fs.String("pidfile", "", "Path of file to which to write process ID") - fs.Bool("watch", false, "Reload changed config file automatically") - return fs - }(), +using 'caddy run' instead to keep it in the foreground. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Configuration file") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply") + cmd.Flags().StringSliceP("envfile", "", []string{}, "Environment file(s) to load") + cmd.Flags().BoolP("watch", "w", false, "Reload changed config file automatically") + cmd.Flags().StringP("pidfile", "", "", "Path of file to which to write process ID") + cmd.RunE = WrapCommandFuncForCobra(cmdStart) + }, }) RegisterCommand(Command{ Name: "run", - Func: cmdRun, - Usage: "[--config <path> [--adapter <name>]] [--envfile <path>] [--environ] [--resume] [--watch] [--pidfile <fil>]", + Usage: "[--config <path> [--adapter <name>]] [--envfile <path>] [--environ] [--resume] [--watch] [--pidfile <file>]", Short: `Starts the Caddy process and blocks indefinitely`, Long: ` Starts the Caddy process, optionally bootstrapped with an initial config file, @@ -126,8 +133,8 @@ As a special case, if the current working directory has a file called that file will be loaded and used to configure Caddy, even without any command line flags. -If --envfile is specified, an environment file with environment variables in -the KEY=VALUE format will be loaded into the Caddy process. +If --envfile is specified, an environment file with environment variables +in the KEY=VALUE format will be loaded into the Caddy process. If --environ is specified, the environment as seen by the Caddy process will be printed before starting. This is the same as the environ command but does @@ -138,44 +145,42 @@ save file. It is not an error if --resume is used and no autosave file exists. If --watch is specified, the config file will be loaded automatically after changes. ⚠️ This can make unintentional config changes easier; only use this -option in a local development environment.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("run", flag.ExitOnError) - fs.String("config", "", "Configuration file") - fs.String("adapter", "", "Name of config adapter to apply") - fs.String("envfile", "", "Environment file to load") - fs.Bool("environ", false, "Print environment") - fs.Bool("resume", false, "Use saved config, if any (and prefer over --config file)") - fs.Bool("watch", false, "Watch config file for changes and reload it automatically") - fs.String("pidfile", "", "Path of file to which to write process ID") - fs.String("pingback", "", "Echo confirmation bytes to this address on success") - return fs - }(), +option in a local development environment. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Configuration file") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply") + cmd.Flags().StringSliceP("envfile", "", []string{}, "Environment file(s) to load") + cmd.Flags().BoolP("environ", "e", false, "Print environment") + cmd.Flags().BoolP("resume", "r", false, "Use saved config, if any (and prefer over --config file)") + cmd.Flags().BoolP("watch", "w", false, "Watch config file for changes and reload it automatically") + cmd.Flags().StringP("pidfile", "", "", "Path of file to which to write process ID") + cmd.Flags().StringP("pingback", "", "", "Echo confirmation bytes to this address on success") + cmd.RunE = WrapCommandFuncForCobra(cmdRun) + }, }) RegisterCommand(Command{ Name: "stop", - Func: cmdStop, - Usage: "[--address <interface>] [--config <path> [--adapter <name>]]", + Usage: "[--config <path> [--adapter <name>]] [--address <interface>]", Short: "Gracefully stops a started Caddy process", Long: ` Stops the background Caddy process as gracefully as possible. It requires that the admin API is enabled and accessible, since it will use the API's /stop endpoint. The address of this request can be customized -using the --address flag, or from the given --config, if not the default.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("stop", flag.ExitOnError) - fs.String("address", "", "The address to use to reach the admin API endpoint, if not the default") - fs.String("config", "", "Configuration file to use to parse the admin address, if --address is not used") - fs.String("adapter", "", "Name of config adapter to apply (when --config is used)") - return fs - }(), +using the --address flag, or from the given --config, if not the default. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Configuration file to use to parse the admin address, if --address is not used") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply (when --config is used)") + cmd.Flags().StringP("address", "", "", "The address to use to reach the admin API endpoint, if not the default") + cmd.RunE = WrapCommandFuncForCobra(cmdStop) + }, }) RegisterCommand(Command{ Name: "reload", - Func: cmdReload, Usage: "--config <path> [--adapter <name>] [--address <interface>]", Short: "Changes the config of the running Caddy instance", Long: ` @@ -185,20 +190,19 @@ workflows revolving around config files. Since the admin endpoint is configurable, the endpoint configuration is loaded from the --address flag if specified; otherwise it is loaded from the given -config file; otherwise the default is assumed.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("reload", flag.ExitOnError) - fs.String("config", "", "Configuration file (required)") - fs.String("adapter", "", "Name of config adapter to apply") - fs.String("address", "", "Address of the administration listener, if different from config") - fs.Bool("force", false, "Force config reload, even if it is the same") - return fs - }(), +config file; otherwise the default is assumed. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Configuration file (required)") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply") + cmd.Flags().StringP("address", "", "", "Address of the administration listener, if different from config") + cmd.Flags().BoolP("force", "f", false, "Force config reload, even if it is the same") + cmd.RunE = WrapCommandFuncForCobra(cmdReload) + }, }) RegisterCommand(Command{ Name: "version", - Func: cmdVersion, Short: "Prints the version", Long: ` Prints the version of this Caddy binary. @@ -213,31 +217,30 @@ detailed version information is printed as given by Go modules. For more details about the full version string, see the Go module documentation: https://go.dev/doc/modules/version-numbers `, + Func: cmdVersion, }) RegisterCommand(Command{ Name: "list-modules", - Func: cmdListModules, - Usage: "[--packages] [--versions]", + Usage: "[--packages] [--versions] [--skip-standard]", Short: "Lists the installed Caddy modules", - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("list-modules", flag.ExitOnError) - fs.Bool("packages", false, "Print package paths") - fs.Bool("versions", false, "Print version information") - fs.Bool("skip-standard", false, "Skip printing standard modules") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().BoolP("packages", "", false, "Print package paths") + cmd.Flags().BoolP("versions", "", false, "Print version information") + cmd.Flags().BoolP("skip-standard", "s", false, "Skip printing standard modules") + cmd.RunE = WrapCommandFuncForCobra(cmdListModules) + }, }) RegisterCommand(Command{ Name: "build-info", - Func: cmdBuildInfo, Short: "Prints information about this build", + Func: cmdBuildInfo, }) RegisterCommand(Command{ Name: "environ", - Func: cmdEnviron, + Usage: "[--envfile <path>]", Short: "Prints the environment", Long: ` Prints the environment as seen by this Caddy process. @@ -247,6 +250,9 @@ configuration uses environment variables (e.g. "{env.VARIABLE}") then this command can be useful for verifying that the variables will have the values you expect in your config. +If --envfile is specified, an environment file with environment variables +in the KEY=VALUE format will be loaded into the Caddy process. + Note that environments may be different depending on how you run Caddy. Environments for Caddy instances started by service managers such as systemd are often different than the environment inherited from your @@ -257,12 +263,15 @@ by adding the "--environ" flag. Environments may contain sensitive data. `, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringSliceP("envfile", "", []string{}, "Environment file(s) to load") + cmd.RunE = WrapCommandFuncForCobra(cmdEnviron) + }, }) RegisterCommand(Command{ Name: "adapt", - Func: cmdAdaptConfig, - Usage: "--config <path> [--adapter <name>] [--pretty] [--validate]", + Usage: "--config <path> [--adapter <name>] [--pretty] [--validate] [--envfile <path>]", Short: "Adapts a configuration to Caddy's native JSON", Long: ` Adapts a configuration to Caddy's native JSON format and writes the @@ -273,20 +282,23 @@ for human readability. If --validate is used, the adapted config will be checked for validity. If the config is invalid, an error will be printed to stderr and a non- -zero exit status will be returned.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("adapt", flag.ExitOnError) - fs.String("config", "", "Configuration file to adapt (required)") - fs.String("adapter", "caddyfile", "Name of config adapter") - fs.Bool("pretty", false, "Format the output for human readability") - fs.Bool("validate", false, "Validate the output") - return fs - }(), +zero exit status will be returned. + +If --envfile is specified, an environment file with environment variables +in the KEY=VALUE format will be loaded into the Caddy process. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Configuration file to adapt (required)") + cmd.Flags().StringP("adapter", "a", "caddyfile", "Name of config adapter") + cmd.Flags().BoolP("pretty", "p", false, "Format the output for human readability") + cmd.Flags().BoolP("validate", "", false, "Validate the output") + cmd.Flags().StringSliceP("envfile", "", []string{}, "Environment file(s) to load") + cmd.RunE = WrapCommandFuncForCobra(cmdAdaptConfig) + }, }) RegisterCommand(Command{ Name: "validate", - Func: cmdValidateConfig, Usage: "--config <path> [--adapter <name>] [--envfile <path>]", Short: "Tests whether a configuration file is valid", Long: ` @@ -294,21 +306,70 @@ Loads and provisions the provided config, but does not start running it. This reveals any errors with the configuration through the loading and provisioning stages. -If --envfile is specified, an environment file with environment variables in -the KEY=VALUE format will be loaded into the Caddy process.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("validate", flag.ExitOnError) - fs.String("config", "", "Input configuration file") - fs.String("adapter", "", "Name of config adapter") - fs.String("envfile", "", "Environment file to load") - return fs - }(), +If --envfile is specified, an environment file with environment variables +in the KEY=VALUE format will be loaded into the Caddy process. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("config", "c", "", "Input configuration file") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter") + cmd.Flags().StringSliceP("envfile", "", []string{}, "Environment file(s) to load") + cmd.RunE = WrapCommandFuncForCobra(cmdValidateConfig) + }, + }) + + RegisterCommand(Command{ + Name: "storage", + Short: "Commands for working with Caddy's storage (EXPERIMENTAL)", + Long: ` +Allows exporting and importing Caddy's storage contents. The two commands can be +combined in a pipeline to transfer directly from one storage to another: + +$ caddy storage export --config Caddyfile.old --output - | +> caddy storage import --config Caddyfile.new --input - + +The - argument refers to stdout and stdin, respectively. + +NOTE: When importing to or exporting from file_system storage (the default), the command +should be run as the user that owns the associated root path. + +EXPERIMENTAL: May be changed or removed. +`, + CobraFunc: func(cmd *cobra.Command) { + exportCmd := &cobra.Command{ + Use: "export --config <path> --output <path>", + Short: "Exports storage assets as a tarball", + Long: ` +The contents of the configured storage module (TLS certificates, etc) +are exported via a tarball. + +--output is required, - can be given for stdout. +`, + RunE: WrapCommandFuncForCobra(cmdExportStorage), + } + exportCmd.Flags().StringP("config", "c", "", "Input configuration file (required)") + exportCmd.Flags().StringP("output", "o", "", "Output path") + cmd.AddCommand(exportCmd) + + importCmd := &cobra.Command{ + Use: "import --config <path> --input <path>", + Short: "Imports storage assets from a tarball.", + Long: ` +Imports storage assets to the configured storage module. The import file must be +a tar archive. + +--input is required, - can be given for stdin. +`, + RunE: WrapCommandFuncForCobra(cmdImportStorage), + } + importCmd.Flags().StringP("config", "c", "", "Configuration file to load (required)") + importCmd.Flags().StringP("input", "i", "", "Tar of assets to load (required)") + cmd.AddCommand(importCmd) + }, }) RegisterCommand(Command{ Name: "fmt", - Func: cmdFmt, - Usage: "[--overwrite] [<path>]", + Usage: "[--overwrite] [--diff] [<path>]", Short: "Formats a Caddyfile", Long: ` Formats the Caddyfile by adding proper indentation and spaces to improve @@ -324,44 +385,41 @@ is not a valid patch format. If you wish you use stdin instead of a regular file, use - as the path. When reading from stdin, the --overwrite flag has no effect: the result -is always printed to stdout.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("fmt", flag.ExitOnError) - fs.Bool("overwrite", false, "Overwrite the input file with the results") - fs.Bool("diff", false, "Print the differences between the input file and the formatted output") - return fs - }(), +is always printed to stdout. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().BoolP("overwrite", "w", false, "Overwrite the input file with the results") + cmd.Flags().BoolP("diff", "d", false, "Print the differences between the input file and the formatted output") + cmd.RunE = WrapCommandFuncForCobra(cmdFmt) + }, }) RegisterCommand(Command{ Name: "upgrade", - Func: cmdUpgrade, Short: "Upgrade Caddy (EXPERIMENTAL)", Long: ` Downloads an updated Caddy binary with the same modules/plugins at the -latest versions. EXPERIMENTAL: May be changed or removed.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("upgrade", flag.ExitOnError) - fs.Bool("keep-backup", false, "Keep the backed up binary, instead of deleting it") - return fs - }(), +latest versions. EXPERIMENTAL: May be changed or removed. +`, + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().BoolP("keep-backup", "k", false, "Keep the backed up binary, instead of deleting it") + cmd.RunE = WrapCommandFuncForCobra(cmdUpgrade) + }, }) RegisterCommand(Command{ Name: "add-package", - Func: cmdAddPackage, Usage: "<packages...>", Short: "Adds Caddy packages (EXPERIMENTAL)", Long: ` Downloads an updated Caddy binary with the specified packages (module/plugin) -added. Retains existing packages. Returns an error if the any of packages are +added. Retains existing packages. Returns an error if the any of packages are already included. EXPERIMENTAL: May be changed or removed. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("add-package", flag.ExitOnError) - fs.Bool("keep-backup", false, "Keep the backed up binary, instead of deleting it") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().BoolP("keep-backup", "k", false, "Keep the backed up binary, instead of deleting it") + cmd.RunE = WrapCommandFuncForCobra(cmdAddPackage) + }, }) RegisterCommand(Command{ @@ -370,35 +428,18 @@ already included. EXPERIMENTAL: May be changed or removed. Usage: "<packages...>", Short: "Removes Caddy packages (EXPERIMENTAL)", Long: ` -Downloads an updated Caddy binaries without the specified packages (module/plugin). -Returns an error if any of the packages are not included. +Downloads an updated Caddy binaries without the specified packages (module/plugin). +Returns an error if any of the packages are not included. EXPERIMENTAL: May be changed or removed. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("remove-package", flag.ExitOnError) - fs.Bool("keep-backup", false, "Keep the backed up binary, instead of deleting it") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().BoolP("keep-backup", "k", false, "Keep the backed up binary, instead of deleting it") + cmd.RunE = WrapCommandFuncForCobra(cmdRemovePackage) + }, }) RegisterCommand(Command{ - Name: "manpage", - Func: func(fl Flags) (int, error) { - dir := strings.TrimSpace(fl.String("directory")) - if dir == "" { - return caddy.ExitCodeFailedQuit, fmt.Errorf("designated output directory and specified section are required") - } - if err := os.MkdirAll(dir, 0755); err != nil { - return caddy.ExitCodeFailedQuit, err - } - if err := doc.GenManTree(rootCmd, &doc.GenManHeader{ - Title: "Caddy", - Section: "8", // https://en.wikipedia.org/wiki/Man_page#Manual_sections - }, dir); err != nil { - return caddy.ExitCodeFailedQuit, err - } - return caddy.ExitCodeSuccess, nil - }, + Name: "manpage", Usage: "--directory <path>", Short: "Generates the manual pages for Caddy commands", Long: ` @@ -408,11 +449,25 @@ tagged into section 8 (System Administration). The manual page files are generated into the directory specified by the argument of --directory. If the directory does not exist, it will be created. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("manpage", flag.ExitOnError) - fs.String("directory", "", "The output directory where the manpages are generated") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("directory", "o", "", "The output directory where the manpages are generated") + cmd.RunE = WrapCommandFuncForCobra(func(fl Flags) (int, error) { + dir := strings.TrimSpace(fl.String("directory")) + if dir == "" { + return caddy.ExitCodeFailedQuit, fmt.Errorf("designated output directory and specified section are required") + } + if err := os.MkdirAll(dir, 0o755); err != nil { + return caddy.ExitCodeFailedQuit, err + } + if err := doc.GenManTree(rootCmd, &doc.GenManHeader{ + Title: "Caddy", + Section: "8", // https://en.wikipedia.org/wiki/Man_page#Manual_sections + }, dir); err != nil { + return caddy.ExitCodeFailedQuit, err + } + return caddy.ExitCodeSuccess, nil + }) + }, }) // source: https://github.com/spf13/cobra/blob/main/shell_completions.md @@ -420,40 +475,40 @@ argument of --directory. If the directory does not exist, it will be created. Use: "completion [bash|zsh|fish|powershell]", Short: "Generate completion script", Long: fmt.Sprintf(`To load completions: - + Bash: - + $ source <(%[1]s completion bash) - + # To load completions for each session, execute once: # Linux: $ %[1]s completion bash > /etc/bash_completion.d/%[1]s # macOS: $ %[1]s completion bash > $(brew --prefix)/etc/bash_completion.d/%[1]s - + Zsh: - + # If shell completion is not already enabled in your environment, # you will need to enable it. You can execute the following once: - + $ echo "autoload -U compinit; compinit" >> ~/.zshrc - + # To load completions for each session, execute once: $ %[1]s completion zsh > "${fpath[1]}/_%[1]s" - + # You will need to start a new shell for this setup to take effect. - + fish: - + $ %[1]s completion fish | source - + # To load completions for each session, execute once: $ %[1]s completion fish > ~/.config/fish/completions/%[1]s.fish - + PowerShell: - + PS> %[1]s completion powershell | Out-String | Invoke-Expression - + # To load completions for every new session, run: PS> %[1]s completion powershell > %[1]s.ps1 # and source this file from your PowerShell profile. @@ -496,7 +551,7 @@ func RegisterCommand(cmd Command) { if cmd.Name == "" { panic("command name is required") } - if cmd.Func == nil { + if cmd.Func == nil && cmd.CobraFunc == nil { panic("command function missing") } if cmd.Short == "" { diff --git a/cmd/main.go b/cmd/main.go index 17da8bd..fa15c08 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -17,6 +17,7 @@ package caddycmd import ( "bufio" "bytes" + "errors" "flag" "fmt" "io" @@ -30,11 +31,12 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/certmagic" "github.com/spf13/pflag" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig" ) func init() { @@ -62,6 +64,10 @@ func Main() { } if err := rootCmd.Execute(); err != nil { + var exitError *exitError + if errors.As(err, &exitError) { + os.Exit(exitError.ExitCode) + } os.Exit(1) } } @@ -89,6 +95,10 @@ func handlePingbackConn(conn net.Conn, expect []byte) error { // and returns the resulting JSON config bytes along with // the name of the loaded config file (if any). func LoadConfig(configFile, adapterName string) ([]byte, string, error) { + return loadConfigWithLogger(caddy.Log(), configFile, adapterName) +} + +func loadConfigWithLogger(logger *zap.Logger, configFile, adapterName string) ([]byte, string, error) { // specifying an adapter without a config file is ambiguous if adapterName != "" && configFile == "" { return nil, "", fmt.Errorf("cannot adapt config without config file (use --config)") @@ -107,13 +117,14 @@ func LoadConfig(configFile, adapterName string) ([]byte, string, error) { if err != nil { return nil, "", fmt.Errorf("reading config file: %v", err) } - caddy.Log().Info("using provided configuration", - zap.String("config_file", configFile), - zap.String("config_adapter", adapterName)) + if logger != nil { + logger.Info("using provided configuration", + zap.String("config_file", configFile), + zap.String("config_adapter", adapterName)) + } } else if adapterName == "" { - // as a special case when no config file or adapter - // is specified, see if the Caddyfile adapter is - // plugged in, and if so, try using a default Caddyfile + // if the Caddyfile adapter is plugged in, we can try using an + // adjacent Caddyfile by default cfgAdapter = caddyconfig.GetAdapter("caddyfile") if cfgAdapter != nil { config, err = os.ReadFile("Caddyfile") @@ -126,7 +137,9 @@ func LoadConfig(configFile, adapterName string) ([]byte, string, error) { } else { // success reading default Caddyfile configFile = "Caddyfile" - caddy.Log().Info("using adjacent Caddyfile") + if logger != nil { + logger.Info("using adjacent Caddyfile") + } } } } @@ -161,7 +174,9 @@ func LoadConfig(configFile, adapterName string) ([]byte, string, error) { if warn.Directive != "" { msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message) } - caddy.Log().Warn(msg, zap.String("adapter", adapterName), zap.String("file", warn.File), zap.Int("line", warn.Line)) + if logger != nil { + logger.Warn(msg, zap.String("adapter", adapterName), zap.String("file", warn.File), zap.Int("line", warn.Line)) + } } config = adaptedConfig } @@ -174,6 +189,8 @@ func LoadConfig(configFile, adapterName string) ([]byte, string, error) { // blocks indefinitely; it only quits if the poller has errors for // long enough time. The filename passed in must be the actual // config file used, not one to be discovered. +// Each second the config files is loaded and parsed into an object +// and is compared to the last config object that was loaded func watchConfigFile(filename, adapterName string) { defer func() { if err := recover(); err != nil { @@ -189,64 +206,36 @@ func watchConfigFile(filename, adapterName string) { With(zap.String("config_file", filename)) } - // get the initial timestamp on the config file - info, err := os.Stat(filename) + // get current config + lastCfg, _, err := loadConfigWithLogger(nil, filename, adapterName) if err != nil { - logger().Error("cannot watch config file", zap.Error(err)) + logger().Error("unable to load latest config", zap.Error(err)) return } - lastModified := info.ModTime() logger().Info("watching config file for changes") - // if the file disappears or something, we can - // stop polling if the error lasts long enough - var lastErr time.Time - finalError := func(err error) bool { - if lastErr.IsZero() { - lastErr = time.Now() - return false - } - if time.Since(lastErr) > 30*time.Second { - logger().Error("giving up watching config file; too many errors", - zap.Error(err)) - return true - } - return false - } - // begin poller //nolint:staticcheck for range time.Tick(1 * time.Second) { - // get the file info - info, err := os.Stat(filename) + // get current config + newCfg, _, err := loadConfigWithLogger(nil, filename, adapterName) if err != nil { - if finalError(err) { - return - } - continue + logger().Error("unable to load latest config", zap.Error(err)) + return } - lastErr = time.Time{} // no error, so clear any memory of one // if it hasn't changed, nothing to do - if !info.ModTime().After(lastModified) { + if bytes.Equal(lastCfg, newCfg) { continue } - logger().Info("config file changed; reloading") - // remember this timestamp - lastModified = info.ModTime() - - // load the contents of the file - config, _, err := LoadConfig(filename, adapterName) - if err != nil { - logger().Error("unable to load latest config", zap.Error(err)) - continue - } + // remember the current config + lastCfg = newCfg // apply the updated config - err = caddy.Load(config, false) + err = caddy.Load(lastCfg, false) if err != nil { logger().Error("applying latest config", zap.Error(err)) continue @@ -316,8 +305,12 @@ func loadEnvFromFile(envFile string) error { } for k, v := range envMap { - if err := os.Setenv(k, v); err != nil { - return fmt.Errorf("setting environment variables: %v", err) + // do not overwrite existing environment variables + _, exists := os.LookupEnv(k) + if !exists { + if err := os.Setenv(k, v); err != nil { + return fmt.Errorf("setting environment variables: %v", err) + } } } @@ -374,18 +367,19 @@ func parseEnvFile(envInput io.Reader) (map[string]string, error) { } // quoted value: support newlines - if strings.HasPrefix(val, `"`) { - for !(strings.HasSuffix(line, `"`) && !strings.HasSuffix(line, `\"`)) { - val = strings.ReplaceAll(val, `\"`, `"`) + if strings.HasPrefix(val, `"`) || strings.HasPrefix(val, "'") { + quote := string(val[0]) + for !(strings.HasSuffix(line, quote) && !strings.HasSuffix(line, `\`+quote)) { + val = strings.ReplaceAll(val, `\`+quote, quote) if !scanner.Scan() { break } lineNumber++ - line = strings.ReplaceAll(scanner.Text(), `\"`, `"`) + line = strings.ReplaceAll(scanner.Text(), `\`+quote, quote) val += "\n" + line } - val = strings.TrimPrefix(val, `"`) - val = strings.TrimSuffix(val, `"`) + val = strings.TrimPrefix(val, quote) + val = strings.TrimSuffix(val, quote) } envMap[key] = val diff --git a/cmd/packagesfuncs.go b/cmd/packagesfuncs.go index 3aed0e8..5d77e4d 100644 --- a/cmd/packagesfuncs.go +++ b/cmd/packagesfuncs.go @@ -27,8 +27,9 @@ import ( "runtime/debug" "strings" - "github.com/caddyserver/caddy/v2" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func cmdUpgrade(fl Flags) (int, error) { diff --git a/cmd/storagefuncs.go b/cmd/storagefuncs.go new file mode 100644 index 0000000..6a6daec --- /dev/null +++ b/cmd/storagefuncs.go @@ -0,0 +1,221 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package caddycmd + +import ( + "archive/tar" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + + "github.com/caddyserver/certmagic" + + "github.com/caddyserver/caddy/v2" +) + +type storVal struct { + StorageRaw json.RawMessage `json:"storage,omitempty" caddy:"namespace=caddy.storage inline_key=module"` +} + +// determineStorage returns the top-level storage module from the given config. +// It may return nil even if no error. +func determineStorage(configFile string, configAdapter string) (*storVal, error) { + cfg, _, err := LoadConfig(configFile, configAdapter) + if err != nil { + return nil, err + } + + // storage defaults to FileStorage if not explicitly + // defined in the config, so the config can be valid + // json but unmarshaling will fail. + if !json.Valid(cfg) { + return nil, &json.SyntaxError{} + } + var tmpStruct storVal + err = json.Unmarshal(cfg, &tmpStruct) + if err != nil { + // default case, ignore the error + var jsonError *json.SyntaxError + if errors.As(err, &jsonError) { + return nil, nil + } + return nil, err + } + + return &tmpStruct, nil +} + +func cmdImportStorage(fl Flags) (int, error) { + importStorageCmdConfigFlag := fl.String("config") + importStorageCmdImportFile := fl.String("input") + + if importStorageCmdConfigFlag == "" { + return caddy.ExitCodeFailedStartup, errors.New("--config is required") + } + if importStorageCmdImportFile == "" { + return caddy.ExitCodeFailedStartup, errors.New("--input is required") + } + + // extract storage from config if possible + storageCfg, err := determineStorage(importStorageCmdConfigFlag, "") + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + + // load specified storage or fallback to default + var stor certmagic.Storage + ctx, cancel := caddy.NewContext(caddy.Context{Context: context.Background()}) + defer cancel() + if storageCfg != nil && storageCfg.StorageRaw != nil { + val, err := ctx.LoadModule(storageCfg, "StorageRaw") + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + stor, err = val.(caddy.StorageConverter).CertMagicStorage() + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + } else { + stor = caddy.DefaultStorage + } + + // setup input + var f *os.File + if importStorageCmdImportFile == "-" { + f = os.Stdin + } else { + f, err = os.Open(importStorageCmdImportFile) + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("opening input file: %v", err) + } + defer f.Close() + } + + // store each archive element + tr := tar.NewReader(f) + for { + hdr, err := tr.Next() + if err == io.EOF { + break + } + if err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("reading archive: %v", err) + } + + b, err := io.ReadAll(tr) + if err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("reading archive: %v", err) + } + + err = stor.Store(ctx, hdr.Name, b) + if err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("reading archive: %v", err) + } + } + + fmt.Println("Successfully imported storage") + return caddy.ExitCodeSuccess, nil +} + +func cmdExportStorage(fl Flags) (int, error) { + exportStorageCmdConfigFlag := fl.String("config") + exportStorageCmdOutputFlag := fl.String("output") + + if exportStorageCmdConfigFlag == "" { + return caddy.ExitCodeFailedStartup, errors.New("--config is required") + } + if exportStorageCmdOutputFlag == "" { + return caddy.ExitCodeFailedStartup, errors.New("--output is required") + } + + // extract storage from config if possible + storageCfg, err := determineStorage(exportStorageCmdConfigFlag, "") + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + + // load specified storage or fallback to default + var stor certmagic.Storage + ctx, cancel := caddy.NewContext(caddy.Context{Context: context.Background()}) + defer cancel() + if storageCfg != nil && storageCfg.StorageRaw != nil { + val, err := ctx.LoadModule(storageCfg, "StorageRaw") + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + stor, err = val.(caddy.StorageConverter).CertMagicStorage() + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + } else { + stor = caddy.DefaultStorage + } + + // enumerate all keys + keys, err := stor.List(ctx, "", true) + if err != nil { + return caddy.ExitCodeFailedStartup, err + } + + // setup output + var f *os.File + if exportStorageCmdOutputFlag == "-" { + f = os.Stdout + } else { + f, err = os.Create(exportStorageCmdOutputFlag) + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("opening output file: %v", err) + } + defer f.Close() + } + + // `IsTerminal: true` keys hold the values we + // care about, write them out + tw := tar.NewWriter(f) + for _, k := range keys { + info, err := stor.Stat(ctx, k) + if err != nil { + return caddy.ExitCodeFailedQuit, err + } + + if info.IsTerminal { + v, err := stor.Load(ctx, k) + if err != nil { + return caddy.ExitCodeFailedQuit, err + } + + hdr := &tar.Header{ + Name: k, + Mode: 0o600, + Size: int64(len(v)), + } + + if err = tw.WriteHeader(hdr); err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("writing archive: %v", err) + } + if _, err = tw.Write(v); err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("writing archive: %v", err) + } + } + } + if err = tw.Close(); err != nil { + return caddy.ExitCodeFailedQuit, fmt.Errorf("writing archive: %v", err) + } + + return caddy.ExitCodeSuccess, nil +} @@ -326,7 +326,7 @@ func (ctx Context) LoadModuleByID(id string, rawMsg json.RawMessage) (any, error // fill in its config only if there is a config to fill in if len(rawMsg) > 0 { - err := strictUnmarshalJSON(rawMsg, &val) + err := StrictUnmarshalJSON(rawMsg, &val) if err != nil { return nil, fmt.Errorf("decoding module config: %s: %v", modInfo, err) } @@ -410,6 +410,11 @@ func (ctx Context) loadModuleInline(moduleNameKey, moduleScope string, raw json. // called during the Provision/Validate phase to reference a // module's own host app (since the parent app module is still // in the process of being provisioned, it is not yet ready). +// +// We return any type instead of the App type because it is NOT +// intended for the caller of this method to be the one to start +// or stop App modules. The caller is expected to assert to the +// concrete type. func (ctx Context) App(name string) (any, error) { if app, ok := ctx.cfg.apps[name]; ok { return app, nil @@ -426,15 +431,23 @@ func (ctx Context) App(name string) (any, error) { return modVal, nil } -// AppIsConfigured returns whether an app named name has been -// configured. Can be called before calling App() to avoid -// instantiating an empty app when that's not desirable. -func (ctx Context) AppIsConfigured(name string) bool { - if _, ok := ctx.cfg.apps[name]; ok { - return true +// AppIfConfigured returns an app by its name if it has been +// configured. Can be called instead of App() to avoid +// instantiating an empty app when that's not desirable. If +// the app has not been loaded, nil is returned. +// +// We return any type instead of the App type because it is not +// intended for the caller of this method to be the one to start +// or stop App modules. The caller is expected to assert to the +// concrete type. +func (ctx Context) AppIfConfigured(name string) any { + if ctx.cfg == nil { + // this can happen if the currently-active context + // is being accessed, but no config has successfully + // been loaded yet + return nil } - appRaw := ctx.cfg.AppsRaw[name] - return appRaw != nil + return ctx.cfg.apps[name] } // Storage returns the configured Caddy storage implementation. @@ -475,6 +488,9 @@ func (ctx Context) Logger(module ...Module) *zap.Logger { if len(module) > 0 { mod = module[0] } + if mod == nil { + return Log() + } return ctx.cfg.Logging.Logger(mod) } @@ -2,11 +2,11 @@ "nodes": { "nixpkgs": { "locked": { - "lastModified": 1697009197, - "narHash": "sha256-viVRhBTFT8fPJTb1N3brQIpFZnttmwo3JVKNuWRVc3s=", + "lastModified": 1698553279, + "narHash": "sha256-T/9P8yBSLcqo/v+FTOBK+0rjzjPMctVymZydbvR/Fak=", "owner": "NixOS", "repo": "nixpkgs", - "rev": "01441e14af5e29c9d27ace398e6dd0b293e25a54", + "rev": "90e85bc7c1a6fc0760a94ace129d3a1c61c3d035", "type": "github" }, "original": { @@ -1,40 +1,44 @@ module github.com/caddyserver/caddy/v2 -go 1.18 +go 1.20 require ( - github.com/BurntSushi/toml v1.2.1 + github.com/BurntSushi/toml v1.3.2 github.com/Masterminds/sprig/v3 v3.2.3 - github.com/alecthomas/chroma/v2 v2.5.0 + github.com/alecthomas/chroma/v2 v2.9.1 github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b - github.com/caddyserver/certmagic v0.17.2 + github.com/caddyserver/certmagic v0.19.2 github.com/dustin/go-humanize v1.0.1 github.com/go-chi/chi v4.1.2+incompatible - github.com/google/cel-go v0.13.0 - github.com/google/uuid v1.3.0 - github.com/klauspost/compress v1.15.15 - github.com/klauspost/cpuid/v2 v2.2.3 - github.com/mholt/acmez v1.1.0 - github.com/prometheus/client_golang v1.14.0 - github.com/quic-go/quic-go v0.32.0 - github.com/smallstep/certificates v0.23.2 - github.com/smallstep/nosql v0.5.0 + github.com/google/cel-go v0.15.1 + github.com/google/uuid v1.3.1 + github.com/klauspost/compress v1.17.0 + github.com/klauspost/cpuid/v2 v2.2.5 + github.com/mastercactapus/proxyprotocol v0.0.4 + github.com/mholt/acmez v1.2.0 + github.com/prometheus/client_golang v1.15.1 + github.com/quic-go/quic-go v0.39.0 + github.com/smallstep/certificates v0.25.0 + github.com/smallstep/nosql v0.6.0 github.com/smallstep/truststore v0.12.1 - github.com/spf13/cobra v1.6.1 + github.com/spf13/cobra v1.7.0 github.com/spf13/pflag v1.0.5 - github.com/tailscale/tscert v0.0.0-20230124224810-c6dc1f4049b2 - github.com/yuin/goldmark v1.5.4 - github.com/yuin/goldmark-highlighting/v2 v2.0.0-20220924101305-151362477c87 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.39.0 - go.opentelemetry.io/otel v1.13.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.4.0 - go.opentelemetry.io/otel/sdk v1.13.0 - go.uber.org/zap v1.24.0 - golang.org/x/crypto v0.5.0 - golang.org/x/net v0.7.0 - golang.org/x/sync v0.1.0 - golang.org/x/term v0.5.0 - google.golang.org/genproto v0.0.0-20230202175211-008b39050e57 + github.com/stretchr/testify v1.8.4 + github.com/tailscale/tscert v0.0.0-20230806124524-28a91b69a046 + github.com/yuin/goldmark v1.5.6 + github.com/yuin/goldmark-highlighting/v2 v2.0.0-20230729083705-37449abec8cc + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.42.0 + go.opentelemetry.io/contrib/propagators/autoprop v0.42.0 + go.opentelemetry.io/otel v1.16.0 + go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.16.0 + go.opentelemetry.io/otel/sdk v1.16.0 + go.uber.org/zap v1.25.0 + golang.org/x/crypto v0.14.0 + golang.org/x/exp v0.0.0-20230310171629-522b1b587ee0 + golang.org/x/net v0.17.0 + golang.org/x/sync v0.4.0 + golang.org/x/term v0.13.0 + google.golang.org/genproto/googleapis/api v0.0.0-20230803162519-f966b187b2e5 gopkg.in/natefinch/lumberjack.v2 v2.2.1 gopkg.in/yaml.v3 v3.0.1 ) @@ -42,20 +46,31 @@ require ( require ( github.com/Microsoft/go-winio v0.6.0 // indirect github.com/aksdb/caddy-cgi/v2 v2.2.1 // indirect + github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230305170008-8188dc5388df // indirect github.com/davecgh/go-spew v1.1.1 // indirect - github.com/fxamacker/cbor/v2 v2.4.0 // indirect + github.com/fxamacker/cbor/v2 v2.5.0 // indirect + github.com/go-chi/chi/v5 v5.0.10 // indirect github.com/go-kit/log v0.2.1 // indirect - github.com/golang/glog v1.0.0 // indirect - github.com/golang/mock v1.6.0 // indirect - github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 // indirect - github.com/onsi/ginkgo/v2 v2.2.0 // indirect + github.com/golang/glog v1.1.0 // indirect + github.com/google/certificate-transparency-go v1.1.6 // indirect + github.com/google/go-tpm v0.9.0 // indirect + github.com/google/go-tspi v0.3.0 // indirect + github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect + github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2 // indirect + github.com/onsi/ginkgo/v2 v2.9.5 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect github.com/quic-go/qpack v0.4.0 // indirect - github.com/quic-go/qtls-go1-18 v0.2.0 // indirect - github.com/quic-go/qtls-go1-19 v0.2.0 // indirect - github.com/quic-go/qtls-go1-20 v0.1.0 // indirect - github.com/rogpeppe/go-internal v1.9.0 // indirect + github.com/quic-go/qtls-go1-20 v0.3.4 // indirect + github.com/smallstep/go-attestation v0.4.4-0.20230627102604-cf579e53cbd2 // indirect github.com/x448/float16 v0.8.4 // indirect - golang.org/x/exp v0.0.0-20221205204356-47842c84f3db // indirect + github.com/zeebo/blake3 v0.2.3 // indirect + go.opentelemetry.io/contrib/propagators/aws v1.17.0 // indirect + go.opentelemetry.io/contrib/propagators/b3 v1.17.0 // indirect + go.opentelemetry.io/contrib/propagators/jaeger v1.17.0 // indirect + go.opentelemetry.io/contrib/propagators/ot v1.17.0 // indirect + go.uber.org/mock v0.3.0 // indirect + google.golang.org/genproto v0.0.0-20230913181813-007df8e322eb // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20230920204549-e6e6cdab5c13 // indirect ) require ( @@ -63,81 +78,78 @@ require ( github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect github.com/Masterminds/goutils v1.1.1 // indirect github.com/Masterminds/semver/v3 v3.2.0 // indirect - github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/cenkalti/backoff/v4 v4.1.2 // indirect + github.com/cenkalti/backoff/v4 v4.2.1 // indirect github.com/cespare/xxhash v1.1.0 // indirect - github.com/cespare/xxhash/v2 v2.1.2 // indirect + github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/chzyer/readline v1.5.1 // indirect github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect github.com/dgraph-io/badger v1.6.2 // indirect github.com/dgraph-io/badger/v2 v2.2007.4 // indirect github.com/dgraph-io/ristretto v0.1.1 // indirect github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect - github.com/dlclark/regexp2 v1.7.0 // indirect + github.com/dlclark/regexp2 v1.10.0 // indirect github.com/felixge/httpsnoop v1.0.3 // indirect github.com/go-kit/kit v0.12.0 // indirect github.com/go-logfmt/logfmt v0.5.1 // indirect - github.com/go-logr/logr v1.2.3 // indirect + github.com/go-logr/logr v1.2.4 // indirect github.com/go-logr/stdr v1.2.2 // indirect - github.com/go-sql-driver/mysql v1.6.0 // indirect - github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 // indirect - github.com/golang/protobuf v1.5.2 // indirect + github.com/go-sql-driver/mysql v1.7.1 // indirect + github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect + github.com/golang/protobuf v1.5.3 // indirect github.com/golang/snappy v0.0.4 // indirect - github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect github.com/huandu/xstrings v1.3.3 // indirect github.com/imdario/mergo v0.3.13 // indirect - github.com/inconshreveable/mousetrap v1.0.1 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jackc/chunkreader/v2 v2.0.1 // indirect - github.com/jackc/pgconn v1.13.0 // indirect + github.com/jackc/pgconn v1.14.0 // indirect github.com/jackc/pgio v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect - github.com/jackc/pgproto3/v2 v2.3.1 // indirect - github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect - github.com/jackc/pgtype v1.12.0 // indirect - github.com/jackc/pgx/v4 v4.17.2 // indirect + github.com/jackc/pgproto3/v2 v2.3.2 // indirect + github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect + github.com/jackc/pgtype v1.14.0 // indirect + github.com/jackc/pgx/v4 v4.18.0 // indirect github.com/libdns/libdns v0.2.1 // indirect github.com/manifoldco/promptui v0.9.0 // indirect github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-isatty v0.0.16 // indirect - github.com/matttproud/golang_protobuf_extensions v1.0.2 // indirect + github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect github.com/micromdm/scep/v2 v2.1.0 // indirect - github.com/miekg/dns v1.1.50 // indirect + github.com/miekg/dns v1.1.55 // indirect github.com/mitchellh/copystructure v1.2.0 // indirect github.com/mitchellh/go-ps v1.0.0 // indirect github.com/mitchellh/reflectwalk v1.0.2 // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/client_model v0.3.0 // indirect - github.com/prometheus/common v0.37.0 // indirect - github.com/prometheus/procfs v0.8.0 // indirect - github.com/rs/xid v1.4.0 // indirect + github.com/prometheus/client_model v0.4.0 // indirect + github.com/prometheus/common v0.42.0 // indirect + github.com/prometheus/procfs v0.9.0 // indirect + github.com/rs/xid v1.5.0 // indirect github.com/russross/blackfriday/v2 v2.1.0 // indirect github.com/shopspring/decimal v1.3.1 // indirect github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect - github.com/sirupsen/logrus v1.9.0 // indirect + github.com/sirupsen/logrus v1.9.3 // indirect github.com/slackhq/nebula v1.6.1 // indirect github.com/spf13/cast v1.5.0 // indirect github.com/stoewer/go-strcase v1.2.0 // indirect - github.com/urfave/cli v1.22.12 // indirect - go.etcd.io/bbolt v1.3.6 // indirect + github.com/urfave/cli v1.22.14 // indirect + go.etcd.io/bbolt v1.3.7 // indirect go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect - go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.4.0 // indirect - go.opentelemetry.io/otel/metric v0.36.0 // indirect - go.opentelemetry.io/otel/trace v1.13.0 // indirect - go.opentelemetry.io/proto/otlp v0.12.0 // indirect - go.step.sm/cli-utils v0.7.5 // indirect - go.step.sm/crypto v0.23.2 - go.step.sm/linkedca v0.19.0 // indirect - go.uber.org/atomic v1.10.0 // indirect - go.uber.org/multierr v1.8.0 // indirect - golang.org/x/mod v0.6.0 // indirect - golang.org/x/sys v0.5.0 - golang.org/x/text v0.7.0 // indirect - golang.org/x/tools v0.2.0 // indirect - google.golang.org/grpc v1.52.3 // indirect - google.golang.org/protobuf v1.28.1 // indirect + go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.16.0 // indirect + go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.16.0 // indirect + go.opentelemetry.io/otel/metric v1.16.0 // indirect + go.opentelemetry.io/otel/trace v1.16.0 + go.opentelemetry.io/proto/otlp v0.19.0 // indirect + go.step.sm/cli-utils v0.8.0 // indirect + go.step.sm/crypto v0.35.1 + go.step.sm/linkedca v0.20.1 // indirect + go.uber.org/multierr v1.11.0 // indirect + golang.org/x/mod v0.11.0 // indirect + golang.org/x/sys v0.13.0 + golang.org/x/text v0.13.0 // indirect + golang.org/x/tools v0.10.0 // indirect + google.golang.org/grpc v1.58.2 // indirect + google.golang.org/protobuf v1.31.0 // indirect gopkg.in/square/go-jose.v2 v2.6.0 // indirect howett.net/plist v1.0.0 // indirect ) @@ -13,19 +13,19 @@ cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKV cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= -cloud.google.com/go v0.107.0 h1:qkj22L7bgkl6vIeZDlOY2po43Mx/TIa2Wsa7VR+PEww= +cloud.google.com/go v0.110.7 h1:rJyC7nWRg2jWGZ4wSJ5nY65GTdYJkg0cd/uXb+ACI6o= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= -cloud.google.com/go/compute v1.14.0 h1:hfm2+FfxVmnRlh6LpB7cg1ZNU+5edAHmW679JePztk0= +cloud.google.com/go/compute v1.23.0 h1:tP41Zoavr8ptEqaW6j+LQOnyBBhO7OkOMAGrgLopTwY= cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= -cloud.google.com/go/iam v0.8.0 h1:E2osAkZzxI/+8pZcxVLcDtAQx/u+hZXVryUaYQ5O0Kk= -cloud.google.com/go/kms v1.8.0 h1:VrJLOsMRzW7IqTTYn+OYupqF3iKSE060Nrn+PECrYjg= +cloud.google.com/go/iam v1.1.1 h1:lW7fzj15aVIXYHREOqjRBV9PsH0Z6u8Y46a1YGvQP4Y= +cloud.google.com/go/kms v1.15.2 h1:lh6qra6oC4AyWe5fUUUBe/S27k12OHAleOOOw6KakdE= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= @@ -41,18 +41,15 @@ filippo.io/edwards25519 v1.0.0/go.mod h1:N1IkdkCkiLB6tki+MYJoSx2JTY9NUlxZE7eHn5E github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIoKjsnZuH8vjyaysT/ses3EvZeaV/1UkF2M= github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/BurntSushi/toml v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak= -github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ= +github.com/BurntSushi/toml v1.3.2 h1:o7IhLm0Msx3BaB+n3Ag7L8EVlByGnpq14C4YWiu/gL8= +github.com/BurntSushi/toml v1.3.2/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0= -github.com/Masterminds/goutils v1.1.0/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU= github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI= github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU= -github.com/Masterminds/semver/v3 v3.1.0/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs= github.com/Masterminds/semver/v3 v3.1.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs= github.com/Masterminds/semver/v3 v3.2.0 h1:3MEsd0SM6jqZojhjLWWeBY+Kcjy9i6MQAeY7YgDP83g= github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ= -github.com/Masterminds/sprig/v3 v3.1.0/go.mod h1:ONGMf7UfYGAbMXCZmQLy8x3lCDIPrEZE/rU8pmrbihA= github.com/Masterminds/sprig/v3 v3.2.3 h1:eL2fZNezLomi0uOLqjQoN6BfsDD+fyLtgbJMAj9n6YA= github.com/Masterminds/sprig/v3 v3.2.3/go.mod h1:rXcFaZ2zZbLRJv/xSysmlgIM1u11eBaRMhvYXJNkGuM= github.com/Microsoft/go-winio v0.6.0 h1:slsWYD/zyx7lCXoZVlvQrj0hPTM1HI4+v1sIda2yDvg= @@ -67,18 +64,17 @@ github.com/aksdb/caddy-cgi/v2 v2.2.1 h1:EvcTTESAkcumUnuoQ7betN/AmzUrmY/8Tlm+aOYy github.com/aksdb/caddy-cgi/v2 v2.2.1/go.mod h1:QRdBNwUhSjN6mEEZgIWvxqplhODkZXNHWNlJgCpmzVM= github.com/alecthomas/assert/v2 v2.2.1 h1:XivOgYcduV98QCahG8T5XTezV5bylXe+lBxLG2K2ink= github.com/alecthomas/chroma/v2 v2.2.0/go.mod h1:vf4zrexSH54oEjJ7EdB65tGNHmH3pGZmVkgTP5RHvAs= -github.com/alecthomas/chroma/v2 v2.5.0 h1:CQCdj1BiBV17sD4Bd32b/Bzuiq/EqoNTrnIhyQAZ+Rk= -github.com/alecthomas/chroma/v2 v2.5.0/go.mod h1:yrkMI9807G1ROx13fhe1v6PN2DDeaR73L3d+1nmYQtw= +github.com/alecthomas/chroma/v2 v2.9.1 h1:0O3lTQh9FxazJ4BYE/MOi/vDGuHn7B+6Bu902N2UZvU= +github.com/alecthomas/chroma/v2 v2.9.1/go.mod h1:4TQu7gdfuPjSh76j78ietmqh9LiurGF0EpseFXdKMBw= github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae/go.mod h1:2kn6fqh/zIyPLmm3ugklbEi5hg5wS435eygvNfaDQL8= github.com/alecthomas/repr v0.2.0 h1:HAzS41CIzNW5syS8Mf9UwXhNH1J9aix/BvDRf1Ml2Yk= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= -github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 h1:yL7+Jz0jTC6yykIK/Wh74gnTJnrGr5AyrNMXuA0gves= -github.com/antlr/antlr4/runtime/Go/antlr v1.4.10/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY= +github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230305170008-8188dc5388df h1:7RFfzj4SSt6nnvCPbCqijJi1nWCd+TqAT3bYCStRC18= +github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230305170008-8188dc5388df/go.mod h1:pSwJ0fSY5KhvocuWSx4fz3BA8OrA1bQn+K1Eli3BRwM= github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= @@ -90,43 +86,42 @@ github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b h1:uUXgbcPDK3KpW29o github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b/go.mod h1:DAHtR1m6lCRdSC2Tm3DSWRPvIPr6xNKyeHdqDQSQT+A= github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU= github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= -github.com/aws/aws-sdk-go v1.44.185 h1:stasiou+Ucx2A0RyXRyPph4sLCBxVQK7DPPK8tNcl5g= +github.com/aws/aws-sdk-go v1.45.12 h1:+bKbbesGNPp+TeGrcqfrWuZoqcIEhjwKyBMHQPp80Jo= github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g= -github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= -github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= +github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps= -github.com/caddyserver/certmagic v0.17.2 h1:o30seC1T/dBqBCNNGNHWwj2i5/I/FMjBbTAhjADP3nE= -github.com/caddyserver/certmagic v0.17.2/go.mod h1:ouWUuC490GOLJzkyN35eXfV8bSbwMwSf4bdhkIxtdQE= +github.com/caddyserver/certmagic v0.19.2 h1:HZd1AKLx4592MalEGQS39DKs2ZOAJCEM/xYPMQ2/ui0= +github.com/caddyserver/certmagic v0.19.2/go.mod h1:fsL01NomQ6N+kE2j37ZCnig2MFosG+MIO4ztnmG/zz8= github.com/casbin/casbin/v2 v2.1.2/go.mod h1:YcPU1XXisHhLzuxH9coDNf2FbKpjGlbCg3n9yuLkIJQ= github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM= -github.com/cenkalti/backoff/v4 v4.1.2 h1:6Yo7N8UP2K6LWZnW94DLVSSrbobcWdVzAYOisuDPIFo= -github.com/cenkalti/backoff/v4 v4.1.2/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= +github.com/cenkalti/backoff/v4 v4.2.1 h1:y4OZtCnogmCPw98Zjyt5a6+QwPLGkiQsYW5oUqylYbM= +github.com/cenkalti/backoff/v4 v4.2.1/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= -github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE= +github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= +github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= +github.com/chzyer/logex v1.2.1 h1:XHDu3E6q+gdHgsdTPH6ImJMIp436vR6MPtH8gP05QzM= github.com/chzyer/logex v1.2.1/go.mod h1:JLbx6lG2kDbNRFnfkgvh4eRJRPX1QCoOIWomwysCBrQ= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/readline v1.5.1 h1:upd/6fQk4src78LMRzh5vItIt361/o4uq553V8B5sGI= github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObkaSkeBlk= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/chzyer/test v1.0.0 h1:p3BQDXSxOhOG0P9z6/hGnII4LGiEPOYBhs8asl/fC04= github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8= github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= +github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= @@ -143,7 +138,6 @@ github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f/go.mod h1:F5haX7 github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE= github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= -github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w= github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= @@ -165,8 +159,9 @@ github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUn github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 h1:fAjc9m62+UWV/WAFKLNi6ZS0675eEUC9y3AlwSbQu1Y= github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw= github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc= -github.com/dlclark/regexp2 v1.7.0 h1:7lJfhqlPssTb1WQx4yvTHN0uElPEv52sbaECrAQxjAo= github.com/dlclark/regexp2 v1.7.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= +github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0= +github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= @@ -180,6 +175,7 @@ github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymF github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= +github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= github.com/envoyproxy/go-control-plane v0.9.10-0.20210907150352-cf90f659a021/go.mod h1:AFq3mo9L8Lqqiid3OhADV3RfLJnjiw63cSpi+fDTRC0= github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= @@ -189,11 +185,13 @@ github.com/felixge/httpsnoop v1.0.3/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSw github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4= github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fxamacker/cbor/v2 v2.4.0 h1:ri0ArlOR+5XunOP8CRUowT0pSJOwhW098ZCUyskZD88= -github.com/fxamacker/cbor/v2 v2.4.0/go.mod h1:TA1xS00nchWmaBnEIxPSE5oHLuJBAVvqrtAnWBwBCVo= +github.com/fxamacker/cbor/v2 v2.5.0 h1:oHsG0V/Q6E/wqTS2O1Cozzsy69nqCiguo5Q1a1ADivE= +github.com/fxamacker/cbor/v2 v2.5.0/go.mod h1:TA1xS00nchWmaBnEIxPSE5oHLuJBAVvqrtAnWBwBCVo= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/go-chi/chi v4.1.2+incompatible h1:fGFk2Gmi/YKXk0OmGfBh0WgmN3XB8lVnEyNz34tQRec= github.com/go-chi/chi v4.1.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ= +github.com/go-chi/chi/v5 v5.0.10 h1:rLz5avzKpjqxrYwXNfmjkrYYXOyLJd37pz53UFHC6vk= +github.com/go-chi/chi/v5 v5.0.10/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= @@ -205,7 +203,6 @@ github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgO github.com/go-kit/kit v0.12.0 h1:e4o3o3IsBfAKQh5Qbbiqyfu97Ku7jrO/JbohvztANh4= github.com/go-kit/kit v0.12.0/go.mod h1:lHd+EkCZPIwYItmGDDRdhinkzX2A1sj+M9biaEaizzs= github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= -github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= github.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU= github.com/go-kit/log v0.2.1/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= @@ -214,19 +211,18 @@ github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG github.com/go-logfmt/logfmt v0.5.1 h1:otpy5pqBCBZ1ng9RQ0dPu4PN7ba75Y/aA+UpowDyNVA= github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= -github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= +github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= -github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= -github.com/go-sql-driver/mysql v1.6.0 h1:BCTh4TKNUYmOmMUcQ3IipzF5prigylS7XXjEkfCHuOE= -github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= +github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI= +github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI= github.com/go-stack/stack v1.6.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-stack/stack v1.8.0 h1:5SgMzNM5HxrEjV0ww2lTmX6E2Izsfxas4+YHWRs3Lsk= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 h1:p104kn46Q8WdvHunIJ9dAyjPVtrBPhSr3KT2yUst43I= -github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= +github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI= +github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls= github.com/gofrs/uuid v4.0.0+incompatible h1:1SD/1F5pU8p29ybwgQSwpQk+mwdRrXCYuPhW6m+TnJw= github.com/gofrs/uuid v4.0.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM= github.com/gogo/googleapis v1.1.0/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s= @@ -234,8 +230,9 @@ github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7a github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/glog v1.0.0 h1:nfP3RFugxnNRyKgeWd4oI1nYvXpxrx8ck8ZrcizshdQ= github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4= +github.com/golang/glog v1.1.0 h1:/d3pCKDPWNnvIWe0vVUpNP32qc8U3PDVxySP/y360qE= +github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP3NQ= github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= @@ -248,8 +245,6 @@ github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= -github.com/golang/mock v1.6.0 h1:ErTB+efbowRARo13NNdxyJji2egdxLGQhRaY+DUumQc= -github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= @@ -265,16 +260,21 @@ github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= -github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM= github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/cel-go v0.13.0 h1:z+8OBOcmh7IeKyqwT/6IlnMvy621fYUqnTVPEdegGlU= -github.com/google/cel-go v0.13.0/go.mod h1:K2hpQgEjDp18J76a2DKFRlPBPpgRZgi6EbnpDgIhJ8s= +github.com/google/btree v1.1.2 h1:xf4v41cLI2Z6FxbKm+8Bu+m8ifhj15JuZ9sa0jZCMUU= +github.com/google/cel-go v0.15.1 h1:iTgVZor2x9okXtmTrqO8cg4uvqIeaBcWhXtruaWFMYQ= +github.com/google/cel-go v0.15.1/go.mod h1:YzWEoI07MC/a/wj9in8GeVatqfypkldgBlwXh9bCwqY= +github.com/google/certificate-transparency-go v1.0.21/go.mod h1:QeJfpSbVSfYc7RgB3gJFj9cbuQMMchQxrWXz8Ruopmg= +github.com/google/certificate-transparency-go v1.1.6 h1:SW5K3sr7ptST/pIvNkSVWMiJqemRmkjJPPT0jzXdOOY= +github.com/google/certificate-transparency-go v1.1.6/go.mod h1:0OJjOsOk+wj6aYQgP7FU0ioQ0AJUmnWPFMqTjQeazPQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= @@ -282,11 +282,14 @@ github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= +github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= +github.com/google/go-tpm v0.9.0 h1:sQF6YqWMi+SCXpsmS3fd21oPy/vSddwZry4JnmltHVk= +github.com/google/go-tpm v0.9.0/go.mod h1:FkNVkc6C+IsvDI9Jw1OveJmxGZUUaKxtrpOS47QWKfU= +github.com/google/go-tpm-tools v0.4.1 h1:gYU6iwRo0tY3V6NDnS6m+XYog+b3g6YFhHQl3sYaUL4= +github.com/google/go-tspi v0.3.0 h1:ADtq8RKfP+jrTyIWIZDIYcKOMecRqNJFOew2IT0Inus= +github.com/google/go-tspi v0.3.0/go.mod h1:xfMGI3G0PhxCdNVcYr1C4C+EizojDg/TXuX5by8CiHI= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= @@ -297,18 +300,19 @@ github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hf github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 h1:yAJXTCF9TqKcTiHJAE8dj7HMvPfh66eeA2JYW7eFpSE= -github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= +github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o= github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= -github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/googleapis/enterprise-certificate-proxy v0.2.1 h1:RY7tHKZcRlk788d5WSo/e83gOyyy742E8GSs771ySpg= +github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4= +github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/googleapis/enterprise-certificate-proxy v0.2.5 h1:UR4rDjcgpgEnqpIEvkiqTYKBCKLNmlge2eVjoZfySzM= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/googleapis/gax-go/v2 v2.7.0 h1:IcsPKeInNvYi7eqSaDjiZqDDKu5rsmunY0Y1YupQSSQ= +github.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/gorilla/context v0.0.0-20160226214623-1ea25387ff6f/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg= github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg= @@ -320,8 +324,10 @@ github.com/groob/finalizer v0.0.0-20170707115354-4c2ed49aabda/go.mod h1:MyndkAZd github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= -github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4M0+kPpLofRdBo= github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2 h1:gDLXvp5S9izjldquuoAhDzccbskOL6tDC5jMSyx3zxE= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2/go.mod h1:7pdNwVWBBHGiCxa9lAszqCJMbfTISJ7oMftp8+UGV08= github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE= github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= @@ -345,21 +351,19 @@ github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2p github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= -github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE= github.com/huandu/xstrings v1.3.3 h1:/Gcsuc1x8JVbJ9/rlye4xZnVAbEkGauT8lbebqcQws4= github.com/huandu/xstrings v1.3.3/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE= github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= -github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/imdario/mergo v0.3.13 h1:lFzP57bqS/wsqKssCGmtLAb8A0wKjLGrve2q3PPVcBk= github.com/imdario/mergo v0.3.13/go.mod h1:4lJ1jqUDcsbIECGy0RUJAXNIhg+6ocWgb1ALK2O4oXg= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= -github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc= -github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo= github.com/jackc/chunkreader v1.0.0/go.mod h1:RT6O25fNZIuasFJRyZ4R/Y2BbhasbmZXF9QQ7T3kePo= github.com/jackc/chunkreader/v2 v2.0.0/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk= @@ -371,9 +375,8 @@ github.com/jackc/pgconn v0.0.0-20190831204454-2fabfa3c18b7/go.mod h1:ZJKsE/KZfsU github.com/jackc/pgconn v1.8.0/go.mod h1:1C2Pb36bGIP9QHGBYCjnyhqu7Rv3sGshaQUvmfGIB/o= github.com/jackc/pgconn v1.9.0/go.mod h1:YctiPyvzfU11JFxoXokUOOKQXQmDMoJL9vJzHH8/2JY= github.com/jackc/pgconn v1.9.1-0.20210724152538-d89c8390a530/go.mod h1:4z2w8XhRbP1hYxkpTuBjTS3ne3J48K83+u0zoyvg2pI= -github.com/jackc/pgconn v1.10.1/go.mod h1:4z2w8XhRbP1hYxkpTuBjTS3ne3J48K83+u0zoyvg2pI= -github.com/jackc/pgconn v1.13.0 h1:3L1XMNV2Zvca/8BYhzcRFS70Lr0WlDg16Di6SFGAbys= -github.com/jackc/pgconn v1.13.0/go.mod h1:AnowpAqO4CMIIJNZl2VJp+KrkAZciAkhEl0W0JIobpI= +github.com/jackc/pgconn v1.14.0 h1:vrbA9Ud87g6JdFWkHTJXppVce58qPIdP7N8y0Ml/A7Q= +github.com/jackc/pgconn v1.14.0/go.mod h1:9mBNlny0UvkgJdCDvdVHYSjI+8tD2rnKK69Wz8ti++E= github.com/jackc/pgio v1.0.0 h1:g12B9UwVnzGhueNavwioyEEpAmqMe1E/BN9ES+8ovkE= github.com/jackc/pgio v1.0.0/go.mod h1:oP+2QK2wFfUWgr+gxjoBH9KGBb31Eio69xUb0w5bYf8= github.com/jackc/pgmock v0.0.0-20190831213851-13a1b77aafa2/go.mod h1:fGZlG77KXmcq05nJLRkk0+p82V8B8Dw8KN2/V9c/OAE= @@ -389,69 +392,61 @@ github.com/jackc/pgproto3/v2 v2.0.0-rc3/go.mod h1:ryONWYqW6dqSg1Lw6vXNMXoBJhpzvW github.com/jackc/pgproto3/v2 v2.0.0-rc3.0.20190831210041-4c03ce451f29/go.mod h1:ryONWYqW6dqSg1Lw6vXNMXoBJhpzvWKnT95C46ckYeM= github.com/jackc/pgproto3/v2 v2.0.6/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA= github.com/jackc/pgproto3/v2 v2.1.1/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA= -github.com/jackc/pgproto3/v2 v2.2.0/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA= -github.com/jackc/pgproto3/v2 v2.3.1 h1:nwj7qwf0S+Q7ISFfBndqeLwSwxs+4DPsbRFjECT1Y4Y= -github.com/jackc/pgproto3/v2 v2.3.1/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA= -github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b h1:C8S2+VttkHFdOOCXJe+YGfa4vHYwlt4Zx+IVXQ97jYg= +github.com/jackc/pgproto3/v2 v2.3.2 h1:7eY55bdBeCz1F2fTzSz69QC+pG46jYq9/jtSPiJ5nn0= +github.com/jackc/pgproto3/v2 v2.3.2/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA= github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b/go.mod h1:vsD4gTJCa9TptPL8sPkXrLZ+hDuNrZCnj29CQpr4X1E= +github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk= +github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM= github.com/jackc/pgtype v0.0.0-20190421001408-4ed0de4755e0/go.mod h1:hdSHsc1V01CGwFsrv11mJRHWJ6aifDLfdV3aVjFF0zg= github.com/jackc/pgtype v0.0.0-20190824184912-ab885b375b90/go.mod h1:KcahbBH1nCMSo2DXpzsoWOAfFkdEtEJpPbVLq8eE+mc= github.com/jackc/pgtype v0.0.0-20190828014616-a8802b16cc59/go.mod h1:MWlu30kVJrUS8lot6TQqcg7mtthZ9T0EoIBFiJcmcyw= github.com/jackc/pgtype v1.8.1-0.20210724151600-32e20a603178/go.mod h1:C516IlIV9NKqfsMCXTdChteoXmwgUceqaLfjg2e3NlM= -github.com/jackc/pgtype v1.9.0/go.mod h1:LUMuVrfsFfdKGLw+AFFVv6KtHOFMwRgDDzBt76IqCA4= -github.com/jackc/pgtype v1.12.0 h1:Dlq8Qvcch7kiehm8wPGIW0W3KsCCHJnRacKW0UM8n5w= -github.com/jackc/pgtype v1.12.0/go.mod h1:LUMuVrfsFfdKGLw+AFFVv6KtHOFMwRgDDzBt76IqCA4= +github.com/jackc/pgtype v1.14.0 h1:y+xUdabmyMkJLyApYuPj38mW+aAIqCe5uuBB51rH3Vw= +github.com/jackc/pgtype v1.14.0/go.mod h1:LUMuVrfsFfdKGLw+AFFVv6KtHOFMwRgDDzBt76IqCA4= github.com/jackc/pgx/v4 v4.0.0-20190420224344-cc3461e65d96/go.mod h1:mdxmSJJuR08CZQyj1PVQBHy9XOp5p8/SHH6a0psbY9Y= github.com/jackc/pgx/v4 v4.0.0-20190421002000-1b8f0016e912/go.mod h1:no/Y67Jkk/9WuGR0JG/JseM9irFbnEPbuWV2EELPNuM= github.com/jackc/pgx/v4 v4.0.0-pre1.0.20190824185557-6972a5742186/go.mod h1:X+GQnOEnf1dqHGpw7JmHqHc1NxDoalibchSk9/RWuDc= github.com/jackc/pgx/v4 v4.12.1-0.20210724153913-640aa07df17c/go.mod h1:1QD0+tgSXP7iUjYm9C1NxKhny7lq6ee99u/z+IHFcgs= -github.com/jackc/pgx/v4 v4.14.0/go.mod h1:jT3ibf/A0ZVCp89rtCIN0zCJxcE74ypROmHEZYsG/j8= -github.com/jackc/pgx/v4 v4.17.2 h1:0Ut0rpeKwvIVbMQ1KbMBU4h6wxehBI535LK6Flheh8E= -github.com/jackc/pgx/v4 v4.17.2/go.mod h1:lcxIZN44yMIrWI78a5CpucdD14hX0SBDbNRvjDBItsw= +github.com/jackc/pgx/v4 v4.18.0 h1:Ltaa1ePvc7msFGALnCrqKJVEByu/qYh5jJBYcDtAno4= +github.com/jackc/pgx/v4 v4.18.0/go.mod h1:FydWkUyadDmdNH/mHnGob881GawxeEm7TcMCzkb+qQE= github.com/jackc/puddle v0.0.0-20190413234325-e4ced69a3a2b/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk= github.com/jackc/puddle v0.0.0-20190608224051-11cab39313c9/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk= github.com/jackc/puddle v1.1.3/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk= -github.com/jackc/puddle v1.2.0/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk= github.com/jackc/puddle v1.3.0/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= -github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= -github.com/klauspost/compress v1.15.15 h1:EF27CXIuDsYJ6mmvtBRlEuB2UVOqHG1tAXgZ7yIO+lw= -github.com/klauspost/compress v1.15.15/go.mod h1:ZcK2JAFqKOpnBlxcLsJzYfrS9X1akm9fHZNnD9+Vo/4= -github.com/klauspost/cpuid/v2 v2.2.3 h1:sxCkb+qR91z4vsqw4vGGZlDgPz3G7gjaLyK3V8y70BU= -github.com/klauspost/cpuid/v2 v2.2.3/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY= +github.com/klauspost/compress v1.17.0 h1:Rnbp4K9EjcDuVuHtd0dgA4qNuv9yKDYKK1ulpJwgrqM= +github.com/klauspost/compress v1.17.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE= +github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c= +github.com/klauspost/cpuid/v2 v2.2.5 h1:0E5MSMDEoAulmXNFquVs//DdoomxaoTY1kUhbc/qbZg= +github.com/klauspost/cpuid/v2 v2.2.5/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw= -github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= github.com/lib/pq v1.1.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= -github.com/lib/pq v1.10.2 h1:AqzbZs4ZoCBp+GtejcpCpcxM3zlSMx29dXbUSeVtJb8= github.com/lib/pq v1.10.2/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o= +github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw= github.com/libdns/libdns v0.2.1 h1:Wu59T7wSHRgtA0cfxC+n1c/e+O3upJGWytknkmFEDis= github.com/libdns/libdns v0.2.1/go.mod h1:yQCXzk1lEZmmCPa857bnk4TsOiqYasqpyOEeSObbb40= github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM= @@ -460,9 +455,10 @@ github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0Q github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/manifoldco/promptui v0.9.0 h1:3V4HzJk1TtXW1MTZMP7mdlwbBpIinw3HztaIlYthEiA= github.com/manifoldco/promptui v0.9.0/go.mod h1:ka04sppxSGFAtxX0qhlYQjISsg9mR4GWtQEhdbn6Pgg= +github.com/mastercactapus/proxyprotocol v0.0.4 h1:qSY75IZF30ZqIU9iW1ip3I7gTnm8wRAnGWqPxCBVgq0= +github.com/mastercactapus/proxyprotocol v0.0.4/go.mod h1:X8FRVEDZz9FkrIoL4QYTBF4Ka4ELwTv0sah0/5NxCPw= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ= -github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= github.com/mattn/go-colorable v0.1.8 h1:c1ghPdyEDarC70ftn0y+A/Ee++9zz8ljHG1b13eJ0s8= github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= @@ -472,27 +468,22 @@ github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNx github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= github.com/mattn/go-isatty v0.0.7/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE= github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= -github.com/mattn/go-isatty v0.0.13 h1:qdl+GuBjcsKKDco5BsxPJlId98mSWNKqYA+Co0SC1yA= -github.com/mattn/go-isatty v0.0.13/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ= github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= -github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= -github.com/matttproud/golang_protobuf_extensions v1.0.2 h1:hAHbPm5IJGijwng3PWk09JkG9WeqChjprR5s9bBZ+OM= -github.com/matttproud/golang_protobuf_extensions v1.0.2/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= +github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo= +github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI= github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= -github.com/mholt/acmez v1.1.0 h1:IQ9CGHKOHokorxnffsqDvmmE30mDenO1lptYZ1AYkHY= -github.com/mholt/acmez v1.1.0/go.mod h1:zwo5+fbLLTowAX8o8ETfQzbDtwGEXnPhkmGdKIP+bgs= +github.com/mholt/acmez v1.2.0 h1:1hhLxSgY5FvH5HCnGUuwbKY2VQVo8IU7rxXKSnZ7F30= +github.com/mholt/acmez v1.2.0/go.mod h1:VT9YwH1xgNX1kmYY89gY8xPJC84BFAisjo8Egigt4kE= github.com/micromdm/scep/v2 v2.1.0 h1:2fS9Rla7qRR266hvUoEauBJ7J6FhgssEiq2OkSKXmaU= github.com/micromdm/scep/v2 v2.1.0/go.mod h1:BkF7TkPPhmgJAMtHfP+sFTKXmgzNJgLQlvvGoOExBcc= github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= -github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA= -github.com/miekg/dns v1.1.50/go.mod h1:e3IlAVfNqAllflbibAZEWOXOQ+Ynzk/dDozDxY7XnME= +github.com/miekg/dns v1.1.55 h1:GoQ4hpsj0nFLYe+bWiCToyrBEJXkQfOOIvFGFy0lEgo= +github.com/miekg/dns v1.1.55/go.mod h1:uInx36IzPl7FYnDcMeVWxj9byh7DutNykX4G9Sj60FY= github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= github.com/mitchellh/copystructure v1.0.0/go.mod h1:SNtv71yrdKgLRyLFxmLdkAbkKEFWgYaq1OVrnRcwhnw= github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw= @@ -513,9 +504,7 @@ github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/nats-io/jwt v0.3.0/go.mod h1:fRYCDE99xlTsqUzISS1Bi75UBJ6ljOJQOAAu5VglpSg= github.com/nats-io/jwt v0.3.2/go.mod h1:/euKqTS1ZD+zzjYrY7pseZrTtWQSjujC7xjPc8wL6eU= github.com/nats-io/nats-server/v2 v2.1.2/go.mod h1:Afk+wRZqkMQs/p45uXdrVLuab3gwv3Z8C4HTBu8GD/k= @@ -528,10 +517,10 @@ github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQ github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo/v2 v2.2.0 h1:3ZNA3L1c5FYDFTTxbFeVGGD8jYvjYauHD30YgLxVsNI= -github.com/onsi/ginkgo/v2 v2.2.0/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk= +github.com/onsi/ginkgo/v2 v2.9.5 h1:+6Hr4uxzP4XIUyAkg61dWBw8lb/gc4/X5luuxN/EC+Q= +github.com/onsi/ginkgo/v2 v2.9.5/go.mod h1:tvAoo1QUJwNEU2ITftXTpR7R1RbCzoZUOs3RonqW57k= github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= -github.com/onsi/gomega v1.20.1 h1:PA/3qinGoukvymdIDV8pii6tiZgC8kbmJO6Z5+b002Q= +github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE= github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk= github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis= github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74= @@ -546,6 +535,7 @@ github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FI github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac= +github.com/peterbourgon/diskv/v3 v3.0.1 h1:x06SQA46+PKIUftmEujdwSEpIx8kR+M9eLYsUxeYveU= github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc= github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= @@ -560,55 +550,40 @@ github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXP github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og= -github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= -github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= -github.com/prometheus/client_golang v1.14.0 h1:nJdhIvne2eSX/XRAFV9PcvFFRbrjbcTUj0VP62TMhnw= -github.com/prometheus/client_golang v1.14.0/go.mod h1:8vpkKitgIVNcqrRBWh1C4TIUQgYNtG/XQE4E/Zae36Y= +github.com/prometheus/client_golang v1.15.1 h1:8tXpTmJbyH5lydzFPoxSIJ0J46jdh3tylbvM1xCv0LI= +github.com/prometheus/client_golang v1.15.1/go.mod h1:e9yaBhRPU2pPNsZwE+JdQl0KEt1N9XgF6zxWmaC0xOk= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.3.0 h1:UBgGFHqYdG/TPFD1B1ogZywDqEkwp3fBMvqdiQ7Xew4= -github.com/prometheus/client_model v0.3.0/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w= +github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY= +github.com/prometheus/client_model v0.4.0/go.mod h1:oMQmHW1/JoDwqLtg57MGgP/Fb1CJEYF2imWWhWtMkYU= github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA= -github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= -github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= -github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE= -github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA= +github.com/prometheus/common v0.42.0 h1:EKsfXEYo4JpWMHH5cg+KOUWeuJSov1Id8zGR8eeI1YM= +github.com/prometheus/common v0.42.0/go.mod h1:xBwqVerjNdUDjgODMpudtOMwlOwf2SaTr1yjz4b7Zbc= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A= -github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= -github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo= -github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4= +github.com/prometheus/procfs v0.9.0 h1:wzCHvIvM5SxWqYvwgVL7yJY8Lz3PKn49KQtpgMYJfhI= +github.com/prometheus/procfs v0.9.0/go.mod h1:+pB4zwohETzFnmlpe6yd2lSc+0/46IYZRB/chUwxUZY= github.com/quic-go/qpack v0.4.0 h1:Cr9BXA1sQS2SmDUWjSofMPNKmvF6IiIfDRmgU0w1ZCo= github.com/quic-go/qpack v0.4.0/go.mod h1:UZVnYIfi5GRk+zI9UMaCPsmZ2xKJP7XBUvVyT1Knj9A= -github.com/quic-go/qtls-go1-18 v0.2.0 h1:5ViXqBZ90wpUcZS0ge79rf029yx0dYB0McyPJwqqj7U= -github.com/quic-go/qtls-go1-18 v0.2.0/go.mod h1:moGulGHK7o6O8lSPSZNoOwcLvJKJ85vVNc7oJFD65bc= -github.com/quic-go/qtls-go1-19 v0.2.0 h1:Cvn2WdhyViFUHoOqK52i51k4nDX8EwIh5VJiVM4nttk= -github.com/quic-go/qtls-go1-19 v0.2.0/go.mod h1:ySOI96ew8lnoKPtSqx2BlI5wCpUVPT05RMAlajtnyOI= -github.com/quic-go/qtls-go1-20 v0.1.0 h1:d1PK3ErFy9t7zxKsG3NXBJXZjp/kMLoIb3y/kV54oAI= -github.com/quic-go/qtls-go1-20 v0.1.0/go.mod h1:JKtK6mjbAVcUTN/9jZpvLbGxvdWIKS8uT7EiStoU1SM= -github.com/quic-go/quic-go v0.32.0 h1:lY02md31s1JgPiiyfqJijpu/UX/Iun304FI3yUqX7tA= -github.com/quic-go/quic-go v0.32.0/go.mod h1:/fCsKANhQIeD5l76c2JFU+07gVE3KaA0FP+0zMWwfwo= +github.com/quic-go/qtls-go1-20 v0.3.4 h1:MfFAPULvst4yoMgY9QmtpYmfij/em7O8UUi+bNVm7Cg= +github.com/quic-go/qtls-go1-20 v0.3.4/go.mod h1:X9Nh97ZL80Z+bX/gUXMbipO6OxdiDi58b/fMC9mAL+k= +github.com/quic-go/quic-go v0.39.0 h1:AgP40iThFMY0bj8jGxROhw3S0FMGa8ryqsmi9tBH3So= +github.com/quic-go/quic-go v0.39.0/go.mod h1:T09QsDQWjLiQ74ZmacDfqZmhY/NLnw5BC40MANNNZ1Q= github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8= -github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ= -github.com/rs/xid v1.4.0 h1:qd7wPTDkN6KQx2VmMBLrpHkiyQwgFXRnkOLacUiaSNY= -github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg= +github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc= +github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg= github.com/rs/zerolog v1.13.0/go.mod h1:YbFCdg8HfsridGWAh22vktObvhZbQsZXe4/zB0OKkWU= github.com/rs/zerolog v1.15.0/go.mod h1:xYTKnLHcpfU2225ny5qZjxnj9NvkumZYjJHlAThCjNc= github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= @@ -618,6 +593,7 @@ github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQD github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E= github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0= +github.com/schollz/jsonstore v1.1.0 h1:WZBDjgezFS34CHI+myb4s8GGpir3UMpy7vWoCeO0n6E= github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24/go.mod h1:M+9NzErvs504Cn4c5DxATwIqPbtswREoFCre64PpcG4= github.com/shopspring/decimal v1.2.0 h1:abSATXmQEYyShuxI4/vyW3tV1MrKAJzCZ/0zLUXYbsQ= @@ -629,19 +605,18 @@ github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeV github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= -github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= -github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0= -github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= +github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= +github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/slackhq/nebula v1.6.1 h1:/OCTR3abj0Sbf2nGoLUrdDXImrCv0ZVFpVPP5qa0DsM= github.com/slackhq/nebula v1.6.1/go.mod h1:UmkqnXe4O53QwToSl/gG7sM4BroQwAB7dd4hUaT6MlI= -github.com/smallstep/assert v0.0.0-20180720014142-de77670473b5/go.mod h1:TC9A4+RjIOS+HyTH7wG17/gSqVv95uDw2J64dQZx7RE= github.com/smallstep/assert v0.0.0-20200723003110-82e2b9b3b262 h1:unQFBIznI+VYD1/1fApl1A+9VcBk+9dcqGfnePY87LY= -github.com/smallstep/assert v0.0.0-20200723003110-82e2b9b3b262/go.mod h1:MyOHs9Po2fbM1LHej6sBUT8ozbxmMOFG+E+rx/GSGuc= -github.com/smallstep/certificates v0.23.2 h1:7KSx9WfZ3CILV0XlsTrl+PK58YE4CHSgqobB6+ieQWs= -github.com/smallstep/certificates v0.23.2/go.mod h1:YuQAlNuYrRA5z+meSH9D0XsUFH45TyAhNgyV5kYuQpQ= -github.com/smallstep/nosql v0.5.0 h1:1BPyHy8bha8qSaxgULGEdqhXpNFXimAfudnauFVqmxw= -github.com/smallstep/nosql v0.5.0/go.mod h1:yKZT5h7cdIVm6wEKM9+jN5dgK80Hljpuy8HNsnI7Gzo= +github.com/smallstep/certificates v0.25.0 h1:WWihtjQ7SprnRxDV44mBp8t5SMsNO5EWsQaEwy1rgFg= +github.com/smallstep/certificates v0.25.0/go.mod h1:thJmekMKUplKYip+la99Lk4IwQej/oVH/zS9PVMagEE= +github.com/smallstep/go-attestation v0.4.4-0.20230627102604-cf579e53cbd2 h1:UIAS8DTWkeclraEGH2aiJPyNPu16VbT41w4JoBlyFfU= +github.com/smallstep/go-attestation v0.4.4-0.20230627102604-cf579e53cbd2/go.mod h1:vNAduivU014fubg6ewygkAvQC0IQVXqdc8vaGl/0er4= +github.com/smallstep/nosql v0.6.0 h1:ur7ysI8s9st0cMXnTvB8tA3+x5Eifmkb6hl4uqNV5jc= +github.com/smallstep/nosql v0.6.0/go.mod h1:jOXwLtockXORUPPZ2MCUcIkGR6w0cN1QGZniY9DITQA= github.com/smallstep/truststore v0.12.1 h1:guLUKkc1UlsXeS3t6BuVMa4leOOpdiv02PCRTiy1WdY= github.com/smallstep/truststore v0.12.1/go.mod h1:M4mebeNy28KusGX3lJxpLARIktLcyqBOrj3ZiZ46pqw= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= @@ -660,8 +635,8 @@ github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w= github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU= github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU= -github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA= -github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY= +github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= +github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= @@ -682,20 +657,21 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= -github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= -github.com/tailscale/tscert v0.0.0-20230124224810-c6dc1f4049b2 h1:TrgfmCXwtWyFw85UkRGXt9qZRzdzt3nWt2Rerdecn0w= -github.com/tailscale/tscert v0.0.0-20230124224810-c6dc1f4049b2/go.mod h1:kNGUQ3VESx3VZwRwA9MSCUegIl6+saPL8Noq82ozCaU= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/tailscale/tscert v0.0.0-20230806124524-28a91b69a046 h1:8rUlviSVOEe7TMk7W0gIPrW8MqEzYfZHpsNWSf8s2vg= +github.com/tailscale/tscert v0.0.0-20230806124524-28a91b69a046/go.mod h1:kNGUQ3VESx3VZwRwA9MSCUegIl6+saPL8Noq82ozCaU= github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0= github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= -github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= -github.com/urfave/cli v1.22.12 h1:igJgVw1JdKH+trcLWLeLwZjU9fEfPesQ+9/e4MQ44S8= -github.com/urfave/cli v1.22.12/go.mod h1:sSBEIC79qR6OvcmsD4U3KABeOTxDqQtdDnaFuUN30b8= +github.com/urfave/cli v1.22.14 h1:ebbhrRiGK2i4naQJr+1Xj92HXZCrK7MsyTS/ob3HnAk= +github.com/urfave/cli v1.22.14/go.mod h1:X0eDS6pD6Exaclxm99NJ3FiCDRED7vIHpx2mDOHLvkA= github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= @@ -703,18 +679,22 @@ github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1: github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= github.com/yuin/goldmark v1.4.15/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -github.com/yuin/goldmark v1.5.4 h1:2uY/xC0roWy8IBEGLgB1ywIoEJFGmRrX21YQcvGZzjU= -github.com/yuin/goldmark v1.5.4/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -github.com/yuin/goldmark-highlighting/v2 v2.0.0-20220924101305-151362477c87 h1:Py16JEzkSdKAtEFJjiaYLYBOWGXc1r/xHj/Q/5lA37k= -github.com/yuin/goldmark-highlighting/v2 v2.0.0-20220924101305-151362477c87/go.mod h1:ovIvrum6DQJA4QsJSovrkC4saKHQVs7TvcaeO8AIl5I= +github.com/yuin/goldmark v1.5.6 h1:COmQAWTCcGetChm3Ig7G/t8AFAN00t+o8Mt4cf7JpwA= +github.com/yuin/goldmark v1.5.6/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +github.com/yuin/goldmark-highlighting/v2 v2.0.0-20230729083705-37449abec8cc h1:+IAOyRda+RLrxa1WC7umKOZRsGq4QrFFMYApOeHzQwQ= +github.com/yuin/goldmark-highlighting/v2 v2.0.0-20230729083705-37449abec8cc/go.mod h1:ovIvrum6DQJA4QsJSovrkC4saKHQVs7TvcaeO8AIl5I= +github.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY= +github.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= +github.com/zeebo/blake3 v0.2.3 h1:TFoLXsjeXqRNFxSbk35Dk4YtszE/MQQGK10BH4ptoTg= +github.com/zeebo/blake3 v0.2.3/go.mod h1:mjJjZpnsyIVtVgTOSpJ9vmRE4wgDeyt2HU3qXvvKCaQ= +github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo= +github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4= github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q= go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= -go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU= -go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= +go.etcd.io/bbolt v1.3.7 h1:j+zJOnnEjF/kyHlDDgGnVL/AIqIJPq8UoB2GSNfkUfQ= +go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw= go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg= go.mozilla.org/pkcs7 v0.0.0-20210730143726-725912489c62/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk= go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 h1:CCriYyAfq1Br1aIYettdHZTy8mBTIPo7We18TuO/bak= @@ -727,59 +707,59 @@ go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.39.0 h1:vFEBG7SieZJzvnRWQ81jxpuEqe6J8Ex+hgc9CqOTzHc= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.39.0/go.mod h1:9rgTcOKdIhDOC0IcAu8a+R+FChqSUBihKpM1lVNi6T0= -go.opentelemetry.io/otel v1.4.0/go.mod h1:jeAqMFKy2uLIxCtKxoFj0FAL5zAPKQagc3+GtBWakzk= -go.opentelemetry.io/otel v1.13.0 h1:1ZAKnNQKwBBxFtww/GwxNUyTf0AxkZzrukO8MeXqe4Y= -go.opentelemetry.io/otel v1.13.0/go.mod h1:FH3RtdZCzRkJYFTCsAKDy9l/XYjMdNv6QrkFFB8DvVg= -go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.0 h1:j7AwzDdAQBJjcqayAaYbvpYeZzII7cEe5qJTu+De6UY= -go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.0/go.mod h1:VpP4/RMn8bv8gNo9uK7/IMY4mtWLELsS+JIP0inH0h4= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.4.0 h1:lRpP10E8oTGVmY1nVXcwelCT1Z8ca41/l5ce7AqLAss= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.4.0/go.mod h1:3oS+j2WUoJVyj6/BzQN/52G17lNJDulngsOxDm1w2PY= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.4.0 h1:buSx4AMC/0Z232slPhicN/fU5KIlj0bMngct5pcZhkI= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.4.0/go.mod h1:ew1NcwkHo0QFT3uTm3m2IVZMkZdVIpbOYNPasgWwpdk= -go.opentelemetry.io/otel/metric v0.36.0 h1:t0lgGI+L68QWt3QtOIlqM9gXoxqxWLhZ3R/e5oOAY0Q= -go.opentelemetry.io/otel/metric v0.36.0/go.mod h1:wKVw57sd2HdSZAzyfOM9gTqqE8v7CbqWsYL6AyrH9qk= -go.opentelemetry.io/otel/sdk v1.4.0/go.mod h1:71GJPNJh4Qju6zJuYl1CrYtXbrgfau/M9UAggqiy1UE= -go.opentelemetry.io/otel/sdk v1.13.0 h1:BHib5g8MvdqS65yo2vV1s6Le42Hm6rrw08qU6yz5JaM= -go.opentelemetry.io/otel/sdk v1.13.0/go.mod h1:YLKPx5+6Vx/o1TCUYYs+bpymtkmazOMT6zoRrC7AQ7I= -go.opentelemetry.io/otel/trace v1.4.0/go.mod h1:uc3eRsqDfWs9R7b92xbQbU42/eTNz4N+gLP8qJCi4aE= -go.opentelemetry.io/otel/trace v1.13.0 h1:CBgRZ6ntv+Amuj1jDsMhZtlAPT6gbyIRdaIzFhfBSdY= -go.opentelemetry.io/otel/trace v1.13.0/go.mod h1:muCvmmO9KKpvuXSf3KKAXXB2ygNYHQ+ZfI5X08d3tds= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.42.0 h1:pginetY7+onl4qN1vl0xW/V/v6OBZ0vVdH+esuJgvmM= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.42.0/go.mod h1:XiYsayHc36K3EByOO6nbAXnAWbrUxdjUROCEeeROOH8= +go.opentelemetry.io/contrib/propagators/autoprop v0.42.0 h1:s2RzYOAqHVgG23q8fPWYChobUoZM6rJZ98EnylJr66w= +go.opentelemetry.io/contrib/propagators/autoprop v0.42.0/go.mod h1:Mv/tWNtZn+NbALDb2XcItP0OM3lWWZjAfSroINxfW+Y= +go.opentelemetry.io/contrib/propagators/aws v1.17.0 h1:IX8d7l2uRw61BlmZBOTQFaK+y22j6vytMVTs9wFrO+c= +go.opentelemetry.io/contrib/propagators/aws v1.17.0/go.mod h1:pAlCYRWff4uGqRXOVn3WP8pDZ5E0K56bEoG7a1VSL4k= +go.opentelemetry.io/contrib/propagators/b3 v1.17.0 h1:ImOVvHnku8jijXqkwCSyYKRDt2YrnGXD4BbhcpfbfJo= +go.opentelemetry.io/contrib/propagators/b3 v1.17.0/go.mod h1:IkfUfMpKWmynvvE0264trz0sf32NRTZL4nuAN9AbWRc= +go.opentelemetry.io/contrib/propagators/jaeger v1.17.0 h1:Zbpbmwav32Ea5jSotpmkWEl3a6Xvd4tw/3xxGO1i05Y= +go.opentelemetry.io/contrib/propagators/jaeger v1.17.0/go.mod h1:tcTUAlmO8nuInPDSBVfG+CP6Mzjy5+gNV4mPxMbL0IA= +go.opentelemetry.io/contrib/propagators/ot v1.17.0 h1:ufo2Vsz8l76eI47jFjuVyjyB3Ae2DmfiCV/o6Vc8ii0= +go.opentelemetry.io/contrib/propagators/ot v1.17.0/go.mod h1:SbKPj5XGp8K/sGm05XblaIABgMgw2jDczP8gGeuaVLk= +go.opentelemetry.io/otel v1.16.0 h1:Z7GVAX/UkAXPKsy94IU+i6thsQS4nb7LviLpnaNeW8s= +go.opentelemetry.io/otel v1.16.0/go.mod h1:vl0h9NUa1D5s1nv3A5vZOYWn8av4K8Ml6JDeHrT/bx4= +go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.16.0 h1:t4ZwRPU+emrcvM2e9DHd0Fsf0JTPVcbfa/BhTDF03d0= +go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.16.0/go.mod h1:vLarbg68dH2Wa77g71zmKQqlQ8+8Rq3GRG31uc0WcWI= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.16.0 h1:cbsD4cUcviQGXdw8+bo5x2wazq10SKz8hEbtCRPcU78= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.16.0/go.mod h1:JgXSGah17croqhJfhByOLVY719k1emAXC8MVhCIJlRs= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.16.0 h1:TVQp/bboR4mhZSav+MdgXB8FaRho1RC8UwVn3T0vjVc= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.16.0/go.mod h1:I33vtIe0sR96wfrUcilIzLoA3mLHhRmz9S9Te0S3gDo= +go.opentelemetry.io/otel/metric v1.16.0 h1:RbrpwVG1Hfv85LgnZ7+txXioPDoh6EdbZHo26Q3hqOo= +go.opentelemetry.io/otel/metric v1.16.0/go.mod h1:QE47cpOmkwipPiefDwo2wDzwJrlfxxNYodqc4xnGCo4= +go.opentelemetry.io/otel/sdk v1.16.0 h1:Z1Ok1YsijYL0CSJpHt4cS3wDDh7p572grzNrBMiMWgE= +go.opentelemetry.io/otel/sdk v1.16.0/go.mod h1:tMsIuKXuuIWPBAOrH+eHtvhTL+SntFtXF9QD68aP6p4= +go.opentelemetry.io/otel/trace v1.16.0 h1:8JRpaObFoW0pxuVPapkgH8UhHQj+bJW8jJsCZEu5MQs= +go.opentelemetry.io/otel/trace v1.16.0/go.mod h1:Yt9vYq1SdNz3xdjZZK7wcXv1qv2pwLkqr2QVwea0ef0= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= -go.opentelemetry.io/proto/otlp v0.12.0 h1:CMJ/3Wp7iOWES+CYLfnBv+DVmPbB+kmy9PJ92XvlR6c= -go.opentelemetry.io/proto/otlp v0.12.0/go.mod h1:TsIjwGWIx5VFYv9KGVlOpxoBl5Dy+63SUguV7GGvlSQ= -go.step.sm/cli-utils v0.7.5 h1:jyp6X8k8mN1B0uWJydTid0C++8tQhm2kaaAdXKQQzdk= -go.step.sm/cli-utils v0.7.5/go.mod h1:taSsY8haLmXoXM3ZkywIyRmVij/4Aj0fQbNTlJvv71I= -go.step.sm/crypto v0.9.0/go.mod h1:+CYG05Mek1YDqi5WK0ERc6cOpKly2i/a5aZmU1sfGj0= -go.step.sm/crypto v0.23.2 h1:XGmQH9Pkpxop47cjYlUhF10L5roPCbu1BCZXopbeW8I= -go.step.sm/crypto v0.23.2/go.mod h1:/IXGz8al8k7u7OV0RTWIi8TRVqO2FMyZVpedV+6Da6U= -go.step.sm/linkedca v0.19.0 h1:xuagkR35wrJI2gnu6FAM+q3VmjwsHScvGcJsfZ0GdsI= -go.step.sm/linkedca v0.19.0/go.mod h1:b7vWPrHfYLEOTSUZitFEcztVCpTc+ileIN85CwEAluM= +go.opentelemetry.io/proto/otlp v0.19.0 h1:IVN6GR+mhC4s5yfcTbmzHYODqvWAp3ZedA2SJPI1Nnw= +go.opentelemetry.io/proto/otlp v0.19.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U= +go.step.sm/cli-utils v0.8.0 h1:b/Tc1/m3YuQq+u3ghTFP7Dz5zUekZj6GUmd5pCvkEXQ= +go.step.sm/cli-utils v0.8.0/go.mod h1:S77aISrC0pKuflqiDfxxJlUbiXcAanyJ4POOnzFSxD4= +go.step.sm/crypto v0.35.1 h1:QAZZ7Q8xaM4TdungGSAYw/zxpyH4fMYTkfaXVV9H7pY= +go.step.sm/crypto v0.35.1/go.mod h1:vn8Vkx/Mbqgoe7AG8btC0qZ995Udm3e+JySuDS1LCJA= +go.step.sm/linkedca v0.20.1 h1:bHDn1+UG1NgRrERkWbbCiAIvv4lD5NOFaswPDTyO5vU= +go.step.sm/linkedca v0.20.1/go.mod h1:Vaq4+Umtjh7DLFI1KuIxeo598vfBzgSYZUjgVJ7Syxw= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= -go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= -go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/atomic v1.10.0 h1:9qC72Qh0+3MqyJbAn8YU5xVq1frD8bn3JtD2oXtafVQ= -go.uber.org/atomic v1.10.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= -go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= -go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= +go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A= +go.uber.org/mock v0.3.0 h1:3mUxI1No2/60yUYax92Pt8eNOEecx2D3lcXZh2NEZJo= +go.uber.org/mock v0.3.0/go.mod h1:a6FSlNadKUHUa9IP5Vyt1zh4fC7uAwxMutEAscFbkZc= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/multierr v1.3.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4= go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU= -go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= -go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= -go.uber.org/multierr v1.8.0 h1:dg6GjLku4EH+249NNmoIciG9N/jURbDG+pFlTkhzIC8= -go.uber.org/multierr v1.8.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= go.uber.org/zap v1.9.1/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM= -go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60= -go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg= +go.uber.org/zap v1.25.0 h1:4Hvk6GtkucQ790dqmj7l1eEnRdKm3k3ZUrUMS2d5+5c= +go.uber.org/zap v1.25.0/go.mod h1:JIAUzQIH94IC4fOJQm7gMmBJP5k7wQfdcnYdPoEXJYk= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= @@ -790,17 +770,15 @@ golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200414173820-0848c9571904/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20201203163018-be400aefbc4c/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4= -golang.org/x/crypto v0.5.0 h1:U/0M97KRkSFvyD/3FSmdP5W5swImpNgle/EHFhOsQPE= -golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU= +golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58= +golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc= +golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -811,8 +789,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0 golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= -golang.org/x/exp v0.0.0-20221205204356-47842c84f3db h1:D/cFflL63o2KSLJIwjlcIt8PR064j/xsmdEJL/YvY/o= -golang.org/x/exp v0.0.0-20221205204356-47842c84f3db/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc= +golang.org/x/exp v0.0.0-20230310171629-522b1b587ee0 h1:LGJsf5LRplCck6jUCH3dBL2dmycNruWNF5xugkSlfXw= +golang.org/x/exp v0.0.0-20230310171629-522b1b587ee0/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= @@ -833,10 +811,9 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= -golang.org/x/mod v0.6.0 h1:b9gGHsz9/HhJ3HF5DHQytPpuwocVTChQJK3AvoLRD5I= -golang.org/x/mod v0.6.0/go.mod h1:4mET923SAdbXp2ki8ey+zGs1SLqsuM2Y0uvdZR/fUNI= +golang.org/x/mod v0.11.0 h1:bUO06HqtnRcc/7l71XBe4WcqTZ+3AH1J59zWDDwLKgU= +golang.org/x/mod v0.11.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= golang.org/x/net v0.0.0-20170726083632-f5079bd7f6f7/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -874,23 +851,18 @@ golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81R golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= -golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210726213435-c6fcb2dbf985/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY= -golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= -golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= +golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= -golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783 h1:nt+Q6cXKz4MosCSpnbMtqiQ8Oz0pxTef2B4Vca2lvfk= +golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.12.0 h1:smVPGxink+n1ZI5pkQa8y6fZT0RW0MgCO5bFpepy4B4= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -899,11 +871,9 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o= -golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= +golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= golang.org/x/sys v0.0.0-20170728174421-0f826bdd13b5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -933,7 +903,6 @@ golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -947,50 +916,42 @@ golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211031064116-611d5d643895/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE= +golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc= -golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= +golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek= +golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k= +golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -1043,12 +1004,9 @@ golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roY golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= -golang.org/x/tools v0.2.0 h1:G6AHpWxTMGY1KyEYoAQ5WTtIekUUvDNjan3ugu60JvE= -golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA= +golang.org/x/tools v0.10.0 h1:tvDr/iQoUqNdohiYm0LmmKcBk+q86lb9EprIUFhHHGg= +golang.org/x/tools v0.10.0/go.mod h1:UJwyiVBsOA2uwvK/e5OY3GTpDUJriEd+/YlqAwLPmyM= golang.org/x/xerrors v0.0.0-20190410155217-1f06c39b4373/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20190513163551-3ee3066db522/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -1072,7 +1030,7 @@ google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0M google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= -google.golang.org/api v0.108.0 h1:WVBc/faN0DkKtR43Q/7+tPny9ZoLZdIiAyG5Q9vFClg= +google.golang.org/api v0.142.0 h1:mf+7EJ94fi5ZcnpPy+m0Yv2dkz8bKm+UL0snTCuwXlY= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= @@ -1112,8 +1070,13 @@ google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7Fc google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20230202175211-008b39050e57 h1:vArvWooPH749rNHpBGgVl+U9B9dATjiEhJzcWGlovNs= -google.golang.org/genproto v0.0.0-20230202175211-008b39050e57/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20211118181313-81c1377c94b1/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20230913181813-007df8e322eb h1:XFBgcDwm7irdHTbz4Zk2h7Mh+eis4nfJEFQFYzJzuIA= +google.golang.org/genproto v0.0.0-20230913181813-007df8e322eb/go.mod h1:yZTlhN0tQnXo3h00fuXNCxJdLdIdnVFVBaRJ5LWBbw4= +google.golang.org/genproto/googleapis/api v0.0.0-20230803162519-f966b187b2e5 h1:nIgk/EEq3/YlnmVVXVnm14rC2oxgs1o0ong4sD/rd44= +google.golang.org/genproto/googleapis/api v0.0.0-20230803162519-f966b187b2e5/go.mod h1:5DZzOUPCLYL3mNkQ0ms0F3EuUNZ7py1Bqeq6sxzI7/Q= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230920204549-e6e6cdab5c13 h1:N3bU/SQDCDyD6R528GJ/PwW9KjYcJA3dgyH+MovAkIM= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230920204549-e6e6cdab5c13/go.mod h1:KSqppvjFjtoCI+KGd4PELB0qLNxdJHRGqRI09mB6pQA= google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM= @@ -1133,10 +1096,10 @@ google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0= google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= -google.golang.org/grpc v1.43.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU= -google.golang.org/grpc v1.44.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU= -google.golang.org/grpc v1.52.3 h1:pf7sOysg4LdgBqduXveGKrcEwbStiK2rtfghdzlUYDQ= -google.golang.org/grpc v1.52.3/go.mod h1:pu6fVzoFb+NBYNAvQL08ic+lvB2IojljRYuun5vorUY= +google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= +google.golang.org/grpc v1.42.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU= +google.golang.org/grpc v1.58.2 h1:SXUpjxeVF3FKrTYQI4f4KvbGD5u2xccdYdurwowix5I= +google.golang.org/grpc v1.58.2/go.mod h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= @@ -1150,8 +1113,8 @@ google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlba google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w= -google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= +google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= @@ -1165,7 +1128,6 @@ gopkg.in/inconshreveable/log15.v2 v2.0.0-20180818164646-67afb5ed74ec/go.mod h1:a gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc= gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= -gopkg.in/square/go-jose.v2 v2.5.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= gopkg.in/square/go-jose.v2 v2.6.0 h1:NGk74WTnPKBNUhNzQX7PYcTLUjoq7mzKk2OKbvwk2iI= gopkg.in/square/go-jose.v2 v2.6.0/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= @@ -1175,8 +1137,6 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= diff --git a/internal/sockets.go b/internal/sockets.go new file mode 100644 index 0000000..56ae9f4 --- /dev/null +++ b/internal/sockets.go @@ -0,0 +1,56 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package internal + +import ( + "fmt" + "io/fs" + "strconv" + "strings" +) + +// SplitUnixSocketPermissionsBits takes a unix socket address in the +// unusual "path|bits" format (e.g. /run/caddy.sock|0222) and tries +// to split it into socket path (host) and permissions bits (port). +// Colons (":") can't be used as separator, as socket paths on Windows +// may include a drive letter (e.g. `unix/c:\absolute\path.sock`). +// Permission bits will default to 0200 if none are specified. +// Throws an error, if the first carrying bit does not +// include write perms (e.g. `0422` or `022`). +// Symbolic permission representation (e.g. `u=w,g=w,o=w`) +// is not supported and will throw an error for now! +func SplitUnixSocketPermissionsBits(addr string) (path string, fileMode fs.FileMode, err error) { + addrSplit := strings.SplitN(addr, "|", 2) + + if len(addrSplit) == 2 { + // parse octal permission bit string as uint32 + fileModeUInt64, err := strconv.ParseUint(addrSplit[1], 8, 32) + if err != nil { + return "", 0, fmt.Errorf("could not parse octal permission bits in %s: %v", addr, err) + } + fileMode = fs.FileMode(fileModeUInt64) + + // FileMode.String() returns a string like `-rwxr-xr--` for `u=rwx,g=rx,o=r` (`0754`) + if string(fileMode.String()[2]) != "w" { + return "", 0, fmt.Errorf("owner of the socket requires '-w-' (write, octal: '2') permissions at least; got '%s' in %s", fileMode.String()[1:4], addr) + } + + return addrSplit[0], fileMode, nil + } + + // default to 0200 (symbolic: `u=w,g=,o=`) + // if no permission bits are specified + return addr, 0o200, nil +} @@ -12,10 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -// TODO: Go 1.19 introduced the "unix" build tag. We have to support Go 1.18 until Go 1.20 is released. -// When Go 1.19 is our minimum, change this build tag to simply "!unix". -// (see similar change needed in listen_unix.go) -//go:build !(aix || android || darwin || dragonfly || freebsd || hurd || illumos || ios || linux || netbsd || openbsd || solaris) +//go:build !unix package caddy diff --git a/listen_unix.go b/listen_unix.go index 7ea6745..8870da5 100644 --- a/listen_unix.go +++ b/listen_unix.go @@ -12,10 +12,10 @@ // See the License for the specific language governing permissions and // limitations under the License. -// TODO: Go 1.19 introduced the "unix" build tag. We have to support Go 1.18 until Go 1.20 is released. -// When Go 1.19 is our minimum, remove this build tag, since "_unix" in the filename will do this. -// (see also change needed in listen.go) -//go:build aix || android || darwin || dragonfly || freebsd || hurd || illumos || ios || linux || netbsd || openbsd || solaris +// Even though the filename ends in _unix.go, we still have to specify the +// build constraint here, because the filename convention only works for +// literal GOOS values, and "unix" is a shortcut unique to build tags. +//go:build unix package caddy @@ -98,7 +98,28 @@ func listenTCPOrUnix(ctx context.Context, lnKey string, network, address string, } return reusePort(network, address, c) } - return config.Listen(ctx, network, address) + + // even though SO_REUSEPORT lets us bind the socket multiple times, + // we still put it in the listenerPool so we can count how many + // configs are using this socket; necessary to ensure we can know + // whether to enforce shutdown delays, for example (see #5393). + ln, err := config.Listen(ctx, network, address) + if err == nil { + listenerPool.LoadOrStore(lnKey, nil) + } + + // if new listener is a unix socket, make sure we can reuse it later + // (we do our own "unlink on close" -- not required, but more tidy) + one := int32(1) + if unix, ok := ln.(*net.UnixListener); ok { + unix.SetUnlinkOnClose(false) + ln = &unixListener{unix, lnKey, &one} + unixSockets[lnKey] = ln.(*unixListener) + } + + // lightly wrap the listener so that when it is closed, + // we can decrement the usage pool counter + return deleteListener{ln, lnKey}, err } // reusePort sets SO_REUSEPORT. Ineffective for unix sockets. @@ -107,7 +128,7 @@ func reusePort(network, address string, conn syscall.RawConn) error { return nil } return conn.Control(func(descriptor uintptr) { - if err := unix.SetsockoptInt(int(descriptor), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1); err != nil { + if err := unix.SetsockoptInt(int(descriptor), unix.SOL_SOCKET, unixSOREUSEPORT, 1); err != nil { Log().Error("setting SO_REUSEPORT", zap.String("network", network), zap.String("address", address), @@ -116,3 +137,37 @@ func reusePort(network, address string, conn syscall.RawConn) error { } }) } + +type unixListener struct { + *net.UnixListener + mapKey string + count *int32 // accessed atomically +} + +func (uln *unixListener) Close() error { + newCount := atomic.AddInt32(uln.count, -1) + if newCount == 0 { + defer func() { + addr := uln.Addr().String() + unixSocketsMu.Lock() + delete(unixSockets, uln.mapKey) + unixSocketsMu.Unlock() + _ = syscall.Unlink(addr) + }() + } + return uln.UnixListener.Close() +} + +// deleteListener is a type that simply deletes itself +// from the listenerPool when it closes. It is used +// solely for the purpose of reference counting (i.e. +// counting how many configs are using a given socket). +type deleteListener struct { + net.Listener + lnKey string +} + +func (dl deleteListener) Close() error { + _, _ = listenerPool.Delete(dl.lnKey) + return dl.Listener.Close() +} diff --git a/listen_unix_setopt.go b/listen_unix_setopt.go new file mode 100644 index 0000000..c9675f9 --- /dev/null +++ b/listen_unix_setopt.go @@ -0,0 +1,7 @@ +//go:build unix && !freebsd + +package caddy + +import "golang.org/x/sys/unix" + +const unixSOREUSEPORT = unix.SO_REUSEPORT diff --git a/listen_unix_setopt_freebsd.go b/listen_unix_setopt_freebsd.go new file mode 100644 index 0000000..0652054 --- /dev/null +++ b/listen_unix_setopt_freebsd.go @@ -0,0 +1,7 @@ +//go:build freebsd + +package caddy + +import "golang.org/x/sys/unix" + +const unixSOREUSEPORT = unix.SO_REUSEPORT_LB diff --git a/listeners.go b/listeners.go index f922144..768d977 100644 --- a/listeners.go +++ b/listeners.go @@ -20,6 +20,7 @@ import ( "errors" "fmt" "io" + "io/fs" "net" "net/netip" "os" @@ -33,6 +34,8 @@ import ( "github.com/quic-go/quic-go" "github.com/quic-go/quic-go/http3" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2/internal" ) // NetworkAddress represents one or more network addresses. @@ -148,11 +151,31 @@ func (na NetworkAddress) Listen(ctx context.Context, portOffset uint, config net func (na NetworkAddress) listen(ctx context.Context, portOffset uint, config net.ListenConfig) (any, error) { var ln any var err error + var address string + var unixFileMode fs.FileMode + var isAbtractUnixSocket bool - address := na.JoinHostPort(portOffset) + // split unix socket addr early so lnKey + // is independent of permissions bits + if na.IsUnixNetwork() { + var err error + address, unixFileMode, err = internal.SplitUnixSocketPermissionsBits(na.Host) + if err != nil { + return nil, err + } + isAbtractUnixSocket = strings.HasPrefix(address, "@") + } else { + address = na.JoinHostPort(portOffset) + } - // if this is a unix socket, see if we already have it open + // if this is a unix socket, see if we already have it open, + // force socket permissions on it and return early if socket, err := reuseUnixSocket(na.Network, address); socket != nil || err != nil { + if !isAbtractUnixSocket { + if err := os.Chmod(address, unixFileMode); err != nil { + return nil, fmt.Errorf("unable to set permissions (%s) on %s: %v", unixFileMode, address, err) + } + } return socket, err } @@ -174,7 +197,8 @@ func (na NetworkAddress) listen(ctx context.Context, portOffset uint, config net if err != nil { return nil, err } - ln = &fakeClosePacketConn{sharedPacketConn: sharedPc.(*sharedPacketConn)} + spc := sharedPc.(*sharedPacketConn) + ln = &fakeClosePacketConn{spc: spc, UDPConn: spc.PacketConn.(*net.UDPConn)} } if strings.HasPrefix(na.Network, "ip") { ln, err = config.ListenPacket(ctx, na.Network, address) @@ -186,17 +210,19 @@ func (na NetworkAddress) listen(ctx context.Context, portOffset uint, config net return nil, fmt.Errorf("unsupported network type: %s", na.Network) } - // if new listener is a unix socket, make sure we can reuse it later - // (we do our own "unlink on close" -- not required, but more tidy) - one := int32(1) - switch unix := ln.(type) { - case *net.UnixListener: - unix.SetUnlinkOnClose(false) - ln = &unixListener{unix, lnKey, &one} - unixSockets[lnKey] = ln.(*unixListener) - case *net.UnixConn: + // TODO: Not 100% sure this is necessary, but we do this for net.UnixListener in listen_unix.go, so... + if unix, ok := ln.(*net.UnixConn); ok { + one := int32(1) ln = &unixConn{unix, address, lnKey, &one} - unixSockets[lnKey] = ln.(*unixConn) + unixSockets[lnKey] = unix + } + + if IsUnixNetwork(na.Network) { + if !isAbtractUnixSocket { + if err := os.Chmod(address, unixFileMode); err != nil { + return nil, fmt.Errorf("unable to set permissions (%s) on %s: %v", unixFileMode, address, err) + } + } } return ln, nil @@ -303,22 +329,32 @@ func IsUnixNetwork(netw string) bool { // Network addresses are distinct from URLs and do not // use URL syntax. func ParseNetworkAddress(addr string) (NetworkAddress, error) { + return ParseNetworkAddressWithDefaults(addr, "tcp", 0) +} + +// ParseNetworkAddressWithDefaults is like ParseNetworkAddress but allows +// the default network and port to be specified. +func ParseNetworkAddressWithDefaults(addr, defaultNetwork string, defaultPort uint) (NetworkAddress, error) { var host, port string network, host, port, err := SplitNetworkAddress(addr) if err != nil { return NetworkAddress{}, err } if network == "" { - network = "tcp" + network = defaultNetwork } if IsUnixNetwork(network) { + _, _, err := internal.SplitUnixSocketPermissionsBits(host) return NetworkAddress{ Network: network, Host: host, - }, nil + }, err } var start, end uint64 - if port != "" { + if port == "" { + start = uint64(defaultPort) + end = uint64(defaultPort) + } else { before, after, found := strings.Cut(port, "-") if !found { after = before @@ -436,11 +472,16 @@ func ListenPacket(network, addr string) (net.PacketConn, error) { // NOTE: This API is EXPERIMENTAL and may be changed or removed. // // TODO: See if we can find a more elegant solution closer to the new NetworkAddress.Listen API. -func ListenQUIC(ln net.PacketConn, tlsConf *tls.Config, activeRequests *int64) (quic.EarlyListener, error) { +func ListenQUIC(ln net.PacketConn, tlsConf *tls.Config, activeRequests *int64) (http3.QUICEarlyListener, error) { lnKey := listenerKey("quic+"+ln.LocalAddr().Network(), ln.LocalAddr().String()) sharedEarlyListener, _, err := listenerPool.LoadOrNew(lnKey, func() (Destructor, error) { - earlyLn, err := quic.ListenEarly(ln, http3.ConfigureTLSConfig(tlsConf), &quic.Config{ + sqtc := newSharedQUICTLSConfig(tlsConf) + // http3.ConfigureTLSConfig only uses this field and tls App sets this field as well + //nolint:gosec + quicTlsConfig := &tls.Config{GetConfigForClient: sqtc.getConfigForClient} + earlyLn, err := quic.ListenEarly(ln, http3.ConfigureTLSConfig(quicTlsConfig), &quic.Config{ + Allow0RTT: true, RequireAddressValidation: func(clientAddr net.Addr) bool { var highLoad bool if activeRequests != nil { @@ -452,12 +493,16 @@ func ListenQUIC(ln net.PacketConn, tlsConf *tls.Config, activeRequests *int64) ( if err != nil { return nil, err } - return &sharedQuicListener{EarlyListener: earlyLn, key: lnKey}, nil + return &sharedQuicListener{EarlyListener: earlyLn, sqtc: sqtc, key: lnKey}, nil }) if err != nil { return nil, err } + sql := sharedEarlyListener.(*sharedQuicListener) + // add current tls.Config to sqtc, so GetConfigForClient will always return the latest tls.Config in case of context cancellation + ctx, cancel := sql.sqtc.addTLSConfig(tlsConf) + // TODO: to serve QUIC over a unix socket, currently we need to hold onto // the underlying net.PacketConn (which we wrap as unixConn to keep count // of closes) because closing the quic.EarlyListener doesn't actually close @@ -469,9 +514,8 @@ func ListenQUIC(ln net.PacketConn, tlsConf *tls.Config, activeRequests *int64) ( unix = uc } - ctx, cancel := context.WithCancel(context.Background()) return &fakeCloseQuicListener{ - sharedQuicListener: sharedEarlyListener.(*sharedQuicListener), + sharedQuicListener: sql, uc: unix, context: ctx, contextCancel: cancel, @@ -484,10 +528,77 @@ func ListenerUsage(network, addr string) int { return count } +// contextAndCancelFunc groups context and its cancelFunc +type contextAndCancelFunc struct { + context.Context + context.CancelFunc +} + +// sharedQUICTLSConfig manages GetConfigForClient +// see issue: https://github.com/caddyserver/caddy/pull/4849 +type sharedQUICTLSConfig struct { + rmu sync.RWMutex + tlsConfs map[*tls.Config]contextAndCancelFunc + activeTlsConf *tls.Config +} + +// newSharedQUICTLSConfig creates a new sharedQUICTLSConfig +func newSharedQUICTLSConfig(tlsConfig *tls.Config) *sharedQUICTLSConfig { + sqtc := &sharedQUICTLSConfig{ + tlsConfs: make(map[*tls.Config]contextAndCancelFunc), + activeTlsConf: tlsConfig, + } + sqtc.addTLSConfig(tlsConfig) + return sqtc +} + +// getConfigForClient is used as tls.Config's GetConfigForClient field +func (sqtc *sharedQUICTLSConfig) getConfigForClient(ch *tls.ClientHelloInfo) (*tls.Config, error) { + sqtc.rmu.RLock() + defer sqtc.rmu.RUnlock() + return sqtc.activeTlsConf.GetConfigForClient(ch) +} + +// addTLSConfig adds tls.Config to the map if not present and returns the corresponding context and its cancelFunc +// so that when cancelled, the active tls.Config will change +func (sqtc *sharedQUICTLSConfig) addTLSConfig(tlsConfig *tls.Config) (context.Context, context.CancelFunc) { + sqtc.rmu.Lock() + defer sqtc.rmu.Unlock() + + if cacc, ok := sqtc.tlsConfs[tlsConfig]; ok { + return cacc.Context, cacc.CancelFunc + } + + ctx, cancel := context.WithCancel(context.Background()) + wrappedCancel := func() { + cancel() + + sqtc.rmu.Lock() + defer sqtc.rmu.Unlock() + + delete(sqtc.tlsConfs, tlsConfig) + if sqtc.activeTlsConf == tlsConfig { + // select another tls.Config, if there is none, + // related sharedQuicListener will be destroyed anyway + for tc := range sqtc.tlsConfs { + sqtc.activeTlsConf = tc + break + } + } + } + sqtc.tlsConfs[tlsConfig] = contextAndCancelFunc{ctx, wrappedCancel} + // there should be at most 2 tls.Configs + if len(sqtc.tlsConfs) > 2 { + Log().Warn("quic listener tls configs are more than 2", zap.Int("number of configs", len(sqtc.tlsConfs))) + } + return ctx, wrappedCancel +} + // sharedQuicListener is like sharedListener, but for quic.EarlyListeners. type sharedQuicListener struct { - quic.EarlyListener - key string + *quic.EarlyListener + sqtc *sharedQUICTLSConfig + key string } // Destruct closes the underlying QUIC listener. @@ -525,37 +636,30 @@ func fakeClosedErr(l interface{ Addr() net.Addr }) error { // socket is actually left open. var errFakeClosed = fmt.Errorf("listener 'closed' 😉") -// fakeClosePacketConn is like fakeCloseListener, but for PacketConns. +// fakeClosePacketConn is like fakeCloseListener, but for PacketConns, +// or more specifically, *net.UDPConn type fakeClosePacketConn struct { - closed int32 // accessed atomically; belongs to this struct only - *sharedPacketConn // embedded, so we also become a net.PacketConn + closed int32 // accessed atomically; belongs to this struct only + spc *sharedPacketConn // its key is used in Close + *net.UDPConn // embedded, so we also become a net.PacketConn and enable several other optimizations done by quic-go } +// interface guard for extra optimizations +// needed by QUIC implementation: https://github.com/caddyserver/caddy/issues/3998, https://github.com/caddyserver/caddy/issues/5605 +var _ quic.OOBCapablePacketConn = (*fakeClosePacketConn)(nil) + +// https://pkg.go.dev/golang.org/x/net/ipv4#NewPacketConn is used by quic-go and requires a net.PacketConn type assertable to a net.Conn, +// but doesn't actually use these methods, the only methods needed are `ReadMsgUDP` and `SyscallConn`. +var _ net.Conn = (*fakeClosePacketConn)(nil) + +// Close won't close the underlying socket unless there is no more reference, then listenerPool will close it. func (fcpc *fakeClosePacketConn) Close() error { if atomic.CompareAndSwapInt32(&fcpc.closed, 0, 1) { - _, _ = listenerPool.Delete(fcpc.sharedPacketConn.key) + _, _ = listenerPool.Delete(fcpc.spc.key) } return nil } -// Supports QUIC implementation: https://github.com/caddyserver/caddy/issues/3998 -func (fcpc fakeClosePacketConn) SetReadBuffer(bytes int) error { - if conn, ok := fcpc.PacketConn.(interface{ SetReadBuffer(int) error }); ok { - return conn.SetReadBuffer(bytes) - } - return fmt.Errorf("SetReadBuffer() not implemented for %T", fcpc.PacketConn) -} - -// Supports QUIC implementation: https://github.com/caddyserver/caddy/issues/3998 -func (fcpc fakeClosePacketConn) SyscallConn() (syscall.RawConn, error) { - if conn, ok := fcpc.PacketConn.(interface { - SyscallConn() (syscall.RawConn, error) - }); ok { - return conn.SyscallConn() - } - return nil, fmt.Errorf("SyscallConn() not implemented for %T", fcpc.PacketConn) -} - type fakeCloseQuicListener struct { closed int32 // accessed atomically; belongs to this struct only *sharedQuicListener // embedded, so we also become a quic.EarlyListener @@ -616,26 +720,6 @@ func RegisterNetwork(network string, getListener ListenerFunc) { networkTypes[network] = getListener } -type unixListener struct { - *net.UnixListener - mapKey string - count *int32 // accessed atomically -} - -func (uln *unixListener) Close() error { - newCount := atomic.AddInt32(uln.count, -1) - if newCount == 0 { - defer func() { - addr := uln.Addr().String() - unixSocketsMu.Lock() - delete(unixSockets, uln.mapKey) - unixSocketsMu.Unlock() - _ = syscall.Unlink(addr) - }() - } - return uln.UnixListener.Close() -} - type unixConn struct { *net.UnixConn filename string diff --git a/listeners_test.go b/listeners_test.go index c5aa527..f8a13ca 100644 --- a/listeners_test.go +++ b/listeners_test.go @@ -17,6 +17,8 @@ package caddy import ( "reflect" "testing" + + "github.com/caddyserver/caddy/v2/internal" ) func TestSplitNetworkAddress(t *testing.T) { @@ -175,47 +177,57 @@ func TestJoinNetworkAddress(t *testing.T) { func TestParseNetworkAddress(t *testing.T) { for i, tc := range []struct { - input string - expectAddr NetworkAddress - expectErr bool + input string + defaultNetwork string + defaultPort uint + expectAddr NetworkAddress + expectErr bool }{ { input: "", expectErr: true, }, { - input: ":", + input: ":", + defaultNetwork: "udp", expectAddr: NetworkAddress{ - Network: "tcp", + Network: "udp", }, }, { - input: "[::]", + input: "[::]", + defaultNetwork: "udp", + defaultPort: 53, expectAddr: NetworkAddress{ - Network: "tcp", - Host: "::", + Network: "udp", + Host: "::", + StartPort: 53, + EndPort: 53, }, }, { - input: ":1234", + input: ":1234", + defaultNetwork: "udp", expectAddr: NetworkAddress{ - Network: "tcp", + Network: "udp", Host: "", StartPort: 1234, EndPort: 1234, }, }, { - input: "tcp/:1234", + input: "udp/:1234", + defaultNetwork: "udp", expectAddr: NetworkAddress{ - Network: "tcp", + Network: "udp", Host: "", StartPort: 1234, EndPort: 1234, }, }, { - input: "tcp6/:1234", + input: "tcp6/:1234", + defaultNetwork: "tcp", expectAddr: NetworkAddress{ Network: "tcp6", Host: "", @@ -224,7 +236,8 @@ func TestParseNetworkAddress(t *testing.T) { }, }, { - input: "tcp4/localhost:1234", + input: "tcp4/localhost:1234", + defaultNetwork: "tcp", expectAddr: NetworkAddress{ Network: "tcp4", Host: "localhost", @@ -233,14 +246,16 @@ func TestParseNetworkAddress(t *testing.T) { }, }, { - input: "unix//foo/bar", + input: "unix//foo/bar", + defaultNetwork: "tcp", expectAddr: NetworkAddress{ Network: "unix", Host: "/foo/bar", }, }, { - input: "localhost:1234-1234", + input: "localhost:1234-1234", + defaultNetwork: "tcp", expectAddr: NetworkAddress{ Network: "tcp", Host: "localhost", @@ -249,11 +264,13 @@ func TestParseNetworkAddress(t *testing.T) { }, }, { - input: "localhost:2-1", - expectErr: true, + input: "localhost:2-1", + defaultNetwork: "tcp", + expectErr: true, }, { - input: "localhost:0", + input: "localhost:0", + defaultNetwork: "tcp", expectAddr: NetworkAddress{ Network: "tcp", Host: "localhost", @@ -262,11 +279,138 @@ func TestParseNetworkAddress(t *testing.T) { }, }, { - input: "localhost:1-999999999999", + input: "localhost:1-999999999999", + defaultNetwork: "tcp", + expectErr: true, + }, + } { + actualAddr, err := ParseNetworkAddressWithDefaults(tc.input, tc.defaultNetwork, tc.defaultPort) + if tc.expectErr && err == nil { + t.Errorf("Test %d: Expected error but got: %v", i, err) + } + if !tc.expectErr && err != nil { + t.Errorf("Test %d: Expected no error but got: %v", i, err) + } + + if actualAddr.Network != tc.expectAddr.Network { + t.Errorf("Test %d: Expected network '%v' but got '%v'", i, tc.expectAddr, actualAddr) + } + if !reflect.DeepEqual(tc.expectAddr, actualAddr) { + t.Errorf("Test %d: Expected addresses %v but got %v", i, tc.expectAddr, actualAddr) + } + } +} + +func TestParseNetworkAddressWithDefaults(t *testing.T) { + for i, tc := range []struct { + input string + defaultNetwork string + defaultPort uint + expectAddr NetworkAddress + expectErr bool + }{ + { + input: "", expectErr: true, }, + { + input: ":", + defaultNetwork: "udp", + expectAddr: NetworkAddress{ + Network: "udp", + }, + }, + { + input: "[::]", + defaultNetwork: "udp", + defaultPort: 53, + expectAddr: NetworkAddress{ + Network: "udp", + Host: "::", + StartPort: 53, + EndPort: 53, + }, + }, + { + input: ":1234", + defaultNetwork: "udp", + expectAddr: NetworkAddress{ + Network: "udp", + Host: "", + StartPort: 1234, + EndPort: 1234, + }, + }, + { + input: "udp/:1234", + defaultNetwork: "udp", + expectAddr: NetworkAddress{ + Network: "udp", + Host: "", + StartPort: 1234, + EndPort: 1234, + }, + }, + { + input: "tcp6/:1234", + defaultNetwork: "tcp", + expectAddr: NetworkAddress{ + Network: "tcp6", + Host: "", + StartPort: 1234, + EndPort: 1234, + }, + }, + { + input: "tcp4/localhost:1234", + defaultNetwork: "tcp", + expectAddr: NetworkAddress{ + Network: "tcp4", + Host: "localhost", + StartPort: 1234, + EndPort: 1234, + }, + }, + { + input: "unix//foo/bar", + defaultNetwork: "tcp", + expectAddr: NetworkAddress{ + Network: "unix", + Host: "/foo/bar", + }, + }, + { + input: "localhost:1234-1234", + defaultNetwork: "tcp", + expectAddr: NetworkAddress{ + Network: "tcp", + Host: "localhost", + StartPort: 1234, + EndPort: 1234, + }, + }, + { + input: "localhost:2-1", + defaultNetwork: "tcp", + expectErr: true, + }, + { + input: "localhost:0", + defaultNetwork: "tcp", + expectAddr: NetworkAddress{ + Network: "tcp", + Host: "localhost", + StartPort: 0, + EndPort: 0, + }, + }, + { + input: "localhost:1-999999999999", + defaultNetwork: "tcp", + expectErr: true, + }, } { - actualAddr, err := ParseNetworkAddress(tc.input) + actualAddr, err := ParseNetworkAddressWithDefaults(tc.input, tc.defaultNetwork, tc.defaultPort) if tc.expectErr && err == nil { t.Errorf("Test %d: Expected error but got: %v", i, err) } @@ -413,3 +557,98 @@ func TestExpand(t *testing.T) { } } } + +func TestSplitUnixSocketPermissionsBits(t *testing.T) { + for i, tc := range []struct { + input string + expectNetwork string + expectPath string + expectFileMode string + expectErr bool + }{ + { + input: "./foo.socket", + expectPath: "./foo.socket", + expectFileMode: "--w-------", + }, + { + input: `.\relative\path.socket`, + expectPath: `.\relative\path.socket`, + expectFileMode: "--w-------", + }, + { + // literal colon in resulting address + // and defaulting to 0200 bits + input: "./foo.socket:0666", + expectPath: "./foo.socket:0666", + expectFileMode: "--w-------", + }, + { + input: "./foo.socket|0220", + expectPath: "./foo.socket", + expectFileMode: "--w--w----", + }, + { + input: "/var/run/foo|222", + expectPath: "/var/run/foo", + expectFileMode: "--w--w--w-", + }, + { + input: "./foo.socket|0660", + expectPath: "./foo.socket", + expectFileMode: "-rw-rw----", + }, + { + input: "./foo.socket|0666", + expectPath: "./foo.socket", + expectFileMode: "-rw-rw-rw-", + }, + { + input: "/var/run/foo|666", + expectPath: "/var/run/foo", + expectFileMode: "-rw-rw-rw-", + }, + { + input: `c:\absolute\path.socket|220`, + expectPath: `c:\absolute\path.socket`, + expectFileMode: "--w--w----", + }, + { + // symbolic permission representation is not supported for now + input: "./foo.socket|u=rw,g=rw,o=rw", + expectErr: true, + }, + { + // octal (base-8) permission representation has to be between + // `0` for no read, no write, no exec (`---`) and + // `7` for read (4), write (2), exec (1) (`rwx` => `4+2+1 = 7`) + input: "./foo.socket|888", + expectErr: true, + }, + { + // too many colons in address + input: "./foo.socket|123456|0660", + expectErr: true, + }, + { + // owner is missing write perms + input: "./foo.socket|0522", + expectErr: true, + }, + } { + actualPath, actualFileMode, err := internal.SplitUnixSocketPermissionsBits(tc.input) + if tc.expectErr && err == nil { + t.Errorf("Test %d: Expected error but got: %v", i, err) + } + if !tc.expectErr && err != nil { + t.Errorf("Test %d: Expected no error but got: %v", i, err) + } + if actualPath != tc.expectPath { + t.Errorf("Test %d: Expected path '%s' but got '%s'", i, tc.expectPath, actualPath) + } + // fileMode.Perm().String() parses 0 to "----------" + if !tc.expectErr && actualFileMode.Perm().String() != tc.expectFileMode { + t.Errorf("Test %d: Expected perms '%s' but got '%s'", i, tc.expectFileMode, actualFileMode.Perm().String()) + } + } +} @@ -62,7 +62,7 @@ type Logging struct { // in dependencies that are not designed specifically for use // in Caddy. Because it is global and unstructured, the sink // lacks most advanced features and customizations. - Sink *StandardLibLog `json:"sink,omitempty"` + Sink *SinkLog `json:"sink,omitempty"` // Logs are your logs, keyed by an arbitrary name of your // choosing. The default log can be customized by defining @@ -259,55 +259,11 @@ func (wdest writerDestructor) Destruct() error { return wdest.Close() } -// StandardLibLog configures the default Go standard library -// global logger in the log package. This is necessary because -// module dependencies which are not built specifically for -// Caddy will use the standard logger. This is also known as -// the "sink" logger. -type StandardLibLog struct { +// BaseLog contains the common logging parameters for logging. +type BaseLog struct { // The module that writes out log entries for the sink. WriterRaw json.RawMessage `json:"writer,omitempty" caddy:"namespace=caddy.logging.writers inline_key=output"` - writer io.WriteCloser -} - -func (sll *StandardLibLog) provision(ctx Context, logging *Logging) error { - if sll.WriterRaw != nil { - mod, err := ctx.LoadModule(sll, "WriterRaw") - if err != nil { - return fmt.Errorf("loading sink log writer module: %v", err) - } - wo := mod.(WriterOpener) - - var isNew bool - sll.writer, isNew, err = logging.openWriter(wo) - if err != nil { - return fmt.Errorf("opening sink log writer %#v: %v", mod, err) - } - - if isNew { - log.Printf("[INFO] Redirecting sink to: %s", wo) - log.SetOutput(sll.writer) - log.Printf("[INFO] Redirected sink to here (%s)", wo) - } - } - - return nil -} - -// CustomLog represents a custom logger configuration. -// -// By default, a log will emit all log entries. Some entries -// will be skipped if sampling is enabled. Further, the Include -// and Exclude parameters define which loggers (by name) are -// allowed or rejected from emitting in this log. If both Include -// and Exclude are populated, their values must be mutually -// exclusive, and longer namespaces have priority. If neither -// are populated, all logs are emitted. -type CustomLog struct { - // The writer defines where log entries are emitted. - WriterRaw json.RawMessage `json:"writer,omitempty" caddy:"namespace=caddy.logging.writers inline_key=output"` - // The encoder is how the log entries are formatted or encoded. EncoderRaw json.RawMessage `json:"encoder,omitempty" caddy:"namespace=caddy.logging.encoders inline_key=format"` @@ -321,16 +277,6 @@ type CustomLog struct { // servers. Sampling *LogSampling `json:"sampling,omitempty"` - // Include defines the names of loggers to emit in this - // log. For example, to include only logs emitted by the - // admin API, you would include "admin.api". - Include []string `json:"include,omitempty"` - - // Exclude defines the names of loggers that should be - // skipped by this log. For example, to exclude only - // HTTP access logs, you would exclude "http.log.access". - Exclude []string `json:"exclude,omitempty"` - writerOpener WriterOpener writer io.WriteCloser encoder zapcore.Encoder @@ -338,8 +284,23 @@ type CustomLog struct { core zapcore.Core } -func (cl *CustomLog) provision(ctx Context, logging *Logging) error { - // Replace placeholder for log level +func (cl *BaseLog) provisionCommon(ctx Context, logging *Logging) error { + if cl.WriterRaw != nil { + mod, err := ctx.LoadModule(cl, "WriterRaw") + if err != nil { + return fmt.Errorf("loading log writer module: %v", err) + } + cl.writerOpener = mod.(WriterOpener) + } + if cl.writerOpener == nil { + cl.writerOpener = StderrWriter{} + } + var err error + cl.writer, _, err = logging.openWriter(cl.writerOpener) + if err != nil { + return fmt.Errorf("opening log writer using %#v: %v", cl.writerOpener, err) + } + repl := NewReplacer() level, err := repl.ReplaceOrErr(cl.Level, true, true) if err != nil { @@ -365,52 +326,6 @@ func (cl *CustomLog) provision(ctx Context, logging *Logging) error { return fmt.Errorf("unrecognized log level: %s", cl.Level) } - // If both Include and Exclude lists are populated, then each item must - // be a superspace or subspace of an item in the other list, because - // populating both lists means that any given item is either a rule - // or an exception to another rule. But if the item is not a super- - // or sub-space of any item in the other list, it is neither a rule - // nor an exception, and is a contradiction. Ensure, too, that the - // sets do not intersect, which is also a contradiction. - if len(cl.Include) > 0 && len(cl.Exclude) > 0 { - // prevent intersections - for _, allow := range cl.Include { - for _, deny := range cl.Exclude { - if allow == deny { - return fmt.Errorf("include and exclude must not intersect, but found %s in both lists", allow) - } - } - } - - // ensure namespaces are nested - outer: - for _, allow := range cl.Include { - for _, deny := range cl.Exclude { - if strings.HasPrefix(allow+".", deny+".") || - strings.HasPrefix(deny+".", allow+".") { - continue outer - } - } - return fmt.Errorf("when both include and exclude are populated, each element must be a superspace or subspace of one in the other list; check '%s' in include", allow) - } - } - - if cl.WriterRaw != nil { - mod, err := ctx.LoadModule(cl, "WriterRaw") - if err != nil { - return fmt.Errorf("loading log writer module: %v", err) - } - cl.writerOpener = mod.(WriterOpener) - } - if cl.writerOpener == nil { - cl.writerOpener = StderrWriter{} - } - - cl.writer, _, err = logging.openWriter(cl.writerOpener) - if err != nil { - return fmt.Errorf("opening log writer using %#v: %v", cl.writerOpener, err) - } - if cl.EncoderRaw != nil { mod, err := ctx.LoadModule(cl, "EncoderRaw") if err != nil { @@ -428,13 +343,11 @@ func (cl *CustomLog) provision(ctx Context, logging *Logging) error { } cl.encoder = newDefaultProductionLogEncoder(colorize) } - cl.buildCore() - return nil } -func (cl *CustomLog) buildCore() { +func (cl *BaseLog) buildCore() { // logs which only discard their output don't need // to perform encoding or any other processing steps // at all, so just shorcut to a nop core instead @@ -463,6 +376,83 @@ func (cl *CustomLog) buildCore() { cl.core = c } +// SinkLog configures the default Go standard library +// global logger in the log package. This is necessary because +// module dependencies which are not built specifically for +// Caddy will use the standard logger. This is also known as +// the "sink" logger. +type SinkLog struct { + BaseLog +} + +func (sll *SinkLog) provision(ctx Context, logging *Logging) error { + if err := sll.provisionCommon(ctx, logging); err != nil { + return err + } + ctx.cleanupFuncs = append(ctx.cleanupFuncs, zap.RedirectStdLog(zap.New(sll.core))) + return nil +} + +// CustomLog represents a custom logger configuration. +// +// By default, a log will emit all log entries. Some entries +// will be skipped if sampling is enabled. Further, the Include +// and Exclude parameters define which loggers (by name) are +// allowed or rejected from emitting in this log. If both Include +// and Exclude are populated, their values must be mutually +// exclusive, and longer namespaces have priority. If neither +// are populated, all logs are emitted. +type CustomLog struct { + BaseLog + + // Include defines the names of loggers to emit in this + // log. For example, to include only logs emitted by the + // admin API, you would include "admin.api". + Include []string `json:"include,omitempty"` + + // Exclude defines the names of loggers that should be + // skipped by this log. For example, to exclude only + // HTTP access logs, you would exclude "http.log.access". + Exclude []string `json:"exclude,omitempty"` +} + +func (cl *CustomLog) provision(ctx Context, logging *Logging) error { + if err := cl.provisionCommon(ctx, logging); err != nil { + return err + } + + // If both Include and Exclude lists are populated, then each item must + // be a superspace or subspace of an item in the other list, because + // populating both lists means that any given item is either a rule + // or an exception to another rule. But if the item is not a super- + // or sub-space of any item in the other list, it is neither a rule + // nor an exception, and is a contradiction. Ensure, too, that the + // sets do not intersect, which is also a contradiction. + if len(cl.Include) > 0 && len(cl.Exclude) > 0 { + // prevent intersections + for _, allow := range cl.Include { + for _, deny := range cl.Exclude { + if allow == deny { + return fmt.Errorf("include and exclude must not intersect, but found %s in both lists", allow) + } + } + } + + // ensure namespaces are nested + outer: + for _, allow := range cl.Include { + for _, deny := range cl.Exclude { + if strings.HasPrefix(allow+".", deny+".") || + strings.HasPrefix(deny+".", allow+".") { + continue outer + } + } + return fmt.Errorf("when both include and exclude are populated, each element must be a superspace or subspace of one in the other list; check '%s' in include", allow) + } + } + return nil +} + func (cl *CustomLog) matchesModule(moduleID string) bool { return cl.loggerAllowed(moduleID, true) } @@ -3,10 +3,11 @@ package caddy import ( "net/http" - "github.com/caddyserver/caddy/v2/internal/metrics" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/collectors" "github.com/prometheus/client_golang/prometheus/promauto" + + "github.com/caddyserver/caddy/v2/internal/metrics" ) // define and register the metrics used in this package. @@ -66,3 +67,9 @@ func (d *delegator) WriteHeader(code int) { d.status = code d.ResponseWriter.WriteHeader(code) } + +// Unwrap returns the underlying ResponseWriter, necessary for +// http.ResponseController to work correctly. +func (d *delegator) Unwrap() http.ResponseWriter { + return d.ResponseWriter +} @@ -333,11 +333,11 @@ func ParseStructTag(tag string) (map[string]string, error) { return results, nil } -// strictUnmarshalJSON is like json.Unmarshal but returns an error +// StrictUnmarshalJSON is like json.Unmarshal but returns an error // if any of the fields are unrecognized. Useful when decoding // module configurations, where you want to be more sure they're // correct. -func strictUnmarshalJSON(data []byte, v any) error { +func StrictUnmarshalJSON(data []byte, v any) error { dec := json.NewDecoder(bytes.NewReader(data)) dec.DisallowUnknownFields() return dec.Decode(v) diff --git a/modules/caddyevents/app.go b/modules/caddyevents/app.go index 6b10460..1684cfd 100644 --- a/modules/caddyevents/app.go +++ b/modules/caddyevents/app.go @@ -22,9 +22,10 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" "github.com/google/uuid" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { diff --git a/modules/caddyhttp/app.go b/modules/caddyhttp/app.go index 36a4011..457a5f4 100644 --- a/modules/caddyhttp/app.go +++ b/modules/caddyhttp/app.go @@ -20,16 +20,19 @@ import ( "fmt" "net" "net/http" + "runtime" "strconv" + "strings" "sync" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyevents" - "github.com/caddyserver/caddy/v2/modules/caddytls" "go.uber.org/zap" "golang.org/x/net/http2" "golang.org/x/net/http2/h2c" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyevents" + "github.com/caddyserver/caddy/v2/modules/caddytls" ) func init() { @@ -232,6 +235,11 @@ func (app *App) Provision(ctx caddy.Context) error { srv.trustedProxies = val.(IPRangeSource) } + // set the default client IP header to read from + if srv.ClientIPHeaders == nil { + srv.ClientIPHeaders = []string{"X-Forwarded-For"} + } + // process each listener address for i := range srv.Listen { lnOut, err := repl.ReplaceOrErr(srv.Listen[i], true, true) @@ -288,11 +296,19 @@ func (app *App) Provision(ctx caddy.Context) error { if srv.Errors != nil { err := srv.Errors.Routes.Provision(ctx) if err != nil { - return fmt.Errorf("server %s: setting up server error handling routes: %v", srvName, err) + return fmt.Errorf("server %s: setting up error handling routes: %v", srvName, err) } srv.errorHandlerChain = srv.Errors.Routes.Compile(errorEmptyHandler) } + // provision the named routes (they get compiled at runtime) + for name, route := range srv.NamedRoutes { + err := route.Provision(ctx, srv.Metrics) + if err != nil { + return fmt.Errorf("server %s: setting up named route '%s' handlers: %v", name, srvName, err) + } + } + // prepare the TLS connection policies err = srv.TLSConnPolicies.Provision(ctx) if err != nil { @@ -312,9 +328,15 @@ func (app *App) Provision(ctx caddy.Context) error { // Validate ensures the app's configuration is valid. func (app *App) Validate() error { + isGo120 := strings.Contains(runtime.Version(), "go1.20") + // each server must use distinct listener addresses lnAddrs := make(map[string]string) for srvName, srv := range app.Servers { + if isGo120 && srv.EnableFullDuplex { + app.logger.Warn("enable_full_duplex is not supported in Go 1.20, use a build made with Go 1.21 or later", zap.String("server", srvName)) + } + for _, addr := range srv.Listen { listenAddr, err := caddy.ParseNetworkAddress(addr) if err != nil { @@ -352,6 +374,14 @@ func (app *App) Start() error { MaxHeaderBytes: srv.MaxHeaderBytes, Handler: srv, ErrorLog: serverLogger, + ConnContext: func(ctx context.Context, c net.Conn) context.Context { + return context.WithValue(ctx, ConnCtxKey, c) + }, + } + h2server := &http2.Server{ + NewWriteScheduler: func() http2.WriteScheduler { + return http2.NewPriorityWriteScheduler(nil) + }, } // disable HTTP/2, which we enabled by default during provisioning @@ -373,6 +403,9 @@ func (app *App) Start() error { } } } + } else { + //nolint:errcheck + http2.ConfigureServer(srv.server, h2server) } // this TLS config is used by the std lib to choose the actual TLS config for connections @@ -382,9 +415,6 @@ func (app *App) Start() error { // enable H2C if configured if srv.protocol("h2c") { - h2server := &http2.Server{ - IdleTimeout: time.Duration(srv.IdleTimeout), - } srv.server.Handler = h2c.NewHandler(srv, h2server) } @@ -451,6 +481,17 @@ func (app *App) Start() error { ln = srv.listenerWrappers[i].WrapListener(ln) } + // handle http2 if use tls listener wrapper + if useTLS { + http2lnWrapper := &http2Listener{ + Listener: ln, + server: srv.server, + h2server: h2server, + } + srv.h2listeners = append(srv.h2listeners, http2lnWrapper) + ln = http2lnWrapper + } + // if binding to port 0, the OS chooses a port for us; // but the user won't know the port unless we print it if !listenAddr.IsUnixNetwork() && listenAddr.StartPort == 0 && listenAddr.EndPort == 0 { @@ -517,7 +558,7 @@ func (app *App) Stop() error { // honor scheduled/delayed shutdown time if delay { - app.logger.Debug("shutdown scheduled", + app.logger.Info("shutdown scheduled", zap.Duration("delay_duration", time.Duration(app.ShutdownDelay)), zap.Time("time", scheduledTime)) time.Sleep(time.Duration(app.ShutdownDelay)) @@ -528,9 +569,9 @@ func (app *App) Stop() error { var cancel context.CancelFunc ctx, cancel = context.WithTimeout(ctx, time.Duration(app.GracePeriod)) defer cancel() - app.logger.Debug("servers shutting down; grace period initiated", zap.Duration("duration", time.Duration(app.GracePeriod))) + app.logger.Info("servers shutting down; grace period initiated", zap.Duration("duration", time.Duration(app.GracePeriod))) } else { - app.logger.Debug("servers shutting down with eternal grace period") + app.logger.Info("servers shutting down with eternal grace period") } // goroutines aren't guaranteed to be scheduled right away, @@ -562,6 +603,21 @@ func (app *App) Stop() error { return } + // First close h3server then close listeners unlike stdlib for several reasons: + // 1, udp has only a single socket, once closed, no more data can be read and + // written. In contrast, closing tcp listeners won't affect established connections. + // This have something to do with graceful shutdown when upstream implements it. + // 2, h3server will only close listeners it's registered (quic listeners). Closing + // listener first and these listeners maybe unregistered thus won't be closed. caddy + // distinguishes quic-listener and underlying datagram sockets. + + // TODO: CloseGracefully, once implemented upstream (see https://github.com/quic-go/quic-go/issues/2103) + if err := server.h3server.Close(); err != nil { + app.logger.Error("HTTP/3 server shutdown", + zap.Error(err), + zap.Strings("addresses", server.Listen)) + } + // TODO: we have to manually close our listeners because quic-go won't // close listeners it didn't create along with the server itself... // see https://github.com/quic-go/quic-go/issues/3560 @@ -572,20 +628,26 @@ func (app *App) Stop() error { zap.String("address", el.LocalAddr().String())) } } + } + stopH2Listener := func(server *Server) { + defer finishedShutdown.Done() + startedShutdown.Done() - // TODO: CloseGracefully, once implemented upstream (see https://github.com/quic-go/quic-go/issues/2103) - if err := server.h3server.Close(); err != nil { - app.logger.Error("HTTP/3 server shutdown", - zap.Error(err), - zap.Strings("addresses", server.Listen)) + for i, s := range server.h2listeners { + if err := s.Shutdown(ctx); err != nil { + app.logger.Error("http2 listener shutdown", + zap.Error(err), + zap.Int("index", i)) + } } } for _, server := range app.Servers { - startedShutdown.Add(2) - finishedShutdown.Add(2) + startedShutdown.Add(3) + finishedShutdown.Add(3) go stopServer(server) go stopH3Server(server) + go stopH2Listener(server) } // block until all the goroutines have been run by the scheduler; diff --git a/modules/caddyhttp/autohttps.go b/modules/caddyhttp/autohttps.go index be229ea..aec43c7 100644 --- a/modules/caddyhttp/autohttps.go +++ b/modules/caddyhttp/autohttps.go @@ -20,10 +20,11 @@ import ( "strconv" "strings" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddytls" "github.com/caddyserver/certmagic" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddytls" ) // AutoHTTPSConfig is used to disable automatic HTTPS @@ -83,6 +84,8 @@ func (ahc AutoHTTPSConfig) Skipped(name string, skipSlice []string) bool { // even servers to the app, which still need to be set up with the // rest of them during provisioning. func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) error { + logger := app.logger.Named("auto_https") + // this map acts as a set to store the domain names // for which we will manage certificates automatically uniqueDomainsForCerts := make(map[string]struct{}) @@ -114,13 +117,13 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er srv.AutoHTTPS = new(AutoHTTPSConfig) } if srv.AutoHTTPS.Disabled { - app.logger.Warn("automatic HTTPS is completely disabled for server", zap.String("server_name", srvName)) + logger.Warn("automatic HTTPS is completely disabled for server", zap.String("server_name", srvName)) continue } // skip if all listeners use the HTTP port if !srv.listenersUseAnyPortOtherThan(app.httpPort()) { - app.logger.Warn("server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server", + logger.Warn("server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server", zap.String("server_name", srvName), zap.Int("http_port", app.httpPort()), ) @@ -134,7 +137,7 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er // needing to specify one empty policy to enable it if srv.TLSConnPolicies == nil && !srv.listenersUseAnyPortOtherThan(app.httpsPort()) { - app.logger.Info("server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS", + logger.Info("server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS", zap.String("server_name", srvName), zap.Int("https_port", app.httpsPort()), ) @@ -186,22 +189,16 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er // a deduplicated list of names for which to obtain certs // (only if cert management not disabled for this server) if srv.AutoHTTPS.DisableCerts { - app.logger.Warn("skipping automated certificate management for server because it is disabled", zap.String("server_name", srvName)) + logger.Warn("skipping automated certificate management for server because it is disabled", zap.String("server_name", srvName)) } else { for d := range serverDomainSet { - // the implicit Tailscale manager module will get its own certs at run-time - if isTailscaleDomain(d) { - continue - } - if certmagic.SubjectQualifiesForCert(d) && !srv.AutoHTTPS.Skipped(d, srv.AutoHTTPS.SkipCerts) { // if a certificate for this name is already loaded, // don't obtain another one for it, unless we are // supposed to ignore loaded certificates - if !srv.AutoHTTPS.IgnoreLoadedCerts && - len(app.tlsApp.AllMatchingCertificates(d)) > 0 { - app.logger.Info("skipping automatic certificate management because one or more matching certificates are already loaded", + if !srv.AutoHTTPS.IgnoreLoadedCerts && app.tlsApp.HasCertificateForSubject(d) { + logger.Info("skipping automatic certificate management because one or more matching certificates are already loaded", zap.String("domain", d), zap.String("server_name", srvName), ) @@ -212,7 +209,7 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er // can handle that, but as a courtesy, warn the user if strings.Contains(d, "*") && strings.Count(strings.Trim(d, "."), ".") == 1 { - app.logger.Warn("most clients do not trust second-level wildcard certificates (*.tld)", + logger.Warn("most clients do not trust second-level wildcard certificates (*.tld)", zap.String("domain", d)) } @@ -228,11 +225,11 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er // nothing left to do if auto redirects are disabled if srv.AutoHTTPS.DisableRedir { - app.logger.Warn("automatic HTTP->HTTPS redirects are disabled", zap.String("server_name", srvName)) + logger.Warn("automatic HTTP->HTTPS redirects are disabled", zap.String("server_name", srvName)) continue } - app.logger.Info("enabling automatic HTTP->HTTPS redirects", zap.String("server_name", srvName)) + logger.Info("enabling automatic HTTP->HTTPS redirects", zap.String("server_name", srvName)) // create HTTP->HTTPS redirects for _, listenAddr := range srv.Listen { @@ -272,12 +269,15 @@ func (app *App) automaticHTTPSPhase1(ctx caddy.Context, repl *caddy.Replacer) er // we now have a list of all the unique names for which we need certs; // turn the set into a slice so that phase 2 can use it app.allCertDomains = make([]string, 0, len(uniqueDomainsForCerts)) - var internal []string + var internal, tailscale []string uniqueDomainsLoop: for d := range uniqueDomainsForCerts { - // whether or not there is already an automation policy for this - // name, we should add it to the list to manage a cert for it - app.allCertDomains = append(app.allCertDomains, d) + if !isTailscaleDomain(d) { + // whether or not there is already an automation policy for this + // name, we should add it to the list to manage a cert for it, + // unless it's a Tailscale domain, because we don't manage those + app.allCertDomains = append(app.allCertDomains, d) + } // some names we've found might already have automation policies // explicitly specified for them; we should exclude those from @@ -285,7 +285,7 @@ uniqueDomainsLoop: // one automation policy would be confusing and an error if app.tlsApp.Automation != nil { for _, ap := range app.tlsApp.Automation.Policies { - for _, apHost := range ap.Subjects { + for _, apHost := range ap.Subjects() { if apHost == d { continue uniqueDomainsLoop } @@ -295,13 +295,15 @@ uniqueDomainsLoop: // if no automation policy exists for the name yet, we // will associate it with an implicit one - if !certmagic.SubjectQualifiesForPublicCert(d) { + if isTailscaleDomain(d) { + tailscale = append(tailscale, d) + } else if !certmagic.SubjectQualifiesForPublicCert(d) { internal = append(internal, d) } } // ensure there is an automation policy to handle these certs - err := app.createAutomationPolicies(ctx, internal) + err := app.createAutomationPolicies(ctx, internal, tailscale) if err != nil { return err } @@ -424,6 +426,10 @@ redirServersLoop: } } + logger.Debug("adjusted config", + zap.Reflect("tls", app.tlsApp), + zap.Reflect("http", app)) + return nil } @@ -466,7 +472,7 @@ func (app *App) makeRedirRoute(redirToPort uint, matcherSet MatcherSet) Route { // automation policy exists, it will be shallow-copied and used as the // base for the new ones (this is important for preserving behavior the // user intends to be "defaults"). -func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []string) error { +func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames, tailscaleNames []string) error { // before we begin, loop through the existing automation policies // and, for any ACMEIssuers we find, make sure they're filled in // with default values that might be specified in our HTTP app; also @@ -480,6 +486,22 @@ func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []stri app.tlsApp.Automation = new(caddytls.AutomationConfig) } for _, ap := range app.tlsApp.Automation.Policies { + // on-demand policies can have the tailscale manager added implicitly + // if there's no explicit manager configured -- for convenience + if ap.OnDemand && len(ap.Managers) == 0 { + var ts caddytls.Tailscale + if err := ts.Provision(ctx); err != nil { + return err + } + ap.Managers = []certmagic.Manager{ts} + + // must reprovision the automation policy so that the underlying + // CertMagic config knows about the updated Managers + if err := ap.Provision(app.tlsApp); err != nil { + return fmt.Errorf("re-provisioning automation policy: %v", err) + } + } + // set up default issuer -- honestly, this is only // really necessary because the HTTP app is opinionated // and has settings which could be inferred as new @@ -501,24 +523,8 @@ func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []stri } } - // if no external managers were configured, enable - // implicit Tailscale support for convenience - if ap.Managers == nil { - ts, err := implicitTailscale(ctx) - if err != nil { - return err - } - ap.Managers = []certmagic.Manager{ts} - - // must reprovision the automation policy so that the underlying - // CertMagic config knows about the updated Managers - if err := ap.Provision(app.tlsApp); err != nil { - return fmt.Errorf("re-provisioning automation policy: %v", err) - } - } - // while we're here, is this the catch-all/base policy? - if !foundBasePolicy && len(ap.Subjects) == 0 { + if !foundBasePolicy && len(ap.SubjectsRaw) == 0 { basePolicy = ap foundBasePolicy = true } @@ -529,15 +535,6 @@ func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []stri basePolicy = new(caddytls.AutomationPolicy) } - if basePolicy.Managers == nil { - // add implicit Tailscale integration, for harmless convenience - ts, err := implicitTailscale(ctx) - if err != nil { - return err - } - basePolicy.Managers = []certmagic.Manager{ts} - } - // if the basePolicy has an existing ACMEIssuer (particularly to // include any type that embeds/wraps an ACMEIssuer), let's use it // (I guess we just use the first one?), otherwise we'll make one @@ -634,7 +631,7 @@ func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []stri // rather they just want to change the CA for the set // of names that would normally use the production API; // anyway, that gets into the weeds a bit... - newPolicy.Subjects = internalNames + newPolicy.SubjectsRaw = internalNames newPolicy.Issuers = []certmagic.Issuer{internalIssuer} err := app.tlsApp.AddAutomationPolicy(newPolicy) if err != nil { @@ -642,6 +639,27 @@ func (app *App) createAutomationPolicies(ctx caddy.Context, internalNames []stri } } + // tailscale names go in their own automation policies because + // they require on-demand TLS to be enabled, which we obviously + // can't enable for everything + if len(tailscaleNames) > 0 { + policyCopy := *basePolicy + newPolicy := &policyCopy + + var ts caddytls.Tailscale + if err := ts.Provision(ctx); err != nil { + return err + } + + newPolicy.SubjectsRaw = tailscaleNames + newPolicy.Issuers = nil + newPolicy.Managers = append(newPolicy.Managers, ts) + err := app.tlsApp.AddAutomationPolicy(newPolicy) + if err != nil { + return err + } + } + // we just changed a lot of stuff, so double-check that it's all good err := app.tlsApp.Validate() if err != nil { @@ -720,13 +738,6 @@ func (app *App) automaticHTTPSPhase2() error { return nil } -// implicitTailscale returns a new and provisioned Tailscale module configured to be optional. -func implicitTailscale(ctx caddy.Context) (caddytls.Tailscale, error) { - ts := caddytls.Tailscale{Optional: true} - err := ts.Provision(ctx) - return ts, err -} - func isTailscaleDomain(name string) bool { return strings.HasSuffix(strings.ToLower(name), ".ts.net") } diff --git a/modules/caddyhttp/caddyauth/basicauth.go b/modules/caddyhttp/caddyauth/basicauth.go index f515a72..f30a869 100644 --- a/modules/caddyhttp/caddyauth/basicauth.go +++ b/modules/caddyhttp/caddyauth/basicauth.go @@ -23,16 +23,14 @@ import ( "net/http" "strings" "sync" - "time" - "github.com/caddyserver/caddy/v2" "golang.org/x/sync/singleflight" + + "github.com/caddyserver/caddy/v2" ) func init() { caddy.RegisterModule(HTTPBasicAuth{}) - - weakrand.Seed(time.Now().UnixNano()) } // HTTPBasicAuth facilitates HTTP basic authentication. diff --git a/modules/caddyhttp/caddyauth/caddyauth.go b/modules/caddyhttp/caddyauth/caddyauth.go index b2bdbc2..c60de88 100644 --- a/modules/caddyhttp/caddyauth/caddyauth.go +++ b/modules/caddyhttp/caddyauth/caddyauth.go @@ -18,9 +18,10 @@ import ( "fmt" "net/http" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "go.uber.org/zap" ) func init() { diff --git a/modules/caddyhttp/caddyauth/command.go b/modules/caddyhttp/caddyauth/command.go index 609de4e..b93b7a4 100644 --- a/modules/caddyhttp/caddyauth/command.go +++ b/modules/caddyhttp/caddyauth/command.go @@ -18,20 +18,21 @@ import ( "bufio" "bytes" "encoding/base64" - "flag" "fmt" "os" "os/signal" - "github.com/caddyserver/caddy/v2" - caddycmd "github.com/caddyserver/caddy/v2/cmd" + "github.com/spf13/cobra" "golang.org/x/term" + + caddycmd "github.com/caddyserver/caddy/v2/cmd" + + "github.com/caddyserver/caddy/v2" ) func init() { caddycmd.RegisterCommand(caddycmd.Command{ Name: "hash-password", - Func: cmdHashPassword, Usage: "[--algorithm <name>] [--salt <string>] [--plaintext <password>]", Short: "Hashes a password and writes base64", Long: ` @@ -50,13 +51,12 @@ be provided (scrypt). Note that scrypt is deprecated. Please use 'bcrypt' instead. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("hash-password", flag.ExitOnError) - fs.String("algorithm", "bcrypt", "Name of the hash algorithm") - fs.String("plaintext", "", "The plaintext password") - fs.String("salt", "", "The password salt") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("plaintext", "p", "", "The plaintext password") + cmd.Flags().StringP("salt", "s", "", "The password salt") + cmd.Flags().StringP("algorithm", "a", "bcrypt", "Name of the hash algorithm") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdHashPassword) + }, }) } diff --git a/modules/caddyhttp/caddyauth/hashes.go b/modules/caddyhttp/caddyauth/hashes.go index 6a651f0..324cf1e 100644 --- a/modules/caddyhttp/caddyauth/hashes.go +++ b/modules/caddyhttp/caddyauth/hashes.go @@ -18,9 +18,10 @@ import ( "crypto/subtle" "encoding/base64" - "github.com/caddyserver/caddy/v2" "golang.org/x/crypto/bcrypt" "golang.org/x/crypto/scrypt" + + "github.com/caddyserver/caddy/v2" ) func init() { diff --git a/modules/caddyhttp/caddyhttp.go b/modules/caddyhttp/caddyhttp.go index c497dc7..f15aec5 100644 --- a/modules/caddyhttp/caddyhttp.go +++ b/modules/caddyhttp/caddyhttp.go @@ -307,5 +307,7 @@ const ( const separator = string(filepath.Separator) // Interface guard -var _ caddy.ListenerWrapper = (*tlsPlaceholderWrapper)(nil) -var _ caddyfile.Unmarshaler = (*tlsPlaceholderWrapper)(nil) +var ( + _ caddy.ListenerWrapper = (*tlsPlaceholderWrapper)(nil) + _ caddyfile.Unmarshaler = (*tlsPlaceholderWrapper)(nil) +) diff --git a/modules/caddyhttp/celmatcher.go b/modules/caddyhttp/celmatcher.go index 60ca00b..e997336 100644 --- a/modules/caddyhttp/celmatcher.go +++ b/modules/caddyhttp/celmatcher.go @@ -25,8 +25,6 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/google/cel-go/cel" "github.com/google/cel-go/common" "github.com/google/cel-go/common/operators" @@ -39,6 +37,9 @@ import ( "github.com/google/cel-go/parser" "go.uber.org/zap" exprpb "google.golang.org/genproto/googleapis/api/expr/v1alpha1" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { @@ -191,15 +192,17 @@ func (m MatchExpression) caddyPlaceholderFunc(lhs, rhs ref.Val) ref.Val { celReq, ok := lhs.(celHTTPRequest) if !ok { return types.NewErr( - "invalid request of type '%v' to "+placeholderFuncName+"(request, placeholderVarName)", + "invalid request of type '%v' to %s(request, placeholderVarName)", lhs.Type(), + placeholderFuncName, ) } phStr, ok := rhs.(types.String) if !ok { return types.NewErr( - "invalid placeholder variable name of type '%v' to "+placeholderFuncName+"(request, placeholderVarName)", + "invalid placeholder variable name of type '%v' to %s(request, placeholderVarName)", rhs.Type(), + placeholderFuncName, ) } @@ -232,9 +235,11 @@ func (cr celHTTPRequest) Parent() interpreter.Activation { func (cr celHTTPRequest) ConvertToNative(typeDesc reflect.Type) (any, error) { return cr.Request, nil } + func (celHTTPRequest) ConvertToType(typeVal ref.Type) ref.Val { panic("not implemented") } + func (cr celHTTPRequest) Equal(other ref.Val) ref.Val { if o, ok := other.Value().(celHTTPRequest); ok { return types.Bool(o.Request == cr.Request) @@ -253,9 +258,14 @@ type celPkixName struct{ *pkix.Name } func (pn celPkixName) ConvertToNative(typeDesc reflect.Type) (any, error) { return pn.Name, nil } -func (celPkixName) ConvertToType(typeVal ref.Type) ref.Val { + +func (pn celPkixName) ConvertToType(typeVal ref.Type) ref.Val { + if typeVal.TypeName() == "string" { + return types.String(pn.Name.String()) + } panic("not implemented") } + func (pn celPkixName) Equal(other ref.Val) ref.Val { if o, ok := other.Value().(string); ok { return types.Bool(pn.Name.String() == o) @@ -491,7 +501,7 @@ func celMatcherStringMacroExpander(funcName string) parser.MacroExpander { } } -// celMatcherStringMacroExpander validates that the macro is called a single +// celMatcherJSONMacroExpander validates that the macro is called a single // map literal argument. // // The following function call is returned: <funcName>(request, arg) diff --git a/modules/caddyhttp/duplex_go120.go b/modules/caddyhttp/duplex_go120.go new file mode 100644 index 0000000..065ccf2 --- /dev/null +++ b/modules/caddyhttp/duplex_go120.go @@ -0,0 +1,26 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +//go:build !go1.21 + +package caddyhttp + +import ( + "net/http" +) + +func enableFullDuplex(w http.ResponseWriter) error { + // Do nothing, Go 1.20 and earlier do not support full duplex + return nil +} diff --git a/modules/caddyhttp/duplex_go121.go b/modules/caddyhttp/duplex_go121.go new file mode 100644 index 0000000..a17d3af --- /dev/null +++ b/modules/caddyhttp/duplex_go121.go @@ -0,0 +1,26 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +//go:build go1.21 + +package caddyhttp + +import ( + "net/http" +) + +func enableFullDuplex(w http.ResponseWriter) error { + //nolint:bodyclose + return http.NewResponseController(w).EnableFullDuplex() +} diff --git a/modules/caddyhttp/encode/encode.go b/modules/caddyhttp/encode/encode.go index e3a6267..dc35fa2 100644 --- a/modules/caddyhttp/encode/encode.go +++ b/modules/caddyhttp/encode/encode.go @@ -20,9 +20,11 @@ package encode import ( + "bufio" "fmt" "io" "math" + "net" "net/http" "sort" "strconv" @@ -91,6 +93,7 @@ func (enc *Encode) Provision(ctx caddy.Context) error { "application/xhtml+xml*", "application/atom+xml*", "application/rss+xml*", + "application/wasm*", "image/svg+xml*", }, }, @@ -165,10 +168,10 @@ func (enc *Encode) openResponseWriter(encodingName string, w http.ResponseWriter // initResponseWriter initializes the responseWriter instance // allocated in openResponseWriter, enabling mid-stack inlining. func (enc *Encode) initResponseWriter(rw *responseWriter, encodingName string, wrappedRW http.ResponseWriter) *responseWriter { - if httpInterfaces, ok := wrappedRW.(caddyhttp.HTTPInterfaces); ok { - rw.HTTPInterfaces = httpInterfaces + if rww, ok := wrappedRW.(*caddyhttp.ResponseWriterWrapper); ok { + rw.ResponseWriter = rww } else { - rw.HTTPInterfaces = &caddyhttp.ResponseWriterWrapper{ResponseWriter: wrappedRW} + rw.ResponseWriter = &caddyhttp.ResponseWriterWrapper{ResponseWriter: wrappedRW} } rw.encodingName = encodingName rw.config = enc @@ -180,7 +183,7 @@ func (enc *Encode) initResponseWriter(rw *responseWriter, encodingName string, w // using the encoding represented by encodingName and // configured by config. type responseWriter struct { - caddyhttp.HTTPInterfaces + http.ResponseWriter encodingName string w Encoder config *Encode @@ -209,7 +212,21 @@ func (rw *responseWriter) Flush() { // to rw.Write (see bug in #4314) return } - rw.HTTPInterfaces.Flush() + //nolint:bodyclose + http.NewResponseController(rw.ResponseWriter).Flush() +} + +// Hijack implements http.Hijacker. It will flush status code if set. We don't track actual hijacked +// status assuming http middlewares will track its status. +func (rw *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) { + if !rw.wroteHeader { + if rw.statusCode != 0 { + rw.ResponseWriter.WriteHeader(rw.statusCode) + } + rw.wroteHeader = true + } + //nolint:bodyclose + return http.NewResponseController(rw.ResponseWriter).Hijack() } // Write writes to the response. If the response qualifies, @@ -246,7 +263,7 @@ func (rw *responseWriter) Write(p []byte) (int, error) { // by the standard library if !rw.wroteHeader { if rw.statusCode != 0 { - rw.HTTPInterfaces.WriteHeader(rw.statusCode) + rw.ResponseWriter.WriteHeader(rw.statusCode) } rw.wroteHeader = true } @@ -254,7 +271,7 @@ func (rw *responseWriter) Write(p []byte) (int, error) { if rw.w != nil { return rw.w.Write(p) } else { - return rw.HTTPInterfaces.Write(p) + return rw.ResponseWriter.Write(p) } } @@ -270,7 +287,7 @@ func (rw *responseWriter) Close() error { // issue #5059, don't write status code if not set explicitly. if rw.statusCode != 0 { - rw.HTTPInterfaces.WriteHeader(rw.statusCode) + rw.ResponseWriter.WriteHeader(rw.statusCode) } rw.wroteHeader = true } @@ -285,13 +302,18 @@ func (rw *responseWriter) Close() error { return err } +// Unwrap returns the underlying ResponseWriter. +func (rw *responseWriter) Unwrap() http.ResponseWriter { + return rw.ResponseWriter +} + // init should be called before we write a response, if rw.buf has contents. func (rw *responseWriter) init() { if rw.Header().Get("Content-Encoding") == "" && isEncodeAllowed(rw.Header()) && rw.config.Match(rw) { rw.w = rw.config.writerPools[rw.encodingName].Get().(Encoder) - rw.w.Reset(rw.HTTPInterfaces) + rw.w.Reset(rw.ResponseWriter) rw.Header().Del("Content-Length") // https://github.com/golang/go/issues/14975 rw.Header().Set("Content-Encoding", rw.encodingName) rw.Header().Add("Vary", "Accept-Encoding") @@ -410,5 +432,4 @@ var ( _ caddy.Provisioner = (*Encode)(nil) _ caddy.Validator = (*Encode)(nil) _ caddyhttp.MiddlewareHandler = (*Encode)(nil) - _ caddyhttp.HTTPInterfaces = (*responseWriter)(nil) ) diff --git a/modules/caddyhttp/encode/gzip/gzip.go b/modules/caddyhttp/encode/gzip/gzip.go index 0212583..0af38b9 100644 --- a/modules/caddyhttp/encode/gzip/gzip.go +++ b/modules/caddyhttp/encode/gzip/gzip.go @@ -18,10 +18,11 @@ import ( "fmt" "strconv" + "github.com/klauspost/compress/gzip" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode" - "github.com/klauspost/compress/gzip" ) func init() { diff --git a/modules/caddyhttp/encode/zstd/zstd.go b/modules/caddyhttp/encode/zstd/zstd.go index 3da9b13..b5a0299 100644 --- a/modules/caddyhttp/encode/zstd/zstd.go +++ b/modules/caddyhttp/encode/zstd/zstd.go @@ -15,10 +15,11 @@ package caddyzstd import ( + "github.com/klauspost/compress/zstd" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode" - "github.com/klauspost/compress/zstd" ) func init() { diff --git a/modules/caddyhttp/errors.go b/modules/caddyhttp/errors.go index 9d1cf47..fc8ffbf 100644 --- a/modules/caddyhttp/errors.go +++ b/modules/caddyhttp/errors.go @@ -15,27 +15,24 @@ package caddyhttp import ( + "errors" "fmt" weakrand "math/rand" "path" "runtime" "strings" - "time" "github.com/caddyserver/caddy/v2" ) -func init() { - weakrand.Seed(time.Now().UnixNano()) -} - // Error is a convenient way for a Handler to populate the // essential fields of a HandlerError. If err is itself a // HandlerError, then any essential fields that are not // set will be populated. func Error(statusCode int, err error) HandlerError { const idLen = 9 - if he, ok := err.(HandlerError); ok { + var he HandlerError + if errors.As(err, &he) { if he.ID == "" { he.ID = randString(idLen, true) } diff --git a/modules/caddyhttp/fileserver/browse.go b/modules/caddyhttp/fileserver/browse.go index a8f5e8a..81eb085 100644 --- a/modules/caddyhttp/fileserver/browse.go +++ b/modules/caddyhttp/fileserver/browse.go @@ -29,18 +29,25 @@ import ( "sync" "text/template" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/templates" - "go.uber.org/zap" ) +// BrowseTemplate is the default template document to use for +// file listings. By default, its default value is an embedded +// document. You can override this value at program start, or +// if you are running Caddy via config, you can specify a +// custom template_file in the browse configuration. +// //go:embed browse.html -var defaultBrowseTemplate string +var BrowseTemplate string // Browse configures directory browsing. type Browse struct { - // Use this template file instead of the default browse template. + // Filename of the template to use instead of the embedded browse template. TemplateFile string `json:"template_file,omitempty"` } @@ -82,8 +89,8 @@ func (fsrv *FileServer) serveBrowse(root, dirPath string, w http.ResponseWriter, repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer) - // calling path.Clean here prevents weird breadcrumbs when URL paths are sketchy like /%2e%2e%2f - listing, err := fsrv.loadDirectoryContents(r.Context(), dir.(fs.ReadDirFile), root, path.Clean(r.URL.Path), repl) + // TODO: not entirely sure if path.Clean() is necessary here but seems like a safe plan (i.e. /%2e%2e%2f) - someone could verify this + listing, err := fsrv.loadDirectoryContents(r.Context(), dir.(fs.ReadDirFile), root, path.Clean(r.URL.EscapedPath()), repl) switch { case os.IsPermission(err): return caddyhttp.Error(http.StatusForbidden, err) @@ -93,7 +100,7 @@ func (fsrv *FileServer) serveBrowse(root, dirPath string, w http.ResponseWriter, return caddyhttp.Error(http.StatusInternalServerError, err) } - fsrv.browseApplyQueryParams(w, r, &listing) + fsrv.browseApplyQueryParams(w, r, listing) buf := bufPool.Get().(*bytes.Buffer) buf.Reset() @@ -113,7 +120,7 @@ func (fsrv *FileServer) serveBrowse(root, dirPath string, w http.ResponseWriter, fs = http.Dir(repl.ReplaceAll(fsrv.Root, ".")) } - var tplCtx = &templateContext{ + tplCtx := &templateContext{ TemplateContext: templates.TemplateContext{ Root: fs, Req: r, @@ -137,10 +144,10 @@ func (fsrv *FileServer) serveBrowse(root, dirPath string, w http.ResponseWriter, return nil } -func (fsrv *FileServer) loadDirectoryContents(ctx context.Context, dir fs.ReadDirFile, root, urlPath string, repl *caddy.Replacer) (browseTemplateContext, error) { +func (fsrv *FileServer) loadDirectoryContents(ctx context.Context, dir fs.ReadDirFile, root, urlPath string, repl *caddy.Replacer) (*browseTemplateContext, error) { files, err := dir.ReadDir(10000) // TODO: this limit should probably be configurable if err != nil && err != io.EOF { - return browseTemplateContext{}, err + return nil, err } // user can presumably browse "up" to parent folder if path is longer than "/" @@ -152,12 +159,20 @@ func (fsrv *FileServer) loadDirectoryContents(ctx context.Context, dir fs.ReadDi // browseApplyQueryParams applies query parameters to the listing. // It mutates the listing and may set cookies. func (fsrv *FileServer) browseApplyQueryParams(w http.ResponseWriter, r *http.Request, listing *browseTemplateContext) { + layoutParam := r.URL.Query().Get("layout") sortParam := r.URL.Query().Get("sort") orderParam := r.URL.Query().Get("order") limitParam := r.URL.Query().Get("limit") offsetParam := r.URL.Query().Get("offset") - // first figure out what to sort by + switch layoutParam { + case "list", "grid", "": + listing.Layout = layoutParam + default: + listing.Layout = "list" + } + + // figure out what to sort by switch sortParam { case "": sortParam = sortByNameDirFirst @@ -196,7 +211,7 @@ func (fsrv *FileServer) makeBrowseTemplate(tplCtx *templateContext) (*template.T } } else { tpl = tplCtx.NewTemplate("default_listing") - tpl, err = tpl.Parse(defaultBrowseTemplate) + tpl, err = tpl.Parse(BrowseTemplate) if err != nil { return nil, fmt.Errorf("parsing default browse template: %v", err) } @@ -229,7 +244,7 @@ func isSymlink(f fs.FileInfo) bool { // features. type templateContext struct { templates.TemplateContext - browseTemplateContext + *browseTemplateContext } // bufPool is used to increase the efficiency of file listings. diff --git a/modules/caddyhttp/fileserver/browse.html b/modules/caddyhttp/fileserver/browse.html index 01537fc..1c8be7f 100644 --- a/modules/caddyhttp/fileserver/browse.html +++ b/modules/caddyhttp/fileserver/browse.html @@ -1,42 +1,376 @@ +{{- define "icon"}} + {{- if .IsDir}} + {{- if .IsSymlink}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-folder-filled" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M9 3a1 1 0 0 1 .608 .206l.1 .087l2.706 2.707h6.586a3 3 0 0 1 2.995 2.824l.005 .176v8a3 3 0 0 1 -2.824 2.995l-.176 .005h-14a3 3 0 0 1 -2.995 -2.824l-.005 -.176v-11a3 3 0 0 1 2.824 -2.995l.176 -.005h4z" stroke-width="0" fill="currentColor"/> + <path fill="#000" d="M2.795 17.306c0-2.374 1.792-4.314 4.078-4.538v-1.104a.38.38 0 0 1 .651-.272l2.45 2.492a.132.132 0 0 1 0 .188l-2.45 2.492a.381.381 0 0 1-.651-.272V15.24c-1.889.297-3.436 1.39-3.817 3.26a2.809 2.809 0 0 1-.261-1.193Z" style="stroke-width:.127478"/> + </svg> + {{- else}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-folder-filled" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M9 3a1 1 0 0 1 .608 .206l.1 .087l2.706 2.707h6.586a3 3 0 0 1 2.995 2.824l.005 .176v8a3 3 0 0 1 -2.824 2.995l-.176 .005h-14a3 3 0 0 1 -2.995 -2.824l-.005 -.176v-11a3 3 0 0 1 2.824 -2.995l.176 -.005h4z" stroke-width="0" fill="currentColor"/> + </svg> + {{- end}} + {{- else if or (eq .Name "LICENSE") (eq .Name "README")}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-license" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M15 21h-9a3 3 0 0 1 -3 -3v-1h10v2a2 2 0 0 0 4 0v-14a2 2 0 1 1 2 2h-2m2 -4h-11a3 3 0 0 0 -3 3v11"/> + <path d="M9 7l4 0"/> + <path d="M9 11l4 0"/> + </svg> + {{- else if .HasExt ".jpg" ".jpeg" ".png" ".gif" ".webp" ".tiff" ".bmp" ".heif" ".heic" ".svg"}} + {{- if eq .Tpl.Layout "grid"}} + <img loading="lazy" src="{{html .Name}}"> + {{- else}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-photo" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M15 8h.01"/> + <path d="M3 6a3 3 0 0 1 3 -3h12a3 3 0 0 1 3 3v12a3 3 0 0 1 -3 3h-12a3 3 0 0 1 -3 -3v-12z"/> + <path d="M3 16l5 -5c.928 -.893 2.072 -.893 3 0l5 5"/> + <path d="M14 14l1 -1c.928 -.893 2.072 -.893 3 0l3 3"/> + </svg> + {{- end}} + {{- else if .HasExt ".mp4" ".mov" ".mpeg" ".mpg" ".avi" ".ogg" ".webm" ".mkv" ".vob" ".gifv" ".3gp"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-movie" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M4 4m0 2a2 2 0 0 1 2 -2h12a2 2 0 0 1 2 2v12a2 2 0 0 1 -2 2h-12a2 2 0 0 1 -2 -2z"/> + <path d="M8 4l0 16"/> + <path d="M16 4l0 16"/> + <path d="M4 8l4 0"/> + <path d="M4 16l4 0"/> + <path d="M4 12l16 0"/> + <path d="M16 8l4 0"/> + <path d="M16 16l4 0"/> + </svg> + {{- else if .HasExt ".mp3" ".m4a" ".aac" ".ogg" ".flac" ".wav" ".wma" ".midi" ".cda"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-music" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 17m-3 0a3 3 0 1 0 6 0a3 3 0 1 0 -6 0"/> + <path d="M16 17m-3 0a3 3 0 1 0 6 0a3 3 0 1 0 -6 0"/> + <path d="M9 17l0 -13l10 0l0 13"/> + <path d="M9 8l10 0"/> + </svg> + {{- else if .HasExt ".pdf"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-pdf" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/> + <path d="M5 18h1.5a1.5 1.5 0 0 0 0 -3h-1.5v6"/> + <path d="M17 18h2"/> + <path d="M20 15h-3v6"/> + <path d="M11 15v6h1a2 2 0 0 0 2 -2v-2a2 2 0 0 0 -2 -2h-1z"/> + </svg> + {{- else if .HasExt ".csv" ".tsv"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-csv" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/> + <path d="M7 16.5a1.5 1.5 0 0 0 -3 0v3a1.5 1.5 0 0 0 3 0"/> + <path d="M10 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + <path d="M16 15l2 6l2 -6"/> + </svg> + {{- else if .HasExt ".txt" ".doc" ".docx" ".odt" ".fodt" ".rtf"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-text" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/> + <path d="M9 9l1 0"/> + <path d="M9 13l6 0"/> + <path d="M9 17l6 0"/> + </svg> + {{- else if .HasExt ".xls" ".xlsx" ".ods" ".fods"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-spreadsheet" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/> + <path d="M8 11h8v7h-8z"/> + <path d="M8 15h8"/> + <path d="M11 11v7"/> + </svg> + {{- else if .HasExt ".ppt" ".pptx" ".odp" ".fodp"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-presentation-analytics" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M9 12v-4"/> + <path d="M15 12v-2"/> + <path d="M12 12v-1"/> + <path d="M3 4h18"/> + <path d="M4 4v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2 -2v-10"/> + <path d="M12 16v4"/> + <path d="M9 20h6"/> + </svg> + {{- else if .HasExt ".zip" ".gz" ".xz" ".tar" ".7z" ".rar" ".xz" ".zst"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-zip" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 20.735a2 2 0 0 1 -1 -1.735v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2h-1"/> + <path d="M11 17a2 2 0 0 1 2 2v2a1 1 0 0 1 -1 1h-2a1 1 0 0 1 -1 -1v-2a2 2 0 0 1 2 -2z"/> + <path d="M11 5l-1 0"/> + <path d="M13 7l-1 0"/> + <path d="M11 9l-1 0"/> + <path d="M13 11l-1 0"/> + <path d="M11 13l-1 0"/> + <path d="M13 15l-1 0"/> + </svg> + {{- else if .HasExt ".deb" ".dpkg"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-debian" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M12 17c-2.397 -.943 -4 -3.153 -4 -5.635c0 -2.19 1.039 -3.14 1.604 -3.595c2.646 -2.133 6.396 -.27 6.396 3.23c0 2.5 -2.905 2.121 -3.5 1.5c-.595 -.621 -1 -1.5 -.5 -2.5"/> + <path d="M12 12m-9 0a9 9 0 1 0 18 0a9 9 0 1 0 -18 0"/> + </svg> + {{- else if .HasExt ".rpm" ".exe" ".flatpak" ".appimage" ".jar" ".msi" ".apk"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-package" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M12 3l8 4.5l0 9l-8 4.5l-8 -4.5l0 -9l8 -4.5"/> + <path d="M12 12l8 -4.5"/> + <path d="M12 12l0 9"/> + <path d="M12 12l-8 -4.5"/> + <path d="M16 5.25l-8 4.5"/> + </svg> + {{- else if .HasExt ".ps1"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-powershell" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M4.887 20h11.868c.893 0 1.664 -.665 1.847 -1.592l2.358 -12c.212 -1.081 -.442 -2.14 -1.462 -2.366a1.784 1.784 0 0 0 -.385 -.042h-11.868c-.893 0 -1.664 .665 -1.847 1.592l-2.358 12c-.212 1.081 .442 2.14 1.462 2.366c.127 .028 .256 .042 .385 .042z"/> + <path d="M9 8l4 4l-6 4"/> + <path d="M12 16h3"/> + </svg> + {{- else if .HasExt ".py" ".pyc" ".pyo"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-python" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M12 9h-7a2 2 0 0 0 -2 2v4a2 2 0 0 0 2 2h3"/> + <path d="M12 15h7a2 2 0 0 0 2 -2v-4a2 2 0 0 0 -2 -2h-3"/> + <path d="M8 9v-4a2 2 0 0 1 2 -2h4a2 2 0 0 1 2 2v5a2 2 0 0 1 -2 2h-4a2 2 0 0 0 -2 2v5a2 2 0 0 0 2 2h4a2 2 0 0 0 2 -2v-4"/> + <path d="M11 6l0 .01"/> + <path d="M13 18l0 .01"/> + </svg> + {{- else if .HasExt ".bash" ".sh" ".com" ".bat" ".dll" ".so"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-script" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M17 20h-11a3 3 0 0 1 0 -6h11a3 3 0 0 0 0 6h1a3 3 0 0 0 3 -3v-11a2 2 0 0 0 -2 -2h-10a2 2 0 0 0 -2 2v8"/> + </svg> + {{- else if .HasExt ".dmg"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-finder" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M3 4m0 1a1 1 0 0 1 1 -1h16a1 1 0 0 1 1 1v14a1 1 0 0 1 -1 1h-16a1 1 0 0 1 -1 -1z"/> + <path d="M7 8v1"/> + <path d="M17 8v1"/> + <path d="M12.5 4c-.654 1.486 -1.26 3.443 -1.5 9h2.5c-.19 2.867 .094 5.024 .5 7"/> + <path d="M7 15.5c3.667 2 6.333 2 10 0"/> + </svg> + {{- else if .HasExt ".iso" ".img"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-disc" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M12 12m-9 0a9 9 0 1 0 18 0a9 9 0 1 0 -18 0"/> + <path d="M12 12m-1 0a1 1 0 1 0 2 0a1 1 0 1 0 -2 0"/> + <path d="M7 12a5 5 0 0 1 5 -5"/> + <path d="M12 17a5 5 0 0 0 5 -5"/> + </svg> + {{- else if .HasExt ".md" ".mdown" ".markdown"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-markdown" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M3 5m0 2a2 2 0 0 1 2 -2h14a2 2 0 0 1 2 2v10a2 2 0 0 1 -2 2h-14a2 2 0 0 1 -2 -2z"/> + <path d="M7 15v-6l2 2l2 -2v6"/> + <path d="M14 13l2 2l2 -2m-2 2v-6"/> + </svg> + {{- else if .HasExt ".ttf" ".otf" ".woff" ".woff2" ".eof"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-typography" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/> + <path d="M11 18h2"/> + <path d="M12 18v-7"/> + <path d="M9 12v-1h6v1"/> + </svg> + {{- else if .HasExt ".go"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-golang" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M15.695 14.305c1.061 1.06 2.953 .888 4.226 -.384c1.272 -1.273 1.444 -3.165 .384 -4.226c-1.061 -1.06 -2.953 -.888 -4.226 .384c-1.272 1.273 -1.444 3.165 -.384 4.226z"/> + <path d="M12.68 9.233c-1.084 -.497 -2.545 -.191 -3.591 .846c-1.284 1.273 -1.457 3.165 -.388 4.226c1.07 1.06 2.978 .888 4.261 -.384a3.669 3.669 0 0 0 1.038 -1.921h-2.427"/> + <path d="M5.5 15h-1.5"/> + <path d="M6 9h-2"/> + <path d="M5 12h-3"/> + </svg> + {{- else if .HasExt ".html" ".htm"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-html" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/> + <path d="M2 21v-6"/> + <path d="M5 15v6"/> + <path d="M2 18h3"/> + <path d="M20 15v6h2"/> + <path d="M13 21v-6l2 3l2 -3v6"/> + <path d="M7.5 15h3"/> + <path d="M9 15v6"/> + </svg> + {{- else if .HasExt ".js"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-js" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M3 15h3v4.5a1.5 1.5 0 0 1 -3 0"/> + <path d="M9 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2h-1"/> + </svg> + {{- else if .HasExt ".css"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-css" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/> + <path d="M8 16.5a1.5 1.5 0 0 0 -3 0v3a1.5 1.5 0 0 0 3 0"/> + <path d="M11 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + <path d="M17 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + </svg> + {{- else if .HasExt ".json" ".json5" ".jsonc"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-json" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M20 16v-8l3 8v-8"/> + <path d="M15 8a2 2 0 0 1 2 2v4a2 2 0 1 1 -4 0v-4a2 2 0 0 1 2 -2z"/> + <path d="M1 8h3v6.5a1.5 1.5 0 0 1 -3 0v-.5"/> + <path d="M7 15a1 1 0 0 0 1 1h1a1 1 0 0 0 1 -1v-2a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-2a1 1 0 0 1 1 -1h1a1 1 0 0 1 1 1"/> + </svg> + {{- else if .HasExt ".ts"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-ts" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2h-1"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M9 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + <path d="M3.5 15h3"/> + <path d="M5 15v6"/> + </svg> + {{- else if .HasExt ".sql"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-type-sql" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 20.25c0 .414 .336 .75 .75 .75h1.25a1 1 0 0 0 1 -1v-1a1 1 0 0 0 -1 -1h-1a1 1 0 0 1 -1 -1v-1a1 1 0 0 1 1 -1h1.25a.75 .75 0 0 1 .75 .75"/> + <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/> + <path d="M18 15v6h2"/> + <path d="M13 15a2 2 0 0 1 2 2v2a2 2 0 1 1 -4 0v-2a2 2 0 0 1 2 -2z"/> + <path d="M14 20l1.5 1.5"/> + </svg> + {{- else if .HasExt ".db" ".sqlite" ".bak" ".mdb"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-database" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M12 6m-8 0a8 3 0 1 0 16 0a8 3 0 1 0 -16 0"/> + <path d="M4 6v6a8 3 0 0 0 16 0v-6"/> + <path d="M4 12v6a8 3 0 0 0 16 0v-6"/> + </svg> + {{- else if .HasExt ".eml" ".email" ".mailbox" ".mbox" ".msg"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-mail" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M3 7a2 2 0 0 1 2 -2h14a2 2 0 0 1 2 2v10a2 2 0 0 1 -2 2h-14a2 2 0 0 1 -2 -2v-10z"/> + <path d="M3 7l9 6l9 -6"/> + </svg> + {{- else if .HasExt ".crt" ".pem" ".x509" ".cer" ".ca-bundle"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-certificate" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M15 15m-3 0a3 3 0 1 0 6 0a3 3 0 1 0 -6 0"/> + <path d="M13 17.5v4.5l2 -1.5l2 1.5v-4.5"/> + <path d="M10 19h-5a2 2 0 0 1 -2 -2v-10c0 -1.1 .9 -2 2 -2h14a2 2 0 0 1 2 2v10a2 2 0 0 1 -1 1.73"/> + <path d="M6 9l12 0"/> + <path d="M6 12l3 0"/> + <path d="M6 15l2 0"/> + </svg> + {{- else if .HasExt ".key" ".keystore" ".jks" ".p12" ".pfx" ".pub"}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-key" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M16.555 3.843l3.602 3.602a2.877 2.877 0 0 1 0 4.069l-2.643 2.643a2.877 2.877 0 0 1 -4.069 0l-.301 -.301l-6.558 6.558a2 2 0 0 1 -1.239 .578l-.175 .008h-1.172a1 1 0 0 1 -.993 -.883l-.007 -.117v-1.172a2 2 0 0 1 .467 -1.284l.119 -.13l.414 -.414h2v-2h2v-2l2.144 -2.144l-.301 -.301a2.877 2.877 0 0 1 0 -4.069l2.643 -2.643a2.877 2.877 0 0 1 4.069 0z"/> + <path d="M15 9h.01"/> + </svg> + {{- else}} + {{- if .IsSymlink}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file-symlink" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M4 21v-4a3 3 0 0 1 3 -3h5"/> + <path d="M9 17l3 -3l-3 -3"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M5 11v-6a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2h-9.5"/> + </svg> + {{- else}} + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-file" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M14 3v4a1 1 0 0 0 1 1h4"/> + <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/> + </svg> + {{- end}} + {{- end}} +{{- end}} <!DOCTYPE html> <html> <head> <title>{{html .Name}}</title> + <link rel="canonical" href="{{.Path}}/" /> <meta charset="utf-8"> + <meta name="color-scheme" content="light dark"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <style> -* { padding: 0; margin: 0; } +* { padding: 0; margin: 0; box-sizing: border-box; } body { - font-family: sans-serif; + font-family: Inter, system-ui, sans-serif; + font-size: 16px; text-rendering: optimizespeed; - background-color: #ffffff; + background-color: #f3f6f7; + min-height: 100vh; } -a { - color: #006ed3; - text-decoration: none; +img, +svg { + vertical-align: middle; + z-index: 1; } -a:hover, -h1 a:hover { - color: #319cff; +img { + max-width: 100%; + max-height: 100%; + border-radius: 5px; } -a:visited { - color: #800080; +td img { + max-width: 1.5em; + max-height: 2em; + object-fit: cover; } -a:visited:hover { - color: #b900b9; +body, +a, +svg, +.layout.current, +.layout.current svg, +.go-up { + color: #333; + text-decoration: none; +} + +.wrapper { + max-width: 1200px; + margin-left: auto; + margin-right: auto; } header, -#summary { +.meta { padding-left: 5%; padding-right: 5%; } +td a { + color: #006ed3; + text-decoration: none; +} + +td a:hover { + color: #0095e4; +} + +td a:visited { + color: #800080; +} + +td a:visited:hover { + color: #b900b9; +} + th:first-child, td:first-child { width: 5%; @@ -47,53 +381,115 @@ td:last-child { width: 5%; } +.size, +.timestamp { + font-size: 14px; +} + +.grid .size { + font-size: 12px; + margin-top: .5em; + color: #496a84; +} + header { - padding-top: 25px; + padding-top: 15px; padding-bottom: 15px; - background-color: #f2f2f2; + box-shadow: 0px 0px 20px 0px rgb(0 0 0 / 10%); +} + +.breadcrumbs { + text-transform: uppercase; + font-size: 10px; + letter-spacing: 1px; + color: #939393; + margin-bottom: 5px; + padding-left: 3px; } h1 { font-size: 20px; + font-family: Poppins, system-ui, sans-serif; font-weight: normal; white-space: nowrap; overflow-x: hidden; text-overflow: ellipsis; - color: #999; + color: #c5c5c5; } -h1 a { +h1 a, +th a { color: #000; - margin: 0 4px; +} + +h1 a { + padding: 0 3px; + margin: 0 1px; } h1 a:hover { - text-decoration: underline; + background: #ffffc4; } h1 a:first-child { margin: 0; } +header, main { - display: block; + background-color: white; +} + +main { + margin: 3em auto 0; + border-radius: 5px; + box-shadow: 0 2px 5px 1px rgb(0 0 0 / 5%); } .meta { - font-size: 12px; - font-family: Verdana, sans-serif; - border-bottom: 1px solid #9C9C9C; - padding-top: 10px; - padding-bottom: 10px; + display: flex; + gap: 1em; + font-size: 14px; + border-bottom: 1px solid #e5e9ea; + padding-top: 1em; + padding-bottom: 1em; } -.meta-item { - margin-right: 1em; +#summary { + display: flex; + gap: 1em; + align-items: center; + margin-right: auto; +} + +.filter-container { + position: relative; + display: inline-block; + margin-left: 1em; +} + +#search-icon { + color: #777; + position: absolute; + height: 1em; + top: .6em; + left: .5em; } #filter { - padding: 4px; + padding: .5em 1em .5em 2.5em; + border: none; border: 1px solid #CCC; + border-radius: 5px; + font-family: inherit; + position: relative; + z-index: 2; + background: none; +} + +.layout, +.layout svg { + color: #9a9a9a; } table { @@ -101,88 +497,132 @@ table { border-collapse: collapse; } -tr { - border-bottom: 1px dashed #dadada; +tbody tr, +tbody tr a, +.entry a { + transition: all .15s; } -tbody tr:hover { - background-color: #ffffec; +tbody tr:hover, +.grid .entry a:hover { + background-color: #f4f9fd; } th, td { text-align: left; - padding: 10px 0; } th { - padding-top: 15px; - padding-bottom: 15px; - font-size: 16px; + position: sticky; + top: 0; + background: white; white-space: nowrap; -} - -th a { - color: black; -} - -th svg { - vertical-align: middle; + z-index: 2; + text-transform: uppercase; + font-size: 14px; + letter-spacing: 1px; + padding: .75em 0; } td { white-space: nowrap; - font-size: 14px; } td:nth-child(2) { - width: 80%; + width: 75%; +} + +td:nth-child(2) a { + padding: 1em 0; + display: block; } td:nth-child(3), th:nth-child(3) { padding: 0 20px 0 20px; + min-width: 150px; } -th:nth-child(4), -td:nth-child(4) { - text-align: right; -} - -td:nth-child(2) svg { - position: absolute; +td .go-up { + text-transform: uppercase; + font-size: 12px; + font-weight: bold; } -td .name, -td .goup { - margin-left: 1.75em; +.name, +.go-up { word-break: break-all; overflow-wrap: break-word; white-space: pre-wrap; } -.icon { - margin-right: 5px; +.listing .icon-tabler { + color: #454545; } -.icon.sort { - display: inline-block; - width: 1em; - height: 1em; +.listing .icon-tabler-folder-filled { + color: #ffb900 !important; +} + +.sizebar { position: relative; - top: .2em; + padding: 0.25rem 0.5rem; + display: flex; } -.icon.sort .top { +.sizebar-bar { + background-color: #dbeeff; position: absolute; + top: 0; + right: 0; + bottom: 0; left: 0; - top: -1px; + z-index: 0; + height: 100%; + pointer-events: none; } -.icon.sort .bottom { - position: absolute; - bottom: -1px; - left: 0; +.sizebar-text { + position: relative; + z-index: 1; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(16em, 1fr)); + gap: 2px; +} + +.grid .entry { + position: relative; + width: 100%; +} + +.grid .entry a { + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + padding: 1.5em; + height: 100%; +} + +.grid .entry svg { + width: 75px; + height: 75px; +} + +.grid .entry img { + max-height: 200px; + object-fit: cover; +} + +.grid .entry .name { + margin-top: 1em; } footer { @@ -191,6 +631,12 @@ footer { text-align: center; } +.caddy-logo { + display: inline-block; + height: 2.5em; + margin: 0 auto; +} + @media (max-width: 600px) { .hideable { display: none; @@ -217,165 +663,273 @@ footer { #filter { max-width: 100px; } + + .grid .entry { + max-width: initial; + } } + @media (prefers-color-scheme: dark) { - body { - background-color: #101010; - color: #dddddd; + html { + background: black; /* overscroll */ } - header { - background-color: #151515; + body { + background: linear-gradient(180deg, rgb(34 50 66) 0%, rgb(26 31 38) 100%); + background-attachment: fixed; } - tbody tr:hover { - background-color: #252525; + body, + a, + svg, + .layout.current, + .layout.current svg, + .go-up { + color: #ccc; } - header a, + h1 a, th a { - color: #dddddd; + color: white; } - a { - color: #5796d1; - text-decoration: none; + h1 { + color: white; } - a:hover, h1 a:hover { - color: #62b2fd; + background: hsl(213deg 100% 73% / 20%); } - a:visited { - color: #c269c2; + header, + main, + .grid .entry { + background-color: #101720; } - a:visited:hover { - color: #d03cd0; + tbody tr:hover, + .grid .entry a:hover { + background-color: #162030; + color: #fff; } - tr { - border-bottom: 1px dashed rgba(255, 255, 255, 0.12); + th { + background-color: #18212c; } - #up-arrow, - #down-arrow { - fill: #dddddd; + td a, + .listing .icon-tabler { + color: #abc8e3; + } + + td a:hover, + td a:hover .icon-tabler { + color: white; + } + + td a:visited { + color: #cd53cd; + } + + td a:visited:hover { + color: #f676f6; + } + + #search-icon { + color: #7798c4; } #filter { - background-color: #151515; color: #ffffff; - border: 1px solid #212121; + border: 1px solid #29435c; } .meta { - border-bottom: 1px solid #212121 + border-bottom: 1px solid #222e3b; + } + + .sizebar-bar { + background-color: #1f3549; + } + + .grid .entry a { + background-color: #080b0f; + } + + #Wordmark path, + #R path { + fill: #ccc !important; + } + #R circle { + stroke: #ccc !important; } } -</style> - </head> - <body onload='initFilter()'> - <svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" height="0" width="0" style="position: absolute;"> - <defs> - <!-- Folder --> - <g id="folder" fill-rule="nonzero" fill="none"> - <path d="M285.22 37.55h-142.6L110.9 0H31.7C14.25 0 0 16.9 0 37.55v75.1h316.92V75.1c0-20.65-14.26-37.55-31.7-37.55z" fill="#FFA000"/> - <path d="M285.22 36H31.7C14.25 36 0 50.28 0 67.74v158.7c0 17.47 14.26 31.75 31.7 31.75H285.2c17.44 0 31.7-14.3 31.7-31.75V67.75c0-17.47-14.26-31.75-31.7-31.75z" fill="#FFCA28"/> - </g> - <g id="folder-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"> - <g id="folder-shortcut-group" fill-rule="nonzero"> - <g id="folder-shortcut-shape"> - <path d="M285.224876,37.5486902 L142.612438,37.5486902 L110.920785,0 L31.6916529,0 C14.2612438,0 0,16.8969106 0,37.5486902 L0,112.646071 L316.916529,112.646071 L316.916529,75.0973805 C316.916529,54.4456008 302.655285,37.5486902 285.224876,37.5486902 Z" id="Shape" fill="#FFA000"></path> - <path d="M285.224876,36 L31.6916529,36 C14.2612438,36 0,50.2838568 0,67.7419039 L0,226.451424 C0,243.909471 14.2612438,258.193328 31.6916529,258.193328 L285.224876,258.193328 C302.655285,258.193328 316.916529,243.909471 316.916529,226.451424 L316.916529,67.7419039 C316.916529,50.2838568 302.655285,36 285.224876,36 Z" id="Shape" fill="#FFCA28"></path> - </g> - <path d="M126.154134,250.559184 C126.850974,251.883673 127.300549,253.006122 127.772602,254.106122 C128.469442,255.206122 128.919016,256.104082 129.638335,257.002041 C130.559962,258.326531 131.728855,259 133.100057,259 C134.493737,259 135.415364,258.55102 136.112204,257.67551 C136.809044,257.002041 137.258619,255.902041 137.258619,254.577551 C137.258619,253.904082 137.258619,252.804082 137.033832,251.457143 C136.786566,249.908163 136.561779,249.032653 136.561779,248.583673 C136.089726,242.814286 135.864939,237.920408 135.864939,233.273469 C135.864939,225.057143 136.786566,217.514286 138.180246,210.846939 C139.798713,204.202041 141.889234,198.634694 144.429328,193.763265 C147.216689,188.869388 150.678411,184.873469 154.836973,181.326531 C158.995535,177.779592 163.626149,174.883673 168.481552,172.661224 C173.336954,170.438776 179.113983,168.665306 185.587852,167.340816 C192.061722,166.218367 198.760378,165.342857 205.481514,164.669388 C212.18017,164.220408 219.598146,163.995918 228.162535,163.995918 L246.055591,163.995918 L246.055591,195.514286 C246.055591,197.736735 246.752431,199.510204 248.370899,201.059184 C250.214153,202.608163 252.079886,203.506122 254.372715,203.506122 C256.463236,203.506122 258.531277,202.608163 260.172223,201.059184 L326.102289,137.797959 C327.720757,136.24898 328.642384,134.47551 328.642384,132.253061 C328.642384,130.030612 327.720757,128.257143 326.102289,126.708163 L260.172223,63.4469388 C258.553756,61.8979592 256.463236,61 254.395194,61 C252.079886,61 250.236632,61.8979592 248.393377,63.4469388 C246.77491,64.9959184 246.07807,66.7693878 246.07807,68.9918367 L246.07807,100.510204 L228.162535,100.510204 C166.863084,100.510204 129.166282,117.167347 115.274437,150.459184 C110.666301,161.54898 108.350993,175.310204 108.350993,191.742857 C108.350993,205.279592 113.903236,223.912245 124.760454,247.438776 C125.00772,248.112245 125.457294,249.010204 126.154134,250.559184 Z" id="Shape" fill="#FFFFFF" transform="translate(218.496689, 160.000000) scale(-1, 1) translate(-218.496689, -160.000000) "></path> - </g> - </g> - - <!-- File --> - <g id="file" stroke="#000" stroke-width="25" fill="#FFF" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"> - <path d="M13 24.12v274.76c0 6.16 5.87 11.12 13.17 11.12H239c7.3 0 13.17-4.96 13.17-11.12V136.15S132.6 13 128.37 13H26.17C18.87 13 13 17.96 13 24.12z"/> - <path d="M129.37 13L129 113.9c0 10.58 7.26 19.1 16.27 19.1H249L129.37 13z"/> - </g> - <g id="file-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"> - <g id="file-shortcut-group" transform="translate(13.000000, 13.000000)"> - <g id="file-shortcut-shape" stroke="#000000" stroke-width="25" fill="#FFFFFF" stroke-linecap="round" stroke-linejoin="round"> - <path d="M0,11.1214886 L0,285.878477 C0,292.039924 5.87498876,296.999983 13.1728373,296.999983 L225.997983,296.999983 C233.295974,296.999983 239.17082,292.039942 239.17082,285.878477 L239.17082,123.145388 C239.17082,123.145388 119.58541,2.84217094e-14 115.369423,2.84217094e-14 L13.1728576,2.84217094e-14 C5.87500907,-1.71479982e-05 0,4.96022995 0,11.1214886 Z" id="rect1171"></path> - <path d="M116.37005,0 L116,100.904964 C116,111.483663 123.258008,120 132.273377,120 L236,120 L116.37005,0 L116.37005,0 Z" id="rect1794"></path> - </g> - <path d="M47.803141,294.093878 C48.4999811,295.177551 48.9495553,296.095918 49.4216083,296.995918 C50.1184484,297.895918 50.5680227,298.630612 51.2873415,299.365306 C52.2089688,300.44898 53.3778619,301 54.7490634,301 C56.1427436,301 57.0643709,300.632653 57.761211,299.916327 C58.4580511,299.365306 58.9076254,298.465306 58.9076254,297.381633 C58.9076254,296.830612 58.9076254,295.930612 58.6828382,294.828571 C58.4355724,293.561224 58.2107852,292.844898 58.2107852,292.477551 C57.7387323,287.757143 57.5139451,283.753061 57.5139451,279.95102 C57.5139451,273.228571 58.4355724,267.057143 59.8292526,261.602041 C61.44772,256.165306 63.5382403,251.610204 66.0783349,247.62449 C68.8656954,243.620408 72.3274172,240.35102 76.4859792,237.44898 C80.6445412,234.546939 85.2751561,232.177551 90.1305582,230.359184 C94.9859603,228.540816 100.76299,227.089796 107.236859,226.006122 C113.710728,225.087755 120.409385,224.371429 127.13052,223.820408 C133.829177,223.453061 141.247152,223.269388 149.811542,223.269388 L167.704598,223.269388 L167.704598,249.057143 C167.704598,250.87551 168.401438,252.326531 170.019905,253.593878 C171.86316,254.861224 173.728893,255.595918 176.021722,255.595918 C178.112242,255.595918 180.180284,254.861224 181.82123,253.593878 L247.751296,201.834694 C249.369763,200.567347 250.291391,199.116327 250.291391,197.297959 C250.291391,195.479592 249.369763,194.028571 247.751296,192.761224 L181.82123,141.002041 C180.202763,139.734694 178.112242,139 176.044201,139 C173.728893,139 171.885639,139.734694 170.042384,141.002041 C168.423917,142.269388 167.727077,143.720408 167.727077,145.538776 L167.727077,171.326531 L149.811542,171.326531 C88.5120908,171.326531 50.8152886,184.955102 36.9234437,212.193878 C32.3153075,221.267347 30,232.526531 30,245.971429 C30,257.046939 35.5522422,272.291837 46.4094607,291.540816 C46.6567266,292.091837 47.1063009,292.826531 47.803141,294.093878 Z" id="Shape-Copy" fill="#000000" fill-rule="nonzero" transform="translate(140.145695, 220.000000) scale(-1, 1) translate(-140.145695, -220.000000) "></path> - </g> - </g> - - <!-- Up arrow --> - <g id="up-arrow" transform="translate(-279.22 -208.12)"> - <path transform="matrix(.22413 0 0 .12089 335.67 164.35)" stroke-width="0" d="m-194.17 412.01h-28.827-28.827l14.414-24.965 14.414-24.965 14.414 24.965z"/> - </g> - - <!-- Down arrow --> - <g id="down-arrow" transform="translate(-279.22 -208.12)"> - <path transform="matrix(.22413 0 0 -.12089 335.67 257.93)" stroke-width="0" d="m-194.17 412.01h-28.827-28.827l14.414-24.965 14.414-24.965 14.414 24.965z"/> - </g> - </defs> - </svg> - <header> - <h1> - {{range $i, $crumb := .Breadcrumbs}}<a href="{{html $crumb.Link}}">{{html $crumb.Text}}</a>{{if ne $i 0}}/{{end}}{{end}} - </h1> +</style> +{{- if eq .Layout "grid"}} +<style>.wrapper { max-width: none; } main { margin-top: 1px; }</style> +{{- end}} +</head> +<body onload="initPage()"> + <header> + <div class="wrapper"> + <div class="breadcrumbs">Folder Path</div> + <h1> + {{range $i, $crumb := .Breadcrumbs}}<a href="{{html $crumb.Link}}">{{html $crumb.Text}}</a>{{if ne $i 0}}/{{end}}{{end}} + </h1> + </div> </header> - <main> - <div class="meta"> - <div id="summary"> - <span class="meta-item"><b>{{.NumDirs}}</b> director{{if eq 1 .NumDirs}}y{{else}}ies{{end}}</span> - <span class="meta-item"><b>{{.NumFiles}}</b> file{{if ne 1 .NumFiles}}s{{end}}</span> - {{- if ne 0 .Limit}} - <span class="meta-item">(of which only <b>{{.Limit}}</b> are displayed)</span> - {{- end}} - <span class="meta-item"><input type="text" placeholder="filter" id="filter" onkeyup='filter()'></span> + <div class="wrapper"> + <main> + <div class="meta"> + <div id="summary"> + <span class="meta-item"> + <b>{{.NumDirs}}</b> director{{if eq 1 .NumDirs}}y{{else}}ies{{end}} + </span> + <span class="meta-item"> + <b>{{.NumFiles}}</b> file{{if ne 1 .NumFiles}}s{{end}} + </span> + {{- if ne 0 .Limit}} + <span class="meta-item"> + (of which only <b>{{.Limit}}</b> are displayed) + </span> + {{- end}} + </div> + <a href="javascript:queryParam('layout', '')" id="layout-list" class='layout{{if eq $.Layout "list" ""}}current{{end}}'> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-layout-list" width="16" height="16" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M4 4m0 2a2 2 0 0 1 2 -2h12a2 2 0 0 1 2 2v2a2 2 0 0 1 -2 2h-12a2 2 0 0 1 -2 -2z"/> + <path d="M4 14m0 2a2 2 0 0 1 2 -2h12a2 2 0 0 1 2 2v2a2 2 0 0 1 -2 2h-12a2 2 0 0 1 -2 -2z"/> + </svg> + List + </a> + <a href="javascript:queryParam('layout', 'grid')" id="layout-grid" class='layout{{if eq $.Layout "grid"}}current{{end}}'> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-layout-grid" width="16" height="16" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M4 4m0 1a1 1 0 0 1 1 -1h4a1 1 0 0 1 1 1v4a1 1 0 0 1 -1 1h-4a1 1 0 0 1 -1 -1z"/> + <path d="M14 4m0 1a1 1 0 0 1 1 -1h4a1 1 0 0 1 1 1v4a1 1 0 0 1 -1 1h-4a1 1 0 0 1 -1 -1z"/> + <path d="M4 14m0 1a1 1 0 0 1 1 -1h4a1 1 0 0 1 1 1v4a1 1 0 0 1 -1 1h-4a1 1 0 0 1 -1 -1z"/> + <path d="M14 14m0 1a1 1 0 0 1 1 -1h4a1 1 0 0 1 1 1v4a1 1 0 0 1 -1 1h-4a1 1 0 0 1 -1 -1z"/> + </svg> + Grid + </a> </div> - </div> - <div class="listing"> + <div class='listing{{if eq .Layout "grid"}} grid{{end}}'> + {{- if eq .Layout "grid"}} + {{- range .Items}} + <div class="entry"> + <a href="{{html .URL}}" title='{{html (.HumanModTime "January 2, 2006 at 15:04:05")}}'> + {{template "icon" .}} + <div class="name">{{html .Name}}</div> + <div class="size">{{.HumanSize}}</div> + </a> + </div> + {{- end}} + {{- else}} <table aria-describedby="summary"> <thead> <tr> <th></th> <th> {{- if and (eq .Sort "namedirfirst") (ne .Order "desc")}} - <a href="?sort=namedirfirst&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon"><svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a> + <a href="?sort=namedirfirst&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon"> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 14l-6 -6l-6 6h12"/> + </svg> + </a> {{- else if and (eq .Sort "namedirfirst") (ne .Order "asc")}} - <a href="?sort=namedirfirst&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon"><svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a> + <a href="?sort=namedirfirst&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon"> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-down" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 10l6 6l6 -6h-12"/> + </svg> + </a> {{- else}} - <a href="?sort=namedirfirst&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon sort"><svg class="top" width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg><svg class="bottom" width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a> + <a href="?sort=namedirfirst&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}" class="icon sort"> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 14l-6 -6l-6 6h12"/> + </svg> + </a> {{- end}} {{- if and (eq .Sort "name") (ne .Order "desc")}} - <a href="?sort=name&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Name <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a> + <a href="?sort=name&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Name + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 14l-6 -6l-6 6h12"/> + </svg> + </a> {{- else if and (eq .Sort "name") (ne .Order "asc")}} - <a href="?sort=name&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Name <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a> + <a href="?sort=name&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Name + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-down" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 10l6 6l6 -6h-12"/> + </svg> + </a> {{- else}} - <a href="?sort=name&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Name</a> + <a href="?sort=name&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Name + </a> {{- end}} + + <div class="filter-container"> + <svg id="search-icon" xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-search" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M10 10m-7 0a7 7 0 1 0 14 0a7 7 0 1 0 -14 0"/> + <path d="M21 21l-6 -6"/> + </svg> + <input type="search" placeholder="Search" id="filter" onkeyup='filter()'> + </div> </th> <th> {{- if and (eq .Sort "size") (ne .Order "desc")}} - <a href="?sort=size&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Size <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a> + <a href="?sort=size&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Size + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 14l-6 -6l-6 6h12"/> + </svg> + </a> {{- else if and (eq .Sort "size") (ne .Order "asc")}} - <a href="?sort=size&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Size <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a> + <a href="?sort=size&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Size + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-down" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 10l6 6l6 -6h-12"/> + </svg> + </a> {{- else}} - <a href="?sort=size&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Size</a> + <a href="?sort=size&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Size + </a> {{- end}} </th> <th class="hideable"> {{- if and (eq .Sort "time") (ne .Order "desc")}} - <a href="?sort=time&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Modified <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a> + <a href="?sort=time&order=desc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Modified + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 14l-6 -6l-6 6h12"/> + </svg> + </a> {{- else if and (eq .Sort "time") (ne .Order "asc")}} - <a href="?sort=time&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Modified <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a> + <a href="?sort=time&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Modified + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-caret-down" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M6 10l6 6l6 -6h-12"/> + </svg> + </a> {{- else}} - <a href="?sort=time&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}">Modified</a> + <a href="?sort=time&order=asc{{if ne 0 .Limit}}&limit={{.Limit}}{{end}}{{if ne 0 .Offset}}&offset={{.Offset}}{{end}}"> + Modified + </a> {{- end}} </th> <th class="hideable"></th> @@ -387,11 +941,15 @@ footer { <td></td> <td> <a href=".."> - <span class="goup">Go up</span> + <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-corner-left-up" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round"> + <path stroke="none" d="M0 0h24v24H0z" fill="none"/> + <path d="M18 18h-6a3 3 0 0 1 -3 -3v-10l-4 4m8 0l-4 -4"/> + </svg> + <span class="go-up">Up</span> </a> </td> - <td>—</td> - <td class="hideable">—</td> + <td></td> + <td class="hideable"></td> <td class="hideable"></td> </tr> {{- end}} @@ -400,54 +958,149 @@ footer { <td></td> <td> <a href="{{html .URL}}"> - {{- if .IsDir}} - <svg width="1.5em" height="1em" version="1.1" viewBox="0 0 317 259"><use xlink:href="#folder{{if .IsSymlink}}-shortcut{{end}}"></use></svg> - {{- else}} - <svg width="1.5em" height="1em" version="1.1" viewBox="0 0 265 323"><use xlink:href="#file{{if .IsSymlink}}-shortcut{{end}}"></use></svg> - {{- end}} + {{template "icon" .}} <span class="name">{{html .Name}}</span> </a> </td> {{- if .IsDir}} - <td data-order="-1">—</td> + <td>—</td> {{- else}} - <td data-order="{{.Size}}">{{.HumanSize}}</td> + <td class="size" data-size="{{.Size}}"> + <div class="sizebar"> + <div class="sizebar-bar"></div> + <div class="sizebar-text"> + {{.HumanSize}} + </div> + </div> + </td> {{- end}} - <td class="hideable"><time datetime="{{.HumanModTime "2006-01-02T15:04:05Z"}}">{{.HumanModTime "01/02/2006 03:04:05 PM -07:00"}}</time></td> + <td class="timestamp hideable"> + <time datetime="{{.HumanModTime "2006-01-02T15:04:05Z"}}">{{.HumanModTime "01/02/2006 03:04:05 PM -07:00"}}</time> + </td> <td class="hideable"></td> </tr> {{- end}} </tbody> </table> + {{- end}} </div> - </main> + </main> + </div> <footer> - Served with <a rel="noopener noreferrer" href="https://caddyserver.com">Caddy</a> + Served with + <a rel="noopener noreferrer" href="https://caddyserver.com"> + <svg class="caddy-logo" viewBox="0 0 379 114" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" xmlns:serif="http://www.serif.com/" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linecap:round;stroke-linejoin:round;"> + <g transform="matrix(1,0,0,1,-1982.99,-530.985)"> + <g transform="matrix(1.16548,0,0,1.10195,1823.12,393.466)"> + <g transform="matrix(1,0,0,1,0.233052,1.17986)"> + <g id="Icon" transform="matrix(0.858013,0,0,0.907485,-3224.99,-1435.83)"> + <g> + <g transform="matrix(-0.191794,-0.715786,0.715786,-0.191794,4329.14,4673.64)"> + <path d="M3901.56,610.734C3893.53,610.261 3886.06,608.1 3879.2,604.877C3872.24,601.608 3866.04,597.093 3860.8,591.633C3858.71,589.457 3856.76,587.149 3854.97,584.709C3853.2,582.281 3851.57,579.733 3850.13,577.066C3845.89,569.224 3843.21,560.381 3842.89,550.868C3842.57,543.321 3843.64,536.055 3845.94,529.307C3848.37,522.203 3852.08,515.696 3856.83,510.049L3855.79,509.095C3850.39,514.54 3846.02,520.981 3842.9,528.125C3839.84,535.125 3838.03,542.781 3837.68,550.868C3837.34,561.391 3839.51,571.425 3843.79,580.306C3845.27,583.38 3847.03,586.304 3849.01,589.049C3851.01,591.806 3853.24,594.39 3855.69,596.742C3861.75,602.568 3869,607.19 3877.03,610.1C3884.66,612.867 3892.96,614.059 3901.56,613.552L3901.56,610.734Z" style="fill:rgb(0,144,221);"/> + </g> + <g transform="matrix(-0.191794,-0.715786,0.715786,-0.191794,4329.14,4673.64)"> + <path d="M3875.69,496.573C3879.62,494.538 3883.8,492.897 3888.2,491.786C3892.49,490.704 3896.96,490.124 3901.56,490.032C3903.82,490.13 3906.03,490.332 3908.21,490.688C3917.13,492.147 3925.19,495.814 3932.31,500.683C3936.13,503.294 3939.59,506.335 3942.81,509.619C3947.09,513.98 3950.89,518.816 3953.85,524.232C3958.2,532.197 3960.96,541.186 3961.32,550.868C3961.61,558.748 3960.46,566.345 3957.88,573.322C3956.09,578.169 3953.7,582.753 3950.66,586.838C3947.22,591.461 3942.96,595.427 3938.27,598.769C3933.66,602.055 3928.53,604.619 3923.09,606.478C3922.37,606.721 3921.6,606.805 3920.93,607.167C3920.42,607.448 3920.14,607.854 3919.69,608.224L3920.37,610.389C3920.98,610.432 3921.47,610.573 3922.07,610.474C3922.86,610.344 3923.55,609.883 3924.28,609.566C3931.99,606.216 3938.82,601.355 3944.57,595.428C3947.02,592.903 3949.25,590.174 3951.31,587.319C3953.59,584.168 3955.66,580.853 3957.43,577.348C3961.47,569.34 3964.01,560.422 3964.36,550.868C3964.74,540.511 3962.66,530.628 3958.48,521.868C3955.57,515.775 3951.72,510.163 3946.95,505.478C3943.37,501.962 3939.26,498.99 3934.84,496.562C3926.88,492.192 3917.87,489.76 3908.37,489.229C3906.12,489.104 3903.86,489.054 3901.56,489.154C3896.87,489.06 3892.3,489.519 3887.89,490.397C3883.3,491.309 3878.89,492.683 3874.71,494.525L3875.69,496.573Z" style="fill:rgb(0,144,221);"/> + </g> + </g> + <g> + <g transform="matrix(-3.37109,-0.514565,0.514565,-3.37109,4078.07,1806.88)"> + <path d="M22,12C22,10.903 21.097,10 20,10C19.421,10 18.897,10.251 18.53,10.649C18.202,11.006 18,11.481 18,12C18,13.097 18.903,14 20,14C21.097,14 22,13.097 22,12Z" style="fill:none;fill-rule:nonzero;stroke:rgb(0,144,221);stroke-width:1.05px;"/> + </g> + <g transform="matrix(-5.33921,-5.26159,-3.12106,-6.96393,4073.87,1861.55)"> + <path d="M10.315,5.333C10.315,5.333 9.748,5.921 9.03,6.673C7.768,7.995 6.054,9.805 6.054,9.805L6.237,9.86C6.237,9.86 8.045,8.077 9.36,6.771C10.107,6.028 10.689,5.444 10.689,5.444L10.315,5.333Z" style="fill:rgb(0,144,221);"/> + </g> + </g> + <g id="Padlock" transform="matrix(3.11426,0,0,3.11426,3938.31,1737.25)"> + <g> + <path d="M9.876,21L18.162,21C18.625,21 19,20.625 19,20.162L19,11.838C19,11.375 18.625,11 18.162,11L5.838,11C5.375,11 5,11.375 5,11.838L5,16.758" style="fill:none;stroke:rgb(34,182,56);stroke-width:1.89px;stroke-linecap:butt;stroke-linejoin:miter;"/> + <path d="M8,11L8,7C8,4.806 9.806,3 12,3C14.194,3 16,4.806 16,7L16,11" style="fill:none;fill-rule:nonzero;stroke:rgb(34,182,56);stroke-width:1.89px;"/> + </g> + </g> + <g> + <g transform="matrix(5.30977,0.697415,-0.697415,5.30977,3852.72,1727.97)"> + <path d="M22,12C22,11.659 21.913,11.337 21.76,11.055C21.421,10.429 20.756,10 20,10C18.903,10 18,10.903 18,12C18,13.097 18.903,14 20,14C21.097,14 22,13.097 22,12Z" style="fill:none;fill-rule:nonzero;stroke:rgb(0,144,221);stroke-width:0.98px;"/> + </g> + <g transform="matrix(4.93114,2.49604,1.11018,5.44847,3921.41,1726.72)"> + <path d="M8.902,6.77C8.902,6.77 7.235,8.253 6.027,9.366C5.343,9.996 4.819,10.502 4.819,10.502L5.52,11.164C5.52,11.164 6.021,10.637 6.646,9.951C7.749,8.739 9.219,7.068 9.219,7.068L8.902,6.77Z" style="fill:rgb(0,144,221);"/> + </g> + </g> + </g> + <g id="Text"> + <g id="Wordmark" transform="matrix(1.32271,0,0,2.60848,-899.259,-791.691)"> + <g id="y" transform="matrix(0.50291,0,0,0.281607,905.533,304.987)"> + <path d="M192.152,286.875L202.629,268.64C187.804,270.106 183.397,265.779 180.143,263.391C176.888,261.004 174.362,257.99 172.563,254.347C170.765,250.705 169.866,246.691 169.866,242.305L169.866,208.107L183.21,208.107L183.21,242.213C183.21,245.188 183.896,247.822 185.268,250.116C186.64,252.41 188.465,254.197 190.743,255.475C193.022,256.754 195.501,257.393 198.182,257.393C200.894,257.393 203.393,256.75 205.68,255.463C207.966,254.177 209.799,252.391 211.178,250.105C212.558,247.818 213.248,245.188 213.248,242.213L213.248,208.107L226.545,208.107L226.545,242.305C226.545,246.707 225.378,258.46 218.079,268.64C215.735,271.909 207.835,286.875 207.835,286.875L192.152,286.875Z" style="fill:rgb(47,47,47);fill-rule:nonzero;"/> + </g> + <g id="add" transform="matrix(0.525075,0,0,0.281607,801.871,304.987)"> + <g transform="matrix(116.242,0,0,116.242,161.846,267.39)"> + <path d="M0.276,0.012C0.227,0.012 0.186,0 0.15,-0.024C0.115,-0.048 0.088,-0.08 0.069,-0.12C0.05,-0.161 0.04,-0.205 0.04,-0.254C0.04,-0.305 0.051,-0.35 0.072,-0.39C0.094,-0.431 0.125,-0.463 0.165,-0.487C0.205,-0.51 0.254,-0.522 0.31,-0.522C0.366,-0.522 0.413,-0.51 0.452,-0.486C0.491,-0.463 0.521,-0.431 0.542,-0.39C0.562,-0.35 0.573,-0.305 0.573,-0.256L0.573,-0L0.458,-0L0.458,-0.095L0.456,-0.095C0.446,-0.076 0.433,-0.058 0.417,-0.042C0.401,-0.026 0.381,-0.013 0.358,-0.003C0.335,0.007 0.307,0.012 0.276,0.012ZM0.307,-0.086C0.337,-0.086 0.363,-0.093 0.386,-0.108C0.408,-0.123 0.426,-0.144 0.438,-0.17C0.45,-0.195 0.456,-0.224 0.456,-0.256C0.456,-0.288 0.45,-0.317 0.438,-0.342C0.426,-0.367 0.409,-0.387 0.387,-0.402C0.365,-0.417 0.338,-0.424 0.308,-0.424C0.276,-0.424 0.249,-0.417 0.226,-0.402C0.204,-0.387 0.186,-0.366 0.174,-0.341C0.162,-0.315 0.156,-0.287 0.156,-0.255C0.156,-0.224 0.162,-0.195 0.174,-0.169C0.186,-0.144 0.203,-0.123 0.226,-0.108C0.248,-0.093 0.275,-0.086 0.307,-0.086Z" style="fill:rgb(47,47,47);fill-rule:nonzero;"/> + </g> + <g transform="matrix(116.242,0,0,116.242,226.592,267.39)"> + <path d="M0.306,0.012C0.265,0.012 0.229,0.006 0.196,-0.008C0.163,-0.021 0.135,-0.039 0.112,-0.064C0.089,-0.088 0.071,-0.117 0.059,-0.151C0.046,-0.185 0.04,-0.222 0.04,-0.263C0.04,-0.315 0.051,-0.36 0.072,-0.399C0.093,-0.437 0.122,-0.468 0.159,-0.489C0.196,-0.511 0.239,-0.522 0.287,-0.522C0.311,-0.522 0.333,-0.518 0.355,-0.511C0.377,-0.504 0.396,-0.493 0.413,-0.48C0.431,-0.466 0.445,-0.451 0.455,-0.433L0.456,-0.433L0.456,-0.73L0.571,-0.73L0.571,-0.261C0.571,-0.205 0.56,-0.156 0.537,-0.115C0.515,-0.074 0.484,-0.043 0.444,-0.021C0.405,0.001 0.358,0.012 0.306,0.012ZM0.306,-0.086C0.335,-0.086 0.361,-0.093 0.384,-0.107C0.406,-0.122 0.423,-0.141 0.436,-0.167C0.448,-0.192 0.455,-0.221 0.455,-0.255C0.455,-0.288 0.448,-0.317 0.436,-0.343C0.423,-0.368 0.406,-0.388 0.383,-0.402C0.361,-0.417 0.335,-0.424 0.305,-0.424C0.276,-0.424 0.251,-0.417 0.228,-0.402C0.206,-0.387 0.188,-0.368 0.175,-0.342C0.163,-0.317 0.156,-0.288 0.156,-0.255C0.156,-0.222 0.163,-0.193 0.175,-0.167C0.188,-0.142 0.206,-0.122 0.229,-0.108C0.251,-0.093 0.277,-0.086 0.306,-0.086Z" style="fill:rgb(47,47,47);fill-rule:nonzero;"/> + </g> + <g transform="matrix(116.242,0,0,116.242,290.293,267.39)"> + <path d="M0.306,0.012C0.265,0.012 0.229,0.006 0.196,-0.008C0.163,-0.021 0.135,-0.039 0.112,-0.064C0.089,-0.088 0.071,-0.117 0.059,-0.151C0.046,-0.185 0.04,-0.222 0.04,-0.263C0.04,-0.315 0.051,-0.36 0.072,-0.399C0.093,-0.437 0.122,-0.468 0.159,-0.489C0.196,-0.511 0.239,-0.522 0.287,-0.522C0.311,-0.522 0.333,-0.518 0.355,-0.511C0.377,-0.504 0.396,-0.493 0.413,-0.48C0.431,-0.466 0.445,-0.451 0.455,-0.433L0.456,-0.433L0.456,-0.73L0.571,-0.73L0.571,-0.261C0.571,-0.205 0.56,-0.156 0.537,-0.115C0.515,-0.074 0.484,-0.043 0.444,-0.021C0.405,0.001 0.358,0.012 0.306,0.012ZM0.306,-0.086C0.335,-0.086 0.361,-0.093 0.384,-0.107C0.406,-0.122 0.423,-0.141 0.436,-0.167C0.448,-0.192 0.455,-0.221 0.455,-0.255C0.455,-0.288 0.448,-0.317 0.436,-0.343C0.423,-0.368 0.406,-0.388 0.383,-0.402C0.361,-0.417 0.335,-0.424 0.305,-0.424C0.276,-0.424 0.251,-0.417 0.228,-0.402C0.206,-0.387 0.188,-0.368 0.175,-0.342C0.163,-0.317 0.156,-0.288 0.156,-0.255C0.156,-0.222 0.163,-0.193 0.175,-0.167C0.188,-0.142 0.206,-0.122 0.229,-0.108C0.251,-0.093 0.277,-0.086 0.306,-0.086Z" style="fill:rgb(47,47,47);fill-rule:nonzero;"/> + </g> + </g> + <g id="c" transform="matrix(-0.0716462,0.31304,-0.583685,-0.0384251,1489.76,-444.051)"> + <path d="M2668.11,700.4C2666.79,703.699 2666.12,707.216 2666.12,710.766C2666.12,726.268 2678.71,738.854 2694.21,738.854C2709.71,738.854 2722.3,726.268 2722.3,710.766C2722.3,704.111 2719.93,697.672 2715.63,692.597L2707.63,699.378C2710.33,702.559 2711.57,706.602 2711.81,710.766C2712.2,717.38 2706.61,724.52 2697.27,726.637C2683.9,728.581 2676.61,720.482 2676.61,710.766C2676.61,708.541 2677.03,706.336 2677.85,704.269L2668.11,700.4Z" style="fill:rgb(46,46,46);"/> + </g> + </g> + <g id="R" transform="matrix(0.426446,0,0,0.451034,-1192.44,-722.167)"> + <g transform="matrix(1,0,0,1,-0.10786,0.450801)"> + <g transform="matrix(12.1247,0,0,12.1247,3862.61,1929.9)"> + <path d="M0.073,-0L0.073,-0.7L0.383,-0.7C0.428,-0.7 0.469,-0.69 0.506,-0.67C0.543,-0.651 0.572,-0.623 0.594,-0.588C0.616,-0.553 0.627,-0.512 0.627,-0.465C0.627,-0.418 0.615,-0.377 0.592,-0.342C0.569,-0.306 0.539,-0.279 0.501,-0.259L0.57,-0.128C0.574,-0.12 0.579,-0.115 0.584,-0.111C0.59,-0.107 0.596,-0.106 0.605,-0.106L0.664,-0.106L0.664,-0L0.587,-0C0.56,-0 0.535,-0.007 0.514,-0.02C0.493,-0.034 0.476,-0.052 0.463,-0.075L0.381,-0.232C0.375,-0.231 0.368,-0.231 0.361,-0.231C0.354,-0.231 0.347,-0.231 0.34,-0.231L0.192,-0.231L0.192,-0L0.073,-0ZM0.192,-0.336L0.368,-0.336C0.394,-0.336 0.417,-0.341 0.438,-0.351C0.459,-0.361 0.476,-0.376 0.489,-0.396C0.501,-0.415 0.507,-0.438 0.507,-0.465C0.507,-0.492 0.501,-0.516 0.488,-0.535C0.475,-0.554 0.459,-0.569 0.438,-0.579C0.417,-0.59 0.394,-0.595 0.369,-0.595L0.192,-0.595L0.192,-0.336Z" style="fill:rgb(46,46,46);fill-rule:nonzero;"/> + </g> + </g> + <g transform="matrix(1,0,0,1,0.278569,0.101881)"> + <circle cx="3866.43" cy="1926.14" r="8.923" style="fill:none;stroke:rgb(46,46,46);stroke-width:2px;stroke-linecap:butt;stroke-linejoin:miter;"/> + </g> + </g> + </g> + </g> + </g> + </g> + </svg> + </a> </footer> + <script> - var filterEl = document.getElementById('filter'); - filterEl.focus({ preventScroll: true }); + const filterEl = document.getElementById('filter'); + filterEl?.focus({ preventScroll: true }); - function initFilter() { - if (!filterEl.value) { - var filterParam = new URL(window.location.href).searchParams.get('filter'); + function initPage() { + // populate and evaluate filter + if (!filterEl?.value) { + const filterParam = new URL(window.location.href).searchParams.get('filter'); if (filterParam) { filterEl.value = filterParam; } } filter(); + + // fill in size bars + let largest = 0; + document.querySelectorAll('.size').forEach(el => { + largest = Math.max(largest, Number(el.dataset.size)); + }); + document.querySelectorAll('.size').forEach(el => { + const size = Number(el.dataset.size); + const sizebar = el.querySelector('.sizebar-bar'); + if (sizebar) { + sizebar.style.width = `${size/largest * 100}%`; + } + }); } function filter() { - var q = filterEl.value.trim().toLowerCase(); - var elems = document.querySelectorAll('tr.file'); - elems.forEach(function(el) { + if (!filterEl) return; + const q = filterEl.value.trim().toLowerCase(); + document.querySelectorAll('tr.file').forEach(function(el) { if (!q) { el.style.display = ''; return; } - var nameEl = el.querySelector('.name'); - var nameVal = nameEl.textContent.trim().toLowerCase(); + const nameEl = el.querySelector('.name'); + const nameVal = nameEl.textContent.trim().toLowerCase(); if (nameVal.indexOf(q) !== -1) { el.style.display = ''; } else { @@ -456,6 +1109,21 @@ footer { }); } + function queryParam(k, v) { + const qs = new URLSearchParams(window.location.search); + if (!v) { + qs.delete(k); + } else { + qs.set(k, v); + } + const qsStr = qs.toString(); + if (qsStr) { + window.location.search = qsStr; + } else { + window.location = window.location.pathname; + } + } + function localizeDatetime(e, index, ar) { if (e.textContent === undefined) { return; @@ -467,7 +1135,7 @@ footer { return; } } - e.textContent = d.toLocaleString([], {day: "2-digit", month: "2-digit", year: "numeric", hour: "2-digit", minute: "2-digit", second: "2-digit"}); + e.textContent = d.toLocaleString(); } var timeList = Array.prototype.slice.call(document.getElementsByTagName("time")); timeList.forEach(localizeDatetime); diff --git a/modules/caddyhttp/fileserver/browsetplcontext.go b/modules/caddyhttp/fileserver/browsetplcontext.go index 172fa50..682273c 100644 --- a/modules/caddyhttp/fileserver/browsetplcontext.go +++ b/modules/caddyhttp/fileserver/browsetplcontext.go @@ -25,17 +25,23 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/dustin/go-humanize" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) -func (fsrv *FileServer) directoryListing(ctx context.Context, entries []fs.DirEntry, canGoUp bool, root, urlPath string, repl *caddy.Replacer) browseTemplateContext { +func (fsrv *FileServer) directoryListing(ctx context.Context, entries []fs.DirEntry, canGoUp bool, root, urlPath string, repl *caddy.Replacer) *browseTemplateContext { filesToHide := fsrv.transformHidePaths(repl) - var dirCount, fileCount int - fileInfos := []fileInfo{} + name, _ := url.PathUnescape(urlPath) + + tplCtx := &browseTemplateContext{ + Name: path.Base(name), + Path: urlPath, + CanGoUp: canGoUp, + } for _, entry := range entries { if err := ctx.Err(); err != nil { @@ -61,9 +67,9 @@ func (fsrv *FileServer) directoryListing(ctx context.Context, entries []fs.DirEn // add the slash after the escape of path to avoid escaping the slash as well if isDir { name += "/" - dirCount++ + tplCtx.NumDirs++ } else { - fileCount++ + tplCtx.NumFiles++ } size := info.Size() @@ -82,7 +88,7 @@ func (fsrv *FileServer) directoryListing(ctx context.Context, entries []fs.DirEn u := url.URL{Path: "./" + name} // prepend with "./" to fix paths with ':' in the name - fileInfos = append(fileInfos, fileInfo{ + tplCtx.Items = append(tplCtx.Items, fileInfo{ IsDir: isDir, IsSymlink: fileIsSymlink, Name: name, @@ -90,17 +96,11 @@ func (fsrv *FileServer) directoryListing(ctx context.Context, entries []fs.DirEn URL: u.String(), ModTime: info.ModTime().UTC(), Mode: info.Mode(), + Tpl: tplCtx, // a reference up to the template context is useful }) } - name, _ := url.PathUnescape(urlPath) - return browseTemplateContext{ - Name: path.Base(name), - Path: urlPath, - CanGoUp: canGoUp, - Items: fileInfos, - NumDirs: dirCount, - NumFiles: fileCount, - } + + return tplCtx } // browseTemplateContext provides the template context for directory listings. @@ -134,6 +134,9 @@ type browseTemplateContext struct { // Sorting order Order string `json:"order,omitempty"` + + // Display format (list or grid) + Layout string `json:"layout,omitempty"` } // Breadcrumbs returns l.Path where every element maps @@ -227,6 +230,19 @@ type fileInfo struct { Mode os.FileMode `json:"mode"` IsDir bool `json:"is_dir"` IsSymlink bool `json:"is_symlink"` + + // a pointer to the template context is useful inside nested templates + Tpl *browseTemplateContext `json:"-"` +} + +// HasExt returns true if the filename has any of the given suffixes, case-insensitive. +func (fi fileInfo) HasExt(exts ...string) bool { + for _, ext := range exts { + if strings.HasSuffix(strings.ToLower(fi.Name), strings.ToLower(ext)) { + return true + } + } + return false } // HumanSize returns the size of the file as a diff --git a/modules/caddyhttp/fileserver/browsetplcontext_test.go b/modules/caddyhttp/fileserver/browsetplcontext_test.go index 9f0d08e..184196f 100644 --- a/modules/caddyhttp/fileserver/browsetplcontext_test.go +++ b/modules/caddyhttp/fileserver/browsetplcontext_test.go @@ -25,6 +25,45 @@ func TestBreadcrumbs(t *testing.T) { }{ {"", []crumb{}}, {"/", []crumb{{Text: "/"}}}, + {"/foo/", []crumb{ + {Link: "../", Text: "/"}, + {Link: "", Text: "foo"}, + }}, + {"/foo/bar/", []crumb{ + {Link: "../../", Text: "/"}, + {Link: "../", Text: "foo"}, + {Link: "", Text: "bar"}, + }}, + {"/foo bar/", []crumb{ + {Link: "../", Text: "/"}, + {Link: "", Text: "foo bar"}, + }}, + {"/foo bar/baz/", []crumb{ + {Link: "../../", Text: "/"}, + {Link: "../", Text: "foo bar"}, + {Link: "", Text: "baz"}, + }}, + {"/100%25 test coverage/is a lie/", []crumb{ + {Link: "../../", Text: "/"}, + {Link: "../", Text: "100% test coverage"}, + {Link: "", Text: "is a lie"}, + }}, + {"/AC%2FDC/", []crumb{ + {Link: "../", Text: "/"}, + {Link: "", Text: "AC/DC"}, + }}, + {"/foo/%2e%2e%2f/bar", []crumb{ + {Link: "../../../", Text: "/"}, + {Link: "../../", Text: "foo"}, + {Link: "../", Text: "../"}, + {Link: "", Text: "bar"}, + }}, + {"/foo/../bar", []crumb{ + {Link: "../../../", Text: "/"}, + {Link: "../../", Text: "foo"}, + {Link: "../", Text: ".."}, + {Link: "", Text: "bar"}, + }}, {"foo/bar/baz", []crumb{ {Link: "../../", Text: "foo"}, {Link: "../", Text: "bar"}, @@ -51,16 +90,16 @@ func TestBreadcrumbs(t *testing.T) { }}, } - for _, d := range testdata { + for testNum, d := range testdata { l := browseTemplateContext{Path: d.path} actual := l.Breadcrumbs() if len(actual) != len(d.expected) { - t.Errorf("wrong size output, got %d elements but expected %d", len(actual), len(d.expected)) + t.Errorf("Test %d: Got %d components but expected %d; got: %+v", testNum, len(actual), len(d.expected), actual) continue } for i, c := range actual { if c != d.expected[i] { - t.Errorf("got %#v but expected %#v at index %d", c, d.expected[i], i) + t.Errorf("Test %d crumb %d: got %#v but expected %#v at index %d", testNum, i, c, d.expected[i], i) } } } diff --git a/modules/caddyhttp/fileserver/command.go b/modules/caddyhttp/fileserver/command.go index bc7f981..d46c204 100644 --- a/modules/caddyhttp/fileserver/command.go +++ b/modules/caddyhttp/fileserver/command.go @@ -16,24 +16,27 @@ package fileserver import ( "encoding/json" - "flag" + "io" "log" + "os" "strconv" "time" + "github.com/caddyserver/certmagic" + "github.com/spf13/cobra" + "go.uber.org/zap" + + caddycmd "github.com/caddyserver/caddy/v2/cmd" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" - caddycmd "github.com/caddyserver/caddy/v2/cmd" "github.com/caddyserver/caddy/v2/modules/caddyhttp" caddytpl "github.com/caddyserver/caddy/v2/modules/caddyhttp/templates" - "github.com/caddyserver/certmagic" - "go.uber.org/zap" ) func init() { caddycmd.RegisterCommand(caddycmd.Command{ Name: "file-server", - Func: cmdFileServer, Usage: "[--domain <example.com>] [--root <path>] [--listen <addr>] [--browse] [--access-log]", Short: "Spins up a production-ready file server", Long: ` @@ -49,17 +52,25 @@ using this option. If --browse is enabled, requests for folders without an index file will respond with a file listing.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("file-server", flag.ExitOnError) - fs.String("domain", "", "Domain name at which to serve the files") - fs.String("root", "", "The path to the root of the site") - fs.String("listen", "", "The address to which to bind the listener") - fs.Bool("browse", false, "Enable directory browsing") - fs.Bool("templates", false, "Enable template rendering") - fs.Bool("access-log", false, "Enable the access log") - fs.Bool("debug", false, "Enable verbose debug logs") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("domain", "d", "", "Domain name at which to serve the files") + cmd.Flags().StringP("root", "r", "", "The path to the root of the site") + cmd.Flags().StringP("listen", "l", "", "The address to which to bind the listener") + cmd.Flags().BoolP("browse", "b", false, "Enable directory browsing") + cmd.Flags().BoolP("templates", "t", false, "Enable template rendering") + cmd.Flags().BoolP("access-log", "a", false, "Enable the access log") + cmd.Flags().BoolP("debug", "v", false, "Enable verbose debug logs") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdFileServer) + cmd.AddCommand(&cobra.Command{ + Use: "export-template", + Short: "Exports the default file browser template", + Example: "caddy file-server export-template > browse.html", + RunE: func(cmd *cobra.Command, args []string) error { + _, err := io.WriteString(os.Stdout, BrowseTemplate) + return err + }, + }) + }, }) } @@ -136,7 +147,9 @@ func cmdFileServer(fs caddycmd.Flags) (int, error) { if debug { cfg.Logging = &caddy.Logging{ Logs: map[string]*caddy.CustomLog{ - "default": {Level: zap.DebugLevel.CapitalString()}, + "default": { + BaseLog: caddy.BaseLog{Level: zap.DebugLevel.CapitalString()}, + }, }, } } diff --git a/modules/caddyhttp/fileserver/matcher.go b/modules/caddyhttp/fileserver/matcher.go index 1cdc87c..c8f5b22 100644 --- a/modules/caddyhttp/fileserver/matcher.go +++ b/modules/caddyhttp/fileserver/matcher.go @@ -26,9 +26,6 @@ import ( "strconv" "strings" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" - "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/google/cel-go/cel" "github.com/google/cel-go/common" "github.com/google/cel-go/common/operators" @@ -36,6 +33,10 @@ import ( "github.com/google/cel-go/parser" "go.uber.org/zap" exprpb "google.golang.org/genproto/googleapis/api/expr/v1alpha1" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) func init() { @@ -558,7 +559,7 @@ func indexFold(haystack, needle string) int { return -1 } -// isCELMapLiteral returns whether the expression resolves to a map literal containing +// isCELTryFilesLiteral returns whether the expression resolves to a map literal containing // only string keys with or a placeholder call. func isCELTryFilesLiteral(e *exprpb.Expr) bool { switch e.GetExprKind().(type) { diff --git a/modules/caddyhttp/fileserver/staticfiles.go b/modules/caddyhttp/fileserver/staticfiles.go index c0fde66..0ed558e 100644 --- a/modules/caddyhttp/fileserver/staticfiles.go +++ b/modules/caddyhttp/fileserver/staticfiles.go @@ -29,17 +29,15 @@ import ( "runtime" "strconv" "strings" - "time" + + "go.uber.org/zap" "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode" - "go.uber.org/zap" ) func init() { - weakrand.Seed(time.Now().UnixNano()) - caddy.RegisterModule(FileServer{}) } @@ -62,7 +60,23 @@ func init() { // requested directory does not have an index file, Caddy writes a // 404 response. Alternatively, file browsing can be enabled with // the "browse" parameter which shows a list of files when directories -// are requested if no index file is present. +// are requested if no index file is present. If "browse" is enabled, +// Caddy may serve a JSON array of the dirctory listing when the `Accept` +// header mentions `application/json` with the following structure: +// +// [{ +// "name": "", +// "size": 0, +// "url": "", +// "mod_time": "", +// "mode": 0, +// "is_dir": false, +// "is_symlink": false +// }] +// +// with the `url` being relative to the request path and `mod_time` in the RFC 3339 format +// with sub-second precision. For any other value for the `Accept` header, the +// respective browse template is executed with `Content-Type: text/html`. // // By default, this handler will canonicalize URIs so that requests to // directories end with a slash, but requests to regular files do not. @@ -250,7 +264,8 @@ func (fsrv *FileServer) ServeHTTP(w http.ResponseWriter, r *http.Request, next c root := repl.ReplaceAll(fsrv.Root, ".") - filename := caddyhttp.SanitizedPathJoin(root, r.URL.Path) + // remove any trailing `/` as it breaks fs.ValidPath() in the stdlib + filename := strings.TrimSuffix(caddyhttp.SanitizedPathJoin(root, r.URL.Path), "/") fsrv.logger.Debug("sanitized path join", zap.String("site_root", root), @@ -355,7 +370,9 @@ func (fsrv *FileServer) ServeHTTP(w http.ResponseWriter, r *http.Request, next c } var file fs.File - var etag string + + // etag is usually unset, but if the user knows what they're doing, let them override it + etag := w.Header().Get("Etag") // check for precompressed files for _, ae := range encode.AcceptedEncodings(r, fsrv.PrecompressedOrder) { @@ -387,7 +404,9 @@ func (fsrv *FileServer) ServeHTTP(w http.ResponseWriter, r *http.Request, next c // don't assign info = compressedInfo because sidecars are kind // of transparent; however we do need to set the Etag: // https://caddy.community/t/gzipped-sidecar-file-wrong-same-etag/16793 - etag = calculateEtag(compressedInfo) + if etag == "" { + etag = calculateEtag(compressedInfo) + } break } @@ -407,20 +426,29 @@ func (fsrv *FileServer) ServeHTTP(w http.ResponseWriter, r *http.Request, next c } defer file.Close() - etag = calculateEtag(info) + if etag == "" { + etag = calculateEtag(info) + } } // at this point, we're serving a file; Go std lib supports only // GET and HEAD, which is sensible for a static file server - reject // any other methods (see issue #5166) if r.Method != http.MethodGet && r.Method != http.MethodHead { - w.Header().Add("Allow", "GET, HEAD") - return caddyhttp.Error(http.StatusMethodNotAllowed, nil) + // if we're in an error context, then it doesn't make sense + // to repeat the error; just continue because we're probably + // trying to write an error page response (see issue #5703) + if _, ok := r.Context().Value(caddyhttp.ErrorCtxKey).(error); !ok { + w.Header().Add("Allow", "GET, HEAD") + return caddyhttp.Error(http.StatusMethodNotAllowed, nil) + } } // set the Etag - note that a conditional If-None-Match request is handled // by http.ServeContent below, which checks against this Etag value - w.Header().Set("Etag", etag) + if etag != "" { + w.Header().Set("Etag", etag) + } if w.Header().Get("Content-Type") == "" { mtyp := mime.TypeByExtension(filepath.Ext(filename)) @@ -607,7 +635,11 @@ func (fsrv *FileServer) notFound(w http.ResponseWriter, r *http.Request, next ca // Prefix the etag with "W/" to convert it into a weak etag. // See: https://tools.ietf.org/html/rfc7232#section-2.3 func calculateEtag(d os.FileInfo) string { - t := strconv.FormatInt(d.ModTime().Unix(), 36) + mtime := d.ModTime().Unix() + if mtime == 0 || mtime == 1 { + return "" // not useful anyway; see issue #5548 + } + t := strconv.FormatInt(mtime, 36) s := strconv.FormatInt(d.Size(), 36) return `"` + t + s + `"` } @@ -635,6 +667,12 @@ func (wr statusOverrideResponseWriter) WriteHeader(int) { wr.ResponseWriter.WriteHeader(wr.code) } +// Unwrap returns the underlying ResponseWriter, necessary for +// http.ResponseController to work correctly. +func (wr statusOverrideResponseWriter) Unwrap() http.ResponseWriter { + return wr.ResponseWriter +} + // osFS is a simple fs.FS implementation that uses the local // file system. (We do not use os.DirFS because we do our own // rooting or path prefixing without being constrained to a single diff --git a/modules/caddyhttp/headers/caddyfile.go b/modules/caddyhttp/headers/caddyfile.go index a6bec95..2b06910 100644 --- a/modules/caddyhttp/headers/caddyfile.go +++ b/modules/caddyhttp/headers/caddyfile.go @@ -32,19 +32,20 @@ func init() { // parseCaddyfile sets up the handler for response headers from // Caddyfile tokens. Syntax: // -// header [<matcher>] [[+|-|?]<field> [<value|regexp>] [<replacement>]] { -// [+]<field> [<value|regexp> [<replacement>]] -// ?<field> <default_value> -// -<field> -// [defer] +// header [<matcher>] [[+|-|?|>]<field> [<value|regexp>] [<replacement>]] { +// [+]<field> [<value|regexp> [<replacement>]] +// ?<field> <default_value> +// -<field> +// ><field> +// [defer] // } // // Either a block can be opened or a single header field can be configured // in the first line, but not both in the same directive. Header operations // are deferred to write-time if any headers are being deleted or if the // 'defer' subdirective is used. + appends a header value, - deletes a field, -// and ? conditionally sets a value only if the header field is not already -// set. +// ? conditionally sets a value only if the header field is not already set, +// and > sets a field with defer enabled. func parseCaddyfile(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error) { if !h.Next() { return nil, h.ArgErr() @@ -246,10 +247,14 @@ func applyHeaderOp(ops *HeaderOps, respHeaderOps *RespHeaderOps, field, value, r respHeaderOps.Set.Set(field, value) case replacement != "": // replace + // allow defer shortcut for replace syntax + if strings.HasPrefix(field, ">") && respHeaderOps != nil { + respHeaderOps.Deferred = true + } if ops.Replace == nil { ops.Replace = make(map[string][]Replacement) } - field = strings.TrimLeft(field, "+-?") + field = strings.TrimLeft(field, "+-?>") ops.Replace[field] = append( ops.Replace[field], Replacement{ @@ -258,6 +263,15 @@ func applyHeaderOp(ops *HeaderOps, respHeaderOps *RespHeaderOps, field, value, r }, ) + case strings.HasPrefix(field, ">"): // set (overwrite) with defer + if ops.Set == nil { + ops.Set = make(http.Header) + } + ops.Set.Set(field[1:], value) + if respHeaderOps != nil { + respHeaderOps.Deferred = true + } + default: // set (overwrite) if ops.Set == nil { ops.Set = make(http.Header) diff --git a/modules/caddyhttp/headers/headers.go b/modules/caddyhttp/headers/headers.go index f8d3fdc..ed503ef 100644 --- a/modules/caddyhttp/headers/headers.go +++ b/modules/caddyhttp/headers/headers.go @@ -192,6 +192,19 @@ type RespHeaderOps struct { // ApplyTo applies ops to hdr using repl. func (ops HeaderOps) ApplyTo(hdr http.Header, repl *caddy.Replacer) { + // before manipulating headers in other ways, check if there + // is configuration to delete all headers, and do that first + // because if a header is to be added, we don't want to delete + // it also + for _, fieldName := range ops.Delete { + fieldName = repl.ReplaceKnown(fieldName, "") + if fieldName == "*" { + for existingField := range hdr { + delete(hdr, existingField) + } + } + } + // add for fieldName, vals := range ops.Add { fieldName = repl.ReplaceKnown(fieldName, "") @@ -215,6 +228,9 @@ func (ops HeaderOps) ApplyTo(hdr http.Header, repl *caddy.Replacer) { // delete for _, fieldName := range ops.Delete { fieldName = strings.ToLower(repl.ReplaceKnown(fieldName, "")) + if fieldName == "*" { + continue // handled above + } switch { case strings.HasPrefix(fieldName, "*") && strings.HasSuffix(fieldName, "*"): for existingField := range hdr { @@ -355,5 +371,5 @@ func (rww *responseWriterWrapper) Write(d []byte) (int, error) { var ( _ caddy.Provisioner = (*Handler)(nil) _ caddyhttp.MiddlewareHandler = (*Handler)(nil) - _ caddyhttp.HTTPInterfaces = (*responseWriterWrapper)(nil) + _ http.ResponseWriter = (*responseWriterWrapper)(nil) ) diff --git a/modules/caddyhttp/http2listener.go b/modules/caddyhttp/http2listener.go new file mode 100644 index 0000000..51b356a --- /dev/null +++ b/modules/caddyhttp/http2listener.go @@ -0,0 +1,102 @@ +package caddyhttp + +import ( + "context" + "crypto/tls" + weakrand "math/rand" + "net" + "net/http" + "sync/atomic" + "time" + + "golang.org/x/net/http2" +) + +// http2Listener wraps the listener to solve the following problems: +// 1. server h2 natively without using h2c hack when listener handles tls connection but +// don't return *tls.Conn +// 2. graceful shutdown. the shutdown logic is copied from stdlib http.Server, it's an extra maintenance burden but +// whatever, the shutdown logic maybe extracted to be used with h2c graceful shutdown. http2.Server supports graceful shutdown +// sending GO_AWAY frame to connected clients, but doesn't track connection status. It requires explicit call of http2.ConfigureServer +type http2Listener struct { + cnt uint64 + net.Listener + server *http.Server + h2server *http2.Server +} + +type connectionStateConn interface { + net.Conn + ConnectionState() tls.ConnectionState +} + +func (h *http2Listener) Accept() (net.Conn, error) { + for { + conn, err := h.Listener.Accept() + if err != nil { + return nil, err + } + + if csc, ok := conn.(connectionStateConn); ok { + // *tls.Conn will return empty string because it's only populated after handshake is complete + if csc.ConnectionState().NegotiatedProtocol == http2.NextProtoTLS { + go h.serveHttp2(csc) + continue + } + } + + return conn, nil + } +} + +func (h *http2Listener) serveHttp2(csc connectionStateConn) { + atomic.AddUint64(&h.cnt, 1) + h.runHook(csc, http.StateNew) + defer func() { + csc.Close() + atomic.AddUint64(&h.cnt, ^uint64(0)) + h.runHook(csc, http.StateClosed) + }() + h.h2server.ServeConn(csc, &http2.ServeConnOpts{ + Context: h.server.ConnContext(context.Background(), csc), + BaseConfig: h.server, + Handler: h.server.Handler, + }) +} + +const shutdownPollIntervalMax = 500 * time.Millisecond + +func (h *http2Listener) Shutdown(ctx context.Context) error { + pollIntervalBase := time.Millisecond + nextPollInterval := func() time.Duration { + // Add 10% jitter. + //nolint:gosec + interval := pollIntervalBase + time.Duration(weakrand.Intn(int(pollIntervalBase/10))) + // Double and clamp for next time. + pollIntervalBase *= 2 + if pollIntervalBase > shutdownPollIntervalMax { + pollIntervalBase = shutdownPollIntervalMax + } + return interval + } + + timer := time.NewTimer(nextPollInterval()) + defer timer.Stop() + for { + if atomic.LoadUint64(&h.cnt) == 0 { + return nil + } + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + timer.Reset(nextPollInterval()) + } + } +} + +func (h *http2Listener) runHook(conn net.Conn, state http.ConnState) { + if h.server.ConnState != nil { + h.server.ConnState(conn, state) + } +} diff --git a/modules/caddyhttp/httpredirectlistener.go b/modules/caddyhttp/httpredirectlistener.go index 3ff79ff..082dc7c 100644 --- a/modules/caddyhttp/httpredirectlistener.go +++ b/modules/caddyhttp/httpredirectlistener.go @@ -17,6 +17,7 @@ package caddyhttp import ( "bufio" "fmt" + "io" "net" "net/http" "sync" @@ -42,7 +43,11 @@ func init() { // // This listener wrapper must be placed BEFORE the "tls" listener // wrapper, for it to work properly. -type HTTPRedirectListenerWrapper struct{} +type HTTPRedirectListenerWrapper struct { + // MaxHeaderBytes is the maximum size to parse from a client's + // HTTP request headers. Default: 1 MB + MaxHeaderBytes int64 `json:"max_header_bytes,omitempty"` +} func (HTTPRedirectListenerWrapper) CaddyModule() caddy.ModuleInfo { return caddy.ModuleInfo{ @@ -56,7 +61,7 @@ func (h *HTTPRedirectListenerWrapper) UnmarshalCaddyfile(d *caddyfile.Dispenser) } func (h *HTTPRedirectListenerWrapper) WrapListener(l net.Listener) net.Listener { - return &httpRedirectListener{l} + return &httpRedirectListener{l, h.MaxHeaderBytes} } // httpRedirectListener is listener that checks the first few bytes @@ -64,6 +69,7 @@ func (h *HTTPRedirectListenerWrapper) WrapListener(l net.Listener) net.Listener // to respond to an HTTP request with a redirect. type httpRedirectListener struct { net.Listener + maxHeaderBytes int64 } // Accept waits for and returns the next connection to the listener, @@ -74,9 +80,14 @@ func (l *httpRedirectListener) Accept() (net.Conn, error) { return nil, err } + maxHeaderBytes := l.maxHeaderBytes + if maxHeaderBytes == 0 { + maxHeaderBytes = 1024 * 1024 + } + return &httpRedirectConn{ Conn: c, - r: bufio.NewReader(c), + r: bufio.NewReader(io.LimitReader(c, maxHeaderBytes)), }, nil } diff --git a/modules/caddyhttp/invoke.go b/modules/caddyhttp/invoke.go new file mode 100644 index 0000000..97fd1cc --- /dev/null +++ b/modules/caddyhttp/invoke.go @@ -0,0 +1,56 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package caddyhttp + +import ( + "fmt" + "net/http" + + "github.com/caddyserver/caddy/v2" +) + +func init() { + caddy.RegisterModule(Invoke{}) +} + +// Invoke implements a handler that compiles and executes a +// named route that was defined on the server. +// +// EXPERIMENTAL: Subject to change or removal. +type Invoke struct { + // Name is the key of the named route to execute + Name string `json:"name,omitempty"` +} + +// CaddyModule returns the Caddy module information. +func (Invoke) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.handlers.invoke", + New: func() caddy.Module { return new(Invoke) }, + } +} + +func (invoke *Invoke) ServeHTTP(w http.ResponseWriter, r *http.Request, next Handler) error { + server := r.Context().Value(ServerCtxKey).(*Server) + if route, ok := server.NamedRoutes[invoke.Name]; ok { + return route.Compile(next).ServeHTTP(w, r) + } + return fmt.Errorf("invoke: route '%s' not found", invoke.Name) +} + +// Interface guards +var ( + _ MiddlewareHandler = (*Invoke)(nil) +) diff --git a/modules/caddyhttp/ip_matchers.go b/modules/caddyhttp/ip_matchers.go new file mode 100644 index 0000000..57a2295 --- /dev/null +++ b/modules/caddyhttp/ip_matchers.go @@ -0,0 +1,345 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package caddyhttp + +import ( + "errors" + "fmt" + "net" + "net/http" + "net/netip" + "reflect" + "strings" + + "github.com/google/cel-go/cel" + "github.com/google/cel-go/common/types/ref" + "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" +) + +// MatchRemoteIP matches requests by the remote IP address, +// i.e. the IP address of the direct connection to Caddy. +type MatchRemoteIP struct { + // The IPs or CIDR ranges to match. + Ranges []string `json:"ranges,omitempty"` + + // If true, prefer the first IP in the request's X-Forwarded-For + // header, if present, rather than the immediate peer's IP, as + // the reference IP against which to match. Note that it is easy + // to spoof request headers. Default: false + // DEPRECATED: This is insecure, MatchClientIP should be used instead. + Forwarded bool `json:"forwarded,omitempty"` + + // cidrs and zones vars should aligned always in the same + // length and indexes for matching later + cidrs []*netip.Prefix + zones []string + logger *zap.Logger +} + +// MatchClientIP matches requests by the client IP address, +// i.e. the resolved address, considering trusted proxies. +type MatchClientIP struct { + // The IPs or CIDR ranges to match. + Ranges []string `json:"ranges,omitempty"` + + // cidrs and zones vars should aligned always in the same + // length and indexes for matching later + cidrs []*netip.Prefix + zones []string + logger *zap.Logger +} + +func init() { + caddy.RegisterModule(MatchRemoteIP{}) + caddy.RegisterModule(MatchClientIP{}) +} + +// CaddyModule returns the Caddy module information. +func (MatchRemoteIP) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.matchers.remote_ip", + New: func() caddy.Module { return new(MatchRemoteIP) }, + } +} + +// UnmarshalCaddyfile implements caddyfile.Unmarshaler. +func (m *MatchRemoteIP) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + for d.NextArg() { + if d.Val() == "forwarded" { + if len(m.Ranges) > 0 { + return d.Err("if used, 'forwarded' must be first argument") + } + m.Forwarded = true + continue + } + if d.Val() == "private_ranges" { + m.Ranges = append(m.Ranges, PrivateRangesCIDR()...) + continue + } + m.Ranges = append(m.Ranges, d.Val()) + } + if d.NextBlock(0) { + return d.Err("malformed remote_ip matcher: blocks are not supported") + } + } + return nil +} + +// CELLibrary produces options that expose this matcher for use in CEL +// expression matchers. +// +// Example: +// +// expression remote_ip('forwarded', '192.168.0.0/16', '172.16.0.0/12', '10.0.0.0/8') +func (MatchRemoteIP) CELLibrary(ctx caddy.Context) (cel.Library, error) { + return CELMatcherImpl( + // name of the macro, this is the function name that users see when writing expressions. + "remote_ip", + // name of the function that the macro will be rewritten to call. + "remote_ip_match_request_list", + // internal data type of the MatchPath value. + []*cel.Type{cel.ListType(cel.StringType)}, + // function to convert a constant list of strings to a MatchPath instance. + func(data ref.Val) (RequestMatcher, error) { + refStringList := reflect.TypeOf([]string{}) + strList, err := data.ConvertToNative(refStringList) + if err != nil { + return nil, err + } + + m := MatchRemoteIP{} + + for _, input := range strList.([]string) { + if input == "forwarded" { + if len(m.Ranges) > 0 { + return nil, errors.New("if used, 'forwarded' must be first argument") + } + m.Forwarded = true + continue + } + m.Ranges = append(m.Ranges, input) + } + + err = m.Provision(ctx) + return m, err + }, + ) +} + +// Provision parses m's IP ranges, either from IP or CIDR expressions. +func (m *MatchRemoteIP) Provision(ctx caddy.Context) error { + m.logger = ctx.Logger() + cidrs, zones, err := provisionCidrsZonesFromRanges(m.Ranges) + if err != nil { + return err + } + m.cidrs = cidrs + m.zones = zones + + if m.Forwarded { + m.logger.Warn("remote_ip's forwarded mode is deprecated; use the 'client_ip' matcher instead") + } + + return nil +} + +// Match returns true if r matches m. +func (m MatchRemoteIP) Match(r *http.Request) bool { + address := r.RemoteAddr + if m.Forwarded { + if fwdFor := r.Header.Get("X-Forwarded-For"); fwdFor != "" { + address = strings.TrimSpace(strings.Split(fwdFor, ",")[0]) + } + } + clientIP, zoneID, err := parseIPZoneFromString(address) + if err != nil { + m.logger.Error("getting remote IP", zap.Error(err)) + return false + } + matches, zoneFilter := matchIPByCidrZones(clientIP, zoneID, m.cidrs, m.zones) + if !matches && !zoneFilter { + m.logger.Debug("zone ID from remote IP did not match", zap.String("zone", zoneID)) + } + return matches +} + +// CaddyModule returns the Caddy module information. +func (MatchClientIP) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.matchers.client_ip", + New: func() caddy.Module { return new(MatchClientIP) }, + } +} + +// UnmarshalCaddyfile implements caddyfile.Unmarshaler. +func (m *MatchClientIP) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + for d.NextArg() { + if d.Val() == "private_ranges" { + m.Ranges = append(m.Ranges, PrivateRangesCIDR()...) + continue + } + m.Ranges = append(m.Ranges, d.Val()) + } + if d.NextBlock(0) { + return d.Err("malformed client_ip matcher: blocks are not supported") + } + } + return nil +} + +// CELLibrary produces options that expose this matcher for use in CEL +// expression matchers. +// +// Example: +// +// expression client_ip('192.168.0.0/16', '172.16.0.0/12', '10.0.0.0/8') +func (MatchClientIP) CELLibrary(ctx caddy.Context) (cel.Library, error) { + return CELMatcherImpl( + // name of the macro, this is the function name that users see when writing expressions. + "client_ip", + // name of the function that the macro will be rewritten to call. + "client_ip_match_request_list", + // internal data type of the MatchPath value. + []*cel.Type{cel.ListType(cel.StringType)}, + // function to convert a constant list of strings to a MatchPath instance. + func(data ref.Val) (RequestMatcher, error) { + refStringList := reflect.TypeOf([]string{}) + strList, err := data.ConvertToNative(refStringList) + if err != nil { + return nil, err + } + + m := MatchClientIP{ + Ranges: strList.([]string), + } + + err = m.Provision(ctx) + return m, err + }, + ) +} + +// Provision parses m's IP ranges, either from IP or CIDR expressions. +func (m *MatchClientIP) Provision(ctx caddy.Context) error { + m.logger = ctx.Logger() + cidrs, zones, err := provisionCidrsZonesFromRanges(m.Ranges) + if err != nil { + return err + } + m.cidrs = cidrs + m.zones = zones + return nil +} + +// Match returns true if r matches m. +func (m MatchClientIP) Match(r *http.Request) bool { + address := GetVar(r.Context(), ClientIPVarKey).(string) + clientIP, zoneID, err := parseIPZoneFromString(address) + if err != nil { + m.logger.Error("getting client IP", zap.Error(err)) + return false + } + matches, zoneFilter := matchIPByCidrZones(clientIP, zoneID, m.cidrs, m.zones) + if !matches && !zoneFilter { + m.logger.Debug("zone ID from client IP did not match", zap.String("zone", zoneID)) + } + return matches +} + +func provisionCidrsZonesFromRanges(ranges []string) ([]*netip.Prefix, []string, error) { + cidrs := []*netip.Prefix{} + zones := []string{} + for _, str := range ranges { + // Exclude the zone_id from the IP + if strings.Contains(str, "%") { + split := strings.Split(str, "%") + str = split[0] + // write zone identifiers in m.zones for matching later + zones = append(zones, split[1]) + } else { + zones = append(zones, "") + } + if strings.Contains(str, "/") { + ipNet, err := netip.ParsePrefix(str) + if err != nil { + return nil, nil, fmt.Errorf("parsing CIDR expression '%s': %v", str, err) + } + cidrs = append(cidrs, &ipNet) + } else { + ipAddr, err := netip.ParseAddr(str) + if err != nil { + return nil, nil, fmt.Errorf("invalid IP address: '%s': %v", str, err) + } + ipNew := netip.PrefixFrom(ipAddr, ipAddr.BitLen()) + cidrs = append(cidrs, &ipNew) + } + } + return cidrs, zones, nil +} + +func parseIPZoneFromString(address string) (netip.Addr, string, error) { + ipStr, _, err := net.SplitHostPort(address) + if err != nil { + ipStr = address // OK; probably didn't have a port + } + + // Some IPv6-Adresses can contain zone identifiers at the end, + // which are separated with "%" + zoneID := "" + if strings.Contains(ipStr, "%") { + split := strings.Split(ipStr, "%") + ipStr = split[0] + zoneID = split[1] + } + + ipAddr, err := netip.ParseAddr(ipStr) + if err != nil { + return netip.IPv4Unspecified(), "", err + } + + return ipAddr, zoneID, nil +} + +func matchIPByCidrZones(clientIP netip.Addr, zoneID string, cidrs []*netip.Prefix, zones []string) (bool, bool) { + zoneFilter := true + for i, ipRange := range cidrs { + if ipRange.Contains(clientIP) { + // Check if there are zone filters assigned and if they match. + if zones[i] == "" || zoneID == zones[i] { + return true, false + } + zoneFilter = false + } + } + return false, zoneFilter +} + +// Interface guards +var ( + _ RequestMatcher = (*MatchRemoteIP)(nil) + _ caddy.Provisioner = (*MatchRemoteIP)(nil) + _ caddyfile.Unmarshaler = (*MatchRemoteIP)(nil) + _ CELLibraryProducer = (*MatchRemoteIP)(nil) + + _ RequestMatcher = (*MatchClientIP)(nil) + _ caddy.Provisioner = (*MatchClientIP)(nil) + _ caddyfile.Unmarshaler = (*MatchClientIP)(nil) + _ CELLibraryProducer = (*MatchClientIP)(nil) +) diff --git a/modules/caddyhttp/logging.go b/modules/caddyhttp/logging.go index 4faaec7..8ecc49a 100644 --- a/modules/caddyhttp/logging.go +++ b/modules/caddyhttp/logging.go @@ -22,6 +22,8 @@ import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" + + "github.com/caddyserver/caddy/v2" ) // ServerLogConfig describes a server's logging configuration. If @@ -139,6 +141,21 @@ func errLogValues(err error) (status int, msg string, fields []zapcore.Field) { return } -// Variable name used to indicate that this request -// should be omitted from the access logs -const SkipLogVar = "skip_log" +// ExtraLogFields is a list of extra fields to log with every request. +type ExtraLogFields struct { + fields []zapcore.Field +} + +// Add adds a field to the list of extra fields to log. +func (e *ExtraLogFields) Add(field zap.Field) { + e.fields = append(e.fields, field) +} + +const ( + // Variable name used to indicate that this request + // should be omitted from the access logs + SkipLogVar string = "skip_log" + + // For adding additional fields to the access logs + ExtraLogFieldsCtxKey caddy.CtxKey = "extra_log_fields" +) diff --git a/modules/caddyhttp/marshalers.go b/modules/caddyhttp/marshalers.go index e6fc3a6..9a955e3 100644 --- a/modules/caddyhttp/marshalers.go +++ b/modules/caddyhttp/marshalers.go @@ -40,6 +40,7 @@ func (r LoggableHTTPRequest) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("remote_ip", ip) enc.AddString("remote_port", port) + enc.AddString("client_ip", GetVar(r.Context(), ClientIPVarKey).(string)) enc.AddString("proto", r.Proto) enc.AddString("method", r.Method) enc.AddString("host", r.Host) diff --git a/modules/caddyhttp/matchers.go b/modules/caddyhttp/matchers.go index 3064300..b385979 100644 --- a/modules/caddyhttp/matchers.go +++ b/modules/caddyhttp/matchers.go @@ -20,22 +20,22 @@ import ( "fmt" "net" "net/http" - "net/netip" "net/textproto" "net/url" "path" "reflect" "regexp" + "runtime" "sort" "strconv" "strings" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/google/cel-go/cel" "github.com/google/cel-go/common/types" "github.com/google/cel-go/common/types/ref" - "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) type ( @@ -176,24 +176,6 @@ type ( // "http/2", "http/3", or minimum versions: "http/2+", etc. MatchProtocol string - // MatchRemoteIP matches requests by client IP (or CIDR range). - MatchRemoteIP struct { - // The IPs or CIDR ranges to match. - Ranges []string `json:"ranges,omitempty"` - - // If true, prefer the first IP in the request's X-Forwarded-For - // header, if present, rather than the immediate peer's IP, as - // the reference IP against which to match. Note that it is easy - // to spoof request headers. Default: false - Forwarded bool `json:"forwarded,omitempty"` - - // cidrs and zones vars should aligned always in the same - // length and indexes for matching later - cidrs []*netip.Prefix - zones []string - logger *zap.Logger - } - // MatchNot matches requests by negating the results of its matcher // sets. A single "not" matcher takes one or more matcher sets. Each // matcher set is OR'ed; in other words, if any matcher set returns @@ -229,7 +211,6 @@ func init() { caddy.RegisterModule(MatchHeader{}) caddy.RegisterModule(MatchHeaderRE{}) caddy.RegisterModule(new(MatchProtocol)) - caddy.RegisterModule(MatchRemoteIP{}) caddy.RegisterModule(MatchNot{}) } @@ -416,7 +397,9 @@ func (m MatchPath) Match(r *http.Request) bool { // security risk (cry) if PHP files end up being served // as static files, exposing the source code, instead of // being matched by *.php to be treated as PHP scripts. - reqPath = strings.TrimRight(reqPath, ". ") + if runtime.GOOS == "windows" { // issue #5613 + reqPath = strings.TrimRight(reqPath, ". ") + } repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer) @@ -1261,159 +1244,6 @@ func (m MatchNot) Match(r *http.Request) bool { return true } -// CaddyModule returns the Caddy module information. -func (MatchRemoteIP) CaddyModule() caddy.ModuleInfo { - return caddy.ModuleInfo{ - ID: "http.matchers.remote_ip", - New: func() caddy.Module { return new(MatchRemoteIP) }, - } -} - -// UnmarshalCaddyfile implements caddyfile.Unmarshaler. -func (m *MatchRemoteIP) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { - for d.Next() { - for d.NextArg() { - if d.Val() == "forwarded" { - if len(m.Ranges) > 0 { - return d.Err("if used, 'forwarded' must be first argument") - } - m.Forwarded = true - continue - } - if d.Val() == "private_ranges" { - m.Ranges = append(m.Ranges, PrivateRangesCIDR()...) - continue - } - m.Ranges = append(m.Ranges, d.Val()) - } - if d.NextBlock(0) { - return d.Err("malformed remote_ip matcher: blocks are not supported") - } - } - return nil -} - -// CELLibrary produces options that expose this matcher for use in CEL -// expression matchers. -// -// Example: -// -// expression remote_ip('forwarded', '192.168.0.0/16', '172.16.0.0/12', '10.0.0.0/8') -func (MatchRemoteIP) CELLibrary(ctx caddy.Context) (cel.Library, error) { - return CELMatcherImpl( - // name of the macro, this is the function name that users see when writing expressions. - "remote_ip", - // name of the function that the macro will be rewritten to call. - "remote_ip_match_request_list", - // internal data type of the MatchPath value. - []*cel.Type{cel.ListType(cel.StringType)}, - // function to convert a constant list of strings to a MatchPath instance. - func(data ref.Val) (RequestMatcher, error) { - refStringList := reflect.TypeOf([]string{}) - strList, err := data.ConvertToNative(refStringList) - if err != nil { - return nil, err - } - - m := MatchRemoteIP{} - - for _, input := range strList.([]string) { - if input == "forwarded" { - if len(m.Ranges) > 0 { - return nil, errors.New("if used, 'forwarded' must be first argument") - } - m.Forwarded = true - continue - } - m.Ranges = append(m.Ranges, input) - } - - err = m.Provision(ctx) - return m, err - }, - ) -} - -// Provision parses m's IP ranges, either from IP or CIDR expressions. -func (m *MatchRemoteIP) Provision(ctx caddy.Context) error { - m.logger = ctx.Logger() - for _, str := range m.Ranges { - // Exclude the zone_id from the IP - if strings.Contains(str, "%") { - split := strings.Split(str, "%") - str = split[0] - // write zone identifiers in m.zones for matching later - m.zones = append(m.zones, split[1]) - } else { - m.zones = append(m.zones, "") - } - if strings.Contains(str, "/") { - ipNet, err := netip.ParsePrefix(str) - if err != nil { - return fmt.Errorf("parsing CIDR expression '%s': %v", str, err) - } - m.cidrs = append(m.cidrs, &ipNet) - } else { - ipAddr, err := netip.ParseAddr(str) - if err != nil { - return fmt.Errorf("invalid IP address: '%s': %v", str, err) - } - ipNew := netip.PrefixFrom(ipAddr, ipAddr.BitLen()) - m.cidrs = append(m.cidrs, &ipNew) - } - } - return nil -} - -func (m MatchRemoteIP) getClientIP(r *http.Request) (netip.Addr, string, error) { - remote := r.RemoteAddr - zoneID := "" - if m.Forwarded { - if fwdFor := r.Header.Get("X-Forwarded-For"); fwdFor != "" { - remote = strings.TrimSpace(strings.Split(fwdFor, ",")[0]) - } - } - ipStr, _, err := net.SplitHostPort(remote) - if err != nil { - ipStr = remote // OK; probably didn't have a port - } - // Some IPv6-Adresses can contain zone identifiers at the end, - // which are separated with "%" - if strings.Contains(ipStr, "%") { - split := strings.Split(ipStr, "%") - ipStr = split[0] - zoneID = split[1] - } - ipAddr, err := netip.ParseAddr(ipStr) - if err != nil { - return netip.IPv4Unspecified(), "", err - } - return ipAddr, zoneID, nil -} - -// Match returns true if r matches m. -func (m MatchRemoteIP) Match(r *http.Request) bool { - clientIP, zoneID, err := m.getClientIP(r) - if err != nil { - m.logger.Error("getting client IP", zap.Error(err)) - return false - } - zoneFilter := true - for i, ipRange := range m.cidrs { - if ipRange.Contains(clientIP) { - // Check if there are zone filters assigned and if they match. - if m.zones[i] == "" || zoneID == m.zones[i] { - return true - } - zoneFilter = false - } - } - if !zoneFilter { - m.logger.Debug("zone ID from remote did not match", zap.String("zone", zoneID)) - } - return false -} - // MatchRegexp is an embedable type for matching // using regular expressions. It adds placeholders // to the request's replacer. @@ -1563,9 +1393,7 @@ func ParseCaddyfileNestedMatcherSet(d *caddyfile.Dispenser) (caddy.ModuleMap, er return matcherSet, nil } -var ( - wordRE = regexp.MustCompile(`\w+`) -) +var wordRE = regexp.MustCompile(`\w+`) const regexpPlaceholderPrefix = "http.regexp" @@ -1588,8 +1416,6 @@ var ( _ RequestMatcher = (*MatchHeaderRE)(nil) _ caddy.Provisioner = (*MatchHeaderRE)(nil) _ RequestMatcher = (*MatchProtocol)(nil) - _ RequestMatcher = (*MatchRemoteIP)(nil) - _ caddy.Provisioner = (*MatchRemoteIP)(nil) _ RequestMatcher = (*MatchNot)(nil) _ caddy.Provisioner = (*MatchNot)(nil) _ caddy.Provisioner = (*MatchRegexp)(nil) @@ -1602,7 +1428,6 @@ var ( _ caddyfile.Unmarshaler = (*MatchHeader)(nil) _ caddyfile.Unmarshaler = (*MatchHeaderRE)(nil) _ caddyfile.Unmarshaler = (*MatchProtocol)(nil) - _ caddyfile.Unmarshaler = (*MatchRemoteIP)(nil) _ caddyfile.Unmarshaler = (*VarsMatcher)(nil) _ caddyfile.Unmarshaler = (*MatchVarsRE)(nil) @@ -1614,7 +1439,6 @@ var ( _ CELLibraryProducer = (*MatchHeader)(nil) _ CELLibraryProducer = (*MatchHeaderRE)(nil) _ CELLibraryProducer = (*MatchProtocol)(nil) - _ CELLibraryProducer = (*MatchRemoteIP)(nil) // _ CELLibraryProducer = (*VarsMatcher)(nil) // _ CELLibraryProducer = (*MatchVarsRE)(nil) diff --git a/modules/caddyhttp/matchers_test.go b/modules/caddyhttp/matchers_test.go index 4f5da69..041975d 100644 --- a/modules/caddyhttp/matchers_test.go +++ b/modules/caddyhttp/matchers_test.go @@ -21,6 +21,7 @@ import ( "net/http/httptest" "net/url" "os" + "runtime" "testing" "github.com/caddyserver/caddy/v2" @@ -254,11 +255,6 @@ func TestPathMatcher(t *testing.T) { expect: true, }, { - match: MatchPath{"*.php"}, - input: "/foo/index.php. .", - expect: true, - }, - { match: MatchPath{"/foo/bar.txt"}, input: "/foo/BAR.txt", expect: true, @@ -435,8 +431,10 @@ func TestPathMatcher(t *testing.T) { func TestPathMatcherWindows(t *testing.T) { // only Windows has this bug where it will ignore - // trailing dots and spaces in a filename, but we - // test for it on all platforms to be more consistent + // trailing dots and spaces in a filename + if runtime.GOOS != "windows" { + return + } req := &http.Request{URL: &url.URL{Path: "/index.php . . .."}} repl := caddy.NewReplacer() diff --git a/modules/caddyhttp/metrics.go b/modules/caddyhttp/metrics.go index 64fbed7..9c0f961 100644 --- a/modules/caddyhttp/metrics.go +++ b/modules/caddyhttp/metrics.go @@ -6,9 +6,10 @@ import ( "sync" "time" - "github.com/caddyserver/caddy/v2/internal/metrics" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promauto" + + "github.com/caddyserver/caddy/v2/internal/metrics" ) // Metrics configures metrics observations. diff --git a/modules/caddyhttp/proxyprotocol/listenerwrapper.go b/modules/caddyhttp/proxyprotocol/listenerwrapper.go new file mode 100644 index 0000000..f404c06 --- /dev/null +++ b/modules/caddyhttp/proxyprotocol/listenerwrapper.go @@ -0,0 +1,69 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package proxyprotocol + +import ( + "fmt" + "net" + "time" + + "github.com/mastercactapus/proxyprotocol" + + "github.com/caddyserver/caddy/v2" +) + +// ListenerWrapper provides PROXY protocol support to Caddy by implementing +// the caddy.ListenerWrapper interface. It must be loaded before the `tls` listener. +// +// Credit goes to https://github.com/mastercactapus/caddy2-proxyprotocol for having +// initially implemented this as a plugin. +type ListenerWrapper struct { + // Timeout specifies an optional maximum time for + // the PROXY header to be received. + // If zero, timeout is disabled. Default is 5s. + Timeout caddy.Duration `json:"timeout,omitempty"` + + // Allow is an optional list of CIDR ranges to + // allow/require PROXY headers from. + Allow []string `json:"allow,omitempty"` + + rules []proxyprotocol.Rule +} + +// Provision sets up the listener wrapper. +func (pp *ListenerWrapper) Provision(ctx caddy.Context) error { + rules := make([]proxyprotocol.Rule, 0, len(pp.Allow)) + for _, s := range pp.Allow { + _, n, err := net.ParseCIDR(s) + if err != nil { + return fmt.Errorf("invalid subnet '%s': %w", s, err) + } + rules = append(rules, proxyprotocol.Rule{ + Timeout: time.Duration(pp.Timeout), + Subnet: n, + }) + } + + pp.rules = rules + + return nil +} + +// WrapListener adds PROXY protocol support to the listener. +func (pp *ListenerWrapper) WrapListener(l net.Listener) net.Listener { + pl := proxyprotocol.NewListener(l, time.Duration(pp.Timeout)) + pl.SetFilter(pp.rules) + return pl +} diff --git a/modules/caddyhttp/proxyprotocol/module.go b/modules/caddyhttp/proxyprotocol/module.go new file mode 100644 index 0000000..78f89c6 --- /dev/null +++ b/modules/caddyhttp/proxyprotocol/module.go @@ -0,0 +1,75 @@ +// Copyright 2015 Matthew Holt and The Caddy Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package proxyprotocol + +import ( + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" +) + +func init() { + caddy.RegisterModule(ListenerWrapper{}) +} + +func (ListenerWrapper) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "caddy.listeners.proxy_protocol", + New: func() caddy.Module { return new(ListenerWrapper) }, + } +} + +// UnmarshalCaddyfile sets up the listener Listenerwrapper from Caddyfile tokens. Syntax: +// +// proxy_protocol { +// timeout <duration> +// allow <IPs...> +// } +func (w *ListenerWrapper) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + // No same-line options are supported + if d.NextArg() { + return d.ArgErr() + } + + for d.NextBlock(0) { + switch d.Val() { + case "timeout": + if !d.NextArg() { + return d.ArgErr() + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("parsing proxy_protocol timeout duration: %v", err) + } + w.Timeout = caddy.Duration(dur) + + case "allow": + w.Allow = append(w.Allow, d.RemainingArgs()...) + + default: + return d.ArgErr() + } + } + } + return nil +} + +// Interface guards +var ( + _ caddy.Provisioner = (*ListenerWrapper)(nil) + _ caddy.Module = (*ListenerWrapper)(nil) + _ caddy.ListenerWrapper = (*ListenerWrapper)(nil) + _ caddyfile.Unmarshaler = (*ListenerWrapper)(nil) +) diff --git a/modules/caddyhttp/push/handler.go b/modules/caddyhttp/push/handler.go index 60eadd0..031a899 100644 --- a/modules/caddyhttp/push/handler.go +++ b/modules/caddyhttp/push/handler.go @@ -19,10 +19,11 @@ import ( "net/http" "strings" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers" - "go.uber.org/zap" ) func init() { @@ -251,5 +252,6 @@ const pushedLink = "http.handlers.push.pushed_link" var ( _ caddy.Provisioner = (*Handler)(nil) _ caddyhttp.MiddlewareHandler = (*Handler)(nil) - _ caddyhttp.HTTPInterfaces = (*linkPusher)(nil) + _ http.ResponseWriter = (*linkPusher)(nil) + _ http.Pusher = (*linkPusher)(nil) ) diff --git a/modules/caddyhttp/replacer.go b/modules/caddyhttp/replacer.go index c58b56e..f6b042c 100644 --- a/modules/caddyhttp/replacer.go +++ b/modules/caddyhttp/replacer.go @@ -39,9 +39,10 @@ import ( "strings" "time" + "github.com/google/uuid" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddytls" - "github.com/google/uuid" ) // NewTestReplacer creates a replacer for an http.Request diff --git a/modules/caddyhttp/requestbody/caddyfile.go b/modules/caddyhttp/requestbody/caddyfile.go index 0a2459f..8a92909 100644 --- a/modules/caddyhttp/requestbody/caddyfile.go +++ b/modules/caddyhttp/requestbody/caddyfile.go @@ -15,9 +15,10 @@ package requestbody import ( + "github.com/dustin/go-humanize" + "github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "github.com/dustin/go-humanize" ) func init() { diff --git a/modules/caddyhttp/responsewriter.go b/modules/caddyhttp/responsewriter.go index 1b28cf0..37c2646 100644 --- a/modules/caddyhttp/responsewriter.go +++ b/modules/caddyhttp/responsewriter.go @@ -24,34 +24,14 @@ import ( ) // ResponseWriterWrapper wraps an underlying ResponseWriter and -// promotes its Pusher/Flusher/Hijacker methods as well. To use -// this type, embed a pointer to it within your own struct type -// that implements the http.ResponseWriter interface, then call -// methods on the embedded value. You can make sure your type -// wraps correctly by asserting that it implements the -// HTTPInterfaces interface. +// promotes its Pusher method as well. To use this type, embed +// a pointer to it within your own struct type that implements +// the http.ResponseWriter interface, then call methods on the +// embedded value. type ResponseWriterWrapper struct { http.ResponseWriter } -// Hijack implements http.Hijacker. It simply calls the underlying -// ResponseWriter's Hijack method if there is one, or returns -// ErrNotImplemented otherwise. -func (rww *ResponseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) { - if hj, ok := rww.ResponseWriter.(http.Hijacker); ok { - return hj.Hijack() - } - return nil, nil, ErrNotImplemented -} - -// Flush implements http.Flusher. It simply calls the underlying -// ResponseWriter's Flush method if there is one. -func (rww *ResponseWriterWrapper) Flush() { - if f, ok := rww.ResponseWriter.(http.Flusher); ok { - f.Flush() - } -} - // Push implements http.Pusher. It simply calls the underlying // ResponseWriter's Push method if there is one, or returns // ErrNotImplemented otherwise. @@ -62,22 +42,16 @@ func (rww *ResponseWriterWrapper) Push(target string, opts *http.PushOptions) er return ErrNotImplemented } -// ReadFrom implements io.ReaderFrom. It simply calls the underlying -// ResponseWriter's ReadFrom method if there is one, otherwise it defaults -// to io.Copy. +// ReadFrom implements io.ReaderFrom. It simply calls io.Copy, +// which uses io.ReaderFrom if available. func (rww *ResponseWriterWrapper) ReadFrom(r io.Reader) (n int64, err error) { - if rf, ok := rww.ResponseWriter.(io.ReaderFrom); ok { - return rf.ReadFrom(r) - } return io.Copy(rww.ResponseWriter, r) } -// HTTPInterfaces mix all the interfaces that middleware ResponseWriters need to support. -type HTTPInterfaces interface { - http.ResponseWriter - http.Pusher - http.Flusher - http.Hijacker +// Unwrap returns the underlying ResponseWriter, necessary for +// http.ResponseController to work correctly. +func (rww *ResponseWriterWrapper) Unwrap() http.ResponseWriter { + return rww.ResponseWriter } // ErrNotImplemented is returned when an underlying @@ -257,7 +231,8 @@ func (rr *responseRecorder) WriteResponse() error { } func (rr *responseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) { - conn, brw, err := rr.ResponseWriterWrapper.Hijack() + //nolint:bodyclose + conn, brw, err := http.NewResponseController(rr.ResponseWriterWrapper).Hijack() if err != nil { return nil, nil, err } @@ -289,7 +264,7 @@ func (hc *hijackedConn) ReadFrom(r io.Reader) (int64, error) { // responses instead of writing them to the client. See // docs for NewResponseRecorder for proper usage. type ResponseRecorder interface { - HTTPInterfaces + http.ResponseWriter Status() int Buffer() *bytes.Buffer Buffered() bool @@ -304,12 +279,13 @@ type ShouldBufferFunc func(status int, header http.Header) bool // Interface guards var ( - _ HTTPInterfaces = (*ResponseWriterWrapper)(nil) - _ ResponseRecorder = (*responseRecorder)(nil) + _ http.ResponseWriter = (*ResponseWriterWrapper)(nil) + _ ResponseRecorder = (*responseRecorder)(nil) // Implementing ReaderFrom can be such a significant // optimization that it should probably be required! // see PR #5022 (25%-50% speedup) _ io.ReaderFrom = (*ResponseWriterWrapper)(nil) _ io.ReaderFrom = (*responseRecorder)(nil) + _ io.ReaderFrom = (*hijackedConn)(nil) ) diff --git a/modules/caddyhttp/responsewriter_test.go b/modules/caddyhttp/responsewriter_test.go index 1913932..492fcad 100644 --- a/modules/caddyhttp/responsewriter_test.go +++ b/modules/caddyhttp/responsewriter_test.go @@ -95,6 +95,14 @@ func TestResponseWriterWrapperReadFrom(t *testing.T) { } } +func TestResponseWriterWrapperUnwrap(t *testing.T) { + w := &ResponseWriterWrapper{&baseRespWriter{}} + + if _, ok := w.Unwrap().(*baseRespWriter); !ok { + t.Errorf("Unwrap() doesn't return the underlying ResponseWriter") + } +} + func TestResponseRecorderReadFrom(t *testing.T) { tests := map[string]struct { responseWriter responseWriterSpy diff --git a/modules/caddyhttp/reverseproxy/addresses.go b/modules/caddyhttp/reverseproxy/addresses.go index 8152108..82c1c79 100644 --- a/modules/caddyhttp/reverseproxy/addresses.go +++ b/modules/caddyhttp/reverseproxy/addresses.go @@ -23,11 +23,46 @@ import ( "github.com/caddyserver/caddy/v2" ) +type parsedAddr struct { + network, scheme, host, port string + valid bool +} + +func (p parsedAddr) dialAddr() string { + if !p.valid { + return "" + } + // for simplest possible config, we only need to include + // the network portion if the user specified one + if p.network != "" { + return caddy.JoinNetworkAddress(p.network, p.host, p.port) + } + + // if the host is a placeholder, then we don't want to join with an empty port, + // because that would just append an extra ':' at the end of the address. + if p.port == "" && strings.Contains(p.host, "{") { + return p.host + } + return net.JoinHostPort(p.host, p.port) +} + +func (p parsedAddr) rangedPort() bool { + return strings.Contains(p.port, "-") +} + +func (p parsedAddr) replaceablePort() bool { + return strings.Contains(p.port, "{") && strings.Contains(p.port, "}") +} + +func (p parsedAddr) isUnix() bool { + return caddy.IsUnixNetwork(p.network) +} + // parseUpstreamDialAddress parses configuration inputs for // the dial address, including support for a scheme in front // as a shortcut for the port number, and a network type, // for example 'unix' to dial a unix socket. -func parseUpstreamDialAddress(upstreamAddr string) (string, string, error) { +func parseUpstreamDialAddress(upstreamAddr string) (parsedAddr, error) { var network, scheme, host, port string if strings.Contains(upstreamAddr, "://") { @@ -35,46 +70,65 @@ func parseUpstreamDialAddress(upstreamAddr string) (string, string, error) { // so we return a more user-friendly error message instead // to explain what to do instead if strings.Contains(upstreamAddr, "{") { - return "", "", fmt.Errorf("due to parsing difficulties, placeholders are not allowed when an upstream address contains a scheme") + return parsedAddr{}, fmt.Errorf("due to parsing difficulties, placeholders are not allowed when an upstream address contains a scheme") } toURL, err := url.Parse(upstreamAddr) if err != nil { - return "", "", fmt.Errorf("parsing upstream URL: %v", err) + // if the error seems to be due to a port range, + // try to replace the port range with a dummy + // single port so that url.Parse() will succeed + if strings.Contains(err.Error(), "invalid port") && strings.Contains(err.Error(), "-") { + index := strings.LastIndex(upstreamAddr, ":") + if index == -1 { + return parsedAddr{}, fmt.Errorf("parsing upstream URL: %v", err) + } + portRange := upstreamAddr[index+1:] + if strings.Count(portRange, "-") != 1 { + return parsedAddr{}, fmt.Errorf("parsing upstream URL: parse \"%v\": port range invalid: %v", upstreamAddr, portRange) + } + toURL, err = url.Parse(strings.ReplaceAll(upstreamAddr, portRange, "0")) + if err != nil { + return parsedAddr{}, fmt.Errorf("parsing upstream URL: %v", err) + } + port = portRange + } else { + return parsedAddr{}, fmt.Errorf("parsing upstream URL: %v", err) + } + } + if port == "" { + port = toURL.Port() } // there is currently no way to perform a URL rewrite between choosing // a backend and proxying to it, so we cannot allow extra components // in backend URLs if toURL.Path != "" || toURL.RawQuery != "" || toURL.Fragment != "" { - return "", "", fmt.Errorf("for now, URLs for proxy upstreams only support scheme, host, and port components") + return parsedAddr{}, fmt.Errorf("for now, URLs for proxy upstreams only support scheme, host, and port components") } // ensure the port and scheme aren't in conflict - urlPort := toURL.Port() - if toURL.Scheme == "http" && urlPort == "443" { - return "", "", fmt.Errorf("upstream address has conflicting scheme (http://) and port (:443, the HTTPS port)") + if toURL.Scheme == "http" && port == "443" { + return parsedAddr{}, fmt.Errorf("upstream address has conflicting scheme (http://) and port (:443, the HTTPS port)") } - if toURL.Scheme == "https" && urlPort == "80" { - return "", "", fmt.Errorf("upstream address has conflicting scheme (https://) and port (:80, the HTTP port)") + if toURL.Scheme == "https" && port == "80" { + return parsedAddr{}, fmt.Errorf("upstream address has conflicting scheme (https://) and port (:80, the HTTP port)") } - if toURL.Scheme == "h2c" && urlPort == "443" { - return "", "", fmt.Errorf("upstream address has conflicting scheme (h2c://) and port (:443, the HTTPS port)") + if toURL.Scheme == "h2c" && port == "443" { + return parsedAddr{}, fmt.Errorf("upstream address has conflicting scheme (h2c://) and port (:443, the HTTPS port)") } // if port is missing, attempt to infer from scheme - if toURL.Port() == "" { - var toPort string + if port == "" { switch toURL.Scheme { case "", "http", "h2c": - toPort = "80" + port = "80" case "https": - toPort = "443" + port = "443" } - toURL.Host = net.JoinHostPort(toURL.Hostname(), toPort) } - scheme, host, port = toURL.Scheme, toURL.Hostname(), toURL.Port() + scheme, host = toURL.Scheme, toURL.Hostname() } else { var err error network, host, port, err = caddy.SplitNetworkAddress(upstreamAddr) @@ -93,18 +147,5 @@ func parseUpstreamDialAddress(upstreamAddr string) (string, string, error) { network = "unix" scheme = "h2c" } - - // for simplest possible config, we only need to include - // the network portion if the user specified one - if network != "" { - return caddy.JoinNetworkAddress(network, host, port), scheme, nil - } - - // if the host is a placeholder, then we don't want to join with an empty port, - // because that would just append an extra ':' at the end of the address. - if port == "" && strings.Contains(host, "{") { - return host, scheme, nil - } - - return net.JoinHostPort(host, port), scheme, nil + return parsedAddr{network, scheme, host, port, true}, nil } diff --git a/modules/caddyhttp/reverseproxy/addresses_test.go b/modules/caddyhttp/reverseproxy/addresses_test.go index 6355c75..0c51419 100644 --- a/modules/caddyhttp/reverseproxy/addresses_test.go +++ b/modules/caddyhttp/reverseproxy/addresses_test.go @@ -150,6 +150,24 @@ func TestParseUpstreamDialAddress(t *testing.T) { expectScheme: "h2c", }, { + input: "localhost:1001-1009", + expectHostPort: "localhost:1001-1009", + }, + { + input: "{host}:1001-1009", + expectHostPort: "{host}:1001-1009", + }, + { + input: "http://localhost:1001-1009", + expectHostPort: "localhost:1001-1009", + expectScheme: "http", + }, + { + input: "https://localhost:1001-1009", + expectHostPort: "localhost:1001-1009", + expectScheme: "https", + }, + { input: "unix//var/php.sock", expectHostPort: "unix//var/php.sock", }, @@ -197,6 +215,26 @@ func TestParseUpstreamDialAddress(t *testing.T) { expectErr: true, }, { + input: "http://localhost:8001-8002-8003", + expectErr: true, + }, + { + input: "http://localhost:8001-8002/foo:bar", + expectErr: true, + }, + { + input: "http://localhost:8001-8002/foo:1", + expectErr: true, + }, + { + input: "http://localhost:8001-8002/foo:1-2", + expectErr: true, + }, + { + input: "http://localhost:8001-8002#foo:1", + expectErr: true, + }, + { input: "http://foo:443", expectErr: true, }, @@ -227,18 +265,18 @@ func TestParseUpstreamDialAddress(t *testing.T) { expectScheme: "h2c", }, } { - actualHostPort, actualScheme, err := parseUpstreamDialAddress(tc.input) + actualAddr, err := parseUpstreamDialAddress(tc.input) if tc.expectErr && err == nil { t.Errorf("Test %d: Expected error but got %v", i, err) } if !tc.expectErr && err != nil { t.Errorf("Test %d: Expected no error but got %v", i, err) } - if actualHostPort != tc.expectHostPort { - t.Errorf("Test %d: Expected host and port '%s' but got '%s'", i, tc.expectHostPort, actualHostPort) + if actualAddr.dialAddr() != tc.expectHostPort { + t.Errorf("Test %d: input %s: Expected host and port '%s' but got '%s'", i, tc.input, tc.expectHostPort, actualAddr.dialAddr()) } - if actualScheme != tc.expectScheme { - t.Errorf("Test %d: Expected scheme '%s' but got '%s'", i, tc.expectScheme, actualScheme) + if actualAddr.scheme != tc.expectScheme { + t.Errorf("Test %d: Expected scheme '%s' but got '%s'", i, tc.expectScheme, actualAddr.scheme) } } } diff --git a/modules/caddyhttp/reverseproxy/caddyfile.go b/modules/caddyhttp/reverseproxy/caddyfile.go index 1211188..bcbe744 100644 --- a/modules/caddyhttp/reverseproxy/caddyfile.go +++ b/modules/caddyhttp/reverseproxy/caddyfile.go @@ -15,12 +15,14 @@ package reverseproxy import ( - "net" + "fmt" "net/http" "reflect" "strconv" "strings" + "github.com/dustin/go-humanize" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" @@ -28,7 +30,6 @@ import ( "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers" "github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite" - "github.com/dustin/go-humanize" ) func init() { @@ -83,10 +84,13 @@ func parseCaddyfile(h httpcaddyfile.Helper) (caddyhttp.MiddlewareHandler, error) // unhealthy_request_count <num> // // # streaming -// flush_interval <duration> +// flush_interval <duration> // buffer_requests // buffer_responses -// max_buffer_size <size> +// max_buffer_size <size> +// stream_timeout <duration> +// stream_close_delay <duration> +// trace_logs // // # request manipulation // trusted_proxies [private_ranges] <ranges...> @@ -142,16 +146,9 @@ func (h *Handler) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { h.responseMatchers = make(map[string]caddyhttp.ResponseMatcher) // appendUpstream creates an upstream for address and adds - // it to the list. If the address starts with "srv+" it is - // treated as a SRV-based upstream, and any port will be - // dropped. + // it to the list. appendUpstream := func(address string) error { - isSRV := strings.HasPrefix(address, "srv+") - if isSRV { - address = strings.TrimPrefix(address, "srv+") - } - - dialAddr, scheme, err := parseUpstreamDialAddress(address) + pa, err := parseUpstreamDialAddress(address) if err != nil { return d.WrapErr(err) } @@ -159,573 +156,641 @@ func (h *Handler) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { // the underlying JSON does not yet support different // transports (protocols or schemes) to each backend, // so we remember the last one we see and compare them - if commonScheme != "" && scheme != commonScheme { + + switch pa.scheme { + case "wss": + return d.Errf("the scheme wss:// is only supported in browsers; use https:// instead") + case "ws": + return d.Errf("the scheme ws:// is only supported in browsers; use http:// instead") + case "https", "http", "h2c", "": + // Do nothing or handle the valid schemes + default: + return d.Errf("unsupported URL scheme %s://", pa.scheme) + } + + if commonScheme != "" && pa.scheme != commonScheme { return d.Errf("for now, all proxy upstreams must use the same scheme (transport protocol); expecting '%s://' but got '%s://'", - commonScheme, scheme) + commonScheme, pa.scheme) } - commonScheme = scheme + commonScheme = pa.scheme - if isSRV { - if host, _, err := net.SplitHostPort(dialAddr); err == nil { - dialAddr = host - } - h.Upstreams = append(h.Upstreams, &Upstream{LookupSRV: dialAddr}) + // if the port of upstream address contains a placeholder, only wrap it with the `Upstream` struct, + // delaying actual resolution of the address until request time. + if pa.replaceablePort() { + h.Upstreams = append(h.Upstreams, &Upstream{Dial: pa.dialAddr()}) + return nil + } + parsedAddr, err := caddy.ParseNetworkAddress(pa.dialAddr()) + if err != nil { + return d.WrapErr(err) + } + + if pa.isUnix() || !pa.rangedPort() { + // unix networks don't have ports + h.Upstreams = append(h.Upstreams, &Upstream{ + Dial: pa.dialAddr(), + }) } else { - h.Upstreams = append(h.Upstreams, &Upstream{Dial: dialAddr}) + // expand a port range into multiple upstreams + for i := parsedAddr.StartPort; i <= parsedAddr.EndPort; i++ { + h.Upstreams = append(h.Upstreams, &Upstream{ + Dial: caddy.JoinNetworkAddress("", parsedAddr.Host, fmt.Sprint(i)), + }) + } } + return nil } - for d.Next() { - for _, up := range d.RemainingArgs() { - err := appendUpstream(up) + d.Next() // consume the directive name + for _, up := range d.RemainingArgs() { + err := appendUpstream(up) + if err != nil { + return fmt.Errorf("parsing upstream '%s': %w", up, err) + } + } + + for nesting := d.Nesting(); d.NextBlock(nesting); { + // if the subdirective has an "@" prefix then we + // parse it as a response matcher for use with "handle_response" + if strings.HasPrefix(d.Val(), matcherPrefix) { + err := caddyhttp.ParseNamedResponseMatcher(d.NewFromNextSegment(), h.responseMatchers) if err != nil { return err } + continue } - for d.NextBlock(0) { - // if the subdirective has an "@" prefix then we - // parse it as a response matcher for use with "handle_response" - if strings.HasPrefix(d.Val(), matcherPrefix) { - err := caddyhttp.ParseNamedResponseMatcher(d.NewFromNextSegment(), h.responseMatchers) + switch d.Val() { + case "to": + args := d.RemainingArgs() + if len(args) == 0 { + return d.ArgErr() + } + for _, up := range args { + err := appendUpstream(up) if err != nil { - return err + return fmt.Errorf("parsing upstream '%s': %w", up, err) } - continue } - switch d.Val() { - case "to": - args := d.RemainingArgs() - if len(args) == 0 { - return d.ArgErr() - } - for _, up := range args { - err := appendUpstream(up) - if err != nil { - return err - } - } + case "dynamic": + if !d.NextArg() { + return d.ArgErr() + } + if h.DynamicUpstreams != nil { + return d.Err("dynamic upstreams already specified") + } + dynModule := d.Val() + modID := "http.reverse_proxy.upstreams." + dynModule + unm, err := caddyfile.UnmarshalModule(d, modID) + if err != nil { + return err + } + source, ok := unm.(UpstreamSource) + if !ok { + return d.Errf("module %s (%T) is not an UpstreamSource", modID, unm) + } + h.DynamicUpstreamsRaw = caddyconfig.JSONModuleObject(source, "source", dynModule, nil) - case "dynamic": - if !d.NextArg() { - return d.ArgErr() - } - if h.DynamicUpstreams != nil { - return d.Err("dynamic upstreams already specified") - } - dynModule := d.Val() - modID := "http.reverse_proxy.upstreams." + dynModule - unm, err := caddyfile.UnmarshalModule(d, modID) - if err != nil { - return err - } - source, ok := unm.(UpstreamSource) - if !ok { - return d.Errf("module %s (%T) is not an UpstreamSource", modID, unm) - } - h.DynamicUpstreamsRaw = caddyconfig.JSONModuleObject(source, "source", dynModule, nil) + case "lb_policy": + if !d.NextArg() { + return d.ArgErr() + } + if h.LoadBalancing != nil && h.LoadBalancing.SelectionPolicyRaw != nil { + return d.Err("load balancing selection policy already specified") + } + name := d.Val() + modID := "http.reverse_proxy.selection_policies." + name + unm, err := caddyfile.UnmarshalModule(d, modID) + if err != nil { + return err + } + sel, ok := unm.(Selector) + if !ok { + return d.Errf("module %s (%T) is not a reverseproxy.Selector", modID, unm) + } + if h.LoadBalancing == nil { + h.LoadBalancing = new(LoadBalancing) + } + h.LoadBalancing.SelectionPolicyRaw = caddyconfig.JSONModuleObject(sel, "policy", name, nil) - case "lb_policy": - if !d.NextArg() { - return d.ArgErr() - } - if h.LoadBalancing != nil && h.LoadBalancing.SelectionPolicyRaw != nil { - return d.Err("load balancing selection policy already specified") - } - name := d.Val() - modID := "http.reverse_proxy.selection_policies." + name - unm, err := caddyfile.UnmarshalModule(d, modID) - if err != nil { - return err - } - sel, ok := unm.(Selector) - if !ok { - return d.Errf("module %s (%T) is not a reverseproxy.Selector", modID, unm) - } - if h.LoadBalancing == nil { - h.LoadBalancing = new(LoadBalancing) - } - h.LoadBalancing.SelectionPolicyRaw = caddyconfig.JSONModuleObject(sel, "policy", name, nil) + case "lb_retries": + if !d.NextArg() { + return d.ArgErr() + } + tries, err := strconv.Atoi(d.Val()) + if err != nil { + return d.Errf("bad lb_retries number '%s': %v", d.Val(), err) + } + if h.LoadBalancing == nil { + h.LoadBalancing = new(LoadBalancing) + } + h.LoadBalancing.Retries = tries - case "lb_retries": - if !d.NextArg() { - return d.ArgErr() - } - tries, err := strconv.Atoi(d.Val()) - if err != nil { - return d.Errf("bad lb_retries number '%s': %v", d.Val(), err) - } - if h.LoadBalancing == nil { - h.LoadBalancing = new(LoadBalancing) - } - h.LoadBalancing.Retries = tries + case "lb_try_duration": + if !d.NextArg() { + return d.ArgErr() + } + if h.LoadBalancing == nil { + h.LoadBalancing = new(LoadBalancing) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad duration value %s: %v", d.Val(), err) + } + h.LoadBalancing.TryDuration = caddy.Duration(dur) - case "lb_try_duration": - if !d.NextArg() { - return d.ArgErr() - } - if h.LoadBalancing == nil { - h.LoadBalancing = new(LoadBalancing) - } - dur, err := caddy.ParseDuration(d.Val()) - if err != nil { - return d.Errf("bad duration value %s: %v", d.Val(), err) - } - h.LoadBalancing.TryDuration = caddy.Duration(dur) + case "lb_try_interval": + if !d.NextArg() { + return d.ArgErr() + } + if h.LoadBalancing == nil { + h.LoadBalancing = new(LoadBalancing) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad interval value '%s': %v", d.Val(), err) + } + h.LoadBalancing.TryInterval = caddy.Duration(dur) - case "lb_try_interval": - if !d.NextArg() { - return d.ArgErr() - } - if h.LoadBalancing == nil { - h.LoadBalancing = new(LoadBalancing) - } - dur, err := caddy.ParseDuration(d.Val()) - if err != nil { - return d.Errf("bad interval value '%s': %v", d.Val(), err) - } - h.LoadBalancing.TryInterval = caddy.Duration(dur) + case "lb_retry_match": + matcherSet, err := caddyhttp.ParseCaddyfileNestedMatcherSet(d) + if err != nil { + return d.Errf("failed to parse lb_retry_match: %v", err) + } + if h.LoadBalancing == nil { + h.LoadBalancing = new(LoadBalancing) + } + h.LoadBalancing.RetryMatchRaw = append(h.LoadBalancing.RetryMatchRaw, matcherSet) - case "lb_retry_match": - matcherSet, err := caddyhttp.ParseCaddyfileNestedMatcherSet(d) - if err != nil { - return d.Errf("failed to parse lb_retry_match: %v", err) - } - if h.LoadBalancing == nil { - h.LoadBalancing = new(LoadBalancing) - } - h.LoadBalancing.RetryMatchRaw = append(h.LoadBalancing.RetryMatchRaw, matcherSet) + case "health_uri": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + h.HealthChecks.Active.URI = d.Val() - case "health_uri": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - h.HealthChecks.Active.URI = d.Val() + case "health_path": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + h.HealthChecks.Active.Path = d.Val() + caddy.Log().Named("config.adapter.caddyfile").Warn("the 'health_path' subdirective is deprecated, please use 'health_uri' instead!") - case "health_path": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - h.HealthChecks.Active.Path = d.Val() - caddy.Log().Named("config.adapter.caddyfile").Warn("the 'health_path' subdirective is deprecated, please use 'health_uri' instead!") + case "health_port": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + portNum, err := strconv.Atoi(d.Val()) + if err != nil { + return d.Errf("bad port number '%s': %v", d.Val(), err) + } + h.HealthChecks.Active.Port = portNum - case "health_port": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) + case "health_headers": + healthHeaders := make(http.Header) + for nesting := d.Nesting(); d.NextBlock(nesting); { + key := d.Val() + values := d.RemainingArgs() + if len(values) == 0 { + values = append(values, "") } - portNum, err := strconv.Atoi(d.Val()) - if err != nil { - return d.Errf("bad port number '%s': %v", d.Val(), err) - } - h.HealthChecks.Active.Port = portNum + healthHeaders[key] = append(healthHeaders[key], values...) + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + h.HealthChecks.Active.Headers = healthHeaders - case "health_headers": - healthHeaders := make(http.Header) - for nesting := d.Nesting(); d.NextBlock(nesting); { - key := d.Val() - values := d.RemainingArgs() - if len(values) == 0 { - values = append(values, "") - } - healthHeaders[key] = values - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - h.HealthChecks.Active.Headers = healthHeaders + case "health_interval": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad interval value %s: %v", d.Val(), err) + } + h.HealthChecks.Active.Interval = caddy.Duration(dur) - case "health_interval": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - dur, err := caddy.ParseDuration(d.Val()) - if err != nil { - return d.Errf("bad interval value %s: %v", d.Val(), err) - } - h.HealthChecks.Active.Interval = caddy.Duration(dur) + case "health_timeout": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad timeout value %s: %v", d.Val(), err) + } + h.HealthChecks.Active.Timeout = caddy.Duration(dur) - case "health_timeout": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - dur, err := caddy.ParseDuration(d.Val()) - if err != nil { - return d.Errf("bad timeout value %s: %v", d.Val(), err) - } - h.HealthChecks.Active.Timeout = caddy.Duration(dur) + case "health_status": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + val := d.Val() + if len(val) == 3 && strings.HasSuffix(val, "xx") { + val = val[:1] + } + statusNum, err := strconv.Atoi(val) + if err != nil { + return d.Errf("bad status value '%s': %v", d.Val(), err) + } + h.HealthChecks.Active.ExpectStatus = statusNum - case "health_status": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - val := d.Val() - if len(val) == 3 && strings.HasSuffix(val, "xx") { - val = val[:1] - } - statusNum, err := strconv.Atoi(val) - if err != nil { - return d.Errf("bad status value '%s': %v", d.Val(), err) - } - h.HealthChecks.Active.ExpectStatus = statusNum + case "health_body": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Active == nil { + h.HealthChecks.Active = new(ActiveHealthChecks) + } + h.HealthChecks.Active.ExpectBody = d.Val() - case "health_body": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Active == nil { - h.HealthChecks.Active = new(ActiveHealthChecks) - } - h.HealthChecks.Active.ExpectBody = d.Val() + case "max_fails": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Passive == nil { + h.HealthChecks.Passive = new(PassiveHealthChecks) + } + maxFails, err := strconv.Atoi(d.Val()) + if err != nil { + return d.Errf("invalid maximum fail count '%s': %v", d.Val(), err) + } + h.HealthChecks.Passive.MaxFails = maxFails - case "max_fails": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Passive == nil { - h.HealthChecks.Passive = new(PassiveHealthChecks) + case "fail_duration": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Passive == nil { + h.HealthChecks.Passive = new(PassiveHealthChecks) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad duration value '%s': %v", d.Val(), err) + } + h.HealthChecks.Passive.FailDuration = caddy.Duration(dur) + + case "unhealthy_request_count": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Passive == nil { + h.HealthChecks.Passive = new(PassiveHealthChecks) + } + maxConns, err := strconv.Atoi(d.Val()) + if err != nil { + return d.Errf("invalid maximum connection count '%s': %v", d.Val(), err) + } + h.HealthChecks.Passive.UnhealthyRequestCount = maxConns + + case "unhealthy_status": + args := d.RemainingArgs() + if len(args) == 0 { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Passive == nil { + h.HealthChecks.Passive = new(PassiveHealthChecks) + } + for _, arg := range args { + if len(arg) == 3 && strings.HasSuffix(arg, "xx") { + arg = arg[:1] } - maxFails, err := strconv.Atoi(d.Val()) + statusNum, err := strconv.Atoi(arg) if err != nil { - return d.Errf("invalid maximum fail count '%s': %v", d.Val(), err) + return d.Errf("bad status value '%s': %v", d.Val(), err) } - h.HealthChecks.Passive.MaxFails = maxFails + h.HealthChecks.Passive.UnhealthyStatus = append(h.HealthChecks.Passive.UnhealthyStatus, statusNum) + } - case "fail_duration": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Passive == nil { - h.HealthChecks.Passive = new(PassiveHealthChecks) - } + case "unhealthy_latency": + if !d.NextArg() { + return d.ArgErr() + } + if h.HealthChecks == nil { + h.HealthChecks = new(HealthChecks) + } + if h.HealthChecks.Passive == nil { + h.HealthChecks.Passive = new(PassiveHealthChecks) + } + dur, err := caddy.ParseDuration(d.Val()) + if err != nil { + return d.Errf("bad duration value '%s': %v", d.Val(), err) + } + h.HealthChecks.Passive.UnhealthyLatency = caddy.Duration(dur) + + case "flush_interval": + if !d.NextArg() { + return d.ArgErr() + } + if fi, err := strconv.Atoi(d.Val()); err == nil { + h.FlushInterval = caddy.Duration(fi) + } else { dur, err := caddy.ParseDuration(d.Val()) if err != nil { return d.Errf("bad duration value '%s': %v", d.Val(), err) } - h.HealthChecks.Passive.FailDuration = caddy.Duration(dur) + h.FlushInterval = caddy.Duration(dur) + } - case "unhealthy_request_count": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Passive == nil { - h.HealthChecks.Passive = new(PassiveHealthChecks) - } - maxConns, err := strconv.Atoi(d.Val()) + case "request_buffers", "response_buffers": + subdir := d.Val() + if !d.NextArg() { + return d.ArgErr() + } + val := d.Val() + var size int64 + if val == "unlimited" { + size = -1 + } else { + usize, err := humanize.ParseBytes(val) if err != nil { - return d.Errf("invalid maximum connection count '%s': %v", d.Val(), err) + return d.Errf("invalid byte size '%s': %v", val, err) } - h.HealthChecks.Passive.UnhealthyRequestCount = maxConns + size = int64(usize) + } + if d.NextArg() { + return d.ArgErr() + } + if subdir == "request_buffers" { + h.RequestBuffers = size + } else if subdir == "response_buffers" { + h.ResponseBuffers = size + } - case "unhealthy_status": - args := d.RemainingArgs() - if len(args) == 0 { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Passive == nil { - h.HealthChecks.Passive = new(PassiveHealthChecks) - } - for _, arg := range args { - if len(arg) == 3 && strings.HasSuffix(arg, "xx") { - arg = arg[:1] - } - statusNum, err := strconv.Atoi(arg) - if err != nil { - return d.Errf("bad status value '%s': %v", d.Val(), err) - } - h.HealthChecks.Passive.UnhealthyStatus = append(h.HealthChecks.Passive.UnhealthyStatus, statusNum) - } + // TODO: These three properties are deprecated; remove them sometime after v2.6.4 + case "buffer_requests": // TODO: deprecated + if d.NextArg() { + return d.ArgErr() + } + caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: buffer_requests: use request_buffers instead (with a maximum buffer size)") + h.DeprecatedBufferRequests = true + case "buffer_responses": // TODO: deprecated + if d.NextArg() { + return d.ArgErr() + } + caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: buffer_responses: use response_buffers instead (with a maximum buffer size)") + h.DeprecatedBufferResponses = true + case "max_buffer_size": // TODO: deprecated + if !d.NextArg() { + return d.ArgErr() + } + size, err := humanize.ParseBytes(d.Val()) + if err != nil { + return d.Errf("invalid byte size '%s': %v", d.Val(), err) + } + if d.NextArg() { + return d.ArgErr() + } + caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: max_buffer_size: use request_buffers and/or response_buffers instead (with maximum buffer sizes)") + h.DeprecatedMaxBufferSize = int64(size) - case "unhealthy_latency": - if !d.NextArg() { - return d.ArgErr() - } - if h.HealthChecks == nil { - h.HealthChecks = new(HealthChecks) - } - if h.HealthChecks.Passive == nil { - h.HealthChecks.Passive = new(PassiveHealthChecks) - } + case "stream_timeout": + if !d.NextArg() { + return d.ArgErr() + } + if fi, err := strconv.Atoi(d.Val()); err == nil { + h.StreamTimeout = caddy.Duration(fi) + } else { dur, err := caddy.ParseDuration(d.Val()) if err != nil { return d.Errf("bad duration value '%s': %v", d.Val(), err) } - h.HealthChecks.Passive.UnhealthyLatency = caddy.Duration(dur) - - case "flush_interval": - if !d.NextArg() { - return d.ArgErr() - } - if fi, err := strconv.Atoi(d.Val()); err == nil { - h.FlushInterval = caddy.Duration(fi) - } else { - dur, err := caddy.ParseDuration(d.Val()) - if err != nil { - return d.Errf("bad duration value '%s': %v", d.Val(), err) - } - h.FlushInterval = caddy.Duration(dur) - } + h.StreamTimeout = caddy.Duration(dur) + } - case "request_buffers", "response_buffers": - subdir := d.Val() - if !d.NextArg() { - return d.ArgErr() - } - size, err := humanize.ParseBytes(d.Val()) + case "stream_close_delay": + if !d.NextArg() { + return d.ArgErr() + } + if fi, err := strconv.Atoi(d.Val()); err == nil { + h.StreamCloseDelay = caddy.Duration(fi) + } else { + dur, err := caddy.ParseDuration(d.Val()) if err != nil { - return d.Errf("invalid byte size '%s': %v", d.Val(), err) - } - if d.NextArg() { - return d.ArgErr() - } - if subdir == "request_buffers" { - h.RequestBuffers = int64(size) - } else if subdir == "response_buffers" { - h.ResponseBuffers = int64(size) - + return d.Errf("bad duration value '%s': %v", d.Val(), err) } + h.StreamCloseDelay = caddy.Duration(dur) + } - // TODO: These three properties are deprecated; remove them sometime after v2.6.4 - case "buffer_requests": // TODO: deprecated - if d.NextArg() { - return d.ArgErr() + case "trusted_proxies": + for d.NextArg() { + if d.Val() == "private_ranges" { + h.TrustedProxies = append(h.TrustedProxies, caddyhttp.PrivateRangesCIDR()...) + continue } - caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: buffer_requests: use request_buffers instead (with a maximum buffer size)") - h.DeprecatedBufferRequests = true - case "buffer_responses": // TODO: deprecated - if d.NextArg() { - return d.ArgErr() - } - caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: buffer_responses: use response_buffers instead (with a maximum buffer size)") - h.DeprecatedBufferResponses = true - case "max_buffer_size": // TODO: deprecated - if !d.NextArg() { - return d.ArgErr() - } - size, err := humanize.ParseBytes(d.Val()) - if err != nil { - return d.Errf("invalid byte size '%s': %v", d.Val(), err) - } - if d.NextArg() { - return d.ArgErr() - } - caddy.Log().Named("config.adapter.caddyfile").Warn("DEPRECATED: max_buffer_size: use request_buffers and/or response_buffers instead (with maximum buffer sizes)") - h.DeprecatedMaxBufferSize = int64(size) + h.TrustedProxies = append(h.TrustedProxies, d.Val()) + } - case "trusted_proxies": - for d.NextArg() { - if d.Val() == "private_ranges" { - h.TrustedProxies = append(h.TrustedProxies, caddyhttp.PrivateRangesCIDR()...) - continue - } - h.TrustedProxies = append(h.TrustedProxies, d.Val()) - } + case "header_up": + var err error - case "header_up": - var err error + if h.Headers == nil { + h.Headers = new(headers.Handler) + } + if h.Headers.Request == nil { + h.Headers.Request = new(headers.HeaderOps) + } + args := d.RemainingArgs() - if h.Headers == nil { - h.Headers = new(headers.Handler) + switch len(args) { + case 1: + err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], "", "") + case 2: + // some lint checks, I guess + if strings.EqualFold(args[0], "host") && (args[1] == "{hostport}" || args[1] == "{http.request.hostport}") { + caddy.Log().Named("caddyfile").Warn("Unnecessary header_up Host: the reverse proxy's default behavior is to pass headers to the upstream") } - if h.Headers.Request == nil { - h.Headers.Request = new(headers.HeaderOps) + if strings.EqualFold(args[0], "x-forwarded-for") && (args[1] == "{remote}" || args[1] == "{http.request.remote}" || args[1] == "{remote_host}" || args[1] == "{http.request.remote.host}") { + caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream") } - args := d.RemainingArgs() - - switch len(args) { - case 1: - err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], "", "") - case 2: - // some lint checks, I guess - if strings.EqualFold(args[0], "host") && (args[1] == "{hostport}" || args[1] == "{http.request.hostport}") { - caddy.Log().Named("caddyfile").Warn("Unnecessary header_up Host: the reverse proxy's default behavior is to pass headers to the upstream") - } - if strings.EqualFold(args[0], "x-forwarded-for") && (args[1] == "{remote}" || args[1] == "{http.request.remote}" || args[1] == "{remote_host}" || args[1] == "{http.request.remote.host}") { - caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream") - } - if strings.EqualFold(args[0], "x-forwarded-proto") && (args[1] == "{scheme}" || args[1] == "{http.request.scheme}") { - caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-Proto: the reverse proxy's default behavior is to pass headers to the upstream") - } - if strings.EqualFold(args[0], "x-forwarded-host") && (args[1] == "{host}" || args[1] == "{http.request.host}" || args[1] == "{hostport}" || args[1] == "{http.request.hostport}") { - caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-Host: the reverse proxy's default behavior is to pass headers to the upstream") - } - err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], args[1], "") - case 3: - err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], args[1], args[2]) - default: - return d.ArgErr() + if strings.EqualFold(args[0], "x-forwarded-proto") && (args[1] == "{scheme}" || args[1] == "{http.request.scheme}") { + caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-Proto: the reverse proxy's default behavior is to pass headers to the upstream") } - - if err != nil { - return d.Err(err.Error()) + if strings.EqualFold(args[0], "x-forwarded-host") && (args[1] == "{host}" || args[1] == "{http.request.host}" || args[1] == "{hostport}" || args[1] == "{http.request.hostport}") { + caddy.Log().Named("caddyfile").Warn("Unnecessary header_up X-Forwarded-Host: the reverse proxy's default behavior is to pass headers to the upstream") } + err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], args[1], "") + case 3: + err = headers.CaddyfileHeaderOp(h.Headers.Request, args[0], args[1], args[2]) + default: + return d.ArgErr() + } - case "header_down": - var err error + if err != nil { + return d.Err(err.Error()) + } - if h.Headers == nil { - h.Headers = new(headers.Handler) - } - if h.Headers.Response == nil { - h.Headers.Response = &headers.RespHeaderOps{ - HeaderOps: new(headers.HeaderOps), - } - } - args := d.RemainingArgs() - switch len(args) { - case 1: - err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], "", "") - case 2: - err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], args[1], "") - case 3: - err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], args[1], args[2]) - default: - return d.ArgErr() - } + case "header_down": + var err error - if err != nil { - return d.Err(err.Error()) - } - - case "method": - if !d.NextArg() { - return d.ArgErr() - } - if h.Rewrite == nil { - h.Rewrite = &rewrite.Rewrite{} - } - h.Rewrite.Method = d.Val() - if d.NextArg() { - return d.ArgErr() + if h.Headers == nil { + h.Headers = new(headers.Handler) + } + if h.Headers.Response == nil { + h.Headers.Response = &headers.RespHeaderOps{ + HeaderOps: new(headers.HeaderOps), } + } + args := d.RemainingArgs() + switch len(args) { + case 1: + err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], "", "") + case 2: + err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], args[1], "") + case 3: + err = headers.CaddyfileHeaderOp(h.Headers.Response.HeaderOps, args[0], args[1], args[2]) + default: + return d.ArgErr() + } - case "rewrite": - if !d.NextArg() { - return d.ArgErr() - } - if h.Rewrite == nil { - h.Rewrite = &rewrite.Rewrite{} - } - h.Rewrite.URI = d.Val() - if d.NextArg() { - return d.ArgErr() - } + if err != nil { + return d.Err(err.Error()) + } - case "transport": - if !d.NextArg() { - return d.ArgErr() - } - if h.TransportRaw != nil { - return d.Err("transport already specified") - } - transportModuleName = d.Val() - modID := "http.reverse_proxy.transport." + transportModuleName - unm, err := caddyfile.UnmarshalModule(d, modID) - if err != nil { - return err - } - rt, ok := unm.(http.RoundTripper) - if !ok { - return d.Errf("module %s (%T) is not a RoundTripper", modID, unm) - } - transport = rt + case "method": + if !d.NextArg() { + return d.ArgErr() + } + if h.Rewrite == nil { + h.Rewrite = &rewrite.Rewrite{} + } + h.Rewrite.Method = d.Val() + if d.NextArg() { + return d.ArgErr() + } - case "handle_response": - // delegate the parsing of handle_response to the caller, - // since we need the httpcaddyfile.Helper to parse subroutes. - // See h.FinalizeUnmarshalCaddyfile - h.handleResponseSegments = append(h.handleResponseSegments, d.NewFromNextSegment()) + case "rewrite": + if !d.NextArg() { + return d.ArgErr() + } + if h.Rewrite == nil { + h.Rewrite = &rewrite.Rewrite{} + } + h.Rewrite.URI = d.Val() + if d.NextArg() { + return d.ArgErr() + } - case "replace_status": - args := d.RemainingArgs() - if len(args) != 1 && len(args) != 2 { - return d.Errf("must have one or two arguments: an optional response matcher, and a status code") - } + case "transport": + if !d.NextArg() { + return d.ArgErr() + } + if h.TransportRaw != nil { + return d.Err("transport already specified") + } + transportModuleName = d.Val() + modID := "http.reverse_proxy.transport." + transportModuleName + unm, err := caddyfile.UnmarshalModule(d, modID) + if err != nil { + return err + } + rt, ok := unm.(http.RoundTripper) + if !ok { + return d.Errf("module %s (%T) is not a RoundTripper", modID, unm) + } + transport = rt + + case "handle_response": + // delegate the parsing of handle_response to the caller, + // since we need the httpcaddyfile.Helper to parse subroutes. + // See h.FinalizeUnmarshalCaddyfile + h.handleResponseSegments = append(h.handleResponseSegments, d.NewFromNextSegment()) + + case "replace_status": + args := d.RemainingArgs() + if len(args) != 1 && len(args) != 2 { + return d.Errf("must have one or two arguments: an optional response matcher, and a status code") + } - responseHandler := caddyhttp.ResponseHandler{} + responseHandler := caddyhttp.ResponseHandler{} - if len(args) == 2 { - if !strings.HasPrefix(args[0], matcherPrefix) { - return d.Errf("must use a named response matcher, starting with '@'") - } - foundMatcher, ok := h.responseMatchers[args[0]] - if !ok { - return d.Errf("no named response matcher defined with name '%s'", args[0][1:]) - } - responseHandler.Match = &foundMatcher - responseHandler.StatusCode = caddyhttp.WeakString(args[1]) - } else if len(args) == 1 { - responseHandler.StatusCode = caddyhttp.WeakString(args[0]) + if len(args) == 2 { + if !strings.HasPrefix(args[0], matcherPrefix) { + return d.Errf("must use a named response matcher, starting with '@'") } - - // make sure there's no block, cause it doesn't make sense - if d.NextBlock(1) { - return d.Errf("cannot define routes for 'replace_status', use 'handle_response' instead.") + foundMatcher, ok := h.responseMatchers[args[0]] + if !ok { + return d.Errf("no named response matcher defined with name '%s'", args[0][1:]) } + responseHandler.Match = &foundMatcher + responseHandler.StatusCode = caddyhttp.WeakString(args[1]) + } else if len(args) == 1 { + responseHandler.StatusCode = caddyhttp.WeakString(args[0]) + } - h.HandleResponse = append( - h.HandleResponse, - responseHandler, - ) + // make sure there's no block, cause it doesn't make sense + if d.NextBlock(1) { + return d.Errf("cannot define routes for 'replace_status', use 'handle_response' instead.") + } - default: - return d.Errf("unrecognized subdirective %s", d.Val()) + h.HandleResponse = append( + h.HandleResponse, + responseHandler, + ) + + case "verbose_logs": + if h.VerboseLogs { + return d.Err("verbose_logs already specified") } + h.VerboseLogs = true + + default: + return d.Errf("unrecognized subdirective %s", d.Val()) } } @@ -918,6 +983,17 @@ func (h *HTTPTransport) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { } h.MaxResponseHeaderSize = int64(size) + case "proxy_protocol": + if !d.NextArg() { + return d.ArgErr() + } + switch proxyProtocol := d.Val(); proxyProtocol { + case "v1", "v2": + h.ProxyProtocol = proxyProtocol + default: + return d.Errf("invalid proxy protocol version '%s'", proxyProtocol) + } + case "dial_timeout": if !d.NextArg() { return d.ArgErr() @@ -1324,6 +1400,7 @@ func (u *SRVUpstreams) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { // resolvers <resolvers...> // dial_timeout <timeout> // dial_fallback_delay <timeout> +// versions ipv4|ipv6 // } func (u *AUpstreams) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { for d.Next() { @@ -1397,8 +1474,30 @@ func (u *AUpstreams) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { } u.FallbackDelay = caddy.Duration(dur) + case "versions": + args := d.RemainingArgs() + if len(args) == 0 { + return d.Errf("must specify at least one version") + } + + if u.Versions == nil { + u.Versions = &IPVersions{} + } + + trueBool := true + for _, arg := range args { + switch arg { + case "ipv4": + u.Versions.IPv4 = &trueBool + case "ipv6": + u.Versions.IPv6 = &trueBool + default: + return d.Errf("unsupported version: '%s'", arg) + } + } + default: - return d.Errf("unrecognized srv option '%s'", d.Val()) + return d.Errf("unrecognized a option '%s'", d.Val()) } } } diff --git a/modules/caddyhttp/reverseproxy/command.go b/modules/caddyhttp/reverseproxy/command.go index 44f4c22..11f935c 100644 --- a/modules/caddyhttp/reverseproxy/command.go +++ b/modules/caddyhttp/reverseproxy/command.go @@ -16,26 +16,28 @@ package reverseproxy import ( "encoding/json" - "flag" "fmt" "net/http" "strconv" + "strings" + + "github.com/spf13/cobra" + "go.uber.org/zap" + + caddycmd "github.com/caddyserver/caddy/v2/cmd" "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile" - caddycmd "github.com/caddyserver/caddy/v2/cmd" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers" "github.com/caddyserver/caddy/v2/modules/caddytls" - "go.uber.org/zap" ) func init() { caddycmd.RegisterCommand(caddycmd.Command{ Name: "reverse-proxy", - Func: cmdReverseProxy, - Usage: "[--from <addr>] [--to <addr>] [--change-host-header]", + Usage: `[--from <addr>] [--to <addr>] [--change-host-header] [--insecure] [--internal-certs] [--disable-redirects] [--header-up "Field: value"] [--header-down "Field: value"] [--access-log] [--debug]`, Short: "A quick and production-ready reverse proxy", Long: ` A simple but production-ready reverse proxy. Useful for quick deployments, @@ -52,21 +54,33 @@ If the --from address has a host or IP, Caddy will attempt to serve the proxy over HTTPS with a certificate (unless overridden by the HTTP scheme or port). -If --change-host-header is set, the Host header on the request will be modified -from its original incoming value to the address of the upstream. (Otherwise, by -default, all incoming headers are passed through unmodified.) +If serving HTTPS: + --disable-redirects can be used to avoid binding to the HTTP port. + --internal-certs can be used to force issuance certs using the internal + CA instead of attempting to issue a public certificate. + +For proxying: + --header-up can be used to set a request header to send to the upstream. + --header-down can be used to set a response header to send back to the client. + --change-host-header sets the Host header on the request to the address + of the upstream, instead of defaulting to the incoming Host header. + This is a shortcut for --header-up "Host: {http.reverse_proxy.upstream.hostport}". + --insecure disables TLS verification with the upstream. WARNING: THIS + DISABLES SECURITY BY NOT VERIFYING THE UPSTREAM'S CERTIFICATE. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("reverse-proxy", flag.ExitOnError) - fs.String("from", "localhost", "Address on which to receive traffic") - fs.Var(&reverseProxyCmdTo, "to", "Upstream address(es) to which traffic should be sent") - fs.Bool("change-host-header", false, "Set upstream Host header to address of upstream") - fs.Bool("insecure", false, "Disable TLS verification (WARNING: DISABLES SECURITY BY NOT VERIFYING SSL CERTIFICATES!)") - fs.Bool("internal-certs", false, "Use internal CA for issuing certs") - fs.Bool("debug", false, "Enable verbose debug logs") - fs.Bool("disable-redirects", false, "Disable HTTP->HTTPS redirects") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("from", "f", "localhost", "Address on which to receive traffic") + cmd.Flags().StringSliceP("to", "t", []string{}, "Upstream address(es) to which traffic should be sent") + cmd.Flags().BoolP("change-host-header", "c", false, "Set upstream Host header to address of upstream") + cmd.Flags().BoolP("insecure", "", false, "Disable TLS verification (WARNING: DISABLES SECURITY BY NOT VERIFYING TLS CERTIFICATES!)") + cmd.Flags().BoolP("disable-redirects", "r", false, "Disable HTTP->HTTPS redirects") + cmd.Flags().BoolP("internal-certs", "i", false, "Use internal CA for issuing certs") + cmd.Flags().StringSliceP("header-up", "H", []string{}, "Set a request header to send to the upstream (format: \"Field: value\")") + cmd.Flags().StringSliceP("header-down", "d", []string{}, "Set a response header to send back to the client (format: \"Field: value\")") + cmd.Flags().BoolP("access-log", "", false, "Enable the access log") + cmd.Flags().BoolP("debug", "v", false, "Enable verbose debug logs") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdReverseProxy) + }, }) } @@ -76,14 +90,19 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { from := fs.String("from") changeHost := fs.Bool("change-host-header") insecure := fs.Bool("insecure") + disableRedir := fs.Bool("disable-redirects") internalCerts := fs.Bool("internal-certs") + accessLog := fs.Bool("access-log") debug := fs.Bool("debug") - disableRedir := fs.Bool("disable-redirects") httpPort := strconv.Itoa(caddyhttp.DefaultHTTPPort) httpsPort := strconv.Itoa(caddyhttp.DefaultHTTPSPort) - if len(reverseProxyCmdTo) == 0 { + to, err := fs.GetStringSlice("to") + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid to flag: %v", err) + } + if len(to) == 0 { return caddy.ExitCodeFailedStartup, fmt.Errorf("--to is required") } @@ -112,17 +131,17 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { // set up the upstream address; assume missing information from given parts // mixing schemes isn't supported, so use first defined (if available) - toAddresses := make([]string, len(reverseProxyCmdTo)) + toAddresses := make([]string, len(to)) var toScheme string - for i, toLoc := range reverseProxyCmdTo { - addr, scheme, err := parseUpstreamDialAddress(toLoc) + for i, toLoc := range to { + addr, err := parseUpstreamDialAddress(toLoc) if err != nil { return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid upstream address %s: %v", toLoc, err) } - if scheme != "" && toScheme == "" { - toScheme = scheme + if addr.scheme != "" && toScheme == "" { + toScheme = addr.scheme } - toAddresses[i] = addr + toAddresses[i] = addr.dialAddr() } // proceed to build the handler and server @@ -136,9 +155,24 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { upstreamPool := UpstreamPool{} for _, toAddr := range toAddresses { - upstreamPool = append(upstreamPool, &Upstream{ - Dial: toAddr, - }) + parsedAddr, err := caddy.ParseNetworkAddress(toAddr) + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid upstream address %s: %v", toAddr, err) + } + + if parsedAddr.StartPort == 0 && parsedAddr.EndPort == 0 { + // unix networks don't have ports + upstreamPool = append(upstreamPool, &Upstream{ + Dial: toAddr, + }) + } else { + // expand a port range into multiple upstreams + for i := parsedAddr.StartPort; i <= parsedAddr.EndPort; i++ { + upstreamPool = append(upstreamPool, &Upstream{ + Dial: caddy.JoinNetworkAddress("", parsedAddr.Host, fmt.Sprint(i)), + }) + } + } } handler := Handler{ @@ -146,16 +180,64 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { Upstreams: upstreamPool, } - if changeHost { + // set up header_up + headerUp, err := fs.GetStringSlice("header-up") + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid header flag: %v", err) + } + if len(headerUp) > 0 { + reqHdr := make(http.Header) + for i, h := range headerUp { + key, val, found := strings.Cut(h, ":") + key, val = strings.TrimSpace(key), strings.TrimSpace(val) + if !found || key == "" || val == "" { + return caddy.ExitCodeFailedStartup, fmt.Errorf("header-up %d: invalid format \"%s\" (expecting \"Field: value\")", i, h) + } + reqHdr.Set(key, val) + } handler.Headers = &headers.Handler{ Request: &headers.HeaderOps{ - Set: http.Header{ - "Host": []string{"{http.reverse_proxy.upstream.hostport}"}, - }, + Set: reqHdr, + }, + } + } + + // set up header_down + headerDown, err := fs.GetStringSlice("header-down") + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid header flag: %v", err) + } + if len(headerDown) > 0 { + respHdr := make(http.Header) + for i, h := range headerDown { + key, val, found := strings.Cut(h, ":") + key, val = strings.TrimSpace(key), strings.TrimSpace(val) + if !found || key == "" || val == "" { + return caddy.ExitCodeFailedStartup, fmt.Errorf("header-down %d: invalid format \"%s\" (expecting \"Field: value\")", i, h) + } + respHdr.Set(key, val) + } + if handler.Headers == nil { + handler.Headers = &headers.Handler{} + } + handler.Headers.Response = &headers.RespHeaderOps{ + HeaderOps: &headers.HeaderOps{ + Set: respHdr, }, } } + if changeHost { + if handler.Headers == nil { + handler.Headers = &headers.Handler{ + Request: &headers.HeaderOps{ + Set: http.Header{}, + }, + } + } + handler.Headers.Request.Set.Set("Host", "{http.reverse_proxy.upstream.hostport}") + } + route := caddyhttp.Route{ HandlersRaw: []json.RawMessage{ caddyconfig.JSONModuleObject(handler, "handler", "reverse_proxy", nil), @@ -173,6 +255,9 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { Routes: caddyhttp.RouteList{route}, Listen: []string{":" + fromAddr.Port}, } + if accessLog { + server.Logs = &caddyhttp.ServerLogConfig{} + } if fromAddr.Scheme == "http" { server.AutoHTTPS = &caddyhttp.AutoHTTPSConfig{Disabled: true} @@ -191,8 +276,8 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { tlsApp := caddytls.TLS{ Automation: &caddytls.AutomationConfig{ Policies: []*caddytls.AutomationPolicy{{ - Subjects: []string{fromAddr.Host}, - IssuersRaw: []json.RawMessage{json.RawMessage(`{"module":"internal"}`)}, + SubjectsRaw: []string{fromAddr.Host}, + IssuersRaw: []json.RawMessage{json.RawMessage(`{"module":"internal"}`)}, }}, }, } @@ -201,7 +286,8 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { var false bool cfg := &caddy.Config{ - Admin: &caddy.AdminConfig{Disabled: true, + Admin: &caddy.AdminConfig{ + Disabled: true, Config: &caddy.ConfigSettings{ Persist: &false, }, @@ -212,7 +298,7 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { if debug { cfg.Logging = &caddy.Logging{ Logs: map[string]*caddy.CustomLog{ - "default": {Level: zap.DebugLevel.CapitalString()}, + "default": {BaseLog: caddy.BaseLog{Level: zap.DebugLevel.CapitalString()}}, }, } } @@ -231,6 +317,3 @@ func cmdReverseProxy(fs caddycmd.Flags) (int, error) { select {} } - -// reverseProxyCmdTo holds the parsed values from repeated use of the --to flag. -var reverseProxyCmdTo caddycmd.StringSlice diff --git a/modules/caddyhttp/reverseproxy/fastcgi/caddyfile.go b/modules/caddyhttp/reverseproxy/fastcgi/caddyfile.go index 799050e..a24a3ed 100644 --- a/modules/caddyhttp/reverseproxy/fastcgi/caddyfile.go +++ b/modules/caddyhttp/reverseproxy/fastcgi/caddyfile.go @@ -217,25 +217,18 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error return nil, dispenser.ArgErr() } fcgiTransport.Root = dispenser.Val() - dispenser.Delete() - dispenser.Delete() + dispenser.DeleteN(2) case "split": extensions = dispenser.RemainingArgs() - dispenser.Delete() - for range extensions { - dispenser.Delete() - } + dispenser.DeleteN(len(extensions) + 1) if len(extensions) == 0 { return nil, dispenser.ArgErr() } case "env": args := dispenser.RemainingArgs() - dispenser.Delete() - for range args { - dispenser.Delete() - } + dispenser.DeleteN(len(args) + 1) if len(args) != 2 { return nil, dispenser.ArgErr() } @@ -246,10 +239,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error case "index": args := dispenser.RemainingArgs() - dispenser.Delete() - for range args { - dispenser.Delete() - } + dispenser.DeleteN(len(args) + 1) if len(args) != 1 { return nil, dispenser.ArgErr() } @@ -257,10 +247,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error case "try_files": args := dispenser.RemainingArgs() - dispenser.Delete() - for range args { - dispenser.Delete() - } + dispenser.DeleteN(len(args) + 1) if len(args) < 1 { return nil, dispenser.ArgErr() } @@ -268,10 +255,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error case "resolve_root_symlink": args := dispenser.RemainingArgs() - dispenser.Delete() - for range args { - dispenser.Delete() - } + dispenser.DeleteN(len(args) + 1) fcgiTransport.ResolveRootSymlink = true case "dial_timeout": @@ -283,8 +267,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error return nil, dispenser.Errf("bad timeout value %s: %v", dispenser.Val(), err) } fcgiTransport.DialTimeout = caddy.Duration(dur) - dispenser.Delete() - dispenser.Delete() + dispenser.DeleteN(2) case "read_timeout": if !dispenser.NextArg() { @@ -295,8 +278,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error return nil, dispenser.Errf("bad timeout value %s: %v", dispenser.Val(), err) } fcgiTransport.ReadTimeout = caddy.Duration(dur) - dispenser.Delete() - dispenser.Delete() + dispenser.DeleteN(2) case "write_timeout": if !dispenser.NextArg() { @@ -307,15 +289,11 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error return nil, dispenser.Errf("bad timeout value %s: %v", dispenser.Val(), err) } fcgiTransport.WriteTimeout = caddy.Duration(dur) - dispenser.Delete() - dispenser.Delete() + dispenser.DeleteN(2) case "capture_stderr": args := dispenser.RemainingArgs() - dispenser.Delete() - for range args { - dispenser.Delete() - } + dispenser.DeleteN(len(args) + 1) fcgiTransport.CaptureStderr = true } } @@ -395,6 +373,7 @@ func parsePHPFastCGI(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error // the rest of the config is specified by the user // using the reverse_proxy directive syntax + dispenser.Next() // consume the directive name err = rpHandler.UnmarshalCaddyfile(dispenser) if err != nil { return nil, err diff --git a/modules/caddyhttp/reverseproxy/fastcgi/client.go b/modules/caddyhttp/reverseproxy/fastcgi/client.go index ae36dd8..04513dd 100644 --- a/modules/caddyhttp/reverseproxy/fastcgi/client.go +++ b/modules/caddyhttp/reverseproxy/fastcgi/client.go @@ -251,7 +251,6 @@ func (c *client) Request(p map[string]string, req io.Reader) (resp *http.Respons // Get issues a GET request to the fcgi responder. func (c *client) Get(p map[string]string, body io.Reader, l int64) (resp *http.Response, err error) { - p["REQUEST_METHOD"] = "GET" p["CONTENT_LENGTH"] = strconv.FormatInt(l, 10) @@ -260,7 +259,6 @@ func (c *client) Get(p map[string]string, body io.Reader, l int64) (resp *http.R // Head issues a HEAD request to the fcgi responder. func (c *client) Head(p map[string]string) (resp *http.Response, err error) { - p["REQUEST_METHOD"] = "HEAD" p["CONTENT_LENGTH"] = "0" @@ -269,7 +267,6 @@ func (c *client) Head(p map[string]string) (resp *http.Response, err error) { // Options issues an OPTIONS request to the fcgi responder. func (c *client) Options(p map[string]string) (resp *http.Response, err error) { - p["REQUEST_METHOD"] = "OPTIONS" p["CONTENT_LENGTH"] = "0" diff --git a/modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go b/modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go index ec194e7..31febdd 100644 --- a/modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go +++ b/modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go @@ -24,13 +24,13 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy" - "github.com/caddyserver/caddy/v2/modules/caddytls" "go.uber.org/zap" "go.uber.org/zap/zapcore" "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" + "github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy" + "github.com/caddyserver/caddy/v2/modules/caddytls" ) var noopLogger = zap.NewNop() @@ -171,6 +171,7 @@ func (t Transport) RoundTrip(r *http.Request) (*http.Response, error) { rwc: conn, reqID: 1, logger: logger, + stderr: t.CaptureStderr, } // read/write timeouts @@ -254,9 +255,7 @@ func (t Transport) buildEnv(r *http.Request) (envVars, error) { // if we didn't get a split result here. // See https://github.com/caddyserver/caddy/issues/3718 if pathInfo == "" { - if remainder, ok := repl.GetString("http.matchers.file.remainder"); ok { - pathInfo = remainder - } + pathInfo, _ = repl.GetString("http.matchers.file.remainder") } // SCRIPT_FILENAME is the absolute path of SCRIPT_NAME @@ -286,10 +285,7 @@ func (t Transport) buildEnv(r *http.Request) (envVars, error) { reqHost = r.Host } - authUser := "" - if val, ok := repl.Get("http.auth.user.id"); ok { - authUser = val.(string) - } + authUser, _ := repl.GetString("http.auth.user.id") // Some variables are unused but cleared explicitly to prevent // the parent environment from interfering. diff --git a/modules/caddyhttp/reverseproxy/forwardauth/caddyfile.go b/modules/caddyhttp/reverseproxy/forwardauth/caddyfile.go index cecc000..8350096 100644 --- a/modules/caddyhttp/reverseproxy/forwardauth/caddyfile.go +++ b/modules/caddyhttp/reverseproxy/forwardauth/caddyfile.go @@ -129,8 +129,7 @@ func parseCaddyfile(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error) return nil, dispenser.ArgErr() } rpHandler.Rewrite.URI = dispenser.Val() - dispenser.Delete() - dispenser.Delete() + dispenser.DeleteN(2) case "copy_headers": args := dispenser.RemainingArgs() @@ -140,13 +139,11 @@ func parseCaddyfile(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error) args = append(args, dispenser.Val()) } - dispenser.Delete() // directive name + // directive name + args + dispenser.DeleteN(len(args) + 1) if hadBlock { - dispenser.Delete() // opening brace - dispenser.Delete() // closing brace - } - for range args { - dispenser.Delete() + // opening & closing brace + dispenser.DeleteN(2) } for _, headerField := range args { @@ -219,6 +216,7 @@ func parseCaddyfile(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error) // the rest of the config is specified by the user // using the reverse_proxy directive syntax + dispenser.Next() // consume the directive name err = rpHandler.UnmarshalCaddyfile(dispenser) if err != nil { return nil, err diff --git a/modules/caddyhttp/reverseproxy/healthchecks.go b/modules/caddyhttp/reverseproxy/healthchecks.go index c27b24f..ad21ccb 100644 --- a/modules/caddyhttp/reverseproxy/healthchecks.go +++ b/modules/caddyhttp/reverseproxy/healthchecks.go @@ -24,12 +24,12 @@ import ( "regexp" "runtime/debug" "strconv" - "strings" "time" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "go.uber.org/zap" ) // HealthChecks configures active and passive health checks. @@ -106,6 +106,76 @@ type ActiveHealthChecks struct { logger *zap.Logger } +// Provision ensures that a is set up properly before use. +func (a *ActiveHealthChecks) Provision(ctx caddy.Context, h *Handler) error { + if !a.IsEnabled() { + return nil + } + + // Canonicalize the header keys ahead of time, since + // JSON unmarshaled headers may be incorrect + cleaned := http.Header{} + for key, hdrs := range a.Headers { + for _, val := range hdrs { + cleaned.Add(key, val) + } + } + a.Headers = cleaned + + h.HealthChecks.Active.logger = h.logger.Named("health_checker.active") + + timeout := time.Duration(a.Timeout) + if timeout == 0 { + timeout = 5 * time.Second + } + + if a.Path != "" { + a.logger.Warn("the 'path' option is deprecated, please use 'uri' instead!") + } + + // parse the URI string (supports path and query) + if a.URI != "" { + parsedURI, err := url.Parse(a.URI) + if err != nil { + return err + } + a.uri = parsedURI + } + + a.httpClient = &http.Client{ + Timeout: timeout, + Transport: h.Transport, + } + + for _, upstream := range h.Upstreams { + // if there's an alternative port for health-check provided in the config, + // then use it, otherwise use the port of upstream. + if a.Port != 0 { + upstream.activeHealthCheckPort = a.Port + } + } + + if a.Interval == 0 { + a.Interval = caddy.Duration(30 * time.Second) + } + + if a.ExpectBody != "" { + var err error + a.bodyRegexp, err = regexp.Compile(a.ExpectBody) + if err != nil { + return fmt.Errorf("expect_body: compiling regular expression: %v", err) + } + } + + return nil +} + +// IsEnabled checks if the active health checks have +// the minimum config necessary to be enabled. +func (a *ActiveHealthChecks) IsEnabled() bool { + return a.Path != "" || a.URI != "" || a.Port != 0 +} + // PassiveHealthChecks holds configuration related to passive // health checks (that is, health checks which occur during // the normal flow of request proxying). @@ -203,7 +273,7 @@ func (h *Handler) doActiveHealthCheckForAllHosts() { } addr.StartPort, addr.EndPort = hcp, hcp } - if upstream.LookupSRV == "" && addr.PortRangeSize() != 1 { + if addr.PortRangeSize() != 1 { h.HealthChecks.Active.logger.Error("multiple addresses (upstream must map to only one address)", zap.String("address", networkAddr), ) @@ -237,16 +307,35 @@ func (h *Handler) doActiveHealthCheckForAllHosts() { // the host's health status fails. func (h *Handler) doActiveHealthCheck(dialInfo DialInfo, hostAddr string, upstream *Upstream) error { // create the URL for the request that acts as a health check - scheme := "http" - if ht, ok := h.Transport.(TLSTransport); ok && ht.TLSEnabled() { - // this is kind of a hacky way to know if we should use HTTPS, but whatever - scheme = "https" - } u := &url.URL{ - Scheme: scheme, + Scheme: "http", Host: hostAddr, } + // split the host and port if possible, override the port if configured + host, port, err := net.SplitHostPort(hostAddr) + if err != nil { + host = hostAddr + } + if h.HealthChecks.Active.Port != 0 { + port := strconv.Itoa(h.HealthChecks.Active.Port) + u.Host = net.JoinHostPort(host, port) + } + + // this is kind of a hacky way to know if we should use HTTPS, but whatever + if tt, ok := h.Transport.(TLSTransport); ok && tt.TLSEnabled() { + u.Scheme = "https" + + // if the port is in the except list, flip back to HTTP + if ht, ok := h.Transport.(*HTTPTransport); ok { + for _, exceptPort := range ht.TLS.ExceptPorts { + if exceptPort == port { + u.Scheme = "http" + } + } + } + } + // if we have a provisioned uri, use that, otherwise use // the deprecated Path option if h.HealthChecks.Active.uri != nil { @@ -256,16 +345,6 @@ func (h *Handler) doActiveHealthCheck(dialInfo DialInfo, hostAddr string, upstre u.Path = h.HealthChecks.Active.Path } - // adjust the port, if configured to be different - if h.HealthChecks.Active.Port != 0 { - portStr := strconv.Itoa(h.HealthChecks.Active.Port) - host, _, err := net.SplitHostPort(hostAddr) - if err != nil { - host = hostAddr - } - u.Host = net.JoinHostPort(host, portStr) - } - // attach dialing information to this request, as well as context values that // may be expected by handlers of this request ctx := h.ctx.Context @@ -279,11 +358,17 @@ func (h *Handler) doActiveHealthCheck(dialInfo DialInfo, hostAddr string, upstre } ctx = context.WithValue(ctx, caddyhttp.OriginalRequestCtxKey, *req) req = req.WithContext(ctx) - for key, hdrs := range h.HealthChecks.Active.Headers { - if strings.ToLower(key) == "host" { - req.Host = h.HealthChecks.Active.Headers.Get(key) - } else { - req.Header[key] = hdrs + + // set headers, using a replacer with only globals (env vars, system info, etc.) + repl := caddy.NewReplacer() + for key, vals := range h.HealthChecks.Active.Headers { + key = repl.ReplaceAll(key, "") + if key == "Host" { + req.Host = repl.ReplaceAll(h.HealthChecks.Active.Headers.Get(key), "") + continue + } + for _, val := range vals { + req.Header.Add(key, repl.ReplaceKnown(val, "")) } } diff --git a/modules/caddyhttp/reverseproxy/hosts.go b/modules/caddyhttp/reverseproxy/hosts.go index a973ecb..83a39d8 100644 --- a/modules/caddyhttp/reverseproxy/hosts.go +++ b/modules/caddyhttp/reverseproxy/hosts.go @@ -17,8 +17,8 @@ package reverseproxy import ( "context" "fmt" - "net" "net/http" + "net/netip" "strconv" "sync/atomic" @@ -47,15 +47,6 @@ type Upstream struct { // backends is down. Also be aware of open proxy vulnerabilities. Dial string `json:"dial,omitempty"` - // DEPRECATED: Use the SRVUpstreams module instead - // (http.reverse_proxy.upstreams.srv). This field will be - // removed in a future version of Caddy. TODO: Remove this field. - // - // If DNS SRV records are used for service discovery with this - // upstream, specify the DNS name for which to look up SRV - // records here, instead of specifying a dial address. - LookupSRV string `json:"lookup_srv,omitempty"` - // The maximum number of simultaneous requests to allow to // this upstream. If set, overrides the global passive health // check UnhealthyRequestCount value. @@ -72,12 +63,10 @@ type Upstream struct { unhealthy int32 // accessed atomically; status from active health checker } -func (u Upstream) String() string { - if u.LookupSRV != "" { - return u.LookupSRV - } - return u.Dial -} +// (pointer receiver necessary to avoid a race condition, since +// copying the Upstream reads the 'unhealthy' field which is +// accessed atomically) +func (u *Upstream) String() string { return u.Dial } // Available returns true if the remote host // is available to receive requests. This is @@ -109,35 +98,21 @@ func (u *Upstream) Full() bool { } // fillDialInfo returns a filled DialInfo for upstream u, using the request -// context. If the upstream has a SRV lookup configured, that is done and a -// returned address is chosen; otherwise, the upstream's regular dial address -// field is used. Note that the returned value is not a pointer. +// context. Note that the returned value is not a pointer. func (u *Upstream) fillDialInfo(r *http.Request) (DialInfo, error) { repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer) var addr caddy.NetworkAddress - if u.LookupSRV != "" { - // perform DNS lookup for SRV records and choose one - TODO: deprecated - srvName := repl.ReplaceAll(u.LookupSRV, "") - _, records, err := net.DefaultResolver.LookupSRV(r.Context(), "", "", srvName) - if err != nil { - return DialInfo{}, err - } - addr.Network = "tcp" - addr.Host = records[0].Target - addr.StartPort, addr.EndPort = uint(records[0].Port), uint(records[0].Port) - } else { - // use provided dial address - var err error - dial := repl.ReplaceAll(u.Dial, "") - addr, err = caddy.ParseNetworkAddress(dial) - if err != nil { - return DialInfo{}, fmt.Errorf("upstream %s: invalid dial address %s: %v", u.Dial, dial, err) - } - if numPorts := addr.PortRangeSize(); numPorts != 1 { - return DialInfo{}, fmt.Errorf("upstream %s: dial address must represent precisely one socket: %s represents %d", - u.Dial, dial, numPorts) - } + // use provided dial address + var err error + dial := repl.ReplaceAll(u.Dial, "") + addr, err = caddy.ParseNetworkAddress(dial) + if err != nil { + return DialInfo{}, fmt.Errorf("upstream %s: invalid dial address %s: %v", u.Dial, dial, err) + } + if numPorts := addr.PortRangeSize(); numPorts != 1 { + return DialInfo{}, fmt.Errorf("upstream %s: dial address must represent precisely one socket: %s represents %d", + u.Dial, dial, numPorts) } return DialInfo{ @@ -259,3 +234,13 @@ var hosts = caddy.NewUsagePool() // dialInfoVarKey is the key used for the variable that holds // the dial info for the upstream connection. const dialInfoVarKey = "reverse_proxy.dial_info" + +// proxyProtocolInfoVarKey is the key used for the variable that holds +// the proxy protocol info for the upstream connection. +const proxyProtocolInfoVarKey = "reverse_proxy.proxy_protocol_info" + +// ProxyProtocolInfo contains information needed to write proxy protocol to a +// connection to an upstream host. +type ProxyProtocolInfo struct { + AddrPort netip.AddrPort +} diff --git a/modules/caddyhttp/reverseproxy/httptransport.go b/modules/caddyhttp/reverseproxy/httptransport.go index ec5d2f2..9290f7e 100644 --- a/modules/caddyhttp/reverseproxy/httptransport.go +++ b/modules/caddyhttp/reverseproxy/httptransport.go @@ -28,10 +28,13 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddytls" + "github.com/mastercactapus/proxyprotocol" "go.uber.org/zap" "golang.org/x/net/http2" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" + "github.com/caddyserver/caddy/v2/modules/caddytls" ) func init() { @@ -64,6 +67,10 @@ type HTTPTransport struct { // Maximum number of connections per host. Default: 0 (no limit) MaxConnsPerHost int `json:"max_conns_per_host,omitempty"` + // If non-empty, which PROXY protocol version to send when + // connecting to an upstream. Default: off. + ProxyProtocol string `json:"proxy_protocol,omitempty"` + // How long to wait before timing out trying to connect to // an upstream. Default: `3s`. DialTimeout caddy.Duration `json:"dial_timeout,omitempty"` @@ -172,12 +179,19 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e } } - // Set up the dialer to pull the correct information from the context dialContext := func(ctx context.Context, network, address string) (net.Conn, error) { - // the proper dialing information should be embedded into the request's context + // For unix socket upstreams, we need to recover the dial info from + // the request's context, because the Host on the request's URL + // will have been modified by directing the request, overwriting + // the unix socket filename. + // Also, we need to avoid overwriting the address at this point + // when not necessary, because http.ProxyFromEnvironment may have + // modified the address according to the user's env proxy config. if dialInfo, ok := GetDialInfo(ctx); ok { - network = dialInfo.Network - address = dialInfo.Address + if strings.HasPrefix(dialInfo.Network, "unix") { + network = dialInfo.Network + address = dialInfo.Address + } } conn, err := dialer.DialContext(ctx, network, address) @@ -188,8 +202,59 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e return nil, DialError{err} } - // if read/write timeouts are configured and this is a TCP connection, enforce the timeouts - // by wrapping the connection with our own type + if h.ProxyProtocol != "" { + proxyProtocolInfo, ok := caddyhttp.GetVar(ctx, proxyProtocolInfoVarKey).(ProxyProtocolInfo) + if !ok { + return nil, fmt.Errorf("failed to get proxy protocol info from context") + } + + // The src and dst have to be of the some address family. As we don't know the original + // dst address (it's kind of impossible to know) and this address is generelly of very + // little interest, we just set it to all zeros. + var destIP net.IP + switch { + case proxyProtocolInfo.AddrPort.Addr().Is4(): + destIP = net.IPv4zero + case proxyProtocolInfo.AddrPort.Addr().Is6(): + destIP = net.IPv6zero + default: + return nil, fmt.Errorf("unexpected remote addr type in proxy protocol info") + } + + // TODO: We should probably migrate away from net.IP to use netip.Addr, + // but due to the upstream dependency, we can't do that yet. + switch h.ProxyProtocol { + case "v1": + header := proxyprotocol.HeaderV1{ + SrcIP: net.IP(proxyProtocolInfo.AddrPort.Addr().AsSlice()), + SrcPort: int(proxyProtocolInfo.AddrPort.Port()), + DestIP: destIP, + DestPort: 0, + } + caddyCtx.Logger().Debug("sending proxy protocol header v1", zap.Any("header", header)) + _, err = header.WriteTo(conn) + case "v2": + header := proxyprotocol.HeaderV2{ + Command: proxyprotocol.CmdProxy, + Src: &net.TCPAddr{IP: net.IP(proxyProtocolInfo.AddrPort.Addr().AsSlice()), Port: int(proxyProtocolInfo.AddrPort.Port())}, + Dest: &net.TCPAddr{IP: destIP, Port: 0}, + } + caddyCtx.Logger().Debug("sending proxy protocol header v2", zap.Any("header", header)) + _, err = header.WriteTo(conn) + default: + return nil, fmt.Errorf("unexpected proxy protocol version") + } + + if err != nil { + // identify this error as one that occurred during + // dialing, which can be important when trying to + // decide whether to retry a request + return nil, DialError{err} + } + } + + // if read/write timeouts are configured and this is a TCP connection, + // enforce the timeouts by wrapping the connection with our own type if tcpConn, ok := conn.(*net.TCPConn); ok && (h.ReadTimeout > 0 || h.WriteTimeout > 0) { conn = &tcpRWTimeoutConn{ TCPConn: tcpConn, @@ -203,6 +268,7 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e } rt := &http.Transport{ + Proxy: http.ProxyFromEnvironment, DialContext: dialContext, MaxConnsPerHost: h.MaxConnsPerHost, ResponseHeaderTimeout: time.Duration(h.ResponseHeaderTimeout), @@ -231,6 +297,14 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e rt.IdleConnTimeout = time.Duration(h.KeepAlive.IdleConnTimeout) } + // The proxy protocol header can only be sent once right after opening the connection. + // So single connection must not be used for multiple requests, which can potentially + // come from different clients. + if !rt.DisableKeepAlives && h.ProxyProtocol != "" { + caddyCtx.Logger().Warn("disabling keepalives, they are incompatible with using PROXY protocol") + rt.DisableKeepAlives = true + } + if h.Compression != nil { rt.DisableCompression = !*h.Compression } @@ -452,10 +526,11 @@ func (t TLSConfig) MakeTLSClientConfig(ctx caddy.Context) (*tls.Config, error) { return nil, fmt.Errorf("managing client certificate: %v", err) } cfg.GetClientCertificate = func(cri *tls.CertificateRequestInfo) (*tls.Certificate, error) { - certs := tlsApp.AllMatchingCertificates(t.ClientCertificateAutomate) + certs := caddytls.AllMatchingCertificates(t.ClientCertificateAutomate) var err error for _, cert := range certs { - err = cri.SupportsCertificate(&cert.Certificate) + certCertificate := cert.Certificate // avoid taking address of iteration variable (gosec warning) + err = cri.SupportsCertificate(&certCertificate) if err == nil { return &cert.Certificate, nil } diff --git a/modules/caddyhttp/reverseproxy/metrics.go b/modules/caddyhttp/reverseproxy/metrics.go index 4272bc4..d3c8ee0 100644 --- a/modules/caddyhttp/reverseproxy/metrics.go +++ b/modules/caddyhttp/reverseproxy/metrics.go @@ -39,6 +39,8 @@ func newMetricsUpstreamsHealthyUpdater(handler *Handler) *metricsUpstreamsHealth initReverseProxyMetrics(handler) }) + reverseProxyMetrics.upstreamsHealthy.Reset() + return &metricsUpstreamsHealthyUpdater{handler} } diff --git a/modules/caddyhttp/reverseproxy/reverseproxy.go b/modules/caddyhttp/reverseproxy/reverseproxy.go index 1449785..08be40d 100644 --- a/modules/caddyhttp/reverseproxy/reverseproxy.go +++ b/modules/caddyhttp/reverseproxy/reverseproxy.go @@ -27,30 +27,23 @@ import ( "net/netip" "net/textproto" "net/url" - "regexp" - "runtime" "strconv" "strings" "sync" "time" + "go.uber.org/zap" + "golang.org/x/net/http/httpguts" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyevents" "github.com/caddyserver/caddy/v2/modules/caddyhttp" "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers" "github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite" - "go.uber.org/zap" - "golang.org/x/net/http/httpguts" ) -var supports1xx bool - func init() { - // Caddy requires at least Go 1.18, but Early Hints requires Go 1.19; thus we can simply check for 1.18 in version string - // TODO: remove this once our minimum Go version is 1.19 - supports1xx = !strings.Contains(runtime.Version(), "go1.18") - caddy.RegisterModule(Handler{}) } @@ -158,6 +151,19 @@ type Handler struct { // could be useful if the backend has tighter memory constraints. ResponseBuffers int64 `json:"response_buffers,omitempty"` + // If nonzero, streaming requests such as WebSockets will be + // forcibly closed at the end of the timeout. Default: no timeout. + StreamTimeout caddy.Duration `json:"stream_timeout,omitempty"` + + // If nonzero, streaming requests such as WebSockets will not be + // closed when the proxy config is unloaded, and instead the stream + // will remain open until the delay is complete. In other words, + // enabling this prevents streams from closing when Caddy's config + // is reloaded. Enabling this may be a good idea to avoid a thundering + // herd of reconnecting clients which had their connections closed + // by the previous config closing. Default: no delay. + StreamCloseDelay caddy.Duration `json:"stream_close_delay,omitempty"` + // If configured, rewrites the copy of the upstream request. // Allows changing the request method and URI (path and query). // Since the rewrite is applied to the copy, it does not persist @@ -185,6 +191,13 @@ type Handler struct { // - `{http.reverse_proxy.header.*}` The headers from the response HandleResponse []caddyhttp.ResponseHandler `json:"handle_response,omitempty"` + // If set, the proxy will write very detailed logs about its + // inner workings. Enable this only when debugging, as it + // will produce a lot of output. + // + // EXPERIMENTAL: This feature is subject to change or removal. + VerboseLogs bool `json:"verbose_logs,omitempty"` + Transport http.RoundTripper `json:"-"` CB CircuitBreaker `json:"-"` DynamicUpstreams UpstreamSource `json:"-"` @@ -199,8 +212,9 @@ type Handler struct { handleResponseSegments []*caddyfile.Dispenser // Stores upgraded requests (hijacked connections) for proper cleanup - connections map[io.ReadWriteCloser]openConnection - connectionsMu *sync.Mutex + connections map[io.ReadWriteCloser]openConnection + connectionsCloseTimer *time.Timer + connectionsMu *sync.Mutex ctx caddy.Context logger *zap.Logger @@ -243,20 +257,6 @@ func (h *Handler) Provision(ctx caddy.Context) error { h.logger.Warn("UNLIMITED BUFFERING: buffering is enabled without any cap on buffer size, which can result in OOM crashes") } - // verify SRV compatibility - TODO: LookupSRV deprecated; will be removed - for i, v := range h.Upstreams { - if v.LookupSRV == "" { - continue - } - h.logger.Warn("DEPRECATED: lookup_srv: will be removed in a near-future version of Caddy; use the http.reverse_proxy.upstreams.srv module instead") - if h.HealthChecks != nil && h.HealthChecks.Active != nil { - return fmt.Errorf(`upstream: lookup_srv is incompatible with active health checks: %d: {"dial": %q, "lookup_srv": %q}`, i, v.Dial, v.LookupSRV) - } - if v.Dial != "" { - return fmt.Errorf(`upstream: specifying dial address is incompatible with lookup_srv: %d: {"dial": %q, "lookup_srv": %q}`, i, v.Dial, v.LookupSRV) - } - } - // start by loading modules if h.TransportRaw != nil { mod, err := ctx.LoadModule(h, "TransportRaw") @@ -363,62 +363,22 @@ func (h *Handler) Provision(ctx caddy.Context) error { if h.HealthChecks != nil { // set defaults on passive health checks, if necessary if h.HealthChecks.Passive != nil { - if h.HealthChecks.Passive.FailDuration > 0 && h.HealthChecks.Passive.MaxFails == 0 { + h.HealthChecks.Passive.logger = h.logger.Named("health_checker.passive") + if h.HealthChecks.Passive.MaxFails == 0 { h.HealthChecks.Passive.MaxFails = 1 } } // if active health checks are enabled, configure them and start a worker - if h.HealthChecks.Active != nil && (h.HealthChecks.Active.Path != "" || - h.HealthChecks.Active.URI != "" || - h.HealthChecks.Active.Port != 0) { - - h.HealthChecks.Active.logger = h.logger.Named("health_checker.active") - - timeout := time.Duration(h.HealthChecks.Active.Timeout) - if timeout == 0 { - timeout = 5 * time.Second - } - - if h.HealthChecks.Active.Path != "" { - h.HealthChecks.Active.logger.Warn("the 'path' option is deprecated, please use 'uri' instead!") - } - - // parse the URI string (supports path and query) - if h.HealthChecks.Active.URI != "" { - parsedURI, err := url.Parse(h.HealthChecks.Active.URI) - if err != nil { - return err - } - h.HealthChecks.Active.uri = parsedURI - } - - h.HealthChecks.Active.httpClient = &http.Client{ - Timeout: timeout, - Transport: h.Transport, - } - - for _, upstream := range h.Upstreams { - // if there's an alternative port for health-check provided in the config, - // then use it, otherwise use the port of upstream. - if h.HealthChecks.Active.Port != 0 { - upstream.activeHealthCheckPort = h.HealthChecks.Active.Port - } + if h.HealthChecks.Active != nil { + err := h.HealthChecks.Active.Provision(ctx, h) + if err != nil { + return err } - if h.HealthChecks.Active.Interval == 0 { - h.HealthChecks.Active.Interval = caddy.Duration(30 * time.Second) + if h.HealthChecks.Active.IsEnabled() { + go h.activeHealthChecker() } - - if h.HealthChecks.Active.ExpectBody != "" { - var err error - h.HealthChecks.Active.bodyRegexp, err = regexp.Compile(h.HealthChecks.Active.ExpectBody) - if err != nil { - return fmt.Errorf("expect_body: compiling regular expression: %v", err) - } - } - - go h.activeHealthChecker() } } @@ -438,25 +398,7 @@ func (h *Handler) Provision(ctx caddy.Context) error { // Cleanup cleans up the resources made by h. func (h *Handler) Cleanup() error { - // close hijacked connections (both to client and backend) - var err error - h.connectionsMu.Lock() - for _, oc := range h.connections { - if oc.gracefulClose != nil { - // this is potentially blocking while we have the lock on the connections - // map, but that should be OK since the server has in theory shut down - // and we are no longer using the connections map - gracefulErr := oc.gracefulClose() - if gracefulErr != nil && err == nil { - err = gracefulErr - } - } - closeErr := oc.conn.Close() - if closeErr != nil && err == nil { - err = closeErr - } - } - h.connectionsMu.Unlock() + err := h.cleanupConnections() // remove hosts from our config from the pool for _, upstream := range h.Upstreams { @@ -517,7 +459,8 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyht // It returns true when the loop is done and should break; false otherwise. The error value returned should // be assigned to the proxyErr value for the next iteration of the loop (or the error handled after break). func (h *Handler) proxyLoopIteration(r *http.Request, origReq *http.Request, w http.ResponseWriter, proxyErr error, start time.Time, retries int, - repl *caddy.Replacer, reqHeader http.Header, reqHost string, next caddyhttp.Handler) (bool, error) { + repl *caddy.Replacer, reqHeader http.Header, reqHost string, next caddyhttp.Handler, +) (bool, error) { // get the updated list of upstreams upstreams := h.Upstreams if h.DynamicUpstreams != nil { @@ -544,7 +487,7 @@ func (h *Handler) proxyLoopIteration(r *http.Request, origReq *http.Request, w h upstream := h.LoadBalancing.SelectionPolicy.Select(upstreams, r, w) if upstream == nil { if proxyErr == nil { - proxyErr = caddyhttp.Error(http.StatusServiceUnavailable, fmt.Errorf("no upstreams available")) + proxyErr = caddyhttp.Error(http.StatusServiceUnavailable, noUpstreamsAvailable) } if !h.LoadBalancing.tryAgain(h.ctx, start, retries, proxyErr, r) { return true, proxyErr @@ -646,7 +589,8 @@ func (h Handler) prepareRequest(req *http.Request, repl *caddy.Replacer) (*http. // feature if absolutely required, if read timeouts are // set, and if body size is limited if h.RequestBuffers != 0 && req.Body != nil { - req.Body, _ = h.bufferedBody(req.Body, h.RequestBuffers) + req.Body, req.ContentLength = h.bufferedBody(req.Body, h.RequestBuffers) + req.Header.Set("Content-Length", strconv.FormatInt(req.ContentLength, 10)) } if req.ContentLength == 0 { @@ -687,8 +631,24 @@ func (h Handler) prepareRequest(req *http.Request, repl *caddy.Replacer) (*http. req.Header.Set("Upgrade", reqUpType) } + // Set up the PROXY protocol info + address := caddyhttp.GetVar(req.Context(), caddyhttp.ClientIPVarKey).(string) + addrPort, err := netip.ParseAddrPort(address) + if err != nil { + // OK; probably didn't have a port + addr, err := netip.ParseAddr(address) + if err != nil { + // Doesn't seem like a valid ip address at all + } else { + // Ok, only the port was missing + addrPort = netip.AddrPortFrom(addr, 0) + } + } + proxyProtocolInfo := ProxyProtocolInfo{AddrPort: addrPort} + caddyhttp.SetVar(req.Context(), proxyProtocolInfoVarKey, proxyProtocolInfo) + // Add the supported X-Forwarded-* headers - err := h.addForwardedHeaders(req) + err = h.addForwardedHeaders(req) if err != nil { return nil, err } @@ -795,25 +755,23 @@ func (h *Handler) reverseProxy(rw http.ResponseWriter, req *http.Request, origRe server := req.Context().Value(caddyhttp.ServerCtxKey).(*caddyhttp.Server) shouldLogCredentials := server.Logs != nil && server.Logs.ShouldLogCredentials - if supports1xx { - // Forward 1xx status codes, backported from https://github.com/golang/go/pull/53164 - trace := &httptrace.ClientTrace{ - Got1xxResponse: func(code int, header textproto.MIMEHeader) error { - h := rw.Header() - copyHeader(h, http.Header(header)) - rw.WriteHeader(code) - - // Clear headers coming from the backend - // (it's not automatically done by ResponseWriter.WriteHeader() for 1xx responses) - for k := range header { - delete(h, k) - } + // Forward 1xx status codes, backported from https://github.com/golang/go/pull/53164 + trace := &httptrace.ClientTrace{ + Got1xxResponse: func(code int, header textproto.MIMEHeader) error { + h := rw.Header() + copyHeader(h, http.Header(header)) + rw.WriteHeader(code) + + // Clear headers coming from the backend + // (it's not automatically done by ResponseWriter.WriteHeader() for 1xx responses) + for k := range header { + delete(h, k) + } - return nil - }, - } - req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace)) + return nil + }, } + req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace)) // if FlushInterval is explicitly configured to -1 (i.e. flush continuously to achieve // low-latency streaming), don't let the transport cancel the request if the client @@ -821,7 +779,7 @@ func (h *Handler) reverseProxy(rw http.ResponseWriter, req *http.Request, origRe // regardless, and we should expect client disconnection in low-latency streaming // scenarios (see issue #4922) if h.FlushInterval == -1 { - req = req.WithContext(ignoreClientGoneContext{req.Context(), h.ctx.Done()}) + req = req.WithContext(ignoreClientGoneContext{req.Context()}) } // do the round-trip; emit debug log with values we know are @@ -897,12 +855,6 @@ func (h *Handler) reverseProxy(rw http.ResponseWriter, req *http.Request, origRe break } - // otherwise, if there are any routes configured, execute those as the - // actual response instead of what we got from the proxy backend - if len(rh.Routes) == 0 { - continue - } - // set up the replacer so that parts of the original response can be // used for routing decisions for field, value := range res.Header { @@ -911,7 +863,7 @@ func (h *Handler) reverseProxy(rw http.ResponseWriter, req *http.Request, origRe repl.Set("http.reverse_proxy.status_code", res.StatusCode) repl.Set("http.reverse_proxy.status_text", res.Status) - h.logger.Debug("handling response", zap.Int("handler", i)) + logger.Debug("handling response", zap.Int("handler", i)) // we make some data available via request context to child routes // so that they may inherit some options and functions from the @@ -956,7 +908,7 @@ func (h *Handler) reverseProxy(rw http.ResponseWriter, req *http.Request, origRe } // finalizeResponse prepares and copies the response. -func (h Handler) finalizeResponse( +func (h *Handler) finalizeResponse( rw http.ResponseWriter, req *http.Request, res *http.Response, @@ -998,15 +950,21 @@ func (h Handler) finalizeResponse( } rw.WriteHeader(res.StatusCode) + if h.VerboseLogs { + logger.Debug("wrote header") + } - err := h.copyResponse(rw, res.Body, h.flushInterval(req, res)) - res.Body.Close() // close now, instead of defer, to populate res.Trailer + err := h.copyResponse(rw, res.Body, h.flushInterval(req, res), logger) + errClose := res.Body.Close() // close now, instead of defer, to populate res.Trailer + if h.VerboseLogs || errClose != nil { + logger.Debug("closed response body from upstream", zap.Error(errClose)) + } if err != nil { // we're streaming the response and we've already written headers, so // there's nothing an error handler can do to recover at this point; // the standard lib's proxy panics at this point, but we'll just log // the error and abort the stream here - h.logger.Error("aborting with incomplete response", zap.Error(err)) + logger.Error("aborting with incomplete response", zap.Error(err)) return nil } @@ -1014,9 +972,8 @@ func (h Handler) finalizeResponse( // Force chunking if we saw a response trailer. // This prevents net/http from calculating the length for short // bodies and adding a Content-Length. - if fl, ok := rw.(http.Flusher); ok { - fl.Flush() - } + //nolint:bodyclose + http.NewResponseController(rw).Flush() } // total duration spent proxying, including writing response body @@ -1035,6 +992,10 @@ func (h Handler) finalizeResponse( } } + if h.VerboseLogs { + logger.Debug("response finalized") + } + return nil } @@ -1066,17 +1027,23 @@ func (lb LoadBalancing) tryAgain(ctx caddy.Context, start time.Time, retries int // should be safe to retry, since without a connection, no // HTTP request can be transmitted; but if the error is not // specifically a dialer error, we need to be careful - if _, ok := proxyErr.(DialError); proxyErr != nil && !ok { + if proxyErr != nil { + _, isDialError := proxyErr.(DialError) + herr, isHandlerError := proxyErr.(caddyhttp.HandlerError) + // if the error occurred after a connection was established, // we have to assume the upstream received the request, and // retries need to be carefully decided, because some requests // are not idempotent - if lb.RetryMatch == nil && req.Method != "GET" { - // by default, don't retry requests if they aren't GET - return false - } - if !lb.RetryMatch.AnyMatch(req) { - return false + if !isDialError && !(isHandlerError && errors.Is(herr, noUpstreamsAvailable)) { + if lb.RetryMatch == nil && req.Method != "GET" { + // by default, don't retry requests if they aren't GET + return false + } + + if !lb.RetryMatch.AnyMatch(req) { + return false + } } } @@ -1128,12 +1095,11 @@ func (h Handler) provisionUpstream(upstream *Upstream) { // without MaxRequests), copy the value into this upstream, since the // value in the upstream (MaxRequests) is what is used during // availability checks - if h.HealthChecks != nil && h.HealthChecks.Passive != nil { - h.HealthChecks.Passive.logger = h.logger.Named("health_checker.passive") - if h.HealthChecks.Passive.UnhealthyRequestCount > 0 && - upstream.MaxRequests == 0 { - upstream.MaxRequests = h.HealthChecks.Passive.UnhealthyRequestCount - } + if h.HealthChecks != nil && + h.HealthChecks.Passive != nil && + h.HealthChecks.Passive.UnhealthyRequestCount > 0 && + upstream.MaxRequests == 0 { + upstream.MaxRequests = h.HealthChecks.Passive.UnhealthyRequestCount } // upstreams need independent access to the passive @@ -1450,21 +1416,36 @@ type handleResponseContext struct { // ignoreClientGoneContext is a special context.Context type // intended for use when doing a RoundTrip where you don't // want a client disconnection to cancel the request during -// the roundtrip. Set its done field to a Done() channel -// of a context that doesn't get canceled when the client -// disconnects, such as caddy.Context.Done() instead. +// the roundtrip. +// This context clears cancellation, error, and deadline methods, +// but still allows values to pass through from its embedded +// context. +// +// TODO: This can be replaced with context.WithoutCancel once +// the minimum required version of Go is 1.21. type ignoreClientGoneContext struct { context.Context - done <-chan struct{} } -func (c ignoreClientGoneContext) Done() <-chan struct{} { return c.done } +func (c ignoreClientGoneContext) Deadline() (deadline time.Time, ok bool) { + return +} + +func (c ignoreClientGoneContext) Done() <-chan struct{} { + return nil +} + +func (c ignoreClientGoneContext) Err() error { + return nil +} // proxyHandleResponseContextCtxKey is the context key for the active proxy handler // so that handle_response routes can inherit some config options // from the proxy handler. const proxyHandleResponseContextCtxKey caddy.CtxKey = "reverse_proxy_handle_response_context" +var noUpstreamsAvailable = fmt.Errorf("no upstreams available") + // Interface guards var ( _ caddy.Provisioner = (*Handler)(nil) diff --git a/modules/caddyhttp/reverseproxy/selectionpolicies.go b/modules/caddyhttp/reverseproxy/selectionpolicies.go index 0b7f50c..acb069a 100644 --- a/modules/caddyhttp/reverseproxy/selectionpolicies.go +++ b/modules/caddyhttp/reverseproxy/selectionpolicies.go @@ -18,17 +18,20 @@ import ( "crypto/hmac" "crypto/sha256" "encoding/hex" + "encoding/json" "fmt" "hash/fnv" weakrand "math/rand" "net" "net/http" "strconv" + "strings" "sync/atomic" - "time" "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) func init() { @@ -36,13 +39,14 @@ func init() { caddy.RegisterModule(RandomChoiceSelection{}) caddy.RegisterModule(LeastConnSelection{}) caddy.RegisterModule(RoundRobinSelection{}) + caddy.RegisterModule(WeightedRoundRobinSelection{}) caddy.RegisterModule(FirstSelection{}) caddy.RegisterModule(IPHashSelection{}) + caddy.RegisterModule(ClientIPHashSelection{}) caddy.RegisterModule(URIHashSelection{}) + caddy.RegisterModule(QueryHashSelection{}) caddy.RegisterModule(HeaderHashSelection{}) caddy.RegisterModule(CookieHashSelection{}) - - weakrand.Seed(time.Now().UTC().UnixNano()) } // RandomSelection is a policy that selects @@ -72,6 +76,90 @@ func (r *RandomSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { return nil } +// WeightedRoundRobinSelection is a policy that selects +// a host based on weighted round-robin ordering. +type WeightedRoundRobinSelection struct { + // The weight of each upstream in order, + // corresponding with the list of upstreams configured. + Weights []int `json:"weights,omitempty"` + index uint32 + totalWeight int +} + +// CaddyModule returns the Caddy module information. +func (WeightedRoundRobinSelection) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.reverse_proxy.selection_policies.weighted_round_robin", + New: func() caddy.Module { + return new(WeightedRoundRobinSelection) + }, + } +} + +// UnmarshalCaddyfile sets up the module from Caddyfile tokens. +func (r *WeightedRoundRobinSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + args := d.RemainingArgs() + if len(args) == 0 { + return d.ArgErr() + } + + for _, weight := range args { + weightInt, err := strconv.Atoi(weight) + if err != nil { + return d.Errf("invalid weight value '%s': %v", weight, err) + } + if weightInt < 1 { + return d.Errf("invalid weight value '%s': weight should be non-zero and positive", weight) + } + r.Weights = append(r.Weights, weightInt) + } + } + return nil +} + +// Provision sets up r. +func (r *WeightedRoundRobinSelection) Provision(ctx caddy.Context) error { + for _, weight := range r.Weights { + r.totalWeight += weight + } + return nil +} + +// Select returns an available host, if any. +func (r *WeightedRoundRobinSelection) Select(pool UpstreamPool, _ *http.Request, _ http.ResponseWriter) *Upstream { + if len(pool) == 0 { + return nil + } + if len(r.Weights) < 2 { + return pool[0] + } + var index, totalWeight int + currentWeight := int(atomic.AddUint32(&r.index, 1)) % r.totalWeight + for i, weight := range r.Weights { + totalWeight += weight + if currentWeight < totalWeight { + index = i + break + } + } + + upstreams := make([]*Upstream, 0, len(r.Weights)) + for _, upstream := range pool { + if !upstream.Available() { + continue + } + upstreams = append(upstreams, upstream) + if len(upstreams) == cap(upstreams) { + break + } + } + if len(upstreams) == 0 { + return nil + } + return upstreams[index%len(upstreams)] +} + // RandomChoiceSelection is a policy that selects // two or more available hosts at random, then // chooses the one with the least load. @@ -181,7 +269,7 @@ func (LeastConnSelection) Select(pool UpstreamPool, _ *http.Request, _ http.Resp // sample: https://en.wikipedia.org/wiki/Reservoir_sampling if numReqs == leastReqs { count++ - if (weakrand.Int() % count) == 0 { //nolint:gosec + if count == 1 || (weakrand.Int()%count) == 0 { //nolint:gosec bestHost = host } } @@ -303,6 +391,39 @@ func (r *IPHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { return nil } +// ClientIPHashSelection is a policy that selects a host +// based on hashing the client IP of the request, as determined +// by the HTTP app's trusted proxies settings. +type ClientIPHashSelection struct{} + +// CaddyModule returns the Caddy module information. +func (ClientIPHashSelection) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.reverse_proxy.selection_policies.client_ip_hash", + New: func() caddy.Module { return new(ClientIPHashSelection) }, + } +} + +// Select returns an available host, if any. +func (ClientIPHashSelection) Select(pool UpstreamPool, req *http.Request, _ http.ResponseWriter) *Upstream { + address := caddyhttp.GetVar(req.Context(), caddyhttp.ClientIPVarKey).(string) + clientIP, _, err := net.SplitHostPort(address) + if err != nil { + clientIP = address // no port + } + return hostByHashing(pool, clientIP) +} + +// UnmarshalCaddyfile sets up the module from Caddyfile tokens. +func (r *ClientIPHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + if d.NextArg() { + return d.ArgErr() + } + } + return nil +} + // URIHashSelection is a policy that selects a // host by hashing the request URI. type URIHashSelection struct{} @@ -330,11 +451,95 @@ func (r *URIHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { return nil } +// QueryHashSelection is a policy that selects +// a host based on a given request query parameter. +type QueryHashSelection struct { + // The query key whose value is to be hashed and used for upstream selection. + Key string `json:"key,omitempty"` + + // The fallback policy to use if the query key is not present. Defaults to `random`. + FallbackRaw json.RawMessage `json:"fallback,omitempty" caddy:"namespace=http.reverse_proxy.selection_policies inline_key=policy"` + fallback Selector +} + +// CaddyModule returns the Caddy module information. +func (QueryHashSelection) CaddyModule() caddy.ModuleInfo { + return caddy.ModuleInfo{ + ID: "http.reverse_proxy.selection_policies.query", + New: func() caddy.Module { return new(QueryHashSelection) }, + } +} + +// Provision sets up the module. +func (s *QueryHashSelection) Provision(ctx caddy.Context) error { + if s.Key == "" { + return fmt.Errorf("query key is required") + } + if s.FallbackRaw == nil { + s.FallbackRaw = caddyconfig.JSONModuleObject(RandomSelection{}, "policy", "random", nil) + } + mod, err := ctx.LoadModule(s, "FallbackRaw") + if err != nil { + return fmt.Errorf("loading fallback selection policy: %s", err) + } + s.fallback = mod.(Selector) + return nil +} + +// Select returns an available host, if any. +func (s QueryHashSelection) Select(pool UpstreamPool, req *http.Request, _ http.ResponseWriter) *Upstream { + // Since the query may have multiple values for the same key, + // we'll join them to avoid a problem where the user can control + // the upstream that the request goes to by sending multiple values + // for the same key, when the upstream only considers the first value. + // Keep in mind that a client changing the order of the values may + // affect which upstream is selected, but this is a semantically + // different request, because the order of the values is significant. + vals := strings.Join(req.URL.Query()[s.Key], ",") + if vals == "" { + return s.fallback.Select(pool, req, nil) + } + return hostByHashing(pool, vals) +} + +// UnmarshalCaddyfile sets up the module from Caddyfile tokens. +func (s *QueryHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { + for d.Next() { + if !d.NextArg() { + return d.ArgErr() + } + s.Key = d.Val() + } + for nesting := d.Nesting(); d.NextBlock(nesting); { + switch d.Val() { + case "fallback": + if !d.NextArg() { + return d.ArgErr() + } + if s.FallbackRaw != nil { + return d.Err("fallback selection policy already specified") + } + mod, err := loadFallbackPolicy(d) + if err != nil { + return err + } + s.FallbackRaw = mod + default: + return d.Errf("unrecognized option '%s'", d.Val()) + } + } + return nil +} + // HeaderHashSelection is a policy that selects // a host based on a given request header. type HeaderHashSelection struct { // The HTTP header field whose value is to be hashed and used for upstream selection. Field string `json:"field,omitempty"` + + // The fallback policy to use if the header is not present. Defaults to `random`. + FallbackRaw json.RawMessage `json:"fallback,omitempty" caddy:"namespace=http.reverse_proxy.selection_policies inline_key=policy"` + fallback Selector } // CaddyModule returns the Caddy module information. @@ -345,12 +550,24 @@ func (HeaderHashSelection) CaddyModule() caddy.ModuleInfo { } } -// Select returns an available host, if any. -func (s HeaderHashSelection) Select(pool UpstreamPool, req *http.Request, _ http.ResponseWriter) *Upstream { +// Provision sets up the module. +func (s *HeaderHashSelection) Provision(ctx caddy.Context) error { if s.Field == "" { - return nil + return fmt.Errorf("header field is required") + } + if s.FallbackRaw == nil { + s.FallbackRaw = caddyconfig.JSONModuleObject(RandomSelection{}, "policy", "random", nil) } + mod, err := ctx.LoadModule(s, "FallbackRaw") + if err != nil { + return fmt.Errorf("loading fallback selection policy: %s", err) + } + s.fallback = mod.(Selector) + return nil +} +// Select returns an available host, if any. +func (s HeaderHashSelection) Select(pool UpstreamPool, req *http.Request, _ http.ResponseWriter) *Upstream { // The Host header should be obtained from the req.Host field // since net/http removes it from the header map. if s.Field == "Host" && req.Host != "" { @@ -359,7 +576,7 @@ func (s HeaderHashSelection) Select(pool UpstreamPool, req *http.Request, _ http val := req.Header.Get(s.Field) if val == "" { - return RandomSelection{}.Select(pool, req, nil) + return s.fallback.Select(pool, req, nil) } return hostByHashing(pool, val) } @@ -372,6 +589,24 @@ func (s *HeaderHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { } s.Field = d.Val() } + for nesting := d.Nesting(); d.NextBlock(nesting); { + switch d.Val() { + case "fallback": + if !d.NextArg() { + return d.ArgErr() + } + if s.FallbackRaw != nil { + return d.Err("fallback selection policy already specified") + } + mod, err := loadFallbackPolicy(d) + if err != nil { + return err + } + s.FallbackRaw = mod + default: + return d.Errf("unrecognized option '%s'", d.Val()) + } + } return nil } @@ -382,6 +617,10 @@ type CookieHashSelection struct { Name string `json:"name,omitempty"` // Secret to hash (Hmac256) chosen upstream in cookie Secret string `json:"secret,omitempty"` + + // The fallback policy to use if the cookie is not present. Defaults to `random`. + FallbackRaw json.RawMessage `json:"fallback,omitempty" caddy:"namespace=http.reverse_proxy.selection_policies inline_key=policy"` + fallback Selector } // CaddyModule returns the Caddy module information. @@ -392,15 +631,48 @@ func (CookieHashSelection) CaddyModule() caddy.ModuleInfo { } } -// Select returns an available host, if any. -func (s CookieHashSelection) Select(pool UpstreamPool, req *http.Request, w http.ResponseWriter) *Upstream { +// Provision sets up the module. +func (s *CookieHashSelection) Provision(ctx caddy.Context) error { if s.Name == "" { s.Name = "lb" } + if s.FallbackRaw == nil { + s.FallbackRaw = caddyconfig.JSONModuleObject(RandomSelection{}, "policy", "random", nil) + } + mod, err := ctx.LoadModule(s, "FallbackRaw") + if err != nil { + return fmt.Errorf("loading fallback selection policy: %s", err) + } + s.fallback = mod.(Selector) + return nil +} + +// Select returns an available host, if any. +func (s CookieHashSelection) Select(pool UpstreamPool, req *http.Request, w http.ResponseWriter) *Upstream { + // selects a new Host using the fallback policy (typically random) + // and write a sticky session cookie to the response. + selectNewHost := func() *Upstream { + upstream := s.fallback.Select(pool, req, w) + if upstream == nil { + return nil + } + sha, err := hashCookie(s.Secret, upstream.Dial) + if err != nil { + return upstream + } + http.SetCookie(w, &http.Cookie{ + Name: s.Name, + Value: sha, + Path: "/", + Secure: false, + }) + return upstream + } + cookie, err := req.Cookie(s.Name) - // If there's no cookie, select new random host + // If there's no cookie, select a host using the fallback policy if err != nil || cookie == nil { - return selectNewHostWithCookieHashSelection(pool, w, s.Secret, s.Name) + return selectNewHost() } // If the cookie is present, loop over the available upstreams until we find a match cookieValue := cookie.Value @@ -413,13 +685,15 @@ func (s CookieHashSelection) Select(pool UpstreamPool, req *http.Request, w http return upstream } } - // If there is no matching host, select new random host - return selectNewHostWithCookieHashSelection(pool, w, s.Secret, s.Name) + // If there is no matching host, select a host using the fallback policy + return selectNewHost() } // UnmarshalCaddyfile sets up the module from Caddyfile tokens. Syntax: // -// lb_policy cookie [<name> [<secret>]] +// lb_policy cookie [<name> [<secret>]] { +// fallback <policy> +// } // // By default name is `lb` func (s *CookieHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { @@ -434,22 +708,25 @@ func (s *CookieHashSelection) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { default: return d.ArgErr() } - return nil -} - -// Select a new Host randomly and add a sticky session cookie -func selectNewHostWithCookieHashSelection(pool []*Upstream, w http.ResponseWriter, cookieSecret string, cookieName string) *Upstream { - randomHost := selectRandomHost(pool) - - if randomHost != nil { - // Hash (HMAC with some key for privacy) the upstream.Dial string as the cookie value - sha, err := hashCookie(cookieSecret, randomHost.Dial) - if err == nil { - // write the cookie. - http.SetCookie(w, &http.Cookie{Name: cookieName, Value: sha, Path: "/", Secure: false}) + for nesting := d.Nesting(); d.NextBlock(nesting); { + switch d.Val() { + case "fallback": + if !d.NextArg() { + return d.ArgErr() + } + if s.FallbackRaw != nil { + return d.Err("fallback selection policy already specified") + } + mod, err := loadFallbackPolicy(d) + if err != nil { + return err + } + s.FallbackRaw = mod + default: + return d.Errf("unrecognized option '%s'", d.Val()) } } - return randomHost + return nil } // hashCookie hashes (HMAC 256) some data with the secret @@ -512,6 +789,9 @@ func leastRequests(upstreams []*Upstream) *Upstream { if len(best) == 0 { return nil } + if len(best) == 1 { + return best[0] + } return best[weakrand.Intn(len(best))] //nolint:gosec } @@ -544,20 +824,40 @@ func hash(s string) uint32 { return h.Sum32() } +func loadFallbackPolicy(d *caddyfile.Dispenser) (json.RawMessage, error) { + name := d.Val() + modID := "http.reverse_proxy.selection_policies." + name + unm, err := caddyfile.UnmarshalModule(d, modID) + if err != nil { + return nil, err + } + sel, ok := unm.(Selector) + if !ok { + return nil, d.Errf("module %s (%T) is not a reverseproxy.Selector", modID, unm) + } + return caddyconfig.JSONModuleObject(sel, "policy", name, nil), nil +} + // Interface guards var ( _ Selector = (*RandomSelection)(nil) _ Selector = (*RandomChoiceSelection)(nil) _ Selector = (*LeastConnSelection)(nil) _ Selector = (*RoundRobinSelection)(nil) + _ Selector = (*WeightedRoundRobinSelection)(nil) _ Selector = (*FirstSelection)(nil) _ Selector = (*IPHashSelection)(nil) + _ Selector = (*ClientIPHashSelection)(nil) _ Selector = (*URIHashSelection)(nil) + _ Selector = (*QueryHashSelection)(nil) _ Selector = (*HeaderHashSelection)(nil) _ Selector = (*CookieHashSelection)(nil) - _ caddy.Validator = (*RandomChoiceSelection)(nil) + _ caddy.Validator = (*RandomChoiceSelection)(nil) + _ caddy.Provisioner = (*RandomChoiceSelection)(nil) + _ caddy.Provisioner = (*WeightedRoundRobinSelection)(nil) _ caddyfile.Unmarshaler = (*RandomChoiceSelection)(nil) + _ caddyfile.Unmarshaler = (*WeightedRoundRobinSelection)(nil) ) diff --git a/modules/caddyhttp/reverseproxy/selectionpolicies_test.go b/modules/caddyhttp/reverseproxy/selectionpolicies_test.go index 546a60d..dc613a5 100644 --- a/modules/caddyhttp/reverseproxy/selectionpolicies_test.go +++ b/modules/caddyhttp/reverseproxy/selectionpolicies_test.go @@ -15,9 +15,14 @@ package reverseproxy import ( + "context" "net/http" "net/http/httptest" "testing" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) func testPool() UpstreamPool { @@ -30,7 +35,7 @@ func testPool() UpstreamPool { func TestRoundRobinPolicy(t *testing.T) { pool := testPool() - rrPolicy := new(RoundRobinSelection) + rrPolicy := RoundRobinSelection{} req, _ := http.NewRequest("GET", "/", nil) h := rrPolicy.Select(pool, req, nil) @@ -69,9 +74,66 @@ func TestRoundRobinPolicy(t *testing.T) { } } +func TestWeightedRoundRobinPolicy(t *testing.T) { + pool := testPool() + wrrPolicy := WeightedRoundRobinSelection{ + Weights: []int{3, 2, 1}, + totalWeight: 6, + } + req, _ := http.NewRequest("GET", "/", nil) + + h := wrrPolicy.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected first weighted round robin host to be first host in the pool.") + } + h = wrrPolicy.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected second weighted round robin host to be first host in the pool.") + } + // Third selected host is 1, because counter starts at 0 + // and increments before host is selected + h = wrrPolicy.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected third weighted round robin host to be second host in the pool.") + } + h = wrrPolicy.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected fourth weighted round robin host to be second host in the pool.") + } + h = wrrPolicy.Select(pool, req, nil) + if h != pool[2] { + t.Error("Expected fifth weighted round robin host to be third host in the pool.") + } + h = wrrPolicy.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected sixth weighted round robin host to be first host in the pool.") + } + + // mark host as down + pool[0].setHealthy(false) + h = wrrPolicy.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected to skip down host.") + } + // mark host as up + pool[0].setHealthy(true) + + h = wrrPolicy.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected to select first host on availablity.") + } + // mark host as full + pool[1].countRequest(1) + pool[1].MaxRequests = 1 + h = wrrPolicy.Select(pool, req, nil) + if h != pool[2] { + t.Error("Expected to skip full host.") + } +} + func TestLeastConnPolicy(t *testing.T) { pool := testPool() - lcPolicy := new(LeastConnSelection) + lcPolicy := LeastConnSelection{} req, _ := http.NewRequest("GET", "/", nil) pool[0].countRequest(10) @@ -89,7 +151,7 @@ func TestLeastConnPolicy(t *testing.T) { func TestIPHashPolicy(t *testing.T) { pool := testPool() - ipHash := new(IPHashSelection) + ipHash := IPHashSelection{} req, _ := http.NewRequest("GET", "/", nil) // We should be able to predict where every request is routed. @@ -229,9 +291,152 @@ func TestIPHashPolicy(t *testing.T) { } } +func TestClientIPHashPolicy(t *testing.T) { + pool := testPool() + ipHash := ClientIPHashSelection{} + req, _ := http.NewRequest("GET", "/", nil) + req = req.WithContext(context.WithValue(req.Context(), caddyhttp.VarsCtxKey, make(map[string]any))) + + // We should be able to predict where every request is routed. + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.1:80") + h := ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.2:80") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.3:80") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.4:80") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + + // we should get the same results without a port + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.1") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.2") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.3") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.4") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + + // we should get a healthy host if the original host is unhealthy and a + // healthy host is available + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.4") + pool[1].setHealthy(false) + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.2") + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + pool[1].setHealthy(true) + + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.3") + pool[2].setHealthy(false) + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.4") + h = ipHash.Select(pool, req, nil) + if h != pool[1] { + t.Error("Expected ip hash policy host to be the second host.") + } + + // We should be able to resize the host pool and still be able to predict + // where a req will be routed with the same IP's used above + pool = UpstreamPool{ + {Host: new(Host), Dial: "0.0.0.2"}, + {Host: new(Host), Dial: "0.0.0.3"}, + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.1:80") + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.2:80") + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.3:80") + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + caddyhttp.SetVar(req.Context(), caddyhttp.ClientIPVarKey, "172.0.0.4:80") + h = ipHash.Select(pool, req, nil) + if h != pool[0] { + t.Error("Expected ip hash policy host to be the first host.") + } + + // We should get nil when there are no healthy hosts + pool[0].setHealthy(false) + pool[1].setHealthy(false) + h = ipHash.Select(pool, req, nil) + if h != nil { + t.Error("Expected ip hash policy host to be nil.") + } + + // Reproduce #4135 + pool = UpstreamPool{ + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + {Host: new(Host)}, + } + pool[0].setHealthy(false) + pool[1].setHealthy(false) + pool[2].setHealthy(false) + pool[3].setHealthy(false) + pool[4].setHealthy(false) + pool[5].setHealthy(false) + pool[6].setHealthy(false) + pool[7].setHealthy(false) + pool[8].setHealthy(true) + + // We should get a result back when there is one healthy host left. + h = ipHash.Select(pool, req, nil) + if h == nil { + // If it is nil, it means we missed a host even though one is available + t.Error("Expected ip hash policy host to not be nil, but it is nil.") + } +} + func TestFirstPolicy(t *testing.T) { pool := testPool() - firstPolicy := new(FirstSelection) + firstPolicy := FirstSelection{} req := httptest.NewRequest(http.MethodGet, "/", nil) h := firstPolicy.Select(pool, req, nil) @@ -246,9 +451,85 @@ func TestFirstPolicy(t *testing.T) { } } +func TestQueryHashPolicy(t *testing.T) { + ctx, cancel := caddy.NewContext(caddy.Context{Context: context.Background()}) + defer cancel() + queryPolicy := QueryHashSelection{Key: "foo"} + if err := queryPolicy.Provision(ctx); err != nil { + t.Errorf("Provision error: %v", err) + t.FailNow() + } + + pool := testPool() + + request := httptest.NewRequest(http.MethodGet, "/?foo=1", nil) + h := queryPolicy.Select(pool, request, nil) + if h != pool[0] { + t.Error("Expected query policy host to be the first host.") + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=100000", nil) + h = queryPolicy.Select(pool, request, nil) + if h != pool[0] { + t.Error("Expected query policy host to be the first host.") + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=1", nil) + pool[0].setHealthy(false) + h = queryPolicy.Select(pool, request, nil) + if h != pool[1] { + t.Error("Expected query policy host to be the second host.") + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=100000", nil) + h = queryPolicy.Select(pool, request, nil) + if h != pool[2] { + t.Error("Expected query policy host to be the third host.") + } + + // We should be able to resize the host pool and still be able to predict + // where a request will be routed with the same query used above + pool = UpstreamPool{ + {Host: new(Host)}, + {Host: new(Host)}, + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=1", nil) + h = queryPolicy.Select(pool, request, nil) + if h != pool[0] { + t.Error("Expected query policy host to be the first host.") + } + + pool[0].setHealthy(false) + h = queryPolicy.Select(pool, request, nil) + if h != pool[1] { + t.Error("Expected query policy host to be the second host.") + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=4", nil) + h = queryPolicy.Select(pool, request, nil) + if h != pool[1] { + t.Error("Expected query policy host to be the second host.") + } + + pool[0].setHealthy(false) + pool[1].setHealthy(false) + h = queryPolicy.Select(pool, request, nil) + if h != nil { + t.Error("Expected query policy policy host to be nil.") + } + + request = httptest.NewRequest(http.MethodGet, "/?foo=aa11&foo=bb22", nil) + pool = testPool() + h = queryPolicy.Select(pool, request, nil) + if h != pool[0] { + t.Error("Expected query policy host to be the first host.") + } +} + func TestURIHashPolicy(t *testing.T) { pool := testPool() - uriPolicy := new(URIHashSelection) + uriPolicy := URIHashSelection{} request := httptest.NewRequest(http.MethodGet, "/test", nil) h := uriPolicy.Select(pool, request, nil) @@ -337,8 +618,7 @@ func TestRandomChoicePolicy(t *testing.T) { pool[2].countRequest(30) request := httptest.NewRequest(http.MethodGet, "/test", nil) - randomChoicePolicy := new(RandomChoiceSelection) - randomChoicePolicy.Choose = 2 + randomChoicePolicy := RandomChoiceSelection{Choose: 2} h := randomChoicePolicy.Select(pool, request, nil) @@ -353,6 +633,14 @@ func TestRandomChoicePolicy(t *testing.T) { } func TestCookieHashPolicy(t *testing.T) { + ctx, cancel := caddy.NewContext(caddy.Context{Context: context.Background()}) + defer cancel() + cookieHashPolicy := CookieHashSelection{} + if err := cookieHashPolicy.Provision(ctx); err != nil { + t.Errorf("Provision error: %v", err) + t.FailNow() + } + pool := testPool() pool[0].Dial = "localhost:8080" pool[1].Dial = "localhost:8081" @@ -362,7 +650,7 @@ func TestCookieHashPolicy(t *testing.T) { pool[2].setHealthy(false) request := httptest.NewRequest(http.MethodGet, "/test", nil) w := httptest.NewRecorder() - cookieHashPolicy := new(CookieHashSelection) + h := cookieHashPolicy.Select(pool, request, w) cookieServer1 := w.Result().Cookies()[0] if cookieServer1 == nil { @@ -399,3 +687,59 @@ func TestCookieHashPolicy(t *testing.T) { t.Error("Expected cookieHashPolicy to set a new cookie.") } } + +func TestCookieHashPolicyWithFirstFallback(t *testing.T) { + ctx, cancel := caddy.NewContext(caddy.Context{Context: context.Background()}) + defer cancel() + cookieHashPolicy := CookieHashSelection{ + FallbackRaw: caddyconfig.JSONModuleObject(FirstSelection{}, "policy", "first", nil), + } + if err := cookieHashPolicy.Provision(ctx); err != nil { + t.Errorf("Provision error: %v", err) + t.FailNow() + } + + pool := testPool() + pool[0].Dial = "localhost:8080" + pool[1].Dial = "localhost:8081" + pool[2].Dial = "localhost:8082" + pool[0].setHealthy(true) + pool[1].setHealthy(true) + pool[2].setHealthy(true) + request := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + + h := cookieHashPolicy.Select(pool, request, w) + cookieServer1 := w.Result().Cookies()[0] + if cookieServer1 == nil { + t.Fatal("cookieHashPolicy should set a cookie") + } + if cookieServer1.Name != "lb" { + t.Error("cookieHashPolicy should set a cookie with name lb") + } + if h != pool[0] { + t.Errorf("Expected cookieHashPolicy host to be the first only available host, got %s", h) + } + request = httptest.NewRequest(http.MethodGet, "/test", nil) + w = httptest.NewRecorder() + request.AddCookie(cookieServer1) + h = cookieHashPolicy.Select(pool, request, w) + if h != pool[0] { + t.Errorf("Expected cookieHashPolicy host to stick to the first host (matching cookie), got %s", h) + } + s := w.Result().Cookies() + if len(s) != 0 { + t.Error("Expected cookieHashPolicy to not set a new cookie.") + } + pool[0].setHealthy(false) + request = httptest.NewRequest(http.MethodGet, "/test", nil) + w = httptest.NewRecorder() + request.AddCookie(cookieServer1) + h = cookieHashPolicy.Select(pool, request, w) + if h != pool[1] { + t.Errorf("Expected cookieHashPolicy to select the next first available host, got %s", h) + } + if w.Result().Cookies() == nil { + t.Error("Expected cookieHashPolicy to set a new cookie.") + } +} diff --git a/modules/caddyhttp/reverseproxy/streaming.go b/modules/caddyhttp/reverseproxy/streaming.go index 1db107a..155a1df 100644 --- a/modules/caddyhttp/reverseproxy/streaming.go +++ b/modules/caddyhttp/reverseproxy/streaming.go @@ -20,6 +20,8 @@ package reverseproxy import ( "context" + "errors" + "fmt" "io" weakrand "math/rand" "mime" @@ -32,32 +34,46 @@ import ( "golang.org/x/net/http/httpguts" ) -func (h Handler) handleUpgradeResponse(logger *zap.Logger, rw http.ResponseWriter, req *http.Request, res *http.Response) { +func (h *Handler) handleUpgradeResponse(logger *zap.Logger, rw http.ResponseWriter, req *http.Request, res *http.Response) { reqUpType := upgradeType(req.Header) resUpType := upgradeType(res.Header) // Taken from https://github.com/golang/go/commit/5c489514bc5e61ad9b5b07bd7d8ec65d66a0512a // We know reqUpType is ASCII, it's checked by the caller. if !asciiIsPrint(resUpType) { - h.logger.Debug("backend tried to switch to invalid protocol", + logger.Debug("backend tried to switch to invalid protocol", zap.String("backend_upgrade", resUpType)) return } if !asciiEqualFold(reqUpType, resUpType) { - h.logger.Debug("backend tried to switch to unexpected protocol via Upgrade header", + logger.Debug("backend tried to switch to unexpected protocol via Upgrade header", zap.String("backend_upgrade", resUpType), zap.String("requested_upgrade", reqUpType)) return } - hj, ok := rw.(http.Hijacker) + backConn, ok := res.Body.(io.ReadWriteCloser) if !ok { + logger.Error("internal error: 101 switching protocols response with non-writable body") + return + } + + // write header first, response headers should not be counted in size + // like the rest of handler chain. + copyHeader(rw.Header(), res.Header) + rw.WriteHeader(res.StatusCode) + + logger.Debug("upgrading connection") + + //nolint:bodyclose + conn, brw, hijackErr := http.NewResponseController(rw).Hijack() + if errors.Is(hijackErr, http.ErrNotSupported) { h.logger.Sugar().Errorf("can't switch protocols using non-Hijacker ResponseWriter type %T", rw) return } - backConn, ok := res.Body.(io.ReadWriteCloser) - if !ok { - h.logger.Error("internal error: 101 switching protocols response with non-writable body") + + if hijackErr != nil { + h.logger.Error("hijack failed on protocol switch", zap.Error(hijackErr)) return } @@ -74,18 +90,6 @@ func (h Handler) handleUpgradeResponse(logger *zap.Logger, rw http.ResponseWrite }() defer close(backConnCloseCh) - // write header first, response headers should not be counted in size - // like the rest of handler chain. - copyHeader(rw.Header(), res.Header) - rw.WriteHeader(res.StatusCode) - - logger.Debug("upgrading connection") - conn, brw, err := hj.Hijack() - if err != nil { - h.logger.Error("hijack failed on protocol switch", zap.Error(err)) - return - } - start := time.Now() defer func() { conn.Close() @@ -93,7 +97,7 @@ func (h Handler) handleUpgradeResponse(logger *zap.Logger, rw http.ResponseWrite }() if err := brw.Flush(); err != nil { - h.logger.Debug("response flush", zap.Error(err)) + logger.Debug("response flush", zap.Error(err)) return } @@ -119,10 +123,23 @@ func (h Handler) handleUpgradeResponse(logger *zap.Logger, rw http.ResponseWrite spc := switchProtocolCopier{user: conn, backend: backConn} + // setup the timeout if requested + var timeoutc <-chan time.Time + if h.StreamTimeout > 0 { + timer := time.NewTimer(time.Duration(h.StreamTimeout)) + defer timer.Stop() + timeoutc = timer.C + } + errc := make(chan error, 1) go spc.copyToBackend(errc) go spc.copyFromBackend(errc) - <-errc + select { + case err := <-errc: + logger.Debug("streaming error", zap.Error(err)) + case time := <-timeoutc: + logger.Debug("stream timed out", zap.Time("timeout", time)) + } } // flushInterval returns the p.FlushInterval value, conditionally @@ -167,38 +184,58 @@ func (h Handler) isBidirectionalStream(req *http.Request, res *http.Response) bo (ae == "identity" || ae == "") } -func (h Handler) copyResponse(dst io.Writer, src io.Reader, flushInterval time.Duration) error { +func (h Handler) copyResponse(dst http.ResponseWriter, src io.Reader, flushInterval time.Duration, logger *zap.Logger) error { + var w io.Writer = dst + if flushInterval != 0 { - if wf, ok := dst.(writeFlusher); ok { - mlw := &maxLatencyWriter{ - dst: wf, - latency: flushInterval, - } - defer mlw.stop() + var mlwLogger *zap.Logger + if h.VerboseLogs { + mlwLogger = logger.Named("max_latency_writer") + } else { + mlwLogger = zap.NewNop() + } + mlw := &maxLatencyWriter{ + dst: dst, + //nolint:bodyclose + flush: http.NewResponseController(dst).Flush, + latency: flushInterval, + logger: mlwLogger, + } + defer mlw.stop() - // set up initial timer so headers get flushed even if body writes are delayed - mlw.flushPending = true - mlw.t = time.AfterFunc(flushInterval, mlw.delayedFlush) + // set up initial timer so headers get flushed even if body writes are delayed + mlw.flushPending = true + mlw.t = time.AfterFunc(flushInterval, mlw.delayedFlush) - dst = mlw - } + w = mlw } buf := streamingBufPool.Get().(*[]byte) defer streamingBufPool.Put(buf) - _, err := h.copyBuffer(dst, src, *buf) + + var copyLogger *zap.Logger + if h.VerboseLogs { + copyLogger = logger + } else { + copyLogger = zap.NewNop() + } + + _, err := h.copyBuffer(w, src, *buf, copyLogger) return err } // copyBuffer returns any write errors or non-EOF read errors, and the amount // of bytes written. -func (h Handler) copyBuffer(dst io.Writer, src io.Reader, buf []byte) (int64, error) { +func (h Handler) copyBuffer(dst io.Writer, src io.Reader, buf []byte, logger *zap.Logger) (int64, error) { if len(buf) == 0 { buf = make([]byte, defaultBufferSize) } var written int64 for { + logger.Debug("waiting to read from upstream") nr, rerr := src.Read(buf) + logger := logger.With(zap.Int("read", nr)) + logger.Debug("read from upstream", zap.Error(rerr)) if rerr != nil && rerr != io.EOF && rerr != context.Canceled { // TODO: this could be useful to know (indeed, it revealed an error in our // fastcgi PoC earlier; but it's this single error report here that necessitates @@ -210,12 +247,17 @@ func (h Handler) copyBuffer(dst io.Writer, src io.Reader, buf []byte) (int64, er h.logger.Error("reading from backend", zap.Error(rerr)) } if nr > 0 { + logger.Debug("writing to downstream") nw, werr := dst.Write(buf[:nr]) if nw > 0 { written += int64(nw) } + logger.Debug("wrote to downstream", + zap.Int("written", nw), + zap.Int64("written_total", written), + zap.Error(werr)) if werr != nil { - return written, werr + return written, fmt.Errorf("writing: %w", werr) } if nr != nw { return written, io.ErrShortWrite @@ -223,9 +265,9 @@ func (h Handler) copyBuffer(dst io.Writer, src io.Reader, buf []byte) (int64, er } if rerr != nil { if rerr == io.EOF { - rerr = nil + return written, nil } - return written, rerr + return written, fmt.Errorf("reading: %w", rerr) } } } @@ -242,10 +284,70 @@ func (h *Handler) registerConnection(conn io.ReadWriteCloser, gracefulClose func return func() { h.connectionsMu.Lock() delete(h.connections, conn) + // if there is no connection left before the connections close timer fires + if len(h.connections) == 0 && h.connectionsCloseTimer != nil { + // we release the timer that holds the reference to Handler + if (*h.connectionsCloseTimer).Stop() { + h.logger.Debug("stopped streaming connections close timer - all connections are already closed") + } + h.connectionsCloseTimer = nil + } h.connectionsMu.Unlock() } } +// closeConnections immediately closes all hijacked connections (both to client and backend). +func (h *Handler) closeConnections() error { + var err error + h.connectionsMu.Lock() + defer h.connectionsMu.Unlock() + + for _, oc := range h.connections { + if oc.gracefulClose != nil { + // this is potentially blocking while we have the lock on the connections + // map, but that should be OK since the server has in theory shut down + // and we are no longer using the connections map + gracefulErr := oc.gracefulClose() + if gracefulErr != nil && err == nil { + err = gracefulErr + } + } + closeErr := oc.conn.Close() + if closeErr != nil && err == nil { + err = closeErr + } + } + return err +} + +// cleanupConnections closes hijacked connections. +// Depending on the value of StreamCloseDelay it does that either immediately +// or sets up a timer that will do that later. +func (h *Handler) cleanupConnections() error { + if h.StreamCloseDelay == 0 { + return h.closeConnections() + } + + h.connectionsMu.Lock() + defer h.connectionsMu.Unlock() + // the handler is shut down, no new connection can appear, + // so we can skip setting up the timer when there are no connections + if len(h.connections) > 0 { + delay := time.Duration(h.StreamCloseDelay) + h.connectionsCloseTimer = time.AfterFunc(delay, func() { + h.logger.Debug("closing streaming connections after delay", + zap.Duration("delay", delay)) + err := h.closeConnections() + if err != nil { + h.logger.Error("failed to closed connections after delay", + zap.Error(err), + zap.Duration("delay", delay)) + } + }) + } + return nil +} + // writeCloseControl sends a best-effort Close control message to the given // WebSocket connection. Thanks to @pascaldekloe who provided inspiration // from his simple implementation of this I was able to learn from at: @@ -365,29 +467,30 @@ type openConnection struct { gracefulClose func() error } -type writeFlusher interface { - io.Writer - http.Flusher -} - type maxLatencyWriter struct { - dst writeFlusher + dst io.Writer + flush func() error latency time.Duration // non-zero; negative means to flush immediately mu sync.Mutex // protects t, flushPending, and dst.Flush t *time.Timer flushPending bool + logger *zap.Logger } func (m *maxLatencyWriter) Write(p []byte) (n int, err error) { m.mu.Lock() defer m.mu.Unlock() n, err = m.dst.Write(p) + m.logger.Debug("wrote bytes", zap.Int("n", n), zap.Error(err)) if m.latency < 0 { - m.dst.Flush() + m.logger.Debug("flushing immediately") + //nolint:errcheck + m.flush() return } if m.flushPending { + m.logger.Debug("delayed flush already pending") return } if m.t == nil { @@ -395,6 +498,7 @@ func (m *maxLatencyWriter) Write(p []byte) (n int, err error) { } else { m.t.Reset(m.latency) } + m.logger.Debug("timer set for delayed flush", zap.Duration("duration", m.latency)) m.flushPending = true return } @@ -403,9 +507,12 @@ func (m *maxLatencyWriter) delayedFlush() { m.mu.Lock() defer m.mu.Unlock() if !m.flushPending { // if stop was called but AfterFunc already started this goroutine + m.logger.Debug("delayed flush is not pending") return } - m.dst.Flush() + m.logger.Debug("delayed flush") + //nolint:errcheck + m.flush() m.flushPending = false } @@ -445,5 +552,7 @@ var streamingBufPool = sync.Pool{ }, } -const defaultBufferSize = 32 * 1024 -const wordSize = int(unsafe.Sizeof(uintptr(0))) +const ( + defaultBufferSize = 32 * 1024 + wordSize = int(unsafe.Sizeof(uintptr(0))) +) diff --git a/modules/caddyhttp/reverseproxy/streaming_test.go b/modules/caddyhttp/reverseproxy/streaming_test.go index 4ed1f1e..3f6da2f 100644 --- a/modules/caddyhttp/reverseproxy/streaming_test.go +++ b/modules/caddyhttp/reverseproxy/streaming_test.go @@ -2,8 +2,11 @@ package reverseproxy import ( "bytes" + "net/http/httptest" "strings" "testing" + + "github.com/caddyserver/caddy/v2" ) func TestHandlerCopyResponse(t *testing.T) { @@ -13,12 +16,15 @@ func TestHandlerCopyResponse(t *testing.T) { strings.Repeat("a", defaultBufferSize), strings.Repeat("123456789 123456789 123456789 12", 3000), } + dst := bytes.NewBuffer(nil) + recorder := httptest.NewRecorder() + recorder.Body = dst for _, d := range testdata { src := bytes.NewBuffer([]byte(d)) dst.Reset() - err := h.copyResponse(dst, src, 0) + err := h.copyResponse(recorder, src, 0, caddy.Log()) if err != nil { t.Errorf("failed with error: %v", err) } diff --git a/modules/caddyhttp/reverseproxy/upstreams.go b/modules/caddyhttp/reverseproxy/upstreams.go index 7a90016..2d21a5c 100644 --- a/modules/caddyhttp/reverseproxy/upstreams.go +++ b/modules/caddyhttp/reverseproxy/upstreams.go @@ -8,12 +8,12 @@ import ( "net" "net/http" "strconv" - "strings" "sync" "time" - "github.com/caddyserver/caddy/v2" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { @@ -114,7 +114,7 @@ func (su SRVUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { cached := srvs[suAddr] srvsMu.RUnlock() if cached.isFresh() { - return cached.upstreams, nil + return allNew(cached.upstreams), nil } // otherwise, obtain a write-lock to update the cached value @@ -126,7 +126,7 @@ func (su SRVUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { // have refreshed it in the meantime before we re-obtained our lock cached = srvs[suAddr] if cached.isFresh() { - return cached.upstreams, nil + return allNew(cached.upstreams), nil } su.logger.Debug("refreshing SRV upstreams", @@ -145,7 +145,7 @@ func (su SRVUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { su.logger.Warn("SRV records filtered", zap.Error(err)) } - upstreams := make([]*Upstream, len(records)) + upstreams := make([]Upstream, len(records)) for i, rec := range records { su.logger.Debug("discovered SRV record", zap.String("target", rec.Target), @@ -153,7 +153,7 @@ func (su SRVUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { zap.Uint16("priority", rec.Priority), zap.Uint16("weight", rec.Weight)) addr := net.JoinHostPort(rec.Target, strconv.Itoa(int(rec.Port))) - upstreams[i] = &Upstream{Dial: addr} + upstreams[i] = Upstream{Dial: addr} } // before adding a new one to the cache (as opposed to replacing stale one), make room if cache is full @@ -170,7 +170,7 @@ func (su SRVUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { upstreams: upstreams, } - return upstreams, nil + return allNew(upstreams), nil } func (su SRVUpstreams) String() string { @@ -206,13 +206,18 @@ func (SRVUpstreams) formattedAddr(service, proto, name string) string { type srvLookup struct { srvUpstreams SRVUpstreams freshness time.Time - upstreams []*Upstream + upstreams []Upstream } func (sl srvLookup) isFresh() bool { return time.Since(sl.freshness) < time.Duration(sl.srvUpstreams.Refresh) } +type IPVersions struct { + IPv4 *bool `json:"ipv4,omitempty"` + IPv6 *bool `json:"ipv6,omitempty"` +} + // AUpstreams provides upstreams from A/AAAA lookups. // Results are cached and refreshed at the configured // refresh interval. @@ -240,7 +245,14 @@ type AUpstreams struct { // A negative value disables this. FallbackDelay caddy.Duration `json:"dial_fallback_delay,omitempty"` + // The IP versions to resolve for. By default, both + // "ipv4" and "ipv6" will be enabled, which + // correspond to A and AAAA records respectively. + Versions *IPVersions `json:"versions,omitempty"` + resolver *net.Resolver + + logger *zap.Logger } // CaddyModule returns the Caddy module information. @@ -251,7 +263,8 @@ func (AUpstreams) CaddyModule() caddy.ModuleInfo { } } -func (au *AUpstreams) Provision(_ caddy.Context) error { +func (au *AUpstreams) Provision(ctx caddy.Context) error { + au.logger = ctx.Logger() if au.Refresh == 0 { au.Refresh = caddy.Duration(time.Minute) } @@ -286,14 +299,36 @@ func (au *AUpstreams) Provision(_ caddy.Context) error { func (au AUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer) - auStr := repl.ReplaceAll(au.String(), "") + + resolveIpv4 := au.Versions == nil || au.Versions.IPv4 == nil || *au.Versions.IPv4 + resolveIpv6 := au.Versions == nil || au.Versions.IPv6 == nil || *au.Versions.IPv6 + + // Map ipVersion early, so we can use it as part of the cache-key. + // This should be fairly inexpensive and comes and the upside of + // allowing the same dynamic upstream (name + port combination) + // to be used multiple times with different ip versions. + // + // It also forced a cache-miss if a previously cached dynamic + // upstream changes its ip version, e.g. after a config reload, + // while keeping the cache-invalidation as simple as it currently is. + var ipVersion string + switch { + case resolveIpv4 && !resolveIpv6: + ipVersion = "ip4" + case !resolveIpv4 && resolveIpv6: + ipVersion = "ip6" + default: + ipVersion = "ip" + } + + auStr := repl.ReplaceAll(au.String()+ipVersion, "") // first, use a cheap read-lock to return a cached result quickly aAaaaMu.RLock() cached := aAaaa[auStr] aAaaaMu.RUnlock() if cached.isFresh() { - return cached.upstreams, nil + return allNew(cached.upstreams), nil } // otherwise, obtain a write-lock to update the cached value @@ -305,26 +340,33 @@ func (au AUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { // have refreshed it in the meantime before we re-obtained our lock cached = aAaaa[auStr] if cached.isFresh() { - return cached.upstreams, nil + return allNew(cached.upstreams), nil } name := repl.ReplaceAll(au.Name, "") port := repl.ReplaceAll(au.Port, "") - ips, err := au.resolver.LookupIPAddr(r.Context(), name) + au.logger.Debug("refreshing A upstreams", + zap.String("version", ipVersion), + zap.String("name", name), + zap.String("port", port)) + + ips, err := au.resolver.LookupIP(r.Context(), ipVersion, name) if err != nil { return nil, err } - upstreams := make([]*Upstream, len(ips)) + upstreams := make([]Upstream, len(ips)) for i, ip := range ips { - upstreams[i] = &Upstream{ + au.logger.Debug("discovered A record", + zap.String("ip", ip.String())) + upstreams[i] = Upstream{ Dial: net.JoinHostPort(ip.String(), port), } } // before adding a new one to the cache (as opposed to replacing stale one), make room if cache is full - if cached.freshness.IsZero() && len(srvs) >= 100 { + if cached.freshness.IsZero() && len(aAaaa) >= 100 { for randomKey := range aAaaa { delete(aAaaa, randomKey) break @@ -337,7 +379,7 @@ func (au AUpstreams) GetUpstreams(r *http.Request) ([]*Upstream, error) { upstreams: upstreams, } - return upstreams, nil + return allNew(upstreams), nil } func (au AUpstreams) String() string { return net.JoinHostPort(au.Name, au.Port) } @@ -345,7 +387,7 @@ func (au AUpstreams) String() string { return net.JoinHostPort(au.Name, au.Port) type aLookup struct { aUpstreams AUpstreams freshness time.Time - upstreams []*Upstream + upstreams []Upstream } func (al aLookup) isFresh() bool { @@ -439,16 +481,9 @@ type UpstreamResolver struct { // and ensures they're ready to be used. func (u *UpstreamResolver) ParseAddresses() error { for _, v := range u.Addresses { - addr, err := caddy.ParseNetworkAddress(v) + addr, err := caddy.ParseNetworkAddressWithDefaults(v, "udp", 53) if err != nil { - // If a port wasn't specified for the resolver, - // try defaulting to 53 and parse again - if strings.Contains(err.Error(), "missing port in address") { - addr, err = caddy.ParseNetworkAddress(v + ":53") - } - if err != nil { - return err - } + return err } if addr.PortRangeSize() != 1 { return fmt.Errorf("resolver address must have exactly one address; cannot call %v", addr) @@ -458,6 +493,14 @@ func (u *UpstreamResolver) ParseAddresses() error { return nil } +func allNew(upstreams []Upstream) []*Upstream { + results := make([]*Upstream, len(upstreams)) + for i := range upstreams { + results[i] = &Upstream{Dial: upstreams[i].Dial} + } + return results +} + var ( srvs = make(map[string]srvLookup) srvsMu sync.RWMutex diff --git a/modules/caddyhttp/rewrite/rewrite.go b/modules/caddyhttp/rewrite/rewrite.go index 31c9778..77ef668 100644 --- a/modules/caddyhttp/rewrite/rewrite.go +++ b/modules/caddyhttp/rewrite/rewrite.go @@ -22,9 +22,10 @@ import ( "strconv" "strings" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "go.uber.org/zap" ) func init() { @@ -195,16 +196,10 @@ func (rewr Rewrite) Rewrite(r *http.Request, repl *caddy.Replacer) bool { var newPath, newQuery, newFrag string if path != "" { - // Since the 'uri' placeholder performs a URL-encode, - // we need to intercept it so that it doesn't, because - // otherwise we risk a double-encode of the path. - uriPlaceholder := "{http.request.uri}" - if strings.Contains(path, uriPlaceholder) { - tmpUri := r.URL.Path - if r.URL.RawQuery != "" { - tmpUri += "?" + r.URL.RawQuery - } - path = strings.ReplaceAll(path, uriPlaceholder, tmpUri) + // replace the `path` placeholder to escaped path + pathPlaceholder := "{http.request.uri.path}" + if strings.Contains(path, pathPlaceholder) { + path = strings.ReplaceAll(path, pathPlaceholder, r.URL.EscapedPath()) } newPath = repl.ReplaceAll(path, "") @@ -232,7 +227,11 @@ func (rewr Rewrite) Rewrite(r *http.Request, repl *caddy.Replacer) bool { // update the URI with the new components // only after building them if pathStart >= 0 { - r.URL.Path = newPath + if path, err := url.PathUnescape(newPath); err != nil { + r.URL.Path = newPath + } else { + r.URL.Path = path + } } if qsStart >= 0 { r.URL.RawQuery = newQuery diff --git a/modules/caddyhttp/rewrite/rewrite_test.go b/modules/caddyhttp/rewrite/rewrite_test.go index 5875983..bb937ec 100644 --- a/modules/caddyhttp/rewrite/rewrite_test.go +++ b/modules/caddyhttp/rewrite/rewrite_test.go @@ -60,6 +60,16 @@ func TestRewrite(t *testing.T) { expect: newRequest(t, "GET", "foo"), }, { + rule: Rewrite{URI: "{http.request.uri}"}, + input: newRequest(t, "GET", "/bar%3Fbaz?c=d"), + expect: newRequest(t, "GET", "/bar%3Fbaz?c=d"), + }, + { + rule: Rewrite{URI: "{http.request.uri.path}"}, + input: newRequest(t, "GET", "/bar%3Fbaz"), + expect: newRequest(t, "GET", "/bar%3Fbaz"), + }, + { rule: Rewrite{URI: "/foo{http.request.uri.path}"}, input: newRequest(t, "GET", "/bar"), expect: newRequest(t, "GET", "/foo/bar"), @@ -323,7 +333,7 @@ func TestRewrite(t *testing.T) { input: newRequest(t, "GET", "/foo/findme%2Fbar"), expect: newRequest(t, "GET", "/foo/replaced%2Fbar"), }, - + { rule: Rewrite{PathRegexp: []*regexReplacer{{Find: "/{2,}", Replace: "/"}}}, input: newRequest(t, "GET", "/foo//bar///baz?a=b//c"), diff --git a/modules/caddyhttp/routes.go b/modules/caddyhttp/routes.go index da25097..9be3d01 100644 --- a/modules/caddyhttp/routes.go +++ b/modules/caddyhttp/routes.go @@ -120,6 +120,59 @@ func (r Route) String() string { r.Group, r.MatcherSetsRaw, handlersRaw, r.Terminal) } +// Provision sets up both the matchers and handlers in the route. +func (r *Route) Provision(ctx caddy.Context, metrics *Metrics) error { + err := r.ProvisionMatchers(ctx) + if err != nil { + return err + } + return r.ProvisionHandlers(ctx, metrics) +} + +// ProvisionMatchers sets up all the matchers by loading the +// matcher modules. Only call this method directly if you need +// to set up matchers and handlers separately without having +// to provision a second time; otherwise use Provision instead. +func (r *Route) ProvisionMatchers(ctx caddy.Context) error { + // matchers + matchersIface, err := ctx.LoadModule(r, "MatcherSetsRaw") + if err != nil { + return fmt.Errorf("loading matcher modules: %v", err) + } + err = r.MatcherSets.FromInterface(matchersIface) + if err != nil { + return err + } + return nil +} + +// ProvisionHandlers sets up all the handlers by loading the +// handler modules. Only call this method directly if you need +// to set up matchers and handlers separately without having +// to provision a second time; otherwise use Provision instead. +func (r *Route) ProvisionHandlers(ctx caddy.Context, metrics *Metrics) error { + handlersIface, err := ctx.LoadModule(r, "HandlersRaw") + if err != nil { + return fmt.Errorf("loading handler modules: %v", err) + } + for _, handler := range handlersIface.([]any) { + r.Handlers = append(r.Handlers, handler.(MiddlewareHandler)) + } + + // pre-compile the middleware handler chain + for _, midhandler := range r.Handlers { + r.middleware = append(r.middleware, wrapMiddleware(ctx, midhandler, metrics)) + } + return nil +} + +// Compile prepares a middleware chain from the route list. +// This should only be done once during the request, just +// before the middleware chain is executed. +func (r Route) Compile(next Handler) Handler { + return wrapRoute(r)(next) +} + // RouteList is a list of server routes that can // create a middleware chain. type RouteList []Route @@ -139,12 +192,7 @@ func (routes RouteList) Provision(ctx caddy.Context) error { // to provision a second time; otherwise use Provision instead. func (routes RouteList) ProvisionMatchers(ctx caddy.Context) error { for i := range routes { - // matchers - matchersIface, err := ctx.LoadModule(&routes[i], "MatcherSetsRaw") - if err != nil { - return fmt.Errorf("route %d: loading matcher modules: %v", i, err) - } - err = routes[i].MatcherSets.FromInterface(matchersIface) + err := routes[i].ProvisionMatchers(ctx) if err != nil { return fmt.Errorf("route %d: %v", i, err) } @@ -158,25 +206,18 @@ func (routes RouteList) ProvisionMatchers(ctx caddy.Context) error { // to provision a second time; otherwise use Provision instead. func (routes RouteList) ProvisionHandlers(ctx caddy.Context, metrics *Metrics) error { for i := range routes { - handlersIface, err := ctx.LoadModule(&routes[i], "HandlersRaw") + err := routes[i].ProvisionHandlers(ctx, metrics) if err != nil { - return fmt.Errorf("route %d: loading handler modules: %v", i, err) - } - for _, handler := range handlersIface.([]any) { - routes[i].Handlers = append(routes[i].Handlers, handler.(MiddlewareHandler)) - } - - // pre-compile the middleware handler chain - for _, midhandler := range routes[i].Handlers { - routes[i].middleware = append(routes[i].middleware, wrapMiddleware(ctx, midhandler, metrics)) + return fmt.Errorf("route %d: %v", i, err) } } return nil } // Compile prepares a middleware chain from the route list. -// This should only be done once: after all the routes have -// been provisioned, and before serving requests. +// This should only be done either once during provisioning +// for top-level routes, or on each request just before the +// middleware chain is executed for subroutes. func (routes RouteList) Compile(next Handler) Handler { mid := make([]Middleware, 0, len(routes)) for _, route := range routes { diff --git a/modules/caddyhttp/server.go b/modules/caddyhttp/server.go index 13ebbe6..cf17609 100644 --- a/modules/caddyhttp/server.go +++ b/modules/caddyhttp/server.go @@ -19,6 +19,7 @@ import ( "crypto/tls" "encoding/json" "fmt" + "io" "net" "net/http" "net/netip" @@ -29,14 +30,15 @@ import ( "sync/atomic" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyevents" - "github.com/caddyserver/caddy/v2/modules/caddytls" "github.com/caddyserver/certmagic" "github.com/quic-go/quic-go" "github.com/quic-go/quic-go/http3" "go.uber.org/zap" "go.uber.org/zap/zapcore" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyevents" + "github.com/caddyserver/caddy/v2/modules/caddytls" ) // Server describes an HTTP server. @@ -81,6 +83,26 @@ type Server struct { // HTTP request headers. MaxHeaderBytes int `json:"max_header_bytes,omitempty"` + // Enable full-duplex communication for HTTP/1 requests. + // Only has an effect if Caddy was built with Go 1.21 or later. + // + // For HTTP/1 requests, the Go HTTP server by default consumes any + // unread portion of the request body before beginning to write the + // response, preventing handlers from concurrently reading from the + // request and writing the response. Enabling this option disables + // this behavior and permits handlers to continue to read from the + // request while concurrently writing the response. + // + // For HTTP/2 requests, the Go HTTP server always permits concurrent + // reads and responses, so this option has no effect. + // + // Test thoroughly with your HTTP clients, as some older clients may + // not support full-duplex HTTP/1 which can cause them to deadlock. + // See https://github.com/golang/go/issues/57786 for more info. + // + // TODO: This is an EXPERIMENTAL feature. Subject to change or removal. + EnableFullDuplex bool `json:"enable_full_duplex,omitempty"` + // Routes describes how this server will handle requests. // Routes are executed sequentially. First a route's matchers // are evaluated, then its grouping. If it matches and has @@ -101,6 +123,16 @@ type Server struct { // The error routes work exactly like the normal routes. Errors *HTTPErrorConfig `json:"errors,omitempty"` + // NamedRoutes describes a mapping of reusable routes that can be + // invoked by their name. This can be used to optimize memory usage + // when the same route is needed for many subroutes, by having + // the handlers and matchers be only provisioned once, but used from + // many places. These routes are not executed unless they are invoked + // from another route. + // + // EXPERIMENTAL: Subject to change or removal. + NamedRoutes map[string]*Route `json:"named_routes,omitempty"` + // How to handle TLS connections. At least one policy is // required to enable HTTPS on this server if automatic // HTTPS is disabled or does not apply. @@ -130,6 +162,17 @@ type Server struct { // to trust sensitive incoming `X-Forwarded-*` headers. TrustedProxiesRaw json.RawMessage `json:"trusted_proxies,omitempty" caddy:"namespace=http.ip_sources inline_key=source"` + // The headers from which the client IP address could be + // read from. These will be considered in order, with the + // first good value being used as the client IP. + // By default, only `X-Forwarded-For` is considered. + // + // This depends on `trusted_proxies` being configured and + // the request being validated as coming from a trusted + // proxy, otherwise the client IP will be set to the direct + // remote IP address. + ClientIPHeaders []string `json:"client_ip_headers,omitempty"` + // Enables access logging and configures how access logs are handled // in this server. To minimally enable access logs, simply set this // to a non-null, empty struct. @@ -186,6 +229,7 @@ type Server struct { server *http.Server h3server *http3.Server h3listeners []net.PacketConn // TODO: we have to hold these because quic-go won't close listeners it didn't create + h2listeners []*http2Listener addresses []caddy.NetworkAddress trustedProxies IPRangeSource @@ -201,6 +245,18 @@ type Server struct { // ServeHTTP is the entry point for all HTTP requests. func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + // If there are listener wrappers that process tls connections but don't return a *tls.Conn, this field will be nil. + // TODO: Can be removed if https://github.com/golang/go/pull/56110 is ever merged. + if r.TLS == nil { + // not all requests have a conn (like virtual requests) - see #5698 + if conn, ok := r.Context().Value(ConnCtxKey).(net.Conn); ok { + if csc, ok := conn.(connectionStateConn); ok { + r.TLS = new(tls.ConnectionState) + *r.TLS = csc.ConnectionState() + } + } + } + w.Header().Set("Server", "Caddy") // advertise HTTP/3, if enabled @@ -231,6 +287,17 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { repl := caddy.NewReplacer() r = PrepareRequest(r, repl, w, s) + // enable full-duplex for HTTP/1, ensuring the entire + // request body gets consumed before writing the response + if s.EnableFullDuplex { + // TODO: Remove duplex_go12*.go abstraction once our + // minimum Go version is 1.21 or later + err := enableFullDuplex(w) + if err != nil { + s.accessLogger.Warn("failed to enable full duplex", zap.Error(err)) + } + } + // encode the request for logging purposes before // it enters any handler chain; this is necessary // to capture the original request in case it gets @@ -248,43 +315,18 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { wrec := NewResponseRecorder(w, nil, nil) w = wrec + // wrap the request body in a LengthReader + // so we can track the number of bytes read from it + var bodyReader *lengthReader + if r.Body != nil { + bodyReader = &lengthReader{Source: r.Body} + r.Body = bodyReader + } + // capture the original version of the request accLog := s.accessLogger.With(loggableReq) - defer func() { - // this request may be flagged as omitted from the logs - if skipLog, ok := GetVar(r.Context(), SkipLogVar).(bool); ok && skipLog { - return - } - - repl.Set("http.response.status", wrec.Status()) // will be 0 if no response is written by us (Go will write 200 to client) - repl.Set("http.response.size", wrec.Size()) - repl.Set("http.response.duration", duration) - repl.Set("http.response.duration_ms", duration.Seconds()*1e3) // multiply seconds to preserve decimal (see #4666) - - logger := accLog - if s.Logs != nil { - logger = s.Logs.wrapLogger(logger, r.Host) - } - - log := logger.Info - if wrec.Status() >= 400 { - log = logger.Error - } - - userID, _ := repl.GetString("http.auth.user.id") - - log("handled request", - zap.String("user_id", userID), - zap.Duration("duration", duration), - zap.Int("size", wrec.Size()), - zap.Int("status", wrec.Status()), - zap.Object("resp_headers", LoggableHTTPHeader{ - Header: wrec.Header(), - ShouldLogCredentials: shouldLogCredentials, - }), - ) - }() + defer s.logRequest(accLog, r, wrec, &duration, repl, bodyReader, shouldLogCredentials) } start := time.Now() @@ -512,17 +554,7 @@ func (s *Server) findLastRouteWithHostMatcher() int { // not already done, and then uses that server to serve HTTP/3 over // the listener, with Server s as the handler. func (s *Server) serveHTTP3(addr caddy.NetworkAddress, tlsCfg *tls.Config) error { - switch addr.Network { - case "unix": - addr.Network = "unixgram" - case "tcp4": - addr.Network = "udp4" - case "tcp6": - addr.Network = "udp6" - default: - addr.Network = "udp" // TODO: Maybe a better default is to not enable HTTP/3 if we do not know the network? - } - + addr.Network = getHTTP3Network(addr.Network) lnAny, err := addr.Listen(s.ctx, 0, net.ListenConfig{}) if err != nil { return err @@ -547,7 +579,7 @@ func (s *Server) serveHTTP3(addr caddy.NetworkAddress, tlsCfg *tls.Config) error } } - s.h3listeners = append(s.h3listeners, lnAny.(net.PacketConn)) + s.h3listeners = append(s.h3listeners, ln) //nolint:errcheck go s.h3server.ServeListener(h3ln) @@ -666,6 +698,57 @@ func (s *Server) shouldLogRequest(r *http.Request) bool { return !s.Logs.SkipUnmappedHosts } +// logRequest logs the request to access logs, unless skipped. +func (s *Server) logRequest( + accLog *zap.Logger, r *http.Request, wrec ResponseRecorder, duration *time.Duration, + repl *caddy.Replacer, bodyReader *lengthReader, shouldLogCredentials bool, +) { + // this request may be flagged as omitted from the logs + if skipLog, ok := GetVar(r.Context(), SkipLogVar).(bool); ok && skipLog { + return + } + + repl.Set("http.response.status", wrec.Status()) // will be 0 if no response is written by us (Go will write 200 to client) + repl.Set("http.response.size", wrec.Size()) + repl.Set("http.response.duration", duration) + repl.Set("http.response.duration_ms", duration.Seconds()*1e3) // multiply seconds to preserve decimal (see #4666) + + logger := accLog + if s.Logs != nil { + logger = s.Logs.wrapLogger(logger, r.Host) + } + + log := logger.Info + if wrec.Status() >= 400 { + log = logger.Error + } + + userID, _ := repl.GetString("http.auth.user.id") + + reqBodyLength := 0 + if bodyReader != nil { + reqBodyLength = bodyReader.Length + } + + extra := r.Context().Value(ExtraLogFieldsCtxKey).(*ExtraLogFields) + + fieldCount := 6 + fields := make([]zapcore.Field, 0, fieldCount+len(extra.fields)) + fields = append(fields, + zap.Int("bytes_read", reqBodyLength), + zap.String("user_id", userID), + zap.Duration("duration", *duration), + zap.Int("size", wrec.Size()), + zap.Int("status", wrec.Status()), + zap.Object("resp_headers", LoggableHTTPHeader{ + Header: wrec.Header(), + ShouldLogCredentials: shouldLogCredentials, + })) + fields = append(fields, extra.fields...) + + log("handled request", fields...) +} + // protocol returns true if the protocol proto is configured/enabled. func (s *Server) protocol(proto string) bool { for _, p := range s.Protocols { @@ -684,18 +767,29 @@ func (s *Server) protocol(proto string) bool { // EXPERIMENTAL: Subject to change or removal. func (s *Server) Listeners() []net.Listener { return s.listeners } +// Name returns the server's name. +func (s *Server) Name() string { return s.name } + // PrepareRequest fills the request r for use in a Caddy HTTP handler chain. w and s can // be nil, but the handlers will lose response placeholders and access to the server. func PrepareRequest(r *http.Request, repl *caddy.Replacer, w http.ResponseWriter, s *Server) *http.Request { // set up the context for the request ctx := context.WithValue(r.Context(), caddy.ReplacerCtxKey, repl) ctx = context.WithValue(ctx, ServerCtxKey, s) + + trusted, clientIP := determineTrustedProxy(r, s) ctx = context.WithValue(ctx, VarsCtxKey, map[string]any{ - TrustedProxyVarKey: determineTrustedProxy(r, s), + TrustedProxyVarKey: trusted, + ClientIPVarKey: clientIP, }) + ctx = context.WithValue(ctx, routeGroupCtxKey, make(map[string]struct{})) + var url2 url.URL // avoid letting this escape to the heap ctx = context.WithValue(ctx, OriginalRequestCtxKey, originalRequest(r, &url2)) + + ctx = context.WithValue(ctx, ExtraLogFieldsCtxKey, new(ExtraLogFields)) + r = r.WithContext(ctx) // once the pointer to the request won't change @@ -724,11 +818,12 @@ func originalRequest(req *http.Request, urlCopy *url.URL) http.Request { // determineTrustedProxy parses the remote IP address of // the request, and determines (if the server configured it) -// if the client is a trusted proxy. -func determineTrustedProxy(r *http.Request, s *Server) bool { +// if the client is a trusted proxy. If trusted, also returns +// the real client IP if possible. +func determineTrustedProxy(r *http.Request, s *Server) (bool, string) { // If there's no server, then we can't check anything if s == nil { - return false + return false, "" } // Parse the remote IP, ignore the error as non-fatal, @@ -738,7 +833,7 @@ func determineTrustedProxy(r *http.Request, s *Server) bool { // remote address and used an invalid value. clientIP, _, err := net.SplitHostPort(r.RemoteAddr) if err != nil { - return false + return false, "" } // Client IP may contain a zone if IPv6, so we need @@ -746,20 +841,56 @@ func determineTrustedProxy(r *http.Request, s *Server) bool { clientIP, _, _ = strings.Cut(clientIP, "%") ipAddr, err := netip.ParseAddr(clientIP) if err != nil { - return false + return false, "" } // Check if the client is a trusted proxy if s.trustedProxies == nil { - return false + return false, ipAddr.String() } for _, ipRange := range s.trustedProxies.GetIPRanges(r) { if ipRange.Contains(ipAddr) { - return true + // We trust the proxy, so let's try to + // determine the real client IP + return true, trustedRealClientIP(r, s.ClientIPHeaders, ipAddr.String()) } } - return false + return false, ipAddr.String() +} + +// trustedRealClientIP finds the client IP from the request assuming it is +// from a trusted client. If there is no client IP headers, then the +// direct remote address is returned. If there are client IP headers, +// then the first value from those headers is used. +func trustedRealClientIP(r *http.Request, headers []string, clientIP string) string { + // Read all the values of the configured client IP headers, in order + var values []string + for _, field := range headers { + values = append(values, r.Header.Values(field)...) + } + + // If we don't have any values, then give up + if len(values) == 0 { + return clientIP + } + + // Since there can be many header values, we need to + // join them together before splitting to get the full list + allValues := strings.Split(strings.Join(values, ","), ",") + + // Get first valid left-most IP address + for _, ip := range allValues { + ip, _, _ = strings.Cut(strings.TrimSpace(ip), "%") + ipAddr, err := netip.ParseAddr(ip) + if err != nil { + continue + } + return ipAddr.String() + } + + // We didn't find a valid IP + return clientIP } // cloneURL makes a copy of r.URL and returns a @@ -773,6 +904,23 @@ func cloneURL(from, to *url.URL) { } } +// lengthReader is an io.ReadCloser that keeps track of the +// number of bytes read from the request body. +type lengthReader struct { + Source io.ReadCloser + Length int +} + +func (r *lengthReader) Read(b []byte) (int, error) { + n, err := r.Source.Read(b) + r.Length += n + return n, err +} + +func (r *lengthReader) Close() error { + return r.Source.Close() +} + // Context keys for HTTP request context values. const ( // For referencing the server instance @@ -785,6 +933,39 @@ const ( // originally came into the server's entry handler OriginalRequestCtxKey caddy.CtxKey = "original_request" + // For referencing underlying net.Conn + ConnCtxKey caddy.CtxKey = "conn" + // For tracking whether the client is a trusted proxy TrustedProxyVarKey string = "trusted_proxy" + + // For tracking the real client IP (affected by trusted_proxy) + ClientIPVarKey string = "client_ip" ) + +var networkTypesHTTP3 = map[string]string{ + "unix": "unixgram", + "tcp4": "udp4", + "tcp6": "udp6", +} + +// RegisterNetworkHTTP3 registers a mapping from non-HTTP/3 network to HTTP/3 +// network. This should be called during init() and will panic if the network +// type is standard, reserved, or already registered. +// +// EXPERIMENTAL: Subject to change. +func RegisterNetworkHTTP3(originalNetwork, h3Network string) { + if _, ok := networkTypesHTTP3[strings.ToLower(originalNetwork)]; ok { + panic("network type " + originalNetwork + " is already registered") + } + networkTypesHTTP3[originalNetwork] = h3Network +} + +func getHTTP3Network(originalNetwork string) string { + h3Network, ok := networkTypesHTTP3[strings.ToLower(originalNetwork)] + if !ok { + // TODO: Maybe a better default is to not enable HTTP/3 if we do not know the network? + return "udp" + } + return h3Network +} diff --git a/modules/caddyhttp/server_test.go b/modules/caddyhttp/server_test.go new file mode 100644 index 0000000..96a241b --- /dev/null +++ b/modules/caddyhttp/server_test.go @@ -0,0 +1,146 @@ +package caddyhttp + +import ( + "bytes" + "context" + "io" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zapcore" +) + +type writeFunc func(p []byte) (int, error) + +type nopSyncer writeFunc + +func (n nopSyncer) Write(p []byte) (int, error) { + return n(p) +} + +func (n nopSyncer) Sync() error { + return nil +} + +// testLogger returns a logger and a buffer to which the logger writes. The +// buffer can be read for asserting log output. +func testLogger(wf writeFunc) *zap.Logger { + ws := nopSyncer(wf) + encoderCfg := zapcore.EncoderConfig{ + MessageKey: "msg", + LevelKey: "level", + NameKey: "logger", + EncodeLevel: zapcore.LowercaseLevelEncoder, + EncodeTime: zapcore.ISO8601TimeEncoder, + EncodeDuration: zapcore.StringDurationEncoder, + } + core := zapcore.NewCore(zapcore.NewJSONEncoder(encoderCfg), ws, zap.DebugLevel) + + return zap.New(core) +} + +func TestServer_LogRequest(t *testing.T) { + s := &Server{} + + ctx := context.Background() + ctx = context.WithValue(ctx, ExtraLogFieldsCtxKey, new(ExtraLogFields)) + req := httptest.NewRequest(http.MethodGet, "/", nil).WithContext(ctx) + rec := httptest.NewRecorder() + wrec := NewResponseRecorder(rec, nil, nil) + + duration := 50 * time.Millisecond + repl := NewTestReplacer(req) + bodyReader := &lengthReader{Source: req.Body} + shouldLogCredentials := false + + buf := bytes.Buffer{} + accLog := testLogger(buf.Write) + s.logRequest(accLog, req, wrec, &duration, repl, bodyReader, shouldLogCredentials) + + assert.JSONEq(t, `{ + "msg":"handled request", "level":"info", "bytes_read":0, + "duration":"50ms", "resp_headers": {}, "size":0, + "status":0, "user_id":"" + }`, buf.String()) +} + +func TestServer_LogRequest_WithTraceID(t *testing.T) { + s := &Server{} + + extra := new(ExtraLogFields) + ctx := context.WithValue(context.Background(), ExtraLogFieldsCtxKey, extra) + extra.Add(zap.String("traceID", "1234567890abcdef")) + + req := httptest.NewRequest(http.MethodGet, "/", nil).WithContext(ctx) + rec := httptest.NewRecorder() + wrec := NewResponseRecorder(rec, nil, nil) + + duration := 50 * time.Millisecond + repl := NewTestReplacer(req) + bodyReader := &lengthReader{Source: req.Body} + shouldLogCredentials := false + + buf := bytes.Buffer{} + accLog := testLogger(buf.Write) + s.logRequest(accLog, req, wrec, &duration, repl, bodyReader, shouldLogCredentials) + + assert.JSONEq(t, `{ + "msg":"handled request", "level":"info", "bytes_read":0, + "duration":"50ms", "resp_headers": {}, "size":0, + "status":0, "user_id":"", + "traceID":"1234567890abcdef" + }`, buf.String()) +} + +func BenchmarkServer_LogRequest(b *testing.B) { + s := &Server{} + + extra := new(ExtraLogFields) + ctx := context.WithValue(context.Background(), ExtraLogFieldsCtxKey, extra) + + req := httptest.NewRequest(http.MethodGet, "/", nil).WithContext(ctx) + rec := httptest.NewRecorder() + wrec := NewResponseRecorder(rec, nil, nil) + + duration := 50 * time.Millisecond + repl := NewTestReplacer(req) + bodyReader := &lengthReader{Source: req.Body} + + buf := io.Discard + accLog := testLogger(buf.Write) + + b.ResetTimer() + + for i := 0; i < b.N; i++ { + s.logRequest(accLog, req, wrec, &duration, repl, bodyReader, false) + } +} + +func BenchmarkServer_LogRequest_WithTraceID(b *testing.B) { + s := &Server{} + + extra := new(ExtraLogFields) + ctx := context.WithValue(context.Background(), ExtraLogFieldsCtxKey, extra) + extra.Add(zap.String("traceID", "1234567890abcdef")) + + req := httptest.NewRequest(http.MethodGet, "/", nil).WithContext(ctx) + rec := httptest.NewRecorder() + wrec := NewResponseRecorder(rec, nil, nil) + + duration := 50 * time.Millisecond + repl := NewTestReplacer(req) + bodyReader := &lengthReader{Source: req.Body} + + buf := io.Discard + accLog := testLogger(buf.Write) + + b.ResetTimer() + + for i := 0; i < b.N; i++ { + s.logRequest(accLog, req, wrec, &duration, repl, bodyReader, false) + } +} diff --git a/modules/caddyhttp/standard/imports.go b/modules/caddyhttp/standard/imports.go index 435569d..d7bb280 100644 --- a/modules/caddyhttp/standard/imports.go +++ b/modules/caddyhttp/standard/imports.go @@ -11,6 +11,7 @@ import ( _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/fileserver" _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers" _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/map" + _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/proxyprotocol" _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/push" _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/requestbody" _ "github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy" diff --git a/modules/caddyhttp/staticresp.go b/modules/caddyhttp/staticresp.go index add5b12..4fe5910 100644 --- a/modules/caddyhttp/staticresp.go +++ b/modules/caddyhttp/staticresp.go @@ -17,7 +17,6 @@ package caddyhttp import ( "bytes" "encoding/json" - "flag" "fmt" "io" "net/http" @@ -28,18 +27,20 @@ import ( "text/template" "time" + "github.com/spf13/cobra" + "go.uber.org/zap" + + caddycmd "github.com/caddyserver/caddy/v2/cmd" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" - caddycmd "github.com/caddyserver/caddy/v2/cmd" - "go.uber.org/zap" ) func init() { caddy.RegisterModule(StaticResponse{}) caddycmd.RegisterCommand(caddycmd.Command{ Name: "respond", - Func: cmdRespond, Usage: `[--status <code>] [--body <content>] [--listen <addr>] [--access-log] [--debug] [--header "Field: value"] <body|status>`, Short: "Simple, hard-coded HTTP responses for development and testing", Long: ` @@ -71,16 +72,15 @@ Access/request logging and more verbose debug logging can also be enabled. Response headers may be added using the --header flag for each header field. `, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("respond", flag.ExitOnError) - fs.String("listen", ":0", "The address to which to bind the listener") - fs.Int("status", http.StatusOK, "The response status code") - fs.String("body", "", "The body of the HTTP response") - fs.Bool("access-log", false, "Enable the access log") - fs.Bool("debug", false, "Enable more verbose debug-level logging") - fs.Var(&respondCmdHeaders, "header", "Set a header on the response (format: \"Field: value\"") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("listen", "l", ":0", "The address to which to bind the listener") + cmd.Flags().IntP("status", "s", http.StatusOK, "The response status code") + cmd.Flags().StringP("body", "b", "", "The body of the HTTP response") + cmd.Flags().BoolP("access-log", "", false, "Enable the access log") + cmd.Flags().BoolP("debug", "v", false, "Enable more verbose debug-level logging") + cmd.Flags().StringSliceP("header", "H", []string{}, "Set a header on the response (format: \"Field: value\")") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdRespond) + }, }) } @@ -318,8 +318,12 @@ func cmdRespond(fl caddycmd.Flags) (int, error) { } // build headers map + headers, err := fl.GetStringSlice("header") + if err != nil { + return caddy.ExitCodeFailedStartup, fmt.Errorf("invalid header flag: %v", err) + } hdr := make(http.Header) - for i, h := range respondCmdHeaders { + for i, h := range headers { key, val, found := strings.Cut(h, ":") key, val = strings.TrimSpace(key), strings.TrimSpace(val) if !found || key == "" || val == "" { @@ -405,7 +409,7 @@ func cmdRespond(fl caddycmd.Flags) (int, error) { if debug { cfg.Logging = &caddy.Logging{ Logs: map[string]*caddy.CustomLog{ - "default": {Level: zap.DebugLevel.CapitalString()}, + "default": {BaseLog: caddy.BaseLog{Level: zap.DebugLevel.CapitalString()}}, }, } } @@ -432,9 +436,6 @@ func cmdRespond(fl caddycmd.Flags) (int, error) { select {} } -// respondCmdHeaders holds the parsed values from repeated use of the --header flag. -var respondCmdHeaders caddycmd.StringSlice - // Interface guards var ( _ MiddlewareHandler = (*StaticResponse)(nil) diff --git a/modules/caddyhttp/templates/templates.go b/modules/caddyhttp/templates/templates.go index 449536a..65359d9 100644 --- a/modules/caddyhttp/templates/templates.go +++ b/modules/caddyhttp/templates/templates.go @@ -88,7 +88,11 @@ func init() { // // ##### `httpInclude` // -// Includes the contents of another file by making a virtual HTTP request (also known as a sub-request). The URI path must exist on the same virtual server because the request does not use sockets; instead, the request is crafted in memory and the handler is invoked directly for increased efficiency. +// Includes the contents of another file, and renders it in-place, +// by making a virtual HTTP request (also known as a sub-request). +// The URI path must exist on the same virtual server because the +// request does not use sockets; instead, the request is crafted in +// memory and the handler is invoked directly for increased efficiency. // // ``` // {{httpInclude "/foo/bar?q=val"}} @@ -96,7 +100,13 @@ func init() { // // ##### `import` // -// Imports the contents of another file and adds any template definitions to the template stack. If there are no defitions, the filepath will be the defition name. Any {{ define }} blocks will be accessible by {{ template }} or {{ block }}. Imports must happen before the template or block action is called +// Reads and returns the contents of another file, and parses it +// as a template, adding any template definitions to the template +// stack. If there are no definitions, the filepath will be the +// definition name. Any {{ define }} blocks will be accessible by +// {{ template }} or {{ block }}. Imports must happen before the +// template or block action is called. Note that the contents are +// NOT escaped, so you should only import trusted template files. // // **filename.html** // ``` @@ -113,13 +123,26 @@ func init() { // // ##### `include` // -// Includes the contents of another file and renders in-place. Optionally can pass key-value pairs as arguments to be accessed by the included file. +// Includes the contents of another file, rendering it in-place. +// Optionally can pass key-value pairs as arguments to be accessed +// by the included file. Note that the contents are NOT escaped, +// so you should only include trusted template files. // // ``` // {{include "path/to/file.html"}} // no arguments // {{include "path/to/file.html" "arg1" 2 "value 3"}} // with arguments // ``` // +// ##### `readFile` +// +// Reads and returns the contents of another file, as-is. +// Note that the contents are NOT escaped, so you should +// only read trusted files. +// +// ``` +// {{readFile "path/to/file.html"}} +// ``` +// // ##### `listFiles` // // Returns a list of the files in the given directory, which is relative to the template context's file root. @@ -130,10 +153,10 @@ func init() { // // ##### `markdown` // -// Renders the given Markdown text as HTML. This uses the +// Renders the given Markdown text as HTML and returns it. This uses the // [Goldmark](https://github.com/yuin/goldmark) library, -// which is CommonMark compliant. It also has these plugins -// enabled: Github Flavored Markdown, Footnote and syntax +// which is CommonMark compliant. It also has these extensions +// enabled: Github Flavored Markdown, Footnote, and syntax // highlighting provided by [Chroma](https://github.com/alecthomas/chroma). // // ``` diff --git a/modules/caddyhttp/templates/tplcontext.go b/modules/caddyhttp/templates/tplcontext.go index ddad24f..a7d5314 100644 --- a/modules/caddyhttp/templates/tplcontext.go +++ b/modules/caddyhttp/templates/tplcontext.go @@ -18,6 +18,7 @@ import ( "bytes" "fmt" "io" + "io/fs" "net" "net/http" "os" @@ -29,15 +30,16 @@ import ( "time" "github.com/Masterminds/sprig/v3" - "github.com/alecthomas/chroma/v2/formatters/html" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyhttp" + chromahtml "github.com/alecthomas/chroma/v2/formatters/html" "github.com/dustin/go-humanize" "github.com/yuin/goldmark" highlighting "github.com/yuin/goldmark-highlighting/v2" "github.com/yuin/goldmark/extension" "github.com/yuin/goldmark/parser" gmhtml "github.com/yuin/goldmark/renderer/html" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) // TemplateContext is the TemplateContext with which HTTP templates are executed. @@ -73,12 +75,14 @@ func (c *TemplateContext) NewTemplate(tplName string) *template.Template { // add our own library c.tpl.Funcs(template.FuncMap{ "include": c.funcInclude, + "readFile": c.funcReadFile, "import": c.funcImport, "httpInclude": c.funcHTTPInclude, "stripHTML": c.funcStripHTML, "markdown": c.funcMarkdown, "splitFrontMatter": c.funcSplitFrontMatter, "listFiles": c.funcListFiles, + "fileStat": c.funcFileStat, "env": c.funcEnv, "placeholder": c.funcPlaceholder, "fileExists": c.funcFileExists, @@ -100,13 +104,11 @@ func (c TemplateContext) OriginalReq() http.Request { // trusted files. If it is not trusted, be sure to use escaping functions // in your template. func (c TemplateContext) funcInclude(filename string, args ...any) (string, error) { - bodyBuf := bufPool.Get().(*bytes.Buffer) bodyBuf.Reset() defer bufPool.Put(bodyBuf) err := c.readFileToBuffer(filename, bodyBuf) - if err != nil { return "", err } @@ -121,6 +123,23 @@ func (c TemplateContext) funcInclude(filename string, args ...any) (string, erro return bodyBuf.String(), nil } +// funcReadFile returns the contents of a filename relative to the site root. +// Note that included files are NOT escaped, so you should only include +// trusted files. If it is not trusted, be sure to use escaping functions +// in your template. +func (c TemplateContext) funcReadFile(filename string) (string, error) { + bodyBuf := bufPool.Get().(*bytes.Buffer) + bodyBuf.Reset() + defer bufPool.Put(bodyBuf) + + err := c.readFileToBuffer(filename, bodyBuf) + if err != nil { + return "", err + } + + return bodyBuf.String(), nil +} + // readFileToBuffer reads a file into a buffer func (c TemplateContext) readFileToBuffer(filename string, bodyBuf *bytes.Buffer) error { if c.Root == nil { @@ -169,6 +188,7 @@ func (c TemplateContext) funcHTTPInclude(uri string) (string, error) { return "", err } virtReq.Host = c.Req.Host + virtReq.RemoteAddr = "127.0.0.1:10000" // https://github.com/caddyserver/caddy/issues/5835 virtReq.Header = c.Req.Header.Clone() virtReq.Header.Set("Accept-Encoding", "identity") // https://github.com/caddyserver/caddy/issues/4352 virtReq.Trailer = c.Req.Trailer.Clone() @@ -195,7 +215,6 @@ func (c TemplateContext) funcHTTPInclude(uri string) (string, error) { // {{ template }} from the standard template library. If the imported file has // no {{ define }} blocks, the name of the import will be the path func (c *TemplateContext) funcImport(filename string) (string, error) { - bodyBuf := bufPool.Get().(*bytes.Buffer) bodyBuf.Reset() defer bufPool.Put(bodyBuf) @@ -313,7 +332,7 @@ func (TemplateContext) funcMarkdown(input any) (string, error) { extension.Footnote, highlighting.NewHighlighting( highlighting.WithFormatOptions( - html.WithClasses(true), + chromahtml.WithClasses(true), ), ), ), @@ -395,6 +414,21 @@ func (c TemplateContext) funcFileExists(filename string) (bool, error) { return false, nil } +// funcFileStat returns Stat of a filename +func (c TemplateContext) funcFileStat(filename string) (fs.FileInfo, error) { + if c.Root == nil { + return nil, fmt.Errorf("root file system not specified") + } + + file, err := c.Root.Open(path.Clean(filename)) + if err != nil { + return nil, err + } + defer file.Close() + + return file.Stat() +} + // funcHTTPError returns a structured HTTP handler error. EXPERIMENTAL; SUBJECT TO CHANGE. // Example usage: `{{if not (fileExists $includeFile)}}{{httpError 404}}{{end}}` func (c TemplateContext) funcHTTPError(statusCode int) (bool, error) { diff --git a/modules/caddyhttp/templates/tplcontext_test.go b/modules/caddyhttp/templates/tplcontext_test.go index 15a369e..fdf2c10 100644 --- a/modules/caddyhttp/templates/tplcontext_test.go +++ b/modules/caddyhttp/templates/tplcontext_test.go @@ -18,7 +18,6 @@ import ( "bytes" "context" "fmt" - "io/ioutil" "net/http" "os" "path/filepath" @@ -221,21 +220,21 @@ func TestNestedInclude(t *testing.T) { // create files and for test case if test.parentFile != "" { absFilePath = filepath.Join(fmt.Sprintf("%s", context.Root), test.parentFile) - if err := ioutil.WriteFile(absFilePath, []byte(test.parent), os.ModePerm); err != nil { + if err := os.WriteFile(absFilePath, []byte(test.parent), os.ModePerm); err != nil { os.Remove(absFilePath) t.Fatalf("Test %d: Expected no error creating file, got: '%s'", i, err.Error()) } } if test.childFile != "" { absFilePath0 = filepath.Join(fmt.Sprintf("%s", context.Root), test.childFile) - if err := ioutil.WriteFile(absFilePath0, []byte(test.child), os.ModePerm); err != nil { + if err := os.WriteFile(absFilePath0, []byte(test.child), os.ModePerm); err != nil { os.Remove(absFilePath0) t.Fatalf("Test %d: Expected no error creating file, got: '%s'", i, err.Error()) } } if test.child2File != "" { absFilePath1 = filepath.Join(fmt.Sprintf("%s", context.Root), test.child2File) - if err := ioutil.WriteFile(absFilePath1, []byte(test.child2), os.ModePerm); err != nil { + if err := os.WriteFile(absFilePath1, []byte(test.child2), os.ModePerm); err != nil { os.Remove(absFilePath0) t.Fatalf("Test %d: Expected no error creating file, got: '%s'", i, err.Error()) } diff --git a/modules/caddyhttp/tracing/module.go b/modules/caddyhttp/tracing/module.go index e3eb84d..fd117c5 100644 --- a/modules/caddyhttp/tracing/module.go +++ b/modules/caddyhttp/tracing/module.go @@ -4,11 +4,12 @@ import ( "fmt" "net/http" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "go.uber.org/zap" ) func init() { diff --git a/modules/caddyhttp/tracing/tracer.go b/modules/caddyhttp/tracing/tracer.go index d113a56..ecd415f 100644 --- a/modules/caddyhttp/tracing/tracer.go +++ b/modules/caddyhttp/tracing/tracer.go @@ -5,16 +5,18 @@ import ( "fmt" "net/http" - "github.com/caddyserver/caddy/v2" - - "github.com/caddyserver/caddy/v2/modules/caddyhttp" "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" + "go.opentelemetry.io/contrib/propagators/autoprop" "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc" "go.opentelemetry.io/otel/propagation" "go.opentelemetry.io/otel/sdk/resource" sdktrace "go.opentelemetry.io/otel/sdk/trace" semconv "go.opentelemetry.io/otel/semconv/v1.17.0" + "go.opentelemetry.io/otel/trace" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" ) const ( @@ -62,7 +64,7 @@ func newOpenTelemetryWrapper( return ot, fmt.Errorf("creating trace exporter error: %w", err) } - ot.propagators = propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}) + ot.propagators = autoprop.NewTextMapPropagator() tracerProvider := globalTracerProvider.getTracerProvider( sdktrace.WithBatcher(traceExporter), @@ -81,8 +83,15 @@ func newOpenTelemetryWrapper( // serveHTTP injects a tracing context and call the next handler. func (ot *openTelemetryWrapper) serveHTTP(w http.ResponseWriter, r *http.Request) { - ot.propagators.Inject(r.Context(), propagation.HeaderCarrier(r.Header)) - next := r.Context().Value(nextCallCtxKey).(*nextCall) + ctx := r.Context() + ot.propagators.Inject(ctx, propagation.HeaderCarrier(r.Header)) + spanCtx := trace.SpanContextFromContext(ctx) + if spanCtx.IsValid() { + if extra, ok := ctx.Value(caddyhttp.ExtraLogFieldsCtxKey).(*caddyhttp.ExtraLogFields); ok { + extra.Add(zap.String("traceID", spanCtx.TraceID().String())) + } + } + next := ctx.Value(nextCallCtxKey).(*nextCall) next.err = next.next.ServeHTTP(w, r) } diff --git a/modules/caddyhttp/vars.go b/modules/caddyhttp/vars.go index b4e1d89..d2d7f62 100644 --- a/modules/caddyhttp/vars.go +++ b/modules/caddyhttp/vars.go @@ -255,9 +255,18 @@ func (m MatchVarsRE) Provision(ctx caddy.Context) error { func (m MatchVarsRE) Match(r *http.Request) bool { vars := r.Context().Value(VarsCtxKey).(map[string]any) repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer) - for k, rm := range m { + for key, val := range m { + var varValue any + if strings.HasPrefix(key, "{") && + strings.HasSuffix(key, "}") && + strings.Count(key, "{") == 1 { + varValue, _ = repl.Get(strings.Trim(key, "{}")) + } else { + varValue = vars[key] + } + var varStr string - switch vv := vars[k].(type) { + switch vv := varValue.(type) { case string: varStr = vv case fmt.Stringer: @@ -267,13 +276,9 @@ func (m MatchVarsRE) Match(r *http.Request) bool { default: varStr = fmt.Sprintf("%v", vv) } - valExpanded := repl.ReplaceAll(varStr, "") - if match := rm.Match(valExpanded, repl); match { - return match - } - replacedVal := repl.ReplaceAll(k, "") - if match := rm.Match(replacedVal, repl); match { + valExpanded := repl.ReplaceAll(varStr, "") + if match := val.Match(valExpanded, repl); match { return match } } diff --git a/modules/caddypki/acmeserver/acmeserver.go b/modules/caddypki/acmeserver/acmeserver.go index 6ecdfdc..8689615 100644 --- a/modules/caddypki/acmeserver/acmeserver.go +++ b/modules/caddypki/acmeserver/acmeserver.go @@ -15,7 +15,10 @@ package acmeserver import ( + "context" "fmt" + weakrand "math/rand" + "net" "net/http" "os" "path/filepath" @@ -23,18 +26,19 @@ import ( "strings" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "github.com/caddyserver/caddy/v2/modules/caddypki" "github.com/go-chi/chi" "github.com/smallstep/certificates/acme" - acmeAPI "github.com/smallstep/certificates/acme/api" + "github.com/smallstep/certificates/acme/api" acmeNoSQL "github.com/smallstep/certificates/acme/db/nosql" "github.com/smallstep/certificates/authority" "github.com/smallstep/certificates/authority/provisioner" "github.com/smallstep/certificates/db" "github.com/smallstep/nosql" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyhttp" + "github.com/caddyserver/caddy/v2/modules/caddypki" ) func init() { @@ -76,8 +80,26 @@ type Handler struct { // changed or removed in the future. SignWithRoot bool `json:"sign_with_root,omitempty"` + // The addresses of DNS resolvers to use when looking up + // the TXT records for solving DNS challenges. + // It accepts [network addresses](/docs/conventions#network-addresses) + // with port range of only 1. If the host is an IP address, + // it will be dialed directly to resolve the upstream server. + // If the host is not an IP address, the addresses are resolved + // using the [name resolution convention](https://golang.org/pkg/net/#hdr-Name_Resolution) + // of the Go standard library. If the array contains more + // than 1 resolver address, one is chosen at random. + Resolvers []string `json:"resolvers,omitempty"` + + logger *zap.Logger + resolvers []caddy.NetworkAddress + ctx caddy.Context + + acmeDB acme.DB + acmeAuth *authority.Authority + acmeClient acme.Client + acmeLinker acme.Linker acmeEndpoints http.Handler - logger *zap.Logger } // CaddyModule returns the Caddy module information. @@ -90,7 +112,9 @@ func (Handler) CaddyModule() caddy.ModuleInfo { // Provision sets up the ACME server handler. func (ash *Handler) Provision(ctx caddy.Context) error { + ash.ctx = ctx ash.logger = ctx.Logger() + // set some defaults if ash.CA == "" { ash.CA = caddypki.DefaultCAID @@ -142,31 +166,30 @@ func (ash *Handler) Provision(ctx caddy.Context) error { DB: database, } - auth, err := ca.NewAuthority(authorityConfig) + ash.acmeAuth, err = ca.NewAuthority(authorityConfig) if err != nil { return err } - var acmeDB acme.DB - if authorityConfig.DB != nil { - acmeDB, err = acmeNoSQL.New(auth.GetDatabase().(nosql.DB)) - if err != nil { - return fmt.Errorf("configuring ACME DB: %v", err) - } + ash.acmeDB, err = acmeNoSQL.New(ash.acmeAuth.GetDatabase().(nosql.DB)) + if err != nil { + return fmt.Errorf("configuring ACME DB: %v", err) } - // create the router for the ACME endpoints - acmeRouterHandler := acmeAPI.NewHandler(acmeAPI.HandlerOptions{ - CA: auth, - DB: acmeDB, // stores all the server state - DNS: ash.Host, // used for directory links - Prefix: strings.Trim(ash.PathPrefix, "/"), // used for directory links - }) + ash.acmeClient, err = ash.makeClient() + if err != nil { + return err + } + + ash.acmeLinker = acme.NewLinker( + ash.Host, + strings.Trim(ash.PathPrefix, "/"), + ) // extract its http.Handler so we can use it directly r := chi.NewRouter() r.Route(ash.PathPrefix, func(r chi.Router) { - acmeRouterHandler.Route(r) + api.Route(r) }) ash.acmeEndpoints = r @@ -175,6 +198,16 @@ func (ash *Handler) Provision(ctx caddy.Context) error { func (ash Handler) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error { if strings.HasPrefix(r.URL.Path, ash.PathPrefix) { + acmeCtx := acme.NewContext( + r.Context(), + ash.acmeDB, + ash.acmeClient, + ash.acmeLinker, + nil, + ) + acmeCtx = authority.NewContext(acmeCtx, ash.acmeAuth) + r = r.WithContext(acmeCtx) + ash.acmeEndpoints.ServeHTTP(w, r) return nil } @@ -207,7 +240,7 @@ func (ash Handler) openDatabase() (*db.AuthDB, error) { dbFolder := filepath.Join(caddy.AppDataDir(), "acme_server", key) dbPath := filepath.Join(dbFolder, "db") - err := os.MkdirAll(dbFolder, 0755) + err := os.MkdirAll(dbFolder, 0o755) if err != nil { return nil, fmt.Errorf("making folder for CA database: %v", err) } @@ -227,10 +260,61 @@ func (ash Handler) openDatabase() (*db.AuthDB, error) { return database.(databaseCloser).DB, err } +// makeClient creates an ACME client which will use a custom +// resolver instead of net.DefaultResolver. +func (ash Handler) makeClient() (acme.Client, error) { + for _, v := range ash.Resolvers { + addr, err := caddy.ParseNetworkAddressWithDefaults(v, "udp", 53) + if err != nil { + return nil, err + } + if addr.PortRangeSize() != 1 { + return nil, fmt.Errorf("resolver address must have exactly one address; cannot call %v", addr) + } + ash.resolvers = append(ash.resolvers, addr) + } + + var resolver *net.Resolver + if len(ash.resolvers) != 0 { + dialer := &net.Dialer{ + Timeout: 2 * time.Second, + } + resolver = &net.Resolver{ + PreferGo: true, + Dial: func(ctx context.Context, network, address string) (net.Conn, error) { + //nolint:gosec + addr := ash.resolvers[weakrand.Intn(len(ash.resolvers))] + return dialer.DialContext(ctx, addr.Network, addr.JoinHostPort(0)) + }, + } + } else { + resolver = net.DefaultResolver + } + + return resolverClient{ + Client: acme.NewClient(), + resolver: resolver, + ctx: ash.ctx, + }, nil +} + +type resolverClient struct { + acme.Client + + resolver *net.Resolver + ctx context.Context +} + +func (c resolverClient) LookupTxt(name string) ([]string, error) { + return c.resolver.LookupTXT(c.ctx, name) +} + const defaultPathPrefix = "/acme/" -var keyCleaner = regexp.MustCompile(`[^\w.-_]`) -var databasePool = caddy.NewUsagePool() +var ( + keyCleaner = regexp.MustCompile(`[^\w.-_]`) + databasePool = caddy.NewUsagePool() +) type databaseCloser struct { DB *db.AuthDB diff --git a/modules/caddypki/acmeserver/caddyfile.go b/modules/caddypki/acmeserver/caddyfile.go index ae2d8ef..3b52113 100644 --- a/modules/caddypki/acmeserver/caddyfile.go +++ b/modules/caddypki/acmeserver/caddyfile.go @@ -29,8 +29,9 @@ func init() { // parseACMEServer sets up an ACME server handler from Caddyfile tokens. // // acme_server [<matcher>] { -// ca <id> -// lifetime <duration> +// ca <id> +// lifetime <duration> +// resolvers <addresses...> // } func parseACMEServer(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error) { if !h.Next() { @@ -74,6 +75,12 @@ func parseACMEServer(h httpcaddyfile.Helper) ([]httpcaddyfile.ConfigValue, error } acmeServer.Lifetime = caddy.Duration(dur) + + case "resolvers": + acmeServer.Resolvers = h.RemainingArgs() + if len(acmeServer.Resolvers) == 0 { + return nil, h.Errf("must specify at least one resolver address") + } } } } diff --git a/modules/caddypki/adminapi.go b/modules/caddypki/adminapi.go index f03c6b6..435af34 100644 --- a/modules/caddypki/adminapi.go +++ b/modules/caddypki/adminapi.go @@ -20,8 +20,9 @@ import ( "net/http" "strings" - "github.com/caddyserver/caddy/v2" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { @@ -49,20 +50,10 @@ func (a *adminAPI) Provision(ctx caddy.Context) error { a.ctx = ctx a.log = ctx.Logger(a) // TODO: passing in 'a' is a hack until the admin API is officially extensible (see #5032) - // First check if the PKI app was configured, because - // a.ctx.App() has the side effect of instantiating - // and provisioning an app even if it wasn't configured. - pkiAppConfigured := a.ctx.AppIsConfigured("pki") - if !pkiAppConfigured { - return nil - } - - // Load the PKI app, so we can query it for information. - appModule, err := a.ctx.App("pki") - if err != nil { - return err + // Avoid initializing PKI if it wasn't configured + if pkiApp := a.ctx.AppIfConfigured("pki"); pkiApp != nil { + a.pkiApp = pkiApp.(*PKI) } - a.pkiApp = appModule.(*PKI) return nil } diff --git a/modules/caddypki/ca.go b/modules/caddypki/ca.go index 1ba0890..6c48da6 100644 --- a/modules/caddypki/ca.go +++ b/modules/caddypki/ca.go @@ -25,12 +25,13 @@ import ( "sync" "time" - "github.com/caddyserver/caddy/v2" "github.com/caddyserver/certmagic" "github.com/smallstep/certificates/authority" "github.com/smallstep/certificates/db" "github.com/smallstep/truststore" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) // CA describes a certificate authority, which consists of @@ -376,15 +377,19 @@ func (ca CA) genIntermediate(rootCert *x509.Certificate, rootKey crypto.Signer) func (ca CA) storageKeyCAPrefix() string { return path.Join("pki", "authorities", certmagic.StorageKeys.Safe(ca.ID)) } + func (ca CA) storageKeyRootCert() string { return path.Join(ca.storageKeyCAPrefix(), "root.crt") } + func (ca CA) storageKeyRootKey() string { return path.Join(ca.storageKeyCAPrefix(), "root.key") } + func (ca CA) storageKeyIntermediateCert() string { return path.Join(ca.storageKeyCAPrefix(), "intermediate.crt") } + func (ca CA) storageKeyIntermediateKey() string { return path.Join(ca.storageKeyCAPrefix(), "intermediate.key") } diff --git a/modules/caddypki/command.go b/modules/caddypki/command.go index cb86c93..b7fa1bb 100644 --- a/modules/caddypki/command.go +++ b/modules/caddypki/command.go @@ -18,21 +18,22 @@ import ( "crypto/x509" "encoding/json" "encoding/pem" - "flag" "fmt" "net/http" "os" "path" - "github.com/caddyserver/caddy/v2" - caddycmd "github.com/caddyserver/caddy/v2/cmd" "github.com/smallstep/truststore" + "github.com/spf13/cobra" + + caddycmd "github.com/caddyserver/caddy/v2/cmd" + + "github.com/caddyserver/caddy/v2" ) func init() { caddycmd.RegisterCommand(caddycmd.Command{ Name: "trust", - Func: cmdTrust, Usage: "[--ca <id>] [--address <listen>] [--config <path> [--adapter <name>]]", Short: "Installs a CA certificate into local trust stores", Long: ` @@ -53,19 +54,17 @@ This command will attempt to connect to Caddy's admin API running at '` + caddy.DefaultAdminListen + `' to fetch the root certificate. You may explicitly specify the --address, or use the --config flag to load the admin address from your config, if not using the default.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("trust", flag.ExitOnError) - fs.String("ca", "", "The ID of the CA to trust (defaults to 'local')") - fs.String("address", "", "Address of the administration API listener (if --config is not used)") - fs.String("config", "", "Configuration file (if --address is not used)") - fs.String("adapter", "", "Name of config adapter to apply (if --config is used)") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("ca", "", "", "The ID of the CA to trust (defaults to 'local')") + cmd.Flags().StringP("address", "", "", "Address of the administration API listener (if --config is not used)") + cmd.Flags().StringP("config", "c", "", "Configuration file (if --address is not used)") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply (if --config is used)") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdTrust) + }, }) caddycmd.RegisterCommand(caddycmd.Command{ Name: "untrust", - Func: cmdUntrust, Usage: "[--cert <path>] | [[--ca <id>] [--address <listen>] [--config <path> [--adapter <name>]]]", Short: "Untrusts a locally-trusted CA certificate", Long: ` @@ -89,15 +88,14 @@ will attempt to connect to the Caddy's admin API running at '` + caddy.DefaultAdminListen + `' to fetch the root certificate. You may explicitly specify the --address, or use the --config flag to load the admin address from your config, if not using the default.`, - Flags: func() *flag.FlagSet { - fs := flag.NewFlagSet("untrust", flag.ExitOnError) - fs.String("cert", "", "The path to the CA certificate to untrust") - fs.String("ca", "", "The ID of the CA to untrust (defaults to 'local')") - fs.String("address", "", "Address of the administration API listener (if --config is not used)") - fs.String("config", "", "Configuration file (if --address is not used)") - fs.String("adapter", "", "Name of config adapter to apply (if --config is used)") - return fs - }(), + CobraFunc: func(cmd *cobra.Command) { + cmd.Flags().StringP("cert", "p", "", "The path to the CA certificate to untrust") + cmd.Flags().StringP("ca", "", "", "The ID of the CA to untrust (defaults to 'local')") + cmd.Flags().StringP("address", "", "", "Address of the administration API listener (if --config is not used)") + cmd.Flags().StringP("config", "c", "", "Configuration file (if --address is not used)") + cmd.Flags().StringP("adapter", "a", "", "Name of config adapter to apply (if --config is used)") + cmd.RunE = caddycmd.WrapCommandFuncForCobra(cmdUntrust) + }, }) } diff --git a/modules/caddypki/pki.go b/modules/caddypki/pki.go index 2caeb2b..9f974a9 100644 --- a/modules/caddypki/pki.go +++ b/modules/caddypki/pki.go @@ -17,8 +17,9 @@ package caddypki import ( "fmt" - "github.com/caddyserver/caddy/v2" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { diff --git a/modules/caddytls/acmeissuer.go b/modules/caddytls/acmeissuer.go index ca79981..5e79c2d 100644 --- a/modules/caddytls/acmeissuer.go +++ b/modules/caddytls/acmeissuer.go @@ -24,13 +24,14 @@ import ( "strconv" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/certmagic" "github.com/mholt/acmez" "github.com/mholt/acmez/acme" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { diff --git a/modules/caddytls/automation.go b/modules/caddytls/automation.go index 7f216d5..ee25400 100644 --- a/modules/caddytls/automation.go +++ b/modules/caddytls/automation.go @@ -19,12 +19,14 @@ import ( "errors" "fmt" "net/http" + "strings" "time" - "github.com/caddyserver/caddy/v2" "github.com/caddyserver/certmagic" "github.com/mholt/acmez" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) // AutomationConfig governs the automated management of TLS certificates. @@ -84,7 +86,14 @@ type AutomationConfig struct { // TLS app to properly provision a new policy. type AutomationPolicy struct { // Which subjects (hostnames or IP addresses) this policy applies to. - Subjects []string `json:"subjects,omitempty"` + // + // This list is a filter, not a command. In other words, it is used + // only to filter whether this policy should apply to a subject that + // needs a certificate; it does NOT command the TLS app to manage a + // certificate for that subject. To have Caddy automate a certificate + // or specific subjects, use the "automate" certificate loader module + // of the TLS app. + SubjectsRaw []string `json:"subjects,omitempty"` // The modules that may issue certificates. Default: internal if all // subjects do not qualify for public certificates; othewise acme and @@ -94,9 +103,11 @@ type AutomationPolicy struct { // Modules that can get a custom certificate to use for any // given TLS handshake at handshake-time. Custom certificates // can be useful if another entity is managing certificates - // and Caddy need only get it and serve it. + // and Caddy need only get it and serve it. Specifying a Manager + // enables on-demand TLS, i.e. it has the side-effect of setting + // the on_demand parameter to `true`. // - // TODO: This is an EXPERIMENTAL feature. It is subject to change or removal. + // TODO: This is an EXPERIMENTAL feature. Subject to change or removal. ManagersRaw []json.RawMessage `json:"get_certificate,omitempty" caddy:"namespace=tls.get_certificate inline_key=via"` // If true, certificates will be requested with MustStaple. Not all @@ -146,12 +157,21 @@ type AutomationPolicy struct { Issuers []certmagic.Issuer `json:"-"` Managers []certmagic.Manager `json:"-"` - magic *certmagic.Config - storage certmagic.Storage + subjects []string + magic *certmagic.Config + storage certmagic.Storage } // Provision sets up ap and builds its underlying CertMagic config. func (ap *AutomationPolicy) Provision(tlsApp *TLS) error { + // replace placeholders in subjects to allow environment variables + repl := caddy.NewReplacer() + subjects := make([]string, len(ap.SubjectsRaw)) + for i, sub := range ap.SubjectsRaw { + subjects[i] = repl.ReplaceAll(sub, "") + } + ap.subjects = subjects + // policy-specific storage implementation if ap.StorageRaw != nil { val, err := tlsApp.ctx.LoadModule(ap, "StorageRaw") @@ -165,36 +185,6 @@ func (ap *AutomationPolicy) Provision(tlsApp *TLS) error { ap.storage = cmStorage } - // on-demand TLS - var ond *certmagic.OnDemandConfig - if ap.OnDemand { - ond = &certmagic.OnDemandConfig{ - DecisionFunc: func(name string) error { - // if an "ask" endpoint was defined, consult it first - if tlsApp.Automation != nil && - tlsApp.Automation.OnDemand != nil && - tlsApp.Automation.OnDemand.Ask != "" { - if err := onDemandAskRequest(tlsApp.logger, tlsApp.Automation.OnDemand.Ask, name); err != nil { - // distinguish true errors from denials, because it's important to log actual errors - if !errors.Is(err, errAskDenied) { - tlsApp.logger.Error("request to 'ask' endpoint failed", - zap.Error(err), - zap.String("endpoint", tlsApp.Automation.OnDemand.Ask), - zap.String("domain", name)) - } - return err - } - } - // check the rate limiter last because - // doing so makes a reservation - if !onDemandRateLimiter.Allow() { - return fmt.Errorf("on-demand rate limit exceeded") - } - return nil - }, - } - } - // we don't store loaded modules directly in the certmagic config since // policy provisioning may happen more than once (during auto-HTTPS) and // loading a module clears its config bytes; thus, load the module and @@ -251,6 +241,46 @@ func (ap *AutomationPolicy) Provision(tlsApp *TLS) error { storage = tlsApp.ctx.Storage() } + // on-demand TLS + var ond *certmagic.OnDemandConfig + if ap.OnDemand || len(ap.Managers) > 0 { + // ask endpoint is now required after a number of negligence cases causing abuse; + // but is still allowed for explicit subjects (non-wildcard, non-unbounded), + // for the internal issuer since it doesn't cause ACME issuer pressure + if ap.isWildcardOrDefault() && !ap.onlyInternalIssuer() && (tlsApp.Automation == nil || tlsApp.Automation.OnDemand == nil || tlsApp.Automation.OnDemand.Ask == "") { + return fmt.Errorf("on-demand TLS cannot be enabled without an 'ask' endpoint to prevent abuse; please refer to documentation for details") + } + ond = &certmagic.OnDemandConfig{ + DecisionFunc: func(name string) error { + if tlsApp.Automation == nil || tlsApp.Automation.OnDemand == nil { + return nil + } + if err := onDemandAskRequest(tlsApp.logger, tlsApp.Automation.OnDemand.Ask, name); err != nil { + // distinguish true errors from denials, because it's important to elevate actual errors + if errors.Is(err, errAskDenied) { + tlsApp.logger.Debug("certificate issuance denied", + zap.String("ask_endpoint", tlsApp.Automation.OnDemand.Ask), + zap.String("domain", name), + zap.Error(err)) + } else { + tlsApp.logger.Error("request to 'ask' endpoint failed", + zap.String("ask_endpoint", tlsApp.Automation.OnDemand.Ask), + zap.String("domain", name), + zap.Error(err)) + } + return err + } + // check the rate limiter last because + // doing so makes a reservation + if !onDemandRateLimiter.Allow() { + return fmt.Errorf("on-demand rate limit exceeded") + } + return nil + }, + Managers: ap.Managers, + } + } + template := certmagic.Config{ MustStaple: ap.MustStaple, RenewalWindowRatio: ap.RenewalWindowRatio, @@ -261,12 +291,13 @@ func (ap *AutomationPolicy) Provision(tlsApp *TLS) error { DisableStapling: ap.DisableOCSPStapling, ResponderOverrides: ap.OCSPOverrides, }, - Storage: storage, - Issuers: issuers, - Managers: ap.Managers, - Logger: tlsApp.logger, + Storage: storage, + Issuers: issuers, + Logger: tlsApp.logger, } - ap.magic = certmagic.New(tlsApp.certCache, template) + certCacheMu.RLock() + ap.magic = certmagic.New(certCache, template) + certCacheMu.RUnlock() // sometimes issuers may need the parent certmagic.Config in // order to function properly (for example, ACMEIssuer needs @@ -282,6 +313,35 @@ func (ap *AutomationPolicy) Provision(tlsApp *TLS) error { return nil } +// Subjects returns the list of subjects with all placeholders replaced. +func (ap *AutomationPolicy) Subjects() []string { + return ap.subjects +} + +func (ap *AutomationPolicy) onlyInternalIssuer() bool { + if len(ap.Issuers) != 1 { + return false + } + _, ok := ap.Issuers[0].(*InternalIssuer) + return ok +} + +// isWildcardOrDefault determines if the subjects include any wildcard domains, +// or is the "default" policy (i.e. no subjects) which is unbounded. +func (ap *AutomationPolicy) isWildcardOrDefault() bool { + isWildcardOrDefault := false + if len(ap.subjects) == 0 { + isWildcardOrDefault = true + } + for _, sub := range ap.subjects { + if strings.HasPrefix(sub, "*") { + isWildcardOrDefault = true + break + } + } + return isWildcardOrDefault +} + // DefaultIssuers returns empty Issuers (not provisioned) to be used as defaults. // This function is experimental and has no compatibility promises. func DefaultIssuers() []certmagic.Issuer { @@ -400,29 +460,32 @@ type DNSChallengeConfig struct { // Caddy can "ask" if it should be allowed to manage // certificates for a given hostname. type OnDemandConfig struct { - // An optional rate limit to throttle the - // issuance of certificates from handshakes. - RateLimit *RateLimit `json:"rate_limit,omitempty"` - - // If Caddy needs to obtain or renew a certificate - // during a TLS handshake, it will perform a quick - // HTTP request to this URL to check if it should be - // allowed to try to get a certificate for the name - // in the "domain" query string parameter, like so: - // `?domain=example.com`. The endpoint must return a - // 200 OK status if a certificate is allowed; - // anything else will cause it to be denied. + // REQUIRED. If Caddy needs to load a certificate from + // storage or obtain/renew a certificate during a TLS + // handshake, it will perform a quick HTTP request to + // this URL to check if it should be allowed to try to + // get a certificate for the name in the "domain" query + // string parameter, like so: `?domain=example.com`. + // The endpoint must return a 200 OK status if a certificate + // is allowed; anything else will cause it to be denied. // Redirects are not followed. Ask string `json:"ask,omitempty"` + + // DEPRECATED. An optional rate limit to throttle + // the checking of storage and the issuance of + // certificates from handshakes if not already in + // storage. WILL BE REMOVED IN A FUTURE RELEASE. + RateLimit *RateLimit `json:"rate_limit,omitempty"` } -// RateLimit specifies an interval with optional burst size. +// DEPRECATED. RateLimit specifies an interval with optional burst size. type RateLimit struct { - // A duration value. A certificate may be obtained 'burst' - // times during this interval. + // A duration value. Storage may be checked and a certificate may be + // obtained 'burst' times during this interval. Interval caddy.Duration `json:"interval,omitempty"` - // How many times during an interval a certificate can be obtained. + // How many times during an interval storage can be checked or a + // certificate can be obtained. Burst int `json:"burst,omitempty"` } diff --git a/modules/caddytls/certmanagers.go b/modules/caddytls/certmanagers.go index 1b701ab..ad26468 100644 --- a/modules/caddytls/certmanagers.go +++ b/modules/caddytls/certmanagers.go @@ -9,11 +9,12 @@ import ( "net/url" "strings" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/certmagic" "github.com/tailscale/tscert" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { @@ -23,14 +24,6 @@ func init() { // Tailscale is a module that can get certificates from the local Tailscale process. type Tailscale struct { - // If true, this module will operate in "best-effort" mode and - // ignore "soft" errors; i.e. try Tailscale, and if it doesn't connect - // or return a certificate, oh well. Failure to connect to Tailscale - // results in a no-op instead of an error. Intended for the use case - // where this module is added implicitly for convenience, even if - // Tailscale isn't necessarily running. - Optional bool `json:"optional,omitempty"` - logger *zap.Logger } @@ -60,16 +53,11 @@ func (ts Tailscale) GetCertificate(ctx context.Context, hello *tls.ClientHelloIn // canHazCertificate returns true if Tailscale reports it can get a certificate for the given ClientHello. func (ts Tailscale) canHazCertificate(ctx context.Context, hello *tls.ClientHelloInfo) (bool, error) { - if ts.Optional && !strings.HasSuffix(strings.ToLower(hello.ServerName), tailscaleDomainAliasEnding) { + if !strings.HasSuffix(strings.ToLower(hello.ServerName), tailscaleDomainAliasEnding) { return false, nil } status, err := tscert.GetStatus(ctx) if err != nil { - if ts.Optional { - // ignore error if we don't expect/require it to work anyway, but log it for debugging - ts.logger.Debug("error getting tailscale status", zap.Error(err), zap.String("server_name", hello.ServerName)) - return false, nil - } return false, err } for _, domain := range status.CertDomains { diff --git a/modules/caddytls/certselection.go b/modules/caddytls/certselection.go index 0311f11..1bef890 100644 --- a/modules/caddytls/certselection.go +++ b/modules/caddytls/certselection.go @@ -58,7 +58,8 @@ nextChoice: if len(p.SerialNumber) > 0 { var found bool for _, sn := range p.SerialNumber { - if cert.Leaf.SerialNumber.Cmp(&sn.Int) == 0 { + snInt := sn.Int // avoid taking address of iteration variable (gosec warning) + if cert.Leaf.SerialNumber.Cmp(&snInt) == 0 { found = true break } diff --git a/modules/caddytls/cf.go b/modules/caddytls/cf.go new file mode 100644 index 0000000..e61a59c --- /dev/null +++ b/modules/caddytls/cf.go @@ -0,0 +1,24 @@ +//go:build cfgo + +package caddytls + +// This file adds support for X25519Kyber768Draft00, a post-quantum +// key agreement that is currently being rolled out by Chrome [1] +// and Cloudflare [2,3]. For more context, see the PR [4]. +// +// [1] https://blog.chromium.org/2023/08/protecting-chrome-traffic-with-hybrid.html +// [2] https://blog.cloudflare.com/post-quantum-for-all/ +// [3] https://blog.cloudflare.com/post-quantum-to-origins/ +// [4] https://github.com/caddyserver/caddy/pull/5852 + +import ( + "crypto/tls" +) + +func init() { + SupportedCurves["X25519Kyber768Draft00"] = tls.X25519Kyber768Draft00 + defaultCurves = append( + []tls.CurveID{tls.X25519Kyber768Draft00}, + defaultCurves..., + ) +} diff --git a/modules/caddytls/connpolicy.go b/modules/caddytls/connpolicy.go index bce69bc..64fdd51 100644 --- a/modules/caddytls/connpolicy.go +++ b/modules/caddytls/connpolicy.go @@ -25,9 +25,10 @@ import ( "path/filepath" "strings" - "github.com/caddyserver/caddy/v2" "github.com/mholt/acmez" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { @@ -54,7 +55,7 @@ func (cp ConnectionPolicies) Provision(ctx caddy.Context) error { } // enable HTTP/2 by default - if len(pol.ALPN) == 0 { + if pol.ALPN == nil { pol.ALPN = append(pol.ALPN, defaultALPN...) } @@ -159,6 +160,18 @@ type ConnectionPolicy struct { // is no policy configured for the empty SNI value. DefaultSNI string `json:"default_sni,omitempty"` + // FallbackSNI becomes the ServerName in a ClientHello if + // the original ServerName doesn't match any certificates + // in the cache. The use cases for this are very niche; + // typically if a client is a CDN and passes through the + // ServerName of the downstream handshake but can accept + // a certificate with the origin's hostname instead, then + // you would set this to your origin's hostname. Note that + // Caddy must be managing a certificate for this name. + // + // This feature is EXPERIMENTAL and subject to change or removal. + FallbackSNI string `json:"fallback_sni,omitempty"` + // Also known as "SSLKEYLOGFILE", TLS secrets will be written to // this file in NSS key log format which can then be parsed by // Wireshark and other tools. This is INSECURE as it allows other @@ -216,6 +229,7 @@ func (p *ConnectionPolicy) buildStandardTLSConfig(ctx caddy.Context) error { cfg.CertSelection = p.CertSelection } cfg.DefaultServerName = p.DefaultSNI + cfg.FallbackServerName = p.FallbackSNI return cfg.GetCertificate(hello) }, MinVersion: tls.VersionTLS12, @@ -270,7 +284,7 @@ func (p *ConnectionPolicy) buildStandardTLSConfig(ctx caddy.Context) error { break } } - if !alpnFound { + if !alpnFound && (cfg.NextProtos == nil || len(cfg.NextProtos) > 0) { cfg.NextProtos = append(cfg.NextProtos, acmez.ACMETLS1Protocol) } @@ -303,7 +317,7 @@ func (p *ConnectionPolicy) buildStandardTLSConfig(ctx caddy.Context) error { return err } logFile, _, err := secretsLogPool.LoadOrNew(filename, func() (caddy.Destructor, error) { - w, err := os.OpenFile(filename, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0600) + w, err := os.OpenFile(filename, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o600) return destructableWriter{w}, err }) if err != nil { diff --git a/modules/caddytls/distributedstek/distributedstek.go b/modules/caddytls/distributedstek/distributedstek.go index 18ed694..f6d0de0 100644 --- a/modules/caddytls/distributedstek/distributedstek.go +++ b/modules/caddytls/distributedstek/distributedstek.go @@ -33,9 +33,10 @@ import ( "runtime/debug" "time" + "github.com/caddyserver/certmagic" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/modules/caddytls" - "github.com/caddyserver/certmagic" ) func init() { diff --git a/modules/caddytls/internalissuer.go b/modules/caddytls/internalissuer.go index 3dd6c35..1cf2461 100644 --- a/modules/caddytls/internalissuer.go +++ b/modules/caddytls/internalissuer.go @@ -21,12 +21,13 @@ import ( "encoding/pem" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" - "github.com/caddyserver/caddy/v2/modules/caddypki" "github.com/caddyserver/certmagic" "github.com/smallstep/certificates/authority/provisioner" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" + "github.com/caddyserver/caddy/v2/modules/caddypki" ) func init() { diff --git a/modules/caddytls/matchers.go b/modules/caddytls/matchers.go index f541220..af1f898 100644 --- a/modules/caddytls/matchers.go +++ b/modules/caddytls/matchers.go @@ -21,9 +21,10 @@ import ( "net/netip" "strings" - "github.com/caddyserver/caddy/v2" "github.com/caddyserver/certmagic" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" ) func init() { diff --git a/modules/caddytls/storageloader.go b/modules/caddytls/storageloader.go index ef9d51e..ddaaa51 100644 --- a/modules/caddytls/storageloader.go +++ b/modules/caddytls/storageloader.go @@ -18,8 +18,9 @@ import ( "crypto/tls" "fmt" - "github.com/caddyserver/caddy/v2" "github.com/caddyserver/certmagic" + + "github.com/caddyserver/caddy/v2" ) func init() { diff --git a/modules/caddytls/tls.go b/modules/caddytls/tls.go index 8051653..02d5aae 100644 --- a/modules/caddytls/tls.go +++ b/modules/caddytls/tls.go @@ -25,10 +25,11 @@ import ( "sync" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/modules/caddyevents" "github.com/caddyserver/certmagic" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/modules/caddyevents" ) func init() { @@ -36,12 +37,24 @@ func init() { caddy.RegisterModule(AutomateLoader{}) } +var ( + certCache *certmagic.Cache + certCacheMu sync.RWMutex +) + // TLS provides TLS facilities including certificate // loading and management, client auth, and more. type TLS struct { - // Caches certificates in memory for quick use during + // Certificates to load into memory for quick recall during // TLS handshakes. Each key is the name of a certificate - // loader module. All loaded certificates get pooled + // loader module. + // + // The "automate" certificate loader module can be used to + // specify a list of subjects that need certificates to be + // managed automatically. The first matching automation + // policy will be applied to manage the certificate(s). + // + // All loaded certificates get pooled // into the same cache and may be used to complete TLS // handshakes for the relevant server names (SNI). // Certificates loaded manually (anything other than @@ -70,12 +83,15 @@ type TLS struct { certificateLoaders []CertificateLoader automateNames []string - certCache *certmagic.Cache ctx caddy.Context storageCleanTicker *time.Ticker storageCleanStop chan struct{} logger *zap.Logger events *caddyevents.App + + // set of subjects with managed certificates, + // and hashes of manually-loaded certificates + managing, loaded map[string]struct{} } // CaddyModule returns the Caddy module information. @@ -96,6 +112,7 @@ func (t *TLS) Provision(ctx caddy.Context) error { t.ctx = ctx t.logger = ctx.Logger() repl := caddy.NewReplacer() + t.managing, t.loaded = make(map[string]struct{}), make(map[string]struct{}) // set up a new certificate cache; this (re)loads all certificates cacheOpts := certmagic.CacheOptions{ @@ -114,7 +131,14 @@ func (t *TLS) Provision(ctx caddy.Context) error { if cacheOpts.Capacity <= 0 { cacheOpts.Capacity = 10000 } - t.certCache = certmagic.NewCache(cacheOpts) + + certCacheMu.Lock() + if certCache == nil { + certCache = certmagic.NewCache(cacheOpts) + } else { + certCache.SetOptions(cacheOpts) + } + certCacheMu.Unlock() // certificate loaders val, err := ctx.LoadModule(t, "CertificatesRaw") @@ -126,7 +150,12 @@ func (t *TLS) Provision(ctx caddy.Context) error { // special case; these will be loaded in later using our automation facilities, // which we want to avoid doing during provisioning if automateNames, ok := modIface.(*AutomateLoader); ok && automateNames != nil { - t.automateNames = []string(*automateNames) + repl := caddy.NewReplacer() + subjects := make([]string, len(*automateNames)) + for i, sub := range *automateNames { + subjects[i] = repl.ReplaceAll(sub, "") + } + t.automateNames = subjects } else { return fmt.Errorf("loading certificates with 'automate' requires array of strings, got: %T", modIface) } @@ -181,8 +210,8 @@ func (t *TLS) Provision(ctx caddy.Context) error { onDemandRateLimiter.SetWindow(time.Duration(t.Automation.OnDemand.RateLimit.Interval)) } else { // remove any existing rate limiter - onDemandRateLimiter.SetMaxEvents(0) onDemandRateLimiter.SetWindow(0) + onDemandRateLimiter.SetMaxEvents(0) } // run replacer on ask URL (for environment variables) -- return errors to prevent surprises (#5036) @@ -197,7 +226,8 @@ func (t *TLS) Provision(ctx caddy.Context) error { // provision so that other apps (such as http) can know which // certificates have been manually loaded, and also so that // commands like validate can be a better test - magic := certmagic.New(t.certCache, certmagic.Config{ + certCacheMu.RLock() + magic := certmagic.New(certCache, certmagic.Config{ Storage: ctx.Storage(), Logger: t.logger, OnEvent: t.onEvent, @@ -205,16 +235,18 @@ func (t *TLS) Provision(ctx caddy.Context) error { DisableStapling: t.DisableOCSPStapling, }, }) + certCacheMu.RUnlock() for _, loader := range t.certificateLoaders { certs, err := loader.LoadCertificates() if err != nil { return fmt.Errorf("loading certificates: %v", err) } for _, cert := range certs { - err := magic.CacheUnmanagedTLSCertificate(ctx, cert.Certificate, cert.Tags) + hash, err := magic.CacheUnmanagedTLSCertificate(ctx, cert.Certificate, cert.Tags) if err != nil { return fmt.Errorf("caching unmanaged certificate: %v", err) } + t.loaded[hash] = struct{}{} } } @@ -231,13 +263,13 @@ func (t *TLS) Validate() error { var hasDefault bool hostSet := make(map[string]int) for i, ap := range t.Automation.Policies { - if len(ap.Subjects) == 0 { + if len(ap.subjects) == 0 { if hasDefault { return fmt.Errorf("automation policy %d is the second policy that acts as default/catch-all, but will never be used", i) } hasDefault = true } - for _, h := range ap.Subjects { + for _, h := range ap.subjects { if first, ok := hostSet[h]; ok { return fmt.Errorf("automation policy %d: cannot apply more than one automation policy to host: %s (first match in policy %d)", i, h, first) } @@ -259,7 +291,7 @@ func (t *TLS) Start() error { if t.Automation.OnDemand == nil || (t.Automation.OnDemand.Ask == "" && t.Automation.OnDemand.RateLimit == nil) { for _, ap := range t.Automation.Policies { - if ap.OnDemand { + if ap.OnDemand && ap.isWildcardOrDefault() { t.logger.Warn("YOUR SERVER MAY BE VULNERABLE TO ABUSE: on-demand TLS is enabled, but no protections are in place", zap.String("docs", "https://caddyserver.com/docs/automatic-https#on-demand-tls")) break @@ -293,16 +325,44 @@ func (t *TLS) Stop() error { // Cleanup frees up resources allocated during Provision. func (t *TLS) Cleanup() error { - // stop the certificate cache - if t.certCache != nil { - t.certCache.Stop() - } - // stop the session ticket rotation goroutine if t.SessionTickets != nil { t.SessionTickets.stop() } + // if a new TLS app was loaded, remove certificates from the cache that are no longer + // being managed or loaded by the new config; if there is no more TLS app running, + // then stop cert maintenance and let the cert cache be GC'ed + if nextTLS := caddy.ActiveContext().AppIfConfigured("tls"); nextTLS != nil { + nextTLSApp := nextTLS.(*TLS) + + // compute which certificates were managed or loaded into the cert cache by this + // app instance (which is being stopped) that are not managed or loaded by the + // new app instance (which just started), and remove them from the cache + var noLongerManaged, noLongerLoaded []string + for subj := range t.managing { + if _, ok := nextTLSApp.managing[subj]; !ok { + noLongerManaged = append(noLongerManaged, subj) + } + } + for hash := range t.loaded { + if _, ok := nextTLSApp.loaded[hash]; !ok { + noLongerLoaded = append(noLongerLoaded, hash) + } + } + + certCacheMu.RLock() + certCache.RemoveManaged(noLongerManaged) + certCache.Remove(noLongerLoaded) + certCacheMu.RUnlock() + } else { + // no more TLS app running, so delete in-memory cert cache + certCache.Stop() + certCacheMu.Lock() + certCache = nil + certCacheMu.Unlock() + } + return nil } @@ -327,6 +387,9 @@ func (t *TLS) Manage(names []string) error { if err != nil { return fmt.Errorf("automate: manage %v: %v", names, err) } + for _, name := range names { + t.managing[name] = struct{}{} + } } return nil @@ -388,8 +451,8 @@ func (t *TLS) AddAutomationPolicy(ap *AutomationPolicy) error { // first see if existing is superset of ap for all names var otherIsSuperset bool outer: - for _, thisSubj := range ap.Subjects { - for _, otherSubj := range existing.Subjects { + for _, thisSubj := range ap.subjects { + for _, otherSubj := range existing.subjects { if certmagic.MatchWildcard(thisSubj, otherSubj) { otherIsSuperset = true break outer @@ -398,7 +461,7 @@ func (t *TLS) AddAutomationPolicy(ap *AutomationPolicy) error { } // if existing AP is a superset or if it contains fewer names (i.e. is // more general), then new AP is more specific, so insert before it - if otherIsSuperset || len(existing.Subjects) < len(ap.Subjects) { + if otherIsSuperset || len(existing.SubjectsRaw) < len(ap.SubjectsRaw) { t.Automation.Policies = append(t.Automation.Policies[:i], append([]*AutomationPolicy{ap}, t.Automation.Policies[i:]...)...) return nil @@ -420,10 +483,10 @@ func (t *TLS) getConfigForName(name string) *certmagic.Config { // public certificate or not. func (t *TLS) getAutomationPolicyForName(name string) *AutomationPolicy { for _, ap := range t.Automation.Policies { - if len(ap.Subjects) == 0 { + if len(ap.subjects) == 0 { return ap // no host filter is an automatic match } - for _, h := range ap.Subjects { + for _, h := range ap.subjects { if certmagic.MatchWildcard(name, h) { return ap } @@ -437,8 +500,27 @@ func (t *TLS) getAutomationPolicyForName(name string) *AutomationPolicy { // AllMatchingCertificates returns the list of all certificates in // the cache which could be used to satisfy the given SAN. -func (t *TLS) AllMatchingCertificates(san string) []certmagic.Certificate { - return t.certCache.AllMatchingCertificates(san) +func AllMatchingCertificates(san string) []certmagic.Certificate { + return certCache.AllMatchingCertificates(san) +} + +func (t *TLS) HasCertificateForSubject(subject string) bool { + certCacheMu.RLock() + allMatchingCerts := certCache.AllMatchingCertificates(subject) + certCacheMu.RUnlock() + for _, cert := range allMatchingCerts { + // check if the cert is manually loaded by this config + if _, ok := t.loaded[cert.Hash()]; ok { + return true + } + // check if the cert is automatically managed by this config + for _, name := range cert.Names { + if _, ok := t.managing[name]; ok { + return true + } + } + } + return false } // keepStorageClean starts a goroutine that immediately cleans up all @@ -552,7 +634,9 @@ type Certificate struct { // // Technically, this is a no-op certificate loader module that is treated as // a special case: it uses this app's automation features to load certificates -// for the list of hostnames, rather than loading certificates manually. +// for the list of hostnames, rather than loading certificates manually. But +// the end result is the same: certificates for these subject names will be +// loaded into the in-memory cache and may then be used. type AutomateLoader []string // CaddyModule returns the Caddy module information. diff --git a/modules/caddytls/zerosslissuer.go b/modules/caddytls/zerosslissuer.go index 0209294..697bab0 100644 --- a/modules/caddytls/zerosslissuer.go +++ b/modules/caddytls/zerosslissuer.go @@ -25,11 +25,12 @@ import ( "strings" "sync" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/certmagic" "github.com/mholt/acmez/acme" "go.uber.org/zap" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { diff --git a/modules/filestorage/filestorage.go b/modules/filestorage/filestorage.go index 629dc27..672c66c 100644 --- a/modules/filestorage/filestorage.go +++ b/modules/filestorage/filestorage.go @@ -15,9 +15,10 @@ package filestorage import ( + "github.com/caddyserver/certmagic" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" - "github.com/caddyserver/certmagic" ) func init() { diff --git a/modules/logging/encoders.go b/modules/logging/encoders.go index 1cfab8e..a4409e7 100644 --- a/modules/logging/encoders.go +++ b/modules/logging/encoders.go @@ -17,11 +17,12 @@ package logging import ( "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "go.uber.org/zap" "go.uber.org/zap/buffer" "go.uber.org/zap/zapcore" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { diff --git a/modules/logging/filewriter.go b/modules/logging/filewriter.go index f902a6d..11c051d 100644 --- a/modules/logging/filewriter.go +++ b/modules/logging/filewriter.go @@ -22,10 +22,11 @@ import ( "path/filepath" "strconv" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/dustin/go-humanize" "gopkg.in/natefinch/lumberjack.v2" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { @@ -126,7 +127,7 @@ func (fw FileWriter) OpenWriter() (io.WriteCloser, error) { } // otherwise just open a regular file - return os.OpenFile(fw.Filename, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0666) + return os.OpenFile(fw.Filename, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0o666) } // UnmarshalCaddyfile sets up the module from Caddyfile tokens. Syntax: diff --git a/modules/logging/filterencoder.go b/modules/logging/filterencoder.go index 1a6842e..4d51e64 100644 --- a/modules/logging/filterencoder.go +++ b/modules/logging/filterencoder.go @@ -19,12 +19,13 @@ import ( "fmt" "time" - "github.com/caddyserver/caddy/v2" - "github.com/caddyserver/caddy/v2/caddyconfig" - "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "go.uber.org/zap" "go.uber.org/zap/buffer" "go.uber.org/zap/zapcore" + + "github.com/caddyserver/caddy/v2" + "github.com/caddyserver/caddy/v2/caddyconfig" + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" ) func init() { diff --git a/modules/logging/filters.go b/modules/logging/filters.go index 8cc84c7..233d5d7 100644 --- a/modules/logging/filters.go +++ b/modules/logging/filters.go @@ -25,10 +25,11 @@ import ( "strconv" "strings" + "go.uber.org/zap/zapcore" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - "go.uber.org/zap/zapcore" ) func init() { @@ -81,8 +82,7 @@ func hash(s string) string { // of the SHA-256 hash of the content. Operates // on string fields, or on arrays of strings // where each string is hashed. -type HashFilter struct { -} +type HashFilter struct{} // CaddyModule returns the Caddy module information. func (HashFilter) CaddyModule() caddy.ModuleInfo { @@ -100,12 +100,15 @@ func (f *HashFilter) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { // Filter filters the input field with the replacement value. func (f *HashFilter) Filter(in zapcore.Field) zapcore.Field { if array, ok := in.Interface.(caddyhttp.LoggableStringArray); ok { + newArray := make(caddyhttp.LoggableStringArray, len(array)) for i, s := range array { - array[i] = hash(s) + newArray[i] = hash(s) } + in.Interface = newArray } else { in.String = hash(in.String) } + return in } @@ -219,9 +222,11 @@ func (m *IPMaskFilter) Provision(ctx caddy.Context) error { // Filter filters the input field. func (m IPMaskFilter) Filter(in zapcore.Field) zapcore.Field { if array, ok := in.Interface.(caddyhttp.LoggableStringArray); ok { + newArray := make(caddyhttp.LoggableStringArray, len(array)) for i, s := range array { - array[i] = m.mask(s) + newArray[i] = m.mask(s) } + in.Interface = newArray } else { in.String = m.mask(in.String) } @@ -368,9 +373,23 @@ func (m *QueryFilter) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { // Filter filters the input field. func (m QueryFilter) Filter(in zapcore.Field) zapcore.Field { - u, err := url.Parse(in.String) + if array, ok := in.Interface.(caddyhttp.LoggableStringArray); ok { + newArray := make(caddyhttp.LoggableStringArray, len(array)) + for i, s := range array { + newArray[i] = m.processQueryString(s) + } + in.Interface = newArray + } else { + in.String = m.processQueryString(in.String) + } + + return in +} + +func (m QueryFilter) processQueryString(s string) string { + u, err := url.Parse(s) if err != nil { - return in + return s } q := u.Query() @@ -392,9 +411,7 @@ func (m QueryFilter) Filter(in zapcore.Field) zapcore.Field { } u.RawQuery = q.Encode() - in.String = u.String() - - return in + return u.String() } type cookieFilterAction struct { @@ -580,9 +597,11 @@ func (m *RegexpFilter) Provision(ctx caddy.Context) error { // Filter filters the input field with the replacement value if it matches the regexp. func (f *RegexpFilter) Filter(in zapcore.Field) zapcore.Field { if array, ok := in.Interface.(caddyhttp.LoggableStringArray); ok { + newArray := make(caddyhttp.LoggableStringArray, len(array)) for i, s := range array { - array[i] = f.regexp.ReplaceAllString(s, f.Value) + newArray[i] = f.regexp.ReplaceAllString(s, f.Value) } + in.Interface = newArray } else { in.String = f.regexp.ReplaceAllString(in.String, f.Value) } diff --git a/modules/logging/filters_test.go b/modules/logging/filters_test.go index e9c3e77..8f7ba0d 100644 --- a/modules/logging/filters_test.go +++ b/modules/logging/filters_test.go @@ -83,7 +83,7 @@ func TestIPMaskMultiValue(t *testing.T) { } } -func TestQueryFilter(t *testing.T) { +func TestQueryFilterSingleValue(t *testing.T) { f := QueryFilter{[]queryFilterAction{ {replaceAction, "foo", "REDACTED"}, {replaceAction, "notexist", "REDACTED"}, @@ -102,6 +102,40 @@ func TestQueryFilter(t *testing.T) { } } +func TestQueryFilterMultiValue(t *testing.T) { + f := QueryFilter{ + Actions: []queryFilterAction{ + {Type: replaceAction, Parameter: "foo", Value: "REDACTED"}, + {Type: replaceAction, Parameter: "notexist", Value: "REDACTED"}, + {Type: deleteAction, Parameter: "bar"}, + {Type: deleteAction, Parameter: "notexist"}, + {Type: hashAction, Parameter: "hash"}, + }, + } + + if f.Validate() != nil { + t.Fatalf("the filter must be valid") + } + + out := f.Filter(zapcore.Field{Interface: caddyhttp.LoggableStringArray{ + "/path1?foo=a&foo=b&bar=c&bar=d&baz=e&hash=hashed", + "/path2?foo=c&foo=d&bar=e&bar=f&baz=g&hash=hashed", + }}) + arr, ok := out.Interface.(caddyhttp.LoggableStringArray) + if !ok { + t.Fatalf("field is wrong type: %T", out.Interface) + } + + expected1 := "/path1?baz=e&foo=REDACTED&foo=REDACTED&hash=e3b0c442" + expected2 := "/path2?baz=g&foo=REDACTED&foo=REDACTED&hash=e3b0c442" + if arr[0] != expected1 { + t.Fatalf("query parameters in entry 0 have not been filtered correctly: got %s, expected %s", arr[0], expected1) + } + if arr[1] != expected2 { + t.Fatalf("query parameters in entry 1 have not been filtered correctly: got %s, expected %s", arr[1], expected2) + } +} + func TestValidateQueryFilter(t *testing.T) { f := QueryFilter{[]queryFilterAction{ {}, diff --git a/modules/logging/netwriter.go b/modules/logging/netwriter.go index 5a6cf39..1939cb7 100644 --- a/modules/logging/netwriter.go +++ b/modules/logging/netwriter.go @@ -40,6 +40,11 @@ type NetWriter struct { // The timeout to wait while connecting to the socket. DialTimeout caddy.Duration `json:"dial_timeout,omitempty"` + // If enabled, allow connections errors when first opening the + // writer. The error and subsequent log entries will be reported + // to stderr instead until a connection can be re-established. + SoftStart bool `json:"soft_start,omitempty"` + addr caddy.NetworkAddress } @@ -92,7 +97,12 @@ func (nw NetWriter) OpenWriter() (io.WriteCloser, error) { } conn, err := reconn.dial() if err != nil { - return nil, err + if !nw.SoftStart { + return nil, err + } + // don't block config load if remote is down or some other external problem; + // we can dump logs to stderr for now (see issue #5520) + fmt.Fprintf(os.Stderr, "[ERROR] net log writer failed to connect: %v (will retry connection and print errors here in the meantime)\n", err) } reconn.connMu.Lock() reconn.Conn = conn @@ -104,6 +114,7 @@ func (nw NetWriter) OpenWriter() (io.WriteCloser, error) { // // net <address> { // dial_timeout <duration> +// soft_start // } func (nw *NetWriter) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { for d.Next() { @@ -128,6 +139,12 @@ func (nw *NetWriter) UnmarshalCaddyfile(d *caddyfile.Dispenser) error { return d.ArgErr() } nw.DialTimeout = caddy.Duration(timeout) + + case "soft_start": + if d.NextArg() { + return d.ArgErr() + } + nw.SoftStart = true } } } @@ -151,8 +168,10 @@ func (reconn *redialerConn) Write(b []byte) (n int, err error) { reconn.connMu.RLock() conn := reconn.Conn reconn.connMu.RUnlock() - if n, err = conn.Write(b); err == nil { - return + if conn != nil { + if n, err = conn.Write(b); err == nil { + return + } } // problem with the connection - lock it and try to fix it @@ -161,8 +180,10 @@ func (reconn *redialerConn) Write(b []byte) (n int, err error) { // if multiple concurrent writes failed on the same broken conn, then // one of them might have already re-dialed by now; try writing again - if n, err = reconn.Conn.Write(b); err == nil { - return + if reconn.Conn != nil { + if n, err = reconn.Conn.Write(b); err == nil { + return + } } // there's still a problem, so try to re-attempt dialing the socket @@ -178,7 +199,9 @@ func (reconn *redialerConn) Write(b []byte) (n int, err error) { return } if n, err = conn2.Write(b); err == nil { - reconn.Conn.Close() + if reconn.Conn != nil { + reconn.Conn.Close() + } reconn.Conn = conn2 } } else { diff --git a/modules/metrics/metrics.go b/modules/metrics/metrics.go index d4ad977..a9e0f0e 100644 --- a/modules/metrics/metrics.go +++ b/modules/metrics/metrics.go @@ -17,15 +17,14 @@ package metrics import ( "net/http" + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promhttp" + "go.uber.org/zap" + "github.com/caddyserver/caddy/v2" "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile" "github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile" "github.com/caddyserver/caddy/v2/modules/caddyhttp" - - "go.uber.org/zap" - - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" ) func init() { diff --git a/replacer.go b/replacer.go index 7f97f34..5d33b7f 100644 --- a/replacer.go +++ b/replacer.go @@ -16,6 +16,7 @@ package caddy import ( "fmt" + "net/http" "os" "path/filepath" "runtime" @@ -130,7 +131,8 @@ func (r *Replacer) ReplaceFunc(input string, f ReplacementFunc) (string, error) func (r *Replacer) replace(input, empty string, treatUnknownAsEmpty, errOnEmpty, errOnUnknown bool, - f ReplacementFunc) (string, error) { + f ReplacementFunc, +) (string, error) { if !strings.Contains(input, string(phOpen)) { return input, nil } @@ -316,6 +318,11 @@ func globalDefaultReplacements(key string) (any, bool) { return runtime.GOARCH, true case "time.now": return nowFunc(), true + case "time.now.http": + // According to the comment for http.TimeFormat, the timezone must be in UTC + // to generate the correct format. + // https://github.com/caddyserver/caddy/issues/5773 + return nowFunc().UTC().Format(http.TimeFormat), true case "time.now.common_log": return nowFunc().Format("02/Jan/2006:15:04:05 -0700"), true case "time.now.year": diff --git a/service_windows.go b/service_windows.go index 357c9ac..4720dba 100644 --- a/service_windows.go +++ b/service_windows.go @@ -18,8 +18,9 @@ import ( "os" "path/filepath" - "github.com/caddyserver/caddy/v2/notify" "golang.org/x/sys/windows/svc" + + "github.com/caddyserver/caddy/v2/notify" ) func init() { |