summaryrefslogtreecommitdiff
path: root/modules/caddyhttp/reverseproxy/reverseproxy.go
AgeCommit message (Collapse)Author
2022-09-29Fix commentMatthew Holt
I apparently read the diff backwards in 2a8c458ffedf886af9542541ea1b1de62370929d
2022-09-29reverseproxy: Parse humanized byte size (fix #5095)Matthew Holt
2022-09-29reverseproxy: On 103 don't delete own headers (#5091)Matt Holt
See #5074
2022-09-16core: Variadic Context.Logger(); soft deprecationMatthew Holt
Ideally I'd just remove the parameter to caddy.Context.Logger(), but this would break most Caddy plugins. Instead, I'm making it variadic and marking it as partially deprecated. In the future, I might completely remove the parameter once most plugins have updated.
2022-09-02reverseproxy: Close hijacked conns on reload/quit (#4895)Matt Holt
* reverseproxy: Close hijacked conns on reload/quit We also send a Close control message to both ends of WebSocket connections. I have tested this many times in my dev environment with consistent success, although the variety of scenarios was limited. * Oops... actually call Close() this time * CloseMessage --> closeMessage Co-authored-by: Francis Lavoie <lavofr@gmail.com> * Use httpguts, duh * Use map instead of sync.Map Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-08-31events: Implement event system (#4912)Francis Lavoie
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-29Minor cleanup, resolve a couple lint warningsMatthew Holt
2022-08-27Remove duplicate words in comments (#4986)Abirdcfly
2022-08-27reverseproxy: Add upstreams healthy metrics (#4935)Dávid Szabó
2022-08-17core: Change net.IP to netip.Addr; use netip.Prefix (#4966)WilczyńskiT
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-12reverseproxy: Ignore context cancel in stream mode (#4952)Matt Holt
2022-08-09reverseproxy: Support 1xx status codes (HTTP early hints) (#4882)Kévin Dunglas
2022-08-06Replace strings.Index with strings.Cut (#4932)Chirag Maheshwari
2022-08-02chore: Bump up to Go 1.19, minimum 1.18 (#4925)Francis Lavoie
2022-07-13reverseproxy: Implement retry count, alternative to try_duration (#4756)Francis Lavoie
* reverseproxy: Implement retry count, alternative to try_duration * Add Caddyfile support for `retry_match` * Refactor to deduplicate matcher parsing logic * Fix lint
2022-07-08reverseproxy: Err 503 if all upstreams unavailableMatthew Holt
2022-06-22reverseproxy: Fix double headers in response handlers (#4847)Francis Lavoie
2022-06-03reverseproxy: HTTP 504 for upstream timeouts (#4824)Matt Holt
Closes #4823
2022-05-06reverseproxy: Support performing pre-check requests (#4739)Francis Lavoie
2022-04-11reverseproxy: Add `_ms` placeholders for proxy durations (#4666)Francis Lavoie
* reverseproxy: Add `_ms` placeholders for proxy durations * Add http.request.duration_ms Also add comments, and change duration_sec to duration_ms * Add response.duration_ms for consistency * Add missing godoc comment Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-09reverseproxy: copy_response and copy_response_headers for handle_response ↵Francis Lavoie
routes (#4391) * reverseproxy: New `copy_response` handler for `handle_response` routes Followup to #4298 and #4388. This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream. Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch. To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream. Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works. Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response. * Delete the "Content-Length" header if we aren't copying Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl: ``` curl: (18) transfer closed with 18 bytes remaining to read ``` To replicate: ``` { admin off debug } :8881 { reverse_proxy 127.0.0.1:8882 { @200 status 200 handle_response @200 { header Foo bar } } } :8882 { header Content-Type application/json respond `{"hello": "world"}` 200 } ``` * Implement `copy_response_headers`, with include/exclude list support * Apply suggestions from code review Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-03-06reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470)Matt Holt
* reverseproxy: Begin refactor to enable dynamic upstreams Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU * Implement SRV and A/AAA upstream sources Also get upstreams at every retry loop iteration instead of just once before the loop. See #4442. * Minor tweaks from review * Limit size of upstreams caches * Add doc notes deprecating LookupSRV * Provision dynamic upstreams Still WIP, preparing to preserve health checker functionality * Rejigger health checks Move active health check results into handler-specific Upstreams. Improve documentation regarding health checks and upstreams. * Deprecation notice * Add Caddyfile support, use `caddy.Duration` * Interface guards * Implement custom resolvers, add resolvers to http transport Caddyfile * SRV: fix Caddyfile `name` inline arg, remove proto condition * Use pointer receiver * Add debug logs Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-06reverseproxy: Implement trusted proxies for `X-Forwarded-*` headers (#4507)Francis Lavoie
2022-03-03reverseproxy: Make shallow-ish clone of the request (#4551)Francis Lavoie
* reverseproxy: Make shallow-ish clone of the request * Refactor request cloning into separate function Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-01-04admin, reverseproxy: Stop timers if canceled to avoid goroutine leak (#4482)Денис Телюх
2021-12-02caddyhttp: Make logging of credential headers opt-in (#4438)Francis Lavoie
2021-11-24reverseproxy: Adjust defaults, document defaults (#4436)Francis Lavoie
* reverseproxy: Adjust defaults, document defaults Related to some of the issues in https://github.com/caddyserver/caddy/issues/4245, a complaint about the proxy transport defaults not being properly documented in https://caddy.community/t/default-values-for-directives/14254/6. - Dug into the stdlib to find the actual defaults for some of the timeouts and buffer limits, documenting them in godoc so the JSON docs get them next release. - Moved the keep-alive and dial-timeout defaults from `reverseproxy.go` to `httptransport.go`. It doesn't make sense to set defaults in the proxy, because then any time the transport is configured with non-defaults, the keep-alive and dial-timeout defaults are lost! - Sped up the dial timeout from 10s to 3s, in practice it rarely makes sense to wait a whole 10s for dialing. A shorter timeout helps a lot with the load balancer retries, so using something lower helps with user experience. * reverseproxy: Make keepalive interval configurable via Caddyfile * fastcgi: DialTimeout default for fastcgi transport too
2021-10-26reverseproxy: Sanitize scheme and host on incoming requests (#4237)Francis Lavoie
* caddyhttp: Sanitize scheme and host on incoming requests * reverseproxy: Sanitize the URL scheme and host before proxying * Apply suggestions from code review Co-authored-by: Matt Holt <mholt@users.noreply.github.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-10-18reverseproxy: Prevent copying the response if a response handler ran (#4388)Francis Lavoie
2021-09-27Revert 3336faf2 (close #4360)Matthew Holt
Debug log is correct level for this
2021-09-24reverseproxy: Log error at error level (fix #4360)Matthew Holt
2021-08-23reverseproxy: Remove redundant flushing (#4299)Francis Lavoie
From reading through the code, I think this code path is now obsoleted by the changes made in https://github.com/caddyserver/caddy/pull/4266. Basically, `h.flushInterval()` will set the flush interval to `-1` if we're in a bi-directional stream, and the recent PR ensured that `h.copyResponse()` properly flushes headers immediately when the flush interval is non-zero. So now there should be no need to call Flush before calling `h.copyResponse()`.
2021-08-12reverseproxy: Incorporate latest proxy changes from stdlib (#4266)Francis Lavoie
I went through the commits that touched stdlib's `reverseproxy.go` file, and copied over all the changes that are to code that was copied into Caddy. The commits I pulled changes from: - https://github.com/golang/go/commit/2cc347382f4df3fb40d8d81ec9331f0748b1c394 - https://github.com/golang/go/commit/a5cea062b305c8502bdc959c0eec279dbcd4391f - https://github.com/golang/go/commit/ecdbffd4ec68b509998792f120868fec319de59b - https://github.com/golang/go/commit/21898524f66c075d7cfb64a38f17684140e57675 -https://github.com/golang/go/commit/ca3c0df1f8e07337ba4048b191bf905118ebe251 - https://github.com/golang/go/commit/9c017ff30dd21bbdcdb11f39458d3944db530d7e This may also fix https://github.com/caddyserver/caddy/issues/4247 because of the change to `copyResponse` to set `mlw.flushPending = true` right away.
2021-06-15reverseproxy: Fix overwriting of max_idle_conns_per_host (closes #4201)Matthew Holt
Also split the Caddyfile subdirective keepalive_idle_conns into two properties so the conns and conns_per_host can be set separately. This is technically a breaking change, but probably anyone who this breaks already had a broken config anyway, and silently fixing it won't help them fix their configs.
2021-06-04reverseproxy: Always remove hop-by-hop headersMatthew Holt
See golang/go#46313 Based on https://github.com/golang/go/commit/950fa11c4cb01a145bb07eeb167d90a1846061b3
2021-05-12reverseproxy: Set the headers in the replacer before `handle_response` (#4165)Francis Lavoie
Turns out this was an oversight, we assumed we could use `{http.response.header.*}` but that doesn't work because those are grabbed from the response writer, and we haven't copied any headers into the response writer yet. So the fix is to set all the response headers into the replacer at a new namespace before running the handlers. This adds the `{http.reverse_proxy.header.*}` replacer. See https://caddy.community/t/empty-http-response-header-x-accel-redirect/12447
2021-05-05reverseproxy: Minor logging improvementsMatthew Holt
2021-05-02reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710) (#4021)Francis Lavoie
* reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710) * reverseproxy: complete handle_response test * reverseproxy: Change handle_response matchers to use named matchers reverseproxy: Add support for changing status code * fastcgi: Remove obsolete TODO We already have d.Err("transport already specified") in the reverse_proxy parsing code which covers this case * reverseproxy: Fix support for "4xx" type status codes * Apply suggestions from code review Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * caddyhttp: Reorganize response matchers * reverseproxy: Reintroduce caddyfile.Unmarshaler * reverseproxy: Add comment mentioning Finalize should be called Co-authored-by: Maxime Soulé <btik-git@scoubidou.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-03-29reverseproxy: Implement health_uri, deprecate health_path, supports query ↵Francis Lavoie
(#4050) * reverseproxy: Implement health_uri, replaces health_path, supports query Also fixes a bug with `health_status` Caddyfile parsing , it would always only take the first character of the status code even if it didn't end with "xx". * reverseproxy: Rename to URI, named logger, warn in Provision (for JSON)
2021-02-25reverseproxy: Fix round robin data race (#4038)Matthew Holt
2021-02-22reverseproxy: Add duration/latency placeholders (close #4012) (#4013)Matt Holt
* reverseproxy: Add duration/latency placeholders (close #4012) (and #2268) Adds 4 placeholders, one is actually outside reverse proxy though: {http.request.duration} is how long since the server decoded the HTTP request (headers). {http.reverse_proxy.upstream.latency} is how long it took a proxy upstream to write the response header. {http.reverse_proxy.upstream.duration} is total time proxying to the upstream, including writing response body to client. {http.reverse_proxy.duration} is total time spent proxying, including selecting an upstream and retries. Obviously, most of these are only useful at the end of a request, like when writing response headers or logs. See also: https://caddy.community/t/any-equivalent-of-request-time-and-upstream-header-time-from-nginx/11418 * Add new placeholders to documentation
2021-02-09reverseproxy: Response buffering & configurable buffer sizeMatthew Holt
Proxy response bodies can now be buffered, and the size of the request body and response body buffer can be limited. Any remaining content that doesn't fit in the buffer will remain on the wire until it can be read; i.e. bodies are not truncated, even if the buffer is not big enough. This fulfills a customer requirement. This was made possible by their sponsorship!
2020-11-25reverseproxy: Handle "operation was canceled" errors (#3816)Daniel Santos
* fix(caddy): Avoid "operation was canceled" errors - Also add error handling for StatusGatewayTimeout * revert(caddy): Revert 504 handling - This will potentially break load balancing and health checks * Handle client cancellation as different error Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-11-22ci: Use golangci's github action for linting (#3794)Dave Henderson
* ci: Use golangci's github action for linting Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix most of the staticcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the prealloc lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the misspell lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the varcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the errcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the bodyclose lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the deadcode lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the unused lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the gosec lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the gosimple lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the ineffassign lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the staticcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Revert the misspell change, use a neutral English Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Remove broken golangci-lint CI job Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Re-add errantly-removed weakrand initialization Signed-off-by: Dave Henderson <dhenderson@gmail.com> * don't break the loop and return * Removing extra handling for null rootKey * unignore RegisterModule/RegisterAdapter Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com> * single-line log message Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * Fix lint after a1808b0dbf209c615e438a496d257ce5e3acdce2 was merged Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Revert ticker change, ignore it instead Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Ignore some of the write errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Remove blank line Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Use lifetime Signed-off-by: Dave Henderson <dhenderson@gmail.com> * close immediately Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * Preallocate configVals Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Update modules/caddytls/distributedstek/distributedstek.go Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-11-20reverseproxy: Implement cookie hash selection policy (#3809)Dimitri Masson
* add CookieHashSelection for session affinity * add CookieHashSelection for session affinity * register module * reverse_proxy: Add and fix cookie lb_policy * reverse_proxy: Manage hmac.write error on cookie hash selection * reverse_proxy: fix some comments * reverse_proxy: variable `cookieValue` is inside the else block * reverse_proxy: Abstract duplicate nuanced logic of reservoir sampling into a function * reverse_proxy: Set a default secret is indeed useless * reverse_proxy: add configuration syntax for cookie lb_policy * reverse_proxy: doc typo and improvement Co-authored-by: utick <123liuqingdong@163.com>
2020-10-13reverseproxy: Fix dial placeholders, SRV, active health checks (#3780)Matt Holt
* reverseproxy: Fix dial placeholders, SRV, active health checks Supercedes #3776 Partially reverts or updates #3756, #3693, and #3695 * reverseproxy: add integration tests Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2020-10-01reverseproxy: allow no port for SRV; fix regression in d55d50b (#3756)Mohammed Al Sahaf
* reverseproxy: fix breakage in handling SRV lookup introduced by 3695 * reverseproxy: validate against incompatible config options with lookup_srv * reverseproxy: add integration test cases for validations involving lookup_srv * reverseproxy: clarify the reason for skipping an iteration * grammar.. Oxford comma Co-authored-by: Francis Lavoie <lavofr@gmail.com> Co-authored-by: Francis Lavoie <lavofr@gmail.com> Fixes #3753
2020-10-01reverseproxy: Change 500 error to 502 for lookup_srv config (#3771)Aleksei
Fixes #3763
2020-09-17reverseproxy: Correct alternate port for active health checks (#3693)Mohammed Al Sahaf
* reverseproxy: construct active health-check transport from scratch (Fixes #3691) * reverseproxy: do upstream health-check on the correct alternative port * reverseproxy: add integration test for health-check on alternative port * reverseproxy: put back the custom transport for health-check http client * reverseproxy: cleanup health-check integration test * reverseproxy: fix health-check of unix socket upstreams * reverseproxy: skip unix socket tests on Windows * tabs > spaces Co-authored-by: Francis Lavoie <lavofr@gmail.com> * make the linter (and @francislavoie) happy Co-authored-by: Francis Lavoie <lavofr@gmail.com> * One more lint fix Co-authored-by: Francis Lavoie <lavofr@gmail.com> Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2020-09-16reverseproxy: Enforce port range size of 1 at provision (#3695)Mohammed Al Sahaf
* reverse_proxy: ensure upstream address has port range of only 1 * reverse_proxy: don't log the error if upstream range size is more than 1
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-08 19:01:38 +0000