summaryrefslogtreecommitdiff
path: root/modules/caddyhttp/reverseproxy/reverseproxy.go
AgeCommit message (Collapse)Author
2021-12-02caddyhttp: Make logging of credential headers opt-in (#4438)Francis Lavoie
2021-11-24reverseproxy: Adjust defaults, document defaults (#4436)Francis Lavoie
* reverseproxy: Adjust defaults, document defaults Related to some of the issues in https://github.com/caddyserver/caddy/issues/4245, a complaint about the proxy transport defaults not being properly documented in https://caddy.community/t/default-values-for-directives/14254/6. - Dug into the stdlib to find the actual defaults for some of the timeouts and buffer limits, documenting them in godoc so the JSON docs get them next release. - Moved the keep-alive and dial-timeout defaults from `reverseproxy.go` to `httptransport.go`. It doesn't make sense to set defaults in the proxy, because then any time the transport is configured with non-defaults, the keep-alive and dial-timeout defaults are lost! - Sped up the dial timeout from 10s to 3s, in practice it rarely makes sense to wait a whole 10s for dialing. A shorter timeout helps a lot with the load balancer retries, so using something lower helps with user experience. * reverseproxy: Make keepalive interval configurable via Caddyfile * fastcgi: DialTimeout default for fastcgi transport too
2021-10-26reverseproxy: Sanitize scheme and host on incoming requests (#4237)Francis Lavoie
* caddyhttp: Sanitize scheme and host on incoming requests * reverseproxy: Sanitize the URL scheme and host before proxying * Apply suggestions from code review Co-authored-by: Matt Holt <mholt@users.noreply.github.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-10-18reverseproxy: Prevent copying the response if a response handler ran (#4388)Francis Lavoie
2021-09-27Revert 3336faf2 (close #4360)Matthew Holt
Debug log is correct level for this
2021-09-24reverseproxy: Log error at error level (fix #4360)Matthew Holt
2021-08-23reverseproxy: Remove redundant flushing (#4299)Francis Lavoie
From reading through the code, I think this code path is now obsoleted by the changes made in https://github.com/caddyserver/caddy/pull/4266. Basically, `h.flushInterval()` will set the flush interval to `-1` if we're in a bi-directional stream, and the recent PR ensured that `h.copyResponse()` properly flushes headers immediately when the flush interval is non-zero. So now there should be no need to call Flush before calling `h.copyResponse()`.
2021-08-12reverseproxy: Incorporate latest proxy changes from stdlib (#4266)Francis Lavoie
I went through the commits that touched stdlib's `reverseproxy.go` file, and copied over all the changes that are to code that was copied into Caddy. The commits I pulled changes from: - https://github.com/golang/go/commit/2cc347382f4df3fb40d8d81ec9331f0748b1c394 - https://github.com/golang/go/commit/a5cea062b305c8502bdc959c0eec279dbcd4391f - https://github.com/golang/go/commit/ecdbffd4ec68b509998792f120868fec319de59b - https://github.com/golang/go/commit/21898524f66c075d7cfb64a38f17684140e57675 -https://github.com/golang/go/commit/ca3c0df1f8e07337ba4048b191bf905118ebe251 - https://github.com/golang/go/commit/9c017ff30dd21bbdcdb11f39458d3944db530d7e This may also fix https://github.com/caddyserver/caddy/issues/4247 because of the change to `copyResponse` to set `mlw.flushPending = true` right away.
2021-06-15reverseproxy: Fix overwriting of max_idle_conns_per_host (closes #4201)Matthew Holt
Also split the Caddyfile subdirective keepalive_idle_conns into two properties so the conns and conns_per_host can be set separately. This is technically a breaking change, but probably anyone who this breaks already had a broken config anyway, and silently fixing it won't help them fix their configs.
2021-06-04reverseproxy: Always remove hop-by-hop headersMatthew Holt
See golang/go#46313 Based on https://github.com/golang/go/commit/950fa11c4cb01a145bb07eeb167d90a1846061b3
2021-05-12reverseproxy: Set the headers in the replacer before `handle_response` (#4165)Francis Lavoie
Turns out this was an oversight, we assumed we could use `{http.response.header.*}` but that doesn't work because those are grabbed from the response writer, and we haven't copied any headers into the response writer yet. So the fix is to set all the response headers into the replacer at a new namespace before running the handlers. This adds the `{http.reverse_proxy.header.*}` replacer. See https://caddy.community/t/empty-http-response-header-x-accel-redirect/12447
2021-05-05reverseproxy: Minor logging improvementsMatthew Holt
2021-05-02reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710) (#4021)Francis Lavoie
* reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710) * reverseproxy: complete handle_response test * reverseproxy: Change handle_response matchers to use named matchers reverseproxy: Add support for changing status code * fastcgi: Remove obsolete TODO We already have d.Err("transport already specified") in the reverse_proxy parsing code which covers this case * reverseproxy: Fix support for "4xx" type status codes * Apply suggestions from code review Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * caddyhttp: Reorganize response matchers * reverseproxy: Reintroduce caddyfile.Unmarshaler * reverseproxy: Add comment mentioning Finalize should be called Co-authored-by: Maxime Soulé <btik-git@scoubidou.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-03-29reverseproxy: Implement health_uri, deprecate health_path, supports query ↵Francis Lavoie
(#4050) * reverseproxy: Implement health_uri, replaces health_path, supports query Also fixes a bug with `health_status` Caddyfile parsing , it would always only take the first character of the status code even if it didn't end with "xx". * reverseproxy: Rename to URI, named logger, warn in Provision (for JSON)
2021-02-25reverseproxy: Fix round robin data race (#4038)Matthew Holt
2021-02-22reverseproxy: Add duration/latency placeholders (close #4012) (#4013)Matt Holt
* reverseproxy: Add duration/latency placeholders (close #4012) (and #2268) Adds 4 placeholders, one is actually outside reverse proxy though: {http.request.duration} is how long since the server decoded the HTTP request (headers). {http.reverse_proxy.upstream.latency} is how long it took a proxy upstream to write the response header. {http.reverse_proxy.upstream.duration} is total time proxying to the upstream, including writing response body to client. {http.reverse_proxy.duration} is total time spent proxying, including selecting an upstream and retries. Obviously, most of these are only useful at the end of a request, like when writing response headers or logs. See also: https://caddy.community/t/any-equivalent-of-request-time-and-upstream-header-time-from-nginx/11418 * Add new placeholders to documentation
2021-02-09reverseproxy: Response buffering & configurable buffer sizeMatthew Holt
Proxy response bodies can now be buffered, and the size of the request body and response body buffer can be limited. Any remaining content that doesn't fit in the buffer will remain on the wire until it can be read; i.e. bodies are not truncated, even if the buffer is not big enough. This fulfills a customer requirement. This was made possible by their sponsorship!
2020-11-25reverseproxy: Handle "operation was canceled" errors (#3816)Daniel Santos
* fix(caddy): Avoid "operation was canceled" errors - Also add error handling for StatusGatewayTimeout * revert(caddy): Revert 504 handling - This will potentially break load balancing and health checks * Handle client cancellation as different error Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-11-22ci: Use golangci's github action for linting (#3794)Dave Henderson
* ci: Use golangci's github action for linting Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix most of the staticcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the prealloc lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the misspell lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the varcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the errcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the bodyclose lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the deadcode lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the unused lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the gosec lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the gosimple lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the ineffassign lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Fix the staticcheck lint errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Revert the misspell change, use a neutral English Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Remove broken golangci-lint CI job Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Re-add errantly-removed weakrand initialization Signed-off-by: Dave Henderson <dhenderson@gmail.com> * don't break the loop and return * Removing extra handling for null rootKey * unignore RegisterModule/RegisterAdapter Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com> * single-line log message Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * Fix lint after a1808b0dbf209c615e438a496d257ce5e3acdce2 was merged Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Revert ticker change, ignore it instead Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Ignore some of the write errors Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Remove blank line Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Use lifetime Signed-off-by: Dave Henderson <dhenderson@gmail.com> * close immediately Co-authored-by: Matt Holt <mholt@users.noreply.github.com> * Preallocate configVals Signed-off-by: Dave Henderson <dhenderson@gmail.com> * Update modules/caddytls/distributedstek/distributedstek.go Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com> Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-11-20reverseproxy: Implement cookie hash selection policy (#3809)Dimitri Masson
* add CookieHashSelection for session affinity * add CookieHashSelection for session affinity * register module * reverse_proxy: Add and fix cookie lb_policy * reverse_proxy: Manage hmac.write error on cookie hash selection * reverse_proxy: fix some comments * reverse_proxy: variable `cookieValue` is inside the else block * reverse_proxy: Abstract duplicate nuanced logic of reservoir sampling into a function * reverse_proxy: Set a default secret is indeed useless * reverse_proxy: add configuration syntax for cookie lb_policy * reverse_proxy: doc typo and improvement Co-authored-by: utick <123liuqingdong@163.com>
2020-10-13reverseproxy: Fix dial placeholders, SRV, active health checks (#3780)Matt Holt
* reverseproxy: Fix dial placeholders, SRV, active health checks Supercedes #3776 Partially reverts or updates #3756, #3693, and #3695 * reverseproxy: add integration tests Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2020-10-01reverseproxy: allow no port for SRV; fix regression in d55d50b (#3756)Mohammed Al Sahaf
* reverseproxy: fix breakage in handling SRV lookup introduced by 3695 * reverseproxy: validate against incompatible config options with lookup_srv * reverseproxy: add integration test cases for validations involving lookup_srv * reverseproxy: clarify the reason for skipping an iteration * grammar.. Oxford comma Co-authored-by: Francis Lavoie <lavofr@gmail.com> Co-authored-by: Francis Lavoie <lavofr@gmail.com> Fixes #3753
2020-10-01reverseproxy: Change 500 error to 502 for lookup_srv config (#3771)Aleksei
Fixes #3763
2020-09-17reverseproxy: Correct alternate port for active health checks (#3693)Mohammed Al Sahaf
* reverseproxy: construct active health-check transport from scratch (Fixes #3691) * reverseproxy: do upstream health-check on the correct alternative port * reverseproxy: add integration test for health-check on alternative port * reverseproxy: put back the custom transport for health-check http client * reverseproxy: cleanup health-check integration test * reverseproxy: fix health-check of unix socket upstreams * reverseproxy: skip unix socket tests on Windows * tabs > spaces Co-authored-by: Francis Lavoie <lavofr@gmail.com> * make the linter (and @francislavoie) happy Co-authored-by: Francis Lavoie <lavofr@gmail.com> * One more lint fix Co-authored-by: Francis Lavoie <lavofr@gmail.com> Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2020-09-16reverseproxy: Enforce port range size of 1 at provision (#3695)Mohammed Al Sahaf
* reverse_proxy: ensure upstream address has port range of only 1 * reverse_proxy: don't log the error if upstream range size is more than 1
2020-08-07reverseproxy: Minor fixes and cleanupMatthew Holt
Now use context cancellation to stop active health checker, which is simpler than and just as effective as using a separate stop channel.
2020-08-03reverse_proxy: fix bidirectional streams with encodings (fix #3606) (#3620)Kevin Lin
* reverse_proxy: fix bi-h2stream breaking gzip encode handle(#3606). * reverse_proxy: check http version of both sides to avoid affecting non-h2 upstream. * Minor cleanup; apply review suggestions Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-07-20reverse_proxy: flush HTTP/2 response when ContentLength is unknown (#3561)Kevin Lin
* reverse proxy: Support more h2 stream scenarios (#3556) * reverse proxy: add integration test for better h2 stream (#3556) * reverse proxy: adjust comments as francislavoie suggests * link to issue #3556 in the comments
2020-07-17reverseproxy: Restore request's original host and header (fix #3509)Matthew Holt
We already restore them within the retry loop, but after successful proxy we didn't reset them, so as handlers bubble back up, they would see the values used for proxying. Thanks to @ziddey for identifying the cause.
2020-06-04reverseproxy: Enable changing only the status code (close #2920)Matthew Holt
2020-05-27reverseproxy: Pool copy buffers (minor optimization)Matthew Holt
2020-05-27reverseproxy: Enable response interception (#1447, #2920)Matthew Holt
It's a raw, low-level implementation for now, but it's very flexible. More sugar-coating can be added after error handling is more developed.
2020-05-20reverseproxy: Don't overwrite existing X-Forwarded-Proto headerMatthew Holt
Correct behavior is not well defined because this is a non-standard header field. This could be a "hop-by-hop" field much like X-Forwarded-For is, but even our X-Forwarded-For implementation preserves prior entries. Or, it could be best to preserve the original value from the first hop, representing the protocol as facing the client. Let's try it the other way for a bit and see how it goes. See https://caddy.community/t/caddy2-w-wordpress-behind-nginx-reverse-proxy/8174/3?u=matt
2020-05-18reverseproxy: Make debug log safe if error occursMatthew Holt
2020-05-18reverseproxy: Emit debug log before checking error (#3425)Matthew Holt
This way the upstream request will always be available even if it failed
2020-05-13reverseproxy: Apply response header ops before copying it (fix #3382) (#3401)Matt Holt
2020-04-22reverseproxy: Don't forget to provision embedded headers handlerMatthew Holt
https://caddy.community/t/set-cookie-manipulation-in-reverse-proxy/7666?u=matt
2020-04-21reverseproxy: always set req.URL.Host with upstream (#3297)westwin
2020-04-17reverseproxy: Set X-Forwarded-Proto (closes #3275) (#3276)Matt Holt
2020-04-07reverseproxy: Remove NTLM transport; refactor and improve docsMatthew Holt
2020-03-30Keep type information with placeholders until replacements happenMatthew Holt
2020-03-27reverse_proxy: Upstream.String() method returns either LookupSRV or DialMatthew Holt
Either Dial or LookupSRV will be set, but if we rely on Dial always being set, we could run into bugs. Note: Health checks don't support SRV upstreams.
2020-03-24reverse_proxy: Add support for SRV backends (#3180)Matt Holt
* reverse_proxy: Begin SRV lookup support (WIP) * reverse_proxy: Finish adding support for SRV-based backends (#3179)
2020-03-20reverseproxy: Add Alt-Svc to Hop-by-hop headers list (#3159)Paolo Barbolini
Adds `Alt-Svc` to the list of headers that get removed when proxying to a backend. This fixes the issue of having the contents of the Alt-Svc header duplicated when proxying to another Caddy server.
2020-02-23reverse_proxy: Health checks: Don't cross the streamsMatthew Holt
Fixes https://caddy.community/t/v2-health-checks-are-going-to-the-wrong-upstream/7084?u=matt ... I think
2020-01-09http: Change routes to sequential matcher evaluation (#2967)Matt Holt
Previously, all matchers in a route would be evaluated before any handlers were executed, and a composite route of the matching routes would be created. This made rewrites especially tricky, since the only way to defer later matchers' evaluation was to wrap them in a subroute, or to invoke a "rehandle" which often caused bugs. Instead, this new sequential design evaluates each route's matchers then its handlers in lock-step; matcher-handlers-matcher-handlers... If the first matching route consists of a rewrite, then the second route will be evaluated against the rewritten request, rather than the original one, and so on. This should do away with any need for rehandling. I've also taken this opportunity to avoid adding new values to the request context in the handler chain, as this creates a copy of the Request struct, which may possibly lead to bugs like it has in the past (see PR #1542, PR #1481, and maybe issue #2463). We now add all the expected context values in the top-level handler at the server, then any new values can be added to the variable table via the VarsCtxKey context key, or just the GetVar/SetVar functions. In particular, we are using this facility to convey dial information in the reverse proxy. Had to be careful in one place as the middleware compilation logic has changed, and moved a bit. We no longer compile a middleware chain per- request; instead, we can compile it at provision-time, and defer only the evaluation of matchers to request-time, which should slightly improve performance. Doing this, however, we take advantage of multiple function closures, and we also changed the use of HandlerFunc (function pointer) to Handler (interface)... this led to a situation where, if we aren't careful, allows one request routed a certain way to permanently change the "next" handler for all/most other requests! We avoid this by making a copy of the interface value (which is a lightweight pointer copy) and using exclusively that within our wrapped handlers. This way, the original stack frame is preserved in a "read-only" fashion. The comments in the code describe this phenomenon. This may very well be a breaking change for some configurations, however I do not expect it to impact many people. I will make it clear in the release notes that this change has occurred.
2019-12-29Improve docs, especially w.r.t. placeholders and template actionsMatthew Holt
2019-12-29Export Replacer and use concrete type instead of interfaceMatthew Holt
The interface was only making things difficult; a concrete pointer is probably best.
2019-12-23Improve godocs all aroundMatthew Holt
These will be used in the new automated documentation system
2019-12-12Minor improvements; comments and shorter placeholders & module IDsMatthew Holt
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-10 00:20:30 +0000