- 30 Aug, 2021 18 commits
-
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
This is the test case that fails if these changes aren't in tree. Note the commented out rxwinup commands that are necessary for the proper fail mode when run without the varnishtest window update changes.
-
Martin Blix Grydeland authored
With the new request body data handling, Varnish changes behaviour significantly wrt to stream window updates sent to the client. Window updates will only be sent once the data is consumed by the client through the request body VFP handling. Test cases that rely on receiving a window update to sync the H/2 stream needs to be adopted.
-
Martin Blix Grydeland authored
This makes it easier to not have to know exactly when and how many window updates to expect in a test case.
-
Martin Blix Grydeland authored
We have a strict min at the protocol default here. This is because we don't have the 'use settings only after peer ack' in place yet. If the value is lower than the protocol default, the very first stream could get a flow control error.
-
Martin Blix Grydeland authored
H2 streams waiting for request body data will timeout after timeout_idle seconds if no new data on the stream is being received. This will ensure that individual H2 streams can be reaped if there is no data received from the peer.
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
This implements stream data handling using a buffer between the H/2 session thread and each stream thread. This is needed to avoid head of line blocking on the session socket when a data frame is received for a stream thread that is not yet ready to receive it. The buffer used will have to be as large as the send window the peer expects at the time the stream is opened. This will typically be 65535 unless the h2_initial_window_size parameter has been changed. Stream window updates will then be issued only once data is removed from the buffer by the request body being consumed from the request handling thread, limited in size to what space is then available in the buffer.
-
Martin Blix Grydeland authored
The H/2 session thread does have a VSL buffer already set up, but the 'wrk->vsl' pointer was not set. This caused issue for e.g. LRU_NukeOne() as it wants to log. Set the buffer for the duration that the worker is dedicated as an H/2 session thread.
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
-
Martin Blix Grydeland authored
This is an API for getting an arbitrary buffer through the stevedores. The stevedore in question may then deploy LRU nuking or other measures to control resource usage.
-
Dridi Boukelmoune authored
Working on the workspace sanitizer (ancestor of the workspace emulator) final rollbacks were needed to unwind allocations. There was however a branch where error handling was missing a workspace release, and it was fine before the introduction of the final rollbacks. To avoid turning the workspace emulator into a DoS vector the rollbacks are now only enforced for emulator builds. The specific "insufficient workspace" log is amended to ensure future changes to the session workspace footprint don't accidentally remove test coverage for that branch. The same could be done for other "insufficient workspace" logs in the PROXY protocol parsing. Refs 0632b846 (req: Prevent early rollback) Refs ce71896a (sess: Plug conceptual leak) Refs 246b1eb1 (busyobj: Plug conceptual leak) Refs 5b4f0f1a (htc: Defer workspace rollbacks for request tasks) Refs #3644 Spotted by Alf's single process fuzzing setup that we should eventually revisit. Refs #3152
-
- 28 Aug, 2021 5 commits
-
-
Poul-Henning Kamp authored
-
Poul-Henning Kamp authored
-
Poul-Henning Kamp authored
Most of this is taken from #3650 but instead of `str[n]casecmp` it uses VCT functionality, in order to avoid LOCALE poisoning. Closes #3650
-
Poul-Henning Kamp authored
-
Poul-Henning Kamp authored
Fixes #3677
-
- 27 Aug, 2021 7 commits
-
-
Dridi Boukelmoune authored
Otherwise we would entirely unravel the task workspace.
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
The goal of the workspace emulator is to replicate the regular workspace behavior with individual allocations and make it work transparently. It's the successor of the workspace sanitizer from #3320 with notable differences: - enabled at configure time instead of run time - in a separate source file instead of mixed in - using sparse allocations instead of built-in red zones This means that the workspace emulator can be combined with regular sanitizer, in particular asan and lsan. If available, asan's public interface is used to mitigate the possible overflow of a reservation after some of it was released. Even without sanitizers, the fact that we integrate with jemalloc by default and enable its abort and junk options in varnishtest is enough to detect a use-after-free in some cases. With sanitizers though, the workspace emulator can observe #3550. One drawback is that the logic is split in two files, and some functions are identical in the two files. It might be possible to split cache_ws.c into something like cache_ws_alloc.c and cache_ws_util.c for example. Closes #3320 Refs #3550 Refs #3600
-
Dridi Boukelmoune authored
When a session has data pipelined we perform a dirty dance to move it at the beginning of the workspace. The rollbacks used to occur between HTC_RxPipeline() and HTC_RxInit() calls until it was centralized in the latter. With a dedicated WS_ReqPipeline() operation we can capture the semantics of initializing an existing connection for its next task with or without data fetched from the previous task. While conceptually there is still a use-after-free since the pipelined data may belong to the same workspace, it is fine if that happens within the bounds of an atomic workspace operation.
-
Dridi Boukelmoune authored
When we take on a new request on a connection from which something was already received, we need to pipeline it and we do so at the beginning of the request workspace. There's a high probability that the pipeline is coming from the same workspace, which is a form of use-after-free only made safe by the workspace implementation details. To avoid the conceptual use-after-free, we defer req workspace rollbacks and perform them during the next HTC_RxInit() call before the pipelining operation. Because HTTP/1 works directly on the session, a worker can safely switch back and forth between sess and req tasks. This means that unless the session goes idle the same workspace is used from one client request to the next, hence the rollback previously happening in Req_Cleanup(). With h2 however there is a disconnect between the session and streams. The connection is received in req0's workspace, and then copied into a stream's req workspace via the pipelining scheme. Rollbacks can be deferred as well, but they need to happen otherwise the session will soon overflow. Independent HTC_RxInit() calls happen for req0 in the h2 session thread, and for h2 streams in the regular request task code path. PROXY Protocol parsing may result in receiving more than the proxy preamble itself and pipelining will happen, whether it is via a req for HTTP/1 or req0 for h2. On the other end of the spectrum when Varnish acts as a client it only sends one HTTP/1 request at a time for a given connection, so we never expect pipelining to occur in fetch task.
-
Dridi Boukelmoune authored
-
- 26 Aug, 2021 1 commit
-
-
Dridi Boukelmoune authored
-
- 25 Aug, 2021 9 commits
-
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
-
Dridi Boukelmoune authored
Some of the autoconf logic is originally from Asad. In addition, the explicit call to __gcov_flush() is now vendored in vdef.h like other toolchain abstractions.
-
Dridi Boukelmoune authored
Similarly to how the ASAN setup was split in two steps.
-