Ref https://github.com/varnishcache/varnish-cache/pull/4035

see https://github.com/varnishcache/varnish-cache/pull/3804 for context

The C compiler checks have been removed as they are already
contained in VARNISH_PREREQ

- ensure we do send VDP_END down
- try to send it with the last bytes (POLLHUP present
  with a read smaller than the buffer size)

... except for stdin, stdout and stderr.

Besides the obvious security issue, this avoids an issue where output
processing would hang with more than one pipe vdp:

When we see a VDP_END, we close the external program's stdin. Yet before
we closed all file descriptors, a second external progam would also hold
the stdin pipe's write end open.

dup2() closes the destination file descriptor anyway

Any real-world object will exceed the pipe buffer in size. Unless the
filter program which we invoke implemented an infinite buffer (and
requiring something like this would miss the point of stream
processing), we can only write to the program as much data as it
accepts for its input buffering (plus the pipe buffer size), before we
need to read its output.

Thus, we need to ensure that write(fds[STDIN_FILENO].fd) does not
block.

Only the first filter has access to the objcore.

fixes:

Assert error in vdp_init(), vdfp_pipe.c line 189:
  Condition((objcore) != NULL) not true.
version = varnish-trunk revision
136736b092206c5bc764f12f80d9092efde6c9dc, vrt api = 13.0
ident = Linux,5.10.12-200.fc33.x86_64,x86_64,-junix,-sfile,-sdefault,-hcritbit,epoll
now = 3808647.356313 (mono), 1618849595.965169 (real)
Backtrace:
  0x559444057399: /usr/sbin/varnishd(+0x57399) [0x559444057399]
  0x5594440ce273: /usr/sbin/varnishd(VAS_Fail+0x43) [0x5594440ce273]
  0x7fc6745f820f: ./vmod_cache/_vmod_pipe.f336a3498bf9765ce992b39693196344e9c1ddda789775d4d0b0fd3b4939272e(+0x220f) [0x7fc6745f820f]
  0x5594440381a3: /usr/sbin/varnishd(VDP_Push+0x103) [0x5594440381a3]
  0x55944406df96: /usr/sbin/varnishd(VCL_StackVDP+0x76) [0x55944406df96]
  0x55944405ce02: /usr/sbin/varnishd(+0x5ce02) [0x55944405ce02]
  0x55944405ec48: /usr/sbin/varnishd(CNT_Request+0xd8) [0x55944405ec48]
  0x559444088bbb: /usr/sbin/varnishd(+0x88bbb) [0x559444088bbb]
  0x55944407cee7: /usr/sbin/varnishd(+0x7cee7) [0x55944407cee7]
  0x55944407d494: /usr/sbin/varnishd(+0x7d494) [0x55944407d494]

- ensure we do send VDP_END down
- try to send it with the last bytes (POLLHUP present
  with a read smaller than the buffer size)

- ensure we do send VDP_END down
- try to send it with the last bytes (POLLHUP present
  with a read smaller than the buffer size)

Somehow related:

- do not wait for the program to close stderr

... except for stdin, stdout and stderr.

Besides the obvious security issue, this avoids an issue where output
processing would hang with more than one pipe vdp:

When we see a VDP_END, we close the external program's stdin. Yet before
we closed all file descriptors, a second external progam would also hold
the stdin pipe's write end open.

dup2() closes the destination file descriptor anyway

Any real-world object will exceed the pipe buffer in size. Unless the
filter program which we invoke implemented an infinite buffer (and
requiring something like this would miss the point of stream
processing), we can only write to the program as much data as it
accepts for its input buffering (plus the pipe buffer size), before we
need to read its output.

Thus, we need to ensure that write(fds[STDIN_FILENO].fd) does not
block.