Commit 1593fdb1 authored by Poul-Henning Kamp's avatar Poul-Henning Kamp

Move the homepage stuff into the dev-guide.

Add Project Governance info.
parent 626ddb02
.. _homepage_contrib:
How to contribute content to varnish-cache.org
==============================================
This is where we walk you through the mechanics of adding content to
varnish-cache.org (see phk's note :ref:`homepage_dogfood` for an
insight into the innards of site).
Git Repository
--------------
The web site contents live in github at:
https://github.com/varnishcache/homepage
To offer your own contribution, fork the project and send us a pull
request.
Sphinx and RST
--------------
The web site sources are written in `RST
<http://docutils.sourceforge.net/rst.html>`_ -- reStructuredText, the
documentation format originally conceived for Python (and also used in
the Varnish distribution, as well as for formatting VMOD
docs). `Sphinx <http://www.sphinx-doc.org/>`_ is used to render web
pages from the RST sources.
So you'll need to `learn markup with RST and Sphinx
<http://www.sphinx-doc.org/en/stable/markup/index.html>`_; and you
will need to `install Sphinx <http://www.sphinx-doc.org/en/stable/install.html>`_ to test the rendering on your local system.
Makefile
--------
Generation of web contents from the sources is driven by the ``Makefile``
in the ``R1`` directory of the repo::
$ cd R1
$ make help
Please use `make <target>' where <target> is one of
html to make standalone HTML files
dirhtml to make HTML files named index.html in directories
singlehtml to make a single large HTML file
pickle to make pickle files
json to make JSON files
htmlhelp to make HTML files and a HTML help project
qthelp to make HTML files and a qthelp project
applehelp to make an Apple Help Book
devhelp to make HTML files and a Devhelp project
epub to make an epub
latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter
latexpdf to make LaTeX files and run them through pdflatex
latexpdfja to make LaTeX files and run them through platex/dvipdfmx
text to make text files
man to make manual pages
texinfo to make Texinfo files
info to make Texinfo files and run them through makeinfo
gettext to make PO message catalogs
changes to make an overview of all changed/added/deprecated items
xml to make Docutils-native XML files
pseudoxml to make pseudoxml-XML files for display purposes
linkcheck to check all external links for integrity
doctest to run all doctests embedded in the documentation (if enabled)
coverage to run coverage check of the documentation (if enabled)
Most of the time, you'll just need ``make html`` to test the rendering
of your contribution.
alabaster theme
---------------
We use the `alabaster theme <https://pypi.python.org/pypi/alabaster>`_,
which you may need to add to your local Python installation::
$ sudo pip install alabaster
We have found that you may need to link the alabaster package install
directory to the directory where Sphinx expects to find themes. For
example (on my machine), alabaster was installed into::
/usr/local/lib/python2.7/dist-packages/alabaster
And Sphinx expects to find themes in::
/usr/share/sphinx/themes
So to get the make targets to run successfully::
$ cd /usr/share/sphinx/themes
$ ln -s /usr/local/lib/python2.7/dist-packages/alabaster
Test the rendering
------------------
Now you can edit contents in the website repo, and test the rendering
by calling make targets in the ``R1`` directory::
$ cd $REPO/R1
$ make html
sphinx-build -b html -d build/doctrees source build/html
Running Sphinx v1.2.3
loading pickled environment... done
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] tips/contribdoc/index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] tips/index
writing additional files... genindex search
copying static files... done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded.
After a successful build, the newly rendered contents are saved in the
``R1/source/build`` directory, so you can have a look with your
browser.
Send us a pull request
----------------------
When you have your contribution building successfully, send us a PR,
we'll be happy to hear from you!
.. _homepage_dogfood:
How our website works
=====================
The principle of eating your own dogfood is important for software
quality, that is how you experience what your users are dealing with,
and I am not the least ashamed to admit that several obvious improvements
have happened to Varnish as a result of running the project webserver.
But it is also important to externalize what you learn doing so, and
therefore I thought I would document here how the projects new "internal
IT" works.
Hardware
--------
Who cares?
Yes, we use some kind of hardware, but to be honest I don't know what
it is.
Our primary site runs on a `RootBSD 'Omega' <https://www.rootbsd.net/>`_
virtual server somewhere near CDG/Paris.
And as backup/integration/testing server we can use any server,
virtual or physical, as long as it has a internet connection and
contemporary performance, because the entire install is scripted
and under version control (more below).
Operating System
----------------
So, dogfood: Obviously FreeBSD.
Apart from the obvious reason that I wrote a lot of FreeBSD and
can get world-class support by bugging my buddies about it, there
are two equally serious reasons for the Varnish Project to run on
FreeBSD: Dogfood and jails.
Varnish Cache is not "software for Linux", it is software for any
competent UNIX-like operating system, and FreeBSD is our primary
"keep us honest about this" platform.
Jails
-----
You have probably heard about Docker and Containers, but FreeBSD
have had jails
`since I wrote them in 1998 <http://phk.freebsd.dk/sagas/jails.html>`_
and they're a wonderful way to keep your server installation
sane.
We currently have three jails:
* Hitch - runs the `Hitch SSL proxy <https://hitch-tls.org/>`_
* Varnish - <a href="rimshot.mp3">You guessed it</a>
* Tools - backend webserver, currently `ACME Labs' thttpd <http://acme.com/software/thttpd/>`_
Script & Version Control All The Things
---------------------------------------
We have a git repos with shell scripts which create these jails
from scratch and also a script to configure the host machine
properly.
That means that the procedure to install a clone of the server
is, unabridged::
# Install FreeBSD (if not already done by hosting)
# Configure networking (if not already done by hosting)
# Set the clock
service ntpdate forcestart
# Get git
env ASSUME_ALWAYS_YES=yes pkg install git
# Clone the private git repo
git clone ssh://example.com/root/Admin
# Edit the machines IP numbers in /etc/pf.conf
# Configure the host
sh build_host.sh |& tee _.bh
# Build the jails
foreach i (Tools Hitch Varnish)
(cd $i ; sh build* |& tee _.bj)
end
From bare hardware to ready system in 15-30 minutes.
It goes without saying that this git repos contains stuff
like ssh host keys, so it should *not* go on github.
Backups
-------
Right now there is nothing we absolutely have to backup, provided
we have an up to date copy of the Admin git repos.
In practice we want to retain history for our development tools
(VTEST, GCOV etc.) and I rsync those file of the server on a
regular basis.
The Homepage
------------
The homepage is built with `Sphinx <http://www.sphinx-doc.org/>`_
and lives in its own
`github project <https://github.com/varnishcache/homepage>`_ (Pull requests
are very welcome!)
We have taken snapshots of some of the old webproperties, Trac, the
Forum etc as static HTML copies.
Why on Earth...
---------------
It is a little bit tedious to get a setup like this going, whenever
you tweak some config file, you need to remember to pull the change
back out and put it in your Admin repos.
But that extra effort pays of so many times later.
You never have to wonder "who made that change and why" or even try
to remember what changes were needed in the first place.
For us as a project, it means, that all our sysadmin people
can build a clone of our infrastructure, if they have a copy of
our "Admin" git repos and access to github.
And when `FreeBSD 11 <https://www.youtube.com/watch?v=KOO5S4vxi0o>`_
comes out, or a new version of sphinx or something else, mucking
about with things until they work can be done at leisure without
guess work. (We're actually at 12 now, but the joke is too good
to delete.)
For instance I just added the forum snapshot, by working out all
the kinks on one of my test-machines.
Once it was as I wanted it, I pushed the changes the live machine and then::
varnishadm vcl.use backup
# The 'backup' VCL does a "pass" of all traffic to my server
cd Admin
git pull
cd Tools
sh build_j_tools.sh |& tee _.bj
varnishadm vcl.load foobar varnish-live.vcl
varnishadm vcl.use foobar
For a few minutes our website was a bit slower (because of the
extra Paris-Denmark hop), but there was never any interruption.
And by doing it this way, I *know* it will work next time also.
2016-04-25 /phk
PS: All that buzz about "reproducible builds" ? Yeah, not a new idea.
......@@ -81,35 +81,20 @@ These rules are imported from the X11 project:
* Provide mechanism, rather than policy.
Bundling VMODs with the Varnish distribution
--------------------------------------------
Decisions about whether to add a new Varnish module (VMOD) to those
bundled with Varnish are guided by these criteria.
* The VMOD is known to be in widespread use and in high demand for
common use cases.
* Or, if the VMOD is relatively new, it provides compelling features
that the developer group agrees will be a valuable enhancement for
the project.
* The VMOD does not create dependencies on additional external
libraries. VMODs that are "glue" for a library come from third
parties.
* We don't want to add new burdens of dependency and compatibility
to the project.
Various policies
----------------
* We don't want to force Varnish deployments to install more than
admins explicitly choose to install.
.. toctree::
:maxdepth: 1
* The VMOD code follows project conventions (passes make distcheck,
follows source code style, and so forth).
policy_governance
policy_vmods
The varnish-cache.org homepage
------------------------------
* A pull request can demonstrate that this is the case (after any
necessary fixups).
.. toctree::
:maxdepth: 1
* The developer group commits to maintaining the code for the long run
(so there will have to be a consensus that we're comfortable with
it).
homepage_dogfood
homepage_contrib
.. _policy-governance:
Project Truck Factor and Governance
-----------------------------------
We recognize that the Truck Factor of the Varnish Cache project is low.
As mitigation we have created a Varnish Governance Board (VGB)
and implemented procedures so no project asset have a Truck Factor
less than 3.
The Varnish Governance Board is the ultimate authority in the
project, but does little to nothing on a daily basis.
The important thing is that there is agreement beforehand about who
has the authority to step in, if need be.
Appointment to the VGB is by acclamation, and we prefer the members
to have enough distance and neutrality to be able to resolve
conflicts, should it ever become necessary.
Current members of the VGB:
* Rogier Mulhuijzen
* Lasse Karstensen
* Poul-Henning Kamp
.. _policy-vmods:
Bundling VMODs with the Varnish distribution
--------------------------------------------
Decisions about whether to add a new Varnish module (VMOD) to those
bundled with Varnish are guided by these criteria.
* The VMOD is known to be in widespread use and in high demand for
common use cases.
* Or, if the VMOD is relatively new, it provides compelling features
that the developer group agrees will be a valuable enhancement for
the project.
* The VMOD does not create dependencies on additional external
libraries. VMODs that are "glue" for a library come from third
parties.
* We don't want to add new burdens of dependency and compatibility
to the project.
* We don't want to force Varnish deployments to install more than
admins explicitly choose to install.
* The VMOD code follows project conventions (passes make distcheck,
follows source code style, and so forth).
* A pull request can demonstrate that this is the case (after any
necessary fixups).
* The developer group commits to maintaining the code for the long run
(so there will have to be a consensus that we're comfortable with
it).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment