Explain how our new server setup works.

R1/source/index.rst

@@ -13,16 +13,25 @@ Please fill the `Varnish User Survey 2016
 Whats happening
 ---------------
 
+2016-04-25
+~~~~~~~~~~
+
+If you are interested, here is an outline of the
+:ref:`projects new IT infrastructure. <20160425_website>` /phk
+
+
 2016-04-11
+~~~~~~~~~~
 
 Sorry if it looks a bit sparse around here, we're in the process
 of moving the Varnish Cache project to its own infrastructure.
 
 If you want to help out, we will gladly take pull requests to the
 new homepage, you can find it in
-`this github repos <https://github.com/varnishcache/homepage>`_.
+`this github repos <https://github.com/varnishcache/homepage>`_.  /phk
 
-/phk
+Future Events
+-------------
 
 2016-05-05 (San Francisco)
 `Cinco de Mayo / High Scale Varnish Meetup <http://www.meetup.com/San-Francisco-Varnish-Meetup/events/230233385/>`_
@@ -36,6 +45,7 @@ new homepage, you can find it in
 2016-07-16 & 17 (Amsterdam) `VarnishCon Europe 2016
 <http://varnishcon2016.eventbrite.com>`_
 
+
 Privacy
 -------
 

R1/source/news/20160425_website.rst

+.. _20160425_website:
+
+How our website works
+=====================
+
+One of the major reasons for the website upgrade the Varnish Project
+has been going through this month, was in an effort to eat more of
+our own dogfood.
+
+The principle of eating your own dogfood is important for software
+quality, that is how you experience what your users are dealing with
+and I am not the least ashamed to admit that several obvious improvements
+have already appeared on my TODO list as a result of this transition.
+
+But it is also important to externalize what you learn doing so, and
+therefore I thought I would document here how the projects new "internal
+IT" works.
+
+Hardware
+--------
+
+Who cares?
+
+Yes, we use some kind of hardware, but to be honest I don't know what
+it is.
+
+Our primary site runs on a `RootBSD 'Omega' <https://www.rootbsd.net/>`_
+virtual server somewhere near CDG/Paris.
+
+And as backup/integration/testing server we can use any server,
+virtual or physical, as long as it has a internet connection and
+contemporary performance, because the entire install is scripted
+and under version control (more below)
+
+Operating System
+----------------
+
+So, dogfood:  Obviously FreeBSD.
+
+Apart from the obvious reason that I wrote a lot of FreeBSD and
+can get world-class support by bugging my buddies about it, there
+are two equally serious reasons for the Varnish Project to run on
+FreeBSD:  Dogfood and jails.
+
+Varnish Cache is not "software for Linux", it is software for any
+competent UNIX-like operating system, and FreeBSD is our primary
+"keep us honest about this" platform.
+
+Jails
+-----
+
+You have probably heard about Docker and Containers, but FreeBSD
+have had jails 
+`since I wrote them in 1998 <http://phk.freebsd.dk/sagas/jails.html>`_
+and they're a wonderful way to keep your server installation
+sane.
+
+We currently have three jails:
+
+* Hitch - runs the `Hitch SSL proxy <https://hitch-tls.org/>`_
+
+* Varnish - <a href="rimshot.mp3">You guessed it</a>
+
+* Tools - backend webserver, currently `ACME Labs' thttpd <http://acme.com/software/thttpd/>`_
+
+Script & Version Control All The Things
+---------------------------------------
+
+We have a git repos with shell scripts which create these jails
+from scratch and also a script to configure the host machine
+properly.
+
+That means that the procedure to install a clone of the server
+is, unabridged::
+
+	# Install FreeBSD 10.3 (if not already done by hosting)
+	# Configure networking (if not already done by hosting)
+	# Set the clock
+	service ntpdate forcestart
+	# Get git
+	env ASSUME_ALWAYS_YES=yes pkg install git
+	# Clone the private git repo
+	git clone ssh://example.com/root/Admin
+	# Edit the machines IP numbers in /etc/pf.conf
+	# Configure the host
+	sh build_host.sh |& tee _.bh
+	# Buld the jails
+	foreach i (Tools Hitch Varnish)
+		(cd $i ; sh build* |& tee _.bj)
+	end
+
+From bare hardware to ready system in 15-30 minutes.
+
+It goes without saying that this git repos contains stuff
+like ssh host keys, so it should *not* go on github.
+
+Backups
+-------
+
+Right now there is nothing we need to backup.
+
+When I move the mailserver/mailman/mailing lists over, those will
+need to be backed up, but here the trick is to *only* backup the
+minimal set of files, and in a "exchange" format, so that future
+migrations and upgrades can slurp them in right away.
+
+The Homepage
+------------
+
+The new homepage is built with `Sphinx <http://www.sphinx-doc.org/>`_
+and lives in its own
+`github project <https://github.com/varnishcache/homepage>`_ (Pull requests
+are very welcome!)
+
+We have taken snapshots of some of the old webproperties, Trac, the
+Forum etc as static HTML copies.
+
+Why on Earth...
+---------------
+
+It is a little bit tedious to get a setup like this going, whenever
+you tweak some config file, you need to remember to pull the change
+back out and put it in your Admin repos.
+
+But that extra effort pays of so many times later.
+
+You never have to wonder "who made that change and why" or even try
+to remember what changes were needed in the first place.
+
+For us as a project, it means, that all our sysadmin people
+can build a clone of our infrastructure, if they have a copy of
+our "Admin" git repos and access to github.
+
+And when `FreeBSD 11 <https://www.youtube.com/watch?v=KOO5S4vxi0o>`_
+comes out, or a new version of sphinx or something else, mucking
+about with things until they work can be done at leisure without
+guess work.
+
+For instance I just added the forum snapshot, by working out all
+the kinks on one of my test-machines.
+
+Once it was as I wanted it, I pushed the changes the live machine and::
+
+Then::
+
+	varnishadm vcl.use backup
+	# The 'backup' VCL does a "pass" of all trafic to my server
+	cd Admin
+	git pull
+	cd Tools
+	sh build_j_tools.sh |& tee _.bj
+	varnishadm vcl.load foobar varnish-live.vcl
+	varnishadm vcl.use foobar
+
+For a few minutes our website was a bit slower (because of the
+extra Paris-Denmark hop), but there was never any interruption.
+
+And by doing it this way, I *know* it will work next time also.
+
+2016-04-25 /phk
+
+All that buzz about "reproducible builds" ?  Yeah, not a new idea.
+

R1/source/news/index.rst

 News, Dates & Events
 ====================
 
+* :ref:`2016-04-25 How our website works <20160425_website>`
+
 * :ref:`2016-04-11 New Website <20160411_website>`
 
 Next major release
@@ -11,6 +13,7 @@ Next major release
 .. toctree::
 	:hidden:
 
+	20160425_website.rst
 	20160411_website.rst
 
 Past events