16:37 flz 

Remove portbuild scripts from pcvs, as they now live in svn/projects.

02:27 linimon 

Although I was going to try to avoid it, it turns out that it is
impossble to have merely "server.conf" and "client.conf" control
everything on a head node.  Some pieces must be refactored into
"common.conf".

22:49 linimon 

Generalize the packge building scripts to be able to be run on more than
one 'head' node, rather than just pointyhat itself.

Constants are factored out into installation-specific files known as
portbuild/conf/server.conf and portbuild/conf/client.conf.  There is
only one server.conf file.  Individual <arch> directories may have
their own client.conf files, or may symlink to ../conf/client.conf.

This first set of files only gets the *.conf files and some cleanup.

Feature safe:   yes

03:58 linimon 

Improve error handling a bit.

14:11 kris 

* Cleanup
* Catch up to build ID directory changes
* Improved support for ZFS builds
* Improved robustness
* Report status verbosely to the caller; whether we succeeded in claiming
  a chroot, whether the caller needs to first set up the client, or
  whether a setup is in progress.
* If we discover that the client has not been set up either because it
  freshly booted and newfs'ed its filesystem, or because a particular
  build has not yet been encountered, atomically claim a cookie and
  report this to the caller to act on

20:27 kris 

Support builds using zfs and tmpfs backing store

zfs:

* Enabled by use_zfs=1 in portbuild.conf

* Populate build chroots by cloning a zfs snapshot instead of maintaining
  many duplicate copies.  In principle this is very efficient since
  everything is copy-on-write and zfs snapshot creation is almost
  instantaneous.  There might be additional overheads from building on zfs
  though.  Currently the snapshot base is hard-wired to y/${branch}@base
  but should be parametrized.  This also must be populated beforehand, e.g.
  during machine startup

* Clean build chroots by just destroying the snapshot.

tmpfs:

* Enabled by use_tmpfs=1 and tmpfs_size in portbuild.conf

* The previous md strategy of mounting in used/, populating and then
  remounting (to avoid possible races from multiple builds claiming the
  same chroot) doesn't work here because tmpfs instances are destroyed at
  umount.  I am not entirely sure the simpler approach is free from races.

05:13 kris 

Silence the clean-chroot invocation

04:40 kris 

The squid rc.d script no longer has the .sh suffix

03:04 kris 

Mount the md image async here too

Approved by:    portmgr (implicit)

07:42 kris 

Add some sanity checking of the build environment:

* Test whether squid is running.  If not, try to kick off
  the rc script in the background in case it can be restarted
  cleanly.

* Test for at least 100MB of free space on the scratch partition.

If either condition fails, set an exception flag and bail out.  This
will be reported back to the server via reportload.

20:07 kris 

Also look for chroots that are marked 'dirty' (previous build left chroot
in unclean state) and run clean-chroot on them to remove them.

04:31 kris 

Mount the md image async since this gives much better performance

03:52 kris 

Add support for use_md_swap.  When this variable is set in a
portbuild.conf, builds will each be done in a separate swap-backed md.
This dramatically improves build performance since

* Every transaction is not written to disk, so disk bandwidth is not
  a bottleneck

* Multiple builds do not contend with each other for the same set of
  filesystem locks and other per-device resources

The size of the md devices is controlled by the md_size variable.  '2g'
seems to be a good size.

Currently we mdconfig -u each device after each port build, since
otherwise dirty blocks accumulate and the md eventually uses a full
amount of backing store (2g in the above example).  This is a problem
if there is unsufficient swap backing to accomodate them all.

XXX This should be made configurable to avoid the performance penalty on
systems that do have enough swap backing

20:35 kris 

Correct typo

10:44 kris 

Atomically claim an unused chroot directory.  The previous algorithm
was not atomic and could have ended up with two port builds trying to use
the same chroot.  Report the claimed directory back to the caller.