notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
This referral link gives you 10% off a account and gives me a discount on my Fastmail account.
New feature planned: get notified when the package is available. Now is the time to contribute ideas/suggestions.
non port: Tools/portbuild/scripts/dopackages

Number of commits found: 41

Sunday, 24 Apr 2011
16:37 flz search for other commits by this committer
Remove portbuild scripts from pcvs, as they now live in svn/projects.
Original commit
Wednesday, 1 Dec 2010
02:30 linimon search for other commits by this committer
Canonicalize the use of SRC_BASE to fix INDEX and duds builds on head
nodes that don't have /usr/src.  On pointyhat, this "just happens to
work"; on a cleanly installed system like pointyhat-west, it didn't.
Original commit
Monday, 16 Aug 2010
06:22 linimon search for other commits by this committer
Add some missing error handling.
Original commit
Tuesday, 10 Aug 2010
05:39 linimon search for other commits by this committer
Correct some misleading documentation.
Original commit
Friday, 25 Jun 2010
23:14 linimon search for other commits by this committer
Generalize the packge building scripts to be able to be run on more than
one 'head' node, rather than just pointyhat itself.

Constants are factored out into installation-specific files known as
portbuild/conf/server.conf and portbuild/conf/client.conf.  There is
only one server.conf file.  Individual <arch> directories may have
their own client.conf files, or may symlink to ../conf/client.conf.

Several bugs are fixed and improvements are made:

 - the name of the journal file is changed from 'make' (ew) to
 - the 'date' parameter becomes 'datestamp' to delinate the script's
   uses of the date command vs. the parameter.  This is mostly for
 - add the -nochecksubdirs option.  This is only useful for restarted
   or incremental runs.
 - refactor the error returns a bit.
 - exit gracefully if no INDEXFILE.
 - fix a bug in the 'packages built' logic that would return -2 if no
   packages were actually built.
 - change the misnamed 'cvsdone' file to '.updated'.
 - clean up the 'task waiting' logic.

Feature safe:   yes
Original commit
Sunday, 23 May 2010
03:09 linimon search for other commits by this committer
Remove obsolete cruft.
Original commit
Tuesday, 10 Nov 2009
08:02 linimon search for other commits by this committer
Harden this script so it will DTRT if any subdirectories of a new build,
or pieces of a pre-existing build, are missing.  These are both the case
when dopackages follows 'build create'.
Original commit
Wednesday, 12 Aug 2009
08:18 erwin search for other commits by this committer
Also put bzipped and checksums of INDEX, UPDATING and MOVED on the mirrors

PR:             131440
Requested by:   Hannes Hauswedell <>
Original commit
Tuesday, 11 Aug 2009
07:51 erwin search for other commits by this committer
- Upload MOVED and UPDATING to the packages directory to make it
  available on the ftp mirrors alongside the packages [1]
- While I'm here, remove a NOOP check for FreeBSD 4.x -exp

PR:             135024 [1]
Requested by:   Dominic Fandrey <>
Original commit
Thursday, 21 May 2009
16:17 pav search for other commits by this committer
- Update to reflect a reality on pointyhat
Original commit
Saturday, 26 Jul 2008
14:49 kris search for other commits by this committer
* Cleanup

* Catch up to build ID directory changes

* Improve usage()

* Fix a variety of small bugs

* Remove support for -ftp builds: we have not supported direct
  uploading for many years due to the desire to manually inspect build
  output for quality

* All data associated to a build is now localized in its own directory
  named according to a build ID:
  /var/portbuild/${arch}/${branch}/builds/${buildid}, where ${buildid}
  is the creation time.  These are actually ZFS filesystems.

* Tasks such as cloning a new build, updating a ZFS snapshot, and
  cleaning up a build are exported to the "build" script, which can be
  used independently.

* Creating a new build is done by ZFS cloning and takes a couple of
  seconds since it is copy-on-write (i.e. no data needs to be copied).

* Ports and source trees are also cloned from pre-updated ZFS images
  (updated regularly from the "updatesnap" cron job).  In most cases
  we do not care if we are building a ports tree that is an hour or so
  old since it will become outdated almost immediately anyway, so no
  matter what we do there will be times when a port has been fixed by
  the time the build error is generated by a client.

* In case an up-to-the-second tree is desired, the -portscvs and
  -srccvs switches update the existing ports tree via CVS.

* -noports and -nosrc can be used to prevent any automatic changes to
   the ports tree.  This is useful for dealing with local
   modifications (e.g. for -exp builds), since the default when
   creating a new build is to replace the previous trees with fresh,
   pristine trees.  If you forget to use this then any local changes
   that are not also present in other trees will be lost.

* By default we keep two builds for each arch/branch pair.  These
  build IDs also may be referred to via "latest" and "previous"
  symlinks.  When creating a new build, the old "previous" build is
  destroyed by default, unless it was originally created using the
  -keep switch.  This prevents the build from being destroyed

* By default when a build finishes all of the clients are completely
  cleaned up (i.e. all build data such as ports trees, tarballs,
  client chroots, etc are deleted).  This is needed to save space on
  the clients.  If you expect to *immediately* perform further builds
  after this one completes, the -nocleanup switch prevents this step.
  Otherwise they will just be set up again if further builds are

* Try to parallelize build pre-processing as much as possible, by
  running jobs in the background wherever possible.  In several places
  we operate on the same parts of the filesystem from multiple jobs,
  so we can make good use of caching to improve performance

* Clients no longer need to be set up explicitly at the start of the
  build, they will be set up on-demand when the first job is
  dispatched to them.  This allows fast clients or those that already
  have been set up to begin building ports as soon as possible, while
  slow clients are set up in the background.  It also improves
  robustness of client recovery, e.g. if the client was offline at the
  time of build startup but later brought back online.

* Optimize copying back in the previous set of restricted packages by
  hardlinking instead of copying.

TODO: The record of failed ports is arch/branch-global still.  This is
the only thing preventing us from running concurrent builds of the
same arch/branch (e.g. while one is stuck building openoffice, the
next build can start to keep the cluster busy).  The difficulty is
that one build from a later ports tree may signal that a build was
successful, then a phase 2 build from an earlier ports tree may
indicate that it was broken.  The solution is probably to migrate this
to a real database instead of a flat file, and query it for the set of
broken ports as of a certain ports tree date.
Original commit
Wednesday, 11 Jun 2008
13:28 kris search for other commits by this committer
* Distfile collection is now the default; replace -distfiles with -nodistfiles

* Record the CVS update stamp in some extra places and make sure to remove it
  if the build is started with -noportscvs (since this probably means the
  ports tree was updated by hand at some random time)
Original commit
Monday, 14 Jan 2008
17:10 erwin search for other commits by this committer
Remove support for 5-exp and add 7-exp, 8 and 8-exp
Original commit
Monday, 24 Dec 2007
01:56 linimon search for other commits by this committer
Copy the duds file to a directory where the web server can find it.  This
makes it possible to correctly analyze why packages were not built for a
specific run.

Add a beginning and ending email notification to help coordinate between
multiple portmgrs doing runs.
Original commit
Sunday, 4 Nov 2007
14:28 kris search for other commits by this committer
Add support for 7-exp and 8 branches

Approved by:    portmgr (self)
Original commit
Sunday, 29 Jul 2007
19:38 kris search for other commits by this committer
* Remove support for 4 and 4-exp builds and add 6-exp2 experimental build
* Only hardlink the old log files instead of anything else that might be
  in the directories
* Add comment that old logfiles should be removed as well as packages, to
  avoid duplicate versions of the same port log
Original commit
Sunday, 18 Feb 2007
08:55 kris search for other commits by this committer
Index building needs to happen before prunefailures is run, otherwise it
will completely erase the list.

When using -continue, don't append to the duds.errors, we don't want to
preserve the old cruft from a previous build
Original commit
Thursday, 14 Sep 2006
05:43 kris search for other commits by this committer
* Back up duds to duds.old before, not after we create the new one

* Add more XXX comments for future work

* Only record a cvsdone timestamp if we updated cvs

* When building with -trybroken, it's safe (and desirable) to run the
  prunefailure script

* Reorganise a few things for better parallelism

* Instead of keeping a duplicate copy of the previous logs and errors
  under bak/, just store a symlink to the archival location

* When doing an incremental build, also cycle out the old logs to avoid
  broken links on the website (the logs from the previous build are
  removed until the packages are rebuilt).  Use cpio to create
  hardlinked copies of the previous logs.  XXX when these are bzipped
  by cron to save space the links will be broken and it might actually
  take more space.

* Don't bother bunzipping old logs, now that the processlogs scripts
  can handle it.  This was a waste of time anyway since they'd all be
  rebzipped by the next nightly cron job.

* When the build is complete, stash a copy of the restricted ports in
  bak/restricted/ before deleting them from packages/, and restore from
  here when doing an incremental build to avoid needlessly rebuilding
  them each time.
Original commit
Sunday, 16 Apr 2006
02:27 kris search for other commits by this committer
Don't bother archiving the ports tree, we don't use it for anything so it
just takes up time.
Original commit
Monday, 3 Apr 2006
04:41 kris search for other commits by this committer
Do not preserve the previous package sets in bak/.  This is taking up
too much space on pointyhat, and it's also very slow.
Original commit
Wednesday, 15 Feb 2006
07:34 kris search for other commits by this committer
Add support for 6-exp builds (FreeBSD 6.x is our new reference platform)
Original commit
Sunday, 22 Jan 2006
04:55 kris search for other commits by this committer
Remove -nodoccvs option, it turns out mounting the doc tree is not
required by any ports
Original commit
Tuesday, 11 Oct 2005
03:53 kris search for other commits by this committer
Change lockf to lockf -k in a commented-out command, just in case it's
ever uncommented.
Original commit
03:31 kris search for other commits by this committer
* Remove unused -nodummy argument

* Finish flipping the switch on -noplistcheck; set NOPLISTCHECK instead
  of PLISTCHECK in the opposite case

* Clean up the distfiles/ directory when starting build, so it is not
  contaminated by old distfiles

* Remove commented-out code to don't back up old distfiles/ directory,
  it's just too big to do this.
Original commit
Sunday, 24 Jul 2005
20:35 kris search for other commits by this committer
Support FreeBSD 7.x
Original commit
Saturday, 2 Apr 2005
20:18 kris search for other commits by this committer
Flip over the -plistcheck switch to -noplistcheck since we now want to
make it the default behaviour.
Original commit
Sunday, 9 Jan 2005
10:21 krion search for other commits by this committer
Remove traling spaces.
Original commit
Thursday, 6 Jan 2005
23:06 kris search for other commits by this committer
* The mlist file now only has one field per line.

* Correctly calculate job concurrency in pass two as well

* In pass two run dosetupnode -rsync to avoid wasting time.

Approved by:    portmgr (self)
Original commit
Tuesday, 28 Dec 2004
05:45 kris search for other commits by this committer
* It does in fact make sense to specify -incremental and -noindex at the
  same time, assuming that the admin has already built the INDEX and
  INDEX.old in advance.

* Adapt to new method of calculating build concurrency, by summing the
  value of ${maxjobs} listed in every portbuild.${machine}

* Support 5-exp builds
Original commit
Thursday, 25 Nov 2004
00:06 kris search for other commits by this committer
Rework the -incremental support slightly; this should now be robust enough
to use.
Original commit
Monday, 25 Oct 2004
04:34 kris search for other commits by this committer
Redefine the -norestr option to mean 'don't build the list of
restricted ports' instead of 'don't build any restricted ports' since
the former is useful when we're not intending to publish the results
of a build, but the latter is not.

Move the build preprocessing (directory setup, old build rotation,
etc) out from under -nobuild, so that we can set up a new build using
that option.
Original commit
Sunday, 26 Sep 2004
22:09 kris search for other commits by this committer
* Break out setupnode into the dosetupnode script so it may be called

* Support 6.x builds

Approved by:    portmgr (self)
Original commit
Wednesday, 14 Jul 2004
10:09 kris search for other commits by this committer
* Remove the quickports stuff, it didn't (ever?) work as designed and
  it's done properly^Wbetter in makeparallel

* Script accepts new arguments:

 -nodoccvs: skip cvs update of the doc tree

 -trybroken: try to build BROKEN ports (off by default because the
  i386 cluster is fast enough now that when doing incremental builds we
  were spending most of the time rebuilding things we know are probably
  going to fail anyway.  Conversely, the other clusters are slow enough
  that we also usually don't want to waste time on BROKEN ports).

 -incremental: compare the interesting fields of the new INDEX with
  the previous one, remove packages and log files for the old ports that
  have changed, and rebuild the rest.  This substantially cuts down on
  build times since we don't rebuild ports that we know have not
  changed.  XXX checkpoint of work-in-progress, not yet working as

* When setting up the nodes, read in per-node config files
  ("portbuild.$(hostname)") before dispatching the setupnode script on
  each node.  For disconnected nodes (which don't mount the master via
  NFS), we also rsync the interesting files required by the builds
  (ports/src/doc trees, bindist tarballs, scripts) into place on the
  client.  They will be mounted locally via nullfs in the build chroots.

* Break out the generation into a makerestr script so it
  can be called manually as needed.

* Remove the -nocvsup argument which has been unused for a long time.

* For now, don't prune the list of failed ports with prunefail,
  since when -trybroken is not specified, every BROKEN port ends up in
  the duds file (so the build is skipped), and as a result we would
  prune almost everything from the list of failed ports.  XXX
  prunefailure should be run conditionally on -trybroken, or I should
  find a way to prune in both cases.

* Don't run index in the background, it was thrashing against makeduds
  and not saving any time by doing it concurrently.

* Build with 'make quickports all' to kick off the quickports builds

* Delete restricted and/or cdrom distfiles *after* post-processing the
  distfiles, otherwise the script doesn't remove any of them since
  they're not in the expected place.

* Miscellaneous other minor changes and cleanups
Original commit
Monday, 8 Mar 2004
01:38 kris search for other commits by this committer
Replace gnome with gnome2 in quickports.  I'm still not sure this actually
works as intended.
Original commit
Thursday, 15 Jan 2004
21:56 marcus search for other commits by this committer
Use editors/openoffice-1.1 instead of editors/openoffice since the latter is
now gone.
Original commit
Sunday, 18 May 2003
00:04 kris search for other commits by this committer
Add a -fetch-original switch to dopackages that fetches the distfiles from
the listed MASTER_SITES instead of ftp-master.  Pass the FETCH_ORIGINAL
environment variable to pdispatch and as a command-line switch to
Original commit
Saturday, 17 May 2003
00:25 kris search for other commits by this committer
Add a -distfiles flag to specify that we should retain distfiles for later
processing (e.g. transfer to ftp site).  It was taking up too much disk
space to collect distfiles for 14 architecture/version combinations
(including the backup set from the previous run).

Put the cvsdone files in the arch/branch subdirectory, not arch/.
Original commit
Sunday, 6 Apr 2003
05:40 kris search for other commits by this committer
Zap some old/non-useful code
Add some new command-line options:
  * -continue : restart the build without rebuilding ports that have
                already failed
  * -nofinish : don't run the post-processing steps when the build finishes
  * -finish : run the post-processing steps only
Reorder some code sections appropriately
Original commit
Friday, 24 Jan 2003
04:45 kris search for other commits by this committer
- Use the ${arch} subdirectory and pass ${arch} around to other scripts
- Use makeparallel script instead of 'make parallel'
- Support building packages as a non-privileged user (needs root access
  to client machines)
Original commit
Friday, 8 Nov 2002
22:20 kris search for other commits by this committer
Catch up to lots of changes from bento/beta:

* Update list of quickports to include kde3

* Add ability to restart builds (-restart)

* Change default to build packages for non-cdrom use (associated option name
  inverted from -nocdrom to -cdrom)

* Cope with .tgz or .tbz packages

* Use buildenv framework

* Use cvs update -PdA instead of -Pd when updating ports/docs tree

* Always delete restricted packages after a build run, since they are
  published on the website.

* Minor cosmetic changes
Original commit
Saturday, 16 Mar 2002
05:48 kris search for other commits by this committer
Pull environment variables from buildenv, and tidy up a bit.
Original commit

Number of commits found: 41