notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
Ukraine
This referral link gives you 10% off a Fastmail.com account and gives me a discount on my Fastmail account.
New feature planned: get notified when the package is available. Now is the time to contribute ideas/suggestions.
non port: Tools/portbuild/scripts/pdispatch

Number of commits found: 30

Sunday, 24 Apr 2011
16:37 flz search for other commits by this committer
Remove portbuild scripts from pcvs, as they now live in svn/projects.
Original commit
Wednesday, 26 Jan 2011
10:41 linimon search for other commits by this committer
Change the "unclean build" handling case to not retry.  Previously, it
was handled the same as "waiting for chroot" and retried.  In certain
circumstances (e.g. a prerequisite package shown as being avaialable
when it really wasn't), this would cause a package build to loop
forever.

Note to future generations: I did not write this code.

Feature safe:   yes
Original commit
Monday, 24 Jan 2011
01:58 linimon search for other commits by this committer
Fix typo in fixed-up tar command to suppress a whoooole lot of junk
output on package builds.

Pointy hat:     self
Feature safe:   yes
Original commit
Sunday, 23 Jan 2011
03:06 linimon search for other commits by this committer
Add anti-foot-shooting measure.

Feature safe:   yes
Original commit
Friday, 17 Dec 2010
01:34 linimon search for other commits by this committer
Remove debugging code that crept in.
Original commit
Thursday, 16 Dec 2010
07:34 linimon search for other commits by this committer
If the logfile was not correctly copied from the client back to the server,
do not attempt to process it.
Original commit
Wednesday, 1 Dec 2010
02:27 linimon search for other commits by this committer
Although I was going to try to avoid it, it turns out that it is
impossble to have merely "server.conf" and "client.conf" control
everything on a head node.  Some pieces must be refactored into
"common.conf".
Original commit
Friday, 25 Jun 2010
23:27 linimon search for other commits by this committer
Generalize the packge building scripts to be able to be run on more than
one 'head' node, rather than just pointyhat itself.

Constants are factored out into installation-specific files known as
portbuild/conf/server.conf and portbuild/conf/client.conf.  There is
only one server.conf file.  Individual <arch> directories may have
their own client.conf files, or may symlink to ../conf/client.conf.

Feature safe:   yes
Original commit
Thursday, 21 May 2009
16:17 pav search for other commits by this committer
- Update to reflect a reality on pointyhat
Original commit
Saturday, 26 Jul 2008
14:01 kris search for other commits by this committer
* Cleanup
* Catch up to build ID directory changes
* Add support for ssh_cmd and scp_cmd to allow using HPN-SSH with the
  none cipher where possible (for performance)
* Lazy client setup; claim-chroot will report if the client needs to be
  set up with this buildid, and we initiate the setup and poll until
  it is complete.  This allows fast clients to begin building before
  slow ones have finished setting up.

TODO: a better solution would be to avoid trying to dispatch jobs onto
clients that are in the process of setting up, since they often have low
loads and are picked preferentially by the job scheduler.
Original commit
Monday, 24 Dec 2007
14:25 linimon search for other commits by this committer
Add the standard 6-line header and URL to all build failure logs, not
just the plist ones.  If the log is less than 1000 lines after the header,
include it all; else, trim to last 1000 lines.

This should help when deciding where to forward logs.

Tested on:      pointyhat
Original commit
Sunday, 18 Feb 2007
08:58 kris search for other commits by this committer
Increase global build timeout to 100 hours
Original commit
Thursday, 14 Sep 2006
05:24 kris search for other commits by this committer
* Drop alpha support

* Increase sparc64 build timeout to 24 hours (we have so few build
  machines that we cannot afford to tie them up for longer)

* Increase other arch build timeout to 100 hours (hello openoffice!)

* If we successfully build a formerly broken package, touch errors/.force
  which will kick off a rebuild of the html files
Original commit
Monday, 13 Mar 2006
03:05 kris search for other commits by this committer
Temporarily back out the pre.log change, it doesn't work properly.

Approved by:    portmgr (implicit)
Original commit
Wednesday, 15 Feb 2006
07:44 kris search for other commits by this committer
Save the output of portbuild run on the client to a temporary log file.
If portbuild bailed out unexpectedly, mail the log to ${mailto}.

Add some XXX comments about improving robustness of this script.

Sleep for 2 minutes before retrying builds, to avoid spamming ${mailto}
with a high rate of failure logs.  In future we might be smarter about
attempting to automatically correct common failure modes.
Original commit
Saturday, 11 Feb 2006
09:49 kris search for other commits by this committer
* Use ${mailto} portbuild.conf variable instead of hard-coded list [1]

* Remove old logs and possible compressed logs before attempting the build

Requested by:   lofi [1]
Submitted by:   linimon [1]
No more accidental portbuild spam:      kris and krion [1]
Original commit
Friday, 3 Feb 2006
20:08 kris search for other commits by this committer
Flip "noclean" to "clean"
Original commit
Tuesday, 11 Oct 2005
03:47 kris search for other commits by this committer
* Use lockf -k instead of lockf.  If there is ever contention from >1
  process, you must use -k or lockf can give out two locks at once
Original commit
03:36 kris search for other commits by this committer
* Remove support for the unused -nodummy function

* Finish flipping the switch on -noplistcheck - this is activated by
  passing in the NOPLISTCHECK environment variable instead of
  using PLISTCHECK in the opposite case

* Always pull in the distfiles from the client if they exist (needed
  for forthcoming ports tree changes to satisfy GPL license requirements)

* If the build did not complete "cleanly", e.g. it was interrupted by a
  network outage or client machine panic, then retry it until it succeeds
  instead of just leaving a dirty truncated log
Original commit
Sunday, 24 Jul 2005
20:38 kris search for other commits by this committer
Check for extra files on !i386 also
Original commit
Thursday, 6 Jan 2005
23:07 kris search for other commits by this committer
* Include portbuild.conf before portbuild.${hostname} to reset the default
value of variables

* Allow krion to drink from the email firehose

Approved by:    portmgr (self)
Original commit
Tuesday, 28 Dec 2004
05:40 kris search for other commits by this committer
Overhaul of the job scheduler.  The new scheduler runs builds
synchronously instead of probabilistically scheduling jobs, which
means that the job load on a machine never exceeds a desired
threshold, and we can preferentially use faster machines when they are
available.  This has a dramatic effect on package build throughput,
although I don't yet have precise measurements of the performance
improvements.

Specifically, the changes are:

* Introduce the new variable maxjobs in portbuild.  This replaces the
build scheduling weights previously listed in the mlist file, which
now changes format to list the build machines only, ranked in order of
preference for job dispatches (i.e. faster machines first).

* The ${arch}/queue directory is used to list machines available for
jobs (file content is the number of jobs currently running on the
machine).  Changes to files in this directory are serialized using
lockf on the .lock file.

* Claim a machine with the getmachine script, with the .lock held.
This picks the machine with the fewestnumber of jobs running, which is
listed highest in the mlist file in case of multiple machines with
equal load.  The job counter is incremented, and the file removed if
the counter reaches ${maxjobs} for that machine.  If all machines are
busy, sleep for 15 seconds and retry.

* After we have claimed a machine, we run claim-chroot on it to claim
an empty chroot, as before.  If the claim fails, release the job from
the queue with the releasemachine script and retry after a 15 second
wait.

* When the build is finished, decrement the job counter with the
releasemachine script, with .lock held.

* The checkmachines script now exists only to poll the load averages
for admin convenience (every 2 minutes), and to ping for unreachable
machines.  When a machine cannot be reached, remove the entry in the
queue directory to stop further job dispatches to it.  This needs more
work to deal with reinitialization of machines after they become
available again.

Additional changes to this file:

* Exit if passed a null package name, to avoid badness later on

* Send a nag-mail if pkg-plist errors are detected in the build
Original commit
Sunday, 26 Sep 2004
22:04 kris search for other commits by this committer
* Bail out if we can't claim a chroot directory in the client (e.g. if
ssh times out)

* Support new portbuild.conf settings:

    client_user = user to connect to on the client (not necessarily root)

    sudo_cmd = If ssh'ing to a non-root user, run this command to gain
               root privs (set to empty string for client_user=root,
               or sudo for !root).  Cannot require interactivity, of
               course.

Approved by:    portmgr (self)
Original commit
Wednesday, 14 Jul 2004
10:33 kris search for other commits by this committer
* Add support for ${TRYBROKEN}.

* Clients no longer have ssh access to the master, so we need to
  push/pull everything on the client from here.  This means we need to
  know where the build took place so we can go in and get the files
  after it finishes.  Introduce the claim-chroot script which
  atomically claims a free chroot directory on the host and returns
  the name.  This directory is later populated by the portbuild script
  if it does not already contain an extracted bindist.

* Use the per-node portbuild.$(hostname) config file to decide where
  in the filesystem to claim the chroot on the build host.

* If a port failed unexpectedly (i.e. is not marked BROKEN), or if
  something strange happened when trying to pull in build results from
  a client, then send me email (XXX should be configurable).

* Clean up after the build finishes and we have everything we need, by
  dispatching the clean-chroot script on the client.
Original commit
Monday, 8 Mar 2004
01:25 kris search for other commits by this committer
Increase timeout on slow build machines (alpha, sparc64) to 16 hours.
This should probably be made a per-machine variable.
Original commit
Sunday, 18 May 2003
00:04 kris search for other commits by this committer
Add a -fetch-original switch to dopackages that fetches the distfiles from
the listed MASTER_SITES instead of ftp-master.  Pass the FETCH_ORIGINAL
environment variable to pdispatch and as a command-line switch to
portbuild.
Original commit
Saturday, 17 May 2003
00:25 kris search for other commits by this committer
Pass down the request to keep distfiles if appropriate.
Original commit
Friday, 14 Feb 2003
09:26 kris search for other commits by this committer
Work around broken rand() in -current by changing the way we pick a
random build host.
Original commit
Friday, 24 Jan 2003
04:49 kris search for other commits by this committer
- Use ${arch}
- Increase timeout to 8 hours (this needs to be made per-arch so it
  doesn't overly pessimize fast client machines)
- Support building as a non-privileged user
Original commit
Sunday, 15 Sep 2002
21:57 kris search for other commits by this committer
Use PKGSUFFIX from buildenv
Original commit

Number of commits found: 30