14:18 Mateusz Piotrowski (0mp) 

framework: Fix fetch-url-list and fetch-urlall-list

The fetch-url-list and fetch-urlall-list targets are meant to produce
a list of URLs from which a port fetches its distfiles and patches.

Currently, those targets were not working as expected as they print
parts of the output meant for other targets like the fetch-list target.

For example:

    mateusz.piotrowski@server /ports/x11-servers/xwayland-devel$ make
fetch-urlall-list
    [...]
    mkdir -p "xorg" &&
    -o xorg/proto-xorgproto-824001c947cb1962209c6a8f2c63c2637877220d_GL0.tar.gz
    [...]

This patch prevents do-fetch.sh from printing the "mkdir" line and the
"-o" line.

While here:

- Remove the outdated comment about escaping. There is no excessive
  escaping happening there anymore.
- Indent cases in the case statement to match the style of the rest of
  the file.

Reviewed by:	pizzamig
Approved by:	portmgr (pizzamig)
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D40808

    5457f71

14:27 Mathieu Arnold (mat) 

framework: Remove $FreeBSD$

Where appropriate fiddle with a few other things.

    5d33e04

13:29 mat 

Add set pipefail in most framework scripts.

set pipefail changes the pipeline return status from being the return
status of the last command to the last non 0 exit status of any command
in the pipeline.  This is needed to make sure all the commands in a
pipeline did actually return a non 0 status and not only the last one.

PR:		250723
Exp-run by:	antoine
Reviewed by:	bapt
Approved by:	bapt
Differential Revision:	https://reviews.freebsd.org/D27007

 

16:51 mat 

Backout r554139.

 

16:42 mat 

Add set pipefail in most framework scripts.

set pipefail changes the pipeline return status from being the return
status of the last command to the last non 0 exit status of any command
in the pipeline.  This is needed to make sure all the commands in a
pipeline did actually return a non 0 status and not only the last one.

 

10:10 mat 

Fix fetch-list when running as a user.

The fetch-list target is used to generate a shell script that will more
or less replicates what do-fetch does. It allows one to do most things
as a regular user, and generate that script to run, say, on another
machine, if the one where you build things does not have access to the
internet, or has much slower access.

It was failing when DISTDIR was not writable by the current user, and
the port had a distribution file with a path in it. (Not using
DIST_SUBDIR, something else, like lang/rust does.) It was failing
because it was trying to create that subdirectory unconditionally,
instead of only creating the subdirectory if actually had to.  This also
fixes the bug that the generated script did not have the appropriate
mkdirs for those directories.

PR:		239293
Submitted by:	tobik (earlier version)
Reported by:	Ruslan Garipov
Differential Revision:	https://reviews.freebsd.org/D21112

 

21:26 bdrewery 

Remove leftover debugging

 

22:58 bdrewery 

Fix RANDOMIZE_MASTER_SITES.

- /usr/games/random moved to /usr/bin/random in 11.0+
- Need to explicitly add in the pipe '|'

Approved by:	portmgr (implicit)

 

08:53 mat 

Fix a edge case in the fetch related targets.

This would have given an error when:

- The port uses DIST_SUBDIR
- That sub-directory does not exist
- The current user does not have permission to write in DISTDIR.

Something like:

$ sudo rmdir /usr/ports/distfiles/gallery2
$ make fetch-urlall-list
mkdir: /usr/ports/distfiles/gallery2: Permission denied
*** Error code 1

PR:		222819
Reported by:	sunpoet
Sponsored by:	Absolight

 

10:52 mat 

Remove some code added in r858[1], still don't know what it was supposed to
be doing at the time.

When fetching, the code was looking for a distfile in either its
DIST_SUBDIR or the main DISTDIR, which was ok.  Until a port
(devel/cargo) moves one of its distfile
(cargo-nightly-x86_64-unknown-freebsd.tar.gz) into a subdirectory.
do-fetch would see the distfile in DISTDIR and say, ok, it's there, and
checksum would not see it in its subdirectory and fail.

1: phabricator's blame mode is really, really, great to unroll history.

PR:		216442
Submitted by:	mat
Reported by:	Bob Willcox, dhw (on ports)
Exp-run by:	antoine
Sponsored by:	Absolight
Differential Revision:	https://reviews.freebsd.org/D9318

 

12:57 mat 

Only try to create DISTDIR if it does not exist already.

It turns out, some people, instead of setting DISTDIR, replace it with a
symlink pointing to where DISTDIR should be pointing.

And mkdir -p <symlink> fails.

PR:		211623
Reported by:	Harald Schmalzbauer
Sponsored by:	Absolight

 

11:25 mat 

Reverse tests to decrease complexity.

PR:		210198
Submitted by:	mat
Exp-run by:	antoine
Sponsored by:	The FreeBSD Foundation, Absolight
Differential Revision:	https://reviews.freebsd.org/D6779

 

11:25 mat 

Remove unnecessary evals that do-fetch was using.

Turns out that env(1) knows how to parse a properly quoted string
using -S "string", it makes the double eval used for the fetch
command to be unnecessary.

This is because running:
eval "foo $(escape $bar)"
is silly when this works:
foo $bar

So remove escaping and quoting, and sillyness.

PR:		210198
Submitted by:	mat
Exp-run by:	antoine
Sponsored by:	The FreeBSD Foundation, Absolight
Differential Revision:	https://reviews.freebsd.org/D6779

 

11:24 mat 

Simplify do-fetch, and remove now unused code.

Turns out that the DEFAULT, and other groups sites were handled in
a different way, ending up having the same code twice, but in
slightly different places.

PR:		210198
Submitted by:	mat
Exp-run by:	antoine
Sponsored by:	The FreeBSD Foundation, Absolight
Differential Revision:	https://reviews.freebsd.org/D6779

 

15:25 mat 

Fix having the :DEFAULT MASTER_SITES entries ending up being there twice
in the end.

Sponsored by:	Absolight

 

08:02 mat 

Fix fetch-list and fetch-url-list-int when DISTDIR is not writable.

PR:		209820
Reported by:	amdmi3
Sponsored by:	Absolight

 

15:58 mat 

Extract the larger bsd.port.mk targets into separate scripts.

Refactor all the fetch code so that there are not 6 slightly different
versions of it but one that does it all.

The targets that have been extracted are:
- check-vulnerable
- do-fetch
- fetch-list
- fetch-url-list-int
- fetch-urlall-list
- checksum.
- makesum.
- check-checksum-algorithms

(Only the first 15 lines of the commit message are shown above )