do-fetch: fix wrongly added newline in 'make fetch-list'
This ensures one series of commands (corresponding to one
distfile) per emitted line on stdout.
Reviewed by: mat
Approved by: portmgr
Differential Revision: https://reviews.freebsd.org/D60140
Mk: fix checksum/fetch/makesum with large distfiles set
When a port has a very large set of DISTFILES, makesum/fetch dies with
an obscure message such as:
> /tmp/makeQEBorT: /usr/bin/env: Argument list too long
The interface from make(1) to sh(1) passes the list of MASTER_SITES and
DISTFILES as arguments on the command line. At one point, it reaches
ARG_MAX and fails with the previous message. This changes the interface
to use temporary files in WRKDIR.
PR: 295029
Differential Revision: https://reviews.freebsd.org/D59475
framework: Fix fetch-url-list and fetch-urlall-list
The fetch-url-list and fetch-urlall-list targets are meant to produce
a list of URLs from which a port fetches its distfiles and patches.
Currently, those targets were not working as expected as they print
parts of the output meant for other targets like the fetch-list target.
For example:
mateusz.piotrowski@server /ports/x11-servers/xwayland-devel$ make
fetch-urlall-list
[...]
mkdir -p "xorg" &&
-o xorg/proto-xorgproto-824001c947cb1962209c6a8f2c63c2637877220d_GL0.tar.gz
[...]
This patch prevents do-fetch.sh from printing the "mkdir" line and the
"-o" line.
While here:
- Remove the outdated comment about escaping. There is no excessive
escaping happening there anymore.
- Indent cases in the case statement to match the style of the rest of
the file.
Reviewed by: pizzamig
Approved by: portmgr (pizzamig)
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D40808
Add set pipefail in most framework scripts.
set pipefail changes the pipeline return status from being the return
status of the last command to the last non 0 exit status of any command
in the pipeline. This is needed to make sure all the commands in a
pipeline did actually return a non 0 status and not only the last one.
PR: 250723
Exp-run by: antoine
Reviewed by: bapt
Approved by: bapt
Differential Revision: https://reviews.freebsd.org/D27007
Add set pipefail in most framework scripts.
set pipefail changes the pipeline return status from being the return
status of the last command to the last non 0 exit status of any command
in the pipeline. This is needed to make sure all the commands in a
pipeline did actually return a non 0 status and not only the last one.
Fix fetch-list when running as a user.
The fetch-list target is used to generate a shell script that will more
or less replicates what do-fetch does. It allows one to do most things
as a regular user, and generate that script to run, say, on another
machine, if the one where you build things does not have access to the
internet, or has much slower access.
It was failing when DISTDIR was not writable by the current user, and
the port had a distribution file with a path in it. (Not using
DIST_SUBDIR, something else, like lang/rust does.) It was failing
because it was trying to create that subdirectory unconditionally,
instead of only creating the subdirectory if actually had to. This also
fixes the bug that the generated script did not have the appropriate
mkdirs for those directories.
PR: 239293
Submitted by: tobik (earlier version)
Reported by: Ruslan Garipov
Differential Revision: https://reviews.freebsd.org/D21112
Fix RANDOMIZE_MASTER_SITES.
- /usr/games/random moved to /usr/bin/random in 11.0+
- Need to explicitly add in the pipe '|'
Approved by: portmgr (implicit)
Fix a edge case in the fetch related targets.
This would have given an error when:
- The port uses DIST_SUBDIR
- That sub-directory does not exist
- The current user does not have permission to write in DISTDIR.
Something like:
$ sudo rmdir /usr/ports/distfiles/gallery2
$ make fetch-urlall-list
mkdir: /usr/ports/distfiles/gallery2: Permission denied
*** Error code 1
PR: 222819
Reported by: sunpoet
Sponsored by: Absolight
Remove some code added in r858[1], still don't know what it was supposed to
be doing at the time.
When fetching, the code was looking for a distfile in either its
DIST_SUBDIR or the main DISTDIR, which was ok. Until a port
(devel/cargo) moves one of its distfile
(cargo-nightly-x86_64-unknown-freebsd.tar.gz) into a subdirectory.
do-fetch would see the distfile in DISTDIR and say, ok, it's there, and
checksum would not see it in its subdirectory and fail.
1: phabricator's blame mode is really, really, great to unroll history.
PR: 216442
Submitted by: mat
Reported by: Bob Willcox, dhw (on ports)
Exp-run by: antoine
Sponsored by: Absolight
Differential Revision: https://reviews.freebsd.org/D9318
Only try to create DISTDIR if it does not exist already.
It turns out, some people, instead of setting DISTDIR, replace it with a
symlink pointing to where DISTDIR should be pointing.
And mkdir -p <symlink> fails.
PR: 211623
Reported by: Harald Schmalzbauer
Sponsored by: Absolight
Remove unnecessary evals that do-fetch was using.
Turns out that env(1) knows how to parse a properly quoted string
using -S "string", it makes the double eval used for the fetch
command to be unnecessary.
This is because running:
eval "foo $(escape $bar)"
is silly when this works:
foo $bar
So remove escaping and quoting, and sillyness.
PR: 210198
Submitted by: mat
Exp-run by: antoine
Sponsored by: The FreeBSD Foundation, Absolight
Differential Revision: https://reviews.freebsd.org/D6779
Simplify do-fetch, and remove now unused code.
Turns out that the DEFAULT, and other groups sites were handled in
a different way, ending up having the same code twice, but in
slightly different places.
PR: 210198
Submitted by: mat
Exp-run by: antoine
Sponsored by: The FreeBSD Foundation, Absolight
Differential Revision: https://reviews.freebsd.org/D6779
Extract the larger bsd.port.mk targets into separate scripts.
Refactor all the fetch code so that there are not 6 slightly different
versions of it but one that does it all.
The targets that have been extracted are:
- check-vulnerable
- do-fetch
- fetch-list
- fetch-url-list-int
- fetch-urlall-list
- checksum.
- makesum.
- check-checksum-algorithms
(Only the first 15 lines of the commit message are shown above )
