05:43 bapt 

Remove support for NO_STAGE
Mark all current non staged ports as BROKEN

Reviewed by:	antoine
Exp-run:	antoine
Differential Revision:	https://reviews.freebsd.org/D693

 

22:10 antoine 

Check that the plist doesn't reference the stage dir during stage-qa

====> Running Q/A tests (stage-qa)
Error: '/usr/ports/www/claroline/work/.PLIST.mktmp' is referring to
/usr/ports/www/claroline/work/stage
*** Error code 1

Differential Revision: https://reviews.freebsd.org/D685
Reviewed by:	bdrewery
With hat:	portmgr

 

12:23 bapt 

Remove manpage leftovers in stage

 

23:01 bapt 

Fix symlinks of manpage after compression

PR:		184395
Submitted by:	oliver.fromme@secnetix.de

 

11:57 tijl 

Let stage-qa detect .la libraries and warn that a port needs USES=libtool.

PR:		ports/189255
Approved by:	portmgr (antoine)

 

17:23 antoine 

Fix a copy/paste error

Reported by:	rene
With hat:	portmgr

 

22:22 bdrewery 

- Fix when USES not defined

With hat:	pormtmgr
Pointyhat to:	bdrewery

 

22:19 bdrewery 

- Fix stage-qa when USES contains desktop-file-utils twice

Reported by:	mandree
With hat:	portmgr

 

06:44 bdrewery 

- Rename check-orphans to check-plist. Keep the old for backwards-compat.
- Bug fixes:
  makeplist/check-plist:
   - Fix showing directories owned by RUN_DEPENDS/LIB_DEPENDS [1]
   - Use proper "rmdir PATH" syntax, not "rmdir >/dev/null... PATH" which
     pkg will not recognize.
   - Never consider base /etc/mtree/BSD.usr.dist or LOCLABASE
     Templates/BSD.local.dist as needing @dirrm handling.
  check-plist:
   - Fix showing PORTDOCS/PORTEXAMPLES files when the OPTIONS are not set.
     makeplist will still suggest them.
   - Fix showing files installed through unselected OPTIONS as orphans,
     by considering "@comment file" to be ignored. [2]
   - Fix @sample, @fc, @fcfontsdir, @fontsdir support
   - Fix return status when orphans are found to be non-zero
   - Add note when PREFIX=!LOCALBASE
   - Be more clear when orphans/no orphans are found.
   - Add a whitelist mechanism for globally approved ignores.
   - Add *.bak/*.orig to orphan whitelist for now.
   - Fix false-positive with dirs installed to /, such as with
     archivers/dpkg creating /var/db/dpkg
   - Fix false-positive with @dirrm ending in /, such as with
     ports-mgmt/poudriere with a @dirrmtry share/zsh/
   - There are likely still some false-positives. I fixed as many as I could
     find. Please let me know of others.
    * One in particular that is not easily fixable is installing a file into
      a directory owned by another port where that other port is not a run-time
      dependency. So the leaf port may create all of the parent dirs and never
      clean them up. Cleaning them up is not proper unless no other package is
      depending on them. This will be addressed by pkg(8) once pkg_install is
      EOL, or sooner.
   - Consider @dirrm of directories owned by run-time dependencies, or
     /etc/mtree/* or Templates/BSD.local.dist (at LOCALBASE) as fatal errors.
     These should not be removed in the plist. @comment lines are not
     considered for this; they will not ignore an error.
        ===> Checking for directories owned by dependencies or MTREEs
        Error: Owned by dependency: @dirrmtry share/locale/af/LC_MESSAGES
        Error: Owned by dependency: @dirrmtry share/locale/af
        Error: Owned by dependency: @dirrmtry %%PERL5_MAN3%%
   - Detect files in plist that do not exist in the stagedir. Pkgng already did
     this, but now we have it unified with this check. @comment lines are
     not considered for this; they will not ignore an error.
        ===> Checking for items in pkg-plist which are not in STAGEDIR
        Error: Missing: foo
        Error: Missing: @dirrmtry bar
   - Change orphaned output due to several new errors introduced:
        ===> Checking for items in STAGEDIR missing from pkg-plist
        Error: Orphaned: foo
   - Send errors to stderr
  check_leftovers.sh:
   - Prefer longer values for PLIST_SUB.
  bsd.options.mk:
   - Fix NOPORTDOCS/WITHOUT_NLS/NOPORTEXAMPLES not unsetting their respective
     OPTIONS.
   - Mark NOPORTDOCS/NOPORTEXAMPLES deprecated and hook them into the
     OPTIONS_WARNING to tell users the new format.
- check-stagedir.sh refactoring:
  - Switch to using new PLIST_SUB_SED
  - Use ! instead of , in sed(1) regex to allow files/dirs with ','
  - Rework PORTEXAMPLES/PORTDOCS handling so it acts on PLIST_SUB_SED'd
    value and not absolutes. Also simplify the regex for these a bit to
    allow reuse.
  - No longer need DOCSDIRS/EXAMPLESDIR in env
  - Wrap long lines
  - Unset some vars in env when they are done being used to free space for
    larger sed vars
  - Cleanup redundant sed regexes
- Add a Scripts/plist_sub_sed_sort.sh to prefer longer values when substituting
  over shorter values.

- To make check-plist ignore a file *as an orphan* do one of the following:
  1. Install it
  2. post-install: ${RM} ${STAGEDIR}file
  3. Put the file behind an OPTION with a PLIST_SUB: %%OPTION%%file
  4. Add to plist as a @comment [2]:
     @comment file
     @comment @dirrmtry dir

Reviewed by:	mat (much earlier version)
Discussed with:	mat, antoine, bapt, swills (various bits)
With hat:	portmgr
PR:		ports/185561 [1]
Reported By:	Alexander Yerenkow <yerenkow@gmail.com> [1]
Tested with:	pkg and pkg_install
Reported by:	many (false-positives)

[2] This is a partial solution, we may still need a plist.ignore too. It
    doesn't make much sense to add files in main pkg-plist we don't care about,
    but maybe it does since you'll see and reconsider them being ignored
    someday. @comment is used as all the OPTION PLIST_SUB deactivations use
    @comment instead of something like @ignore.

 

17:02 bapt 

Allow to properly run make check-orphans and make makeplist for gnome ports

 

16:09 mat 

Remove bits in PLIST_SUB who have spaces in them for makeplist usage.

Sponsored by:	Absolight

 

22:19 antoine 

- Do not check for ${WRKDIR} in staged files, there are lots of false
  positives as some .o files may embed the source path [1]
- Only check shebang of files with executable bit set

Requested by:	bapt [1]
With hat:	portmgr

 

14:16 mat 

Greatly enhance makeplist to replace all it can find in PLIST_SUB.

With hat:	portmgr
Sponsored by:	Absolight

 

20:49 antoine 

Add WRKDIR to QA_ENV so that the paths QA check can search for it

 

08:17 bapt 

Add an internal variable to decide if DESTDIR should ve define in MAKE_ENV or
MAKE_ARGS

 

17:48 antoine 

Do not create a BSD local hierarchy under ${STAGEDIR}${LINUXBASE} when
USE_LINUX=yes and PREFIX!=LINUXBASE.
This is not needed and creates a bunch of stage orphaned directories.

PR:		ports/184477
Approved by:	portmgr (bapt)
Tested by:	poudriere bulk -t games/dsnake emulators/rtc
www/linux-f10-flashplugin11 (no error, no orphans)

 

09:48 bapt 

Fix handling of info files with pkgng, when dealing with .info-X files and also
what INFO_SUBDIR.
The .info-X handling was wrong since the beginning of pkgng, the INFO_SUBDIR
regression was introduced with stage

PR:		ports/184178
Reported by:	gerald
Tested by:	gerald

 

10:36 rene 

Fix 'make stage' for ports which define USE_LINUX by passing a defined
variable to MTREE_CMD.  The previous variable, MTREE_LINUX_ARGS, seems to
be a leftover.

Approved by:	portmgr (bapt)

 

19:07 mandree 

bsd.stage.mk: Fewer false positives, much faster, easier maintenance.

Bugfixes:

* @cmd in pkg-plist is now properly handled.
  - It was previously treated the same as though there was a directory
    following it, missing the prefix. (ordering matters in case...esac)

  - Due to the cwd=${PREFIX} inside the while read line loop, state
    tracking was broken and every new line assumed that cwd were the
    prefix.

* stage-qa no longer complains about unstripped binaries if debugging is
  active (WITH_DEBUG set && WITHOUT_DEBUG unset).

* The compress-man target uses ECHO_MSG, not ECHO_CMD, to print its
  build step.

Additions:

* The plist parser now understands @unexec rmdir ... || ... lines,
  including those with redirections, so that there are no false
  positives for directories stripped with @unexec rmdir (usually
  happens on stuff installed outside $PREFIX, as in /var).

* The system's root and var mtrees are now also expanded to avoid
  false @dirrm positives if a port installs directories under /var
  and has to create parents in the stagedir that are present in a fully
  installed system (i. e. in the real $PREFIX).

* Given that pkg_create is deemed beyond repair with respect to deleting
  files outside prefix, generate @unexec rmdir statements for such
  directories, rather than @dirrmtry, to sidestep the problem.

Speedups:

* the orphan check now generates sorted lists of staged files,
  and plisted/mtree files, and compares them with comm(1).
  This saves us the overhead of running one grep process per file
  and up to two per directory, and defers the actual list
  processing to a shell utility.  Complexity has not changed,
  but overhead per item has.

* the orphan check now uses one file for directories and one file for
  files mentioned in pkg-plist, so we need not decorate them with "dir "
  and parse them out any longer.

* qa.sh's shebang scanner only looks at the first line of a file,
  sed is told to exit from the 2nd line.

Other Changes:

* Split the makeplist/check-orphans logic out of bsd.stage.mk,
  it is too unwieldy to maintain in make-escaped shell syntax,
  and permits shell tracing with "SH=sh -x" (including quotes!)

* Unify the functions "makeplist" and "check-orphans" in one
  script.  The only difference is that makeplist assumes an empty
  pkg-plist, whereas check-orphans parses it.

* overhaul the mtree extractor, avoiding awk.

Reviewed by:	bapt
Approved by:	portmgr (bapt)

 

13:09 rene 

Fix a typo in bsd.stage.mk, LINUXBASE_REL should be LINUXBASE.  Nothing
defines the former variable.

Approved by:	portmgr (bapt)

 

17:09 antoine 

Add two QA tests for staged ports:
- check for desktop files handling MIME types
- check for share MIME-Info database

Approved by:	portmgr (bapt)

 

15:11 bapt 

First set of Q/A for staged ports.
A couple of Q/A tests are done if the DEVELOPER macros is set in make.conf
Right now the tests are:
- Check if the symlinks are properly created
- Check if the binaries are stripped (just warn)
- Check if the STAGEDIR or the WORKDIR are referenced in the final files
- Check if the ports provide script with bad shebangs.

 

16:43 bapt 

MTREE_FILE is not yet defined bsd.stage.mk is parsed let the shell script test
it.

 

15:33 bapt 

Fix makeplist with no MTREE_FILE

 

15:31 bapt 

Fix check-orphans with no MTREE_FILE

 

14:11 mat 

Fixup the MLINKS replacement code.

Approved by:	bapt

 

22:09 bapt 

New target make check-orphans to help discovering the files in staged not
present in plist

Reviewed by:	bdrewery

 

20:07 bapt 

Introduce a new MANDIRS macros to make the compress-man determine where the
makefiles should be found
Previous implementation was based on etc/man.d which doesn't exists in FreeBSD
8.
By default MANDIRS is ${MANPREFIX}/man. if a port adds USES=perl5 then the perl
default locations is added

Reviewed by:	bdrewery

 

11:45 bapt 

Use macro when possible

Reported by:	ak

 

10:23 bapt 

Sort earlier to preserver the FS ordering

Reported by:	danfe

 

10:09 bapt 

Make the new compress-man respect LOCALBASE and PREFIX

 

05:56 bapt 

Add support for staging area in the ports tree

The ports tree is now staged by default. With pkgng the sequence hasn't
changed, the main difference is that creating package is now independent
from installing it. With pkg_install, the package is now created first
and make install, do install the package.

New macros:
- STAGEDIR: PATH to the directory where the port will be staged.
- NO_STAGE: Keep the old behaviour of the ports tree (aka no staging area)

Unsupported macro if NO_STAGE is not set:
- MAN* with staging man page compression and handling of hardlinks and
symlinks is automatically done in the stage. the manpages becomes then a
"normal" plist files and should be tracked in pkg-plist.

(Only the first 15 lines of the commit message are shown above )