Opened 4 years ago

Last modified 14 months ago

#26753 needs_review defect

Incremental docker build fails when removing SPKGs

Reported by: Julian Rüth Owned by:
Priority: minor Milestone: sage-8.5
Component: build: configure Keywords: docker, ContinuousIntegration
Cc: Erik Bray Merged in:
Authors: Julian Rüth Reviewers:
Report Upstream: N/A Work issues:
Branch: u/saraedum/incremental (Commits, GitHub, GitLab) Commit: fb8b9f5a3652f57e871280aeda6235b13f40316a
Dependencies: Stopgaps:

Status badges

Description (last modified by Erik Bray)

The incremental build seems to fail (on its second run) when an SPKG is removed.

https://trac.sagemath.org/ticket/26732#comment:6

The error that occurs is:

configure: error: "/home/sage/sage/build/pkgs/cephes/type" is missing.
If you would like to try to build Sage anyway (to help porting),
export the variable 'SAGE_PORT' to something non-empty.
Makefile:39: recipe for target 'build/make/Makefile' failed
make[1]: *** [build/make/Makefile] Error 1
make[1]: Leaving directory '/home/sage/sage'
Makefile:31: recipe for target 'base-toolchain' failed
make: *** [base-toolchain] Error 2
ERROR: Job failed: exit code 2

Change History (10)

comment:1 Changed 4 years ago by Julian Rüth

It could be that our patching of the incremental images does not handle deletions correctly.

Or maybe it's just a general bug in incremental builds and nothing specific about the docker setup.

Last edited 4 years ago by Julian Rüth (previous) (diff)

comment:2 in reply to:  1 Changed 4 years ago by Erik Bray

Replying to saraedum:

It could be that our patching of the incremental images does not handle deletions correctly.

Or maybe it's just a general bug in incremental builds and nothing specific about the docker setup.

Perhaps a little of both. In general there should be no problem with removing packages in incremental builds. But there might also still be some bugs in the uninstall system, and perhaps it only impacts the docker image builds even if it is in some way a more generic bug.

comment:3 Changed 4 years ago by Erik Bray

Description: modified (diff)

The code causing this particular error is from m4/sage_spkg_collect.m4:

for DIR in $SAGE_ROOT/build/pkgs/*; do
    test -d "$DIR" || continue

    PKG_TYPE_FILE="$DIR/type"
    if test -f "$PKG_TYPE_FILE"; then
        PKG_TYPE=`cat $PKG_TYPE_FILE`
    else
        AC_MSG_ERROR(["$PKG_TYPE_FILE" is missing.])
    fi

I think this code could be made more robust. I don't think there's a reason to return an error if a directory under build/pkgs/* does not actually contain an SPKG. It should probably check a few other things to determine if a directory is an SPKG, and at worst issue a warning. Maybe only error here if for all intents and purposes it still looks like an SPKG but is missing some required file.

On the Docker build end of things the issue probably has something to do with empty directories not being deleted properly.

comment:4 Changed 4 years ago by Erik Bray

I think maybe what I'll do fornow is require there to be an SPKG.txt, at a minimum, for the directory to be considered an SPKG, and ignore any directory that doesn't have it.

I would still like to redo the proliferation of file that make up an "SPKG" but that's something I can tackle another time.

comment:5 Changed 3 years ago by Julian Rüth

Keywords: ContinuousIntegration added; CI removed

comment:6 Changed 3 years ago by Julian Rüth

Some of the packages actually do not have an SPKG.txt (python3 e.g.)

So let's test for type.

comment:7 Changed 3 years ago by Julian Rüth

Branch: u/saraedum/incremental

comment:8 Changed 3 years ago by Julian Rüth

Commit: fb8b9f5a3652f57e871280aeda6235b13f40316a
Component: PLEASE CHANGEbuild: configure
Status: newneeds_review

New commits:

fb8b9f5Ignore empty package directories

comment:9 Changed 16 months ago by Samuel Lelièvre

Authors: Julian Rüth

comment:10 Changed 14 months ago by Dima Pasechnik

is this outdated?

Note: See TracTickets for help on using tickets.