Re: revamped gdb_mbuild.sh

[Andrew Cagney <ac131313 at redhat dot com>]

Ok,

Attached is an updated script:

> The changes are:
>
>
> - restartable
>
> Instead of starting over from scratch, it tries to continue on from where the last run left off (after being CNTRL-Ced or failing).  To force a complete rebuild of each target, use the option:
>
> `-f' (force) will force a new run
>
>
> - stop on failure
>
> The script, by default, aborts on the first failed build.  The assumption is that the problem will be fixed and then the script re-started.  This can be overriden with the option:
>
> `-k' (keep going) will force the script to try and build everything possible (the flag is passed to make as well).
>
>
> - parallel
It is possible to specify parallism at two levels.  Firstly the number 
of builds, and secondly the number of jobs MAKE tries to sustain.  Two 
options control this:

`-b MAXBUILDS' The number of builds to run in parallel.
`-j MAXJOBS' The number of jobs, whithin a build, that MAKE should run 
in parallel (using `make -j MAXJOBS').

These options are orthoginal, `-b 3 -j 2' will result in the script 
running up to three simultaneous `make -j 2'.

> - testing
>
> In addition to checking the configure and build, the script also tries to run the built GDB to the point where it checks that '(gdb) maint print architecture' succeeds.
>
> (NB: The h8500hms target currently barfs when run, I think I might propose that it be booted out of the build list).
>
>
> - selecting targets
>
> A sub-set of targets can be selected using the option:
>
> `-e <targexp>' Select a subset of the targets using the (grep) regular expression <targexp>.  Multiple expressions can be specified.
- more verbose output

By default the output from the configure and build is supressed (saved 
to a log file).  This can be disabled with:

-v (verbose) Provide more verbose output (e.g., output the configure, 
build and run results to stdout).

This is very useful from within emacs.  Something like `M-X compile' 
`./gdb_build.sh -v .. /tmp' works.


David, yes, `gdb_build.sh -k -f' should give you back something like the 
old behavior.

Richard, should `-j 2' be the default?  Now that this doesn't lead to 
two parallel configures (which is what I think hammers the poor host).

Andrew
#!/bin/sh

#  Multi-build script for testing compilation of all maintained configs of GDB.
#  Copyright (C) 2002  Free Software Foundation, Inc.
#  Contributed by Richard Earnshaw  (rearnsha@arm.com)

#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.

#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.

#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

usage() {
    cat <<EOF
Usage: gdb_mbuild.sh [ <options> ... ] <srcdir> <builddir>
 Options:
  -j <makejobs>  Run <makejobs> in parallel.  Passed to make.
		 On a single cpu machine, 2 is recommended.
  -b <maxbuilds> Run <maxbuild> builds in parallel.
		 On a single cpu machine, 1 is recommended.
  -v             Be more (and more, and more) verbose.
  -k             Keep going.  Do not stop after the first build fails.
  -e <regexp>    Regular expression for selecting the targets to build.
  -f             Force rebuild.  Even rebuild previously built directories.
 Environment variables examined (with default if not defined):
  MAKE (make)"
 Note: Everything in <builddir>/gdb-allcross will be blown away.
EOF
    exit 1;
}

### COMMAND LINE OPTIONS

makejobs=
maxbuilds=1
keepgoing=false
force=false
targexp=""
verbose=0
while test $# -gt 0
do
    case "$1" in
    -j )
	# Number of parallel make jobs.
	shift
	test $# -ge 1 || usage
	makejobs="-j $1"
	;;
    -b | -c )
	# Number of builds to fire off in parallel.
	shift
	test $# -ge 1 || usage
	maxbuilds=$1
	;;
    -k )
	# Should we soldier on after the first build fails?
	keepgoing=true
	;;
    -e )
	# A regular expression for selecting targets
	shift
	test $# -ge 1 || usage
	targexp="${targexp} -e ${1}"
	;;
    -f )
	# Force a rebuild
	force=true ; shift ;;
    -v )
	# Be more, and more, and more, verbose
	verbose=`expr ${verbose} + 1`
	;;
    -* ) usage ;;
    *) break ;;
    esac
    shift
done


### COMMAND LINE PARAMETERS

if test $# -ne 2
then
    usage
fi

# Convert these to absolute directory paths.

# Where the sources live
srcdir=`cd $1 && /bin/pwd` || exit 1

# Where the builds occur
builddir=`cd $2 && /bin/pwd` || exit 1


### ENVIRONMENT PARAMETERS

# Version of make to use
make=${MAKE:-make}
MAKE=${make}
export MAKE


# Where to look for the list of targets to test
maintainers=${srcdir}/gdb/MAINTAINERS
if [ ! -r ${maintainers} ]
then
    echo Maintainers file ${maintainers} not found
    exit 1
fi

# Get the list of targets and the build options
alltarg=`cat ${maintainers} | tr -s '[\t]' '[ ]' | sed -n '
/^[ ]*[-a-z0-9\.]*[ ]*[(]*--target=.*/ !d
s/^.*--target=//
s/).*$//
h
:loop
  g
  /^[^ ]*,/ !b end
  s/,[^ ]*//
  p
  g
  s/^[^,]*,//
  h
b loop
:end
p
' | if test "${targexp}" = ""
then
    grep -v -e broken -e OBSOLETE
else
    grep ${targexp}
fi`


# Run several jobs in parallel.
set --


# Usage: fail <message> <test-that-should-succeed>.  Should the build
# fail?  If the test is true, and we don't want to keep going, print
# the message and shoot everything in sight and abort the build.

fail ()
{
    msg="$1" ; shift
    if test "$@"
    then
	echo "${target}: ${msg}"
	if ${keepgoing}
	then
	    exit 1
	else
	    kill $$
	    exit 1
	fi
    fi
}


# Usage: log <logfile>.  Write standard input to <logfile> and (if
# verbose) stdout.

log ()
{
    if test ${verbose} -gt 0
    then
	tee $1
    else
	cat > $1
    fi
}



# Warn the user of what is comming, print the list of targets
echo "$alltarg"
echo ""


# For each target, configure and build it.
echo "$alltarg" | while true
do

    # Keep forking builds until we've exceed our build capacity

    while test $# -lt ${maxbuilds} && read target gdbopts simopts
    do

	(
	    trap "exit 1"  1 2 15
	    dir=${builddir}/${target}

	    # Should a scratch rebuild be forced?
	    if ${force}
	    then
		echo ... forcing ${target}
		rm -rf ${dir}
	    fi
	    
	    # Don't bother re-building a built target (indicated by
	    # ${dir} being a file).
	    if test -f ${dir}
	    then
		echo "${target}"
		exit 0
	    fi

	    echo ${target} ...

	    # Did the previous configure attempt fail?  If it did
	    # restart from scratch.
	    if test -d ${dir} -a ! -r ${dir}/gdb/Makefile
	    then
		echo ... removing partially configured ${target}
		rm -rf ${dir}
		if test -d ${dir}
		then
		    echo "${target}: unable to remove directory ${dir}"
		    exit 1
		fi
	    fi

	    # From now on, we're in this target's build directory
	    mkdir -p ${dir}
	    cd ${dir} || exit 1

	    # Configure, if not already.
	    if test ! -r gdb/Makefile
	    then
		# The config options
		__target="--target=${target}"
		__enable_gdb_warnings=`test -z "${gdbopts}" \
		    || echo "--enable-gdb-warnings=${gdbopts}"`
		__enable_sim_warnings=`test -z "${simopts}" \
		    || echo "--enable-sim-warnings=${simopts}"`
		echo ... configure ${target} \
		    ${__enable_gdb_warnings} \
		    ${__enable_sim_warnings}
		trap "echo Removing partially configured ${dir} directory ...; rm -rf ${dir}; exit 1" 1 2 15
		${srcdir}/configure \
		    ${__target} \
		    ${__enable_gdb_warnings} \
		    ${__enable_sim_warnings} \
		2>&1 | log Config.log
		trap "exit 1"  1 2 15
	    fi
	    fail "configure failed" ! -r gdb/Makefile

	    # Build, if not built.
	    if test ! -x gdb/gdb -a ! -x gdb/gdb.exe
	    then
		echo ... make ${makejobs} ${target}
		${make} ${makejobs} all-gdb 2>&1 | log Build.log
	    fi
	    fail "compile failed" ! -x gdb/gdb -a ! -x gdb/gdb.exe
	
	    # Check that the built GDB can at least print it's architecture.
	    echo ... run ${target}
	    rm -f core gdb.core ${dir}/gdb/x
	    cat <<EOF > x
maint print architecture
quit
EOF
	    ./gdb/gdb -batch -nx -x x 2>&1 | log Gdb.log
	    fail "gdb dumped core" -r core -o -r gdb.core
	    fail "gdb printed no output" ! -s Gdb.log
	    grep -e internal-error Gdb.log && fail "gdb panic" 1

	    # Replace the build directory with a file as semaphore
	    # that stops a rebuild. (should the logs be saved?)
	    cd ${builddir}
	    rm -f ${target}.tmp
	    mv ${target}/Gdb.log ${target}.tmp
	    rm -rf ${target}
	    mv ${target}.tmp ${target}

	    # Success!
	    echo ... ${target} built

	) &

	# Append this to the list of things to wait for.  Grr, there
	# is a race condition as the cntrl-c could come in before the
	# trap is updated.
	set -- $* $!
	trap "echo Killing $*; kill $*; wait ; exit 1" 1 2 15

    done

    # If we've got zero jobs, we're finished
    if test $# -eq 0
    then
	break
    fi

    # Reap the first running process.  Unfortunatly, this doesn't help
    # if that process turns out to be the slowest.  Anyone got a
    # better suggestion?
    wait $1 ; shift

    # Prune the remaining processes, cuting out any others that have
    # finished.  This hopefully helps a little with the above problem
    # by maximising the number of jobs started each time round the
    # loop.
    jobs=""
    while test $# -gt 0
    do
	# still running?
	if kill -0 $1 > /dev/null 2>&1
	then
	    jobs="${jobs} $1"
	else
	    wait $1
	fi
	shift
    done
    set -- ${jobs}

done

exit 0