Testing (was: Re: help: question for porting gas/ld to MIPS-X3)

[Hans-Peter Nilsson <hp at bitrange dot com>]

On Sun, 24 Feb 2002, David Zhang wrote:

>    I have found the gas/ld port examples in the maillist. But I am a
> novice, the testsuite confused me. I don't know what's function of the
> "*.s" and "*.d" files in the testsuite directory and how to build
> testsuite. I can't find the doc introducing them. Could you give me some
> pointers in this regard?

(To all: There really should be a document describing this
procedure, besides the generalizations in the dejagnu manual at
<URL:http://www.gnu.org/manual/dejagnu-1.3/dejagnu.html>.
Maybe there is; if so I've only forgotten where I've read this.
Maybe someone told me or I read it myself on this list long ago;
I honestly don't remember.)

I think you are referring to the test files for run_dump_tests,
as run from for example ld/testsuite/ld-mips-elf/mips-elf.exp
(or better IMHO: ld/testsuite/ld-mmix/mmix.exp).

You write the .s file(s) yourself, as you add functionality or
investigate a bug that is causing errors or incorrect object
files to be generated by gas or ld.  Then you write the head
part of the .d file.  You have to decide on what kind of
"inspection program" you want to run for the test.  The most
common ones are readelf and objdump.  See run_dump_test in
ld/testsuite/lib/ld-lib.exp for what little documentation there
is about this.

The simplest way forward then is that you run just that test,
for example for ld/testsuite/ld-mmix/bpo-1.d with
 "make check-ld RUNTESTFLAGS=mmix.exp=bpo-1".
The test will fail at this point because the expected output is
empty.  The point is that you use the output generated in
ld/tmpdir/dump.out when running the test, and paste it at the
end of the .d file. In the text you pasted in, you have to add \
escapes in front of regexp metacharacters, and you should
replace the output parts that are likely to change and that are
uninteresting for your test, with generalized regexps.

You don't have to do it the way it's described in the paragraph
above (I've just found that to be easiest for me); you can of
course for example run the "inspection program" manually or
cut-and-paste and use the command lines logged in
ld/testsuite/ld.log.

Note the "#..." and "#pass" run_dump_test match features.
Also, note that the gas run_dump_test differs from the ld one.

A run_dump_test doesn't cover all types of tests, and some
features are lacking, for example the ability to test that some
specific output does *not* appear.  One other type of test is
that invalid input is caught correctly in the assembler.  It is
best done with a "dg" test.  Perhaps the best reading for that
is /usr/share/dejagnu/dg.exp (your filepaths may vary) or by
example from grepping for dg-do in the gas testsuited.  I think
it's also described to some extent in the dejagnu manual.

Hope this helps.  If there's anything that I didn't explain good
enough, please don't hesitate to ask (on the list, as always).

brgds, H-P