Re: [rfa/testsuite/PING] asm-source.exp: use UNTESTED

[Andrew Cagney <cagney at gnu dot org>]

> [Originally submitted 2003-12-18]
>
> I'm walking through the gdb.sum file for native hppa2.0w-hp-hpux11.11,
> cleaning up crap in the test suite so that I can see the condition of
> gdb better.  So here's the first patch.
>
> This patch changes asm-source.exp for architectures that aren't
> implemented yet.  The existing code calls gdb_suppress_entire_file, a
> nasty function that doesn't actually suppress any tests: it just forces
> a lot of test results to FAIL.  So I get: 4 ERROR, 5 WARNING, 28 FAIL, 1
> UNRESOLVED.
>
> My patch just reports UNTESTED and then returns.
>
> I think that UNTESTED is the right test result here.  The dejagnu doco
> says:
>
>   @item UNTESTED
>   @kindex UNTESTED
>   @cindex untested properties
>   A test case is not yet complete, and in particular cannot yet produce a
>   @code{PASS} or @code{FAIL}.  You can also use this outcome in dummy
>   ``tests'' that note explicitly the absence of a real test case
>   for a particular property.

Technically yes.

However, remember why this test was originally changed to fail messy - 
the test was being skipped and, as a demonstratable consequence, 
everyone chose to ignore it rather than fix the testcase :-(

What about UNTESTED, and then KFAIL everything?  Knowing my luck that 
will be much harder than it seems ...

The other solution is to require people to post test results (so that 
they are recorded) when submitting a new architecture.

> If you want the output to be WARNING and then UNTESTED, I wouldn't
> object.  Or if you want a FAIL in there too so that people who ignore
> everything but FAIL would see it.  I think UNTESTED is completely right
> for this situation, but I'm flexible about happens.
>
> But the call to gdb_suppress_entire_file really must die.  I can't deal
> with 4 ERROR, 5 WARNING, 28 FAIL, and 1 UNRESOLVED just because no one
> has written the assembly language test yet.  And I'm not going to write
> it yet because I have to deal with testing HP's compilers and assemblers
> first; it can't be just another hunk of gnu assembly code with a few
> opcodes changed.
>
> I tested this on native hppa2.0w-hp-hpux11.11, with a result
> of UNTESTED.
>
> Okay to commit?
>
> Michael C
>
> 2003-12-17  Michael Chastain  <mec.gnu@mindspring.com>
>
> 	* gdb.asm/asm-source.exp: Return UNTESTED for platforms that
> 	have not implemented the assembly source test.
>
> *** ORIGINAL-asm-source.exp	2003-12-18 01:15:58.000000000 -0500
> --- asm-source.exp	2003-12-18 01:16:07.000000000 -0500
> *************** switch -glob -- [istarget] {
> *** 115,121 ****
>   }
>   
>   if { "${asm-arch}" == "" } {
> !     gdb_suppress_entire_file "Assembly source test -- not implemented for this target."
>   }
>   
>   # On FreeBSD and NetBSD, crt1.o the final link will fail because of
> --- 115,122 ----
>   }
>   
>   if { "${asm-arch}" == "" } {
> !     untested "assembly source test not implemented for this target"
> !     return
>   }
>   
>   # On FreeBSD and NetBSD, crt1.o the final link will fail because of