This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: Remove unused libm-test expected-failure mechanism
- From: "Carlos O'Donell" <carlos at redhat dot com>
- To: "Joseph S. Myers" <joseph at codesourcery dot com>
- Cc: libc-alpha at sourceware dot org
- Date: Sun, 05 May 2013 10:41:00 -0400
- Subject: Re: Remove unused libm-test expected-failure mechanism
- References: <Pine dot LNX dot 4 dot 64 dot 1305041343210 dot 658 at digraph dot polyomino dot org dot uk>
On 05/04/2013 09:43 AM, Joseph S. Myers wrote:
> libm-test.inc has a mechanism for libm-test-ulps files to specify
> expected failures as well as ulps values. No architecture uses it, it
> can only handle certain kinds of errors (e.g. one of result and
> expected value being infinity/NaN and the other not; if the error is
> simply different nonzero values, an ulps value could just be specified
> directly), in particular not those relating to exceptions or errno
> values, and there is no convenient system for generating such expected
> failures if (re)generating a libm-test-ulps file from scratch.
>
> We now have a de facto convention of skipping tests known to fail via
> #if conditions, or relaxing what's allowed in the case of exception
> tests, with a comment referring to the bug filed in Bugzilla; while
> libm-test-ulps files can also contain comments, any regeneration of
> them at all (e.g. "make regen-ulps"), not just from scratch, will lose
> any manually-added comments, so indicating a further advantage of
> skipping tests this way rather than using the libm-test-ulps
> expected-failure mechanism - the ability to have comments referring to
> the associated bug, which don't automatically disappear.
>
> Thus, I propose this patch to remove the unused expected-failure
> mechanism. This of course makes the generated libm-test.c smaller and
> presumably faster to compile (one fewer function argument in thousands
> of generated function calls), as well as simplifying the conversion to
> data-driven tests (one fewer bit of information for each test to put
> in the arrays of test inputs and then pass to appropriate functions).
>
> Tested x86_64.
This looks good to me.
It is my opinion that such an expected failure mechanism should be
handled by a higher level test framework, and that libm-test.inc
should simply report the list of all passed or failed tests given
that each test has a unique name.
Cheers,
Carlos.