Re: Implementation of C11 Bounds-checking interfaces

[Ulrich Bayer <ubayer at sba-research dot org>]

On 31.10.2012 18:27, Joseph S. Myers wrote:
> It would be good to know what issues were encountered in implementing the 
> standard requirements outside of glibc, since that's the main factor I 
> think should determine the best place for these functions.
Thanks for your feedback. Please find my comments below.

> * I'd think the K.3.1.1 paragraph 4 requirement ("Within a preprocessing 
> translation unit, __STDC_WANT_LIB_EXT1__ shall be defined identically for 
> all inclusions of any headers from subclause K.3. If 
> __STDC_WANT_LIB_EXT1__ is defined differently for any such inclusion, the 
> implementation shall issue a diagnostic as if a preprocessor error 
> directive were used.") may need a compiler feature (a GCC pragma to say 
> that a header is using a standard feature test macro like this) to be met 
> fully.
I'd agree. Currently, no such warning is issued for conflicting
definitions of __STDC_WANT_LIB_EXT1__.

> * Although you can probably meet the header requirements using 
> #include_next, and may be able to predefine __STDC_LIB_EXT1__ by providing 
> your own stdc-predef.h that uses #include_next, doing so would certainly 
> be simpler with closer library integration.
__STDC_LIB_EXT1__ is not defined at the moment because the user must
explicitly change
the compiler search path by means of -I slibc_new_headers/. This results
in the Annex K header files being found before the original header file.
Probably this compiler search path could  be changed at the system
level. In that case, it would make sense to define such a symbol in a
own stdc-predef in the way that you describe.

The #include_next way could be a bit problematic for headers that are
included multiple times with different intentions. stdef.h  and the
different __need_TYPE (e.g. __need_size_t) definitions are an example.
We arrived at our current solution through trial and error and it seems
to work but there might be problems on other systems.

> * It's the printf/scanf requirements that seem hardest to meet separately 
> (hence this implementation having a modified copy of thousands of lines of 
> scanf code, which certainly isn't good practice for maintainability).
True. I agree it's bad practice to duplicate thousands of lines of scanf
code. We actually had two different (broken) scanf_s implementations
before we determined that there is no other viable solution than
directly modifying scanf. This is because the length check needs to be
put directly inside the function.
scanf_s differs from scanf mainly by expecting each buffer argument to
be followed by a buffer length. For example:
   scanf_s("%s", buffer, buffer_len);
One idea that we tried but had to give up on was to transform the given
buffer length arguments into format string width specifiers. However,
this approach did not work well because scanf's matching behavior
slightly differs for these two cases.  Adding the length check to the
existing scanf was a matter of adding around 10 new lines and could be
easily protected by #ifdefs.

An remaining problem for scanf_s is that GCC does not understand the new
scanf_s format string and cannot warn if the variable argument list does
not correspond to the format string. Something a long the way of
__attribute__((/format/(/scanf/s,m,n))) would be required.

The printf_s functions differ from the printf functions in that they do
not allow %n in the format string and are required to check for null
pointer arguments. At the moment, this is done with the help of the
glibc function parse_printf_format. This allows an implementation of the
printf_s functions to be a wrapper without duplicating code.

> * As already noted, there are namespace issues with implementing outside 
> of glibc, since the functions should only use implementation-namespace 
> external names and C11-namespace (including Annex K-namespace, of course) 
> external names, not POSIX or glibc names that for C11 are in the user 
> namespace.
>
> Given the above I suspect there may be advantages in implementing in glibc 
> - though possibly in a separate library from libc.so that glibc also 
> installs, rather than directly in libc.so.  (I don't know if there is any 
> existing practice from other POSIX systems regarding what library to put 
> these functions in.)  If a separate library built by glibc is used, 
> libc.so could export any non-C11-namespace symbols required with __* names 
> at GLIBC_PRIVATE symbol version.
In the beginning, we thought the Annex K functions could be simply
interpreted as wrappers that perform additional argument checks before
calling the original C functions. However, this approach does not work
for specific functions. For these functions, the only viable
implementation in my opinion is a direct modification of the original C
code.
In more detail:
1) There are functions where the length check is required to be
performed inside the original C function and cannot be performed before:
 * strtok_s
 * scanf_s
2) There are functions where the signature of the callback function has
changed. These functions both add a void *context argument to the
comparison function.
 * bsearch_s
 * qsort_s
3)  There is one other function where wrapping seems difficult: fopen_s
. The current implementation does not precisely correspond to the
standard. It's difficult to do so without directly modifying libc code.

--Ulrich