This is the mail archive of the libc-alpha@sourceware.cygnus.com mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]

Re: strtok bug


On Thu, 13 Jan 2000, Geoff Keating wrote:

> > int main() {
> > 	strtok("",":");    /* returns NULL */
> > 	strtok(NULL,":");  /* should return NULL - but dups core on ppc */
> > }
> 
> I think you've found a bug in the C standard.  The standard says:

[ snip quote ]

> The problem is that the standard doesn't say that it saves a pointer
> when the first call returns a null pointer.

This is not a problem at all. Why should it?  The strtok function is intended
to extract tokens one by one. When it returns null, the search is over due to
hitting the end of string.  You must start a new search.

There is some extra behavior that can be deduced from the standard's
description; namely once you hit the end of the string after extracting one or
more tokens, you can keep calling strtok with a null argument to try to extract
more tokens, but it will keep failing.

This behavior isn't required if the first call doesn't extract a token.

The deducible behavior is a consequence of having to keep the pointer from one
call to the next. The standard makes the minimum possible requirements with
respect to the saving of the pointer. In the situation when no token can be
extracted, the pointer is not required to saved. If it was already saved from
previous extraction, there is no state change in this case making it possible
to perform any number of additional calls with the first argument null. That is
simply not useful behavior; a reasonably written program will terminate its
search after the first null return.

It's not clear why strtok is defined this way in the standard. Either 
it's a deliberate attempt to make the fewest requirements, or it 
reflected the behavior of some existing implementations. 

The Rationale is not helpful, unfortunately. All it does is provide an
apologetic excuse for why this function was included in the standard at all:

    This function has been included to provide a convenient solution to many
    simple problems of lexical analysis, such as scanning command line
    arguments.

Ah well. I never want to think about strtok again! ;)


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]