This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: documentation issues


Hello Roland,

Thank you for your lengthy and considered mail. Probably, I said most
of what I needed to say already, so I'll try to minimize repetition...

On Wed, Apr 25, 2012 at 11:11 AM, Roland McGrath <roland@hack.frob.com> wrote:
> This thread got quite long and strayed quite far from its original
> subject.  I won't attempt to answer every point raised individually.
> Instead, I'll just summarize my views on the larger issues raised.
> Accordingly, I have adjusted the Subject: line to be meaningful, and
> trimmed the CC list of everyone I'm sure is on the mailing list.
>
> There is strong consensus that Michael is a fine fellow and pleasant
> to interact with.  I concur.

You're so disarming... ;-).

> It's clear that the glibc manual as it stands has some major
> omissions and some (perhaps many) minor errors.  I certainly
> wouldn't deny it.  The omissions came about in the usual way: more
> effort was put into implementing new features than was put into
> documenting them.  Large areas of functionality such as pthreads did
> not exist at all when the great bulk of the work on the manual was
> done.  The manual was written mostly around 1991-1992, when it had
> the benefit of both a full-time technical writer (Sandra Loosemore)
> and major time commitments from technical experts (mostly me).  It
> was fairly complete at the time, though certainly imperfect.  (The
> pthreads interfaces did not exist in POSIX until 1996, and were not
> implemented in libc for some more years after that.)  Since then, it
> got far less attention for the simple reasons that there was no
> technical writer on the project and that the programmers on the
> project spent their time on code rather than on documentation.
> Various additions were made over the years by some of the
> programmers who worked on libc, but these never benefited from a
> fine writer like Sandra and some never even got edited by a native
> English speaker, both of which show in the quality of the text.
>
> I do not agree at all with the assertions that the Linux man-pages
> project constitutes the de facto documentation for glibc.

(I wouldn't have phrased it quite as Dave Miller did, but I'm pretty
sure that man pages are where the majority of programmers look first
for documentation of glibc APIs, which I imagine was David's point.)

> There are
> many sources of documentation that different people consult.  The
> overwhelming majority of the interfaces glibc provides are specified
> by standards; so for the most common needs, people have a large
> number of sources of information available, including the standards
> themselves, other systems' documentation, books, etc., many of which
> do not even purport to specifically document the interfaces of glibc
> per se.  For interfaces specific to glibc, or for details of
> standard interfaces (e.g. implementation-defined behavior) that are
> specific to glibc, there is often a lack of reliable documentation
> anywhere.  That is a sorry state of affairs, but the fact is that it
> reflects the true state that the exact interface details were often
> never clearly specified when the features were added, so it may be
> that nobody knows (or remembers), or nobody is entirely sure, what
> the intended specification of a given interface originally was or
> should be now.  For some interfaces, there were always fairly
> thorough (though usually rather tersely succinct) comments in the
> public header files describing the interfaces, but that is
> unfortunately not universal.
>
> I am all for friendly cooperation with any and all people working on
> freely redistributable documentation.

Great to hear! Thanks!

> When there are conventions
> and procedures we can adopt that aid those people without creating
> substantial new burdens for glibc developers, we should do so
> because it's the friendly thing to do.  That is quite different from
> saying that someone who wants to contribute new features to glibc is
> obliged to contribute work to some documentation project outside
> glibc.  I would object to any attempt to create such an obligation.
> It's also quite different from saying that when people are
> interested in contributing their efforts to improve glibc
> documentation, we would encourage them instead to contribute to some
> documentation project outside glibc.  Of course there is nothing
> wrong with helping potential contributors to be aware of all the
> various freely-redistributable documentation efforts going on in the
> world, nor with encouraging them to contribute to whichever projects
> they think are most worthy of their efforts.  People who are glibc
> developers certainly may offer their personal opinions about such
> subjects, and even do so on libc mailing lists (as long as that does
> not involve promoting non-free work such as books that are not
> freely redistributable--such is never appropriate on any GNU
> project's mailing lists).  Everyone offering a personal opinion
> should be obliged to make clear that personal is what it is, and not
> purport to represent it as the guidance of the glibc community.
> IMHO, the guidance of the glibc project should be that if a person
> wants to contribute to the glibc project in the area of
> documentation, then that person should contribute to glibc's
> documentation, which exists in the web pages, the wiki, and the
> manual.
>
> Assertions of the form "programmers want..." are useless
> overgeneralization, and frankly laughable.  Programmers are people.
> People want documentation that they find useful and/or pleasant to
> read.  What works best varies widely among different people.  I am
> a programmer, and I think Unix man pages are a lousy way to
> organize documentation.  I would certainly not presume to claim
> that this means that "programmers want" documentation in some other
> particular style.

Agreed. Above all, programmers want complete, useful, readable
documentation. And I agree that man pages are not a perfect format,
but they're sufficient when done completely and well (as would info
pages be, if done completely and well). I've rarely solicited opinions
on the subject, but every now and then I hear opinions expressed, and
I listen. Every opinion I've heard that expressed a preference favored
"man" format over "info". But this is arguable anecdotal, rather than
scientific.

> The libc manual was written with the original philosophy of GNU
> manuals.  That is, it's intended to be both tutorial and reference.
> (The mark-up format is a very small issue in comparison to the
> philosophy of a manual.)  It endeavors not only to explain, in
> complete detail, how to use each interface in the library, but also
> when and why.  When complete and correct, such a manual contains all
> the information necessary to use every aspect of every interface
> (the "reference").  But it is never simply a litany of features and
> details.  When written well, it also helps the reader navigate among
> the features available and decide which are most appropriate to use
> for particular situations (the "tutorial").  It's intended to be
> readable as a book, where the styles of navigation available are
> thumbing through the pages, a table of contents, indices, and cross
> references that cite page numbers.

(I think the "tutorial" aspect of the glibc manual is where it
succeeds best, but as things stand, as a "reference" it falls far
short of what programmers need. See below.)

> It's also intended to be useful
> as on-line documentation, where the styles of navigation available
> include hypertext methods within the manual and direct top-level
> pointers such as the system Info directory.  This has always been
> the philosophy behind the libc manual, and I would object to any
> attempts to recast it as a different kind of document.  The manual
> needs to be improved and expanded.  It does not need to have its
> purpose changed.
>
> If you like man pages and want to improve them, then work on that.
> It doesn't have anything to do with with contributing to glibc, but
> there are lots of places you can choose to contribute.  If you think
> the libc manual is hopeless or ill-conceived and you don't want to
> put time into improving it, then put your time elsewhere.  It's up
> to you to decide where you want to put your time and which, of all
> the contributions you might make, you think are the most beneficial
> to the user community.
>
> I don't have any real quarrel with what Joseph proposed as a
> standard for documentation accompanying code contributions to libc.
> But I also don't mind leaving the current absence of any formal
> documentation obligation on contributors.  New public interfaces
> being added to libc is relatively rare.  People reviewing proposed
> changes can always ask the contributor to add some documentation,
> without having a formal policy about it.
>
> I would like to see the libc manual improved in numerous ways.  But
> that is not where I plan to devote my contributions to the project.
> I think it would be best if we could inspire some new contributors
> who are skilled technical writers to do some major new work on the
> manual.  That could include writing whole new chapters on the large
> areas of functionality that aren't documented.  It could include
> general editing of the text (the portions added after 1992 or so
> could especially use such attention).  It could include filling in
> missing technical details in existing sections of the manual.  (That
> last task could be split into the task of enumerating what's
> missing, which could be done by someone who is moderately technical
> but not necessarily a skilled writer, and the actual writing, which
> may be more daunting to someone who is not an experienced writer.)
> We've been needing this work for a very long time and we haven't had
> people show up to do it.  But maybe things will be different now.
> If anyone does work on this, I'll be glad to review the work as a
> technical expert.

Roland, much of what you say makes sense, and I understand the
philosophical view point about the motivations of the glibc manual.

But, you said elsewhere "users mostly [...] just experiment and
observe, and then expect the behavior they observed to be preserved in
the future".

And therein lies the problem (that I already stated): programmers
using the many nonstandard interfaces (or the many unspecified details
of standardized interfaces) in glibc have no predictable contract
about glibc behavior. This is a systemic problem in glibc. By way of
example, here's a more or less complete list of functions added in
glibc >= 2.10 that (as far as I know) are not covered by any standard:

scandirat() 2.15
pthread_sigqueue() 2.11
malloc_info() 2.10
register_printf_modifier(), register_printf_type(),
register_printf_specifier() 2.10
pthread_setname_np(), pthread_getname_np() 2.12

fgetsgent(), fgetsgent_r() 2.10
getsgent(), getsgent_r() 2.10
sgetsgent(), sgetsgent_r() 2.10
getsgnam(), getsgnam_r() 2.10
putsgent() 2.10
setsgent(), endsgent() 2.10

(One might argue that the shadow group functions are like the
standardized passwd/group functions, and to a certain extent this may
be true, but the devil's in the undocumented details.)

As far as I can see, not one of the above functions is documented in
the glibc manual. (man-pages is a little better off, but also lacking:
the first two have man pages, and malloc_info() will have one very
soon; the rest have have no man pages yet.)

Quoting you at further length from the other thread:

> Sometimes those expectations are reasonable and sometimes they are not.
> We are responsible to all the users, past, present, and future--not just
> particular vociferous past users who rail against some change.  We are
> the experts and it's our responsibility to judge what's best overall,
> not that of some subset of users and not that of some man-page writer.
> Sometimes slavish compatibility with the past bad assumptions of
> application developers or outside documentation writers does a
> disservice to the users at large.  That's one reason we have major
> releases and minor releases.  Certainly controversial cases don't change
> in minor releases.

My reading (possibly incorrect) of your comments is: in cases where
documentation is absent (or silent on a point), users may experiment
and they may make (wrong) assumptions, but we reserve the right to
make changes to the behavior (possibly negating users' assumptions)
that we deem reasonable.

Now, I know that glibc doesn't gratuitously break the ABI, but my
reading of your comments is that they provide a rather weaker
guarantee than the statements from Linux kernel developers, where
ABI/API compatibility is an oft-stated and overriding priority (and
there is at least a part of the culture that emphasizes *developers*
contributing to man pages and other documentation).

To be concrete: lacking any "authorized" documentation for the
interfaces I listed above, programmers have the problem not only of
determining how to use the interfaces, but also determining what
aspects of the behavior of these interfaces they can rely on in the
future. That's a pretty uncomfortable position to be in. And it's not
clear that things are going to get better (here I am alluding to your
statement above "But I also don't mind leaving the current absence of
any formal documentation obligation on contributors").

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface"; http://man7.org/tlpi/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]