This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC][PATCH] Add new getgrgid2(), getgrnam2() interfaces to glibc


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/18/2010 07:42 PM, Petr Baudis wrote:
> This RFC patch adds support for new interfaces: getgrgid2(),
> getgrnam2() and their *_r() variants. These interfaces allow
> the user to specify whether the group.gr_mem field shall be
> filled in.
> 
> The issue we are trying to solve is that needlessly high load is
> generated in large deployments with NIS/LDAP user groups with possibly
> thousands of members. Even when nscd is used, the information is
> requested and transferred frequently even when there never arises
> any real use for the huge lists of members at the node at all.
> 
> We have looked at some commonly used applications and e.g. for a normal
> desktop, if some basic utilities and file managers would be trivially
> modified to use the new interface, the full lists of users in a group
> would never have to be downloaded. Right now, even ls will trigger
> getting full lists of users in all groups it encounters. Moreover, e.g.
> the sssd guys are reportedly trying to solve this problem by evil hacks
> like randomly not filling or only partially filling the gr_mem field.
> 

Actually, your information is a little bit incorrect. We recently
discovered that we had a few bugs in SSSD that were causing us to only
present a subset of the gr_mem field. This was entirely incorrect, and
as of SSSD 1.2.4 and 1.4.0 we have rectified this problem.

The way we do things in SSSD is, however, designed to significantly
reduce the load on LDAP servers. When we perform a group lookup with
getgrnam() or getgrgid(), we request ONLY this group from the LDAP
server. What we then do on our client-side is to create in our cache
mechanism a series of "fake" users in our data store. These users will
be reported back to getgrnam(), but they are not separately looked up
and populated with their complete user information unless they are
directly requested by a getpwnam() or getpwid() request.

The net result is that we will report all of the users that belong in a
group, but we will not make additional requests to populate those users
unless they are directly requested.

Yes, for groups that have many thousands of users, we may make a single
request that transfers a couple hundred kilobytes, but we do not make
subsequent requests for each of those users in turn.

RFC2307bis servers require slightly more processing, as we need to make
multiple requests to LDAP if there are nested groups in play. But we
never make more requests than the configured nesting limit.

> Any comments on the API are welcome, including thoughts on how to
> make it more elegant. Originally, I wanted to implement it as
> gid_t getgidbyname(const char *name) etc., but I would have to redo
> some parts of the NSS infrastructure to easily allow returning gid_t
> instead of a struct.
> 
> If there is a consensus that this is a good idea and Ulrich gives his
> blessing, I will of course extend the patch to be compatible with older
> NSS modules and add NIS support and nscd support.
> 


I'm really not sure what the value of this approach is. Realistically,
you're talking about creating an interface whose sole purpose is to map
GID<->groupname and nothing else?

I'm not sure there's sufficient value in this approach, given that it's
possible as described above to limit the network requests. I think it
makes much more sense to guide the respective name-service providers
along the path of handling these requests more efficiently.


I'm expanding this mailing to include the sssd-devel list for broader
discussion.

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAky9iTYACgkQeiVVYja6o6PwsACaAsWK6+lv/jsVLqQJhsml00hB
z8IAni+cKfv2z9PNWLSe6qKO8Ol/KP53
=PeBB
-----END PGP SIGNATURE-----


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]