Re: get rid of getpwent? (Was: cygwin-1.7.28 getpwent header declaration changes ?)

[Warren Young <warren at etr-usa dot com>]

On 2/7/2014 02:49, Corinna Vinschen wrote:
> On Feb  6 14:43, Warren Young wrote:
> > On 2/6/2014 07:13, Corinna Vinschen wrote:
>
> it would, of course, be possible to implement Cygwin
> command line tools along the lines of useradd/usermod/groupdel.  For AD,
> they would just have to use LDAP,

If by "use LDAP" you mean the ldap_* functions in the OpenLDAP library, 
I can't recommend it.  (See my other post on LDAP books.)

Such programs need not be portable.  It doesn't look like it will be 
helpful to start with the ones from Linux's shadow-utils, since those 
modify /etc/foo directly.  They don't even attempt to abstract away 
/etc/foo, NIS and LDAP, so they're no use to us except as design 
exemplars.  Cygwin equivalents should use the same command line format 
and flags, they should use /etc/skel, etc.

I don't see why such programs shouldn't be written straight to the 
Windows API, even though this is naughty on Cygwin.  The Win32 security 
API fills the same role as libldap does on a Linux box configured for LDAP.

You're right that such programs are probably going to be necessary, if 
Cygwin moves to SAM/AD as primary.  Windows Home edition user management 
probably won't be powerful enough to do what Cygwin needs, if SAM is 
Cygwin's Single Point of Truth on such systems.

> > I don't see a reason for this to change, given that so many other
> > POSIX systems share aspects of this behavior.
>
> Sorry, I lost you there.  What exactly of the current behaviour do
> you want to keep?

I want getpwent() and friends to remain available, but to switch to 
AD/SAM as primary, like OS X does all the time, or like Linux does when 
LDAP is enabled in NSS.[*]

I want the mkpasswd and mkgroup utilities to remain available to 
generate these files on systems where there are programs that insist on 
scanning them directly.  Such files will be understood to be potentially 
stale caches of records that really live in SAM/AD.  Plus, these files 
are useful as debug dumps, and for casual grepping.

Corinna, an earlier post of yours suggested that /etc/foo was being kept 
as primary for speed reasons, but are you comparing to SAM or to AD? 
And have you tested it lately?

I don't think it's fair or even useful to compare the speed of /etc/foo 
to AD.  Sites using AD have implicitly said they're willing to pay a 
speed hit to get certain benefits.  The fact that AD is slower is like 
pointing out that container ships are slower than a Maserati.

I'd expect a SAM lookup to be *faster* than any /etc/foo lookup, even 
with very small /etc/foo files.

This takes 7.1 seconds on my system, with a 12-line /etc/passwd file:

     #include <pwd.h>
     #include <stdio.h>
     #include <stdlib.h>

     int main(int argc, const char* argv[])
     {
         int i;
         const char* user = argv[1];

         if (!user) {
             printf("usage: %s username\n", argv[0]);
             exit(1);
         }

         for (i = 0; i < 1000000; ++i) {
             struct passwd* pw = getpwnam(user);
             if (!pw) {
                 printf("User %s doesn't exist!\n", user);
                 exit(2);
             }
             else if (i == 0) {
                 printf("User %s is UID %d\n", user, pw->pw_uid);
             }
         }
     }

So, each getpwnam() call takes 7.1 microseconds on average.

It used to take 5.5 seconds, until I moved my user record down to the 
end of the /etc/passwd file, just to make it a worst-case test.  3 lines 
of movement added ~1.6 us to the call time.  Then I moved it to the top 
of the file, and the program ran in 1.2 seconds.  6x delta between best 
and worst case, in such a small file!

If you send me the Win32 code for a SAM equivalent, I'll run it here, 
for comparison.  Please keep the unnecessary overhead, like the 
printf()s and logic inside the main work loop, so we can null out the 
difference in the comparison.

> Also, the usual problem:  What exactly
> do you write to passwd/group?  If you write all users of a big corp,
> the files get really big.

If Cygwin moves to SAM/AD as primary, then this doesn't matter.

The disk space hit is small in modern terms.  Didn't someone calculate 
135 bytes on average per /etc/passwd line recently?  So, a massive 
100,000 user corporation's /etc/passwd would be 13.5 MB.  Big, but not 
TeX big.  Not even Vim big, these days.  Sigh.

In such a system, Joe User probably isn't allowed to pull all 100,000 
records anyway, for security reasons.  Won't mkpasswd skip all those?

The speed hit doesn't really matter, since it will only be paid by 
programs that insist on scanning these files directly, if AD/SAM are 
primary.  That's going to be things like misguided Perl scripts that 
parse the file directly, instead of using Perl's built-in getpwent() and 
such.

> > (I assume getpwnam() consults SAM/AD in Cygwin, now or in Cygwin.next.)
>
> It checks for the file first, then it asks SAM/AD.

I realize getting rid of /etc/foo checks in these functions is a very 
large change, from a system operations point of view.  Large enough to 
call the result Cygwin 1.9, IMO, simply to give people a forking point, 
to buy migration time.




-----

[*] The comparison to NSS breaks down quickly.  A "files" fall-back 
doesn't make sense, since you can't make up the proper NT security token 
to populate /etc/passwd with a Cygwin-only user that doesn't exist in 
SAM/AD.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple