This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Excessive thrashing when popen()ing after a large malloc()


Eric Blake <ebb9 <at> byu.net> writes:
> Eric Blake <ebb9 <at> byu.net> writes:
>
> > >     else
> > >       sprintf(cmd, "sh -c '%s' %d>&-", program, pdes[1])
> > >   }
> > >   pid = spawnl(_P_NOWAIT, _PATH_BSHELL, "sh", "-c", cmd, NULL);
> > 
> > Why are you going through two levels of sh?  That seems like a waste to
> > me; the whole idea of using spawn is to avoid a fork(), but when you
> > invoke "sh" "-c" "sh -c 'cmd'", you are right back to a fork.  True, the
> > new invocation of sh uses less memory than the 1 GB process that invoked
> > popen, so less thrashing will occur, but your whole approach seems
> > fundamentally flawed if you are trying to use spawn to avoid a fork.
>
> You know, maybe a better approach to this would be to first implement 
> posix_spawn() in cygwin, since that API seems like it would be possible
> to do the necessary fd manipulation in a spawned child process without the
> penalty of a full fork/exec.  Then you could teach popen how to use
> posix_spawn when it is available, which is probably a patch more likely to
> be accepted in newlib, rather than your current idea of having to go through
> an intermediary sh invocation.

    That intermediate sh invocation can probably be slightly streamlined
  by doing something similar to

sprintf(cmd, "(%s) >&%d %d>&- %d>&-", program, pdes[1], pdes[0], pdes[1]);
pid = spawnl(_P_NOWAIT, _PATH_BSHELL, "sh", "-c", cmd, NULL);

  but you misunderstand my objective: I'm not "trying to have my patch
  accepted", I'm just signaling a problem and proposing a patch for the
  upstream maintainers: cygwin and/or newlib. I've modified the program
  that uses popen and it is now using a _popen which is a kludge around
  the ideas that I have suggested. If my ideas are useful for anybody else,
  I'm really happy to contribute and help; if nobody cares, neither do I.

> But I do not have a copyright on file for cygwin contributions, so I am
> currently unable to work on contributing posix_spawn;

    That's very nice of you to suggest it.

> you are at the mercy of someone willing and able to write such a patch.

    Actually, as said above, /I/ am not at the mercy of anyone, but anybody
  else who popen()'s with a large memory space is.

> -- 
> Eric Blake

    Loïc Grenié

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]