python errors with popen/threads/select

[Satish Balay]

We are encountering errors in our python code when invoking system
commands using popen2.Popen3() in a separate thread. This code works
fine on linux [and many other OSes ] - but not on cygwin/python/WinXP.

I'm attaching a minimal example code that demonstrates this problem.
This test code runs the command '/bin/true' in 2 modes. [multiple
times in a loop]

- run the command in a separate thread [using popen]
- run the command directly [using popen]

The thread version fails numerous times [with wait() on pipe returning
error codes] - whereas the one run directly does not.

Hoping the bug can be identified.

thanks,
Satish

----------------------------

$ /usr/bin/cygcheck.exe -c cygwin python
Cygwin Package Information
Package              Version        Status
cygwin               1.5.24-2       OK
python               2.5-1          OK

$ python th-bug.py 
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
pipe.wait() returns error code: 1280
thread+pipe overhead 2.47299981117
pipe overhead 2.26300001144
-------------- next part --------------
import popen2
import threading
import time
import os
timeout = 10

def openPipe(command):
  '''We need to use the asynchronous version here since we want to avoid blocking reads'''
  pipe   = popen2.Popen3(command, 1)
  input  = pipe.tochild
  output = pipe.fromchild
  err    = pipe.childerr
  return (input, output, err, pipe)

def runShellCommand(command):
  import select

  ret        = None
  out        = ''
  err        = ''
  count      = 0
	
  (input, output, error, pipe) = openPipe(command)
  input.close()
  lst = [output, error]
  while lst:
    ready = select.select(lst, [], [])
    count = count + 1 # somehow this line triggers the bug more frequently
    if len(ready[0]):
      if error in ready[0]:
        msg = error.readline()
        if msg:
          err += msg
        else:
          lst.remove(error)
      if output in ready[0]:
        msg = output.readline()
        if msg:
          out += msg
        else:
          lst.remove(output)
  output.close()
  error.close()
  if pipe:
    ret = pipe.wait()
  if ret: print 'pipe.wait() returns error code:', ret
  return (out, err, ret)

#####################################################


def runThreadShellCommand(command):
  thread = threading.Thread(target = runShellCommand, name = 'Shell Command', args = (command,))
  thread.setDaemon(1)
  thread.start()
  thread.join(timeout)
  return 0

time1 = time.time()
for count in range(0,30):
  runThreadShellCommand('/bin/true')
time2 = time.time()
print 'thread+pipe overhead',time2-time1

time1 = time.time()
for count in range(0,30):
  runShellCommand('/bin/true')
time2 = time.time()
print 'pipe overhead', time2-time1
-------------- next part --------------
--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/