PyIter_Check false positives

airplanemath airplanemath@aol.com
Fri May 27 21:26:30 GMT 2022


I found this problem running the tests for pandas, but can reproduce it
more simply.

Compile the following cython file to create a thin wrapper around the
C-API function:

> from cpython.iterator cimport PyIter_Check
> 
> def is_iterator(obj: object) -> bool:
>     return PyIter_Check(obj)

with `cythonize --build --inplace test_iterator.pyx` (probably requires
python39-cython), changing the name if needed, then try the following in
Python:

>>> from collections.abc import Iterator
>>> from fractions import Fraction
>>> from os import environ
>>> from test_iterator import is_iterator
>>> isinstance(environ, Iterator)
False
>>> is_iterator(environ)
True
>>> next(environ)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '_Environ' object is not an iterator
>>> isinstance(Fraction(0, 1), Iterator)
False
>>> is_iterator(Fraction(0, 1))
True
>>> next(Fraction(0, 1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Fraction' object is not an iterator

>From the Python documentation [1], it appears that `PyIter_Check` is
intended to return True only if the `PyIter_Next` will work, and
`PyIter_Next` is the C equivalent of the python `next` function.  That
is, `isinstance(thing, Iterator)` and `is_iterator(thing)` should agree
with each other, and should return True only if `next(thing)` works
(produces an element or says there aren't any).
On Linux, both checks agree with each other, returning False, and the
attempts to advance the iterator with `next` still fail.
As seen above, on (my) Cygwin, PyIter_Check disagrees with
`collections.abc.Iterator` and succeeds in cases where `next` produces a
TypeError.  I can reproduce this with python3.9 and with python2.7 on Cygwin.

Am I missing something in the documentation?  Is this a side-effect of
working on top of Windows?

In case it's relevant, I'm using an unpatched Cython with the
distribution Python packages.

[1] https://docs.python.org/3/c-api/iter.html#c.PyIter_Check

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_iterator.pyx
Type: text/x-python3
Size: 308 bytes
Desc: PyIter_Next Cython wrapper module source
URL: <https://cygwin.com/pipermail/cygwin/attachments/20220527/f651eb07/attachment.bin>


More information about the Cygwin mailing list