Extracting strings from Python beyond the terminating NULL

[Phil Muldoon <pmuldoon at redhat dot com>]

I've been working on the second part of this support for counted 
strings. I've been puzzled by an issue the last couple of days, and I'm 
fast running out of ideas ;) My issue is the way that Python extracts 
strings and passes them back to the C code (and in this case GDB).  Here 
is the narrative:

Now that we have support for counted strings (e.g. value.string(length = 
20)) we can capture strings beyond an embedded null (e.g. foo = 
bar.string(length = 7) where bar is "foo\0bar"). And that works: the 
full "foo\0bar" is captured. I've been working on handling these counted 
strings correctly within the Python code. So far so good. Pretty much 
throughout this code the fully captured string is preserved until it 
becomes time to turn it back into something GDB can print. At this point 
we progress:

#3  unicode_to_encoded_string (unicode_str=0x7ffff329de10,
   charset=0x743240 "ISO-8859-1")
   at ../../archer/gdb/python/python-utils.c:118
#4  0x00000000004bdbb4 in unicode_to_target_string 
(unicode_str=0x7ffff329de10)
   at ../../archer/gdb/python/python-utils.c:133
#5  0x00000000004bdbe9 in python_string_to_target_string 
(obj=0x7ffff329de10)
   at ../../archer/gdb/python/python-value.c:900
#7  0x00000000004b3823 in pretty_print_one_value (printer=0x7ffff31f1368,
   out_value=0x7fffffffd830) at ../../archer/gdb/python/python.c:850
#8  0x00000000004b39b6 in print_string_repr (printer=0x7ffff31f1368,
   hint=0x11fd900 "string", stream=0xbb8dd0, recurse=0,
   options=0x7fffffffd9b0, language=0x77af00)
   at ../../archer/gdb/python/python.c:907
   at ../../archer/gdb/python/python.c:1220


(The line numbers will be different to your code, mine is modified with 
the string preservation bits). Anyway, this is where the problem lay. As 
far as I an see, the only methods available to convert a Python string 
to a C string are ... you guessed it, null terminated:

http://docs.python.org/c-api/string.html?highlight=pystring#PyString_AsString

And the code in question uses that. So even though we dutifully preserve 
the embedded null, when the call to Python is made to extract the 
string, it is terminated on the first null. I'm hoping I missed a length 
based api call that some python hacker can point me too. 
PyString_Size(string) actually returns the correct (non null terminated) 
length btw.

Any ideas?

Regards

Phil