Is there anyway to reduce cygwin perl's String memory usage?

[mike marchywka]

On 12/31/09, Reini Urban  wrote:
> 2009/12/30 Zhao, Dong:
>> Dear All:
>> Recently our Perl project run into the "Out of memory" issue on Windows
>> Cygwin, while it works well on Linux, after optimized the code ,it still
>> didn't work on Windows Cygwin perl(Because we do a lot of analysis bases
>> on string arrays and hashes) . We found the reason is Cygwin perl uses
>> much more memory than Linux perl or Windows As perl.
>> We did a lot searches over the internet but didn't found a solution yet.
>> I did tune the  "heap_chunk_in_mb" flag to "2048"  in registry
>> "Software/Cygnus Solutions/Cygwin/". But our Perl script still exhausted
>> over 1GB memory and crashes on Cygwin perl.
>>
>> Then I compared the memory usage difference between Cygwin perl, Windows
>> AS perl, and Linux Perl use this simple script:
>>
>>     my @array = ();
>>     my $str = "A" x 1024;  #A string with 1024 len
>>     for(my $i=0;$i<100*1024;$i++){
>>         push @array,$str;
>>     }
>>     <STDIN>; #No we can take a look at memory.
>>
>> I expected it to use about 100M memory for all platforms. But Windows
>> cygwin perl use about 200M, while AS perl and Linux perl only used 100M.
>> Our current cygwin perl version is v5.10.0. And cygwin version is 1.7.1.
>> I also tried the latest cygwin perl version : v5.10.1. But still the same
>> result.
>>
>> Can anyone explain to me?  It there a simple way to reduce the cygwin
>> perl's memory usage(Set some flag?Or recomple cygwin perl code)
>> Thanks for any answer!
>
> I 'll investigate. Best is always to pre-allocate huge memory to help
> those malloc calls.

Are there any additional memory management features available?
I gave up on a cygwin perl approach  and went to c++ due to VM thrashing and
didn't know if there were options to fix it. It turned out IIRC all I
needed from
perl was regex and I could get that from some other libraries. Even if you
get the total memory usage down, there may be other issues with access
patterns.

>
> We are using the standard perl malloc -Dusemymalloc=y (hints/cygwin.sh)
> Maybe the other platforms are using their system malloc.
>
> You can override that with -Dusemymalloc=n
>
> ../perl-5.10.1/Configure -de -Dlibperl=cygperl5_10_sysmalloc.dll
> -Dmksymlinks -Dusethreads -Dmad=y -Dusemymalloc=n
> (untested)
> --
> Reini Urban
> http://phpwiki.org/           http://murbreak.at/
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
>


-- 
marchywka@gmail.com
note new address 2009-12-16:
Mike Marchywka
1975 Village Round
Marietta GA 30064
415-264-8477 (w)<- use this
404-788-1216 (C)<- leave message
989-348-4796 (P)<- emergency only
marchywka@hotmail.com
Note: If I am asking for free stuff, I normally use for hobby/non-profit
information but may use in investment forums, public and private.
Please indicate any concerns if applicable.
Note: hotmail is censoring incoming mail using random criteria beyond
my control and often hangs my browser
but all my subscriptions are here..., try also marchywka@yahoo.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple