This is the mail archive of the
libc-help@sourceware.org
mailing list for the glibc project.
Re: Memory consumption of iconv
- From: "Carlos O'Donell" <carlos at systemhalted dot org>
- To: dl9obn at darc dot de
- Cc: "libc-help at sourceware dot org" <libc-help at sourceware dot org>
- Date: Fri, 21 Mar 2014 14:00:12 -0400
- Subject: Re: Memory consumption of iconv
- Authentication-results: sourceware.org; auth=none
- References: <2766265 dot zxRTdxtCPy at gentoo> <532C6154 dot 4050404 at gmx dot de>
On Fri, Mar 21, 2014 at 11:57 AM, Dirk Bächle <tshortik@gmx.de> wrote:
> Hi Oliver,
>
>
> On 21.03.2014 12:53, Oliver Becker wrote:
>>
>> Hi,
>> I am developing a server software which spawns up to thousands of client
>> processes which use iconv_open (through Qt) to later transform strings
>> between
>> different encodings. This works like a charm of course but after a while I
>> recognized that every process used about 1MB (non shared) memory for the
>> iconv
>> data.
>> This allone is not that much but If you multiply it by the number of child
>> processes, I get a huge size of multiple GB. If you take into account that
>> I
>> use the same locales in every child process this is not easy to
>> understand.
>>
>
>
> I've opened a related thread for the build system SCons, see
>
> https://www.sourceware.org/ml/libc-help/2014-01/msg00038.html
>
> . Seems that we both suffer from the same behaviour (all memory of the
> parent process gets copied for a spawned child), so I'm very interested in
> any answers you get. ;)
Use vfork() if possible? You can't get away from copying the entire address
space unless you are doing a very restricted set of operations after the
*fork(). Alternatively use system() and expect that the implementation does
a good job.
Cheers,
Carlos.