Shared memory handling for mixed C/FORTRAN program
Christoph Weise
cfweise@yahoo.com
Sat Apr 11 20:25:00 GMT 2015
>On Apr 10 22:20, Christoph Weise wrote:
>> >PAGESIZE on Cygwin is not 1024, and the right value to use for
>> >XSI SHM is SHMLBA (== 64K on Cygwin)
>>
>> Setting PAGESIZE to SHMLBA creates problems elsewhere in the program
>> (then PAGESIZE is too big for the program to handle, a problem I have
>> yet to debug) so only unless there is no workaround (please see below)
>> I'd like to avoid trying to use SHMLBA for the time being.
>
>The PAGESIZE is an OS-specific value. If you set it to some arbitrary
>value and use it for your own purposes it's ok, but if the application
>expects the OS to follow its lead, you'll run into problems.
>
>> Regardless of the choice of PAGESIZE, only a total of 4kB can be
>> stored sequentially in the shared memory section handed by shmat. If I
>> try to write more than this number of bytes to the shm section the
>> program crashes.
>
>Not that I know of.
Please see below.
>> > Did you check if it works from plain C? If not, do you have a
>> > simple testcase in plain C?
>>
>> I modified the first C utlity, which sets up the shared memory
>> section, to use the addresses it receives from shmat to try to write
>> as much data as possible into the section. I could read/write to the
>> full shm section from within that utility.
>
>Which means, the memory has been allocated the right size.
>
>> I also modified a second C routine which looks for the memory section
>> and returns addresses into the shm section. When attempting to
>> read/write within that second routine the program crashes when
>> accessing addresses more than 4 kB from the start of the shm section.
>
>Is that inside the same executable?
No, they are separate executables, please see below.
>
>> I found this
>> https://www.cygwin.com/ml/cygwin/2009-01/msg00492.html
>> and links from there, which suggest that this 4kB "barrier" is probably related to cygwin's memory handling.
>
>That has nothing to do with it. It's just a question I replied to.
>Cygwin always returns memory in 64K chunks because that's what the
>underlying OS does.
>
>Unless the OS is asked to return shared memory backed by a file. In
>this case it returns memory ending on a 4K boundary. That's weird, but
>Cygwin has code trying to workaround this problem. But that doesn't
>affect shmat.
>
>> If this is in fact a likely source of the problem,
>
>No, it isn't. If you can provide a simple testcase in plan C which
>allows to reproduce the issue with minimal code, I'd take a look into
>the issue.
>
Please see below, I provide minimal C code for three separate executables, one creates the shm section, another finds it, the third removes it. I include also a bash test script that executes the routines in order. Please beware as I removed some checks to reduce the length of the code, but it should run ok. The PAGESIZE parameter is hardcoded (here 512 bytes). The desired shared mem section size is set interactively as input to makeshm, or automatically with the bash script.
I can write to the shm section with the second routine when the requested section <= 4096 bytes. Otherwise it's not happy.
Thanks for your help.
#####################################################################
bash test script
#####################################################################
#!/bin/bash
./makeshm <<EOF
4096
EOF
# try >4096 for a seg fault
# check for shms
ipcs
./findshm
./removeshm
# check for shms
ipcs
#####################################################################
makeshm.c
#####################################################################
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/stat.h>
#undef PAGESIZE
#define PAGESIZE 512
#define MYKEY 0xEDCB /* A code for our workspaces */
#ifndef IPC_ALLOC
#define IPC_ALLOC 0
#endif
int main(int argc, char **argv)
{
unsigned uid;
key_t key;
int shmid;
int shmsuccess;
struct shmid_ds sbuf;
int i, size;
char *p, *shmaddr;
char answer[10];
printf("-------------IN MAKESHM----------------\n");
/* Check for override environment variable set */
uid = getuid();
key = (MYKEY << 16) | (uid & 0xFFFF);
shmid = shmget(key, PAGESIZE, IPC_ALLOC);
if (shmid < 0){
printf("Creating the shared memory section\n");
/* Now try to create it */
printf("Enter buffer size (bytes): ");
fgets(answer, sizeof(answer), stdin);
i = sscanf(answer, "%d", &size);
printf("%d\n",size);
shmid = shmget(key, size, IPC_CREAT | 0666);
}
else {
perror("Could not create shared memory section");
return 1;
}
/* Print the section information */
shmsuccess = shmctl(shmid, IPC_STAT, &sbuf);
/* Attach to the shared memory section */
shmaddr = 0;
p = shmat(shmid, shmaddr, (S_IRUSR | S_IWUSR));
if (-1 == (int) p)
perror("Unable to attach to the section\n");
else {
printf("Attached at address %x.\n", p);
}
/* !!! hardcoded !!! */
int foffset = 1020;
printf("Check writing into the buffer at addresses starting at %x + %d bytes (buffer size: %d bytes)\n",p,4*foffset,sbuf.shm_segsz);
float *pfloop = (float*) (p + foffset*4);
float *piorig = (float*)p;
*piorig = (float)0;
printf("storing in address %x (%x + %d floats): %f\n",piorig,piorig,0,*piorig);
printf(" .................\n");
for (i=0;i<sbuf.shm_segsz/4 -foffset;i++,pfloop++) {
*pfloop = (float)(i+foffset);
printf("storing in address %x (%x + %d floats): %f\n",pfloop,piorig,i+foffset,*pfloop);
}
printf("--------------------------------------------\n");
/* Detach from the memory section */
if (0 == shmdt(p))
printf("Detached from the shared memory section.\n");
else
perror("Could not detach the section");
return 0;
}
#####################################################################
removeshm.c
#####################################################################
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#undef PAGESIZE
#define PAGESIZE 512
#define MYKEY 0xEDCBu /* A code for our workspaces */
#ifndef IPC_ALLOC
#define IPC_ALLOC 0
#endif
int main(int argc, char **argv)
{
unsigned uid;
key_t key;
int shmid;
struct shmid_ds sbuf;
int i;
printf("-------------IN REMOVESHM----------------\n");
uid = getuid();
key = (MYKEY << 16) | (uid & 0xFFFF);
shmid = shmget(key, PAGESIZE, IPC_ALLOC);
i = shmctl(shmid, IPC_RMID, &sbuf);
if (0 == i)
printf("Removal was successful.\n");
else {
perror("Error in removal");
return 1;
}
return 0;
}
#####################################################################
findshm.c
#####################################################################
#include <stdio.h>
#include <stdlib.h>
#include <string.h> /* memcpy */
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/stat.h>
#undef PAGESIZE
#define PAGESIZE 512
#define MYKEY 0xEDCBu /* A code for our workspaces */
#ifndef IPC_ALLOC
#define IPC_ALLOC 0
#endif
int main(int argc, char **argv)
{
int uid;
key_t key;
int shmid;
int shmsuccess;
char *p, *shmaddr;
struct shmid_ds sbuf;
int npages;
printf("-------------IN RUNME/FINDSHM----------------\n");
printf("Mapping the shared memory section ... \n");
uid = getuid();
/* See that the section exists */
key = (MYKEY << 16) | (uid & 0xFFFF);
shmid = shmget(key, PAGESIZE, IPC_ALLOC);
/* Determine the shared memory size */
shmsuccess=shmctl(shmid, IPC_STAT, &sbuf);
/* Attach to the shared memory section and check for other users */
shmaddr = 0;
p = shmat(shmid, shmaddr, (S_IRUSR | S_IWUSR));
if (-1 == (int) p) {
perror("Unable to attach to the section\n");
return -1;
}
else {
printf("Attached at address %x.\n", p);
}
/* Store the pointers */
char *pptr = p;
float *cptr = (float *) (p + PAGESIZE);
printf("--------------------------------------------\n");
printf("Checking memory pointers \n");
printf("pptr: %x\n",pptr);
printf("cptr: %x\n",cptr);
printf("cptr-pptr: %i\n",((char *)(cptr))-(pptr));
printf("--------------------------------------------\n");
/*check contents of buffer */
int i, foffset = 1020;
printf("Check writing into the buffer at addresses starting at %x + %d bytes (buffer size: %d bytes)\n",p,4*foffset,sbuf.shm_segsz);
float *pfloop = (float*) (p + foffset*4);
float *piorig = (float*) p;
*piorig = (float)0;
printf("storing in address %x (%x + %d floats): %f\n",piorig,piorig,0,*piorig);
printf(" .................\n");
for (i=0;i<sbuf.shm_segsz/4 -foffset;i++,pfloop++) {
printf("content of address %x (%x + %d floats): %f\n",pfloop,piorig,i+foffset,*pfloop);
*pfloop = (float)2*(i+foffset);
printf("storing in address %x (%x + %d floats): %f\n",pfloop,piorig,i+foffset,*pfloop);
}
printf("--------------------------------------------\n");
return 0;
}
#####################################################################
#./makefile#
#####################################################################
.SUFFIXES: .f .c .o .a
CC= gcc
CFLAGS= -c -ansi
.c.o:
$(CC) $(CFLAGS) -c $<
EXEFILES= makeshm removeshm findshm
all: $(EXEFILES)
makeshm: makeshm.c
$(CC) makeshm.c -o makeshm.exe
chmod 4755 makeshm.exe
removeshm: removeshm.c
$(CC) removeshm.c -o removeshm.exe
chmod 4755 removeshm.exe
findshm: findshm.c
$(CC) findshm.c -o findshm.exe
#
# Cleanup of non-source files
#
clean:
rm -f 2>/dev/null *~ *.o *.exe
#####################################################################
typical output
#####################################################################
$ ./test1.exe
-------------IN MAKESHM----------------
Creating the shared memory section
Enter buffer size (bytes): 4096
Attached at address 7fea0000.
Check writing into the buffer at addresses starting at 7fea0000 + 4080 bytes (buffer size: 4096 bytes)
storing in address 7fea0000 (7fea0000 + 0 floats): 0.000000
.................
storing in address 7fea0ff0 (7fea0000 + 1020 floats): 1020.000000
storing in address 7fea0ff4 (7fea0000 + 1021 floats): 1021.000000
storing in address 7fea0ff8 (7fea0000 + 1022 floats): 1022.000000
storing in address 7fea0ffc (7fea0000 + 1023 floats): 1023.000000
--------------------------------------------
Detached from the shared memory section.
Message Queues:
T ID KEY MODE OWNER GROUP
Shared Memory:
T ID KEY MODE OWNER GROUP
m 12845056 3989505001 --rw-rw-rw- John None
Semaphores:
T ID KEY MODE OWNER GROUP
-------------IN RUNME/FINDSHM----------------
Mapping the shared memory section ...
Attached at address 7fea0000.
--------------------------------------------
Checking memory pointers
pptr: 7fea0000
cptr: 7fea0200
cptr-pptr: 512
--------------------------------------------
Check writing into the buffer at addresses starting at 7fea0000 + 4080 bytes (buffer size: 4096 bytes)
storing in address 7fea0000 (7fea0000 + 0 floats): 0.000000
.................
content of address 7fea0ff0 (7fea0000 + 1020 floats): 1020.000000
storing in address 7fea0ff0 (7fea0000 + 1020 floats): 2040.000000
content of address 7fea0ff4 (7fea0000 + 1021 floats): 1021.000000
storing in address 7fea0ff4 (7fea0000 + 1021 floats): 2042.000000
content of address 7fea0ff8 (7fea0000 + 1022 floats): 1022.000000
storing in address 7fea0ff8 (7fea0000 + 1022 floats): 2044.000000
content of address 7fea0ffc (7fea0000 + 1023 floats): 1023.000000
storing in address 7fea0ffc (7fea0000 + 1023 floats): 2046.000000
--------------------------------------------
-------------IN REMOVESHM----------------
Removal was successful.
Message Queues:
T ID KEY MODE OWNER GROUP
Shared Memory:
T ID KEY MODE OWNER GROUP
Semaphores:
T ID KEY MODE OWNER GROUP
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
More information about the Cygwin
mailing list