Mutex and Shared memory Problem

QNX USERS QNX6 - Applications
1

Hi
I am porting some linux diver and application to QNX. In Linux it uses a shared memory to access mutex across processes. The following is the pseudo code used in Linux

fd = creat(//dev/shm/shmfile, 0664);
ftruncate(fd, pagesize);
fd = open(/dev/shm/shmfile, O_RDWR);
ptr = mmap(0x40000000, ps, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

I am following same steps in QNX. Instead of creating the shared memory file in /dev/shm/ it is created under /dev/shmem

But the mutex shared using this method is not able to be accessed on the other process when the shared memory was mapped. Do I need to use shm_open/shm_ctl/ instead of creat/open?

thanks
Dilip.

2

Please show your mutex initialization code.

Recall:

[code]NAME

pthread_mutexattr_getpshared, pthread_mutexattr_setpshared - set and get process-shared attribute

SYNOPSIS

#include <pthread.h>

int pthread_mutexattr_getpshared(const pthread_mutexattr_t *attr,
    int *pshared);
int pthread_mutexattr_setpshared(pthread_mutexattr_t *attr,
    int pshared);

[/code]

DESCRIPTION

The pthread_mutexattr_getpshared() function obtains the value of the process-shared attribute from the attributes object referenced by attr. The pthread_mutexattr_setpshared() function is used to set the process-shared attribute in an initialised attributes object referenced by attr.

The process-shared attribute is set to PTHREAD_PROCESS_SHARED to permit a mutex to be operated upon by any thread that has access to the memory where the mutex is allocated, even if the mutex is allocated in memory that is shared by multiple processes. If the process-shared attribute is PTHREAD_PROCESS_PRIVATE, the mutex will only be operated upon by threads created within the same process as the thread that initialised the mutex; if threads of differing processes attempt to operate on such a mutex, the behaviour is undefined. [b]The default value of the attribute is PTHREAD_PROCESS_PRIVATE[/b]
3

This is the mutex initialization code.

rv = pthread_mutexattr_init(&attr);
if (rv) {
	perror("Error initializing mutex attr\n");
	exit(-1);
}
rv = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);
if (rv) {
	perror("Error setting mutex attr recursive\n");
	exit(-1);
}
rv = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
if (rv) {
	perror("Error setting mutex attr shared\n");
	exit(-1);
}
rv = pthread_mutex_init(&lib_glob->mutex, &attr);
if (rv) {
	perror("Error initializing mutex\n");
	exit(-1);
}
/*
 * Init the alloc pool mutex
 */
pthread_mutex_init(&lib_glob->allocpoolmutex, &attr);

Does the creat() call instead of shm_open() makes any difference? Also, how to print the mutex number after pthread_mutex_init() call?

4

Sure does (I thought that your original comment implied you were already looking into that)

Here is a code snippet that shows how Posix shm_open is used (consider the code with “#define USE_POSIX_SHM 1”).

google.com/codesearch/p?hl=e … q=shm_open

with proper shm_open() call from above snippet, and with the shared attribute set on the mutex, it should work fine for you.

5

I am not able to understand the difference between the shared memory created using creat() and shm_open(). Able to print some variable initialized in one process on the other process when used creat() call for making shared memory and share the variable. But when tried to lock the mutex same way, the program crashed giving following error

Process 45071 (dev-axec) terminated SIGSEGV code=1 fltno=11 ip=fe321450(libc.so.3@pthread_mutex_lock+0x24) mapaddr=00021450. ref=40122018
Memory fault

6

Using creat() is not posix, so it doesn’t matter. Use shm_open() it is the one and only correct way to create a shared memory object.

7

Finally got the shared mutex working fine with both creat() and shm_open() calls. The only change done was adding MAP_FIXED flag in mmap call after which the memory got mapped to the required location which was not happening without this flag.

8

MAP_FIXED?

I didn’t see you mention anywhere that this was device memory. Also, putting a mutex in device memory is unusual.

I now see the 0x40000000, but since this is ignored without MAP_FIXED, and since you never mentioned that this was device memory it didn’t seem relevant.

I am curious as to why you are putting a mutex in device memory? Surely the device itself doesn’t understand what a posix mutex is?

In any case, stick with shm_open(), as it has defined semantics. Creating an object in /dev/shmem is not necessarily the same as creating an object with shm_open().

Good to hear it is working for you, but if this is device memory, you may need to consider adding PROT_NOCACHE as well (depending on whether the device memory is cache-coherent or not).

9

This is done for a driver which handle communication between PowerPC core and DSP core using hardware FIFO’s. QNX is running on PPC. The driver was originally written for Linux when linux was running on PPC. I am also not very sure of the design since it was not documented. Also I am very new to QNX. So I would like to understand more about this issue.

  1. How are you telling that the mutex is in the device memory? What I was thinking is that the address 0x40000000 is just a virtual address in process address space where the shared memory region should be mapped.

  2. What you mean by stating device memory? Is it the physical memory, I mean DRAM address?

  3. Another interesting thing I noticed was, when the MAP_FIXED was not there in flags, the PROCESS1 mmap() return address was 0x40122000 and the PROCESS2 mmap() return address was 0x40000000 even though the mmap() call was same,
    ie,
    ptr = mmap(0x40000000, ps, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

Following is the structure shared using shared memory between processes,

struct
{
pthread_mutex_t mutex1;
pthread_cond_t cond1;
pthread_mutex_t mutex2;
int testpattern;
}

The variable test pattern was initialized to 0xabcdef in PROCESS1 and after the mmap() in PROCESS2, i was able to print the value properly. But when tried to lock the mutex, the program crashed. Any idea why is this happening?

10

Sorry my bad. The first arg is not the physical address, but the VA hint. I was thinking that this was the address specified as the last arg (the physical offset).

Ok, so it sounds as if the code you’re porting assumes that the pointer address returned by the mmap call will be the same in different processes (I am surprised it worked on Linux).

You need to change the mmap call to set the first arg to zero; remove MAP_FIXED, then adjust the code to derive pointers into the shared memory area by adding offsets to the pointer returned from mmap().

Is there any place in this code where the last arg to mmap is non-zero?

11

No. There is no place where the last arg to mmap is non zero. BTW, in linux without using MAP_FIXED flag the mmap returns 0x40000000 as return address.

12

Shared memory create always problems. It is my owned experience…that’s why i always recommend the use of non-sharing memory.

13

You should not recommend the use of non-sharing memory (double negative). Instead you should learn how to use it properly.