Multi threaded resource manager with a twist

I have a requirement wherein I need to go with single resource manager and 2 threads.
These 2 threads will be accessing the same device but different parts of the memory. SO I need to allow the client apps to have concurrent access to the same device.
So I came up with a design wherein each thread registers a separate pathname space and client apps should be able to read/write registers at the same time. So that means io_read/io_write/io_devctl function may need to be accessed at the same time since they need to have concurrent access to the device. I couldn’t follow the sample code in QNX on “ multiple threaded resource manager” since access to io_read/io_write are synchronized (allows only one access at a time)
So I wrote a code like this. Please let me know if my method has any other repercussions. Or is there any elegant way of achieving this. The thing I don’t like about my method is I need to duplicate the code for io_read, io_write etc so that clients can access the device at the same time. Is there anyway I can avoid these code duplication and achieve the same result ?

Thanks a lot in advance.

My code:

#define NumDevices 2
char *names [NumDevices] =
{
“/dev/fpgaDev”,
“/dev/mdmDev”
};

int createThread2 ();

void* Thread2Handler (void*) ;

//Reg first device and create second thread for handling msgs to access other part of the memory

int main(int argc, char *argv)
{
/
declare variables we’ll be using */
resmgr_attr_t resmgr_attr;
dispatch_t *dpp;
dispatch_context_t *ctp;
int id;

/* initialize dispatch interface */
if((dpp = dispatch_create()) == NULL) {
    fprintf(stderr,
            "%s: Unable to allocate dispatch handle.\n",
            argv[0]);
    return EXIT_FAILURE;
}

/* initialize resource manager attributes */
memset(&resmgr_attr, 0, sizeof(resmgr_attr));
resmgr_attr.nparts_max = 1;
resmgr_attr.msg_max_size = 2048;

/* initialize functions for handling messages */
iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &connect_funcs,
                 _RESMGR_IO_NFUNCS, &io_funcs);

connect_funcs.open = io_open;
io_funcs.read = io_read;
io_funcs.write = io_write;
io_funcs.devctl = io_devctl;
	
/* initialize attribute structure used by the device */
iofunc_attr_init(&attr, S_IFNAM | 0666, 0, 0);

/* attach our device name */
id = resmgr_attach(
        dpp,            /* dispatch handle        */
        &resmgr_attr,   /* resource manager attrs */
        names[0],  /* device name            */
        _FTYPE_ANY,     /* open type              */
        0,              /* flags                  */
        &connect_funcs, /* connect routines       */
        &io_funcs,      /* I/O routines           */
        &attr);    /* handle      */

if(id == -1) {
    fprintf(stderr, "%s: Unable to attach name.\n", argv[0]);
    return EXIT_FAILURE;
}


//create Thread2.
createThread2();

 
/* allocate a context structure.The context structure contains a buffer where messages will be received. The size of
the buffer was set when we initialized the resource manager attribute structure */
ctp = dispatch_context_alloc(dpp);

/* start the resource manager message loop */

while(1) {
    if((ctp = dispatch_block(ctp)) == NULL) {
        fprintf(stderr, "block error\n");
        return EXIT_FAILURE;
    }

    dispatch_handler(ctp);
}

return EXIT_SUCCESS;

}

//Thread2

int createThread2 () {
/* Set the attributes for the thread. */
struct sched_param SchedParam;
pthread_attr_t Attr;
SchedParam.sched_priority = MDM_MSG_HANDLER_THREAD_PRIO;
pthread_attr_init(&Attr);
pthread_attr_setstacksize(&Attr, OSIX_DEFAULT_STACK_SIZE * 2);
pthread_attr_setinheritsched(&Attr, PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&Attr, SCHED_RR);
pthread_attr_setschedparam(&Attr, &SchedParam);
pthread_attr_setdetachstate(&Attr, PTHREAD_CREATE_DETACHED);
int pthread_setname_np(pthread_t thread, const char *name);

pthread_t             thread;
int                   rc=0;
char                  theName[16];

if (pthread_create(&thread, &Attr, &Thread2Handler, NULL) != EOK)
{
    printf("\r\nThread 2 creation failed");
}
else {

    rc = pthread_setname_np(thread, " Thread2 RM");
   
    if(0 == rc)
    {
      rc = pthread_getname_np(thread, theName, sizeof(theName));
      if(0 == rc)
      {
        printf("The thread name is %s.\n", theName);
      }
    }
}

}

void* Thread2Handler (void* ) {
/* declare variables we’ll be using */
resmgr_attr_t resmgr_attr;
dispatch_t *dpp;
dispatch_context_t *ctp;
int id;

/* initialize dispatch interface */
if((dpp = dispatch_create()) == NULL) {
    printf("\r\nUnable to allocate dispatch handle.\n");
    return 0;
}

/* initialize resource manager attributes */
memset(&resmgr_attr, 0, sizeof(resmgr_attr));
resmgr_attr.nparts_max = 1;
resmgr_attr.msg_max_size = 2048;

/* initialize functions for handling messages */
iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &mdmConnect_funcs,
                 _RESMGR_IO_NFUNCS, &mdmio_funcs);

mdmConnect_funcs.open = mdmRm_io_open;
mdmio_funcs.read = mdmRm_io_read;   //same io functions but diff name which I don’t like..
mdmio_funcs.write = mdmRm_io_write;
mdmio_funcs.devctl = mdmRm_io_devctl; 

 /* initialize attribute structure used by the device */
iofunc_attr_init(&mdmAttr, S_IFNAM | 0666, 0, 0);

/* attach our device name */
id = resmgr_attach(
        dpp,            /* dispatch handle        */
        &resmgr_attr,   /* resource manager attrs */
        names[1],       /* diff device name            */
        _FTYPE_ANY,     /* open type              */
        0,              /* flags                  */
        &mdmConnect_funcs, /* connect routines       */
        &mdmio_funcs,      /* I/O routines           */
        &mdmAttr);    /* handle      */

if(id == -1) {
    printf("\r\nUnable to attach name.\n");
    return NULL;
}

  /* allocate a context structure.The context structure contains a buffer where messages will be received. The size of
the buffer was set when we initialized the resource manager attribute structure */
ctp = dispatch_context_alloc(dpp);

/* start the resource manager message loop */

while(1) {
    if((ctp = dispatch_block(ctp)) == NULL) {
        fprintf(stderr, "block error\n");
        return 0;
    }

    dispatch_handler(ctp);
}
return 0;

}

The synchronization of io_read/io_write/io_devctl is controlled by a mutex designated to each device. There are calls that you can use to override this allowing simultaneous access to these functions. Usually this is used when an i/o call will block, and there is no need to keep a 2nd thread waiting. You still probably want to synchronize the hardware accessing software which you can be done with a mutex under your own control.

I think the preferred way to have more than one thread is to use the existing thread pool feature of the resource manager libraries.

Thanks maschoen.

But in my case, the main program is the first thread and this thread in turn creates another thread. And these threads will be blocked waiting for messages as long as the application is running. So I thought since I need to create only 1 more thread , I don’t have to go for thread pool. I read thread pool feature is more useful if the threads are created/destroyed dynamically which is not the case with me.

There are calls that you can use to override this allowing simultaneous access to these functions. Usually this is used when an i/o call will block, and there is no need to keep a 2nd thread waiting.”
Are you talking about achieving this concurrent access to these io_functions by using "iofunc_attr_unlock() " at the beginning of io_read()/io_write() ? Or is there any other functions you are referring to ?

Yes, I’ll be synchronizing accessing the hardware using separate mutex for each thread .

How long will each io_read, io_write operation take. Will any of these operation result in some sort of waiting for other resources, do polling etc.

The point Im trying to get at is that ultimatly you endup with some sort of serialisation anyway because everyting has to go though a single pipe; the PCI bus (I am assuming its a device on the PCI bus).

If a read or write operation takes less then 1ms (QNX default timeslice) it might be totaly useles to use thread .

You make it sound like going through a thread pool is some difficult and time consuming procedure that is to be avoided. This is software. You write it once and then it’s done. If you know exactly how many threads you will need, you can set that in the thread pool parameters. Yes, the thread pool is useful if the threads are created/destroyed dynamically. That doesn’t make it un-useful if you don’t have dynamic threads.

Yes.

This again sounds very strange. The mutex that iofunc_attr_unlock() will unlock for you would do the same. Unless you are doing something like a blocking read, or waiting for an event inside this code, you are doing things the hard way.