IPC message which can be accessed randomly?

I’m looking for a way of multiple processes (on different hosts!) updating something similar to a std::deque; I need to be able to insert/pop and get/update ‘at’* any place in the container. I’ve had a few ideas of how to do this but I don’t want to roll my own, so could anyone suggest something I could use? like a message queue implementation that I’ve not heard of?

I’m wondering about wrapping a deque in its own process and calling it MyOwnSpecialMessageQueue?
or a static array in shared mem but I can see the push/pop being expensive and I’d have to protect against another process modifying it at the same time.
or… chaining multiple mq’s together, thus letting me get the random access I want.

Any ideas much appreciated, thanks :slight_smile:

(*note once setup, the at’s will always be at the same index into a fixed size queue)

My guess is you will have to write this stuff pretty much from scratch.

That being said you don’t give any detail about the requirement or the exact task you are trying to acheive. My guess is you are looking at too complexe of a solution for something that could be done simpler. That’s just a gut feeling ;-)

I’m modelling a conveyor belt (I’m not), sand goes on to it one end and falls off the other, at fixed points along the belt extra sand is tipped on or falls off.

->xxxxxxxxxxxxxxxxxxxxxxxxxx-> ^ v ^ v e.g. sand being put on and falling off

the things modifying my container in the middle will be on separate hosts, my ‘main’ host can cope with the insert/pop bit. Can you see why a deque is so appealing? but it has to be available over qnet (or similar) for separate hosts so I like mqueue as well :slight_smile:

What kind of refresh rate are you looking at; how often will other host modify the data?
What is the nature of the data/operation done by the remote computers?

nothing crazy, 10 updates per second, each update is inserting/overwriting a double

Ok then I would create a process that would handle all this, then provide an API for that process. How the data is handle in the process is somewhat irrelavent ( but indeed a deque look good). Make that process as “dump” as possible, just like a conveyor belt.

I would give the API real world mecanical term, like drop point, sample station, etc. I can see getting real creative with this ;-)