QNX 7.1 and Qnet

According to the 7.1 release notes 7.1 removes the ability to use Qnet with QNX6.5 systems. With 7.0 this works very well - why kill it?

Maybe the answer is further down in the release notes. I had to read it a couple of times - maybe it was a dream?

Qnet is to be deprecated in future releases. Again, why kill this? It is one of the features that set QNX apart from other operating systems - a secret weapon if you like.

There are still some very good features of this operating system. For example, the ease of driver development - coupled tightly to the resource manager architecture (meaning no need to delve into the kernel as one has to do (apparently) with Linux)) - and the easy message passing between threads. And of course, if you need it, it’s an RTOS.

But is this enough to warrant continued use of it? For me I’ll have to think about that. I have been culturing post-grad students (a couple at PhD level who will be future managers) in QNX who I would have expected to carry it forward into the future (as I am close to retiring). I’m not sure if there’s much point in this endeavour now as who knows what else will be ripped out of this operating system? Future planning is impossible if this is to occur.

It is not without precedent. Photon was lost after 6.5.0. What will be next and when?

Geoff.

Hi Geoff,

I just took a look at the 7.1 release notes (because we use QNX 7).

QNet in general is not being deprecated. It’s only deprecated in terms of working with QNX 6.5X systems. The reason for this appears to be in the line above:
-support for 32-bit x86 targets, including _intr_v86()

I take this to mean 7.1 is 64 bit only. Since 6.5 is 32 bit they are no longer compatible because you can’t execute 64 bit kernel calls on 32 bit kernels.

I didn’t read further on whether it means you can no longer compile 32 bit code any more. This matters for us because all our code is 32 bit (ported from 6.5) and we run QNX 7 in 32 bit mode. If that’s true it’s a much much bigger deal for those of us who may not have portable code (ie sizeof() calls change).

Tim

Hi Tim,

Buried deep down in the 7.1 release notes, under “Deprecation Notices” it states a list of items that “will be removed in future releases”. One of them is “Transparent Distributed Pricessing (Qnet)”.

7.0 running on a 64-bit ARM A53 works very well with 6.5.0 running on a a small x86 SBC. Of course one needs to be mindful of element sizes in messages (for example long is now 64 bits as is time_t - so struct timespec is now 16 bytes instead of 8 ). But I am using it as a sort of file server as I can’t write to the internal SD card of the Xilinx ZCU104 (reason unknown but I don’t think it’s QNX as the ZCU102 has no such problem). But by setting up a series of soft links I have the Xilinx board thinking that it’s using an internal drive - but in fact it is transparently using a remote drive on a foreign architecture! A wonderful feature that saved my a$$! :slight_smile:

But I can live without that easily enough although all my small 32-bit x86 SBC’s become somewhat redundant.

Some other things break also but I can live with that also.

I am worried about losing QNET completely some time down the track. And that is what the deprecation notice states. Like when down the track? 7.11? an 8.0? I have asked and am waiting for an answer. As there’s no point in exploiting such a feature now only to have it ripped away later (should I last that long)! Like what happened with Photon.

Having said all that, on reflection overnight I have more or less decided that I will most likely perservere with QNX as there are still heaps of features that stand it apart from Linux and make life easier for me, for what I want to do. But losing Qnet would be a major blow after having exploited it since 1987. And it was around long before then of course!

Arguably QNX ain’t QNX without Qnet.Well, not complete anyway! :slight_smile:

Geoff.

Hi Geoff,

I can totally understand why QNX would want to remove ‘transparent distributed processing’. The speed of modern CPU’s combined with multi-core has made distributed processing mostly irrelevant. How often do you need to split computations across multiple processors these days? Last time I did stuff like that was in the 90’s on single core systems with 486/Pentiums.

As far as your file server goes, you can accomplish the same thing with Samba (we use it) so you don’t need Qnet for that. Are you actually doing distributed processing over Qnet or could you accomplish everything using basic message passing over TCP/UDP (what we do).

Tim

Hi Tim,

I gather from a conversation I had yesterday that the dropping of Qnet could be some time (years?) away so this discussion is really academic at this stage. But I’d like nonetheless to put forward my view on the subject.

While parallel processing may have been an original justification nearly 40 years ago for developing the distributed processing feature (known thesedays as Qnet) I don’t think it’s been a viable justification since the days of QNX 2/3 and ARCNET some 25 years ago. So on that score I agree entirely with you.

For me Qnet allows me to transparently implement the powerful features of IPC (Inter Process Communications) either by use of the Resource Manager model (that I use extensively) or simple the message passing paradigm (SRR - that I also use extensively) seamlessly across a network. Why do I want to do this?

Well, I sometimes need to split processes/threads out of one single CPU environment (be it single or multi-core) to CPU’s that are for whatever reason physically detached. In one case I have a CPU that acts as a master system controller (MC) that also incorporates an instance of a different software suite (an “application” if you like) that the MC manages. Often I require multiple CPU systems, connected by a network (be it wired ethernet or wireless) that also run these “applications” that the MC provides overall management and control.

Qnet allows me to simply move the applications to different CPU’s without any need to change anything other than the path names. I don’t have to implement TCP/UDP mechanisms - from the point of view of the MC “/dev/application” simply becomes “/net/cpuname/dev/application” and that’s it!

Using Qnet I can continue to employ pulses for event notifications without changing anything in the code. In fact, everything to do with QNX IPC that can be done locally can be done remotely without requiring any fundamental changes to the system architecture. This to me (at least) is a Very Powerful Feature that sets QNX apart from anything else.

For 25 or so years I have been maintaining a QNX serial driver for a well known PCI multi-port serial I/O product (the RocketPort). This product also had (of course) Windows and Linux drivers. But it was becoming harder and harder to run serial lines/cables around buildings/factories. Often the system controllers (hosts if you like) would be in one location but the machinery requiring the serial interfaces would be somewhere else. While there was resistance to laying serial cables generally there would be networking cables (Cat5/Cat6) going everywhere. As a result a new serial product was developed, often referred to as a “serial device server”. It’s basically an ethernet to serial port converter. We looked at developing a QNX host to connect to these serial device servers but it quickly became apparent that the effort was simply not required. As one guy in the broadcast industry said to me (20+ years ago and I quote) “if I want access to remote serial ports I’ll simply install a QNX box with a RocketPort and manage it remotely across the network”. He was simply referring to the capabilities of Qnet - his was a QNX shop. We simply didn’t need to develop a special driver as QNX provided us with a solution simply “out of the box”. So I never did.

I can think of many such cases. The ability to seamlessly communicate between multiple remote CPU environments (such as wind farms, even cars/motorbikes, etc) without having to resort to 3rd party solutions or a diferent protocol suite should not be under-estimated!

As for the little “file server”, using Qnet I didn’t require a Samba server to be running. I’m not even sure if there is a fs-cifs Samba client available in 7.0 (I’ll go looking later this morning). While I have a Samba server running on 6.5.0 I don’t remember seeing one that runs on 7.0. I don’t know about 7.1 as I don’t yet have 7.1. Maybe there is - if not what’s involved in building it? To me none of this mattered - with Qnet I didn’t need to even consider that approach. I was able to quickly solve my problem using the tools provided to me by the operating system in a couple of hours and with no need to implement 3rd party solutions to a problem QNX solved decades ago.

I fervently hope that Qnet is not removed in the future. But even if it is dropped in the future, there are so many other powerful features of QNX that I can’t at this time see myself moving away from it. But the flagged possibility does force me to alter my approach to the research project I’m involved with at one of the local universities, where I’m trying to teach students (at least when they come back post COVID!) the dark arts of QNX as opposed to Linux! :slight_smile:

Cheers,

Geoff.

There doesn’t appear to be an ARM Samba client (fs-cifs) distributed with QNX 7.0. Not that I could find anyway. So the Samba option was not available to me. Had Qnet not been available I would have been in deep strife. Basically, for this project, dead in the water.

Nor does there seem to be a Samba server (smbd).

I didn’t bother looking in the x86 stuff.

Geoff.

I see you are still passionate Geoff.
cheers,
Ian

Hi Geoff,

You are right that there is no Samba client or server for QNX 7. I had to get the source and compile it. I think it took me a few hours of time so it wasn’t difficult at all. I miss the days when QNX had the pre-compiled packages for QNX 6.*. Now you have to do them yourself.

It’s interesting what you wrote about the Rocketport (serial to Ethernet). We extensively make use of remote Serial to Ethernet in our application. But we went an entirely different direction by simply putting in 3rd party box (Digi 8 port serial to Ethernet converter) at the remote location. This eliminates entirely the need for any drivers since it’s just basic UDP / TCP from QNX to the Digi device. The upside for us is that we can (and have) used any serial to Ethernet converter so we are not tied down to Rocketport or maintaining drivers.

Tim

Hi Ian,

Call me some time! Contact details haven’t changed (but I’ve lost yours). It’s been a long time…

Geoff.

Personally I find the removal of Qnet shocking. 30+ years ago I had in my hands one of the most elegant OS architectures one could imagine. From a a console I could direct a program to run on any node in the system, taking input from files on any node in the system, the same for console input and output, and without the program written in any unusual manner. On top of that, the development was self hosting, a huge advantage in development. All that was missing back then was a good GUI. It took two attempts but then QNX had that too.

Since then, GUI gone. Self hosted development gone. Now Qnet to be gone. All that is left is, microkernel archicture and real time. The former is mostly important for writing drivers. The latter is less and less important as processors increase in speed and as cores multiply on chip sets. On the dark down side, the development environment is horrid compared to the competitors. The developer seems willing to throw any customer it doesn’t deem profitable enough under the bus. It’s pretty sad. If they don’t win the auto automation marketing war, and I don’t see any reason to think they will, then QNX is a goner.

Maschoen,

Not sure why you find Qnet’s demise shocking. It’s hardly needed anymore by anyone. The things you talk about (running programs on other nodes) can more or less be done by just about any O/S now (Windows remote desktop or VNC Server) including GUI/keyboard etc. You can mount shared network drives across multiple O/S’s and share files/data and so on. Maybe not quite as seamlessly as Qnet but you are 90% of the way there in just about any O/S and you can code the last 10% if you require it.

Multi-core CPU’s (and threading) have rendered obsolete many of the things that once made QNX stand out. Priority inversion for example was very important in the 90’s and early 2000’s when you had 1 core CPU and no threading. Now, multi-core means the lower thread (or process) doesn’t really need the boost because it will simple execute and finish on another core and release the resource. Maybe not quite as fast as if it was boosted immediately (depending on how loaded your system is) but unless you need real hard time it’s quite sufficient. Executing processes on another node across QNet was similarly important to get parallel processing in 1 core days but now the network delay is millions of times slower than using a 2nd or 4th or even 16th core on modern CPU’s.

GUI development continues to undergo big changes. We exclusively develop ours under Windows and yet customers for the last 5 years are no longer happy with that because of tablets/phones etc and now we are switching to a browser based model and serving up webpages instead of a customized C# app because the browser based approach can run anywhere/any hardware. That’s the issue QNX faced in that they were going to forever be chasing what the customer demands. At least you can now port over some webservers to QNX fairly easily and pump out webpages for GUI’s.

I suspect what helps QNX in the car market is that it still boots extremely fast (important), can be embedded without much need to write to a filesystem and is still unknown enough that there aren’t a lot of virus’s or other malware written for it so it’s immune to hacking. Plus QNX has gone to great lengths to get SIL certified (easier with a microkernel that virtually never crashes) which is going to continue to be important in automation in cars/medical etc.

Tim

P.S. I hate the loss of self hosted. I now code under Visual Studio (IMHO, vastly better than Eclipse for managing code) and compile for QNX from the command line using Makefiles. I debug with printfs/log messages rather than use Eclipse. It’s sad because Makefile/GDB debugging could still have remained under self hosting because the GNU compiler is all that would need to be cross compiled for QNX.

Hi Tim,

I need a recent Samba server for QNX6.6.0. Can you explain me how you compiled one for your usage ?

Regards,
Nicolas

Nicolas,

The more recent versions (depending on how recent you need) have technically removed QNX support because I believe no one was available/willing to do the compilation and tests to ensure it works (or maybe it’s a cross compile issue). So just be aware of that when you go looking for Samba source (you should be able to find the last supported version easily with a google search).

For me it was very straightforward because I was doing self-hosted compiles for QNX 6.5.

  1. Get the source code for the version you want
  2. Run ‘Make Configure’ (I believe that’s the command that sets up the compile environment for QNX) in the root Samba directory
  3. Do a make and get the binaries (for the Samba client and server).

I am not sure what you’d do if you weren’t running self hosted because I think the configure gets confused because it would think it’s a Windows/Linux system instead of a QNX one. Maybe someone who has cross compiled might know the answer?

How I got around this on 7 where you must cross compile is that I took the QNX 6 configuration and just copied verbatim to 7 and since the make environment was already setup for QNX so the cross compile worked.

If this is confusing or you have any more questions, let me know.

Tim

Tim, As far as it “in general not being deprecated” hold onto your hat. I discovered a bug with 7.0’s name_locate() in GLOBAL mode, meaning across the network. Then I talked with Geoff who had discovered this a year before. QNX’s response was, pay us $20K and we’ll fix it. There is an obvious work around, make the target a resource manager, but with an attitude like that, how long do you think they will keep QNET running?

Hi Tim,

Thanks for your answer.

I tried to cross-compile the last SAMBA version without success. I didn’t know QNX support has been removed.

With recent SAMBA versions, building is much more complicated since WAF is used as the build system. So you have to deal with Python+cross-compiler.
I spent many hours trying to understand how all this building stuff works. I made some progress but as I am far from an expert in building systems I’m stuck on a problem I don’t understand.

I need SMBV3 support so I have to use SAMBA V4.1.0 or later.
WAF is used since SAMBA V4.0 so I have to deal with it.
With QNX support removed, I wonder if cross-compiling is still possible…

Regards,
Nicolas

Hi Nicolas,

The version I built is older than V4 so it’s all pre-WAF. But this bug report shows that someone was building it for QNX in V4.0 at least.
https://bugzilla.samba.org/show_bug.cgi?id=6842

I found this link.
https://wiki.samba.org/index.php/Waf

It looks like you can still do the ./configure, make route even though the build still uses WAF.

I think the ‘cross-answers’ build option is the one to use for QNX so you can specify the QNX compiler directly and then at each step of the build when it fails with UNKNOWN you’ll replace with OK (as per the instructions) in order to get the compilation to finish.

I’d definitely suggest doing this under Linux as opposed to Windows. The reason for that is that the make configure step for Linux is going to be much closer to QNX than Windows would be in terms of include files/names/ etc. You may have to modify a few differences between QNX and Linux for names/paths etc.

You might also email them from their contact email on the homepage and ask what the last supported version of QNX was (I can’t recall now which of the release notes/comments that said official QNX support stopped).

Apologies if that’s what you tried.

Tim

Tim,

Unfortunately, yes, I tried all that.

I use a virtual machine (on my Windows box) running Ubuntu for my tests.
I tried ‘cross-answers’ but it is very hard to find the correct answers (is the answer, ‘yes’, ‘1’, ‘on’ ?).
So I use the ‘cross-execute’ method.

For this I made a Python script which loads the generated binary on the target, executes it on the target and gets output back in the host. This script is given as a parameter to the build command. This allowed me to go much further in the compilation process.
But the whole thing errors at some point.
I asked the SAMBA mailing list. One guy provided me with an old patch for cross-compilation. This allowed me to go further in the compilation process. But it still fails.
It looks like cross-compilation is not really maintained, but I can be wrong.

I also tried to go another route and do the compilation on the target by using pkgsrc scripts and a Python3.2 interpreter I compiled for my target.
First, Python3.2 is said to not be compatible with the WAF version used. But sometimes we have good surprises.
Second, I made pkgsrc scripts to redirect the compilation process on my Windows box where I also installed a NFS server. But it looks like the NFS server I used has synchronisation problems (what is written on the Windows side not reflected correctly on the remote side).
At this point, I got bored and had other things to do so I gave up (I should have tried with my linux VM).

To conclude, I think cross-compilation on the linux VM is the best option even if I failed. I have the feeling that my problem is not related to QNX but to cross-compilation :

image

If I remember well, “com_err.h” is in the sources tree. That makes me think cross-compilation is broken.
I asked the SAMBA mailing list but didn’t get any answer this time.
Maybe I should ask again…

Reagards,
Nicolas

Hi Nicolas,

When you say you did “I use a virtual machine (on my Windows box) running Ubuntu for my tests.”, was this just to test if cross compilation in general was working? Because this setup is only going to work/make sense if you are cross compiling for Ubuntu.

When I was compiling for QNX 7, I totally ignored all the test stuff. I just wanted the binaries built and didn’t care if the tests themselves passed. It’s been a while now (a couple years) since it I did but I think I had to manually adjust a few include file paths for QNX. I also think I went into each folder manually and did a make there as opposed to the top level make that runs all the test stuff.

Exactly how far along are you getting in the build process. It may be that what you want is already compiled (ie the server or the client) and just the test stuff is failing.

I think cross-answers is the only way you will get it to work for QNX because you need to get the CC line set to use the QNX compiler and include paths etc.

Tim

Hi Tim,

When you say you did “I use a virtual machine (on my Windows box) running Ubuntu for my tests.”, was this just to test if cross compilation in general was working? Because this setup is only going to work/make sense if you are cross compiling for Ubuntu.

I use the Ubuntu VM for cross-compiling for QNX target. I have installed QNX-SDP on it.
I’ve already successfully compiled other software with this setup.

When I was compiling for QNX 7, I totally ignored all the test stuff. I just wanted the binaries built and didn’t care if the tests themselves passed. It’s been a while now (a couple years) since it I did but I think I had to manually adjust a few include file paths for QNX. I also think I went into each folder manually and did a make there as opposed to the top level make that runs all the test stuff.

I use many switches to removed parts of the build process. Maybe I have to remove more.

Exactly how far along are you getting in the build process. It may be that what you want is already compiled (ie the server or the client) and just the test stuff is failing.

The configure part pass. The build part stops at some point. I’ll check if binaries are built but I don’t think so, the build process stops quite early.

I think cross-answers is the only way you will get it to work for QNX because you need to get the CC line set to use the QNX compiler and include paths etc.

I gave up with cross-answers since it is really not obvious which answer to write in the file.
Cross-execute works well. At least, it seems to work well since it goes till the end without error.

Thanks for taking time to answer my questions.

Nicolas

If you zip up what you have (source only please) on your Ubuntu VM, I can see if I can make it compile on my self hosted QNX system. You’d just have to let me know what binaries you are interested in (server and/or client).

That may get the process a bit further.

Tim