Low memory issue, many fds are open

QNX USERS QNX6 - OS
1

Hi,

I am facing low memory issue where only 5kb is free out of 250MB RAM. The memory used by the processes look same as it was used when system is stable. Only difference I could find is that that around 5000 Fds are open by procnto (80 fds are opens when system is stable). Any hint to find the root cause of this issue shall be greatly appreciated. Thanks in advance for you help.

Thanks,
Ikshu

2

It sounds like an application program bug. A program continually doing an open but not closing it could cause this. What is running in your system.

3

Hi Maschoen,

Thanks so much for your response. It is a confidential code, so I cannot share the code here. It is a very huge code and I have analysed if the fds are being correctly closed in the application. I am suspecting if it is a qnx issue (I mean is the qnx not closing the fd correctly). Is it possible that 5000 fds can consume around 40MB memory?

Regards,
Ikshu

4

I don’t think you’ve mentioned which version of QNX you are running. It is unlikely to be a QNX bug. The problem is not that 5000 fds can consume 40MB of memory. The problem is that the number of fds is a limited resource. The error you would see if you exceed it is probably the same as if you run out of memory. If you are running some version of QNX 6, then the command

$ pidin fd

will show you which process has fd’s open. If one process has a lot, it will be obvious.

If you are using QNX 4 the command is

$ sin fd

Looking back at your original post, I would guess that if you only have 5K left of 40MB or 250MB it is probably not the fds that are the problem. The technical term for you what you have is probably a memory leak.

5

Hi Maschoen,

Thanks for your time for this issue.
I am using Qnx 6.5.0. I used pdin fd to see which process is having more fds open. It was procnto process. I initially suspected it to be memory leak and compared the memory of the processes in stable and issue state. The memory consumed by all the processes are same when the issue occurs. I compared pidin mem output during stable and issue state. Any malloc will fail after available RAM memory reduces. Please let me know if there are any other items related to memory that I can compare in stable and issue state to find what is causing the memory leak.

Thanks,
Ikshu

6

Hi,

Have you tried to use the “QNX System Information” perspective in Momentics ?
With this perspective, you can monitor CPU, RAM, fds…

Regards,
Nicolas

7

Your system is acting very strange.

In general procnto has only one reason I can think of for creating an fd, to open an executable and load it.

I suspect you are doing something unusual in your application somewhere. Whether the problem is a QNX bug or user error is impossible to know given the information you’ve provided.

8

In procnto : Is the fd closed as soon as the executable is loaded/running ? Or is the fd closed when the executable exits ?

9

That’s a good question, one that I don’t know the answer too. Having the source might answer that. Some kernel and/or proc source was once released before QNX was bought by Blackberry. Then everything was reversed.

I do know that sometimes when I want to update a module, if it is currently running I get an error. That suggests that there is an open fd. However it doesn’t always work this way, and I don’t know the details of when.

I checked before my last reply on a slightly older version I have running, QNX 6.32, and procnto only listed one open fd, though there were many programs running. All very confusing.

10

I’m currently using QNX6.6.
In “QNX System Information” perspective, “System Resources” tab, procnto-smp is shown to be using 95 file descriptors.
The system is running 81 processes (procnto-smp included).

11

Upon closer look,I’m getting the same sort of thing. I was running this:

pidin fds | grep proc

Which only showed 1 fd. This of course was not the way to view this.

So the obvious question is, since pidin fds shows what the fd has open, what is open on the system with 5000 fds?

Clearly the theory that a running process means proc has the executable open is correct.

procnto document shows the following parameter

-F number
The maximum number of file descriptors that can be open at the same time. The minimum allowable value is 100. The default value is 1000, but might be constrained by the RLIMIT_NOFILE system resource.

So if a single process is opening more than 1000 fd’s it should fail unless the default is increased. Maybe this is the problem?

12

Hi Nicolas,

I do not have qnx IDE support. So I cannot use this.

Thanks,
Ikshu

13

Hi Maschoen,

procnto can create fd for process management, memory management and path management as per this link qnx.com/developers/docs/6.5. … /proc.html . Please let me know if my understanding is not correct.

Thanks,
Ikshu

14

Hi,

This is a good point, I am looking at setting this limit and trying to debug to find the root cause.

Thanks,
Ikshu