I am working on a QNX 6.5 project which requires fairly frequent capture of the screen using PgReadScreen() (Every 200ms or so) while the user interacts with the photon GUI. For about 80% of the frames it works fine, but we are having issues in which the bitmap returned by the PgReadScreen function is just a grey square. This typically happens when a new terminal/application is opened. Or when any sort of text is output to the screen by another program (Like running the “ls” command in a terminal).
Below is a program which is the most simplified version of the problem. It just saves the resulting bitmap to the QNX filesystem every 200ms 25 times.
We have tried:
-Locking the photon library with PtEnter(0) and PtExit(0)
-Creating and releasing a direct-mode context to the graphics driver
-Explicitly creating and destroying the shared memory object used by PgReadScreen
Any Ideas?
Thanks
#include <stdlib.h>
#include <stdio.h>
#include <stdbool.h>
#include <signal.h>
#include <time.h>
#include <errno.h>
#include <Pt.h>
#include <photon/PxImage.h>
#include <photon/PhT.h>
#include <photon/Pg.h>
#include <photon/PkKeyDef.h>
#include <sys/neutrino.h>
#include <rfb/rfb.h>
#include <rfb/keysym.h>
#define PX_IMAGE_MODULES
#define PX_PHIMAGE_SUPPORT
#define PX_BMP_SUPPORT
#include <photon/PxImage.h>
#include <unistd.h>
#include <stdio.h>
#include <limits.h>
#define FRAME_RATE_MSEC 200
void exit_handler(int temp);
double msec();
void exit_handler(int temp)
{
printf("Exiting...\n");
PtExit(0);
}
// msec resolution time based on system clock time
double msec()
{
uint64_t nsec;
if (ClockTime_r( CLOCK_MONOTONIC, NULL, &nsec) == EOK)
return( nsec * 1e-6);
else
return(0);
}
int main(int argc,char *argv[])
{
PhImage_t* ph_image;
double start;
PhRect_t current_window_rect;
int count=0;
char filename[20];
//initialize rect
current_window_rect.ul.x=0;
current_window_rect.ul.y=0;
current_window_rect.lr.x=1279;
current_window_rect.lr.y=1023;
signal(SIGINT, exit_handler);
/* Initialize Photon image data */
// get a connection to Photon
if (PtInit("/dev/photon") != 0)
{
printf("Error: Could not connect to Photon.");
return -1;
}
start = msec();
while (1)
{
// Frames per second
if ((msec() - start) > FRAME_RATE_MSEC)
{
if (ph_image = PgReadScreen(¤t_window_rect, NULL))
{
//Save bitmap file
sprintf(filename, "/ph_image%d.bmp", count);
if(PxWriteImage(filename,ph_image,NULL,PX_IMAGE_BMP,0)!=0){
printf("IMG SAVE FAILED\n");
} else {
printf("IMG SAVE WORKED\n");
}
PgShmemDestroy(ph_image);
//stop after 25 captures
if(count>25) return 0;
count++;
}
else {
printf("get_frame FAILED!\n");
}
start = msec();
}
}
}
A quick and dirty test would be to set the priority of your test program REALLY high. As in higher than anything else including the Photon drivers. That way when your program wakes up it will get exclusive access during it’s running time.
With luck that will get you a screen capture without any of the grey issues when other programs are writing to Photon.
I like Tim’s idea but I think you need to re-evaluate what you are doing.
You want to read a video screen 5 times a second and you are having a synchronization problem. This is not surprising. The screen is rendered through at least three layers, the Photon space, message passing and the video driver. Apparently the data you retrieve from the driver when reading the screen is not always what you see on the screen, So why would this happen? Have you ever taken a photograph of a scanning (tube) video screen with a high shutter speed? You can get part or all of the screen black. Your eye doesn’t see this because the screen pixels are illuminated 30 or more times a second. Something similar is happening with Photon. There are intermediate moments when a view of the screen is not what you want to see. As I recall the widget library will often use a painter’s algorithm, repainting in the order of the z-axis. I once had a problem with the screen flashing a lot. I had to go in to the widgets and make their backgrounds transparent so the user wouldn’t see the flash as background was painted and the over-painted.
You should see if bumping the priority helps. You might try a kludge fix where you identify bad images and discard them. If you really need 5 clean frames a second, I think you need to rethink what you are doing.
You could separately render what goes to the screen using a double buffer in your application and provide an interface to grab the last rendered screen. I don’t know enough about what you are doing to know if that would work or is practical.
I see you also asked about the screen grab utility. I suspect that if you found a way to have it grab 5 frames a second, it would give you the same results.
Unfortunately, bumping the priority did not help. Also just for good measure I increased the increment all the way up to 10 seconds. Still no change in behavior.
Is there any way to counter the screen refresh behavior you are describing by making sure I am only attempting to capture the screen when it is displaying what I want to see?
To give you some background, we are trying to build custom remote desktop tool. And will need at least a crude video capture. Would there be a way to make the background of all on-screen qnx applications transparent like you describe?
I’m not surprised that bumping the priority didn’t help. At best I thought it would make the grey images less common.
Aside from what I mentioned earlier, doing your own rendering and creating an interface to grab a fully rendered image, I have one more idea.
There is a widget called a double buffer widget. I forget the actual name. It’s a container widget that renders locally and then sends an entire image to the driver. It might be the case that putting your widget inside this widget will solve the problem. It will almost undoubtedly come with additional overhead, however it might not be noticable.
Do you have any pointers on how exactly I would achieve this? I’m very greenhorn to qnx as a whole.
It looks like I can create an offscreen memory context and set up a double buffer there, but I’m not sure how I would go about interfacing with/capturing it after the fact.
Offscreen memory contexts in Photon. No, not for the faint of heart. I remember having to read through the docs to get it to work. I think once you have an offscreen memory context, it should not be that hard to transfer it to the screen. I’d have to look up how to do this.
If you are not familiar with QNX however, the trickier part may be figuring out to have a separate process get ahold of the image using message passing. This is quite straight forward if you are familiar with the resource manager structure, but if not, you would need to learn how.
I’m curious though why you are doing new development with QNX 6.5. There are lots of good reasons to do this, but as you probably know, QNX the company has thrown in the towel on Photon a number of years ago.
Now on the flip side, if you absolutely postively must have this working, I know consultant who does this kind of work.
BTW, my suggestion about using a double buffer widget was to keep it on screen. It is possible that this will allow the proper synchronization as just one message pass will (hopefully) deliver the completed screen image to the driver. I would try it and see.