The idea behind this program is to simply access the ram and download the data from it to a txt file.
Later Ill convert the txt file to jpeg and hopefully it will be readable .
However when I try and read from the RAM using NEW[] it takes waaaaaay to long to actually copy all the values into the file?
Isnt it suppose to be really fast? I mean I save pictures everyday and it doesn’t even take a second?
Is there some other method I can use to dump memory to a file?
#include <stdio.h>
#include <stdlib.h>
#include <hw/pci.h>
#include <hw/inout.h>
#include <sys/mman.h>
main()
{
FILE *fp;
fp = fopen ("test.txt","w+d");
int NumberOfPciCards = 3;
struct pci_dev_info info[NumberOfPciCards];
void *PciDeviceHandler1,*PciDeviceHandler2,*PciDeviceHandler3;
uint32_t *Buffer;
int *BusNumb; //int Buffer;
uint32_t counter =0;
int i;
int r;
int y;
volatile uint32_t *NEW,*NEW2;
uintptr_t iobase;
volatile uint32_t *regbase;
NEW = (uint32_t *)malloc(sizeof(uint32_t));
NEW2 = (uint32_t *)malloc(sizeof(uint32_t));
Buffer = (uint32_t *)malloc(sizeof(uint32_t));
BusNumb = (int*)malloc(sizeof(int));
printf ("\n 1");
for (r=0;r<NumberOfPciCards;r++)
{
memset(&info[r], 0, sizeof(info[r]));
}
printf ("\n 2");
//Here the attach takes place.
for (r=0;r<NumberOfPciCards;r++)
{
(pci_attach(r) < 0) ? FuncPrint(1,r) : FuncPrint(0,r);
}
printf ("\n 3");
info[0].VendorId = 0x8086; //Wont be using this one
info[0].DeviceId = 0x3582; //Or this one
info[1].VendorId = 0x10B5; //WIll only be using this one PLX 9054 chip
info[1].DeviceId = 0x9054; //Also PLX 9054
info[2].VendorId = 0x8086; //Not used
info[2].DeviceId = 0x24cb; //Not used
printf ("\n 4");
//I attached the device and give it a handler and set some setting.
if ((PciDeviceHandler1 = pci_attach_device(0,PCI_SHARE|PCI_INIT_ALL, 0, &info[1])) == 0)
{
perror("pci_attach_device fail");
exit(EXIT_FAILURE);
}
for (i = 0; i < 6; i++)
//This just prints out some details of the card.
{
if (info[1].BaseAddressSize[i] > 0)
printf("Aperture %d: "
"Base 0x%llx Length %d bytes Type %s\n", i,
PCI_IS_MEM(info[1].CpuBaseAddress[i]) ? PCI_MEM_ADDR(info[1].CpuBaseAddress[i]) : PCI_IO_ADDR(info[1].CpuBaseAddress[i]),
info[1].BaseAddressSize[i],PCI_IS_MEM(info[1].CpuBaseAddress[i]) ? "MEM" : "IO");
}
printf("\nEnd of Device random info dump---\n");
printf("\nNEWs Address : %d\n",*(int*)NEW);
//Not sure if this is a legitimate way of memory allocation but I cant see to read the ram any other way.
NEW = mmap_device_memory(NULL, info[1].BaseAddressSize[3],PROT_READ|PROT_WRITE|PROT_NOCACHE, 0,info[1].CpuBaseAddress[3]);
//Here is where things are starting to get messy and REALLY long to just run through all the ram and dump it.
//Is there some other way I can dump the data in the ram into a file?
while (counter!=info[1].BaseAddressSize[3])
{
fprintf(fp, "%x",NEW[counter]);
counter++;
}
fclose(fp);
printf("0x%x",*Buffer);
}
The reason it’s taking so long is because you are copying the data to the file in a loop. So you are literally writing to the file probably hundreds or thousands of times. That’s going to be VERY slow since the file system has to make sure each fprintf is successfully written to disk. Think of this way as 100 people walking into a house. The way you are doing that is each person opens the front door, enters the house then closes the door behind them. Then the next person does the same. That opening and closing of the door takes time.
I suggest you write it all at once using fwrite(); That would be the equivalent of opening the front door once and having 100 people walk in and then closing the door.
Tim
P.S. When you are talking about saving pictures everyday if you mean on a cell phone or camera those may be using faster physical media (compact flash/SSD) vs a hard drive (assuming you are using a hard drive and not a compact flash/SSD card).
Thanks for the quick response.
Your little explanation there was awesome by the way !
Ok I have tried fwrite() which sounds in all cases like the better idea… but now im really not getting the data that I wanted…
Im getting trash data out of the ram…Ill post a little sample in CODE brackets.
Well first I have to disagree with Tim. If you did an fopen() and an fclose() between each fprintf()I would agree with the open/close door analogy.
When you are writing to a file using fprintf() you are using buffered writes. The data accumulates in a buffer in your programs address space before it even gets sent to the file system. When it hits the file system, unless you have changed the default settings, it goes into a memory cache where it can age a bit. So even there you can accumulate multiple dirty sectors that need to be flushed to disk. Finally they are all flushed to disk in block writes.
So why is it slow? I don’t know. How slow, and how much memory are you moving?
As to your weird output, I don’t see the new code. One possibility is that the info[1].BaseAddressSize[3] is not the physical address of your device, but some other place in memory. Another possibility is that your code is looking in the wrong place.
Reading over the PCI bus is a very slow process. I don’t recall the exact speed but it’s in the LOW megahertz I thing. Around 8Meg or something like that, FAR slower then normal RAM. PCI is efficient when operation are DMA driven. Plus PCI memory, unlike normal memory is always mapped as NONCACHE, which again has a drastic impact.
You might get better performance if you use memcpy, into a buffer in ram instead of reading each value one by one. Then do one signle fprintf of that buffer. If the memory you are trying to read is really big, just breakdown the memcpy into smaller chunks.
I’d say there is a door opening/closing with fprintf Although in reality it’s imperceptibly small and not the cause of his slowness since my example wrote 1 Meg in under 1 second in either case. I also agree with your detailed explanation of how things work but there whether it’s fprintf library or something else it’s definitely vastly slower to use fprintf than fwrite.
Tim
P.S. From the doc’s on mmap_device_memory: You need I/O privileges to use the result of the mmap_device_memory() function. The calling thread may call ThreadCtl() with the _NTO_TCTL_IO command to establish these privileges.
I don’t see a call to ThreadCtl() in your code. You need this even if you are running as root.
I guess it depends on how you look at it. In the example below you are measuring just the process time. There’s no accounting for the time the file system putting the data to disk. That can easily all happen after the program has exited.
I do agree that the cpu time to do all those printf’s is much more (apparently 20 times more) than one fwrite(). It’s just that that’s not what takes the most time.
Much faster than fprintf but not quite as fast as fwrite.
Most interesting to me is that repeated runs (say 10 in a row) give VERY smooth times for frwrite. The numbers vary from 0.04 to 0.06 real. But they vary wildly for fprintf (.25-.36 which I averaged to .31) and fputc (.10-.21) which I averaged to .14.
I would guess the fprintf is much slower because of the parse done to the format string “%c” and then to the conversion from bin to ascii. Fputc has no parsing and no conversion. So the difference in time should be striclty cause by extra overhead of calling fputc 1000000 times versus 1 fwrite. I think that the fwrite will be split down to 1000000/buffer size operations. I think buffer size is 4k or 8k not sur. Making the buffer bigger with setvbuf would help. Using open/write/close would be much faster yet.
We are digressing from the OP
Can’t explain the variation of time either. If I had to guess I would have said they should have similar behavior, as the buffering should ensure similar number of messages and context switches. The System profiler would probably help explain it
Although in reality it’s imperceptibly small and not the cause of his slowness since my example wrote 1 Meg in under 1 second in either case. I also agree with your detailed explanation of how things work but there whether it’s fprintf library or something else it’s definitely vastly slower to use fprintf than fwrite.