Open Source Difficulties

Sean,

I like that.   You might also want to be a little clearer about how big the tree is, and how long it takes to download.   I'm downloading just HEAD right now, and its been going 3 hours without an end in sight.  I'm not sure if it is possible, but with 8000+ packages, maybe there could be a version that had just a few hundred of the more commonly needed ones.  

Mitchell

pkgsrc is not that big :slight_smile: A full pkgsrc tarball, (as you can see on netbsd.org) is only 17Mb.

Everything in pkgsrc is just a pointer (a url to tell the build envrionment where to download the real source archive). Then, when you “bmake” a project, it will know where to download the archive, chksum the downloaded archive to make sure it is the right one. extract the archive, and patch it so it can be built on your QNX6 machine …

It also have dependency check, so before download wget-1.10.2nb1.tar.gz, it knows it also need openssl; if openssl package is not build yet, (not in your /usr/pkg :smiley:), then it will go security/openssl to bmake that package first. (and if it depends on other package…)

So you see, it is not that easy to just tar up “some of the package” as it could easily broken the dependency.

It’s big enough when you have to check it all out :slight_smile:

But xtang’s right. We don’t want to get into maintaining our own dependency tree. Plus I wouldn’t want to discourage someone from porting KDE, for example.

-seanb

While the tar ball may only be 20Meg, when run through gunzip, it expands to 200Meg.
It gets worse. When you de-archive it, it breaks up into thousands of files. I’m guessing
at least 100,000. So it took a short time to download, and a long time to de-archive, a surprising
few hours on my development machine. When I first started using it, I would foolishly search for
a package as follows:

find /usr/pkgsrc | grep apache

This took a few minutes. I eventually ended up creating a directory index. Inside index are files name after the program category, and list long form ls listing of that directory. So now I can do the following:

cd index; grep apache *

I’m probably going to move everything off to a removable volume.
I’m used to searching for things on my system (yes I’m old and I forget a lot,
but I’m clever so who can tell?) by the following:

find / | grep lost-it

This useful technique is no longer possible because of pkgsrc.

I downloaded it via svn ( already dearchived) took 20 minutes I think.

Makes me wonder what is different about our systems. What type of hard drives are you using. Have you modified anything in the file system such as the cache write delay? I’ve noticed that Linux systems seem to have much better throughput on disk I/O, but that seems related to not caring as much when the metadata and data get to disk.

CPU is most definitely not a limiting factor for me.

Mitchell

When I am feeling lucky, I usually do:

cd */apache

or

cd /aparche

And it usually land me into where I want. :slight_smile:

The “NetBSD Sweden” have a web fron pkgsrc, (pkgsrc.se/) which is also very useful.

Will QNX release a new XPhoton?

Maybe the nature of your internet connection?

I ALWAYS change the filesystem setting on development machine:
diskboot … -odevb-*,“blk noatime,commit=none,cache=64m”

  • Mario

How do you change the settings on the development machine?
I assume you are building your own .boot file?
I’ve not been able to do that up until now, because I’ve only
been able to use the provided SMP .boot. I guess
that is changed now.

My DSL is 2Mb download, so unless it’s because you are closer to Ottawa,
I don’t think there would be much difference.

Well before I say anything else, I’d first like to say that pkgsrc is a life saver. It is not perfect, and probably will never be. A common compile problem is when a typedef, eg. ulong, is missing. These are pretty easy to fix. Sometimes an include is not in the right place. Many packages just install without a hitch.

If you get involved with something like this, use the fastest cpu’s with maximum cores. Also, don’t start until you’ve modified the diskboot command in your .boot file to delay writes. This will save you many hours of waiting for configure and bmake to run.

I finally have the latest Apache2/PHP5/Mysql running together, along with a host of PHP5 features and extensions. This required solving three major problems. 1st, pkgsrc is currently useless for creating the PHP5 module for Apache due to a linker configuration problem. While this is troubling, it is the only package that I know of with this problem. So I was able to build all the packages that Apache2 and PHP5 depended on, along with Apache2, but I had to work with the standard PHP5 download myself. This is not terribly difficult. You get the latest source. You setup a configure command, which is complicated only because there are so many options. You have to point some of the options at /usr/pkg so that it will find the shared libraries needed. You run configure/make/make install, and you are done.

The second problem has to do with libtool. PHP5 comes with a libtool script that gets run explicitly, and which causes a linker problem. libphp5.la will always be built static, which doesn’t work as an Apache load module. The solution is as follows. Use pkgsrc to get the latest libtool and install it. Once installed, copy /usr/pkg/bin/libtool into the directory where you are building PHP5.

The third problem is the one that was most difficult to solve, although at its core it is quite simple. Part of my problem was that pkgsrc lulled me into thinking that all dependencies would be created automatically. But that just isn’t so for PHP5. When you configure PHP5, you might for example want to include PDFlib. If you haven’t retrieved it with pkgsrc, the PHP5 make will proceed along until the link step where it will complain about not being able to resolve all the needed symbols. But it doesn’t give you a hint as to which ones. I came up with a straight forward strategy to figure this out, but it is too unwieldy. After completing the make, you delete libphp5.la and run make redirected to a script file. You can then edit this file, removing the libtool prefix, and then change all the foobar.lo’s to lib/foobar.o’s. When you run this script it should display the missing symbols which will tell you what is missing. This was too unwieldy for me because this script file was one line that was about 4000 characters. In case QSSL is interested, the ped display started barfing, although the replace seemed to work ok.

So instead, I found that trial and error worked best. I went through my configure script, and made sure I had each shared library built by pkgsrc. This can be a bit tricky. For example, PDFlib needed both the pdflib package and the php-pdflib package built. Since the former is a dependence of the latter, you can just build the latter, but it isn’t always obvious. BTW, pay attention to the messages you get at the end of building any php-* package about modifying the php.ini and httpd.conf files, as I didn’t see this documented anywhere else.

Another gotcha is that for some strange reason, Mysql sets itself up in /usr/pkg/mysql/lib rather than /usr/pkg/lib/mysql. I created some soft links that solved this. There are lots of little problems to overcome, but with a little time and effort it will work. I never figured out how to get the older mysql_() interface routines to work, which was fine since I only use the newer mysqli_() interface.

I haven’t tried moving the whole shebang to my server yet, but I think it is just a matter of setting up paths correctly and copying /usr/pkg. If anyone else runs into trouble, I’m happy to provide my /usr/pkg directory, or any working pkgsrc packages needed. This task was so arduous that I gave up on putting in the pkgsrc fixes along the way, which was inconsiderate, but necessary.

One last note, I was tempted to put in postgresql support, but found that the latest pkgsrc version needs some idiosyncratic configuration for QNX. Hopefully someone will do this some day. I was able to install the 6.2.1 version with the installer and that worked fine although I think that it is a little out of date.