Hello all!
Is QNET over IP with loadbalancing supported in QNX 6.4.0?
I have 2 computers, each of them has 2 network ethernet adapters. IP protocol is esteblished, IP addresses are set manually, netmasks 255.255.255.0. Cross-cables are used:
hostname = host31 --------------------- hostname = host32
| en0 10.0.3.1 | << ========== >> | 10.0.3.2 en0 |
| en1 10.0.4.1 | << ========== >> | 10.0.4.2 en1 |
The aim is to set up QNET connection over IP with loadbalance. So if one cabel is disconnected (for example, 10.0.3.1(2) ), QNET has to use another connection (10.0.4.1(2) ).
I start QNET like this:
mount -Tio-pkt -o bind=ip,resolve=dns,qos_verbose=20 /lib/dll/lsm-qnet.so
io-pkt-v4-hc starts automaticly when system starts up, using devn-el900.so with shim.
ping 10.0.3.1(2), ping 10.0.4.1(2) work OK.
_CS_RESOLVE is set to lookup_file_bind
/etc/hosts (the same on both nodes):
Host Database
127.1 localhost.localdomain localhost
#::1 localhost.localdomain localhost
10.0.3.1 host31.lic.ru host31
10.0.3.2 host32.lic.ru host32
10.0.4.1 host31.lic.ru host31
10.0.4.2 host32.lic.ru host32
Question 1. Is it possible to set two IP addresses for the same host in /etc/hosts in QNX 6.4.0? Are there other ways to do it?
Then. Here is the /proc/qnetstats output :
…
**** Qnet compiled on Oct 20 2008 at 22:52:13 running on host31
**** Tx Connections:
**** Rx Connections:
**** L4 Status:
slot 0 mtu 1476 ack 1 crc 1
txd ok 0
txd bad 0
txd descr 0
txd still 0
tx timeouts 0
tx slow 0
rxd ok 0
rxd bad dr 0
rxd bad L4 0
rxd dropped 0
rxd duplic 0
rxd nacks 0
slot 1 is unused
**** Last 8192 bytes of circular qnet_error() log:
02035100: l4_init(): starting
02035100: l4_driver_init_en_iopkt(): starting
02035100: try_ifp(): ignoring interface en0 as per bind option
02035100: try_ifp(): ignoring interface en1 as per bind option
02035100: l4_driver_init_en_iopkt(): not ethernet bind, en io-pkt driver not running
02035100: l4_driver_init_ip_iopkt(): starting
02035100: l4_init(): ending
02035100: qnet_birth(): qnet_init() - complete
02035100: l4_resolve_node_up_ip_ionet(): nd 0 L4 0 has IP address 103000A in ndb
02035100: l4_resolve_node_up_ip_ionet(): saved our IP address of 103000A to L4 struct
02035100): nd_change_notify(): Node Up: nd 0 host31.lic.ru
Question 2. WHY «slot 1 is unused» and «saved our IP address of 103000A to L4 struct»? What about another interface/IP address? How to point to QNET to use all the interfaces avaliable? Is it possible (over IP)?
Then, if we execute on host31:
ls /net/host32
That’s all right. There are two node names in /net folders on both nodes and they are ready for work with them.
BUT !!!
Now and forward QNET uses ONLY ONE interface, with IP address 10.0.4.1(2), NOT that one witch has been saved in L4 struct!
If anoher cable ( 10.0.3.1(2) ) is connected or not - it has no diffirence. But when another cable ( 10.0.4.1(2) ) is disconnected, QNET fails and restores back to normal only when cabel is connected.
Here is the sloginfo output:
Jan 02 08:45:42 7 15 0 qnet(L4): l4_resolve_node_up_ip_ionet(): nd 1 L4 0 has IP address 203000A in ndb
Jan 02 08:45:42 7 15 0 qnet(QOS): nd_change_notify(): Node Up: nd 1 host32.lic.ru
Jan 02 08:45:42 7 15 0 qnet(kif): server_lookup(): invalid scoid 35, 0
Jan 02 08:45:42 7 15 0 qnet(QOS): tx_conn_create(): nd:1 conn id:1
Jan 02 08:45:43 7 15 0 qnet(QOS): tx_xmit_init_conn_pkt(): to nd 1 on L4 0
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(QOS): tx_xmit_init_conn_pkt(): to nd 1 on L4 0 retry 1
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(L4): qnet_ip_input(): RX: SRC: 204000A DST: 104000A
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(L4): qnet_ip_input(): RX: SRC: 204000A DST: 104000A
Jan 02 08:45:43 7 15 0 qnet(L4): qnet_ip_input(): RX: SRC: 204000A DST: 104000A
Jan 02 08:45:43 7 15 0 qnet(QOS): rx_qos_init_pkt(): nd 1 new rx_conn 1
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(L4): qnet_ip_input(): RX: SRC: 204000A DST: 104000A
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(L4): qnet_ip_input(): RX: SRC: 204000A DST: 104000A
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
Jan 02 08:45:43 7 15 0 qnet(L4): ip_iopkt_tx_pkt(): TX: SRC: 0 DST: 204000A
…
And so on, RX and TX for 10.0.4.1(2).
Question 3. Why the Server Connection ID error occurs (qnet(kif): server_lookup(): invalid scoid 35, 0) ? How to fix it?
Question 4. Should somethihg like /dev/io-net/qnet_ip and io-pkt directory appear, as it was in previous versions? I haven’t it.
Sorry for so long post. I hope for your help, it is a great problem now. If someone has expirience QNET over IP (particulary, with PPP) with loadbalance on several interfaces in 6.4.0, I would be happy:)))
Best regards,
Evgeniy.
P.S. Sorry for my English skill.