Failing at a network loadtest (for now)

Problem statement

I want to test packet forwarding performance diffrence between ERLite-3, ER-4, apu2 (w/ debian), and apu2 (w/ danos).

Summary of the fail

I started following an awesome guide for loadtesting using TRex my buddy Pim wrote up¹.

I installed debian on a spare apu4d4 (that’s supposed to be source/sink of the traffic)² and started configuring the TRex.

I created /etc/trex_cfg.yaml:

- version: 2
  interfaces: ["02:00.0", "04:00.0"]
  port_limit: 2
  cores: 4
  port_info:
    - dest_mac: [0x00,0x0d,0xb9,0x??,0x??,0xd5]
      src_mac: [0x00,0x0d,0xb9,0x??,0x??,0xd7]
    - src_mac: [0x00,0x0d,0xb9,0x??,0x??,0xd5]
      dest_mac: [0x00,0x0d,0xb9,0x??,0x??,0xd7]

and connected the two network ports (2 and 4) with a patch cable.

And I got almost immediate failure afterwards:

root@nettest:/home/trex/v2.89# ./t-rex-64 -i 
WARNING: tried to configure 2048 hugepages for socket 0, but result is: 1869
Starting Scapy server...................... Scapy server is started
The ports are bound/configured.
Starting  TRex v2.89 please wait  ... 
 set driver name net_e1000_igb 
 driver capability  : TCP_UDP_OFFLOAD  TSO 
 set dpdk queues mode to DROP_QUE_FILTER 
 Number of ports found: 2
zmq publisher at: tcp://*:4500
Ethdev port_id=0 nb_tx_queues=3 > 2
EAL: Error - exiting with code: 1
  Cause: Cannot configure device: err=-22, port=0
Killing Scapy server... Scapy server is killed

Turns out, the APU network card doesn’t have enough queues for TRex to work in multi-queue mode. A bit of fiddling around led me to the --software switch, with which it works:

root@nettest:/home/trex/v2.89# ./t-rex-64 -i --software
WARNING: tried to configure 2048 hugepages for socket 0, but result is: 1869
Starting Scapy server......... Scapy server is started
The ports are bound/configured.
Starting  TRex v2.89 please wait  ... 
 set driver name net_e1000_igb 
 driver capability  : TCP_UDP_OFFLOAD  TSO 
 set dpdk queues mode to ONE_QUE 
 Number of ports found: 2
zmq publisher at: tcp://*:4500
 wait 7 sec ....
[...]

So, yay!

But it turns out, APU2 isn’t capable of saturating the gbit link with empty UDP packets.

As suggested in Pim’s guide, I tried:

tui> start -f stl/udp_1pkt_simple_bdir.py -m 500kpps

The 500kpps works reasonably well, but taking it up to 700kpps³ (which is still well below linerate – not even 50%) pegs the rx CPU at ~100% and starts dropping:

Conclusion

So it seems that APU2, while otherwise a magnificent machine, is not well suited for a proper source/sink for the network test.

So I have scored a dual-port Intel NIC on the Amazon(.de) Marketplace (39€ delivered), and will retry this test with my desktop as a sink/source… once the card gets here.