SFP+ Support on the Honeycomb LX2

A few years ago, I bought a SolidRun Honeycomb LX2. The LX2 is a 16-core ARM board (NXP LX2160A) with four SFP+ ports and a 1GbE RJ45. Paired with the extremely nice looking Lone Industries L5 mini itx case, it’s a very cool small form factor machine that requires very little power. When I first got it, I followed Liz Fong-Jones’ tips for getting Ubuntu 20.04 running and since then it’s been happily existing as a node in a heterogeneous kubernetes cluster in my homelab.

Recently, I wanted to repurpose it as a router and take advantage of the SFP+ ports to build a 10Gbit backbone. I also was using a USB ethernet adapter because I had problems getting the onboard 1GbE NIC to work back when I first set it up, so I wanted to see if I could fix that. I wanted to start with a fresh install of Ubuntu 24.04 LTS. The onboard NIC uses NXP’s ENETC driver, which was merged in kernel 5.14. I figured by running 6.8 which ships with Ubuntu 24.04, I should be good to go there.

The Problem

NXP’s LX2160A uses a subsystem called DPAA2 (Data Path Acceleration Architecture) for its high-speed network ports. DPAA2 uses a hardware component called the Management Complex (or MC) for managing the hardware resources. Unlike normal NICs that show up as eth0, the MC is used to create and wire up network objects before Linux can see them. The tool used to do this is called restool and it has a convenience wrapper called ls-addni:

sudo ls-addni dpmac.7   # should create an eth interface for SFP+ port 7

On a fresh Ubuntu 24.04 install with the prebuilt UEFI firmware from SolidRun’s image server, this fails:

MC error: Unsupported operation (status 0xb)
Error: dpmcp object was not created!

The MC firmware bundled in SolidRun’s prebuilt UEFI images was version 10.28.1, which does not support dynamic object creation. This has been a known issue and has been fixed, but I don’t think the prebuilt images have been updated yet. Time to build a new version of the firmware!

I performed all of the following steps using minicom over the micro-USB serial console port from my laptop. Follow Liz’s instructions linked above if you have trouble getting that working. I downloaded Ubuntu 24.04 and put it on a USB thumb drive, then selected the device from the BIOS when booting the Honeycomb LX2.

Step 0: Install Ubuntu 24.04 with IOMMU Workaround

Ubuntu 24.04 (kernel 6.8) won’t install cleanly on the LX2. The installer crashes with ARM SMMU context (arm-smmu arm-smmu.0.auto: Unhandled context fault). At the GRUB menu during install, press e to edit the boot entry and add to the kernel command line:

iommu.passthrough=1 arm-smmu.disable_bypass=0

After install, make it permanent:

sudo sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT=".*"/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0,115200 iommu.passthrough=1 arm-smmu.disable_bypass=0"/' /etc/default/grub
sudo update-grub

Step 1: Verify the 1GbE Works

The onboard RJ45 uses the ENETC driver, which works out of the box on kernel 6.8. Hooray, first success!

sudo ip link set eth0 up
sudo ethtool eth0   # Look for "Link detected: yes"

Now I could get rid of the annoying USB ethernet adapter.

Step 2: Build and install restool

You need restool to interact with the DPAA2 subsystem. It’s not packaged for Ubuntu, so clone the repo and download it from source. Building the man pages required pandoc as a dependency, and I didn’t want to screw around with that, so I just disabled it:

git clone https://github.com/nxp-qoriq/restool.git
cd restool
make MANPAGE= install

Now time to verify the MC firmware version and confirm the SFP+ MAC objects exist:

sudo restool --mc-version          # shows 10.28.1 on prebuilt firmware
ls /sys/bus/fsl-mc/devices/        # should show dpmac.7, dpmac.8, dpmac.9, dpmac.10

The MAC objects are there. The MC firmware just refuses to do anything with them:

sudo ls-addni dpmac.7   # should create an eth interface for SFP+ port 7
MC error: Unsupported operation (status 0xb)
Error: dpmcp object was not created!

Step 3: Fix the UEFI Build Environment

SolidRun’s UEFI build repo uses Docker. Unfortunately, it seems to be broken as of 2025. Here’s what I did to fix it:

Problem 1 - Debian Buster is Dead

The Dockerfile uses debian:buster-slim. Buster reached end-of-life and its repos return 404. Change to Bookworm:

cd lx2160a_uefi/docker
sed -i 's/debian:buster-slim/debian:bookworm-slim/' Dockerfile
sed -i 's|/usr/bin/python3.7|/usr/bin/python3|' Dockerfile
cd ..
docker build -t lx2160a_uefi docker/

Problem 2 - The Entrypoint Fails Silently

Docker entrypoint (entry.sh) uses #!/bin/sh -e and runs git rev-parse early. Because Docker volume mounts trigger git’s safe directory check, this command fails with exit code 128, and set -e kills the script before it produces any output. You get no error message, no log, nothing, just an instant return.

Bypass the entrypoint entirely:

docker run ... --entrypoint /bin/bash lx2160a_uefi -c '
 git config --global --add safe.directory /work
 cd /work
 INITIALIZE=1 ./runme.sh
 ./runme.sh
'

Problem 3 - GCC 12 Breaks edk2 BaseTools

BookWorm ships GCC 12, which enables new warnings under -Wall that edk2’s -Werror promotes to build failures. Four suppresions were needed:

The fix uses edk2’s existing EXTRA_OPTFLAGS hook in header.makefile. In runme.sh, change the BaseTools make invocation to:

EXTRA_OPTFLAGS="-Wno-vla-parameter -Wno-use-after-free -Wno-dangling-pointer -Wno-stringop-overflow" \
 PYTHON_COMMAND=/usr/bin/python3 make -C BaseTools

This survives INITIALIZE=1 because it lives in runme.sh (parent repo), not in the edk2 submodule which gets reset.

Step 4 - Get a Modern MC Firmware

Here’s the actual fix. Everything above is just getting the build to compile.

The Management Complex is a separate microcontroller embedded in the LX2160A SoC that runs its own firmware, independent of Linux. It manages all DPAA2 network objects: the MACs, buffer pools, channels, and network interfaces that make the SFP+ ports work. When you run ls-addni dpmac.7, the restool utility sends commands to the MC over an ioctl, asking it to dynamically allocate these objects and wire them together into a usable network interface.

MC firmware 10.28.1 (bundled in SolidRun’s UEFI images) has all these objects pre-allocated at boot in a static layout, but it does not support creating new ones at runtime. When ls-addni asks it to create a dpmcp (management command portal), the MC responds with “Unsupported operation.” The objects you need are technically there - you can see them in /sys/bus/fsl-mc/devices/ - but without the ability to create new dpni (network interface) objects and connect them to the existing dpmac objects, they’re useless from Linux.

Newer MC firmware versions support dynamic object creation. NXP publishes these binaries on GitHub:

git clone --depth 1 https://github.com/NXP/qoriq-mc-binary.git
ls qoriq-mc-binary/lx216xa/
# mc_lx2160a_10.40.0.itb

The MC firmware is embedded in the UEFI image at a specific offset on the SD card - it’s loaded by the SoC before Linux boots. There’s no way to update it independently; you have to rebuild the entire UEFI firmware image with the new binary included.

I added an MC_FW environment variable hook to runme.sh that copies a user-supplied .itb into the edk2-non-osi tree and patches the FDF filename before building. This is opt-in - without MC_FW, you get the old bundled version.

Step 5: Build and Flash

Putting it all together:

git clone https://github.com/SolidRun/lx2160a_uefi.git
cd lx2160a_uefi
git submodule update --init --recursive

# Apply the Dockerfile and runme.sh fixes described above

docker build -t lx2160a_uefi docker/

docker run \
  -e DDR_SPEED=2600 \
  -e MC_FW=/work/mc_fw.itb \
  -v "$PWD":/work:Z \
  -v /path/to/mc_lx2160a_10.40.0.itb:/work/mc_fw.itb:ro \
  --rm -t --entrypoint /bin/bash lx2160a_uefi -c '
    git config --global --add safe.directory /work
    cd /work
    INITIALIZE=1 ./runme.sh
    ./runme.sh
  '

Output lands in images/lx2160acex7_2000_700_2600_8_5_2_sd_<hash>.img.

DDR speed matters. My RAM is DDR4-2666, so I use DDR_SPEED=2600. My first build attempt used 3200, which produced a firmware that failed at boot with “DDR Clk is faster than DIMM can support.” The prebuilt images report 3200 in dmidecode because the firmware attempted that speed, not because the RAM supports it.

Flash to microSD on another machine (early HoneyComb revisions can’t write to the microSD from Linux):

sudo dd if=images/lx2160acex7_2000_700_2600_8_5_2_sd_*.img of=/dev/sdX bs=4M status=progress

Step 6: Activate SFP+ Ports

Boot with the new firmware and verify:

$ restool --mc-version
MC firmware version: 10.40.0

$ sudo ls-addni dpmac.7
created eth1

Hooray! We’ve got functioning SFP+ Ports now.

Wrapup

The HoneyComb LX2 is a genuinely interesting piece of hardware, but the toolchain can be a little tough to work with sometimes.

If you’re running a HoneyComb LX2 with UEFI and your SFP+ ports don’t work, the answer is almost certainly that your MC firmware is too old. Build your own UEFI image with a newer one from NXP’s qoriq-mc-binary repo.

What’s Next

I still need to actually configure this as a router - nftables NAT, DHCP, the whole deal. That’s for a future post once my switch and SFP modules arrive. But at least now, hard part is done!