Skip to content

Troubleshooting

Start with the symptom, then collect evidence before reflashing. The gateway has two processors and two distinct failure domains:

  • if SSH and Ethernet work but Zigbee/Thread does not, investigate the EFR32 radio path;
  • if the whole gateway disappears, investigate Linux, Ethernet, power, or the RTL8196E boot path;
  • if this is the first installation, begin with UART and bootloader networking.

Find the right section

Symptom Start here
No serial text, or unreadable characters No readable serial output
Linux boots before you see <RealTek> Cannot enter the bootloader
Installer cannot find 192.168.1.6 Bootloader not detected
TFTP times out or upload fails TFTP transfer fails
First boot loops after a full flash Boot loop after the first full flash
Gateway boots but SSH is unavailable SSH unavailable after boot
Zigbee2MQTT/ZHA cannot open port 8888 Radio client cannot connect
Zigbee/Thread stops, but SSH still works EFR32 radio unresponsive
The entire gateway vanishes from the LAN Gateway disappears from the network
RX drop counter is unexpectedly large Large RX drop count

First-install and bootloader problems

No readable serial output

The RTL8196E console must use 38400 baud, 8N1, no flow control. For the Lidl J1 header, verify all three connections:

J1 Adapter
Pin 2 GND GND
Pin 3 gateway TX RX
Pin 4 gateway RX TX

Leave pin 1 VCC disconnected and power the gateway normally.

Then check, in order:

  1. the host opened the correct /dev/ttyUSB* device;
  2. the adapter uses 3.3 V TTL logic, not RS-232 or 5 V logic;
  3. ground is common;
  4. TX and RX are crossed;
  5. the temporary contacts or solder joints are reliable;
  6. another program is not already holding the serial device.

Garbled but changing text almost always indicates the wrong baud. No text at all usually indicates wiring, device selection, or power.

Cannot enter the bootloader

Open the terminal before applying power. Press Esc repeatedly from the first serial character until <RealTek> appears. A single late key press is easy for the short boot window to miss.

If Linux starts:

  1. disconnect power;
  2. confirm the terminal still has 38400 8N1 and no flow control;
  3. reconnect power while repeatedly pressing Esc.

On a gateway already running this project's firmware, serial entry is usually unnecessary: use the upgrade guide, which invokes boothold over SSH.

Bootloader not detected

The stock bootloader normally listens at 192.168.1.6. The computer needs an address on the same direct Ethernet segment:

ip route get 192.168.1.6

If the output contains via <router>, add a temporary address to the interface that faces the gateway:

sudo ip addr add 192.168.1.10/24 dev <interface>

Check that:

  • the gateway is still at the <RealTek> prompt;
  • Ethernet link is up;
  • no other device uses 192.168.1.6;
  • the computer does not itself own 192.168.1.6;
  • a host firewall is not blocking TFTP/UDP.

The installer requires an actual TFTP response, not only ping or an ARP entry. With a non-default bootloader address, pass the same address to the script with --boot-ip and configure the stock bootloader with IPCONFIG when required.

Host prerequisites are missing

The full installer checks its required commands before modifying the gateway. For a normal pre-built-image installation on Ubuntu:

sudo apt install fakeroot gcc mtd-utils squashfs-tools tftp-hpa \
  netcat-openbsd iproute2 iputils-ping openssh-client

Ubuntu's tftp-hpa is required; another program named tftp may not support the command-line interface used by the scripts. The complete cross-toolchain is only required when rebuilding firmware. xxd is not required; the image and radio scripts use od from the base coreutils package. EFR32 flashing also needs python3, python3-venv, and patch.

TFTP transfer fails

Check:

  1. computer and bootloader are on the same L2 subnet;
  2. UDP port 69 is not blocked;
  3. no host TFTP server conflicts with the bootloader service;
  4. the Ethernet interface and temporary address are still up;
  5. the serial console remains at the bootloader prompt.

If the transfer never began, nothing has been written. If a flash write began, do not remove power merely because the operation is slow. The userdata region can take one or two minutes on the stock loader; wait for the serial success or failure message and the installer's result.

Boot loop after the first full flash

This can occur only on the first handoff from an older, pre-V2.9 bootloader. The old bootloader can leave switch DMA active while the newly flashed kernel starts.

Unplug the gateway for a few seconds, then power it on again. A warm reboot does not clear the same hardware state. The replacement V2.9 bootloader stops the DMA engine, so later full flashes boot normally.

SSH unavailable after boot

Wait at least 30 seconds, then check the serial console for the assigned address:

ip addr show dev eth0

Confirm whether the install selected DHCP or a static address. For DHCP, inspect the router's lease table. For static networking, the address is the one accepted at the installer's prompt; when the prompt was left at its default, that is an address in the subnet of the machine that ran the installer, with host part 88 (192.168.0.88 on a 192.168.0.0/24 LAN). The value is also recorded as GW_IP in .gateway-state at the repository root.

If DHCP was selected and no lease was ever obtained, the gateway falls back to the static configuration in /userdata/etc/eth0.bak — check that file for the address to try.

If the serial console shows Linux but no address:

cat /userdata/etc/eth0.conf 2>/dev/null
dmesg | tail -n 50

An eth0.conf file selects static mode; its absence selects DHCP. Correct the file from the serial console and reboot. If the gateway never reaches Linux, keep the serial boot log and use the bootloader restore path rather than repeating blind flashes.

Radio and client problems

Radio client cannot connect

Only one process can own TCP port 8888. Stop all other candidates — Zigbee2MQTT, ZHA, cpcd, a flasher, or a test socket — then retry.

On the gateway, inspect the persisted mode and bridge:

cat /userdata/etc/radio.conf
cat /sys/module/rtl8196e_uart_bridge/parameters/armed
cat /sys/module/rtl8196e_uart_bridge/parameters/stats

For an NCP Zigbee setup, radio.conf should contain FIRMWARE=ncp and should not contain MODE=otbr. Zigbee2MQTT uses:

serial:
  port: tcp://<gateway-ip>:8888
  adapter: ember

Home Assistant ZHA uses an EmberZNet radio with socket://<gateway-ip>:8888. Do not set a client-side UART baud for the TCP connection; Linux gets the physical baud from radio.conf.

If BRIDGE_BIND=127.0.0.1 is set, remote connections are intentionally refused. Use the configured SSH tunnel or restore a trusted-LAN bind.

The EFR32 radio is unresponsive

This condition means Linux and SSH still work, but Z2M, ZHA, or OTBR cannot communicate with the EFR32.

Try these recovery surfaces in order:

  1. Hold the front-panel button for five seconds. The status LED gives hold feedback; the service pulses EFR32 reset and restarts the radio daemon.
  2. Run ssh root@<gateway-ip> recover_efr32.
  3. Run ssh root@<gateway-ip> reboot.

If none works, stop the normal radio client, power-cycle once, and rerun the flasher with the intended firmware:

./flash_efr32.sh -y -g <gateway-ip> ncp

Use the correct BOARD for Sengled. The script probes the running application and Gecko bootloader and includes protocol-specific fallbacks.

Repeated HandleRcpTimeout() or Failed to communicate with RCP errors on an old v3.1.x/v3.2.x OT-RCP installation at 460800 baud were fixed in v3.3.0 by enabling UART hardware flow control in otbr-agent. Upgrade the Linux firmware before treating that historical failure as a damaged radio.

The recovery architecture and limits are documented in the EFR32 bootloader recovery post-mortem.

UART errors or intermittent radio timeouts

If radio failures correlate with traffic bursts, measure the RTL8196E UART1 counters instead of relying on an old cumulative oe: value:

scp -O 3-Main-SoC-Realtek-RTL8196E/32-Kernel/tools/uart-overrun-monitor \
  root@<gateway-ip>:/tmp/
ssh root@<gateway-ip> 'chmod +x /tmp/uart-overrun-monitor'
ssh root@<gateway-ip> \
  '/tmp/uart-overrun-monitor -i 2 -d 120 -o /tmp/uart-errors.csv'
scp -O root@<gateway-ip>:/tmp/uart-errors.csv .

Reproduce the workload during the two-minute capture. Positive d_oe values mean the UART hardware dropped bytes in that interval; positive d_fe or d_pe values indicate framing or parity errors. Verify that the EFR32 firmware, FIRMWARE_BAUD, and FIRMWARE_FLOW_CTRL in /userdata/etc/radio.conf agree. Do not simply increase the baud. The Sengled G4 has no RTS/CTS wiring and uses lower board defaults for this reason.

See the UART overrun monitor reference for every CSV column and interpretation guidance.

EFR32 flash fails

Before retrying:

  1. stop Zigbee2MQTT, ZHA, cpcd, or OTBR so the flasher has exclusive access;
  2. confirm SSH reaches the correct gateway;
  3. confirm the selected board matches cat /proc/device-tree/model;
  4. keep the gateway powered and rerun without --force;
  5. save the complete flasher output.

flash_efr32.sh temporarily changes the bridge to the Gecko bootloader's 115200/no-flow-control mode and restores runtime configuration afterwards. Do not manually pre-set bridge parameters unless following a specific recovery procedure.

Whole-gateway and network problems

The gateway disappears from the network

This is distinct from a radio failure: SSH, radio services, and every network response stop together, and a power cycle restores the box.

After recovery, collect persistent evidence before another power cycle:

dmesg
cat /userdata/netwatch/incidents.log 2>/dev/null
ls -l /userdata/panic 2>/dev/null
cat /proc/net/dev
ip route

The hardware watchdog detects a stopped CPU, but it cannot detect a live userspace whose network path is dead because its userspace feeder continues to kick the watchdog. The optional netwatch service covers that case by writing a persistent incident snapshot and rebooting after a long failure with carrier still up.

netwatch is shipped disabled because it can reboot the gateway. For a remote installation, read the netwatch reference before enabling it. DRY_RUN=1 records evidence without rebooting.

eth0 reports a large RX drop count

ifconfig, ip -s link, or /proc/net/dev can show a steadily growing RX drop value while errs and fifo remain zero. This does not necessarily mean the Ethernet driver lost useful traffic.

If ethtool is present, check the driver's own allocation/drop counters:

ethtool -S eth0 | grep -E 'rx_no_skb|rx_alloc_fail|rx_mbuf_no_shadow'

When those counters are zero, the higher RX drop total is commonly the kernel counting intact Ethernet frames for protocols the gateway does not consume. A frequent source is Realtek loop-detection traffic with ethertype 0x8899, broadcast by other equipment about once every two seconds.

Watch errs, fifo, the driver-specific counters, TCP retransmissions, and actual application symptoms. The detailed mechanism and measurements are in the Ethernet driver design.

Ask for help effectively

For setup questions, open a GitHub Discussion. For a reproducible defect, open an Issue.

Include:

  • exact board model;
  • project version and uname -r;
  • whether the gateway still answers SSH;
  • /userdata/etc/radio.conf for radio problems;
  • complete command output, not only the final line;
  • relevant serial boot log or dmesg;
  • what changed immediately before the failure.

Remove passwords, SSH private keys, Thread credentials, serial numbers, and user backups before posting logs.