The Phantom Network Drops

I recently made the jump to a 2.5 GbE home network, and I can honestly say it was absolutely worth the effort. The project initially kicked off as a troubleshooting mission to fix some bizarre network gremlins on my older garage desktop.

Out of nowhere, any file download over 1MB would stall and fail after just a few seconds. It didn’t matter which browser I threw at it—Edge, Chrome, Firefox, Opera, Brave, or Vivaldi—they all exhibited the exact same behavior. Clicking “resume” only bought me another second or two before the connection choked again. Browsing was equally frustrating; websites would only partially load. On YouTube, for example, the sidebar would render, but the top loading indicator would just freeze. To make matters more confusing, every other computer in the house was working perfectly.

The Operating System Swap

I had been running Windows 10 for years and figured a clean OS upgrade might resolve what I thought was a driver issue, but moving to Windows 11 didn’t change a thing. I tried falling back to WiFi, dropping in a separate 1GbE NIC, and testing out various USB-to-Ethernet adapters, but the phantom drops refused to go away.

Ultimately, I decided to cut my losses with Windows and switch the machine over to Ubuntu. I’ve been running Linux since around 2000 and feel much more at home in it anyway, especially since it’s my daily driver for nearly everything I do at work. Plus, thanks to my extensive library of Bash and Python automation scripts, getting the initial OS and environment dialed in was essentially a push-button deployment.

Total Hardware Failure

For a week or so after installing Ubuntu, the switch seemed to do the trick. I was able to browse and download without issue. Then, the exact same behavior crept back in.

Eventually, it reached a breaking point. I went to use the garage desktop one morning and the network was entirely gone. Running ifconfig showed nothing but the local loopback interface. It was as if someone had physically ripped the ethernet connection out of the motherboard. The NIC was dead.

Going Overkill: The LACP Solution

At this point, I decided that if I was having issues with a single network port, the obvious solution was to over-engineer it with four. I went online and tracked down a Realtek-based 4-port 2.5 GbE PCIe card (utilizing the RTL8125 controller) for $57.

I wasn’t just going to connect each NIC to an unmanaged switch and call it a day. My plan was to use LACP (Link Aggregation Control Protocol) to bond those four physical copper links into a single, highly resilient virtual ethernet port. At an engineering level, LACP (originally IEEE 802.3ad, later 802.1AX) is a dynamic protocol that negotiates and manages multiple physical links into one logical interface. Because it adds hardware-level redundancy, I could physically unplug one of the four cables and experience zero drops in connectivity.

The Switch Upgrade

By bonding four 2.5 GbE ports, I technically had a 10 GbE connection to my LAN. However, to actually take advantage of LACP, the upstream network switch must support the IEEE 802.3ad protocol. If you plug an LACP bond into a “dumb” unmanaged switch, the switch won’t understand the negotiation, and each physical connection will just be treated as a separate, confusing link.

To bring it all together, I ended up going with the TrendNet TEW-3102WS Managed 10-Port 2.5 GbE switch. Each switch was $169 on Amazon. I purchased two of these, one for my living room and the other for the garage. It is a smart switch with a proper web interface that allows for the configuration of VLANs, Trunk Ports, LLDP, and, most importantly, the LACP bond my new garage desktop required. What I really liked about this switch was the fact that it had 2 SFP+ 10GbE uplink ports.

For a 2.5GbE (2.5 Gigabit Ethernet) network, you need at least a Cat 5e cable, though Cat 6 is the ideal standard for optimal performance and reliability. I went ahead and purchased a 500 foot spool of CAT6E, along with some EZ RJ-45’s and a EZ RJ-45 Crimp Tool.

Configuring 802.3ad Network Bonding in Ubuntu Using Netplan

Setting up a Link Aggregation Group (LAG) using the 802.3ad (LACP) protocol across multiple interfaces is an excellent way to increase throughput and ensure high availability on your Ubuntu server. Because Netplan relies on YAML, success comes down to exact indentation and proper testing.

Here is how to deploy and verify a four-interface LACP bond.

Preparing the YAML Configuration

Begin by ensuring your configuration file is created with the correct permissions and exact spacing. Open the configuration file in a text editor with root privileges:

Bash

sudo nano /etc/netplan/01-bond0.yaml

Paste your configuration exactly as shown below. YAML is strictly dependent on correct indentation, so ensure you are using spaces rather than tabs:

YAML

network:
  version: 2
  ethernets:
    enp4s0:
      dhcp4: no
    enp5s0:
      dhcp4: no
    enp6s0:
      dhcp4: no
    enp7s0:
      dhcp4: no
  bonds:
    bond0:
      interfaces: [enp4s0, enp5s0, enp6s0, enp7s0]
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        transmit-hash-policy: layer3+4
      dhcp4: yes

After saving and closing the file, restrict its permissions. Netplan will often throw a security warning if network configuration files are left too open:

Bash

sudo chmod 600 /etc/netplan/01-bond0.yaml

Testing and Applying the Changes

Whenever you make network changes—especially over an SSH connection—it is highly recommended to test them first before committing. Use the try command to apply the configuration temporarily:

Bash

sudo netplan try

If there is a syntax error in your YAML file, Netplan will pinpoint the failing line so you can fix it. If your connection drops, simply wait 120 seconds, and Ubuntu will automatically revert to the previous working state. If the configuration is accepted and your connection is stable, press Enter to keep the changes.

If you prefer to skip the test phase or need to apply the configuration directly, you can commit the changes immediately:

Bash

sudo netplan apply

Verifying the Bond State

Once applied, you need to verify that the bond interface is up, has received an IP address, and that the LACP negotiation with your switch was successful.

First, check if bond0 successfully pulled an IP address via DHCP:

Bash

ip addr show bond0

8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether da:0c:d9:b7:2b:3c brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.51/22 brd 192.168.3.255 scope global noprefixroute bond0
       valid_lft forever preferred_lft forever
    inet6 2600:1700:dc0:5b80:d80c:d9ff:feb7:2b3c/64 scope global dynamic mngtmpaddr proto kernel_ra 
       valid_lft 3592sec preferred_lft 3592sec
    inet6 fe80::d80c:d9ff:feb7:2b3c/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
ifconfig
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet 192.168.0.51  netmask 255.255.252.0  broadcast 192.168.3.255
        inet6 2600:1700:dc0:5b80:d80c:d9ff:feb7:2b3c  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::d80c:d9ff:feb7:2b3c  prefixlen 64  scopeid 0x20<link>
        ether da:0c:d9:b7:2b:3c  txqueuelen 1000  (Ethernet)
        RX packets 11969568  bytes 13630767697 (13.6 GB)
        RX errors 0  dropped 81  overruns 0  frame 0
        TX packets 2140724  bytes 397045377 (397.0 MB)
        TX errors 0  dropped 25 overruns 0  carrier 0  collisions 0

Next, check the bonding module status. This reads the kernel parameters for the bond and details the status of all your physical interfaces:

Bash

cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v7.0.0-22-generic
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
802.3ad info
LACP active: on
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Slave Interface: enp4s0
MII Status: up
Speed: 2500 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 1c:86:0b:37:54:70
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
Slave Interface: enp6s0
MII Status: up
Speed: 2500 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 1c:86:0b:37:54:72
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
Slave Interface: enp5s0
MII Status: up
Speed: 2500 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 1c:86:0b:37:54:71
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
Slave Interface: enp7s0
MII Status: up
Speed: 2500 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 1c:86:0b:37:54:73
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 0
Partner Churned Count: 0

When reviewing the output, look for these specific indicators of a healthy bond:

• Bonding Mode: Should display IEEE 802.3ad Dynamic link aggregation.
• MII Status: Should report up for the master bond and all individual slave interfaces.
• Aggregator ID: All four interfaces must share the exact same aggregator ID. If they are grouped into different aggregators, there is a configuration mismatch between the server and the switch.

Items Purchased