MTU Update Day!

It's MTU update day in my lab. I'm mainly writing this to remember the steps I took and the order in which I took them that made things work right.

First few steps were pretty easy:
- On the 10Gb switch, set the port connected to my desktop to L2 MTU 9000
- Set desktop to MTU 9000
- Ping switch IP from Windows with MTU 9000:

ping -f -l 8972 <switchip>
Pinging switch with 8972 bytes of data:
Reply from switch: bytes=8972 time<1ms TTL=64
Reply from switch: bytes=8972 time<1ms TTL=64

So far so good. Now for a scary step, update TrueNAS Scale.
- On the 10Gb switch, set the port connected to my desktop to L2 MTU 9000
- On TrueNAS go to Network > Interfaces > <Interface> > Set MTU 9000
- It tests network changes just like a monitor setting to will revert in 60 seconds if you don't confirm.

VMWare time. I have two ESXi hosts under vSphere. First I found the underlying interface of one ESXi host and changed the mtu:

esxcli network ip interface list
# find interface
esxcli network ip interface set -i vmk0 -m 9000

At this point, I lost connectivity with vSphere and the ESXi gui. I was still connected vi SSH. I backed out.

esxcli network ip interface set -i vmk0 -m 1500

In my idiocy I hadn't set the vSwitch to MTU 9000, so I did that step next.

esxcli network vswitch standard list
# find network
esxcli network vswitch standard set -v vSwitch0 -m 9000

With no ill effect, I set that on the other ESXi host as well. Then I set the interface back to MTU 9000 and everything seemed to work. I set that on the other host as well.

Setting MTU 9000 on Linux hosts is no big deal.

# CentOS/EL:
vim /etc/sysconfig/network-scripts/ifcfg-eth0
# set MTU=9000

# Ubuntu:
vim /etc/netplan/50-cloud-init.yaml
# set mtu: 9000 in the interface
netplan apply

Docker was a funny one. I tried setting {"mtu":9000} in /etc/docker/daemon.json and that didn't seem to do anything. I set --mtu 9000 in the docker.service file. The docker0 bridge would not flip to MTU 9000 unless I manually set it that way. I don't believe there's much benefit in having docker communicating with a higher MTU since the underlying OS is handling the network traffic to the NAS. I'm leaving it for now.

This is a bit of a disjointed post, but it's just for my notes. It might help someone some day.