I'm doing some contract work for Citylink, working on the migration of WIX peers into a vlan dedicated to WIX. At the moment, there are about 130 individual devices responding to ARP in the WIX IP range (202.7.0.0/23). Of those, 50 are already in the new WIX vlan, about 80 remain in the "mud" vlan.

In order to kick the process along a bit, at 6am on Jun 13 (the flag day), I'll be moving the majority of WIX users remaining in the mud vlan into the WIX vlan.

Of the outstanding devices:

  • 20 do modest amounts of traffic, they'll be moved into the new VLAN over the next fortnight. 15 done as of 8/6/2011

  • 26 do more significant amounts of traffic, they'll be moved in one block on the flag day. The 26 includes most of the major traffic generators on WIX - Trademe, Harmony, DTS, Xtreme, Actrix, Vodafone, ICONZ, CatalystIT/Stuff, Datacom, Revera, Orcon, VUW.

  • According to arpwatch, ~30 appear to use more than one IP address in the mud vlan (ie, they're doing transit on the same interface), or are on a port with multiple MAC addresses. Unless these peers make changes they'll remain in the mud vlan, as moving them to the WIX vlan will break whatever other transit stuff they're doing. The full list of devices is below, most notable in it are 6 TelstraClear devices, 2 each for FX and NZWireless, and a few other smaller ISP's (ACS/Linuxnet, Knossos, InspireNET, Datalight, Parliament).

Most of the multi-homed devices will need new WIX ports provisioned, and they'll need to make config changes to split their WIX/non-WIX traffic, so while I'm not counting on any of them being ready to move by the flag day, I'm hoping some will. They'll continue to peer as before, they'll see a 1-2ms increase in latency to peers in the WIX vlan only.

The rationale for the flag day is pretty straightforward - between the new vlan and the old vlan there is a linux box acting as a filtering bridge, allowing WIX v4/v6 traffic and blocking everything else. That box has only a finite capacity, by moving the majority of the high bandwidth peers at (more or less) the same time, we avoid the possibility of DOSing the filtering PC. I know it can sustain ~500Mb/s, so while I don't believe there's going to be a capacity problem, I'd rather not run the risk.

There are also some devices for whom I've no idea what they're actually doing with the WIX addresses. They don't peer with the route servers, so I'm unclear if they make any actual use of WIX or if the numbers are just residual config left over from a previous age. Once the flag day has passed, and the majority of peers are "across the bridge", then I'll be able to do packet captures on the bridge to see if some of these mystery devices are actually doing anything on WIX.

For those peers remaining in the mud vlan, this box represents a new SPOF for their WIX peering - if it dies, WIX will fail for those peers. So there are modest incentives for both Citylink and the peers concerned to migrate in a timely fashion.

While the two vlan's are connected via the linux bridge, it is absolutely crucial that those splitting their services don't attach the same MAC address to both the WIX vlan and the mud vlan - things will go badly wrong for you if you do. The ultimate aim is to remove the linux bridge - once that is done, it'll be OK to attach the same MAC into both vlans.

Devices remaining in the mud vlan:

ISP IP MAC Comments
tc2 202.7.0.77 0:90:1a:40:21:3a
telstraclear 202.7.0.70 0:90:1a:40:3a:7d
tsc8lh 202.7.0.66 0:8:20:4c:1c:1a
tsb1lh 202.7.0.65 0:8:20:4c:1c:1a
telstraclear2 202.7.0.73 0:90:1a:9f:f9:7c
clear-ba1-atm1 202.7.0.192 0:d0:bb:a8:d8:c0
xtra 202.7.1.241 0:d0:97:5:64:0
knossos 202.7.0.76 0:c:76:7e:92:80
nzwireless 202.7.0.59 0:12:1e:7b:3e:14
nzwireless2 202.7.1.69 0:18:b9:a6:68:41
linuxnet/acs 202.7.0.245 0:12:44:ab:34:1b
inspire1 202.7.0.123 0:1b:c0:53:14:81
parliament1 202.7.0.86 0:1f:9e:fd:58:c0
parliament2 202.7.0.142 0:23:5e:fe:33:e0
fx3 202.7.1.175 0:21:a0:56:2c:19
fx4 202.7.1.176 0:23:4:aa:80:19
datalight 202.7.0.211 0:24:dc:12:e1:1
juniper 202.7.0.94 0:5:85:ca:d0:c0
asnet1 202.7.0.243 0:9:f:67:1d:f7
asnet2 202.7.1.239 0:60:d1:4f:ee:f8
businessonline 202.7.0.212 0:f:35:2c:61:56 Vector?
intergen1 202.7.0.121 0:12:44:ab:30:1b
tepapa 202.7.0.95 0:13:21:c9:37:2
3months 202.7.0.58 0:14:a9:73:cb:d0
natlib1 202.7.1.193 0:1b:54:e1:eb:10
inz1 202.7.0.187 0:23:33:ed:5d:8a
ssc 202.7.0.67 0:25:45:f3:50:e1