Archive for the ‘Networks’ Category

Apple Airport devices and guest networks in bridging mode

Friday, January 9th, 2015

The feature of being able to configure a separate guest Wi-Fi network has been present for a while now on Apple Airport gear. However, the documentation only ever alludes to this feature working when the Airport device is running in router/NAT mode, i.e. it’s in charge of connecting directly to the Internet and sharing the connection from your ISP.

Quite often people (like myself) use the Airport device(s) in bridging mode, i.e. as regular access points to attach Wi-Fi clients onto the Local Area Network and then some other device handles the routing and NAT function, (in my case an OpenBSD host), and it would be nice if you could also create a guest Wi-Fi network in this mode. The obvious way I would expect it to work is to utilise VLANs; the Airport device uses a single unique VLAN ID to tag Ethernet frames from clients on the guest Wi-Fi network. As the administrator you can then use that tag to segregate the traffic on your network; usually to allow guests some form of heavily-restricted Internet-only access, and not be able to access the rest of the network. The Airport Utility lets you create the guest network in bridging mode but doesn’t give you any details as to the mechanics of it.

Well it turns out my hunch was correct, it does use VLANs; Ethernet frames from clients on the normal Wireless network stay untagged so they Just Work on your network as before, however Ethernet frames from clients on your guest network are tagged with the VLAN ID 1003. This ID is not mentioned anywhere, nor can it be changed so you’d better hope you’ve not already used that ID for something else.

Armed with that information, I configured my Cisco SG300 switch like so:

switch#configure terminal
switch(config)#vlan database
switch(config-vlan)#vlan 1003
switch(config)#interface vlan 1003
switch(config-if)#name wifi
switch(config)#interface GigabitEthernet 1
switch(config-if)#switchport mode trunk
switch(config-if)#switchport trunk allowed vlan add 1003
switch(config)#interface GigabitEthernet 2
switch(config-if)#switchport mode trunk
switch(config-if)#switchport trunk allowed vlan add 1003
switch#show vlan
Created by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, V-Voice VLAN
Vlan       Name           Tagged Ports      UnTagged Ports      Created by    
---- ----------------- ------------------ ------------------ ---------------- 
 1           1                               gi1-8,Po1-8            D         
1003       wifi             gi1,gi2                                 S

This creates the VLAN, names it, configures two trunk ports where my Airport and OpenBSD router are attached and adds the new VLAN to the list of allowed ones on each port. Finally all that is left to do is to create a VLAN interface on the OpenBSD router:

# cat /etc/hostname.vlan1003
inet vlan 1003 vlandev em0

Providing any DHCP and/or DNS services and firewalling the traffic is outside the scope of this post but you now have a separate interface and subnet that you can treat it like any other regular network.

So now I have two separate wireless networks; one that gives me access to my LAN which I can secure using WPA2 Enterprise and another that can only reach the Internet which can be unrestricted or more likely secured with WPA2 Personal.

IPsec between Meraki and OpenBSD

Tuesday, August 6th, 2013

I recently acquired some (Cisco) Meraki networking kit including an MX60 security appliance (read: router, firewall, NAT, etc.).

Once it’s set up and running, I was browsing the dashboard and the site-to-site VPN configuration options. Normally with multiple Meraki devices in use, a fully-meshed VPN can be created automatically with very little configuration.

I also noticed the ability to add non-Meraki VPN peers so I added details for my OpenBSD-based gateway. You can see from the screenshot the details are basic:

meraki dashboard

On the OpenBSD side, I started with a basic configuration in /etc/ipsec.conf:

ike esp from to \
        peer \
        psk mekmitasdigoat

The Meraki dashboard allows you to register a dynamic DNS record for the MX60 which will be in the domain so you can use this to refer to the remote peer in the configuration, (especially if you have a dynamic IP address).

This configuration didn’t work, but using isakmpd(8)‘s handy -L option to write to /var/run/isakmpd.pcap all of the unencrypted IKE traffic it then becomes apparent the OpenBSD and the Meraki disagree on the phase 1 negotiation; mainly that the Meraki end wants to use Triple DES and OpenBSD prefers AES by default. As there’s no way to tweak the cryptography options on the Meraki side, we change the OpenBED side like so:

ike esp from to \
        peer \
        main auth hmac-sha1 enc 3des group modp1024 lifetime 28800 \
        quick auth hmac-sha1 enc aes-256 group none lifetime 28800 \
        psk mekmitasdigoat

The main option sets the phase 1 parameters and the quick option sets the phase 2 parameters that match the highest settings out of the handful proposed by the Meraki side.

With that done, all that remains is to ensure isakmpd(8) starts at boot and the rules in /etc/ipsec.conf are automatically loaded by adding the following to /etc/rc.conf.local:


Elasticsearch clusters plugged into Cisco switches

Friday, May 31st, 2013

I was building up an Elasticsearch cluster as a storage backend for Logstash. Using the default Zen discovery method, I found that some of the nodes in the cluster could not automatically find each other.

Zen uses multicast to locate other nodes and by using tcpdump I could see that some nodes weren’t receiving the multicast traffic. This lead me straight to the network layer and I found the culprit; a Cisco Nexus 5000 switch.

Normally a switch often treats multicast traffic much like broadcast traffic, it floods the packet to every port on the same VLAN. However Cisco switches try to be clever and learn which ports are interested in receiving the traffic by listening for IGMP packets, (referred to as snooping), but IGMP is only sent if there’s a multicast router on the network, (note also I’m not trying to route multicast, all nodes are on the same VLAN).

The solution was to enable a feature on the Nexus known as an “IGMP querier”. What this does is mimic enough of a multicast router that nodes report which multicast groups they’re interested in receiving traffic for and the switch can then learn which ports to forward multicast traffic on.

On the Nexus 5000, I needed to add the following configuration:

vlan configuration 1234
  ip igmp snooping querier

(If you have a Catalyst switch running IOS, the configuration should be very similar)

The VLAN should match whatever VLAN the nodes are attached to, and you can basically make up the IP address used here, the switch sends IGMP packets with it as the source, but it’s never used as the destination for packets, nodes use a specific multicast group instead.

As soon as I added this configuration my Elasticsearch cluster sprung into life.

OpenBSD PPPoE and RFC 4638

Friday, January 20th, 2012

I upgraded my Internet connection from ADSL 2+ to FTTC a while ago. I’m with Eclipse as an ISP, but it’s basically the same product as BT Infinity, right down to the Openreach-branded modem, (a Huawei Echolife HG612 to be exact).

With this modem, you need to use a router or some software that can do RFC 2516 PPPoE so I simply kept using OpenBSD on my trusty Soekris net4501 and set up a pppoe(4) interface, job done. However what became apparent is the 133 MHz AMD Elan CPU couldn’t fully utilise the 40 Mb/s bandwidth I now had, at best I could get 16-20 Mb/s with a favourable wind. An upgrade was needed.

Given I’d had around 8 years of flawless service from the net4501, another Soekris board was the way to go. Enter the net6501 with comparatively loads more CPU grunt, RAM and interestingly Gigabit NIC chips; not necessarily for the faster speed, but because they can naturally handle a larger MTU.

The reason for this was that I had read that the Huawei modem and BT FTTC network fully supported RFC 4638, which means you can have an MTU of 1,500 bytes on your PPPoE connection which matches what you’ll have on your internal network. Traditionally a PPPoE connection only allowed 1,492 bytes on account of the overhead of 8 bytes of PPPoE headers in every Ethernet frame payload. Because of this it was almost mandatory to perform MSS clamping on traffic to prevent problems. So having an MTU of 1,500 bytes should avoid the need for any clamping trickery, but means your Ethernet interface needs to cope with an MTU of 1,508 bytes, hence the Gigabit NIC (which can accommodate an MTU of 9,000 bytes with no problems).

One small problem remained, pppoe(4) on OpenBSD 5.0 didn’t support RFC 4638. While I sat down and started to add support I noticed someone had added this to the NetBSD driver already, (which is where the OpenBSD driver originated from), so based on their changes I created a similar patch and with some necessary improvements based on feedback from OpenBSD developers it has now been committed to CVS in time for the 5.1 release.

To make use of the larger MTU is fairly obvious, simply set the MTU explicitly on both the Ethernet and PPPoE interfaces to 8 bytes higher than their default. As an example, my /etc/hostname.em0 now contains:

mtu 1508 up

And similarly my /etc/hostname.pppoe0 contains:

inet NONE mtu 1500 \
        pppoedev em0 authproto chap \
        authname 'user' authkey 'secret' up
!/sbin/route add default -ifp \$if

I also added support to tcpdump(8) to display the additional PPPoE tag used to negotiate the larger MTU, so when you bring the interface up, watch for PPP-Max-Payload tags going back and forth during the discovery phase.

With that done the remaining step is to remove any scrub max-mss rules in pf.conf(5) as with any luck they should no longer be required.

Using SNMP to upgrade IOS on Cisco devices

Friday, April 29th, 2011

Following on from my previous post regarding using SNMP to initiate TFTP backups, it’s also possible to use SNMP to remotely upgrade IOS on a Cisco device.

I’ll reuse and extend the setup and configuration from the previous article so apply that first. The IOS device then needs the following additions:

router(config)#snmp-server view myview ciscoFlashOps included
router(config)#snmp-server view myview lts.9 included
router(config)#snmp-server system-shutdown

The first command grants SNMP write access to allow us to manipulate the flash storage in the device. The remaining commands allow us to trigger a reload/reboot of the device remotely via SNMP.

On our “management station” we’ll need a few more MIB files installing under ~/mibs or wherever you put them:


Update the $MIBS environment variables to include them, you’ll need something like the following:


Now, my devices don’t have enough flash storage to hold two IOS images so I normally have to delete the current version (which is loaded into memory, so it doesn’t affect the operation of the device) to free up space and then upload the new version, reload, and pray it works.

To find the filename of the current IOS image we can use the SNMP equivalent of dir flash:/:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser -Cb -Ci -Cw 80 ciscoFlashFileTable
SNMP table: CISCO-FLASH-MIB::ciscoFlashFileTable
 index           Size     Checksum Status                                Name
 1.1.1      576 bytes        "0x0"  valid                                   /
 1.1.2 16459360 bytes "0xDEADBEEF"  valid c870-advsecurityk9-mz.124-15.T9.bin
SNMP table CISCO-FLASH-MIB::ciscoFlashFileTable, part 2
 index      Type Date
 1.1.1 directory    ?
 1.1.2   unknown    ?

The index is a composite of the flash device, partition and file, so 1.1.2 is the second file on the first partition of the first flash device. Unless you have multiple flash devices and/or partitions you can largely ignore these details, take a look at ciscoFlashDeviceTable & ciscoFlashPartitionTable if you want more information.

With the filename, we now need to construct a request to delete it. This sort of operation along with tasks such as erasing the flash, etc. uses the ciscoFlashMiscOpTable SNMP table. Using the same technique I demonstrated in the previous article insert a new row and activate it:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
ciscoFlashMiscOpCommand.23 i delete \
ciscoFlashMiscOpDestinationName.23 s c870-advsecurityk9-mz.124-15.T9.bin \
ciscoFlashMiscOpEntryStatus.23 i createAndGo
CISCO-FLASH-MIB::ciscoFlashMiscOpCommand.23 = INTEGER: delete(3)
CISCO-FLASH-MIB::ciscoFlashMiscOpDestinationName.23 = STRING: c870-advsecurityk9-mz.124-15.T9.bin
CISCO-FLASH-MIB::ciscoFlashMiscOpEntryStatus.23 = INTEGER: createAndGo(4)

Note that I’m using createAndGo instead of active for the row status because it’s part of the same initial set request. Now check the table to see if the operation was successful:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser -Cb -Ci -Cw 80 ciscoFlashMiscOpTableSNMP table: CISCO-FLASH-MIB::ciscoFlashMiscOpTable
 index Command                     DestinationName                 Status
    23  delete c870-advsecurityk9-mz.124-15.T9.bin miscOpOperationSuccess
SNMP table CISCO-FLASH-MIB::ciscoFlashMiscOpTable, part 2
 index NotifyOnCompletion         Time EntryStatus
    23              false 0:0:00:56.24      active

It worked so checking the flash storage should confirm the file is now absent. Now we can upload the new IOS image, you’ll need to have it copied to /tftpboot on your TFTP server. This uses another SNMP table but the procedure is exactly the same, insert a row describing the operation we want to perform:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
ciscoFlashCopyCommand.23 i copyToFlashWithoutErase \
ciscoFlashCopyProtocol.23 i tftp \
ciscoFlashCopyServerAddress.23 a \
ciscoFlashCopySourceName.23 s c870-advsecurityk9-mz.124-24.T4.bin \
ciscoFlashCopyEntryStatus.23 i createAndGo
CISCO-FLASH-MIB::ciscoFlashCopyCommand.23 = INTEGER: copyToFlashWithoutErase(2)
CISCO-FLASH-MIB::ciscoFlashCopyProtocol.23 = INTEGER: tftp(1)
CISCO-FLASH-MIB::ciscoFlashCopyServerAddress.23 = IpAddress:
CISCO-FLASH-MIB::ciscoFlashCopySourceName.23 = STRING: c870-advsecurityk9-mz.124-24.T4.bin
CISCO-FLASH-MIB::ciscoFlashCopyEntryStatus.23 = INTEGER: createAndGo(4)

Chances are this operation will take a minute or two to complete, keep checking ciscoFlashCopyTable until it completes:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser -Cb -Ci -Cw 80 ciscoFlashCopyTable
SNMP table: CISCO-FLASH-MIB::ciscoFlashCopyTable
 index                 Command Protocol ServerAddress
    23 copyToFlashWithoutErase     tftp
SNMP table CISCO-FLASH-MIB::ciscoFlashCopyTable, part 2
 index                          SourceName DestinationName RemoteUserName
    23 c870-advsecurityk9-mz.124-24.T4.bin                               
SNMP table CISCO-FLASH-MIB::ciscoFlashCopyTable, part 3
 index               Status NotifyOnCompletion         Time EntryStatus Verify
    23 copyOperationSuccess              false 0:0:05:03.71      active  false
SNMP table CISCO-FLASH-MIB::ciscoFlashCopyTable, part 4
 index ServerAddrType ServerAddrRev1 RemotePassword
    23           ipv4    ""              ?

With the new image uploaded it’s time for the coup de grĂ¢ce, reloading the device:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
tsMsgSend.0 i reload
OLD-CISCO-TS-MIB::tsMsgSend.0 = INTEGER: reload(2)

Your device should now reboot and after a short while come back up running the updated IOS image. You can confirm the source of the reboot request with the following:

router#show version | i reason
Last reload reason: snmp shutdown request

Powerful stuff. Needless to say, don’t make this accessible to the world with an SNMPv2 community of “private”.

Using SNMP to trigger Cisco TFTP backups

Sunday, April 24th, 2011

I recently discovered you can use SNMP to trigger a Cisco device to write its configuration to a remote TFTP server. Apparently this feature has existed in Cisco devices in one form or another for over a decade, but somehow I’d missed it completely. This seems a far better alternative to getting in a muddle with Expect-style scripts to telnet or SSH into the device. One or more Access Control Lists can also be utilised to restrict access.

First of all, let’s configure the IOS device which will need to be running at least IOS 12.0. The first thing to do is create an Access Control List which matches the IP address of the “management station” which will be issuing the SNMP commands and for simplicities sake is also the TFTP server:

router(config)#access-list 20 permit host
router(config)#access-list 20 deny any

Next we define an SNMP view that restricts how much of the tree is writable, allowing write access to everything is both unnecessary and inadvisable:

router(config)#snmp-server view myview ccCopyTable included

Now we create an SNMP group that ties the ACL and view together:

router(config)#snmp-server group mygroup v3 priv write myview access 20

Note that I’ve specified SNMPv3 along with the authPriv security level, not all devices support that so adjust accordingly. I’ve also not specified a read view so it will use the default v1default view which can read pretty much everything. Next create a new user as a member of this group:

router(config)#snmp-server user myuser mygroup v3 auth md5 authsecret priv des privsecret

Finally, restrict which TFTP servers can be written to, triggered by SNMP. This stops someone from making your device write files to an arbitrary TFTP server somewhere:

router(config)#snmp-server tftp-server-list 20

That should be all of the configuration needed for the Cisco device.

Now on our “management station” which can be anything provided it has a recent version of Net-SNMP installed, (CentOS 5.x works fine), first make sure TFTP is installed and running. Normally the TFTP server cannot create new files so create an empty file ready:

# touch /tftpboot/router.conf

Now install the various MIB files we’ll need. You don’t need these but for the sake of readability it’s easier to use the symbolic names rather than raw OID strings. You’ll need to download the following MIB files:


Put them in a directory such as ~/mibs and then configure Net-SNMP with the following two environment variables:

# export MIBDIRS=+${HOME}/mibs

You should be able to test that everything is configured correctly:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser ccCopyTable
CISCO-CONFIG-COPY-MIB::ccCopyTable: No entries

Now we need to create a row in this table describing the copy operation we’d like to perform. Without getting too heavy into how SNMP tables work, we basically need to use a unique index number to refer to the row. As we’ve already shown, the table is empty so pick any random number, umm, 23! Create a new row with the following column values:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
> ccCopyProtocol.23 i tftp \
> ccCopySourceFileType.23 i runningConfig \
> ccCopyDestFileType.23 i networkFile \
> ccCopyServerAddress.23 a \
> ccCopyFileName.23 s router.conf
CISCO-CONFIG-COPY-MIB::ccCopyProtocol.23 = INTEGER: tftp(1)
CISCO-CONFIG-COPY-MIB::ccCopySourceFileType.23 = INTEGER: runningConfig(4)
CISCO-CONFIG-COPY-MIB::ccCopyDestFileType.23 = INTEGER: networkFile(1)
CISCO-CONFIG-COPY-MIB::ccCopyServerAddress.23 = IpAddress:
CISCO-CONFIG-COPY-MIB::ccCopyFileName.23 = STRING: router.conf

Now if you re-run the snmptable command (with some extra flagsoup to help readability) you’ll see there’s now a row:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser -Cb -Ci -Cw 80 ccCopyTable
 index Protocol SourceFileType DestFileType ServerAddress    FileName UserName
    23     tftp  runningConfig  networkFile router.conf        ?
SNMP table CISCO-CONFIG-COPY-MIB::ccCopyTable, part 2
 index UserPassword NotificationOnCompletion State TimeStarted TimeCompleted
    23            ?                    false     ?           ?             ?
SNMP table CISCO-CONFIG-COPY-MIB::ccCopyTable, part 3
 index FailCause EntryRowStatus ServerAddressType ServerAddressRev1
    23         ?              ?              ipv4       ""

This won’t initiate the operation until we set one more column to activate the table row:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
> ccCopyEntryRowStatus.23 i active
CISCO-CONFIG-COPY-MIB::ccCopyEntryRowStatus.23 = INTEGER: active(1)

Now check the table again:

# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X authsecret -u myuser -Cb -Ci -Cw 80 ccCopyTable
 index Protocol SourceFileType DestFileType ServerAddress    FileName UserName
    23     tftp  runningConfig  networkFile router.conf        ?
SNMP table CISCO-CONFIG-COPY-MIB::ccCopyTable, part 2
 index UserPassword NotificationOnCompletion      State  TimeStarted
    23            ?                    false successful 5:0:20:49.92
SNMP table CISCO-CONFIG-COPY-MIB::ccCopyTable, part 3
 index TimeCompleted FailCause EntryRowStatus ServerAddressType
    23  5:0:20:51.39         ?         active              ipv4
SNMP table CISCO-CONFIG-COPY-MIB::ccCopyTable, part 4
 index ServerAddressRev1
    23       ""

You can see the state column now reads successful so check the empty file you created under /tftpboot, it should now have the contents of the current configuration for your device.

The row in the table will be automatically removed after a few minutes but you can also explicitly remove it:

# snmpset -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser \
> ccCopyEntryRowStatus.23 i destroy
CISCO-CONFIG-COPY-MIB::ccCopyEntryRowStatus.23 = INTEGER: destroy(6)
# snmptable -v 3 -l authPriv -a MD5 -A authsecret -x DES -X privsecret -u myuser ccCopyTable
CISCO-CONFIG-COPY-MIB::ccCopyTable: No entries

There’s still a fair bit of functionality that I’ve skipped over, you might be able to use something other than TFTP or generate an SNMP trap on completion. Experiment with changing the Access Control Lists and stopping the TFTP server to see how the table output changes.