Securing the wired network with 802.1X

This post covers an innovation project I did to secure the wired network at a shared conf center with 802.1X.

Every few months we had to disable the wired network in order to prevent non-employees from being able to get online. This was not scalable, was prone to human error, and scheduling confusion. I planned to automate the process by enabling 802.1X aka dot1q on the switches using our Windows AD via Cisco ACS.

Any Domain joined devices that plugged in would get access to our corp VLAN, and unknown devices would go into a dead VLAN. Long term I planned to enable a wired guest VLAN and had it labbed out for non local switched wifi where the guest VLAN exists on the switch you’re connected to but didn’t around to labbing local switching using CAPWAP tunnels.

Wired Guest Access using Cisco WLAN Controllers Configuration Example:

http://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-lan-wlan/99470-config-wiredguest-00.html

Phones would end up in the VoIP VLAN but they weren’t equipped for dot1x authentication. So I had two options either manually add each MAC address to a list in the ACS which is not scalable or supportable. Instead I removed the user VLAN from the ports with phones connected. The voice VLAN itself was locked down with a strict ACL that only allowed communication with the VoIP server.

Client configs:

In this environment there were only Windows clients. Windows needs to be configured to enable their supplicant for dot1x. Anything with Windows can and should be controlled by Group Policy. I researched the needed settings and how to set them via GPO. Then I worked with the Windows team to roll out the GPO to a pilot group and finally deploy globally.

Configuring 802.1X Wired Authentication on a Windows 7 Client:

https://documentation.meraki.com/MS/Access_Control/Configuring_802.1X_Wired_Authentication_on_a_Windows_7_Client

You can do the same thing with other versions of Windows just this was the one I worked with.

Windows AD GPO guide:

https://msdn.microsoft.com/en-us/library/dd759237.aspx

When these Win 7 machines were upgraded to Win 10 the GPO still worked.

 

Switch configs:

!Debug 802.1x all

!Debug radius all

conf t

aaa new-model

aaa authentication dot1x default group radius

aaa authorization network default group radius

aaa accounting network default start-stop group radius

dot1x system-auth-control

dot1x guest-vlan supplicant

radius server acs1.foo.com

 address ipv4 10.1.181.2 auth-port 1812 acct-port 1813

 key 0 This-IsTheSharedSecret123

exit

radius server acs2.foo.com

 address  ipv4 10.2.181.2 auth-port 1812 acct-port 1813

 key 0 This-IsTheSharedSecret123

exit

!User ports

interface <interface>

 authentication port-control auto

 authentication host-mode multi-domain

 dot1x pae authenticator

 authentication event no-response action authorize vlan 15

 authentication event fail action authorize vlan 15

!Phone ports

interface <interface>

no switchport access vlan 1010

end

copy run start

ACS Configs:

I already had the ACS configured to do dot1x auth for wifi clients so it was simple to just add the new switches to the rule set.

 

It was a success and opened the door to securing all wired networks.

NTP redesign

This post is about a bug that affected NTP (Network Time Protocol) and our redesign of the environment bypass the issue.

In this environment the core Cisco 7604 IOS routers were the NTP stratum 2 servers (x.x.x.123 because fun with port numbers). The IP was an HSRP standby IP. There were several downstream Linux NTP servers and Window Domain Controllers serving NTP to Windows clients. As unsupported Linux servers died their IP’s were just added to servers that were still alive. Eventually this got messy.

After the 7604 routers were replaced with a pair of ASR1006X we ran into some interesting issues. Windows users we no longer able to log. Turns out the Domain Controllers were falling out of sync. My Infoblox DDI servers also showed stale time. Users were eventually able to log into the Domain either before or after the Windows team changed their NTP config. The sys admins were now syncing with one of the Stratum 3 Linux servers. Knowing that the only thing that had changed in our environment was the ASR routers I knew this wouldn’t be the end of the issue.

I opened a ticket with Cisco TAC to troubleshoot with the ASR’s. TAC thought maybe it was because we were using a standby IP. But I couldn’t get resources to help test so we got nowhere. Eventually a bug ID CSCsq31723 was made which I think is related. Fast forward 6 months and Windows users can’t log in again. This time we decided to go nuclear and redesign the whole NTP layout.

The new design removed all servers running on unsupported hardware and OS’s. It also made use of our Infoblox DDI grid which is a purpose made tool for DNS, DHCP, IPAM, NTP, and File Distribution.

We decided on this:

  • 3 internet Stratum 1 servers
  • 3 Stratum 2 servers. Infoblox DDI Grid Master at HQ, the Grid Master Candidate at our DR location, and a Linux server to diversify technologies.
  • Place the Stratum 2 servers in a mesh.
  • The Infoblox DDI Grid Master and Grid Master Candidate fed the 2 HA pair (4 servers with 2 VIP’s) Grid Members.
  • Create Access Lists on the Stratum 2 servers so only the Stratum 3 servers can sync with them.
  • All clients would then sync with the 3 HA pairs of Infoblox DDI Grid Members.
  • Set up NTP Authentication (https://www.nist.gov/pml/time-and-frequency-division/time-services/nist-authenticated-ntp-service).

Both VIP’s of the HA paired Grid Members had a user friendly DNS record and some systems accept a hostname/DNS for the NTP config. That would be handy if we ever needed to change the IP’s again. For extra flexibility we could use F5 BIG-IP GTM (DNS load balancing). But network devices like routers and switches don’t support using DNS for NTP meaning there would be two sets of configs. One with NTP configs using DNS and another hard set to static IP’s. We wanted a global config so we went with static IP’s everywhere.

The IP’s were given out and people were told to migrate. We setup a span and periodically checked to see who was still pointed at the old servers before finally retiring the old IP’s/servers.

It wasn’t perfect but it was a big improvement. Long term I’d want to install our own Stratum 0 GPS antennas instead of using internet hosted servers. For a home project I’m thinking of using a Raspberry Pi to make one using this link as a guide (https://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html).