Multi-Domain-SDA-ACI-Lab Guide
Multi-Domain-SDA-ACI-Lab Guide
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Page 1 of 41
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
Page 2 of 41
NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE
PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR
APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION
PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO
LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE
PROVIDED 'AS IS' WITH ALL FAULTS. CISCO AND THIRD-PARTY SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED,
INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL
DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR
INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of
Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks.
Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
Copyright 2012 Cisco Systems, Inc. All rights reserved.
Page 3 of 41
Table of Contents
Table of Contents ................................................................................................................................. 4
Cisco Multidomain: SDA-ACI Integration.............................................................................................. 6
Lab Overview .................................................................................................................................... 7
Section 1: Campus Policy ......................................................................................................... 8
Section 2: SDA-ACI Integration ................................................................................................. 8
Section 3: Common Use Cases................................................................................................. 8
Overall Lab Topology .................................................................................................................... 8
Assumptions ..................................................................................................................................... 8
Before you begin ............................................................................................................................... 9
Terminology................................................................................................................................... 9
Connecting to Lab Devices (for reference only)................................................................................ 9
Connect to a POD ......................................................................................................................... 9
Lab Exercise 1: SD-Access Baseline (Segmentation Policy Focus Only) ........................................ 9
Cisco Identity Services Engine ...................................................................................................... 9
Cisco pxGrid ................................................................................................................................ 10
Cisco Software Defined Access .................................................................................................. 10
Cisco DNA Center ....................................................................................................................... 11
Sharing SGTs from ISE to DNAC ............................................................................................ 11
Lab Exercise 2: Authenticating and Authorizing a host to connect to the SDA fabric ..................... 14
SGT Assignment for Authenticated Users ............................................................................... 14
Connecting a host to the network ............................................................................................ 16
Lab Exericse 3: Enabling ISE to ACI Integration ............................................................................ 19
Import APIC Certificate into ISE .................................................................................................. 19
Security Group Tag Numbering for APIC EPGs.......................................................................... 21
Learning IP-EPG mappings......................................................................................................... 22
Enabling ISE SXP .................................................................................................................... 22
Configuring a SXP Peer........................................................................................................... 23
ACI Settings in ISE ...................................................................................................................... 24
SXP Propagation ..................................................................................................................... 25
Lab Exercise 4: Shared Policy Group Verification .......................................................................... 26
APIC Internal EPGs converted to Security Groups ..................................................................... 26
TrustSec Security Groups converted to ACI External EPGs ....................................................... 27
Internal Endpoints (IEPs) converted in ISE as IP-Mappings ....................................................... 28
Verify ISE IP Mappings are converted to External EPG Subnets ............................................... 28
Page 4 of 41
Lab Exercise 5: Campus Identity provided to ACI ....................................................................... 29
Lab Exercise 6: ACI Context provided to SDA Policy Domain .................................................... 32
Enabling Group-Based Enforcement on the SDA Border ........................................................... 32
Making the SDA Border aware of APIC IP-SGT (EPG) mapping................................................ 33
Appendix............................................................................................................................................. 39
Phase 1 Solution Testing ........................................................... Error! Bookmark not defined.
Phase 1 Considerations............................................................. Error! Bookmark not defined.
Page 5 of 41
Cisco Multidomain: SDA-ACI Integration
Cisco Multidomain is a program to enable customers to use common policies across multiple operational
domains using Cisco products, essentially to simplify policy management for customers using Cisco
technology in multiple operational domains.
This is necessary because customer’s policy challenges are not specific to a place in the network, but
spans all across campus, wan, data center, and cloud environments.
This program is also intended to enable customers to get started with advanced Cisco policy capabilities
in any given domain and make it easy to extend their policy management capabilities into other domains
easily; to provide competitive differentiation for all Cisco products in scope.
The initial phase of this program starts with SDA-ACI Integration which is focused on enabling policy
objects to be shared across between the sd-access domain.
ACI shares some base characteristics with SDA in that it provides policy for base segmentation, QoS,
path selection, and service chaining via a construct called endpoint groups (EPGs). Endpoint groups to
customers are synonymous with Scalable Groups used in SDA, hence customers have asked for
interoperability between EPGs and SGTs, so that they can apply policy within the data center leveraging
groups using context from SDA. In addition, customers would like to use the EPGs from the data center
to invoke services in the campus and WAN based on SGT/EPG pairs into and out of the data center.
SDA-ACI Scenarios
This program is also intended to enable the development of common policy management across
domains, by providing common groups for future multi-domain management applications to leverage.
Page 6 of 41
The starting point for this program is the Phase 1 SDA-ACI integration illustrated below :
SDA-ACI integration via the use of ISE provides the ability to interconnect the administrative domains of
SDA and ACI to provide policy normalization to simplify security design, operations, and compliance.
Lab Overview
This lab is divided into three main sections:
Page 7 of 41
Section 1: Campus Policy
• Exercise 1: Lab topology and brief introduction to how DNAC and ISE interact with each other to
provide segmentation within a sd-access environment.
• Exercise 4: Validation that DNAC, ISE, and APIC are integrated and sharing group information
Assumptions
This lab assumes:
• The student is familiar with the Cisco Identity Services Engine features and functions
• The student is familiar with DNA Center features and functions
• The student is familiar with Scalable Group/SGT and SGACL functions
Page 8 of 41
Before you begin
Today, terminology for the same policy constructs differ between Cisco Identity Services Engine, Cisco
Application Centric Infrastructure, and Cisco DNA Center. In this lab the term “group” is used to mean
scalable group/security group/endpoint group. The term “policy” refers to policies defined by the use of
groups. `
Terminology
DNAC ISE APIC
Scalable Group Security Group Endpoint Group
Access Contract Security Group ACLs Contract
Group-Based Access TrustSec Policy Group based access
Control
Note: Admin PC access is through RDP, therefore you must have an RDP client installed on your computer
Connect to a POD
Step 1 Launch the Remote Desktop application on your system
a. Connect to your POD Admin PC using RDP
b. Login as admin / ISEisC00L
Note: All lab configurations can be performed from the Admin client PC.
Page 9 of 41
ISE is used to manage authentication, authorization and deploy SGT policies within Cisco SD Access.
ISE is also the AAA server used in SD-Access.
Cisco pxGrid
Cisco pxGrid (Platform Exchange Grid) is an open, scalable, and IETF standards-driven data-sharing
platform
Cisco DNAC subscribes as a pxGrid client to ISE to obtain the SGT’s from ISE through this pxGrid
(Publish Exchange Grid) client/server relationship.
In this lab, the sd-access network is divided into two VNs, User and IOT, to completely isolate users
from IOT devices. Then scalable groups needed to achieve the segmentation goals within these VNs
have been mapped by DNA Center.
Page 10 of 41
Cisco DNA Center 1.3.1
DNAC is the central policy authoring tool for sd-access networks. Policy administrators create policies
on DNAC and DNAC writes these policies to ISE. ISE then deploys the policies to the network devices.
The groups used in these policies are learned from ISE via pxGrid.
In this lab, the pxgrid connection between DNAC and ISE is already established. In the steps below you
will be able to see how quickly group information is exchanged between DNAC and ISE.
Page 11 of 41
These default groups are automatically synchronized with ISE. These groups are visible as Security
Groups on ISE
Create a scalable group named Doctors. Be sure to select the “Users_VN” and to check the
box “Propagate to ACI”
Page 12 of 41
Note: Checking the box “propagate to ACI” tells ISE to share this group with APIC
Click Save
Upon clicking save, a sync status indicator appears beside the Doctors group to indicate the sync status
with ISE
Page 13 of 41
Log in to ISE. Use the Chrome bookmark, “ISE” (admin/SGTsr0x!)
Navigate to PolicyàGroup-Based Access ControlàScalable Groups to view the newly
added Doctors Security Group with “propagate to ACI” enabled
In the steps below, you will walk through configuring an ISE policy set. A policy set is a grouping of
authentication and authorization policies logically grouped together for Administrative ease.
This lab utilizes a pre-configured “Campus Users” policy set. The policy set is configured to allow any
wired 802.1x users authenticate against any configured user identity store and if any one of the users is
an employee, the Doctors SGT is assigned to them. Let’s see what this policy set looks like:
On ISE, navigate to PolicyàPolicy Sets. Take a moment to digest the configured Campus
Users policy set. This policy will match any authentications that match the “Wired_802.1X”
condition.
Now click the “>” icon in the Campus Users policy window as shown below
Click on the Authentication Policy as shown below and view the policy configuration
Page 14 of 41
As shown above, we are making use of the default authentication policy for simplicity. This policy has
no specific conditions that must be met and any configured user identity store, indicated by
“All_User_ID_Stores”, is used to authenticate the 802.1x users.
The pre-configured authorization policy assigns the Employees security group to ISE internal users in
the Employees group.
For simplicity, change the Security Groups column from Employees to Doctors will associate
the newly created Doctors SGT to the users.
In the steps below, we will connect a user to the network. The client pc is connected to int g1/0/1 which
has been preconfigured for Dot1x. Upon successful authentication, the user will be dynamically
assigned the Doctor SGT by matching the authorization rule configured in the steps in the previous
section.
Note: To limit the need to go back and forth between applications, you will view the preconfigured “Permit All”
contract in APIC later in the lab
Page 16 of 41
Note: Client1 is a windows virtual machine, not a physical pc.
Within the remote desktop app, click on Edge to navigate to the switch’s console
Edge#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Edge(config)#int g1/0/1
Edge(config-if)#shut
Edge(config-if)#no shut
Server Policies:
Vlan Group: Vlan: 1021
SGT Value: 17
Note: If the status is not “Authorized”, please go back and shut/no shut the switchport.
Note: Please make a note of the assigned IP address and SGT value. These will be needed later in this lab.
Page 18 of 41
On the client pc, open Chrome and click the EMR bookmark. The Cisco HealthConnections
Center portal should appear
From the Browser window of the APIC controller click to the left of the URL bar as shown
below. Select “Certificate” to view the certificate. When viewing the certificate, click the
“Details” tab to export the certificate to your local computer.
Page 19 of 41
From the ISE UI, navigate to AdministrationàSystemàCertificatesàTrusted
Certificates.
Click import to import the APIC certificate to the ISE Certificate Store.
Page 20 of 41
Security Group Tag Numbering for APIC EPGs
Under Work Centers àTrustSec àSettingsà General TrustSec Settings
The EPGs that are received by ISE will be assigned a SGT value. Go to the Security Group Tag
Numbering for APIC EPGs and Check to modify the SGT value to any number. By default it is 10,000
and is unchecked. If unchecked the EPGs, which are received by ISE, will be assigned a SGT value
from 10,000.
Page 21 of 41
Learning IP-EPG mappings
ISE SXP is used to learn and share IP to group mappings to and from ISE. While the communication
between ISE and APIC is handled by REST APIs, not SXP, ISE SXP MUST be enabled and configured
in order for ISE to register the IP to EPG mappings learned from APIC
Note: You may skip to “ACI Settings in ISE” since the items are pre-configured. This section is added because
often the items configured here are the MOST COMMON reason why SDA-ACI integration “doesn’t work”
Page 22 of 41
Navigate to Work CentersàTrustSecàSettingsàSXP Settings
Check “Add radius mappings into SXP IP SGT mapping table”
Optional: Check “Publish SXP bindings on PxGrid”. This is necessary to share bindings with
PxGrid consumers
Note: This peer is the network device that you want to target IP-SGT mappings to for enforcement. For example,
in this lab, we will use the sd-access border to enforce North to South communications.
Note: A dummy address can be used when there is no need to actually share mappings to a network device
Page 23 of 41
ACI Settings in ISE
Under Work CentersàTrustSecà SettingsàACI Settings
Check the TrustSec-ACI Policy Element Exchange and fill in the settings as pictured below.
Note: The tenant and L3Out names are case sensitive. Additionally, double check your pod information.
Entering in the incorrect information can cause you to connect to a different pod.
Page 24 of 41
Note: ISE supports a single cluster of 3 APIC controllers. Currently, ISE supports single Tenant and a single L3Out
logical connection
Review the Naming Convention settings for new SGTs and EPGs. The configured suffix will
be appended to the converted SGTs and EPGs in ISE and APIC. In the example bellow,
_EPG and _SGT will be appended
Check “TrustSec-ACI Policy Element Exchange” at the top of the screen before clicking Save
SXP Propagation
Before configuring the SXP Propagation settings there is a concept of SXP Domains that needs to be
understood. An SXP Domain is a collection of SXP Devices and the administrator can decide which
domain to send IP-SGT mappings to. This is not mandatory to create a new SXP domain, as a Default
Domain already exists in ISE.
Click Save.
Once saved ISE and the APIC controller will start sharing the policy group information (SGTs & EPGs)
with each other. The internal EPGs from APIC will be converted to Security Groups on ISE and
automatically shared to Cisco DNA Center via PxGrid. Likewise, the Security Groups originating from
ISE will be converted to external EPGs on APIC.
Page 25 of 41
Lab Exercise 4: Shared Policy Group Verification
Page 26 of 41
Note: ISE supports only 32 characters SGT name whereas ACI supports a 64 character EPG. In these cases the
name will be truncated and the full EPG name details can be viewed in the description.
Page 27 of 41
Internal Endpoints (IEPs) converted in ISE as IP-Mappings
Once the EPGs are converted to the relevant SGTs in ISE, the Endpoints (EPs) of the EPGs are
converted to IP-Mappings under the All SXP Mappings in ISE
Look for the newly created IP-SGT Mappings of the Security Groups (of EPGs) with the relevant SXP
Domain
Note: The mapping for the Finance EPG is not listed because the VM does not have an IP.
Now that EPGs and SGTs are shared, let’s look at some common use cases
Page 28 of 41
Lab Exercise 5: Campus Identity provided to ACI
Now that APIC has learned about the Doctors group of campus users and their associated addresses,
APIC has more specific context on and visibility of the traffic incoming to the data center. Administrators
can use this context to create more granular segmentation policies.
In the steps below you will create a contract to allow Doctors access to the EMR server. But before
creating a new contract, let’s revisit the user’s access.
On the client pc, clear the browser cache and re-launch the EMR portal page. THIS WILL
FAIL.
From the DOS prompt, launch a continuous ping to EMR (get the IP address from Chrome).
These pings will also FAIL.
Failure reason: Now that the user’s IP is identified as “Doctor”, a contract specifically allowing doctors
access to EMR is required.
Page 29 of 41
On the resulting window, you can see the preconfigured policy for L3Out to EMR access
Now create a contract specifically to allow Doctors to EMR
Select the “Layer 3” icon, , drag it to center of the screen, and release your mouse. The
result will look similar to this:
Page 30 of 41
Select the Contract icon, , and without releasing your mouse drag it to touch EMR and
then Doctors_SGT. The result will look similar to this:
Once you let go of the mouse, configure the resulting screen as follows:
Page 31 of 41
Click OK to finalize the configuration
Navigate back to the client pc. The pings and EMR portal should work now.
In the following steps, you will configure and deploy a group-based policy to drop traffic sourced from
Doctors SGT to EMR EPG on the sd-access border switch (as the traffic exists the SDA fabric) so that
the traffic does not need to traverse the network to only to be dropped at the ACI fabric leaf.
Navigate to the Remote Desktop Client Manager application and select Border.
Note: “cts role-based enforcement vlan 3002” enables enforcement specifically for the User_VN. To enable enforcement for
any configured VLAN, the command would be “cts role-based enforcement vlan all”
Page 32 of 41
Note: Due to DNAC automation, vlan 3002 may not be the User_VN. Type “show int vlan 3003”. If vlan 3003 is mapped to
the User_VN, enable enforcement for vlan 3003 instead
Click Save
Go back to the ssh connection to the border to configure the SXP peering back to ISE
Border#conf t
cts sxp enable
cts sxp connection peer 10.1.100.21 source 172.16.101.254 password default mode local listener
hold-time 0 0 vrf User_VN
cts sxp default password 0 Cisco123
Note: If the connection status shows as OFF, double check the SXP settings in ISE, in particular the password-both sides of
the connection must use the same password
Now the border is ready to enforce traffic destined to the EMR server
Page 34 of 41
Create a policy to drop all traffic from doctors to EMR as shown below:
Page 35 of 41
Page 36 of 41
Now that the policy has been created, it still needs to be deployed to ISE. Click the “Deploy”
button to send the policy to ISE
Note: In future releases the deploy function will include an approval process flow.
Connect back to the border to view the policy change
Page 37 of 41
This completes the
SDA-ACI Integration Lab.
Great Job!!
Page 38 of 41
Appendix
Scale
Tested with SDA 1.2.10 using ISE 2.3 patch 7 and ISE 2.4 patch 6.
Scale
Number of SGTs mapped to External EPGs 250
Max number of mappings per External EPG 8,000
Transaction rate 100 IP/Group Mappings per second
Page 39 of 41
Scale
Number of Internal EPGs mapped to SGTs 1000
Max Number of IP/Group Mappings 256,000
Max Number of SXP Peers 100 per pair of ISE SXP nodes
Max number of pxGrid peers 200 per ISE pxGrid node
Design Considerations
Page 40 of 41
Page 41 of 41