Configuring Nexus 7000

Virtualization
LTRCRT-2605

Barry Gursky CCIE#7208

bgursky@fireflyeducate.com
Sr. Data Center Architect
www.fireflyeducate.com

Dr. Peter J. Welcher

pjw@netcraftsmen.net
Principal Consultant
www.netcraftsmen.net

Agenda
Describe the Cisco Nexus 7x00 Series Switch
Hardware and Software Features
Discuss VDC Design Practices and
Configuration
Review vPC Design and Configuration
Hands-on Lab Deploying VDC
Hands-on Lab Configuring Double-Sided vPC
Hands-on Lab Establishing L3 Communication

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Cisco Nexus 7x00 Data Center Switch

Objectives
Discuss the Cisco Nexus 7000
Describe the architecture of VDCs
Describe the fabric module capacity and redundancy capability

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Cisco Nexus 7x00

Cisco Nexus 7000 Platform

1.92 - 15+ Tb/s System

DCB and FCoE Ready
Modular OS
Device Virtualization
Continuous Operations

Nexus 7004

Nexus 7009

Nexus 7010

Nexus 7018

Slots

2 I/O + 2 Sup

7 I/O + 2 sup

8 I/O + 2 sup

16 I/O + 2 sup

Height

7 RU

14 RU

21 RU

25 RU

BW / Slot Fab 1

N/A

N/A

230 Gig / slot

230 Gig / slot

BW / Slot Fab 2

1.92Tbs Fixed

550 Gig / Slot

550 Gig / slot

550 Gig / slot

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Cisco Nexus 7004 Chassis

Side-to-back
airflow

Supervisor
Slots (1-2)
I/O Slots
(2-3)

LTRCRT-2605

1.92Tb/s in 4RU modular chassis

2 I/O module slots, supporting 1, 10, 40 and 100 Gb M-series and
F-series
The chassis does not have fabric modules, the I/O modules
connect directly through the backplane
Side-to-back air flow

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Cisco Nexus 7009 Chassis

Front

Rear

Supervisor
Slots (1-2)

Summary
LEDs

Optional
Front Doors

Side-to-side
airflow

Locking
Ejector
Levers

Crossbar
Fabric
Modules
I/O Slots
(3-9)

Integrated
Cable
Management
LTRCRT-2605

N7K-C7009
2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Power Supplies

Fan Tray

Cisco Nexus 7010 Chassis

System Status
LEDs

ID LEDs on
all FRUs

Front-to-back
airflow

Integrated Cable
Management with cover

Air Exhaust

Optional Locking
Front Doors

System Fan Trays

Fabric Fan Trays

Locking Ejector
Levers

21RU

Supervisor
Slots (5-6)

Two Chassis
per 7 Rack
Crossbar Fabric
Modules

I/O Module Slots

(1-4, 7-10)

Power Supplies
Air Intake with
Optional Filter

Front
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

N7K-C7010
Cisco Public

10

Common Equipment
Removes from Rear

Rear

Cisco Nexus 7018 Chassis

ID LEDs on all
FRUs

Optional front
door
Locking
ejector levers

Side-to-side
airflow

Supervisor
slots (9-10)

25RU

Payload slots
(1-8, 11-18)

Crossbar
fabric
modules

Common equipment
removes from rear

Power supplies
(2 - 4)

Power supply
air intake

Front
LTRCRT-2605

System
fan trays

System status
LEDs

Integrated cable
management

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

11

Rear

Cisco Nexus 7706 Chassis

Front

9RU

8 Payload Slots
(1.3T/slot)

Redundant Supervisor
Engines

Front-to-Back Airflow
Up to 8x 3kW
AC/DC Power Supplies
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Cisco Nexus 7710 Chassis

8 payload slots
(1.3T/slot)
6 fabric modules
(behind fan trays)
Redundant Supervisor
Engines

14RU

3 fan trays
Front-to-back airflow

Front
LTRCRT-2605

Up to 8x 3kW
AC/DC power supplies

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

13

34

Rear

Cisco Nexus 7718 Chassis

16 payload slots
(1.3T/slot)
Redundant Supervisor
Engines
26RU

6 fabric modules
(behind fan trays)
Front-to-back airflow
3 fan trays

Front
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

16 3000W AC/DC
power supplies
Cisco Public

14

35

Rear

Cisco Nexus 7x00 Switch Line Modules

Supervisor Engine 2/2E

N7K-SUP1
Beacon LED
AUX
Console

Management
Ethernet

Compact Flash

USB Ports

N7K-SUP2
N7K-SUP2E

N77-SUP2E
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

16

CMP Ethernet

Management Ethernet Interface

10/100/1000 interface
Belongs to dedicated
management VRF
Supports IEEE 802.1ae
LinkSec encryption

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

17

Cisco Nexus 7000 10GE M1 I/O Modules

8-port 10G with X2 transceivers
80G full-duplex fabric connectivity
Two integrated forwarding engines
(120Mpps):
Support for XL forwarding tables (licensed
feature)

8 ports wire-rate L3 multicast replication

802.1AE LinkSec

32-port 10G with SFP+ transceivers

80G full-duplex fabric connectivity
Integrated 60Mpps forwarding engine:
XL forwarding engine on L version
Oversubscription option for higher density (up to
4:1)
8 ports wire-rate L3 multicast replication
802.1AE LinkSec

N7K-M132XP-12L

N7K-M108X2-12L
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

18

Cisco Nexus 7000 48-Port 1G M1 I/O Modules

Two 1G I/O module options:
48 1G SFP ports with XL forwarding
engine (N7K-M148GS-11L)
48 10/100/1000 RJ-45 ports with XL forwarding engine
(N7K-M148GT-11L)
N7K-M148GS-11L

Integrated 60Mpps forwarding engine

46G full duplex fabric connectivity:
Line rate on 48-ports with some
local switching

N7K-M148GT-11L

48 ports wire-rate L3 multicast replication

802.1AE LinkSec

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved. Cisco

Cisco
Public
Public

19

Cisco Nexus 7000 F1 Series I/O Module

SFP+ 10G I/O module
1G/10G dual-speed system-on-chip
(SoC) design
Layer 2 forwarding with L3/L4 services
(ACL/QoS)
Multi-protocol Classic Ethernet,
FabricPath, DCB

High performance:
230Gb/s fabric connectivity
20 line-rate ports per slot over fabric at 64
bytes
32 line-rate ports per slot with local
switching
320512 line-rate ports per system (7018
chassis)

Low latency unicast/multicast 5sec

module-to-module at 64 bytes

N7K-F132XP-15

sometimes called switch-on-chip

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

20

Cisco Nexus 7000 Enhanced F2-Series I/O Module

1/10Gb SFP+/1/10GBaseT
48 SFP/SFP+ 1/10G I/O module (N7K-F248XP25E)
48 port 1/10GBASE-T (N7K-F248XT-25E)
1G/10G dual-speed system-on-chip (SoC) design
L2/L3 forwarding with L3/L4 services (ACL/QoS)
Multi-protocol Classic Ethernet, FabricPath,
FCoE, DCB
Support for the Nexus 2000 Series Fabric
Extenders

N7K-F248XP-25E
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

High performance:

480Gb/s fabric connectivity

48 line-rate ports per slot
Up to 768 line-rate ports per system (7018
chassis)
Less that 7.5W per port (N7K-F248XP-25E)
Less that 9W per port (N7K-F248XT-25E)

N7K-F248XT-25E
21

Cisco Nexus 7000 M2 Series 24-port 10Gb

I/O Module

High performance:

10G Line-rate forwarding

Compatible with Fab1 or Fab2
L2/L3 switching functionality
Support for Nexus 2000 Fabric
Extender

24 non-blocking ports per slot

Up to 384 line-rate ports per system (7018
chassis)

N7K-M224XP-23L

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

22

Cisco Nexus 7000 M2 Series 6-port 40Gb

I/O Module

High performance:

40G/10G dual-speed
Common QSFP interface for 40G and
4x10G
Compatible with Fab1 or Fab2
L2/L3 switching functionality

550Gb/s fabric connectivity

16 non-blocking ports per slot
Up to 96 line-rate ports per system (7018
chassis)

N7K-M206FQ-23L

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

23

Cisco Nexus 7000 M2 2-Port 100Gb I/O Module

Dual speed capability:
Flexible 40GbE / 100GbE capability
Common CFP Interfaces for 100G and 40G
Optics for Single Mode:

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

Cisco Nexus 7000 F3-Series 12-Port 40Gb Module

12 Ports line-rate 40Gbps
QSFP+ modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and
VPLS.

N7K-F312FQ-25
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

25

Cisco Nexus 7000 F3-Series 6-Port 100Gb Module

6 Ports line-rate 100Gbps
CPAK modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and
VPLS.

CPAK-100G-LR4

N7K-F306CK-25

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

Nexus 7700 F2E 48-Port 1G/10G Module

48-port 1G/10G SFP/SFP+ module

Multi-protocol Classic Ethernet,

FabricPath, DCB/FCoE

Based on F2E ASIC technology

*Same exact functionalities of the
F2E on Nexus 7000

32K FIB TCAM/16K adjacency table

Wire-rate L2/L3 IPv4/IPv6

Nexus 2000 (FEX) support

16K MAC address table

480 Gbps/slot
720 Mpps/slot

VOQ Buffering: 72MB per module

N77-F248XP-23E
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

27

Cisco Nexus 7700 F3-Series 24-Port 40Gb Module

24 Ports line-rate 40Gbps
QSFP+ modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV,
MPLS, and VPLS.

N77-F324FQ-25
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

28

Cisco Nexus 7700 F3-Series 12-Port 100Gb Module

12 Ports line-rate 100Gbps
CPAK modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV,
MPLS, and VPLS.

N77-F312CK-26
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

The Cisco Nexus 2000 Series Fabric Extender

Fabric Extender
24 or 48TX 100/1000M host interfaces; 2 or 4x 10GE uplink interfaces
32 or 48 SFP/SPF+ 1/10G host interfaces with 8x 10GE or 4x 40G uplinks
32 port 1/10G TX host interfaces with 8x 10GE
Host port-channel support up to 24 port-channels per FEX
SPAN source/destination support
4 10GE SFP+ Uplinks

48 100/1000 RJ45 Downlinks

4 40GE QSFP+ Uplinks

2 10GE SFP+ Uplinks

48 1/10G SFP/SFP+ Downlinks

24 100/1000 RJ45 Downlinks

8 10GE SFP+ Uplinks

32 1/10G SFP/SFP+ Downlinks

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

FET-10G
Fabric Extender
transceiver
For FEX links only
Cisco Public

30

8 10GE SFP+ Uplinks

32 1/10GTX Downlinks

Cisco Nexus 7x00 Fabric Modules

31

Fabric Module

N7K-C7010-FAB-1
N7K-C7010-FAB-2

N7K-C7009-FAB-2
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

32

N7K-C7018-FAB-1
N7K-C7018-FAB-2

Nexus 7700 Fabric-2 Modules

N77-C7718-FAB-2

Consistent Nexus 7700 Fabric Architecture:

1.32 Tbps per slot with 6 Fabric modules

Multilevel redundancy with all modules All

modules share the total fabric bandwidth
helping to ensure a lossless forwarding during
failover

VoQ provides a QoS aware lossless fabric

Arbitrated Cross for Unicast

N77-C7710-FAB-2
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

33

Fabric Capacity and Redundancy

Fabrics

N7000
46 Gb/s
110 Gb/s
40 G

480 G

Module
Slots

1 G Module
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

34

N7700
220 Gb/s

F2 10 G Module

Fabric Capacity and Redundancy (Cont.)

Fabrics

N7000
230 Gb/s
550 Gb/s
Module
Slots

480 G

40 G

N7700
1.32 Tb/s (6 Fabric)

1G Module
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

35

10G Module

Cisco Nexus 7x00 Power Supplies

36

System Power

Load-sharing
Hot-swappable

N7K-AC-7.5 KW
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

N7K-AC-6.0 KW

DC System Power

N7K-DC-PIU
N7K-DC-6.0KW

Load-sharing
Hot-swappable

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

Nexus 7700 Power Supplies

90+% power supply efficiency above 50%

3000W AC

load

N77-AC-3KW

Typical power draw values up to 30%

less due to optimized cooling design

Redundancy modes for power supply or

grid failure

3000W DC

AC power supply accepts 110v or 220v

inputs at 20A with 10 different power

cables.
DC power supply accepts 47v or 60v

inputs at 40A
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

39

N77-DC-3KW

Why Use Many Smaller Power Supplies?

More flexibility for provisioning power redundancy
Provide grid redundancy for smaller configurations

CB 18
Slot

Pay-as-you-grow power
Headroom for future growth
Minimum bootup power:*
CB 18 ~4.5kW (2 PSUs)
CB 10 ~3kW (1 PSU)

Grid 1

Grid 2

Fully loaded w/F2E:

CB 18 ~12.5 kW (6 PSUs)
CB 10 ~7kW (4PSUs)

CB 10
Slot

Fully loaded, grid redundant:

CB 18 ~24kW (10 PSUs)
CB 10 ~14kW (6 PSUs)

* Chassis w/ 2 sups, 6 fabs, 3 fans

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

40

Nexus 7009 Cooling System

Rear Accessible Variable Speed Fan Tray
- Dynamically adjusted based on system
temperature to reduce power
NEW Independent Fan Speed Control
- Further Optimizes System Cooling
- Fans can be off when slots are empty
avoiding cooling of open slots

- Reduces fan speeds by slot

Power Reporting of Fan Tray Power Draw
- Visibility into system power usage

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

41

System Cooling for 7010/7018

Redundant system fan trays
provide cooling of I/O
modules and supervisor
engines.

Redundant fabric fans provide

cooling of crossbar fabric
modules.

N7K-C7010-FAN-S

N7K-C7010-FAN-F

Protection against any single fan, controller

or connector failure
Variable fan speed allowing speed reduction
for lower power usage in well-controlled
environments
Two system fan trays top and bottom:

Fabric Fans

Full redundancy for single fan failure

Fully redundant fan controllers

Dual Connectors
Single Fan

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

42

Cisco Nexus 7x00 supervisor Redundancy

43

Supervisor Synchronization
Request snapshot
State synchronized
Start services in standby and notify

Provide snapshot
Provide event-driven sync messages

Standby
etc.

HA Manager

HA Manager

Linux Kernel

Linux Kernel

NX7K Data Plane

Data plane streams

LTRCRT-2605

PIM

BGP

OSPF

etc.

PIM

BGP

OSPF

Active

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

44

Supervisor Failure

Standby
etc.

HA Manager

HA Manager

Linux Kernel

Linux Kernel

NX7K Data Plane

Data plane streams

LTRCRT-2605

PIM

BGP

OSPF

etc.

PIM

BGP

OSPF

Active

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

45

Supervisor Switchover

Go Active

Switchover

Active
etc.

HA Manager

HA Manager

Linux Kernel

Linux Kernel

NX7K Data Plane

Data plane streams

LTRCRT-2605

PIM

BGP

OSPF

etc.

PIM

BGP

OSPF

Reload

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

46

Supervisor Reactivation
Run boot diags
Request snapshot
State synchronized
Start services in standby and notify

Provide snapshot
Provide event driven sync messages

Active
etc.

HA Manager

HA Manager

Linux Kernel

Linux Kernel

NX7K Data Plane

Data plane streams

LTRCRT-2605

PIM

BGP

OSPF

etc.

PIM

BGP

OSPF

Standby

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

47

Nexus 7x00 Licensing

48

Cisco Nexus 7000 License Summary

Description

Part Number

Cisco NX-OS Enterprise LAN License

N7K-LAN1K9

Cisco NX-OS Advanced LAN License

N7K-ADV1K9

VDC license Supports 8 VDCs on Sup2E

N7K-VDC1K9

Cisco NX-Cisco NX-OS Transport Services License*

N7K-TRS1K9

Cisco NX-OS Enhanced Layer 2 License

N7K-EL21K9

Cisco Nexus 7000 MPLS License **

N7K-MPLS1K9

Cisco FCoE License for Nexus 7000 32-port 10G SFP+ (F1)
Cisco Nexus 7000 SAN Enterprise License

N7K-FCOEF132XP
N7K-SAN1K9

Cisco FCoE License for Nexus 7000 48 port 10G SFP/SFP+ (F2)

N7K-FCOEF248XP

Cisco Nexus 7004 Scalable Feature License

N7K-C7004-XL

Cisco Nexus 7009 Scalable Feature License

N7K-C7009-XL

Cisco Nexus 7010 Scalable Feature License

N7K-C7010-XL

Cisco Nexus 7018 Scalable Feature License

N7K-C7018-XL

Notes: * For OTV deployment, Enterprise and Advanced packages are required.
** For MPLS deployment Enterprise package is required.
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

49

Cisco NX-OS Licensing (7700 specific)

Description

Part Number

Cisco NX-OS Enterprise LAN License

N77-LAN1K9

VDC license Supports 8 VDCs on Sup2E

N77-VDC1K9

Cisco NX-OS Enhanced Layer 2 License

N77-EL21K9

Cisco Nexus 7000 SAN Enterprise License

N77-SAN1K9

Cisco FCoE License for Nexus 7700 48 port 10G SFP+ (F2e)

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

50

N77-FCOEF248XP

Nexus 7x00 Virtualization

51

Various Degrees of Virtualization

Data/Control Plane
Data/Control Plane
+
Management Plane

Data/Control Plane
+
Management Plane
+
Resources
+
Operating Environment

Hypervisor Model
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

52

Introduction to the VDC Architecture

L2 Protocols

L3 Protocols

L2 Protocols

VLAN Mgr

UDLD

OSPF

GLBP

VLAN Mgr

UDLD

OSPF

GLBP

VLAN Mgr

UDLD

BGP

HSRP

VLAN Mgr

UDLD

BGP

HSRP

LACP

CTS

EIGRP

VRRP

LACP

CTS

EIGRP

VRRP

IGMP

802.1x

PIM

SNMP

IGMP

802.1x

PIM

SNMP

MAC Table

RIB

MAC Table

Protocol Stack (IPv4 / IPv6 / L2)

VDCn

Infrastructure
Linux 2.6 Kernel

Physical Switch
2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

RIB

Protocol Stack (IPv4 / IPv6 / L2)

VDC1

LTRCRT-2605

L3 Protocols

53

Virtualization Hierarchy
Scalability:
4096 VLANs/VDC
1000 VRFs/VDC

N7K VDC
VLAN VLAN VLAN
VLAN VLAN VLAN
VLAN VLAN VLAN

VRF VRF VRF

VRF VRF VRF
VRF VRF VRF

VDC VLAN VLAN VLAN

VRF VRF VRF

VRF VRF VRF
VRF VRF VRF

VLAN VLAN VLAN

VLAN VLAN VLAN

Admin + 4 VDCs Sup2

Admin + 8 VDCs Sup 2E

VDC VLAN VLAN VLAN

VLAN VLAN VLAN
VLAN VLAN VLAN

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

54

VRF VRF VRF

VRF VRF VRF
VRF VRF VRF

Virtual Device Contexts

Consolidates physical network by
virtualizing onto common data center
networking infrastructure
Secures traffic between user departments
Allows departmental administration
Provides testing capability with no impact
on production systems

VDC
Prod
VDC
Extranet

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

55

VDC
DMZ

VDC Use Case Examples

Vertical Consolidation
Objective: Consolidate vertical infrastructure that delivers orthogonal roles to the same
administrative or operational domain
Benefits: Reduced power and space requirements, can maximize density of the platform,
provides smooth growth path, easy migration to physical separation in future
Considerations: Number of VDCs (4), Four VDCs != Four CPU Intra-Nexus 7000 cabling
needed for connectivity between layers
core1

core2

Core Devices
Aggregation Devices

LTRCRT-2605

agg3

agg4

accN

accY

2014 Cisco and/or its affiliates. All rights reserved.

core VDC

core VDC

agg VDC

agg VDC

accN

Cisco Public

56

accY

Core VDCs
Aggregation
VDCs

VDC Use Case Examples

Vertical & Horizontal Consolidation
Combined vertical & horizontal consolidation in small to medium designs (2 aggregation blocks or less)
Power, cooling and real estate optimization for multiple layers
Maximize the benefits of a high-density platform

Simplified growth migration path

core1

core2

core

core

Core Devices

Core VDCs
Aggregation Devices

LTRCRT-2605

agg
agg1

agg2

agg3

agg4

acc1

acc2

accN

accY

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

agg

Aggregation VDCs

acc1

57

acc2

accN

accY

The Default VDC

L2 Protocols
VLAN Mgr
VLAN Mgr
LACP
IGMP

UDLD
UDLD
CTS
802.1x

MAC Table

L3 Protocols
OSPF
BGP
EIGRP
PIM

VDC1 is the default VDC for Sup1, This is

replaced with the Admin VDC on Sup2/2E

GLBP
HSRP
VRRP

The default VDC has several Unique features:

Has all ports are assigned initially assigned

SNMP

Enabled when the system is activated

RIB

Cannot be deleted

Protocol Stack (IPv4 / IPv6 / L2)

Default
VDC

Is responsible for other VDC administration

not necessarily configuration

VDC X

Handles all Software Installation

Infrastructure
Linux 2.6 Kernel
Physical Switch
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Controls Systemwide parameters such as

Licensing, VDC Resources, CoPP, NTP

Cisco Public

58

VDC Fault Domain

Each VDC is a separate fault domain
A process crashes in any VDC
Processes in the other VDCs are not affected and continue to run unimpeded

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

59

VDC Administration
Super User

VDC Administrator

VDC User
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

60

Nexus 7x00 Features

61

Feature Overview & Terminology

Intelligent L2 Domains POD Evolution
Inter-POD Connectivity across L3
(Failure Boundary Preservation)
Failure
Boundary

IP Cloud

Core

L3

Aggregation

L3

vPC

L2

L2MP

Access

L2

vPC

vPC

Servers

STP+

vPC
NIC Teaming

16x ECMP

Simplified loop-free trees

Low Latency / Lossless

2x Multi-pathing

MAC Scaling

STP Enhancements

Bridge Assurance

Cisco FapricPath (L2MP)

Operational Flexibility
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

62

vPC and VSS Comparison

Virtual Port Channel (vPC) is a version of VSS for the Nexus.
vPC

VSS

(Virtual Port Channels)

(Virtual Switching System)

Multi-Chassis Port Channel

Yes

Yes

Loop-free Topology
(no blocking ports)

Yes

Yes

STP as a fail-safe protocol only

Yes

Yes

Two Independent Nodes, both

active

Single Logical Node

Intra-chassis

Inter-chassis

Instances per Node

Single instance

Separate Configs
(w/ consistency checker)

Combined Configs

Yes

12.2(33)SXI

32 Port 10GE Module

PFC3C mode, Sup 70 10G,

6708, 6716

Switch Control Plane

Switch Redundancy (sup failover)

Control Plane Protocols

Switch Configuration

Maximum Physical Nodes

ISSU Support
Inter-switch Link Hardware

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

vPC Features
Allow a single device to use a port
channel across two upstream switches
Eliminate STP blocked ports
Uses all available uplink bandwidth
Dual-homed server operate in activeactive mode
Provide fast convergence upon
link/device failure

vPC

Reduce CAPEX and OPEX

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

64

vPC Terminology
A virtual port channel (vPC) allows multiple links that are physically
connected to two different Cisco Nexus 7000 Series to appear as a single
port channel to a third device.
vPC Peer
Keepalive Link

Layer 3
Cloud
vPC Domain

vPC
Peer

Peer
Link

Orphan
Port

CFS
vPC Member
Port
Normal
Port Channel

vPC
Orphan
Device
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

65

Double-Sided vPC
vPC is supported on both the Cisco Nexus
5000 and Cisco Nexus 7000 Series
Switches.

vPC can be deployed in multiple layers of

the data center simultaneously:
Server to access
Access to aggregation

Double-sided vPC enables a unique 16-way

port channel:
Can be scaled to 32-way port channels with
F-series modules

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

vPC
Domain 1

Max 16 Ports
vPC
Domain 2

Introducing Cisco FabricPath

FabricPath brings Layer 3 routing benefits to flexible Layer 2
bridged Ethernet networks.
Switching
Routing
Easy Configuration
Plug & Play
Provisioning Flexibility

Multi-pathing (ECMP)
Fast Convergence
Highly Scalable

FabricPath

Control plane: IS-IS

Load balancing: ECMP and multi-topology
Frame format: MAC-in-MAC
Interoperability with Classical Ethernet
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

67

FabricPath Operation
Control Plane:
L2 IS-IS is running in the L2MP Core network no STP
Data Plane:
L2MP Core: Ethernet frames are encapsulated with MAC-in-MAC
(MiM) header and forwarded based on switch table derived from L2
IS-IS
S1

S2

L2

L1
Switch

S42

L5

S3
L3
L6

A
C

S11

L9
L8
L2MP Fabric

S12

IF

L12

L4

L7

L10

IF

L1, L2, L3,

L4
MAC
IF

Switch

S42
S4

L11

L12
AC
S42

MAC
MA

1/1
S42

1/1
A

C
C
A

3/1
B

Forwarding of Multicast is through distinct SPF Trees.

LTRCRT-2605

S11 S42

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

68

IF
IF
3/1
S11

FabricPath Port
CE Port

Loop Mitigation with FabricPath

Root

STP Domain

S1

Root

S2

TTL=2

TTL=1

L2 Fabric
S10

TTL=3
TTL=0

Block redundant paths to ensure loop-free

topology

TTL is part of FabricPath header

Frames loop indefinitely if STP failed

Frames are discarded when TTL=0

Could result in complete network meltdown as the result of flooding

RPF check for multicast based on tree

info

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Decrement by 1 at each hop

69

Cisco Nexus 7000 OTV Topology

Ethernet traffic between sites is tunneled in IP Packets
Allows simple Ethernet connectivity across an IP network
Provides simplicity of Ethernet with the feature rich characteristics of IP
IP Packet

Ethernet Traffic

Ethernet Traffic

Ethernet Traffic

Encap

Decap
OTV

OTV

IP A

Server 1

LTRCRT-2605

IP B

Communication between
Server 1(site 1) and Server 2(site 2)
2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

70

Server 2

Nexus 7x00 Switch Configuration

71

Configuration Steps: Switch Mode

1. Configure basic connectivity and administrative access
2. Provision VDCs
3. Configure Ethernet interface
4. Configure IP routing protocols
5. Validate interface configurations
6. Validate routing configuration

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Configuring Basic Connectivity and

Administrative Access

73

Initial Switch Configuration

Do you want to enforce secure password standard (yes/no): yes
Enter the password for "admin": 1234Qwer

Confirm the password for "admin":1234Qwer

You will be prompted for secure password configuration on a Nexus 7000 switch with no previous
configuration
If a password is weak (short, easy-to-decipher), your password configuration is rejected.
Passwords are case-sensitive.
Password must be at least 8 characters with a mix of letters, numbers and capitals
Must not contain dollar signs ($) or spaces anywhere in the password.
Cannot include quotation marks (" or '),vertical bars (|), or right angle brackets (>). at the beginning
of the password

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

74

Basic System Configuration

---- Basic System Configuration Dialog VDC: V ---This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.
Please register Cisco Nexus7000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus7000 devices must be registered to receive
entitled support services.

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

to skip the remaining dialogs.
Would you like to enter the basic configuration dialog (yes/no): yes

The Nexus 7000 includes a CLI setup script

This script will appear in three cases:
Upon initial configuration of a new switch
After a write erase reload

Upon typing setup from the command line

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

75

Basic Manual Configuration

Assign the switch hostname and domain
Assign the switch IP address and gateway ( ensure in management vrf)
Define additional usernames if required with network-admin credentials
N7K-1 # conf
N7K-1(config)# hostname N7K-P
N7K-1(config)# ip domain-name pod1.com
N7K-1(config)# vrf context management
N7K-1(config-vrf)# ip route 0.0.0.0/0 10.1.1.1
N7K-1(config)# username tarzan password Jane123 role network-admin
N7K-1#(config)# show user-account
...

user:tarzan
this user account has no expiry date
roles:network-admin

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

76

Management Administrative Access

Ensure that you verify management access with ping from mgmt vrf
N7K-1# ping 10.x.1.1 vrf management (Where
PING 10.1.1.1 (10.1.1.1): 56 data bytes
Request 0 timed out
64 bytes from 10.1.1.1: icmp_seq=1 ttl=254
64 bytes from 10.1.1.1: icmp_seq=2 ttl=254
64 bytes from 10.1.1.1: icmp_seq=3 ttl=254

x is your pod number.)

time=1.28 ms
time=1.197 ms
time=1.094 ms

Configure RSA keys and enable the SSH server process

Verify that the SSH server is running
N7K-1(config)# ssh key rsa 1024 force
deleting old rsa key.....
generating rsa key(1024 bits).....
generated rsa key
N7K-1(config)# show ssh server
ssh is enabled
version 2 enabledN7K-1(config)# feature ssh

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

77

Nexus VDC Configuration

78

VDC Configuration
Nondefault VDCs are created from within the default VDC global configuration context:
N7K-1(config)# vdc engineering
N7K-1(config-vdc)#
N7K-1(config-vdc)# show vdc
vdc_id vdc_name
------ -------1
N7K-1
2
engineering

state
----active
active

mac
---------00:22:55:79:1d:41
00:22:55:79:1d:42

Nondefault VDCs are removed from within the default VDC global configuration context:
N7K-1# config t
N7K-1(config)# no vdc engineering
Deleting this vdc will remove its config. Continue deleting this vdc? [no]
yes
Note: VDC deletion is a time consuming process, please wait until the
command completes

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

VDC Resource Assignment

N7K-1(config-vdc)# show run vdc
vdc N7K-7 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 96 maximum 96
limit-resource u6route-mem minimum 24 maximum 24
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
vdc engineering id 2
boot-order 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 8 maximum 8
limit-resource u6route-mem minimum 4 maximum 4
limit-resource m4route-mem minimum 8 maximum 8
limit-resource m6route-mem minimum 2 maximum 2
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

80

Configuring Resource Assignment

N7K-1(config)# vdc engineering
N7K-1(config-vdc)# limit-resource vlan minimum 32 maximum 100
N7K-1(config-vdc)# show run | begin vdc
vdc N7K-7 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 96 maximum 96
limit-resource u6route-mem minimum 24 maximum 24
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
vdc engineering id 2
boot-order 1
limit-resource vlan minimum 32 maximum 100
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
-More--

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

81

VDC Interface Allocation

Allocating a single Ethernet interface to a VDC:
N7K-1# config t
N7K-1(config)# vdc engineering
N7K-1(config-vdc)# allocate interface ethernet 2/47
Moving ports will cause all config associated to them in source vdc to
be removed. Are you sure you want to move the ports? [yes] yes

Allocating a range of Ethernet interfaces to a VDC:

N7K-1# config t
N7K-1(config)# vdc engineering
N7K-1(config-vdc)# allocate interface ethernet 2/1-2, e2/5
Moving ports will cause all config associated to them in source vdc to
be removed. Are you sure you want to move the ports? [yes] yes

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

82

Validating VDC Interfaces

Display VDC interface information from within the default VDC:
N7K-1# show vdc membership
vdc_id: 1 vdc_name: N7K-1
Ethernet2/1 Ethernet2/2
Ethernet2/7 Ethernet2/8
Ethernet2/13 Ethernet2/14
Ethernet2/19 Ethernet2/20
Ethernet2/25 Ethernet2/26
Ethernet2/31 Ethernet2/32
Ethernet2/37 Ethernet2/38
Ethernet2/43 Ethernet2/44

interfaces:
Ethernet2/3
Ethernet2/9
Ethernet2/15
Ethernet2/21
Ethernet2/27
Ethernet2/33
Ethernet2/39
Ethernet2/45

Ethernet2/4
Ethernet2/10
Ethernet2/16
Ethernet2/22
Ethernet2/28
Ethernet2/34
Ethernet2/40
Ethernet2/48

vdc_id: 2 vdc_name: engineering interfaces:

Ethernet2/47

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

83

Ethernet2/5
Ethernet2/11
Ethernet2/17
Ethernet2/23
Ethernet2/29
Ethernet2/35
Ethernet2/41

Ethernet2/6
Ethernet2/12
Ethernet2/18
Ethernet2/24
Ethernet2/30
Ethernet2/36
Ethernet2/42

VDC Navigation
Navigating between the default and nondefault VDCs:
N7K-1# switchto vdc engineering
TAC support: http://www.cisco.com/tac Copyright (c) 2002-2008, Cisco Systems, Inc.
All rights reserved. The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under license. Certain
components of this software are licensed under the GNU General Public License (GPL)
version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of
each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
N7K-1-engineering#

Switch from a nondefault VDC back to default VDC:

N7K-1-engineering# switchback

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

VDC Running Configuration

Copy the running configuration for all VDCs on the physical device to the startup
configuration:
N7K-1# copy running-config startup-config vdc-all

Display the running configurations for all VDCs:

N7K-1# show running-config vdc-all
!Running config for default vdc: N7K-7
!Command: show running-config
!Time: Mon Aug 2 03:30:42 2010
version 5.0(3)
license grace-period
feature telnet
username admin password 5 $1$pjCtSd9F$FLCKjyWF9c74BBAhUXOkr.
admin
--Remaining output omitted-LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

85

role network-

Nexus 7x00 Interface Configuration

86

CLI L2 Interface Configuration

N7K-1(config)# interface eth1/1-3
N7K-1(config-if-range)# switchport
N7K-1(config-if-range)# no shut
N7K-1(config-if)# interface eth1/4,e1/7-8
N7K-1(config-if-range)# switchport
N7K-1(config-if-range)# switchport mode trunk
N7K-1(config-if-range)# swichport trunk allowed vlan 10,20

All Cisco Nexus 7000 interfaces are designated

interface ethernet slot/port.

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

87

CLI Slash Notation

N7K-1(config)# interface e1/1
N7K-1(config-if)# no switchport

N7K-1(config-if)# ip address 10.1.23.1/24

N7K-1(config-if)# ipv6 add ::abcd:223/120
N7K-1(config)# ip access-list test
N7K-1(config-acl)# permit ip 10.1.1.0/24 any

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

88

Nexus 7000 vPC Configuration

89

vPC Configuration
Step 1: Enable feature vPC and LACP
Step 2: Configure interface that you want to be peer link to be a port channel
Step 3: Create the vPC domain
N7K-1(config)# feature vpc

N7K-1(config)# feature lacp

N7K-1(config)# interface ethernet 7/1, e8/1
N7K-1(config-if)# switchport mode trunk

N7K-1(config-if)# channel-group 20 mode active

N7K-1(config-if)# exit
N7K-1(config)# vpc domain 1
N7K-1(config-vpc-domain)# peer-keepalive source 10.2.2.1 destination 10.2.2.2
vrf keepalive
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

90

vPC Configuration (Cont.)

Step 4: Configure the vPC peer link
Step 5: Configure interface that connects to the vPC device as Layer 2 LACP
port channel
Step 6: Add the port channels that connect to the downstream device to the vPC
N7K-1(config)# interface port-channel 20
N7K-1(config-if)# vpc peer-link
N7K-1(config-if)# exit
N7K-1(config)# interface e3/1
N7K-1(config-if)# channel-group 50 mode active
N7K-1(config-if)# exit
N7K-1(config)# interface port-channel 50
N7K-1(config-if)# description Link To Access
N7K-1(config-if)# vpc 50
N7K-1(config-if)# switchport
N7K-1(config-if)# switchport mode trunk
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

91

Verifying vPC Domain Status

To verify the status of the vPC peer relationship use the show vpc brief
command:
N7K-1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id
:
Peer status
:
vPC keep-alive status
:
Configuration consistency status:
Type-2 consistency reason
:
vPC role
:
Number of vPCs configured
:
Peer Gateway
:
Dual-active excluded VLANs
:

10
peer adjacency formed ok
peer is alive
success
Consistency Check Not Performed
primary
0
Disabled
-

vPC Peer-link status

--------------------------------------------------------------------id
Port
Status Active vlans
---------- -------------------------------------------------1
Po20
up
100-105
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

92

Verifying vPC Consistency

To check for potential vPC configuration consistency problems use the show vpc
consistency-parameters command:
N7K-1# show vpc consistency-parameters vpc
Legend:
Type 1 : vPC will be suspended in case of mismatch
Name
Type Local Value
Peer Value
---------------- ---------------------- ----------------STP Port Type
1
Default
Default
STP Port Guard
1
None
None
STP MST Simulate PVST
1
Default
Default
lag-id
1
[(7f9b,
[(7f9b,
0-23-4-ee-be-a, 8007, 0-23-4-ee-be-a, 8007,
0, 0), (8000,
0, 0), (8000,
0-5-9b-1f-89-fc, 0, 0, 0-5-9b-1f-89-fc, 0,0,
0)]
0)]
mode
1
active
active
Speed
1
10 Gb/s
10 Gb/s
Duplex
1
full
full
Port Mode
1
trunk
trunk
Native Vlan
1
1
1
MTU
1
1500
1500
Allowed VLANs
1-3967,4048-4093
1-3967,4048-4093
Local suspended VLANs
1,10
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

93

Summary

LTRCRT-2605

Data Center Aggregation and Core layers typically consist of highly available
redundantly interconnected switches providing advanced services.
The Cisco Nexus 7000 is typically deployed either as a Core switch or as an
Aggregation switch, in parallel with Catalyst Series Switches.
The Cisco Nexus 7000 integrated core provides high density 10 GE ports
alongside Catalyst series services.
When the VDC is created, a default resource allocation is made for this VDC.
Networks and interfaces are configured on a interface basis per VDC from
interface configuration mode.

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

94

Lab

Continue Your Education

Demos in the Cisco Campus
Walk-in Self-Paced Labs
Table Topics

Meet the Engineer 1:1 meetings

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

96

DC/V Related Official Cisco Training Offerings

Course

Description

Cisco Certification

Implement Cisco Data Center

Unified Fabric/Unified Computing
(DCUFI and DCUCI)

Learn how to deploy complex virtualized Data Center Fabric

and Computing environments with Nexus and UCS families

CCNP Data Center

Cisco Data Center CCIE Unified

Fabric/Computing Workshop
(DCXUF and DCXUC)

Prepare for your CCIE Data Center Practical exam with

hands on lab exercises running on a dedicated
comprehensive topology

CCIE Data Center

Configuring Cisco MDS 9000

switches

Deep dive into the MDS Storage Networking Product family:

hands on lab exercises, newest features and platforms
explained

Introducing Cisco Data Center

Networking and Technologies
(DCICN and DCICT)

Start your career in Data Center learning all the different

technologies contributing to build a Data Center
infrastructure

CCNA Data Center

For more details please visit : http://learningnetwork.cisco.com

Questions: Visit the Learning@Cisco Booth
LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

97

Participate in the My Favorite Speaker Contest

Promote Your Favorite Speaker and You Could be a Winner
Promote your favorite speaker through Twitter and you could win $200 of Cisco
Press products (@CiscoPress)
Send a tweet and include
Your favorite speakers Twitter handle <Speaker enter your twitter handle here>
Two hashtags: #CLUS #MyFavoriteSpeaker

You can submit an entry for more than one of your favorite speakers
Dont forget to follow @CiscoLive and @CiscoPress
View the official rules at http://bit.ly/CLUSwin

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

98

Complete Your Online Session Evaluation

Give us your feedback and you
could win fabulous prizes. Winners
announced daily.
Complete your session evaluation
through the Cisco Live mobile app
or visit one of the interactive kiosks
located throughout the convention
center.
Dont forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

99

Continue Your Education

Demos
Labs
Lunch

Topics
Final copy TBD

LTRCRT-2605

2014 Cisco and/or its affiliates. All rights reserved.

Cisco Public

100