FCoE Design and Configuration

FCoE Design and Configuration on Nexus 5596

 The vPC is intelligent enough to keep the FCoE VLANs isolated per FC storage best-practices (i.e. VLAN 11 on Fabric A, VLAN 12 on Fabric B), but allows the Data VLAN (i.e. VLAN 50) across the LACP port-channel.

Physical Architecture

FCoE VLAN Architecture

Data VLAN Architecture

Configuration

N5K-A:
vlan 11
fcoe vsan 11
name FCOE_VSAN11
!
interface port-channel3
switchport mode trunk
vpc 3
switchport trunk allowed vlan 50,11
!
interface Ethernet1/7
description File Server
switchport mode trunk
switchport trunk allowed vlan 50,11
spanning-tree port type edge trunk
channel-group 3 mode active
!
interface vfc1
bind interface port-channel3
no shutdown
!
vsan database
vsan 11 interface vfc1
!

N5K-B:
vlan 12
fcoe vsan 12
name FCOE_VSAN12
!
interface port-channel3
switchport mode trunk
vpc 3
switchport trunk allowed vlan 50,12
!
interface Ethernet1/7
description File Server
switchport mode trunk
switchport trunk allowed vlan 50,12
spanning-tree port type edge trunk
channel-group 3 mode active
!
interface vfc1
bind interface port-channel3
no shutdown
!
vsan database
vsan 12 interface vfc1
!

 

Networking on the Oracle VMs ( Ldoms)

Hardware used : Oracle blade chassis (6K), CNA, Brocade 5100, Cisco Nexus 5596, Oracle Pillar
OS on Sparc blades : Solaris 10, Solaris 11
OS on Windows Blades : ESXi 5.1

Configuring networking on Oracle VMs ( Solaris 11) using CNA with multiple VLANs

On Primary

#ldm set-vsw  vid=3 primary-vsw0
#dladm create-vlan -l net8 -v 3 net3008 ( net8 is primary-vsw)
#dladm create-vlan -l net10 -v 3 net3010
#ipadm create-ip net3008
#ipadm create-addr -T static -a 10.xx.xx.xx/24 net3008/v4address
 #ldm add-vnet  vid=3 vnet1 primary-vsw0 6k02bl09vm01

On guest ldom
dladm create-vlan -l net0 -v 3 net3000
ipadm create-ip net3000
ipadm create-addr -T static -a <ip>/24 net3000/v4address