Nexus1000v for VMware Integration with Virtual Connect

This presentation highlights how to configure Nexus1000v with Virtual Connect. The presentation can be downloaded here.

Nexus1000v VSM deployment design for Virtual Connect is the same as general Cisco recommendation. However, two points need to be pointed out related with Virtual Connect environment. (It’d be much better to review presentation first before watching videos below to understand configuration context.)

1) Adding VEM on ESXi host connecting with Virtual Connect

2) Configuring MAC-pinning host port-channel for VEM uplinks

Configuring Cisco B22 FEX module with HP Proliant Blade Server

For HP Proliant server customers using Cisco B22 FEX modules, this presentation can help you with understanding how to design and configure B22 for ethernet and FCOE connectivity. The presentation can be downloaded here.


HP Proliant Server with Emulex CAN PXE and FCOE Configuration Without Virtual Connect(Using Cisco B22 FEX module etc)

It’s common that server admins need to set up PXE and FCOE configuration for HP Proliant servers. For HP blade server with Virtual Connect modules, users can configure all PXE and FCOE settings in either OneView or classic Virtual Connect Manager GUI.

However, some customers may need to use other switches other than HP Virtual Connect for HP Proliant server network connectivity. This includes Proliant blade servers using Cisco B22 FEX module or other blade switches as well as Proliant rack servers. Meanwhile, many HP Proliant servers these days are shipped with Emulex CNAs as LOM or FlexLOM.

It can be tricky to set up PXE and FCOE without Virtual Connect profile concept because users need to enter server RBSU and Emulex ROM to configure relevant settings.

This presentation is to help users with server RBSU and Emulex ROM PXE and FCOE configuration. At the end of the presentation, there is also a section of Emulex BFS booting messages where you can see the common booting errors and related causes. This actually will even help you to troubleshoot when you are using Virtual Connect module.

Hope it helps your deployment. The presentation can be downloaded here.


HP Announce OneView for Infrastructure Management

HP’s new infrastructure management OneView is announced today. This is a very big announcement for HP.

It’s the next-generation HP management software to combine management of HP Proliant blade and rack servers. In near future, it’ll also provide HP storage(3PAR) and networking management.

In version 1.0, here are some key technical facts

  • The management controller will be available to download as virtual appliance format in early Q4 2013
  • The virtual appliance can manage up to 640 servers(or 40 c7000 enclosures)
  • Rackmount Gen8 and Blade Gen8/G7 servers are supported but blade Gen8 will have full features supported while some of OneView features won’t be available on G7 blade servers.  Some features doesn’t apply to rack Gen8 server in this version because rack servers don’t have virtual connect component(which is the key component to enable service mobility).
  • Virtual Connect Flex-10/10D, Flexfabric, Flex-10 are supported. Virtual Connect Fibre channel module will be supported in future release. VC-1/10 module will NOT be supported.
  • c7000 enclosure is supported. Not c3000 enclosure.

You can watch quick product demos and OneView marketing page here. There are 4 good technical demos after you click “Get An HP OneView Demo”.

Also, this is a video link showing how OneView is different than Cisco UCS in the way it operates.

Virtual Connect LACP Long Timeout Option in VC4.01 Release

One of VC 4.01 features is an option to let users to configure LACP timeout to Long Timeout(30sec instead of default short timeout 1 sec). Sometimes data center switches would like their LACP neighbors to use Long Timeout and this feature will meet such requirement. One of the use cases is to help Cisco Nexus customers who wish to perform ISSU. One of ISSU requirements is all LACP timers need to be set as Long Timeout.

Before VC4.01, Nexus customers will see this on Nexus side when connecting with Virtual Connect.

N5K-1# show lldp neigh
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
Device ID Local Intf Hold-time Capability Port ID
ProCurveSwitch2900-24Gmgmt0 120 BR 14
N5K-2 Eth1/1 120 B Eth1/1
N5K-2 Eth1/2 120 B Eth1/2
Enc-FF-VC-1 SERIAL NO:TW201300H8 BAY:1Eth1/3 120 X1
Enc-FF-VC-1 SERIAL NO:TW201300H8 BAY:1Eth1/4 120 X2

N5K-1# show lacp interface e1/3 Interface Ethernet1/3 is up Channel group is 1 port channel is Po1


Neighbor: 0x11 MAC Address= 0-24-81-f7-c6-f7 System Identifier=0x1,0-24-81-f7-c6-f7 Port Identifier=0x1,0x11 Operational key=2 LACP_Activity=active LACP_Timeout=short Timeout (1s) Synchronization=IN_SYNC Collecting=true Distributing=true Partner Admin State=(Ac-0:To-1:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0) Partner Oper State=(Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0) N5K-1#

N5K-1# show lacp issu-impact
For ISSU to Proceed, Check the following:
1. The port-channel member port should be in a steady state.
2. Check if any of the port-channel member has fast timer enabled from the remote peer
The following ports are not ISSU ready
Eth1/3 , Eth1/4 ,

With 4.01, VC has the long timeout option now.


and this is how Nexus end should look like after VC set his end to do LACP Long Timeout.

N5K-1# show lacp interface e1/3 <snip> Neighbor: 0x11 MAC Address= 0-24-81-f7-c6-f7 System Identifier=0x1,0-24-81-f7-c6-f7 Port Identifier=0x1,0x11 Operational key=2 LACP_Activity=active LACP_Timeout=Long Timeout (30s) Synchronization=IN_SYNC Collecting=true Distributing=true Partner Admin State=(Ac-0:To-1:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0) Partner Oper State=(Ac-1:To-0:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0)

N5K-1# show lacp issu-impact
For ISSU to Proceed, Check the following:
1. The port-channel member port should be in a steady state.
2. Check if any of the port-channel member has fast timer enabled from the remote peer
All the port are ISSU ready


Please Note:

1. If you still see Nexus show VC neighbor as “Short_Timeout” even after you set VC as Long, try to set VC back to Short and apply and then try to set VC back to Long again and apply to see if it can negotiate successfully.

2. Resetting LACP timer might have short traffic interruption for your application. I lost one ping during LACP timer changes on VC side while other times I didn’t lose any pings when doing the same LACP timer changes.

Virtual Connect Initial Configuration Video Tutorial

I still get questions from time to time in regard to Virtual Connect initial configuration so I made this video. Hopefully it can help users to get VC up quickly. In the video, I also explained some common VC terms like Shared Uplink Set, Multiple Networks, native/untagged network, Smartlink etc.

Virtual Connect IfInDiscards and IfOutDiscards Counters

Two of Virtual Connect interface counters: IfInDiscards and IfOutDiscards, are often seen as the indication of traffic packet drop issues. Actually, they are normal behavior inside VC operation. Please take a look at this VC advisory.


The intent of this Customer Notice is to explain how rfc1213_IfOutDiscards and rfc1213_IfInDiscards function in a Virtual Connect environment. There are several normal traffic switching behaviors in Virtual Connect that will result in an accumulation of these Discard counters. This document describes the counter’s behaviors. The rfc1213_IfOutDiscards and rfc1213_IfInDiscards counter’s behavior is normal and harmless to the customer environment. The counters have no adverse affect on the performance of the Virtual Connect environment.


The VC advisory collection link is as below. I use Flexfabric as example as most of VC problems are not individual model number dependent.


Another useful link is Emulex CNA advisory link.

If you are using Proliant blade server G7 and above, the good chance is that you are using Emulex CNA(Intel G7 has NC553i as built-in LOM, AMD G7 has NC551i as built-in LOM, NC553m and NC551m are just Mezz card form factor for G7 with same architecture as “i”(built-in); Gen8 will have NC554 as FlexLOM, which uses same ASIC as NC553, just different form factor).  In many cases, the traffic issues are caused by end-point NIC/CNAs instead of Virtual Connect so it’s very important that you have up to date NIC/CNA firmware/driver.

Virtual Connect 4.01 Release New Features Demo

I just updated my Virtual Connect Technical Overview deck to reflect some new features releases in Virtual Connect 4.0.1. They include:

  • Min/Max BW Optimization
  • Dual-Hop FCOE to Nexus5500/5000
  • QOS

If you have viewed this deck before and would like to directly jump to 4.0.1 new feature section, it starts from slide 67.

I also created a youtube video showing the above three features. Hopefully it can help you with better understanding these features.

I also updated my “VC Bookmark” tab with latest 4.01 links and added a “Download” tab for VC 4.01 image download(too difficult to find on web).

How to View All Profile Configuration Summary and Cut/Paste fields like WWN

I’m not sure how many people know this. If you want to quickly view the configuration for all Virtual Connect server profiles in a list format, there is a small GUI button for it. It’s there but not very obvious.

If you look at top right corner of “Server Profiles” page, you should see a button like “printer” sign.


If you click it, it will pop up a separate window showing config for all server profiles and you can cut/paste any field like WWN/MAC, which is something you can’t do from a profile edit view.


You can even filter by specific profile and it will switch promptly to the configuration for that particular profile.


Feel free to let me know anything else you like or don’t like for VC Manager GUI or CLI. I can aggregate feedbacks back to VC engineering.

Virtual Connect 3.70 Release New Features

VC3.70 was just released and it’s a feature release, which means that it includes more features compared with VC3.60 and VC3.51 where both of them are primarily bug-fix releases with VC3.51 starts to support Gen8 server(BL460Gen8).

One of the major features in VC3.70 is Virtual Connect 3PAR storage array direct attach. Now you can direct attach 3PAR storage array directly with VC FlexFabric module without any SAN switching layer. I created a presentation to show how to config 3PAR and VC FlexFabric for this feature. Please download the deck Here.

Another feature in VC3.70 is to support newly release Virtual Connect Flex-10/10D module. Comparing this module with original Flex-10, this module provides more uplink ports(10) and also 4 internal 10G stack links. I don’t see you’ll use too much of internal 40G stacking bandwidth, especially if you use VC Active/Active design, where your user traffic will go directly from downlink to uplink and vice versa without using stacking link. The best thing is that you can now use all 10 uplink ports without worrying about original Flex-10 X7/X8 shared stacking link. You can check the module spec from the quick link in my VC bookmark page at the top of the web page.

Also, Flex-10/10D module is planned to provide FCOE uplink(Not FC uplink) and FCOE downlink in the future.

For more information on this new module and 3.70 new features, please check my Virtual Connect Technical Overview Presentation. I just updated it with the new contents.


Get every new post delivered to your Inbox.

Join 74 other followers