Some Networking Stuff

vmxnet — a paravirtualized device that works only if VMware Tools is installed in the guest operating system. A paravirtualized device is one designed with specific awareness that it is running in a virtualized environment. The vmxnet adapter is designed for high performance.

• vswif — a paravirtualized device similar to vmxnet that is used only by the ESX Server service console.
• vmknic — a virtual device in the VMkernel, the software layer that manages most of the physical resources on the ESX Server host. The vmknic is used by the TCP/IP stack that services VMotion, NFS and software iSCSI clients that run at the VMkernel level, and remote console traffic.

There is no way to interconnect multiple vSwitches

vSwitches cannot share physical ethernet adapters

Each virtual switch can have up to 1,016 virtual ports, with a limit of 4,096 ports on all virtual switches on a host.

You can create a maximum of 512 port groups on a single host.

You can think of port groups as templates for creating virtual ports with particular sets of specifications.

REFERENCE: http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

ESX TIPS

Maximum amount of memory you can assign to service console is 800MB, 1600 MB is should be the

swap file size. VCB is license per host and not per processor as are VC,ESX,DRS,HA It is important to

note that virtual center uses a heartbeat per 5 min to check if license is up and changes are made to licenses.

HA Services running on the hosts itself and are NOT part of virtual center(VC), they are just configured using VC .

However DRS is controlled and managed by Virtual Center. DRS is driven by default interval of 5 min or when host

is added/moved from the cluster. So ideally DRS queries cluster 12 times in a hour. Based on this it prioritize series

of recommendations to level load across the cluster.

 

After logging level,Virtual Machine is the major factor in deciding the Virtual center database. Beware changing your logging level wipes out all your previous logged data.

---

One of the physical NICs will be allocated exclusively to the ESX console. It is through this ESX Server Service Console NIC that all connections to the ESX node are made, as well as SCP, SSH, or any other tool to access the ESX Server’s file system.

vmxnet virtual NICs can be utilized only after you install the VMware tools onto your virtual machine. When you replace Vlance NIC type to vmxnet NIC, you would have to re-ip the Virtual Machine. 

---

What is trunk port ?

It is port configured to carry traffic from multiple VLAN’s. This means every packet flowing through this port is tagged and other end(typically switch) knows where this packet belongs to. So in order to enable VLAN’s in Port which connects to NIC on ESX should be trunked(802.1q VLAN Trunk). Never forget to assign Native VLAN. Anything on Native VLAN is not tagged and whichever packet is not tagged it is not seen by Guest on ESX. And also configure port to allow other VLAN on it, this makes sure that all VLAN’s can been seen by the port.

---

What happens when you do “Service network restart” ?

It restarts eth0, service console nic and lo which is loopback adapter.

---

 

VMWare Tools, VLANS

Few good articles i came across and found rich in knowledge, would like to share with you all

Configuring VLANs in VMware Infrastructure 3 (VI3)

 

How to Install VMWare Tools Through Windows Group Policy

Advance concepts

WORLD SWITCH:

The process by which one VM is unscheduled and another scheduled to execute is known as a world switch. This process involves capturing one VM’s processor registers and writing these registers to memory, and reading the registers for the other VM from main memory and, finally, writing these registers to the processor.

Beacon monitoring allows ESX Server to test the links in a bond by sending a packet from one adapter to the other adapters within a virtual switch across the physical links.

AMD PCNet PCI Ethernet adapter (vlance):. This device is used as the default because of its near-universal compatibility – there are DOS drivers for this adapter, as well as Linux, Netware, and all versions of Windows. However the virtual adapter reports link speeds of 10Mbps with only a half-duplex interface,

Vmxnet adapter: If the vlance adapter is not delivering acceptable throughput or if the physical host is suffering from excessive CPU utilization, higher throughput may be possible by changing to the vmxnet adapter, which is a highly-tuned virtual network adapter for VMs.

To handle multiple source MAC addresses, the physical network interface of the ESX server is put into promiscuous mode. This causes its physical MAC address to be masked; all packets transmitted on the network segment are presented to the VMkernel virtual switch interface. Any packets destined for a VM are forwarded to the virtual network adapter through the virtual switch interface. Packets not destined for a VM are immediately discarded.

In virtualization port-based tagging at the physical switch does not provide VLAN isolation between VMs that share the same physical network connection. To address the scenario where broadcast-domain isolation is required between two VMs sharing the same physical network, virtual switches support the creation of port groups that can provide VLAN tagging isolation between VMs within the confines of a virtual switch. Each port group is identified by a network label, which is unique to the current host, and can optionally have a VLAN tagging ID.

When an application within the VM issues a file read or write request to the operating system, the operating system performs a file-to-block conversion and passes the request to the driver. However, the driver in an ESX Server environment does not “talk“ directly to the hardware; instead, the driver passes the block read/write request to the VMkernel where the physical device driver resides and then the read/write request is forwarded to the actual physical hardware device and forwarded to the storage controller.

Unlike Windows and Linux operating systems, ESX Server does not lock a LUN when it is mounted. VMFS is inherently a distributed file system, allowing more than one ESX Server to view the same LUN. This means that, while numerous ESX Server instances may view the contents of a VMFS LUN, only one ESX Server may open a file at any given moment. To an ESX Server and VMFS, when a VM is powered on, the VM disk file is locked.

Networking – VMWARE

If you are using VLAN, in the VLAN ID field, enter a number between 1 and 4094. If you use VLAN ID 4095, port group would see traffic on any VLAN. Blade servers have limited number of NIC, it will be likely be necessary to use VLANs to separate traffic for SVC Console,Vmotion,IP Storage and various group of VMs.

IP Storage refers to any form of storage that uses TCP/IP to access SCSI devices

VMKernel TCP/IP networking stack has been extended to include

iSCSI & NFS (Virtual Machine Datastore,ISO files)

and Vmotion

Service Console and VMKernel Services have completely separate TCP/IP Stacks.

When only one service console connection is present, changing service console configuration is not allowed. Using DHCP for SVC Console, DNS server should be able to map SVC Console’s hostname to dynamically generated IP address. Otherwise you can use IP Address which again might change if lease expires, DHCP is supported only when virtual interface is configured and attached to the network where DHCP server resides.

iSCSI also has service console components, so networks that are used to access targets should be reachable by both Service console and VMKernel TCP/IP Stacks. As result, after you create VMKernel port for iSCSI you must create a SVC Console connection on the same vSwitch as the Vmkernel port.

When you select “Use this port group for VMotion” you broadcasting other ESX server to send Vmotion traffic on this network connection.

Uplink adapters default speed is Autonegotiate.

Layer 2 Security policy are promiscuous mode (by default Reject), MAC address change and forged transmits.Layer is data link layer.

Traffic shaping policies are set to each virtual adapter attached to the port group and not the vSwitch as a whole

Load balancing and failover policy allow you to determine how network traffic is distributed between adapters and how to re-route in case of failure of NIC. Outgoing traffic is controlled by this policy and incoming traffic by physical switch.

Use port group with different sets of active adapters in their teaming policy to separate VM into groups. These can use separate adapters as long as all adapters are up.

esxcfg-vswif -l

Provides a list of the service console’s current network interfaces.

Check that vswif0 is present and that the current IP address and Netmask are

correct.

esxcfg-vswitch -l

Provides a list of current virtual switch configurations.

Check that the uplink adapter configured for the service console is connected to the

appropriate physical network.

exscfg-nics -l

Provides a list of current network adapters along with their names

Check that the uplink adapter configured for the service console is up and that the

speed and duplex are both correct.

esxcfg-nics -s <speed> <nic>

Changes the speed of a network adapter.

esxcfg-nics -d <duplex> <nic>

Changes the duplex of a network adapter.

esxcfg-vswif -i <new ip address> vswifX

Changes the service console’s IP address.

esxcfg-vswif -n <new netmask> vswifX

Changes the service console’s netmask.

esxcfg-vswitch -U <old vmnic> <service console vswitch>

Removes the NIC for the service console

esxcfg-vswitch -L <new vmnic> <service console vswitch>

Changes the uplink for the service console.

If you encounter long waits when using esxcfg-* commands, it is possible that DNS is misconfigured.

Connection from Virtual network adapters to port group is made by name, any change in the name would cause loss of connection when VMs are rebooted. It won’t affect already running VMs. Best practise is to avoid renaming networks after they are in use.

STORAGE-Advance Concepts

For preparing VCP you first need to read the Exam Blue print available on vmware site, after going through it you would realize that one should go through

1. Basic Administration Guide
2. Server configuration Guide
3. Resource Administration

All the above guides and additional guides are available at Vi3 Documents in PDF

Below are the contents from all three guide, they are actually few important concepts rather than entire text. This blog talks about storage.

STORAGE

TYPES OF STORAGE

• Local
• Fibre Channel (FC)
• ISCSI (Hardware iniatiated)
• ISCSI (software iniatiated)
• NFS (NFS client is built-in into ESX server)

iSCSI

With iSCSI, SCSI storage cmds are send by VM to its VMDKs & are converted into TCP/IP protocol packets and transmitted to a remote device or target, that stores the virtual disk. ISCSI initiators are responsible for transporting SCSI requests between ESX Server and the target storage device on the IP Network.

There are two types of ISCSI initiators

1. Software based

2. Hardware based

Software based iSCSI initiators have a code built into VMKernel which carries out the transporting job, using software initiators, the ESX server connects to a LAN through an existing NIC card using network stacks, in short you can implement iSCSI without purchasing specialized hardware. You also need to open a firewall port by enabling the iSCSI software client service.

Hardware based iSCSI initiators requires HBA cards which are specialized to transport iSCSI cmds over LAN to the target. Currently ESX Server supports only Qlogic QLA4010 iSCSI HBA.

NB: ESX 3.0 does not support both types of initiators on single system.

Naming requirements:

IQN (iSCSI qualified name)

e.g. iqn.1998-01.com.mycompany:myserver

Format Template: iqn.<year-mo>.<reversed_domain_name>:<unique_name>

Discovery methods

Initiator discovers iSCSI targets by sending a sendtargets requested for specific target address.

Static: Only available for Hardware based iSCSI initiators, you can manually add additional targets or remove unneeded targets. If you remove a dynamically discovery static target, the target can be returned to the list the next time a rescan happens, the HBA is reset, or the system is rebooted.

Dynamic: to use this method enter the address of the target device so that the initiator can established a discovery session with this target. The target device then responds by forwarding a list of additional targets that the initiator is allowed to access.

iSCSI Security

Since iSCSI communications between initiator and target happens over TCP/IP stack, it is necessary to ensure security of the connection. ESX server supports CHAP that iSCSI initiators can use for authentication purposes.

You can’t store VM on IDE or SATA, but on SCSI,NAS or FC storage only.

VMs communicate with datastore (where vmdk is placed) using SCSI commands, SCSI commands are encapsulated into various protocols e.g. FC,iSCSI, NFS depending type of physical storage.

HBA Naming convention vmhba1:1:3:1, Hba card 1, on Storage processor 1, using LUN3 and partition 1. First 2 numbers can change but last will remain unchanged

Select a large LUN if you plan to create multiple virtual machines on it., if more space is needed you can increase the VMS volume at any time –up to 64 TB.

Network Policy: NIC TEAMING

NIC Teaming policies, which includes load balancing and failover settings, allow you to determine how network traffic is distributed between adapters and how to re-route traffic in the event of adapter failure. Default NIC teaming policies are set for the entire virtual switch. And of course can be overridden at port group level.

To modify NIC teaming policies of a port group, click your ESX server’s configuration ->Networking ->Properties link next to virtual switch on which port group is allocated, select port group in the list of ports, Edit and under port group properties NIC Teaming

Load balancing Methods:

Route based on the originating port ID (default) [PortBased] : With this method, VM’s outbound traffic is mapped to specific PhyNIC based on the portID of Virtual Port, to which this virtual machine is connected.This method is simple and fast and doesn’t require VMKernel to examine the frame for necessary information.

Route based on IP hash [IPBased]:In this case NIC card is choosing based upon source and destination address of hte outbound packet. This method has slightly overhead and is not compatible with all switches. but traffic is distributed across NICS

Route based on source MAC hash: Each virtual machine outbound traffic is mapped to phyNIC based upon the VM’s NIC MAC address. It has low overhead and is compatible with all switches but may not spread traffic evenly.

Image can be found at http://techstarts.spaces.live.com/

Detecting and Handling failover

Network Failover Detection: Network failure is detected by VMkernel , which monitors Link state /+ beaconing. Monitoring link state will helps us detect cable pulls or switch failures.Beaconing, VMKernel sends out and listen probe packets on all NICS in team.

Notify Switches: Whenever failure occurs, virtual NIC’s traffic is routed to different NIC, notification is send to switch to modify routing table, in most cases it is desirable to avoid network latency after failovers and VMotion

Rolling failover : Determines how a physical network is returned to active duty after recovering from a failure. If rolling is set toNo, the adapter is returned to active duty immediately upon recovery.