Lets Design, Implement and do Administration of ESX3

Virtualization with VMWare Infrastructure 3.0

Musings from security guide –Part 02

Posted by Preetam on September 9, 2008

in ESX3i all logs are lost during reboot. Therefore a syslog server is required to record and archive all ESX Server 3i logs since a simple reboot will erase all activity of all users.

ISO Images consumes a lot of disk space since they are not compressed when they are created.

Users should create hash checksums on all ISO OS images on the ESX server before utilizing the ISO OS images for virtual machines.

Shares and Resource allocation: Minimum and Maximum resource settings within ESX Servers are absolute values, whereas shares are used to give preference to a guest OS when resource is scare. Minimum guarantee a specific amount of a resource to the virtual machine but deny that much of the resource to other virtual machine, While Maximum deny the virtual machine a portion of the resource while allowing other virtual machine more access to that resources. Do not configure the minimum virtual machine CPU and memory setting equal the total physical amount available. Use reservations,shares and limits to allocate resources.

Time management, synchronizing the virtual machine with the ESX Server is the preferred method for time synchronization.

Posted in Advance Concepts, Security, Virtual Center, VMWare | Leave a Comment »

Musings from security guide –Part 01

Posted by Preetam on September 9, 2008

  • The ESX server virtual switch port groups will be configured with any value between 2 and 4094. Utilizing VLAN1 will cause a denial of service since the ESX Server drops this traffic. The maximum port group that may be configured on a virtual switch is 512. Each port group is identified by a network label and a VLAN ID.

  • Ports Groups may have VLAN ID between  0 –4095.

  • VLAN ID 4095 specifies that the port group should use trunk mode or Virtual Guest Tagging (VGT) mode.

  • A value of Zero or blank VLAN ID is default value for External Switch Tagging(EST). EST is default configuration for all virtual switches within ESX Server. EST mode has 1-to-1 relationship, the number of VLAN’s are limited to the number of physical network adapter ports assigned to ESX.

  • Virtual Switch Tagging(VST) allows virtual switch to handle it’s own VLAN tagging. This processing is handled by Pnic and this overhead never comes to VMkernel. Each physical switch port that connects to virtual switch is configured in trunk mode. VLAN’s can span across multiple PSwitch. VLAN is enabled by trunked link connecting the virtual switch and PSwitch thru frame tags. Trunk links can carry the traffic of multiple VLANs simultaneously.Within Switch fabric, switches uses frame tagging to direct frames to the appropriate switch and port. Frame tagging assigns frame id prior to traversing trunked link. After the frame reaches the access link, VLAN ID is removed and the end device receives the frame.

  • Each Virtual Nic (VNic) has two MAC Address. effective and initial MAC address. Both the MAC address are same when they are first created.

  • Forged Transmits (set to accept by default): When effective MAC Address and initial MAC address are different, which means effective MAC address is always compared with initial MAC address.it is considered as forged transmits.

  • MAC Address Changes (set to accept by default): When effective MAC address is changed compared to initial one.

  • Promiscuous mode: When promiscuous mode is applied, all virtual machine connected to virtual switch have potential of reading all packets.

  • STP is not supported on vSwitch.Spanning Tree Protocol (STP) is either needs to disabled or Port fast needs to be enabled on PSwitch.

The vpxuser has privileges of a root user on the ESX server host,, but has no file privileges on the ESX server console. The vpxuser is created when the ESX server host is attached to Virtual Center. It is not present on the ESX Server host unless the host is being managed through VirtualCenter.

Virtual Center has two default roles defined, system roles and sample roles. System Roles are permanent and the permissions associated with these roles cannot be changed. All changes made to permissions of custom roles are effective immediately not requiring users to log off and log back in.

Posted in Advance Concepts, Security, Virtual Center, VMWare | Leave a Comment »

ESX,Storage and HBACMD’s -02

Posted by Preetam on September 1, 2008

[root@FirstESX hbanyware]# ./hbacmd PortStat 10:00:00:00:c9:4f:47:6f

Port Statistics for 10:00:00:00:c9:4f:47:6f

Secs Since Last Reset   :  700883
Exchange Count          :  5729859
Responder Exchange Count:  23
TX Seq Count            :  6842854
RX Seq Count            :  10893170
TX Frame Count          :  6845254
RX Frame Count          :  108920755
TX Word Count           :  223818240
RX Word Count           :  643851008
TX KB Count             :  874290
RX KB Count             :  2515043
LIP Count               :  1
NOS Count               :  N/A
Error Frame Count       :  0
Dumped Frame Count      :  N/A
Link Failure Count      :  0
Loss of Sync Count      :  46
Loss of Signal Count    :  1
Prim Seq Prot Err Count :  0
Invalid TX Word Count   :  235
Invalid RX Frame CRC Cnt:  0

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd ServerAttrib 10:00:00:00:c9:49:06:da

Server Attributes for 10:00:00:00:c9:49:06:da

Host Name       : FirstESX.vmzare.com
FW Resource Path: /usr/sbin/hbanyware/RMRepository/
DR Resource Path: /usr/sbin/hbanyware/RMRepository/
HBAnyware Server Version: 2.1a35

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd AllNodeInfo 10:00:00:00:c9:49:06:da

All Node Info for 10:00:00:00:c9:49:06:da

Node Type      : READY
FCP ID         : D2
SCSI Bus Number: 0
SCSI Target Num: 0
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9B
OS Device Name : /proc/scsi/lpfc/00,0

Node Type      : READY
FCP ID         : B5
SCSI Bus Number: 0
SCSI Target Num: 3
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9C
OS Device Name : /proc/scsi/lpfc/00,3

Node Type      : READY
FCP ID         : C6
SCSI Bus Number: 0
SCSI Target Num: 10
Node WWN       : 20:00:00:20:37:17:D4:8E
Port WWN       : 22:00:00:20:37:17:D4:8E
OS Device Name : /proc/scsi/lpfc/00,10

Node Type      : READY
FCP ID         : CD
SCSI Bus Number: 0
SCSI Target Num: 9
Node WWN       : 20:00:00:20:37:17:83:92
Port WWN       : 22:00:00:20:37:17:83:92
OS Device Name : /proc/scsi/lpfc/00,9

Node Type      : READY
FCP ID         : CC
SCSI Bus Number: 0
SCSI Target Num: 11
Node WWN       : 20:00:00:20:37:17:C5:14
Port WWN       : 22:00:00:20:37:17:C5:14
OS Device Name : /proc/scsi/lpfc/00,11

Node Type      : READY
FCP ID         : CB
SCSI Bus Number: 0
SCSI Target Num: 12
Node WWN       : 20:00:00:20:37:17:D0:E4
Port WWN       : 22:00:00:20:37:17:D0:E4
OS Device Name : /proc/scsi/lpfc/00,12

Node Type      : READY
FCP ID         : C9
SCSI Bus Number: 0
SCSI Target Num: 13
Node WWN       : 20:00:00:20:37:17:C6:11
Port WWN       : 22:00:00:20:37:17:C6:11
OS Device Name : /proc/scsi/lpfc/00,13

Node Type      : READY
FCP ID         : C7
SCSI Bus Number: 0
SCSI Target Num: 14
Node WWN       : 20:00:00:20:37:17:C2:3D
Port WWN       : 22:00:00:20:37:17:C2:3D
OS Device Name : /proc/scsi/lpfc/00,14

Node Type      : READY
FCP ID         : EF
SCSI Bus Number: 0
SCSI Target Num: 1
Node WWN       : 20:00:00:20:37:17:E5:F1
Port WWN       : 22:00:00:20:37:17:E5:F1
OS Device Name : /proc/scsi/lpfc/00,1

Node Type      : READY
FCP ID         : E8
SCSI Bus Number: 0
SCSI Target Num: 2
Node WWN       : 20:00:00:20:37:17:90:6F
Port WWN       : 22:00:00:20:37:17:90:6F
OS Device Name : /proc/scsi/lpfc/00,2

Node Type      : READY
FCP ID         : E4
SCSI Bus Number: 0
SCSI Target Num: 4
Node WWN       : 20:00:00:20:37:17:DA:7C
Port WWN       : 22:00:00:20:37:17:DA:7C
OS Device Name : /proc/scsi/lpfc/00,4

Node Type      : READY
FCP ID         : E2
SCSI Bus Number: 0
SCSI Target Num: 5
Node WWN       : 20:00:00:20:37:11:E8:CD
Port WWN       : 22:00:00:20:37:11:E8:CD
OS Device Name : /proc/scsi/lpfc/00,5

Node Type      : READY
FCP ID         : E1
SCSI Bus Number: 0
SCSI Target Num: 6
Node WWN       : 20:00:00:20:37:51:46:DD
Port WWN       : 22:00:00:20:37:51:46:DD
OS Device Name : /proc/scsi/lpfc/00,6

Node Type      : READY
FCP ID         : E0
SCSI Bus Number: 0
SCSI Target Num: 7
Node WWN       : 20:00:00:20:37:17:BB:A8
Port WWN       : 22:00:00:20:37:17:BB:A8
OS Device Name : /proc/scsi/lpfc/00,7

Node Type      : READY
FCP ID         : DC
SCSI Bus Number: 0
SCSI Target Num: 8
Node WWN       : 20:00:00:20:37:24:83:26
Port WWN       : 22:00:00:20:37:24:83:26
OS Device Name : /proc/scsi/lpfc/00,8

********************************************************************************************************

[root@FirstESX log]# cat /etc/logrotate.d/vmkernel
/var/log/vmkernel{
    create 0640 root patrol
    missingok
    nocompress         =======> change it to compress
   # keep a history over 3 years.
    monthly
    rotate 36
   #   max log size of 200k  (thus limiting total disk usage to under 8megs)
    size 200k          =======> change it to 2046
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

********************************************************************************************************
Check the nodes in a cluster

[root@FirstESX config]# cat vmware-sites
FULLTIME_SITES_TID 00000022
+ 1:8042,8042,8043 FirstESX    vmware #FT_Agent_Port=8045
+ 2:8042,8042,8043 SecondESX vmware

Posted in Advance Concepts, ESX-CMDs, HBACMD, Storage, VMWare | Leave a Comment »

ESX,Storage and HBACMD’s -01

Posted by Preetam on September 1, 2008

Found good article on how manage HBA and listed some command output

[root@FirstESX hbanyware]# ./hbacmd listhbas

Manageable HBA List

Port WWN   : 10:00:00:00:c9:4f:47:6f
Node WWN   : 20:00:00:00:c9:4f:47:6f
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 8000fa00
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

Port WWN   : 10:00:00:00:c9:49:06:da
Node WWN   : 20:00:00:00:c9:49:06:da
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 0000f902
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:4f:47:6f

HBA Attributes for 10:00:00:00:c9:4f:47:6f

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS53963943
Model          : LP10000
Model Desc     : Emulex LP10000 2Gb PCI-X Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 4f 47 6f
Node Symname   :
HW Version     : 1001206d
Opt ROM Version:
FW Version     : 1.90A4 (T2D1.90A4)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : FA00
HBA Type       : LP10000
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 1.90a4
SLI2 FW        : SLI-2 Overlay 1.90a4
IEEE Address   : 00 00 c9 4f 47 6f
Boot BIOS      : Boot Bios Firmware5.00a7
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:49:06:da

HBA Attributes for 10:00:00:00:c9:49:06:da

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS51195140
Model          : LP9002
Model Desc     : Emulex LP9002 2Gb PCI Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 49 06 da
Node Symname   :
HW Version     : 2002606d
Opt ROM Version:
FW Version     : 3.92A2 (C2D3.92A2)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : F900
HBA Type       : LP9002
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 3.92a2
SLI2 FW        : SLI-2 Overlay 3.92a2
IEEE Address   : 00 00 c9 49 06 da
Boot BIOS      : Disabled
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd PortAAttrib 10:00:00:00:c9:49:06:da

Port Attributes for 10:00:00:00:c9:49:06:da

Node WWN            : 20 00 00 00 c9 49 06 da
Port WWN            : 10 00 00 00 c9 49 06 da
Port Symname        :
Port FCID           : 0001
Port Type           : Private Loop
Port State          : Operational
Port Service Type   : 12
Port Supported FC4  : 00 00 01 20 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Active FC4     : 00 00 01 00 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Supported Speed: 2 GBit/sec.
Port Speed          : 1 GBit/sec.
Max Frame Size      : 2048
OS Device Name      : /proc/scsi/lpfc/0
Num Discovered Ports: 16
Fabric Name         : 00 00 00 00 00 00 00 00

Posted in Advance Concepts, ESX-CMDs, HBACMD, Storage, VMWare | Leave a Comment »

VMware: VI Toolkit (for Windows)

Posted by Preetam on August 18, 2008

Best website to start with powershell on vmware

VMware: VI Toolkit (for Windows)

and for quick start guide refer http://www.rtfm-ed.eu/docs/vmwdocs/whitepaper-powershell.pdf

Posted in Advance Concepts, powershell, VMWare | Leave a Comment »

ESX3.5

Posted by Preetam on August 4, 2008

Below paper provides good analysis of which Storage protocol to choose from.

storage protocol performance

see VMware knowledge base article 1003469 “Tuning ESX Server 3.5 for Better Storage Performance by Modifying the Maximum I/O Block Size”

Tuning ESX Server 3.5 for Better Storage Performance by Modifying the Maximum I/O Block Size

Posted in Advance Concepts, ESX3.5, Storage, VMWare, VMWARE_KBs | Leave a Comment »

Good VMWare [Official] blogs

Posted by Preetam on August 4, 2008

Good VMWare blog sites

Networking Blog

http://blogs.vmware.com/networking/

VMware Infrastructure

http://blogs.vmware.com/vi/

Posted in Advance Concepts, Tips, VMWare, VMWare NEWS | Leave a Comment »

Some Networking Stuff

Posted by Preetam on July 20, 2008

vmxnet — a paravirtualized device that works only if VMware Tools is installed in the guest operating system. A paravirtualized device is one designed with specific awareness that it is running in a virtualized environment. The vmxnet adapter is designed for high performance.

• vswif — a paravirtualized device similar to vmxnet that is used only by the ESX Server service console.
• vmknic — a virtual device in the VMkernel, the software layer that manages most of the physical resources on the ESX Server host. The vmknic is used by the TCP/IP stack that services VMotion, NFS and software iSCSI clients that run at the VMkernel level, and remote console traffic.

There is no way to interconnect multiple vSwitches

vSwitches cannot share physical ethernet adapters

Each virtual switch can have up to 1,016 virtual ports, with a limit of 4,096 ports on all virtual switches on a host.

You can create a maximum of 512 port groups on a single host.

You can think of port groups as templates for creating virtual ports with particular sets of specifications.

REFERENCE: http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

Posted in Advance Concepts, Networking | Leave a Comment »

Few more FAQ on managing ESX Server

Posted by Preetam on July 20, 2008

How to start the daemon by default when server starts/reboots?

chkconfig –level 345 ntpd on

 


How to instruct the kernel to occasionally synchronize the system time(when NTP is configured) back to the hardware clock?

hwclock –systohc

 


How to check which services are auto-start in Service console?

ls -Al /etc/rc3.d/S*

 


Where are the review logs located in Service console?

VMKernel –>> /var/log/vmkernel

VMKernel Warning –>> /var/log/vmkwarning

VMKernel Summary –>> /var/log/vmksummary.html

Host agent log –>> /var/log/vmware/hostd.log

WebAccess –>> /var/log/vmware/webaccess

Service Console –>> /var/log/messages

Authentication log –>> /var/log/secure


How to check what is the password policy set at Service console?

esxcfg-auth  –probe

Posted in Advance Concepts, ESX-CMDs, How to, VMWare | Leave a Comment »

ESX3.5 Notes -Part02

Posted by Preetam on April 30, 2008

Lab Manager 2.5.1 does not support ESX Server 3.5.
All hosts in a VMware HA cluster must have DNS configured so that the short host name (without the domain suffix) of any host in the cluster can be resolved to the appropriate IP address from any other host in the cluster.

If a host is added to a cluster, you can no longer create child resource pools of that host. You can create child resource pools of the cluster if the cluster is enabled for Distributed Resource Scheduler (DRS).

You cannot use VMotion to migrate a virtual machine with a guest operating system with 16GB of memory or more to ESX Sever 3.5 hosts or earlier. Resize the guest operating system memory or migrate to a compatible version of ESX Server 3.

Using VI Client or VI Web Access ensures that the starting sectors of partitions are 64K aligned, which improves storage performance.

In centralized license server mode, license files are located at the following default location on the machine running the VMware license server: C:\Program Files\VMware\VMware License Server\Licenses. This is different from VirtualCenter 2.0, where the default location of the license file was C:\Documents and Settings\All Users\Application Data\VMware\VMware License Server\vmware.lic. which no longer exists.

The VI Client installer installs Microsoft .NET Framework 2.0 on your machine. If you have an older version, the VirtualCenter Server installer upgrades your version to version 2.0.

While installing ESX Server 3.5, the option to create a default network for virtual machines is selected by default. If you proceed with installing ESX Server 3.5 with this option selected, your virtual machines share a network adapter with the service console, which does not provide optimal security.

Manage remote console connections—You can now configure VirtualCenter 2.5 to set the maximum number of allowed console connections (0 to 100) to all virtual machines.

VirtualCenter 2.5 provides an unlicensed evaluation mode that doesn’t require that you install and configure a license server while installing VirtualCenter 2.5 and ESX Server 3.

Virtual Center 2.5 can Manage up to 200 hosts and 2000 virtual machines
ESX Server 3.5 supports 256GB of physical memory and virtual machines with 64GB of RAM.
ESX Server hosts support for up to 32 logical processors
SATA support—ESX Server 3.5 supports selected SATA devices connected to dual SAS/SATA controllers
ESX Server 3.5 introduces support for N-Port ID Virtualization (NPIV) for Fibre Channel SANs. Each virtual machine can now have its own World Wide Port Name (WWPN).

VMotion migration of virtual machines with local swap files is supported only across ESX Server 3.5 hosts and later with VirtualCenter 2.5 and later

Enhanced HA provides experimental support for monitoring individual virtual machine failures. VMware HA can now be set up to either restart the failed virtual machine or send a notification to the administrator.

Storage VMotion simplifies array migration and upgrade tasks and reduces I/O bottlenecks by moving virtual machines to the best available storage resource in your environment.Migrations using Storage VMotion must be administered through the Remote Command Line Interface (Remote CLI)

VirtualCenter 2.5 provides support for batch installations of VMware Tools where VMware Tools can now be updated for selected groups of virtual machines. VMware Tools upgrades can now be scheduled for the next boot cycle

Posted in Advance Concepts, DRS, Limits, System Requirements, Virtual Center, VMWare, VMWare Tools | Leave a Comment »