ESX,Storage and HBACMD’s -02

[root@FirstESX hbanyware]# ./hbacmd PortStat 10:00:00:00:c9:4f:47:6f

Port Statistics for 10:00:00:00:c9:4f:47:6f

Secs Since Last Reset   :  700883
Exchange Count          :  5729859
Responder Exchange Count:  23
TX Seq Count            :  6842854
RX Seq Count            :  10893170
TX Frame Count          :  6845254
RX Frame Count          :  108920755
TX Word Count           :  223818240
RX Word Count           :  643851008
TX KB Count             :  874290
RX KB Count             :  2515043
LIP Count               :  1
NOS Count               :  N/A
Error Frame Count       :  0
Dumped Frame Count      :  N/A
Link Failure Count      :  0
Loss of Sync Count      :  46
Loss of Signal Count    :  1
Prim Seq Prot Err Count :  0
Invalid TX Word Count   :  235
Invalid RX Frame CRC Cnt:  0

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd ServerAttrib 10:00:00:00:c9:49:06:da

Server Attributes for 10:00:00:00:c9:49:06:da

Host Name       : FirstESX.vmzare.com
FW Resource Path: /usr/sbin/hbanyware/RMRepository/
DR Resource Path: /usr/sbin/hbanyware/RMRepository/
HBAnyware Server Version: 2.1a35

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd AllNodeInfo 10:00:00:00:c9:49:06:da

All Node Info for 10:00:00:00:c9:49:06:da

Node Type      : READY
FCP ID         : D2
SCSI Bus Number: 0
SCSI Target Num: 0
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9B
OS Device Name : /proc/scsi/lpfc/00,0

Node Type      : READY
FCP ID         : B5
SCSI Bus Number: 0
SCSI Target Num: 3
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9C
OS Device Name : /proc/scsi/lpfc/00,3

Node Type      : READY
FCP ID         : C6
SCSI Bus Number: 0
SCSI Target Num: 10
Node WWN       : 20:00:00:20:37:17:D4:8E
Port WWN       : 22:00:00:20:37:17:D4:8E
OS Device Name : /proc/scsi/lpfc/00,10

Node Type      : READY
FCP ID         : CD
SCSI Bus Number: 0
SCSI Target Num: 9
Node WWN       : 20:00:00:20:37:17:83:92
Port WWN       : 22:00:00:20:37:17:83:92
OS Device Name : /proc/scsi/lpfc/00,9

Node Type      : READY
FCP ID         : CC
SCSI Bus Number: 0
SCSI Target Num: 11
Node WWN       : 20:00:00:20:37:17:C5:14
Port WWN       : 22:00:00:20:37:17:C5:14
OS Device Name : /proc/scsi/lpfc/00,11

Node Type      : READY
FCP ID         : CB
SCSI Bus Number: 0
SCSI Target Num: 12
Node WWN       : 20:00:00:20:37:17:D0:E4
Port WWN       : 22:00:00:20:37:17:D0:E4
OS Device Name : /proc/scsi/lpfc/00,12

Node Type      : READY
FCP ID         : C9
SCSI Bus Number: 0
SCSI Target Num: 13
Node WWN       : 20:00:00:20:37:17:C6:11
Port WWN       : 22:00:00:20:37:17:C6:11
OS Device Name : /proc/scsi/lpfc/00,13

Node Type      : READY
FCP ID         : C7
SCSI Bus Number: 0
SCSI Target Num: 14
Node WWN       : 20:00:00:20:37:17:C2:3D
Port WWN       : 22:00:00:20:37:17:C2:3D
OS Device Name : /proc/scsi/lpfc/00,14

Node Type      : READY
FCP ID         : EF
SCSI Bus Number: 0
SCSI Target Num: 1
Node WWN       : 20:00:00:20:37:17:E5:F1
Port WWN       : 22:00:00:20:37:17:E5:F1
OS Device Name : /proc/scsi/lpfc/00,1

Node Type      : READY
FCP ID         : E8
SCSI Bus Number: 0
SCSI Target Num: 2
Node WWN       : 20:00:00:20:37:17:90:6F
Port WWN       : 22:00:00:20:37:17:90:6F
OS Device Name : /proc/scsi/lpfc/00,2

Node Type      : READY
FCP ID         : E4
SCSI Bus Number: 0
SCSI Target Num: 4
Node WWN       : 20:00:00:20:37:17:DA:7C
Port WWN       : 22:00:00:20:37:17:DA:7C
OS Device Name : /proc/scsi/lpfc/00,4

Node Type      : READY
FCP ID         : E2
SCSI Bus Number: 0
SCSI Target Num: 5
Node WWN       : 20:00:00:20:37:11:E8:CD
Port WWN       : 22:00:00:20:37:11:E8:CD
OS Device Name : /proc/scsi/lpfc/00,5

Node Type      : READY
FCP ID         : E1
SCSI Bus Number: 0
SCSI Target Num: 6
Node WWN       : 20:00:00:20:37:51:46:DD
Port WWN       : 22:00:00:20:37:51:46:DD
OS Device Name : /proc/scsi/lpfc/00,6

Node Type      : READY
FCP ID         : E0
SCSI Bus Number: 0
SCSI Target Num: 7
Node WWN       : 20:00:00:20:37:17:BB:A8
Port WWN       : 22:00:00:20:37:17:BB:A8
OS Device Name : /proc/scsi/lpfc/00,7

Node Type      : READY
FCP ID         : DC
SCSI Bus Number: 0
SCSI Target Num: 8
Node WWN       : 20:00:00:20:37:24:83:26
Port WWN       : 22:00:00:20:37:24:83:26
OS Device Name : /proc/scsi/lpfc/00,8

********************************************************************************************************

[root@FirstESX log]# cat /etc/logrotate.d/vmkernel
/var/log/vmkernel{
    create 0640 root patrol
    missingok
    nocompress         =======> change it to compress
   # keep a history over 3 years.
    monthly
    rotate 36
   #   max log size of 200k  (thus limiting total disk usage to under 8megs)
    size 200k          =======> change it to 2046
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

********************************************************************************************************
Check the nodes in a cluster

[root@FirstESX config]# cat vmware-sites
FULLTIME_SITES_TID 00000022
+ 1:8042,8042,8043 FirstESX    vmware #FT_Agent_Port=8045
+ 2:8042,8042,8043 SecondESX vmware

ESX,Storage and HBACMD’s -01

Found good article on how manage HBA and listed some command output

[root@FirstESX hbanyware]# ./hbacmd listhbas

Manageable HBA List

Port WWN   : 10:00:00:00:c9:4f:47:6f
Node WWN   : 20:00:00:00:c9:4f:47:6f
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 8000fa00
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

Port WWN   : 10:00:00:00:c9:49:06:da
Node WWN   : 20:00:00:00:c9:49:06:da
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 0000f902
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:4f:47:6f

HBA Attributes for 10:00:00:00:c9:4f:47:6f

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS53963943
Model          : LP10000
Model Desc     : Emulex LP10000 2Gb PCI-X Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 4f 47 6f
Node Symname   :
HW Version     : 1001206d
Opt ROM Version:
FW Version     : 1.90A4 (T2D1.90A4)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : FA00
HBA Type       : LP10000
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 1.90a4
SLI2 FW        : SLI-2 Overlay 1.90a4
IEEE Address   : 00 00 c9 4f 47 6f
Boot BIOS      : Boot Bios Firmware5.00a7
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:49:06:da

HBA Attributes for 10:00:00:00:c9:49:06:da

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS51195140
Model          : LP9002
Model Desc     : Emulex LP9002 2Gb PCI Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 49 06 da
Node Symname   :
HW Version     : 2002606d
Opt ROM Version:
FW Version     : 3.92A2 (C2D3.92A2)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : F900
HBA Type       : LP9002
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 3.92a2
SLI2 FW        : SLI-2 Overlay 3.92a2
IEEE Address   : 00 00 c9 49 06 da
Boot BIOS      : Disabled
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd PortAAttrib 10:00:00:00:c9:49:06:da

Port Attributes for 10:00:00:00:c9:49:06:da

Node WWN            : 20 00 00 00 c9 49 06 da
Port WWN            : 10 00 00 00 c9 49 06 da
Port Symname        :
Port FCID           : 0001
Port Type           : Private Loop
Port State          : Operational
Port Service Type   : 12
Port Supported FC4  : 00 00 01 20 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Active FC4     : 00 00 01 00 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Supported Speed: 2 GBit/sec.
Port Speed          : 1 GBit/sec.
Max Frame Size      : 2048
OS Device Name      : /proc/scsi/lpfc/0
Num Discovered Ports: 16
Fabric Name         : 00 00 00 00 00 00 00 00

Few more FAQ on managing ESX Server

How to start the daemon by default when server starts/reboots?

chkconfig –level 345 ntpd on

 

---

How to instruct the kernel to occasionally synchronize the system time(when NTP is configured) back to the hardware clock?

hwclock –systohc

 

---

How to check which services are auto-start in Service console?

ls -Al /etc/rc3.d/S*

 

---

Where are the review logs located in Service console?

VMKernel –>> /var/log/vmkernel

VMKernel Warning –>> /var/log/vmkwarning

VMKernel Summary –>> /var/log/vmksummary.html

Host agent log –>> /var/log/vmware/hostd.log

WebAccess –>> /var/log/vmware/webaccess

Service Console –>> /var/log/messages

Authentication log –>> /var/log/secure

---

How to check what is the password policy set at Service console?

esxcfg-auth  –probe

query RPM Packages

[root@UDA account]# rpm -qi zip-2.31-1.2.1
Name        : zip                          Relocations: (not relocatable)
Version     : 2.31                              Vendor: Red Hat, Inc.
Release     : 1.2.1                         Build Date: Sun 12 Feb 2006 09:36:31 PM CET
Install Date: Fri 02 Mar 2007 08:39:59 PM CET      Build Host: hs20-bc1-6.build.redhat.com
Group       : Applications/Archiving        Source RPM: zip-2.31-1.2.1.src.rpm
Size        : 303340                           License: distributable
Signature   : DSA/SHA1, Mon 06 Mar 2006 10:22:55 PM CET, Key ID b44269d04f2a6fd2
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL         : http://www.info-zip.org/pub/infozip/Zip.html
Summary     : A file compression and packaging utility compatible with PKZIP.
Description :
The zip program is a compression and file packaging utility.  Zip is
analogous to a combination of the UNIX tar and compress commands and
is compatible with PKZIP (a compression and file packaging utility for
MS-DOS systems).

Install the zip package if you need to compress files using the zip
program.

---

[root@UDA account]# rpm -ql zip-2.31-1.2.1
/usr/bin/zip
/usr/bin/zipcloak
/usr/bin/zipnote
/usr/bin/zipsplit
/usr/share/doc/zip-2.31
/usr/share/doc/zip-2.31/BUGS
/usr/share/doc/zip-2.31/CHANGES
/usr/share/doc/zip-2.31/LICENSE
/usr/share/doc/zip-2.31/MANUAL
/usr/share/doc/zip-2.31/README
/usr/share/doc/zip-2.31/TODO
/usr/share/doc/zip-2.31/WHATSNEW
/usr/share/doc/zip-2.31/WHERE
/usr/share/doc/zip-2.31/algorith.txt
/usr/share/man/man1/zip.1.gz

---

cat /etc/vmware/netmap.conf to view contents of file

grep .name /etc/vmware/netmap.conf
find -name “*.vmx”

If you need to create an archive containing a group of files, type

tar cvf filename.tar directory
tar cvf archives/vm-configs.tar vmware

tar tvf filename.tar to view the contents of the file

ps -ef to view all process running on the service cosnole
ps -efH to know parent process

to list all progam running the background

jobs

to restart service

service service-name restart
e.g. service http.vmware restart

to start service at boot time

chkconfig service-name on

available services can be found at ls /etc/init.d

view disk

vdf -h

view the content of vmfs volume

vmkfstools -l vmfs_label
vmkfstools -l vmhba0:0:11

To find the size of the directory

du -sk directoryname

How to’s

How to find version of virtual center you are running?

D:\Program Files\VMware\VMware VirtualCenter 2.0>vpxd.exe -v
VMware VirtualCenter 2.0.1 build-32042

---

Few ESX Commands

vmkping – use vmkernel to ping a device

esxcfg-vswif for configuring network interface

vm-support to run diagnostic commands

esxupdate -l query query patches installed on ESX Server

esxcfg-rescan vmhba1 to rescan HBA’s

ps -ef | grep hostd To check hostd is running

esxcfg-mpath -l to list path details

---

Important Log and their locations

VMKernel Logs = > /var/log/vmware/

VIClient logs = > /var/log/vmware/hostd.log

Virtual Center interaction logs = > /var/log/vmware/vpx/vpxa-*.log

VMWare Patch activity logs = > /var/log/vmware/esxupdate.log

[root@esx2007a config]# cat vmware-sites
FULLTIME_SITES_TID 00000061
+ 1:8042,8042,8043 esx2007e vmware #FT_Agent_Port=8045
+ 2:8042,8042,8043 esx2007d vmware
+ 3:8042,8042,8043 esx2007b vmware
+ 4:8042,8042,8043 esx2007a vmware
+ 5:8042,8042,8043 esx2007c vmware

License server logs = > %ALLUSERSPROFILE%\Application Data\VMware\VMware License Server\lmgrd.log

---

STORAGE

When ESX Server scans the SAN, each HBA reports all LUNs visible on the storage network; each LUN reports an ID that uniquely identifies it to all nodes on the storage network. After detecting the same unique LUN ID reported by the storage network, the VMkernel automatically enables multiple, redundant paths to this LUN, known as multi-pathing.ESX Server uses a single storage path for a particular LUN until the LUN becomes unavailable over this path. After noting the path failure, ESX Server switches to an operational path.

---

VIRTUAL CENTER:

SQL Server authentication will work predictably on local and remote database servers. NT authentication requires VC to run with a service account that has access to the DB; “Local System”account does not have access to remote servers.

vpxd.exe VirtualCenter Server service

vpxd.cfg VirtualCenter Server configuration

vpxa VirtualCenterAgent

If you create partition from virtual center then that partition is automatically aligned, in case you are creating partition using vmkfstools then you will have to manually align the partition using fdisk. But this is not enough you also have to align partition at guest level as well, which can be easily achieved using diskpart if the os is windows.

How to

How to Re-start VMWare Hostd service ?

service mgmt-vmware restart

when you make change from command line, those changes might not be visible in Virtual center, to make them visible re-start of hostd is recommended.This restart process re-reads esx.conf file

---

TO FIND THE STATUS OF ALL SERVICES

E.g. => service –status-all

E.g. => service vmware-vpxa status