Lets Design, Implement and do Administration of ESX3

Virtualization with VMWare Infrastructure 3.0

Archive for the ‘Storage’ Category

How to find Queue depth of HBA ?

Posted by Preetam on April 30, 2009

Here is way

How to find queue depth for HBA ?

[root@pz-vmware07 scripts]# tail /etc/vmware/esx.conf
/vmkmodule[0003]/enabled = “true”
/vmkmodule[0003]/module = “tg3.o”
/vmkmodule[0003]/type = “nic”
/vmkmodule[0004]/enabled = “true”
/vmkmodule[0004]/module = “cciss.o”
/vmkmodule[0004]/type = “scsi”
/vmkmodule[0005]/enabled = “true”
/vmkmodule[0005]/module = “lpfcdd_732.o”
/vmkmodule[0005]/options = “lpfc0_lun_queue_depth=16 lpfc1_lun_queue_depth=16 lpfc_tgt_queue_depth=31”
/vmkmodule[0005]/type = “fc”

 

Or

[root@pz-vmware07 scripts]# esxcfg-module -l
Device Driver Modules
Module Type Enabled Loaded
vmkapimod vmkapimod true true
vmklinux linux true true
bnx2 nic true true
tg3 nic true true
cciss scsi true true
lpfcdd_732 fc true true
[root@ci-vmware07 scripts]# esxcfg-module -g lpfcdd_732
lpfcdd_732 enabled = 1 options = ‘lpfc0_lun_queue_depth=16 lpfc1_lun_queue_depth=16 lpfc_tgt_queue_depth=31’

Advertisements

Posted in Advance Concepts, How to, Storage, VMWare | Leave a Comment »

What are Thin Disks,Thick Disks

Posted by Preetam on September 13, 2008

Excellent explanation of various storage technologies, especially Thin disks from vmmba.

 There are three main technologies that can accomplish storage oversubscription:

  1. Linked clones
    • This feature is available in VMware Lab Manager and VMware Workstation at the virtual disk level.  When a linked clone is used, the new VM uses pointers to the original VM for all common data.
    • The additional advantage of linked clones is that whitespace is not stored – for example if an empty data disk is part of a clone operation, the new disk will act as a "thin" disk and only consume the storage that it really requires for data
    • Linked clones can also be accomplished at the datastore level using technologies such as NetApp FlexClone (useful when cloning many VMs at once)
    • Keep in mind: linked clones pay a performance penalty on write operations (using copy-on-write), and put added stress on the source disks on read operations
  2. Thin Disks
    • Thin-provisioned disks are virtual disks that "appear" to the VM as one size, but only consume up to the amount of data that is required by that disk.  So, a 10 GB drive that is 50% utilized will only store 5 GB on disk (a traditional "thick" virtual disk would consume the entire 10 GB on disk)
    • Thin disks are options in VMware Workstation, and are the default disk type when using NFS storage in VMware ESX Server – however, VMs cloned from templates are always thick
    • Storage vendors such as Hitachi and NetApp have LUN-level thin provisioning, but that would only apply to VMware if using RDMs
  3. Deduplication
    • Deduplication is a technology similar to memory page sharing (above), where common data is stored only once.  It is done "after the fact" (ex poste), meaning de-duplication opportunities are scanned using a background process
    • Deduplication is primarily used for backups (e.g. Symantec PureDisk, EMC Avamar, or Quantum DXi-Series), but can also be used on the filesystem itself (today, using NetApp Deduplication, formerly A-SIS)

REFERENCE: www.vmmba.com

Posted in Advance Concepts, Storage, VM Provisioning, VMFS, VMWare | 1 Comment »

ESX,Storage and HBACMD’s -02

Posted by Preetam on September 1, 2008

[root@FirstESX hbanyware]# ./hbacmd PortStat 10:00:00:00:c9:4f:47:6f

Port Statistics for 10:00:00:00:c9:4f:47:6f

Secs Since Last Reset   :  700883
Exchange Count          :  5729859
Responder Exchange Count:  23
TX Seq Count            :  6842854
RX Seq Count            :  10893170
TX Frame Count          :  6845254
RX Frame Count          :  108920755
TX Word Count           :  223818240
RX Word Count           :  643851008
TX KB Count             :  874290
RX KB Count             :  2515043
LIP Count               :  1
NOS Count               :  N/A
Error Frame Count       :  0
Dumped Frame Count      :  N/A
Link Failure Count      :  0
Loss of Sync Count      :  46
Loss of Signal Count    :  1
Prim Seq Prot Err Count :  0
Invalid TX Word Count   :  235
Invalid RX Frame CRC Cnt:  0

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd ServerAttrib 10:00:00:00:c9:49:06:da

Server Attributes for 10:00:00:00:c9:49:06:da

Host Name       : FirstESX.vmzare.com
FW Resource Path: /usr/sbin/hbanyware/RMRepository/
DR Resource Path: /usr/sbin/hbanyware/RMRepository/
HBAnyware Server Version: 2.1a35

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd AllNodeInfo 10:00:00:00:c9:49:06:da

All Node Info for 10:00:00:00:c9:49:06:da

Node Type      : READY
FCP ID         : D2
SCSI Bus Number: 0
SCSI Target Num: 0
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9B
OS Device Name : /proc/scsi/lpfc/00,0

Node Type      : READY
FCP ID         : B5
SCSI Bus Number: 0
SCSI Target Num: 3
Node WWN       : 50:80:02:00:00:03:19:98
Port WWN       : 50:80:02:00:00:03:19:9C
OS Device Name : /proc/scsi/lpfc/00,3

Node Type      : READY
FCP ID         : C6
SCSI Bus Number: 0
SCSI Target Num: 10
Node WWN       : 20:00:00:20:37:17:D4:8E
Port WWN       : 22:00:00:20:37:17:D4:8E
OS Device Name : /proc/scsi/lpfc/00,10

Node Type      : READY
FCP ID         : CD
SCSI Bus Number: 0
SCSI Target Num: 9
Node WWN       : 20:00:00:20:37:17:83:92
Port WWN       : 22:00:00:20:37:17:83:92
OS Device Name : /proc/scsi/lpfc/00,9

Node Type      : READY
FCP ID         : CC
SCSI Bus Number: 0
SCSI Target Num: 11
Node WWN       : 20:00:00:20:37:17:C5:14
Port WWN       : 22:00:00:20:37:17:C5:14
OS Device Name : /proc/scsi/lpfc/00,11

Node Type      : READY
FCP ID         : CB
SCSI Bus Number: 0
SCSI Target Num: 12
Node WWN       : 20:00:00:20:37:17:D0:E4
Port WWN       : 22:00:00:20:37:17:D0:E4
OS Device Name : /proc/scsi/lpfc/00,12

Node Type      : READY
FCP ID         : C9
SCSI Bus Number: 0
SCSI Target Num: 13
Node WWN       : 20:00:00:20:37:17:C6:11
Port WWN       : 22:00:00:20:37:17:C6:11
OS Device Name : /proc/scsi/lpfc/00,13

Node Type      : READY
FCP ID         : C7
SCSI Bus Number: 0
SCSI Target Num: 14
Node WWN       : 20:00:00:20:37:17:C2:3D
Port WWN       : 22:00:00:20:37:17:C2:3D
OS Device Name : /proc/scsi/lpfc/00,14

Node Type      : READY
FCP ID         : EF
SCSI Bus Number: 0
SCSI Target Num: 1
Node WWN       : 20:00:00:20:37:17:E5:F1
Port WWN       : 22:00:00:20:37:17:E5:F1
OS Device Name : /proc/scsi/lpfc/00,1

Node Type      : READY
FCP ID         : E8
SCSI Bus Number: 0
SCSI Target Num: 2
Node WWN       : 20:00:00:20:37:17:90:6F
Port WWN       : 22:00:00:20:37:17:90:6F
OS Device Name : /proc/scsi/lpfc/00,2

Node Type      : READY
FCP ID         : E4
SCSI Bus Number: 0
SCSI Target Num: 4
Node WWN       : 20:00:00:20:37:17:DA:7C
Port WWN       : 22:00:00:20:37:17:DA:7C
OS Device Name : /proc/scsi/lpfc/00,4

Node Type      : READY
FCP ID         : E2
SCSI Bus Number: 0
SCSI Target Num: 5
Node WWN       : 20:00:00:20:37:11:E8:CD
Port WWN       : 22:00:00:20:37:11:E8:CD
OS Device Name : /proc/scsi/lpfc/00,5

Node Type      : READY
FCP ID         : E1
SCSI Bus Number: 0
SCSI Target Num: 6
Node WWN       : 20:00:00:20:37:51:46:DD
Port WWN       : 22:00:00:20:37:51:46:DD
OS Device Name : /proc/scsi/lpfc/00,6

Node Type      : READY
FCP ID         : E0
SCSI Bus Number: 0
SCSI Target Num: 7
Node WWN       : 20:00:00:20:37:17:BB:A8
Port WWN       : 22:00:00:20:37:17:BB:A8
OS Device Name : /proc/scsi/lpfc/00,7

Node Type      : READY
FCP ID         : DC
SCSI Bus Number: 0
SCSI Target Num: 8
Node WWN       : 20:00:00:20:37:24:83:26
Port WWN       : 22:00:00:20:37:24:83:26
OS Device Name : /proc/scsi/lpfc/00,8

********************************************************************************************************

[root@FirstESX log]# cat /etc/logrotate.d/vmkernel
/var/log/vmkernel{
    create 0640 root patrol
    missingok
    nocompress         =======> change it to compress
   # keep a history over 3 years.
    monthly
    rotate 36
   #   max log size of 200k  (thus limiting total disk usage to under 8megs)
    size 200k          =======> change it to 2046
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

********************************************************************************************************
Check the nodes in a cluster

[root@FirstESX config]# cat vmware-sites
FULLTIME_SITES_TID 00000022
+ 1:8042,8042,8043 FirstESX    vmware #FT_Agent_Port=8045
+ 2:8042,8042,8043 SecondESX vmware

Posted in Advance Concepts, ESX-CMDs, HBACMD, Storage, VMWare | Leave a Comment »

ESX,Storage and HBACMD’s -01

Posted by Preetam on September 1, 2008

Found good article on how manage HBA and listed some command output

[root@FirstESX hbanyware]# ./hbacmd listhbas

Manageable HBA List

Port WWN   : 10:00:00:00:c9:4f:47:6f
Node WWN   : 20:00:00:00:c9:4f:47:6f
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 8000fa00
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

Port WWN   : 10:00:00:00:c9:49:06:da
Node WWN   : 20:00:00:00:c9:49:06:da
Fabric Name: 00:00:00:00:00:00:00:00
Flags      : 0000f902
Host Name  : FirstESX.vmzare.com
Mfg        : Emulex Corporation

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:4f:47:6f

HBA Attributes for 10:00:00:00:c9:4f:47:6f

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS53963943
Model          : LP10000
Model Desc     : Emulex LP10000 2Gb PCI-X Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 4f 47 6f
Node Symname   :
HW Version     : 1001206d
Opt ROM Version:
FW Version     : 1.90A4 (T2D1.90A4)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : FA00
HBA Type       : LP10000
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 1.90a4
SLI2 FW        : SLI-2 Overlay 1.90a4
IEEE Address   : 00 00 c9 4f 47 6f
Boot BIOS      : Boot Bios Firmware5.00a7
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@FirstESX hbanyware]# ./hbacmd HBAAttrib 10:00:00:00:c9:49:06:da

HBA Attributes for 10:00:00:00:c9:49:06:da

Host Name      : FirstESX.vmzare.com
Manufacturer   : Emulex Corporation
Serial Number  : MS51195140
Model          : LP9002
Model Desc     : Emulex LP9002 2Gb PCI Fibre Channel Adapter
Node WWN       : 20 00 00 00 c9 49 06 da
Node Symname   :
HW Version     : 2002606d
Opt ROM Version:
FW Version     : 3.92A2 (C2D3.92A2)
Vender Spec ID : 10DF
Number of Ports: 1
Driver Name    : lpfcdd_732
Device ID      : F900
HBA Type       : LP9002
Operational FW : SLI-2 Overlay
SLI1 FW        : SLI-1 Overlay 3.92a2
SLI2 FW        : SLI-2 Overlay 3.92a2
IEEE Address   : 00 00 c9 49 06 da
Boot BIOS      : Disabled
Driver Version : 7.3.2_vmw; HBAAPI(I) v2.0.f, 12-01-03

******************************************************************************************************************************

[root@SecondESX hbanyware]# ./hbacmd PortAAttrib 10:00:00:00:c9:49:06:da

Port Attributes for 10:00:00:00:c9:49:06:da

Node WWN            : 20 00 00 00 c9 49 06 da
Port WWN            : 10 00 00 00 c9 49 06 da
Port Symname        :
Port FCID           : 0001
Port Type           : Private Loop
Port State          : Operational
Port Service Type   : 12
Port Supported FC4  : 00 00 01 20 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Active FC4     : 00 00 01 00 00 00 00 01
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
                      00 00 00 00 00 00 00 00
Port Supported Speed: 2 GBit/sec.
Port Speed          : 1 GBit/sec.
Max Frame Size      : 2048
OS Device Name      : /proc/scsi/lpfc/0
Num Discovered Ports: 16
Fabric Name         : 00 00 00 00 00 00 00 00

Posted in Advance Concepts, ESX-CMDs, HBACMD, Storage, VMWare | Leave a Comment »

ESX3.5

Posted by Preetam on August 4, 2008

Below paper provides good analysis of which Storage protocol to choose from.

storage protocol performance

see VMware knowledge base article 1003469 “Tuning ESX Server 3.5 for Better Storage Performance by Modifying the Maximum I/O Block Size”

Tuning ESX Server 3.5 for Better Storage Performance by Modifying the Maximum I/O Block Size

Posted in Advance Concepts, ESX3.5, Storage, VMWare, VMWARE_KBs | Leave a Comment »