Lets Design, Implement and do Administration of ESX3

Virtualization with VMWare Infrastructure 3.0

Archive for the ‘Resources’ Category

Resource Mgmt Guide -02

Posted by Preetam on March 31, 2007

If a host is added to the cluster, you can no longer create child resource in the host
While creation resource pools, and assigning limits, reservation, shares, if any of the value is not valid, you will see yellow triangle against resource pool.

E.g.You created a resource pool of 10GB, in that you create resource pool of 6 GB and again you try to create resource pool of another 6 GB and type is fixed, you will get yellow triangle

When you move VMs into new resource pool, VMs existing limits & reservations don’t not change. If shares value is customized, it is not changed as well, but it is set to normal.high, low % share value changes. Also unreserved resource values changes to reflect newly added reservations. That being said, if VMs reservations are not met by resource pool, move will fail

When you move hosts into cluster, depending upon what you choose like

  • DRS enabled
  • DRS disabled

Existing resource pools are affected accordingly, when you enable DRS on cluster, you have option of moving the entire tree of resource pool into new cluster or you have option to put this host’s VMs into the cluster’s root resources, when you enable second option, tree structure is changed to flat structure and all VMs & Resource pool becomes child of cluster root, instead of host, However when you happen to move out Host from the cluster, resource pool heirarchy is not moved, in short it becomes completely independent of host.

If DRS is disabled all resource pools are deleted and VMs become direct child of the cluster.

Also in Non-DRS cluster, there is no cluster-wide resource management based on the shares. Shares remain relative to host.

You can create cluster without special license, but you must have a license to enable a cluster for DRS or HA.

What happens to DRS & HA when Virtual center goes down?

HA – Continues to work and can still restart VMs on other hosts in case of failover; however information specific to VMs like cluster properties (priority or isolation response) is based on the state of the cluster before the virtual center goes down

DRS – No recommendations are made for resource optimization.

By default automation level (HA) is enabled at cluster level but you can customize it at VM level as well. Migration recommendation made by VMHA is based on the priority and associated reasons.

When in maintenance mode, the host does not allow you to deploy VMs, VMs that are running on maintenance mode continues to run, you either migrate them to another host or shutdown. When no VMs are running on the host, host’s icon changes to include under maintenance mode. If DRS cluster is in automation mode, all VMs are migrated to different host, when host moved to maintenance mode. This makes sense when other host fails and tries to failover VMs on this host which is in maintenance, being in maintenance mode it won’t allow any VMs. Also if host goes into maintenance mode VMHA will compute current failover capacity excluding host which are in maintenance mode. When host exist maintenance mode, failover capacity is again computed by VMHA.

When you allow VM to be started even if they violate availability constraints deselected (i.e. disable) you will also not able to

  • Reverting VMs to last snapshot
  • Change CPU/Memory reservations
  • Migration VM into the cluster


Posted in Advance Concepts, DRS, iSCSI, Limits, Reservations, Resource Pools, Resources, VMHA, VMWare | Leave a Comment »

Resource Mgmt Guide -01

Posted by Preetam on March 31, 2007

Resource Management Guide points filtered out. It is must to read resource management guide for VCP. Below are the points I feel are important & gives quick glimpse of Resource mgmt in VI3


The amount of physical memory VMs can used is less than what is present on physical host because Virtualization layer uses some part of it.

 e.g. Physical Host with 3.2 GHz with dual core and 2 GB RAM, will make 6 GHz and 1.5GB RAM to VMs 

Reservation [GHz/MB]:

By default no reservation is specified on CPU and Memory on VMs, however it is good idea to specify reservation for Memory/CPU for VMs, so that resource is always available for VMs.

  • Use reservation to specify minimum amount of cpu/memory, host assigns additional resources as available based on the limits and number of shares. Don’t set reservation too high because it will restrict VM to that resource pool only.
  • Reservations are used by running virtual machines and resource pools with reservations.
  • Reserved and unreserved CPU and Memory change only if the virtual machine is powered on.


E.g. Consider VM with 2GHz reservation, but it is not using this reservation. This reservation cannot be used for reservation for other VMs however this 2GHz can be used by other VMs so the idle CPU resource is not wasted.

 E.g. VM-A and VM-B have 1GHz of CPU reservation, if VM-A is using 500 MHz of CPU, then VM-B can use 1.5 GHz of CPU to use idle resources. 

The ESX server will allocate number of MB reserved for VM directly, anything beyond the reservation is allocated using the host’s physical resources or when physical resources aren’t available, handled using techniques like balloning or swapping.


Limit [CPU/MB]:

By default no limit is specified on CPU/MB, in this case whatever memory/cpu you assign to Virtual Machine becomes its limit.


Each VM is entitled to resources in proportion to its specified shares, bounded by reservations and limits. Shares are typical specified as high, normal and low. They are in the ratio of 4:2:1. If you expect frequent change in the resources, then use shares than reservations to allocate resources fairly across VMs.


  • High — 2000 shares per virtual machines, 20 shares per MB of VM memory
  • Normal — 1000 shares per virtual machines, 10 shares per MB of VM memory
  • Low — 500 shares per virtual machines, 5 shares per MB of VM memory 

In short high means twice as much as normal, normal means twice as much as low. You can also specify custom shares

e.g. VM with 2 Virtual CPU & 1 GB RAM, with shares set to normal for CPU/RAM, then2×1000=2000Ghz shares of CPU and 1024×10=10240 MB memory shares.  

By default all VMs associated with single ESX host received

q       Equal share of CPU per Virtual CPU. E.g. If VM-A has one 1 CPU and VM-B has two CPU, then VM-A will receive half of CPU of VM-B

q       Equal MB of per share of virtual memory size.


When you power on VMs, system checks amount of CPU/Memory unreserved available, depending upon reservation set on VMs, it compares and checks if this VM can be powered on, this process is called Admission Control.


Posted in Advance Concepts, Limits, Reservations, Resources, Shares, VMWare | Leave a Comment »

Basic System Administration -Part 3

Posted by Preetam on March 24, 2007

With ESX 3.0 you can migrate suspended VMs under cold migration process. You can move files of VMs only when they are power off and suspended.

When you modify User’s permissions, Users do not need to log off and log on into Virtual Center for changes to take effect. All changes take effect immediately.

You can define permission on

  • VM
  • Folders
  • Datacenters
  • Resource pools
  • Templates
  • Host
  • Clusters

You cannot directy define permission on

  1. Networks
  2. Datastores

In virtual center you can assign permission to

  • Folder
  • Datacenter
  • Host
  • VM
  • Templates
  • Cluster
  • Resource Pools
  • In ESX server you can assign permission to
  • Resource Pool
  • VMs
  • Hosts

There are two types of roles

System (cannot be modified): administrator/read-only/No-access

Sample (can be modified):

Detail Description is available in Excel Sheet HERE

Posted in Advance Concepts, Resources, VM Permissions, VMWare | 1 Comment »

Creating resource Pools

Posted by Preetam on February 27, 2007

VMKernel manages all memory, except the memory that is allocated to Service console. VM will only power on when Server is sure to allocate that VM’s it’s reservations. This is calculated by Server by checking how much unreserved resources are available and if that meets the reservation of the VM.

Let’s learn how to create resource pools.Resource pool can be created on individual hosts, only if hosts are not clustered. In order to create resource pool, you need to have following information in hand.

  1. Name: Name of the resource pool
  2. Shares: Number of shares to be allocated to every VM
  3. Reservation: Minimum resources guranteed to VM’s
  4. Expandable reservation: There are two options YES/NO, if say YES, if reservation of existing resource pools are not available they can be used from Parent resource pool, If select NO, then host won;t be able to power ON the VM.
  5. Limit: Maximum resources any VM, under this resource pool would get.

Similiar option you would get for memory configuration. Once this step is completed you might get yellow triangle, which suggest something is incorrect. 

Now resource pool is created, let’s play around with it.

Select resource pool(left hand-side), select on summary tab (right-side), you see below mentioned screen

I’ve underline things which could be of worth observation when you see them in Virtual center.

Now select resource allocation, resource allocation gives what reservation have been used, what is available, In order understand this clearly, let take look at picture below

You can see currently there are no reservation defined in this resource pool, but you still see CPU/Memory reservation used,

because VM-Marketing has been allocated reservation which is consumed from total reservation available

Other important information you get from here is the unreserved status, this unreserved status helps you in allocating resources in future. You also see resources are set to unlimited field, there is custom shares defined for particular VM.

Of course it is true, you can certainly change values under resource pools by editing it after you create resource pools

Lets discuss another important topic, How to add VM to existing Resource pool or move vm to another resource pool. You can vm to existing pools in two ways

  • When you create VM, you can assign VM to any specific resource pool
  • You can also drag and drop VM to any resource pool even after it is created

NB: you don;t need to power off VM in order to just move it to different resource pool.

When you move drag & drop VM to resource pool, its Memory/CPU/Shares/Limit don’t change, this change is reflected only when the VM is powered ON, in power off state resource pool’s resources are not affected by it or not being allocated to this new VM. Though you will observe in resource allocation tab Reservations for both CPU/Memory will reflect the changes in value. Also a point to be noted if Resource pool is not in a position to allocate the reservation for this VM, it generate error and move will fail.

Posted in Resource Pools, Resources, Virtual Center, VMWare | Leave a Comment »

Shares,Reservation,Limits -CPU/Memory Resource settings

Posted by Preetam on February 27, 2007

Before we talk about Resource Pool we need to know what are resources in VMware terminology. Resources are

  • CPU’s
  • Memory
  • Disk (Manages which proportional share mechanisim)
  • Network (are controlled by using Network-traffic shaping)

Resources are provided by ESX host and they are consumed by VM’s. if ESX host are clustered, then Cluster is actual resource provider.

Every VM will be allocated Memory and CPU resources, there are 3 settings in resource pool which influence as to what memory and CPU any VM would get.

RESERVATION:Reservation specified for the resource pool or VM, if the reservation is set to Zero (which is by default), it means no reservation is set. VM will not start if its reservation is not met or guaranteed. When reservations are not utilized ESX host can assign them to other VM’s. Lets take example

You have VM’s A and B. A & B is configured for 1GHZ reservation. Now during some days A only goes as far a 0.5 GHz, under such case B can use 1.5 GHz. But if B is using 1.5GHz and A is poweroff, Once A is powerON B has to give away 0.5 GHz

LIMIT:Specified limit for the VM, default is unlimited. Server can allocate more memory/CPU than reservation but it cannot assign more than it’s limit. Applying limit will vary depending upon the circumstances or your design requirement.

It is recommended to apply limit, when you wish to manage few VM’s and you know Max Memory/CPU utilization of these VM’s applications. Because one you apply limit, even if the resource on one system is under utilize and other machine is going to require more CPU/Memory sometime, it won’t be able to get that idle resource and there are chance paging would start happening.

SHARES: Number of shares determine which VM will get resources when there is competition for resources among VM’s.Specifying shares makes sense only with regard to sibling virtual machines or resource pools, that is, virtual machines or resource pools with the same parent in the resource pool hierarchy.

The amount of resources represented by each share changes when a new virtual machine is powered on. This affects all virtual machines. Shares are typically specified as high, normal, or low. High, normal, and low specify share values with a 4:2:1 ratio.

For example:Two virtual machines run on a host with 8GHz. Both are set to Normal and get 4GHz each.A third virtual machine is powered on. It is set to High, which means it should have twice as many shares as the machines set to Normal. The new virtual machine receives 4GHz and the two other machines get only 2GHz each.

Share values default to:

  • High — 2000 shares per virtual CPU, 20 shares per MB of virtual machine memory
  • Normal — 1000 shares per virtual CPU, 10 shares per MB of virtual machine memory
  • Low — 500 shares per virtual CPU, 5 shares per MB of virtual machine memory

By going by above defination, if we’ve VM with 2 CPU’s and 1 GB RAM and shares is defined as High, so this VM is going to get

  • 2000 Shares x 2 CPU’s =Shares of CPU Power
  • 20 Shares x 1024MB=Shares of Memory

Mind you, this is just shares and they will be useful in cases where VM is competing either for CPU or Memory resources.

Resource pool is used to distribute CPU’s and Memory across VM. resource pools can have child resource pool or VM or Both.Resource pool or VM which are at the same level of heirarchy, it is called sibling. Picture below gives very good information about the relation.

You can actually create resources pool as per the department’s requirement and can guarantee that particular resource pool will provide required processing power when the condition demands.

You don’t need to configure resources to each VM, in fact you apply common settings across group of VM using resource poo. In above figure RP-Marketing is a resource pool, all resources defined in it, will be automatically applied to VM’s under it, with configuring such setting for each VM.

In order to really leverage the power of resource pools, Group hosts into cluster. When you create cluster, resource are managed at the cluster level rather host level. In short resources on each host are combined into one. This one resource is total resource which can be allocated to all VM’s under that cluster. Figure below explains it briefly.

Posted in Limits, Reservations, Resource Pools, Resources, Shares, VMWare | 3 Comments »