I’ve found a few situations in which snapshots get stuck, like glue, to a running VM, and despite your best effort to delete them, they wont go away. Like in-laws, they stick around, a bit longer than is pleasant.

If a snapshot has not been removed cleanly on the first try, you may want to create a new snapshot first (just to make sure the system knows there IS a snapshot present) before using the methods below.

Here are a few ways of getting rid of the ones that don’t want to go away:

From the Service Console (ESX)

[root@esx esx:storage1]# vmware-cmd ./VMname/VMname.vmx removesnapshots
removesnapshots() = 1

From PowerShell

PS C:\> get-vm VMname* | get-snapshot | Remove-Snapshot -Confirm:$false

Using the rCLI

C:\Program Files\VMware\VMware VI Remote CLI\bin>vmware-cmd.pl -H esx -U root –P  password /vmfs/volumes/esx:storage1/VMname/VMname.vmx removesnapshots
removesnapshot () = 1

Using the Perl Toolkit

This brought to us by Andrew, an avid Perl Toolkit user:
./snapshotmanager.pl –server your.vc.host –operation removeall –vmname some.bad.vm

Warning, depending on the size of the snapshot, it may take a LONG while for them to go away! You can get an idea of how long it’s going to take by looking at how large the snapshots are with ls -lh and note the size of your largest snapshots. They bigger than a few deciGig (for those who don’t get it, that’s a few tens of Gigs, 10G-20G)? Shame on you.

Tom Howarth from PlanetVM turned me on to a great series of posts by Ed Haletky on how to manage different combinations of NIC’s in VMware ESX:

• Blue Gears – 2 Physical NICs with VMware ESX
• Blue Gears – 3 Physical NICs with VMware ESX
• Blue Gears – 4 Physical NICs with VMware ESX
• Blue Gears – 5 Physical NICs with VMware ESX
• Blue Gears – 6 Physical NICs with VMware ESX

He covers the information quite thoroughly, making it a worthwhile read.

Well now. We’ve come a long way since parts 1 & 2 haven’t we? That is all well and good considering, and makes for some light background reading.  However, the game changes with ESXi, as there is no “Supported” Service Console on it’s platform.

In reality, the vmkernel runs a busybox executable. (Busybox is a linux in an executable binary sort of thing), and you can enable DropBear SSH, and the like on it. Doing this however, will likely void your warranty or support agreement, or prevent you from getting your per incident support taken care of, so I strongly recommend against it.

With that said then.  How does ESXi handle all of the communication we talked about before, how do you make it redundant, and how do you interact with ESXi in a meaningful way?  Let’s take a look at the first two, the last will be covered in a future post.

“Service Console” Communications in ESXi

How does ESXi overcome it’s lack of a good and proper “Service Console”? At the physical console you get something that resembles a server BIOS screen from which you can configure the ESXi host.

So that’s cool, but how does it communicate with the VI Client or VMware’s vCenter (formerly Virtual Center)? It does this using a “Management IP”. By default this Management IP is set by DHCP at host boot time, or can be configured manually from the console above. Remember this IP.

Making ESXi Management Communication Redundant

This actually works in quite the same way it did for the “Fat” version of ESX (Classic). You can setup NIC teaming, and a redundant vSwitch on another network. Read Part 2 for more information there.

Now that we’ve discussed what the service console is and what it does for you. We’ll talk about making it redundant, why Service Console Redundancy is important, and some ways of going about that. Another “Pretty Big Task” but one worth sticking around for (at least for the cookies at the end).

The Department of Redundancy Department

What is redundancy? I think perhaps that redundantly describing redundancy in an article on making redundant things redundant would be… well, redundant. Enough of that! In the context of the Service Console, redundancy is simply a means to provide continuous access to the Service Console and it’s services. A big task considering everything that was covered in Part 1 (link).

So why is this important? Lets cover it on an item by item basis from part 1:

Communications: Losing communication with your ESX host may not seem to be that big a deal. After all, your Virtual Machines will continue to run, wont they? If you’ve separated your Virtual Machine network from your Service Console network, then they (the VM’s that is) are still communicating with the remainder of the network. So why is losing communication with the Service Console bad? Well, you lose connectivity to the following:

• Command Line Interface (CLI)
• SSH Access
• Web Management (this includes SOAP, and the other various API’s)
• Using the VMware Infrastructure Client
• Communication with Virtual Center

So in some environments this may not be an issue, but alas, one has to manage and monitor the ESX host, so already you need… well all of them. Further, if you have this host in a cluster of ESX hosts this will very well likely break High Availability. What? My HA? My Precious HA? You mean I spent all of that money on HA licensing, and something as simple as service console communication can take that out? Look again at the last bullet point there, go ahead, I’ll wait. So knowing that the service console is responsible for communicating with virtual center, and knowing that the Virtual Center checks for HA, and communicates HA decisions back to ESX, it then becomes critical that you have redundant service consoles. At least if you don’t want your VM’s rebooting for no reason.

How do I make it redundant?

There are as many was to do this as there are episodes of Seinfeld. Really.  That said, we’ll go over a few of those ways.

Redundant NICS

That’s right, as simple as that.  On your Service Console vSwitch, just add a second  NIC and away you go. What this does is setup a fail over pair, and open up the opportunity for load balancing, should you also have VM traffic flowing over that NIC.

Redundant vSwitches

This one takes a bit more doing, as it requires two (or more) networks, and at least two NICs as well. Having two vSwitches, with one separate for management is a best practice anyways.  To implement redundant SC’s in this setup, is to add a Service Console to both your management vSwitch and your VM vSwitch.  While you may be mixing traffic in this case (management with the VM’s) your risk should be mitigated by the ESX firewall and strong user/passwords, as this service console would only be used in emergencies. Please note, this can be any network that supports VM traffic, so if you have a network on your ESX host(s) dedicated to Backup, you can likely put the additional Service Console there.

Redundant NICS with Redundant vSwitches

Ok, so this one was to be a neat little pain in the rear, really.  It’ basically combines both of the above, by throwing redundant nic’s onto the redundant vSwitches.  While we’re at it, to make it more redundant, lets throw in redundant Physical switches on the uplink end as well. 

For further discussion on Service Console Redundancy, I’d like to point you at yellow-bricks. (Note: We’ve been talking about option 3 for most of ours)

Found a great resource explaining quite a few of the esxcfg-* commands, what they are and their use. They’re copied here for future reference.

NOTE: These will work from the rcli, SSH to the service console, or via VIMA. I’ve not tested these on ESXi (as the Service Console is ‘unsupported’)

To get a feel for what commands are available, log on to the console (or via ssh if you like), and type

esxcfg-[TAB][TAB].

[root@vi3host root]# esxcfg-

esxcfg-advcfg esxcfg-firewall esxcfg-module esxcfg-pciid esxcfg-swiscsi esxcfg-vswif

esxcfg-auth esxcfg-info esxcfg-mpath esxcfg-rescan esxcfg-upgrade esxcfg-vswitch

esxcfg-boot esxcfg-init esxcfg-nas esxcfg-resgrp esxcfg-vmhbadevs

esxcfg-dumppart esxcfg-linuxnet esxcfg-nics esxcfg-route esxcfg-vmknic

esxcfg-advcfg

From the man page “…provides an interface to query and modify the advanced options of the VMkernel. These options control a wide variety of different areas in the VMkernel including storage, resource, and networking global settings.”

Use esxcfg-info -o (discussed below) to output the advanced parameters to a file for further investigation.

Here are some examples:

[root@vi3host root]# esxcfg-advcfg -g /VMFS3/ZeroedThickVirtualDisks

Value of ZeroedThickVirtualDisks is 1

[root@vi3host root]# esxcfg-advcfg -g /Disk/SupportSparseLUN

Value of SupportSparseLUN is 1

[root@vi3host root]# esxcfg-advcfg -g /Disk/MaxLUN

Value of MaxLUN is 256

[root@vi3host root]# esxcfg-advcfg -g /Scsi/ConflictRetries

Value of ConflictRetries is 80

[root@vi3host root]# esxcfg-advcfg -g /LVM/EnableResignature

Value of EnableResignature is 0

[root@vi3host root]# esxcfg-advcfg -g /Net/NotifySwitch

Value of NotifySwitch is 1

So what can you configure? Read through the output of esxcfg-info -o and look at how the values are listed.

esxcfg-auth

From the man page “…provides an easy way to configure your server to allow network based authentication as well as password complexity settings for your machine. It supports setting up your system to do authentication against an Active Directory Server, but not user management, as well as authentication against a NIS server, a Kerberos server, or an LDAP server. You can configure the way that passwords are stored and the complexity of the password when a user sets a new password.

This utility is experimental. It is likely to change.”

So let’s do something with it and update vmware-authd as well.

esxcfg-auth -enablead -addomain=domain.net -addc=adserver.domain.net -krb5realm=domain.net -krb5kdc=adserver.domain.net -krb5adminserver=adserver.domain.net

esxcfg-auth -passmaxdays=0

Update vmware-authd to allow AD logon

echo “#%PAM-1.0″ > /etc/pam.d/vmware-authd

echo “# Built by script” >> /etc/pam.d/vmware-authd

echo “auth sufficient /lib/security/pam_unix_auth.so shadow nullok” >> /etc/pam.d/vmware-authd

echo “auth required /lib/security/pam_krb5.so use_first_pass” >> /etc/pam.d/vmware-authd

echo “auth sufficient /lib/security/pam_ldap.so” >> /etc/pam.d/vmware-authd

echo “account required /lib/security/pam_unix_acct.so” >> /etc/pam.d/vmware-authd

echo “account sufficient /lib/security/pam_ldap.so” >> /etc/pam.d/vmware-authd

echo “vmware-authd updated”

esxcfg-boot

esxcfg-boot -h -help

-q -query boot|vmkmod

-p -update-pci

-b -update-boot

-d -rootdev UUID=<uuid>

-a -kernelappend <kernel append>

-r -refresh-initrd

-g -regenerate-grub

Queries cannot be combined with each other or other options.

Passing -p or -d enables -b even if it is not passed explicitly.

-b implies -g plus a new initrd creation.

-b and -r are incompatible, but -g and -r can be combined.

esxcfg-dumppart

From the man page “…provides an interface to query, set, and scan for diagnostic partitions on an ESX Server. That is, it can be used to configure or list the VMkernel crash dump partition.”

[root@vi3host root]# esxcfg-dumppart -l

VM Kernel Name Console Name Is Active Is Configured

vmhba0:0:0:10 /dev/sda10 yes yes

[root@vi3host root]# esxcfg-dumppart -f

Partition number 10 on vml.0000000000766d686261303a303a30 -> vmhba0:0:0:10 -> /dev/sda10

esxcfg-firewall

esxcfg-firewall “…provides an interface to query and modify the service console firewall settings.”

For example, if you want to open all of the ports on your firewall, issue the following commands:

esxcfg-firewall -allowIncoming

esxcfg-firewall -allowOutgoing

You might then want to re-enable the firewall, by doing the following:

esxcfg-firewall -blockIncoming

esxcfg-firewall -blockOutgoing

Rules are no fun without exceptions, so here’s how to add them to your ruleset. ESX comes with a number of pre-canned exceptions that can be added by name. Some examples are below:

esxcfg-firewall -enableService snmpd

esxcfg-firewall -enableService ntpClient

These are the services you can enable by name:

nfsClient

ftpServer

ntpClient

dellom

nisClient

vncServer

tmpLicenseClient

swISCSIClient

CIMHttpsServer

sshClient

snmpd

tmpAAMClient

vpxHeartbeats

smbClient

hpim

tmpHostVmdbServer

tmpHostdSOAPServer

ftpClient

sshServer

ibmdirector

CIMHttpServer

telnetClient

If, however, you want to do something non-standard (in this case for IBM Director), do this:

esxcfg-firewall -openPort 14247,tcp,out,IBMDirector

esxcfg-firewall -openPort 14247,tcp,in,IBMDirector

esxcfg-firewall -openPort 14247,udp,in,IBMDirector

esxcfg-firewall -openPort 14247,udp,out,IBMDirector

esxcfg-firewall -openPort 14248,tcp,out,IBMDirector

esxcfg-firewall -openPort 14248,tcp,in,IBMDirector

Once you’ve made your changes, restart the firewall service:

service firewall restart

esxcfg-info

From the man page “…provides a view of the internal state of the VMkernel and Service Console components. This tool is designed to provide information used in debugging and troubleshooting VMware ESX Servers.”

I suggest you pipe these to a text file, as the output is substantial, and takes an age to scroll through on my terminal.

Usage: esxcfg-info mode

-a, -all Print all information

-w, -hardware Print hardware information

-r, -resource Print resource information

-s, -storage Print storage information

-n, -network Print network information

-y, -system Print system information

-o, -advopt Print advanced options

-h, -help Print this message.

[root@vi3host root]# esxcfg-info -a >esxcfg-info.output

[root@vi3host root]# less esxcfg-info.output

esxcfg-init

From the man page “…is used to initialize pci device names and advanced configuration options for the VMkernel on system boot

and is NOT intended for use outside of initialization scripts.”

esxcfg-linuxnet

esxcfg-linuxnet -setup

-remove

-h -help

The -setup option cannot be combined with the -remove option.

Used to replicate the IP parameters assigned to vswif0 to eth0 in Linux troubleshooting mode

esxcfg-module

From the man page “…provides an interface into the driver modules loaded on system startup and allows modules to be disabled or added.”

Example usage:

[root@vi3host root]# esxcfg-module -l

Device Driver Modules

Module Type Enabled Loaded

vmkapimod vmkapimod true true

vmklinux linux true true

bnx2 nic true true

aacraid_esx30 scsi true true

[root@vi3host root]# esxcfg-module -q

vmkapimod vmkapimod

vmklinux linux

bnx2 nic

aacraid_esx30 scsi

esxcfg-mpath

esxcfg-mpath configures multipath settings for Fibre Channel or iSCSI LUNs.

This is useful to check your zoning on large SAN implementations.

[root@vi3host root]# esxcfg-mpath -l

Disk vmhba0:0:0 /dev/sda (139900MB) has 1 paths and policy of Fixed

Local 4:0.0 vmhba0:0:0 On active preferred

Disk vmhba1:0:1 /dev/sdb (307200MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:1 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:1 Standby

Disk vmhba1:0:2 /dev/sdd (307200MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:2 Standby preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:2 On active

Disk vmhba1:0:3 /dev/sde (307200MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:3 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:3 Standby

Disk vmhba1:0:4 /dev/sdg (204800MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:4 Standby preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:4 On active

Disk vmhba1:0:5 /dev/sdh (204800MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:5 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:5 Standby

Disk vmhba1:0:6 /dev/sdi (25600MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:6 Standby preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:6 On active

Disk vmhba1:0:7 /dev/sdj (25600MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:7 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:7 Standby

Disk vmhba1:0:8 /dev/sdl (691200MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:8 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:8 Standby

Disk vmhba1:0:31 /dev/sdf (20MB) has 1 paths and policy of Fixed

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:31 On active preferred

Disk vmhba1:0:10 /dev/sdc (691200MB) has 2 paths and policy of Most Recently Used

FC 28:0.0 210000e08b9d2a7a<->200400a0b829871d vmhba1:0:10 On active preferred

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:10 Standby

Disk vmhba2:0:31 /dev/sdk (20MB) has 1 paths and policy of Fixed

FC 28:0.1 210100e08bbd2a7a<->200500a0b829871e vmhba2:0:31 On active preferred

esxcfg-nas

esxcfg-nas provides an interface to manipulate NAS file systems on an ESX Server.

esxcfg-nas <options> [<label>]

-a|-add Add a new NAS filesystem to /vmfs volumes.

Requires -host and -share options.

-o|-host <host> Set the host name or ip address for a NAS mount.

-s|-share <share> Set the name of the NAS share on the remote system.

-d|-delete Unmount and delete a filesystem.

-l|-list List the currently mounted NAS file systems.

-r|-restore Restore all NAS mounts from the configuration file.

(FOR INTERNAL USE ONLY).

-h|-help Show this message.

For example, if you want to connect to a NAS

[root@vi3host root]# esxcfg-nas -a -o 100.100.100.253 -s /Test NFS02

esxcfg-nics

From the man page “…provides information about the Physical NICs in use by the VMkernel. This will print the VMkernel name for the NIC, its PCI ID, Driver, Link state, Speed, Duplex, and a shore PCI description of the card. It also allows users to set speed and duplex settings for a specific NIC.”

[root@vi3host root]# esxcfg-nics

esxcfg-nics <options> [nic]

-s|-speed <speed> Set the speed of this NIC to one of 10/100/1000/10000.

Requires a NIC parameter.

-d|-duplex <duplex> Set the duplex of this NIC to one of ‘full’ or ‘half’.

Requires a NIC parameter.

-a|-auto Set speed and duplexity automatically. Requires a NIC parameter.

-l|-list Print the list of NICs and their settings.

-r|-restore Restore the nics configured speed/duplex settings (INTERNAL ONLY)

-h|-help Display this message.

[root@vi3host root]# esxcfg-nics -l

Name PCI Driver Link Speed Duplex Description

vmnic0 03:00.00 bnx2 Up 1000Mbps Full Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T

vmnic1 06:00.00 bnx2 Up 1000Mbps Full Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T

vmnic2 08:03.00 tg3 Up 1000Mbps Full Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet

vmnic3 08:03.01 tg3 Up 1000Mbps Full Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet

esxcfg-pciid

esxcfg-rescan

esxcfg-rescan <vmkernel SCSI adapter name>

[root@vi3host root]# esxcfg-rescan vmhba0

Rescanning vmhba0…done.

On scsi0, removing: 0:0.

On scsi0, adding: 0:0.

[root@vi3host root]# esxcfg-rescan vmhba1

Rescanning vmhba1…done.

On scsi1, removing: 0:1 0:10 0:2 0:3 0:31 0:4 0:5 0:6 0:7 0:8.

On scsi1, adding: 0:1 0:10 0:2 0:3 0:31 0:4 0:5 0:6 0:7 0:8.

[root@vi3host root]# esxcfg-rescan vmhba2

Rescanning vmhba2…done.

On scsi2, removing: 0:31.

On scsi2, adding: 0:31.

[root@vi3host root]#

esxcfg-resgrp

From the man page “…provides an interface to manipulate resource group hierarchy.

Usage: esxcfg-resgrp [options] [resgrp path]

-a, -add rgname Add a new group under the path

-d, -delete Delete the resource group

-l, -list List all the resource groups

Or stats about a specific resource group

-restore Restore all resource groups from configuration

file (FOR INTERNAL USE ONLY).

-h, -help Show this message

esxcfg-route

From the man page “…provides an interface to manipulate VMkernel IP stack’s default route entry.”

esxcfg-route <options> [<network> [<netmask] <gateway>]

<network> can be specified in 2 ways:

* As a single argument in <Network>/<Mask> format

* Or as a <Network> <Netmask> pair.

<gateway> is either an IP address or ‘default’

-a|-add Add route, to the VMkernel,

requires network address (or default)

and gateway IP address

-d|-del Delete route from VMkernel.

Requires network address (or “default”)

-l|-list List configured routes for the service console

-r|-restore Restore route setting to configured values

on system start. (INTERNAL USE ONLY)

-h|-help Show this message.

[root@vi3host root]# esxcfg-route -l

VM Kernel Routes:

Network Netmask Gateway

192.168.100.0 255.255.255.0 Local Subnet

default 0.0.0.0 Local Subnet

esxcfg-swiscsi

From the man page “…provides an interface to configure Software iSCSI. if no arguments are given, esxcfg-swiscsi displays the help message.”

Usage: esxcfg-swiscsi options

-e, -enable Enable sw iscsi

-d, -disable Disable sw iscsi

-q, -query Check if sw iscsi is on/off

-s, -scan Scan for disk available through sw iscsi interface

-k, -kill Try to forcibly remove iscsi sw stack

-r, -restore Restore sw iscsi configuration from file

(FOR INTERNAL USE ONLY)

-h, -help Show this message

[root@vi3host root]# esxcfg-swiscsi -q

Software iSCSI is enabled

[root@vi3host root]# vmkiscsi-tool -h

vmkiscsi-tool -h -help

-R -discoveryStatus : Print discovery status.

-D -discovery

-S -static: Static Discovery Targets

-A -Authentication

-T -Target

-L -Lun

-P -Phba

-N -Network: network properties

-p -Pnp: Physical Network Portal properties

-t -ipv4AddrType

-i -ipv4Address

-d -dnsserver

-g -gateway

-s -subnetmask

-I -iSCSIname

-k -Alias

-e -ethernet: Link Status

-c -ipconfig: enable/disable DHCP, ARP redirect

-X -Reset

Subcommands

-l -list

-r -remove

-a -add

-m -authMethod : specify method for add/remove

-f -flag: set a discovery or authentication flag

adapterName

Combine -l with an option to display the current information.

[root@vi3host root]# vmkiscsi-tool vmhba40 -P -l

=========PHBA Properties for Adapter vmhba40=========

VENDOR : VMware

MODEL : VMware-Isoft

DESCRIPTION : VMware Software Initiator

SERIAL NUMBER :

=========Node Properties for Adapter vmhba40=========

NODE NAME VALID : 1

NODE NAME : iqn.1998-01.com.vmware:vi3host-3fc07e31

NODE ALIAS VALID : 1

NODE ALIAS : vi3host.network.internal

NODE NAME AND ALIAS SETTABLE: 1

So if you combine this command, vmkiscsi-tool and esxcfg-rescan, you can automate iscsi setup in scripted installs (useful when combined with UDA).

Set-up iSCSI Software Emulator and Force a Rescan of vmhba40

esxcfg-swiscsi -e

vmkiscsi-tool -D -a 192.168.100.139 vmhba40

esxcfg-swiscsi -s

esxcfg-rescan vmhba40

esxcfg-upgrade

esxcfg-upgrade -h -help

-g -convert-grub

-f -convert-fstab

-r -upgrade-pre-vmkernel

-o -upgrade-post-vmkernel

The -g option may only be used with the -r option.

esxcfg-vmhbadevs

From the man page “…provides information about the LUNs available on the ESX Server. By default, the command will print a mapping of vmhbaX:X:X names to console /dev/ names.”

This is useful when you want to set VMFS filesystem alignment on EMC CLARiiON arrays but don’t know what LUNs are mapped to what devices in fdisk.

esxcfg-vmhbadevs

esxcfg-vmhbadevs <options>

Print the mappings between vmhba names and /dev names

-m|-vmfs Print mappings for VMFS volumes to their

Service Console partitions and vmhba names.

-q|-query Print mapping in 2.5 compatibility mode to mimic

vmkpcidivy -q vmhba_devs.

-h|-help Show this message.

[root@vi3host root]# esxcfg-vmhbadevs -m

vmhba0:0:0:2 /dev/sda2 4667158e-899da7f8-ae73-00145eccccba

vmhba40:1:0:1 /dev/sdd1 4692c9ad-1a501875-f252-00145eccccba

vmhba40:0:0:1 /dev/sdb1 4692c987-14967ce4-a9a1-00145eccccba

[root@vi3host root]# esxcfg-vmhbadevs -q

vmhba0:0:0 /dev/sda

vmhba1:0:1 /dev/sdb

vmhba1:0:2 /dev/sdd

vmhba1:0:3 /dev/sde

vmhba1:0:4 /dev/sdg

vmhba1:0:5 /dev/sdh

vmhba1:0:6 /dev/sdi

vmhba1:0:7 /dev/sdj

vmhba1:0:8 /dev/sdl

vmhba1:0:31 /dev/sdf

vmhba1:0:10 /dev/sdc

vmhba2:0:31 /dev/sdk

esxcfg-vmknic

From the man page “…provides an interface to configure VMkernel NIC. if no arguments are given, esxcfg-vmknic displays the help message. The Portgroup option specifies the portgroup to which the VMkernel NIC is associated.”

esxcfg-vmknic <options> [[<portgroup>]]

-a|-add Add a VMkernel NIC to the system,

requires IP parameters and portgroup name.

-d|-del Delete VMkernel NIC on given portgroup.

-e|-enable Enable the given NIC if disabled.

-D|-disable Disable the given NIC if enabled.

-l|-list List VMkernel NICs.

-i|-ip <X.X.X.X> The IP address for this VMkernel NIC.

Setting an IP address requires that the

-netmask option be given in same command.

-n|-netmask <X.X.X.X> The IP netmask for this VMkernel NIC.

Setting the IP netmask requires that the -ip

option be given in the same command.

-r|-restore Restore VMkernel TCP/IP interfaces from

Configuration file (FOR INTERNAL USE ONLY).

-h|-help Show this message.

[root@vi3host root]# esxcfg-vmknic -l

Port Group IP Address Netmask Broadcast MAC Address MTU Enabled

VMOTION 10.0.0.2 255.0.0.0 10.255.255.255 00:50:56:61:d8:2b 1514 true

esxcfg-vswif

From the man page “…provides an interface to configure Service Console NIC. if no arguments are given, esxcfg-vswif displays the help message. VSWIF is the name of the interface. This should begin with ‘vswif’ followed by a unit number, for example ‘vswif0′ for the first Service Console interface.”

esxcfg-vswitch

From the man page “…provides an interface for adding, removing, and modifying virtual switches and their settings. By default, there is a single virtual switch called vSwitch0.”

Sample DEV network configuration script

# !/bin/bash

# Tear down and rebuild network

# Remove current config

echo “Removing current network configuration”

/usr/sbin/esxcfg-vswitch -U vmnic0 vSwitch0

/usr/sbin/esxcfg-vswitch -U vmnic1 vSwitch0

/usr/sbin/esxcfg-vswif -d vswif0

/usr/sbin/esxcfg-vswitch -d vSwitch0

service mgmt-vmware restart

# Recreate SC

echo “Reconfiguring Service Console NIC”

cp /etc/vmware/esx.conf /tmp/esx.conf.bak

/usr/sbin/esxcfg-vswitch -a vSwitch0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic1

/usr/sbin/esxcfg-vswitch vSwitch0 -A “Service Console” -p “Service Console”

/usr/sbin/esxcfg-vswif -a vswif0 -p “Service Console” -i 192.168.100.7 -n 255.255.255.0

route add default gw 192.168.100.253

# echo “GATEWAY=192.168.100.253″ >> /etc/sysconfig/network

echo “Service Console configured”

# Create VM Network

echo “Configuring VM Network”

/usr/sbin/esxcfg-vswitch vSwitch0 -A VMnetwork -p VMnetwork

echo “VM network configured”

# Setup VMotion

echo “Setting up VMotion”

/usr/sbin/esxcfg-vswitch vSwitch0 -A VMotion

/usr/sbin/esxcfg-vmknic -a VMotion -i 10.0.0.3 -n 255.255.255.0

/usr/sbin/esxcfg-route 10.0.0.3

echo “Activating vMotion”

service mgmt-vmware restart

sleep 20

vimsh -n -e “/hostsvc/vmotion/vnic_set portgroup4″

echo “VMotion configured”

# Create internal-only vswitch

echo “Configuring Private Network”

/usr/sbin/esxcfg-vswitch -a vSwitch1

/usr/sbin/esxcfg-vswitch vSwitch1 -A Internal -p Internal

echo “Private network configured”

# Setup iSCSI Network

echo “Configuring iSCSI Network”

esxcfg-vswitch -A iSCSI vSwitch0

esxcfg-vmknic -a iSCSI -i 192.168.200.100 -n 255.255.255.0

esxcfg-vswitch -A iSCSI-COS vSwitch3

esxcfg-vswif -a vswif1 -p iSCSI-COS -i 192.168.200.101 -n 255.255.255.0

Technorati : esx, esx cli config, esxcfg, esxi, rcli, service console, vima

Del.icio.us : esx, esx cli config, esxcfg, esxi, rcli, service console, vima

Zooomr : esx, esx cli config, esxcfg, esxi, rcli, service console, vima

Flickr : esx, esx cli config, esxcfg, esxi, rcli, service console, vima

So I wanted to talk on best practices for your Service Console, what it’s for, and why and how to make it redundant. As that’s a lot of information, we’ll break it out in some parts.

This first part we’ll talk about what the service console is, and what it does.

So what is the ‘Service Console’? It is the “Console Operating System” that is responsible for user interaction with ESX. In the full ESX 3/3.5 (as in not ESXi), the service console is a VM that is based on RHEL 3. In fact this is often what keeps folks confused and arguing that “ESX is just Linux”. However, this is very much not the case. VMware has also tried to do everything possible to deemphasize the use of the service console, and, in the ESXi release, and perhaps future releases, it will go away completely, to be replaced by things like VIMA (VMware Infrastructure Management Assistant).

What’s it do this service console? I suppose my single line above wasn’t quite a good enough, but that is more or less what it is responsible for: User interaction with ESX. This is a bigger task that it first appears when you consider the various ways one can communicate with ESX:

• Command Line Interface (CLI)
• SSH Access
• Web Management (this includes SOAP, and the other various API’s)
• Using the VMware Infrastructure Client
• Communication with Virtual Center

The Service Console is also responsible for keeping this communication secure. This is done in several ways as well.

• ESX Firewall (iptables)
• SSL to the web services
• Standard Linux authentication methods (keep your eyes open for a future article on getting ESX to talk to AD)

The last two things the Service Console does are manage access to ‘non-critical’ hardware, and provide support applications. Interesting, no? Lets explore those further.

What is ‘non-critical’ hardware? ‘Non-critical’ hardware is anything outside the Core 4 (Memory, Disk, Networking, CPU), things like serial ports and cd-roms. This keeps the VMKernel light and agile to better support your VM’s.

The last bit is support applications, and these are simply applications, like vm-support, or the esxcfg-* set of commands that allow you to keep the host running in tip top shape. With ESXi, these are handled either via the VIMA, or rCLI, which can both be downloaded from VMware’s site.

So, we’ve now covered what the Service Console is, and what it does. Next go around we’ll cover the why and how of making it redundant.

Technorati : esx, esx3.5, service console, vmware

Del.icio.us : esx, esx3.5, service console, vmware

Zooomr : esx, esx3.5, service console, vmware

Flickr : esx, esx3.5, service console, vmware