Preparing Internal Flash Cards on Dell R710 for ESXi 5.x Installation.

On my last project I worked once again with the Dell PowerEdge R710 servers but this time the customer followed our advice and purchased the servers with internal 2 GB flash cards. Auto deploy would of course be more awesome but due to the limited knowledge of vSphere at our customer, we decided to go with the flash cards. During the installation of ESXi 5.0 I noticed something unusual. The flash cards were detected correctly in the BIOS but the ESXi installer failed to install the software. Apparently the flash cards were not prepared for the installation.

Here is a quick guide on how to prepare the flash cards for the installation of vSphere 5 Hypervizor.

1. Place the flash card in the card reader of your laptop or PC.
2. Windows will detect the card and will ask you to format it. In my case it failed.
3. Fire up Diskpart and create a new partition.
4. First, list the disks in your system: DISKPART> list disk
5. Select the correct disk (in my case disk 2): DISKPART>  select disk 2
6. Create new partition: DISKPART> create partition primary
7. Format the disk with FAT32 as you normally would
8. Place the flash card back in the server, power on and go to BIOS.
9. Make sure the flash card is the first boot device in the Boot Sequence settings.
10. Verify that the USB Flash Drive Emulation Type is set to Hard Disk.
11. Save your settings and reboot the server.

Wait for the ESXi installer to start and follow the default procedure. The vSphere 5 Hypervizor should now install correctly.

Cheers!

- Marek.Z

P.S. Don’t forget to wear an antistatic wrist strap when you remove and install the flash cards in the server

Troubleshooting IP-Hash load balancing on a NFS storage network.

When configuring a NFS storage network at one of our customers some time ago, I noticed that the ESXi host wasn’t utilizing all NICs assigned to the NIC team for the VMkernel traffic. After some research, I have found this article written by Frank Denneman a while ago and this VMware KB document. According to the blog post and the article mentioned above, this issue may occur if the calculated hash returns the same result based on the source IP and both destination IP’s. Before we jump in to troubleshooting, let’s take a look at what is exactly going wrong.

Setup

The setup consisted of 4 Dell R710 ESXi hosts connected through 2 stacked Cisco 2960 switches to a NetApp FAS3210 filer. Four NICs per server have been dedicated to NFS storage and cabled in a redundant configuration (2 per switch in EtherChannel). See the drawing for more details.

Solution

To see what is going wrong, we need to calculate the IP-Hash manually. The formula is:

Source IP XOR Destination IP = x MOD y = z where:

Source IP = VMkernel IP address in Hexadecimal
Destination IP = IP address of the NFS filer in Hexadecimal
x = Exclusive OR operation output
y = Number of physical NICs
z = Modulo operation output

First, let’s calculate the IP-Hash value of the IP addresses in the current setup. To do this, we need to convert the IP addresses from decimal to hexadecimal. I used the BitCricket IP Calculator to do the conversion.

Next, calculate the IP-Hash with the formula specified earlier and take a look at the outcome. You can use Windows Calculator to do this, just set the view to Programmer and make sure it is set to Hex and Qword.

1.    C0A86465 XOR C0A86478 = 1D MOD 4 = 1
2.    C0A86465 XOR C0A86482 = E7 MOD 4 = 3
3.    C0A86465 XOR C0A8648C = E9 MOD 4 = 1
4.    C0A86465 XOR C0A86496 = F3 MOD 4 = 3

As you can see, the values are not unique. That’s what causes the problem. The IP-Hash calculation only returns 2 different values instead of 4. To correct this, we need to reconfigure the destination IP addresses (on the NFS Filer) so that every IP-Hash calculations return a unique value. The IP addresses have been reconfigured as follow:

Let’s have a look at the IP-Hash calculations now.

1.    C0A86465 XOR C0A8646F = A MOD 4 = 2
2.    C0A86465 XOR C0A86470 = 15 MOD 4 = 1
3.    C0A86465 XOR C0A86471 = 14 MOD 4 = 0
4.    C0A86465 XOR C0A86472 = 17 MOD 4 = 3

As you can see, the IP-Hash calculation now returns unique values in all four cases. This will now allow utilization of all four NICs from the ESXi host to the NFS Filer.

Cheers!

- Marek.Z

Book review: VMware vSphere 5 Clustering Technical Deepdive.

The VMware vSphere 5 Clustering Technical Deepdive book has been published a while ago and probably most of you already read it. I purchased the book right after the release and I was reading the book really slowly due to some time management issues. Today, I have finally finished reading this awesome book that was written by Duncan Epping (@DuncanYB) and Frank Denneman (@FrankDenneman) and now it’s time for a little review.

This book consists of more than 340 pages about the most important key features of VMware vSphere 5: the VMware High Availability (HA), VMware Distributed Resource Scheduler (DRS) including Distributed Power Management (DPM) and new in this book is the vSphere Storage DRS (SDRS) which was released in vSphere 5.

The first part of the book concentrates on the vSphere HA and includes topics about the redesigned HA stack, new components like the Fault Domain Manager (FDM) and the new Master/Slave node concept. The new “Datastore Heartbeating” feature and the state where a cluster can be isolated or partitioned are also explained in this chapter.

The second part of the book gives you detailed information about the vSphere DRS. It includes topics about some new features of DRS in vSphere 5 as well as vMotion and EVC. The DRS and DPM calculations and recommendations are extensively explained with good examples and of course the resource pools and it’s control.

Part three of the book is about the new vSphere Storage DRS feature in vSphere 5. In my opinion, this is one of the coolest features of vSphere 5. The chapters in this part of the book will discuss all key features of SDRS like resource aggregation, load balancing, datastore maintenance mode but also how it is calculated and when it is invoked. The Storage vMotion (including the new Mirror Mode), Storage I/O Control and some new enhancement in vSphere 5 are also described in this part.

Part four, called “Integration” is an interesting chapter. It describes the integration points of HA, DRS and SDRS. For example, the sub-topic HA and Stateless ESXi describe what happens with the HA configuration when a host reboots. The topics are small, to the point and give you good understanding about the integration points in vSphere 5.

This is just a small summary of the main topics of the VMware vSphere 5 Clustering Technical Deepdive book, there is a lot more. Just like in the previous version of the book, the basic design principle quotes are also included but not indexed which was the case in the previous book. The technical level is high but the topics are interesting for all VMware enthusiasts. The figures and tables are clear and make some topics easier to understand.

Summary

Although the old HA and DRS Technical Deepdive book is great, the new vSphere 5 Clustering book is even better. I know from twitter that Duncan and Frank worked really hard on this book and managed to complete it just in time the vSphere 5 was released. I guess they really have raised the bar for themselves this time. Just like I said earlier, this book is great and this is the bible you must have read if you are a VMware System Administrator, Engineer, Consultant or Architect.

You can order your copy here or here.

Cheers!

- Marek.Z

Dutch VMUG 2011 – recap.

Just like in 2010, the Dutch VMUG event was held once again at NBC in Nieuwegein. More than 700 VMware enthusiasts gathered this day to attend the sessions and the workshops. The agenda was full of good sessions so it was hard to choose a session to attend. Luckily, some sessions were repeated. Here is a small recap of the sessions and the workshop that I attended.

The opening session was (as always) presented by Victor van den Berg and this years general keynote session was presented by Willem van Enter and Jeremy van Doorn from VMware. Good general info about working in the cloud and the products that VMware offers to accompany your journey to the cloud. Almost forgot, the Monster VM appeared on the stage, this was kind of weird because he was “sleeping with the fishes” after an encounter with the @Dutch_vMaffia at VMworld 2011 in Copenhagen…

Sessions

• Een dag uit het leven van een cloud consument: presented by Stefan Willems and Marcel Steenman (VMUG/Platani). This was really hilarious presentation! A comparison between a cloud infrastructure and a family. Really funny but, with no disrespect to the presenters, not very informative. Although I appreciate your enthusiasm.
• Mythbusters go virtual: presented by Eric Sloof (VMUG/NTPro.nl) and Mattias Sundling (VMUG/Quest). I really enjoyed this session at VMworld 2011 in Copenhagen and I heard on twitter that there were some new myths so I decided to check it out! Great session guys!

Workshop

After the lunch I attended the vSphere Auto Deploy workshop that was organized by XTG. Marcel van Oss was leading the workshop and explained the Auto Deploy feature in a big picture. After the instruction, hands-on time! The instructions were clear and in no time I had deployed my first stateless ESXi host. I really love this feature in vSphere 5. Great workshop! Thanks XTG.

Sessions

• vSphere Heathcheck: presented by Gabrie van Zanten (VMUG/gabesvirtualworld.com/OpenLine). A lot of info about tools and best practices that Gabrie uses when he is performing a health check on a vSphere environment. Good presentations, helpful tips, too bad Gabrie run out of time.
• vCenter Operations: presented by Andre van der Werff (VMware). General info about the product and good explanation about the features. I really liked the demo that Andre presented; it gives you a bit more “feel” how vCO works.

So, to recap, the Dutch VMUG 2011 event was great once again. Good sessions and speakers made this event a success. I would like to thank the Dutch VMUG and the staff for organizing this great event once again. Special thanks to @Veeam for sponsoring the VMware vSphere 5 Clustering Technical Deepdive book for all Dutch VMUG attendees!

Cheers!

- Marek.Z

Veeam releases Backup & Replication version 6.

Veeam has just released version 6 of their Backup & Replication software that provides the most comprehensive protection for your virtual environment by offering backup and replication in a single solution so you can have the flexibility to meet different recovery objectives for the VMs. So, let’s have a look at the specs of the new release and see what is new.

• 2-in-1 backup and replication: Offers backup and replication in a single solution with one price point.
• Multi-hypervisor support: Protect all VMware vSphere and Microsoft Hyper-V VMs with a single product, from a single console, minimizing cost and complexity.
• Synthetic full backups: Takes previous full and incremental backups to make a new full backup recovery point without duress on primary storage.
• Built-in de-duplication and compression: Cuts backup storage requirements up to 75% by compressing backup and network traffic and eliminating duplicate blocks of data.
• Near-continuous data protection: Allows you to capture changes and then updates VM images as often as every few minutes for on-site and off-site replication. This allows you to avoid having to buy additional expensive/specialized products.
• Agentless: There are no agents for you to license or deploy, manage, monitor or update on hosts or VMs.

This release comes also with vPower which is a technology that can help you meet recovery time and recovery point objectives for your VMware environment with less time, effort, cost and risk. The vPower offers the following features:

• Instant VM Recovery: Restore an entire machine from backup in a matter of minutes.
• U-AIR (Universal Application-Item Recovery): Recover individual objects from any virtualized application, without adding application-specific agents.
• SureBackup: Automatically verify the recoverability of every backup, every time.
• On-Demand Sandbox: Restore from any point (full or incremental) and run production VMs from the backup file in a sandbox for testing or troubleshooting.
• Universal File Level Recovery: Quickly recover an individual file from any file system.

To learn more please visit Veeam website.

Cheers!

Marek.Z