Skip navigation

Blog Posts

Total : 3,741

Blog Posts

In this days more and more customer start using hardware VTEP instead of VMK on esxi. Stories about why customer use HWVTEP varies between performance , capacity or already existing hardware in place. To hones in my experience I didn;t see much of benefit choosing HWVTEP against esxi VTEP ones. But what is definitely happens is introducing complexity in the network and troubleshooting issues. In this post can be found useful steps to troubleshoot this kind of environments.

First before we start with any kind of troubleshooting examples lest follow a normal process of identifying how packect a travel between Physical PC and VM in vSphere/ NSX environment






As we can see from the example we have a flow

1 PC sends and ARP request which is broadcast to find VM 1

2 HW VTEP sends PC MAC learn on port to NSX Controller

3 HW VTEP learns VM mac from Controller (Controller should know mac address of the VM via a <JOIN> message send to controller when VM is attached to VNI)

4 vDS prepared for NSX shares VM MAC with Controller

5 NSX vDS request remote MAC of PC from controllers which should be shared between HW VTEP and Controller.

6 VM sends ARP back to PC via tunnel between ESXI and HW VTEP

7 HW VTEP forwards ARP response to PC


with this steps should be communication happens.

As can be seen there is a lot communication handshakes between HW VTEP and NSX components, and here most of the issues happens and we need to focus on this.

Main troubleshooting area for HW VTEP is the HW VTEp it self.

From NSX side we just need to ensure that environment is ready like host preparation.

Most common configuration are

  • Bind to non-existing physical ports
  • ToR certificate is not configured properly
  • Not connecting ToR to anyone of controllers nodes. HWVTEP need to be connected to one of the controllers and the controller push OVSDB transaction to make ToR connected to all controllers.
  • Forget to prepare host which is necessary, meaning install vibs.

If we are sure that all this look OK and no alerts or strange messages in NSX , then we can go to ToR and check <show bfd neighbors>

When we add replication hosts and bindings to Logical Switch need to check BFD is up or not BFD will not be up until we not specify any replication host and binding to Logical Switch

Please refer to documentation for specific vendor as an example is Juniper…


Summary : most of trouble shooting in HWVTEP  need to be done on ToR not in NSX environment.  To confirm what where and where is the packet (as my Colleague Jose say's) we need to capture 

VMGator Novice

Hello World ! in Hal Venoff's Blog

Posted by VMGator Feb 28, 2018

More information to come.............Stay tuned.

Rideout421 Enthusiast

Rideout421 in Michael Rideout's Blog

Posted by Rideout421 Feb 27, 2018

Financially I'm unable to afford a $5,000 certification class at New Horizon's. I'm working using self study resources to stay sharp on VMware solutions. Recently from some training i'm taking I found out about VMware Foundations which is $125.00 I plan to study up and take that exam to get it under my belt. The training that I found references 6.0 however I found 6.5 available so I'm going to take 6.5.

Have you ever needed or wanted to dynamically get the users within an Active Directory group through vRA? The process I hoped would already exist but, unfortunately as I found out, the existing vRO Active Directory plug-in does not contain a pre-built workflow or action to handle this. As I typically do, I find my own way, but my trials and tribulations can easily be to your benefit as you’ll see. The use case for getting users within a group can vary. In my case, there is a customer for whom I’m building a CMP and they wish to dynamically pull in all the users within one specific AD group to which they can assign provisioned machines for metadata purposes. This metadata would then be sent in a custom notification to a recipient which tells them of the new machine(s) that has/have been built and this user chosen at request time. So this group membership needs to be displayed in a request form in a drop-down and the requestor able to pick a member. With a little bit of vRO work and a cool trick in vRA I learned, this is pretty easy and others may find it useful.


              In my test environment here, I’m on vRA 7.3 with an external vRO 7.3. None of that should matter as all these steps should be applicable to earlier versions as well. That said, I updated my vRO Active Directory plug-in to the latest available here. The newest version is v3.0.7 as of this writing. Once the plug-in is updated, run the “Add an Active Directory server” workflow found in Library -> Microsoft -> Active Directory -> Configuration.


Step 1a is fairly self-explanatory. Host in my case is just a single AD server. Base is the root object of your AD in DN format.



Since this is for use in vRA, we want to use a shared session with a service account and avoid per-user sessions.


What’s nice in step 1c is you can provide multiple AD servers that can be attempted. The algorithms are Single Server, Round Robin, or Fail-Over.



And, finally, add a timeout value to wait before failing. I chose 5 seconds but, depending on your AD size, this may need to be longer.


Once the workflow successfully completes, verify you have it in your inventory explorer.



The tree here shows it is indeed working.


Once you’re good there, import the action I’ve pre-built. The JavaScript required in order for it to function is fairly rudimentary:


userArray = new Array();

var usersInGroup = userGroup.userMembers

for each (user in usersInGroup) {





return userArray;


I’ve commented out a couple lines that allow logging the user names to System just for development and troubleshooting purposes to ensure the results are returned and in the correct format.


With this action imported, flip over to vRA. Go create a new Custom Property definition. In the one below, I’m calling it CZAD.Users and choosing to get external values from the new action.


Click on the userGroup input parameter and edit it. The value is going to be the identifier in vRO’s AD inventory that corresponds to the user group which we want to list the users.




Flip back over to vRO and browse in your AD inventory tree to find the group whose members you wish to list. Click the user group in the tree and check the General tab.



You see VSO ID as shown above? This is the value we’ll copy and paste into the input parameter definition back in vRA. This is a unique ID which essentially is an API “shortcut” to reference this one specific group and no other. With this, we can avoid having to hardcode the name of this group into our action or pull it from some other place in vRO. Since userGroup is the input object, we can simply supply this value with vRA and be done with it—so one place to go if you wish to change that group later.


With this VSO ID copied and pasted into the input parameter, save the custom property. Let’s add it to a blueprint and see if it works.



Go to the request form now.



And boom, there are the users in that AD group! Don’t believe me? Go check the membership for yourself in AD to compare.



And there you have it! With this simple action, you can pick a user from an AD user group and consume it as a string-based custom property anywhere you like in vRA.


Very simple little action that does a simple thing, but it solves a small use case and others may find it useful as well.



Download getADGroupMembership from VMware {code}

Use NVMe U.2 SFF 8639 disk drive form factor SSD in PCIe slot

server storage I/O data infrastructure trends

Need to install or use an Intel Optane NVMe 900P or other Nonvolatile Memory (NVM) Express  NVMe based U.2 SFF 8639 disk drive form factor Solid State Device (SSD) into PCIe a slot?


For example, I needed to connect an Intel Optane NVMe 900P U.2 SFF 8639 drive form factor SSD into one of my servers using an available PCIe slot.


The solution I used was an carrier adapter card such as those from Ableconn (PEXU2-132 NVMe 2.5-inch U.2 [SFF-8639] via among other global venues.


Top Intel 750 NVMe PCIe AiC SSD, bottom Intel Optane NVMe 900P U.2 SSD with Ableconn carrier


The above image shows top an Intel 750 NVMe PCIe Add in Card (AiC) SSD and on the bottom an Intel Optane NVMe 900P 280GB U.2 (SFF 8639) drive form factor SSD mounted on an Ableconn carrier adapter.


NVMe server storage I/O sddc

NVMe Tradecraft Refresher

NVMe is the protocol that is implemented with different topologies including local via PCIe using U.2 aka SFF-8639 (aka disk drive form factor), M.2 aka Next Generation Form Factor (NGFF) also known as "gum stick", along with PCIe Add in Card (AiC). NVMe accessed devices can be installed in laptops, ultra books, workstations, servers and storage systems using the various form factors. U.2 drives are also refereed to by some as PCIe drives in that the NVMe command set protocol is implemented using PCIe x4 physical connection to the devices. Jump ahead if you want to skip over the NVMe primer refresh material to learn more about U.2 8639 devices.


data infrastructure nvme u.2 8639 ssd
Various SSD device form factors and interfaces


In addition to form factor, NVMe devices can be direct attached and dedicated, rack and shared, as well as accessed via networks also known as fabrics such as NVMe over Fabrics.


The many facets of NVMe as a front-end, back-end, direct attach and fabric


Context is important with NVMe in that fabric can mean NVMe over Fibre Channel (FC-NVMe) where the NVMe command set protocol is used in place of SCSI Fibre Channel Protocol (e.g. SCSI_FCP) aka FCP or what many simply know and refer to as Fibre Channel. NVMe over Fabric can also mean NVMe command set implemented over an RDMA over Converged Ethernet (RoCE) based network.


NVM and NVMe accessed flash SCM SSD storage


Another point of context is not to confuse Nonvolatile Memory (NVM) which are the storage or memory media and NVMe which is the interface for accessing storage (e.g. similar to SAS,


SATA and others). As a refresher, NVM or the media  are the various persistent memories (PM) including NVRAM, NAND Flash, 3D XPoint along with other storage class memories (SCM) used in SSD (in various packaging).


Learn more about 3D XPoint with the following resources:


Learn more (or refresh) your  NVMe server storage I/O knowledge, experience tradecraft skill set with  this post here. View this piece here looking at NVM vs. NVMe and how one is the media where data is stored, while the other is an access protocol (e.g. NVMe). Also  visit to view additional NVMe tips, tools, technologies, and related resources.

NVMe U.2 SFF-8639 aka 8639 SSD

On quick glance, an NVMe U.2 SFF-8639 SSD may look like a SAS small form factor (SFF) 2.5" HDD or SSD. Also, keep in mind that HDD and SSD with SAS interface have a small tab to prevent inserting them into a SATA port. As a reminder, SATA devices can plug into SAS ports, however not the other way around which is what the key tab function does (prevents accidental insertion of SAS into SATA). Looking at the left-hand side of the following image you will see an NVMe SFF 8639 aka U.2 backplane connector which looks similar to a SAS port.


Note that depending on how implemented including its internal controller, flash translation layer (FTL), firmware and other considerations, an NVMe U.2 or 8639 x4 SSD should have similar performance to a comparable NVMe x4 PCIe AiC (e.g. card) device. By comparable device, I mean the same type of NVM media (e.g. flash or 3D XPoint), FTL and controller. Likewise generally an PCIe x8 should be faster than an x4, however more PCIe lanes does not mean more performance, its what's inside and how those lanes are actually used that matter.


NVMe U.2 8639 2.5" 1.8" SSD driveNVMe U.2 8639 2.5 1.8 SSD drive slot pin
NVMe U.2 SFF 8639 Drive (Software Defined Data Infrastructure Essentials CRC Press)


With U.2 devices the key tab that prevents SAS drives from inserting into a SATA port is where four pins that support PCIe x4 are located. What this all means is that a U.2 8639 port or socket can accept an NVMe, SAS or SATA device depending on how the port is configured. Note that the U.2 8639 port is either connected to a SAS controller for SAS and SATA devices or a PCIe port, riser or adapter.


On the left of the above figure is a view towards the backplane of a storage enclosure in a server that supports SAS, SATA, and NVMe (e.g. 8639). On the right of the above figure is the connector end of an 8639 NVM SSD showing addition pin connectors compared to a SAS or SATA device. Those extra pins give PCIe x4 connectivity to the NVMe devices. The 8639 drive connectors enable a device such as an NVM, or NAND flash SSD to share a common physical storage enclosure with SAS and SATA devices, including optional dual-pathing.


More PCIe lanes may not mean faster performance, verify if those lanes (e.g. x4 x8 x16 etc) are present just for mechanical (e.g. physical) as well as electrical (they are also usable) and actually being used. Also, note that some PCIe storage devices or adapters might be for example an x8 for supporting two channels or devices each at x4. Likewise, some devices might be x16 yet only support four x4 devices.


NVMe U.2 SFF 8639 PCIe Drive SSD FAQ

Some common questions pertaining NVMe U.2 aka SFF 8639 interface and form factor based SSD include:


Why use U.2 type devices?


Compatibility with what's available for server storage I/O slots in a server, appliance, storage enclosure. Ability to mix and match SAS, SATA and NVMe with some caveats in the same enclosure. Support higher density storage configurations maximizing available PCIe slots and enclosure density.


Is PCIe x4 with NVMe U.2 devices fast enough?


While not as fast as a PCIe AiC that fully supports x8 or x16 or higher, an x4 U.2 NVMe accessed SSD should be plenty fast for many applications. If you need more performance, then go with a faster AiC card.


Why not go with all PCIe AiC?


If you need the speed, simplicity, have available PCIe card slots, then put as many of those in your systems or appliances as possible. Otoh, some servers or appliances are PCIe slot constrained so U.2 devices can be used to increase the number of devices attached to a PCIe backplane while also supporting SAS, SATA based SSD or HDDs.


Why not use M.2 devices?


If your system or appliances supports NVMe M.2 those are good options. Some systems even support a combination of M.2 for local boot, staging, logs, work and other storage space while PCIe AiC are for performance along with U.2 devices.


Why not use NVMeoF?


Good question, why not, that is, if your shared storage system supports NVMeoF or FC-NVMe go ahead and use that, however, you might also need some local NVMe devices. Likewise, if yours is a software-defined storage platform that needs local storage, then NVMe U.2, M.2 and AiC or custom cards are an option. On the other hand, a shared fabric NVMe based solution may support a mixed pool of SAS, SATA along with NVMe U.2, M.2, AiC or custom cards as its back-end storage resources.


When not to use U.2?


If your system, appliance or enclosure does not support U.2 and you do not have a need for it. Or, if you need more performance such as from an x8 or x16 based AiC, or you need shared storage. Granted a shared storage system may have U.2 based SSD drives as back-end storage among other options.

How does the U.2 backplane connector attach to PCIe?


Via enclosures backplane, there is either a direct hardwire connection to the PCIe backplane, or, via a connector cable to a riser card or similar mechanism.


Does NVMe replace SAS, SATA or Fibre Channel as an interface?


The NVMe command set is an alternative to the traditional SCSI command set used in SAS and Fibre Channel. That means it can replace, or co-exist depending on your needs and preferences for access various storage devices.


Who supports U.2 devices?


Dell has supported U.2 aka PCIe drives in some of their servers for many years, as has Intel and many others. Likewise, U.2 8639 SSD drives including 3D Xpoint and NAND flash-based are available from Intel among others.


Can you have AiC, U.2 and M.2 devices in the same system?


If your server or appliance or storage system support them then yes. Likewise, there are M.2 to PCIe AiC, M.2 to SATA along with other adapters available for your servers, workstations or software-defined storage system platform.

NVMe U.2 carrier to PCIe adapter

The following images show examples of mounting an Intel Optane NVMe 900P accessed U.2 8639 SSD on an Ableconn PCIe AiC carrier. Once U.2 SSD is mounted, the Ableconn adapter inserts into an available PCIe slot similar to other AiC devices. From a server or storage appliances software perspective, the Ableconn is a pass-through device so your normal device drivers are used, for example VMware vSphere ESXi 6.5 recognizes the Intel Optane device, similar with Windows and other operating systems.


intel optane 900p u.2 8639 nvme drive bottom view
  Intel Optane NVMe 900P U.2 SSD and Ableconn PCIe AiC carrier


The above image shows the Ableconn adapter carrier card along with NVMe U.2 8639 pins on the Intel Optane NVMe 900P.


intel optane 900p u.2 8639 nvme drive end view
Views of Intel Optane NVMe 900P U.2 8639 and Ableconn carrier connectors


The above image shows an edge view of the NVMe U.2 SFF 8639 Intel Optane NVMe 900P SSD along with those on the Ableconn adapter carrier. The following images show an Intel Optane NVMe 900P SSD installed in a PCIe AiC slot using an Ableconn carrier, along with how VMware vSphere ESXi 6.5 sees the device using plug and play NVMe device drivers.


NVMe U.2 8639 installed in PCIe AiC Slot
Intel Optane NVMe 900P U.2 SSD installed in PCIe AiC Slot


NVMe U.2 8639 and VMware vSphere ESXi
How VMware vSphere ESXi 6.5 sees NVMe U.2 device


Intel NVMe Optane NVMe 3D XPoint based and other SSDs

Here are some links to various Intel Optane NVMe 3D XPoint based SSDs in different packaging form factors:


Here are some links to various Intel and other vendor NAND flash based NVMe accessed SSDs including U.2, M.2 and AiC form factors:

Note in addition to carriers to adapt U.2 8639 devices to PCIe AiC form factor and interfaces, there are also M.2 NGFF to PCIe AiC among others. An example is the Ableconn M.2 NGFF PCIe SSD to PCI Express 3.0 x4 Host Adapter Card.


In addition to,, Ebay and many other venues carry NVMe related technologies. The Intel Optane NVMe 900P are newer, however the Intel 750 Series along with other Intel NAND Flash based SSDs are still good price performers and as well as provide value. I have accumulated several Intel 750 NVMe devices over past few years as they are great price performers. Check out this related post Get in the NVMe SSD game (if you are not already).

Where To Learn More

View additional NVMe, SSD, NVM, SCM, Data Infrastructure and related topics via the following links.


Additional  learning experiences along with  common questions (and answers), as well as  tips can be found in  Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

NVMe accessed storage is in your future, however there are various questions to address including exploring your options for type of devices, form factors, configurations among other topics. Some NVMe accessed storage is direct attached and dedicated in laptops, ultrabooks, workstations and servers including PCIe AiC, M.2 and U.2 SSDs, while others are shared networked aka fabric based. NVMe over fabric (e.g. NVMeoF) includes RDMA over converged Ethernet (RoCE) as well as NVMe over Fibre Channel (e.g. FC-NVMe). Networked fabric accessed NVMe access of pooled shared storage systems and appliances can also include internal NVMe attached devices (e.g. as part of back-end storage) as well as other SSDs (e.g. SAS, SATA).


General wrap-up (for now) NVMe U.2 8639 and related tips include:

  • Verify the performance of the device vs. how many PCIe lanes exist
  • Update any applicable BIOS/UEFI, device drivers and other software
  • Check the form factor and interface needed (e.g. U.2, M.2 / NGFF, AiC) for a given scenario
  • Look carefully at the NVMe devices being ordered for proper form factor and interface
  • With M.2 verify that it is an NVMe enabled device vs. SATA


Learn more about NVMe at including how to use Intel Optane NVMe 900P U.2 SFF 8639 disk drive form factor SSDs in PCIe slots as well as for fabric among other scenarios.


Ok, nuff said, for now.


World Backup Day 2018 Data Protection Readiness Reminder

server storage I/O trends

It's that time of year again, World Backup Day 2018 Data Protection Readiness Reminder.


In case you have forgotten, or were not aware, this coming Saturday March 31 is World Backup (and recovery day). The annual day is a to remember to make sure you are protecting your applications, data, information, configuration settings as well as data infrastructures. While the emphasis is on Backup, that also means recovery as well as testing to make sure everything is working properly.


data infrastructure data protection


Its time that the  focus of world backup day should expand from just a focus on backup to also broader data protection and things that start with R. Some data protection (and backup) related things, tools, tradecraft techniques, technologies and trends that start with R include  readiness, recovery, reconstruct, restore, restart, resume, replication, rollback, roll forward, RAID and erasure codes, resiliency, recovery time objective (RTO), recovery point objective (RPO), replication among others.


data protection threats ransomware software defined


Keep in mind that Data Protection  is a broader focus than just backup and recovery. Data protection includes  disaster recovery DR, business continuance BC, business resiliency BR, security (logical and physical), standard and high availability HA, as well as durability, archiving, data footprint reduction, copy data management CDM along with various technologies, tradecraft techniques, tools.


data protection 4 3 2 1 rule and 3 2 1 rule

Quick Data Protection, Backup and Recovery Checklist

    • Keep the 4 3 2 1 or shorter older 3 2 1 data protection rules in mind
    • Do you know what data, applications, configuration settings, meta data, keys, certificates are being protected?
    • Do you know how many versions, copies, where stored and what is on or off-site, on or off-line?
    • Implement data protection at different intervals and coverage of various layers (application, transaction, database, file system, operating system, hypervisors, device or volume among others)


    data infrastructure backup data protection



          • Have you protected your data protection environment including software, configuration, catalogs, indexes, databases along with management tools?
          • Verify that data protection point in time copies (backups, snapshots, consistency points, checkpoints, version, replicas) are working as intended
          • Make sure that not only are the point in time protection copies running when scheduled, also that they are protected what's intended


          data infrastructure backup data protection



              • Test to see if the protection copies can actually be used, this means restoring as well as accessing the data via applications
              • Watch out to prevent a disaster in the course of testing, plan, prepare, practice, learn, refine, improve
              • In addition to verifying your data protection (backup, bc, dr) for work, also take time to see how your home or personal data is protected
              • View additional tips, techniques, checklist items in this Data Protection fundamentals series of posts here.

                storageio data protection toolbox

              Where To Learn More

              View additional Data Infrastructure Data Protection and related tools, trends, technology and tradecraft skills topics  via the following links.


              data protection rto rpo

              Additional  learning experiences along with  common questions (and answers), as well as  tips can be found in  Software Defined Data Infrastructure Essentials book.

              Software Defined Data Infrastructure Essentials Book SDDC

              What This All Means

              You can not go forward if you can not go back to a particular point in time (e.g. recovery point objective or RPO). Likewise, if you can not go back to a given RPO, how can you go forward with your business as well as meet your recovery time objective (RTO)?


              data protection restore rto rpo


              Backup is as important as restore, without a good backup or data protection point in time copy, how can you restore? Some will say backup is more important than recovery, however its the enablement that matters, in other words being able to provide data protection and recover, restart, resume or other things that start with R. World backup day should be a reminder to think about broader data protection which also means recovery, restore and realizing if your copies and versions are good. Keep the above in mind and this is your World Backup Day 2018 Data Protection Readiness Reminder.


              Ok, nuff said, for now.


              以前に、vSAN のストレージ ポリシーを PowerCLI で変更してみる投稿をしました。

              今回は、PowerCLI で vSAN データストアのデフォルト ストレージ ポリシーを確認してみます。

              vSAN の仮想マシン ストレージ ポリシー を PowerCLI で変更してみる。


              vSAN データストアでは、デフォルト ストレージ ポリシーを設定できます。



              PowerCLI でデフォルト ストレージ ポリシーの確認をするのは難しそうです。

              vSAN Cluster の設定を確認すると「StoragePolicy」プロパティがあります。

              PowerCLI> Get-Cluster vsan-cluster-03 | Get-VsanClusterConfiguration | fl Cluster,VsanEnabled,StoragePolicy


              Cluster       : vsan-cluster-03

              VsanEnabled   : True

              StoragePolicy :



              しかし、これは vSAN のパフォーマンスサービスの

              「統計オブジェクトのストレージ ポリシー」が格納されているようです。

              このクラスタでは vSAN のパフォーマンス サービスがオフのままなので、

              PowerCLI の StoragePolicy は空欄になっていますが、いずれにせよ

              Get-VsanClusterConfiguration の StoragePolicy ではデフォルト ストレージ ポリシーはわかりません。



              そこで今回はバッド プラクティスですが、

              実際に vSAN Cluster に VM を作成することで、デフォルト ストレージ ポリシーを確認してみます。


              ここでは「check-vm」という名前の VM を vSAN データストアに作成してみます。

              PowerCLI> New-VM -Name check-vm -ResourcePool vsan-cluster-03 -Datastore vsanDatastore-03 -StorageFormat Thin


              Name                 PowerState Num CPUs MemoryGB

              ----                 ---------- -------- --------

              check-vm             PoweredOff 1        0.250



              ストレージ ポリシーを指定せずに作成したので、

              この VM のストレージ ポリシーを確認することでデフォルト ストレージ ポリシーがわかります。

              この vSAN データストアのデフォルト ストレージ ポリシーは「vSAN Default Storage Policy」でした。

              PowerCLI> Get-VM check-vm | Get-SpbmEntityConfiguration | fl Entity,StoragePolicy


              Entity        : check-vm

              StoragePolicy : vSAN Default Storage Policy



              ということで、デフォルト ストレージ ポリシーを確認するスクリプトを作成してみました。


              vSAN の情報を PowerCLI 6.5 R1 で見てみる。



              get_vsan_default_storage_policy.ps1 · GitHub



              PowerCLI> .\get_vsan_default_storage_policy.ps1 <vSAN クラスタ名>



              • あらかじめ、PowerCLI のウインドウ幅は広げてあります。(120 くらい)
              • vSAN Default Storage Policy はデフォルトで作成されるポリシーですが、
              • テスト VM を作成して、情報を取得したあとに削除しています。





              以上、PowerCLI で vSAN のデフォルト ストレージ ポリシーを見てみる話でした。

              PowerCLI に含まれる Image Builder では、標準では含まれない VIB を追加したインストーラの

              ISO イメージファイルを作成することができます。

              今回は ESXi 6.5 に、RealTek の NIC ドライバを追加してみます。


              ちなみに、ドライバを追加した ISO イメージファイルを作成する場合、

              最近では Andreas Peetz さんが作成した ESXi-Customizer-PS という

              PowerShell / PowerCLI ベースのツールが利用されることが多いです。






              しかし今回は、あえて標準的な PowerCLI / Image Builder を使用して

              ISO イメージファイルを作成してみます。



              ESXi 6.5 は最新のパッチ(を MyVMware からダウンロードずみです。


              RealTek の NIC ドライバの VIB は、おなじく

              Andreas Peetz さんの下記の Web サイトで公開されているもの(Net55-r8168)を



              .vib ファイルではなくオフラインバンドルを利用してみます。

              これは VIB が含まれた ZIP ファイルで、 です。

              ※せっかくオフライン バンドルも提供していただいているので・・・


              List of currently available ESXi packages




              PowerCLI> ls .\, | select Name,Length


              Name                                         Length

              ----                                         ------





              なお、PowerCLI も、インストールずみです。


              ISO イメージ ファイルの作成。

              まず、ESXi のパッチ(オフラインバンドル)を読み込みます。

              今回も、不具合修正&セキュリティ修正の両方と、VMware Tools が含まれるイメージ プロファイルである

              ESXi-6.5.0-20171204001-standard を利用します。

              PowerCLI> Add-EsxSoftwareDepot .\



              PowerCLI> Get-EsxImageProfile | where {$_.Name -notmatch "s-standard|-no-tools"} | ft -AutoSize


              Name                            Vendor       Last Modified       Acceptance Level

              ----                            ------       -------------       ----------------

              ESXi-6.5.0-20171204001-standard VMware, Inc. 2017/12/18 11:40:25 PartnerSupported



              RealTek ドライバのオフラインバンドルを読み込みつつ、

              ついでに含まれている VIB の名前を見ておきます。

              VIB の名前は、net55-r8168 です。

              PowerCLI> Add-EsxSoftwareDepot .\ | Get-EsxSoftwarePackage | ft -AutoSize


              Name        Version    Vendor  Creation Date

              ----        -------    ------  -------------

              net55-r8168 8.045-napi Realtek 2018/02/08 12:46:36



              ちなみに、VIB のファイル名も確認できたりします。

              PowerCLI> Get-EsxSoftwarePackage -AcceptanceLevel CommunitySupported -Name net55-r8168 | select SourceUrls







              今回 読み込んだ VIB パッケージの中で、net55-r8168 だけは

              他よりも許容レベル(AcceptanceLevel)が低く、CommunitySupported のパッケージです。

              PowerCLI> Get-EsxSoftwarePackage -AcceptanceLevel CommunitySupported | ft -AutoSize


              Name        Version    Vendor  Creation Date

              ----        -------    ------  -------------

              net55-r8168 8.045-napi Realtek 2018/02/08 12:46:36



              イメージ プロファイルの 許容レベル については、ドキュメントでは下記のあたりに説明があります。

              VIB およびホストの許容レベルについて


              そこで新しく作成するイメージ プロファイルも、許容レベルを CommunitySupported にします。

              新しいプロファイル名は、わかりやすく ESXi-6.5.0-20171204001-with-realtek にしました。

              Vendor の指定も必須になるので、今回は HomeLab としています。

              PowerCLI> New-EsxImageProfile -CloneProfile $image_profile -Vendor HomeLab -Name ESXi-6.5.0-20171204001-with-realtek -AcceptanceLevel CommunitySupported | fl Name,Vendor,AcceptanceLevel


              Name            : ESXi-6.5.0-20171204001-with-realtek

              Vendor          : HomeLab

              AcceptanceLevel : CommunitySupported




              PowerCLI> Add-EsxSoftwarePackage -ImageProfile ESXi-6.5.0-20171204001-with-realtek -SoftwarePackage net55-r8168


              Name                           Vendor          Last Modified   Acceptance Level

              ----                           ------          -------------   ----------------

              ESXi-6.5.0-20171204001-with... HomeLab         2018/02/10 1... CommunitySupported



              ドライバの VIB パッケージが追加されています。

              PowerCLI> Get-EsxImageProfile -Name ESXi-6.5.0-20171204001-with-realtek | select -ExpandProperty VibList | where {$_.Name -eq "net55-r8168"} | ft -AutoSize


              Name        Version    Vendor  Creation Date

              ----        -------    ------  -------------

              net55-r8168 8.045-napi Realtek 2018/02/08 12:46:36



              ISO イメージ ファイルとしてエクスポートします。

              PowerCLI> Export-EsxImageProfile -ImageProfile ESXi-6.5.0-20171204001-with-realtek -ExportToIso -FilePath .\ESXi-6.5.0-20171204001-with-realtek.iso


              ついでに、ドライバを追加したイメージ プロファイルの


              これは、後でこのイメージにさらに VIB を追加したい場合などに利用できます。

              PowerCLI> Export-EsxImageProfile -ImageProfile ESXi-6.5.0-20171204001-with-realtek -ExportToBundle -FilePath .\



              PowerCLI> ls | select Name,Length


              Name                                         Length

              ----                                         ------

              ESXi-6.5.0-20171204001-with-realtek.iso   350390272






              この ISO イメージファイルで ESXi をインストールすると、

              通常のインストーラでは認識できない RealTek の NIC が認識できるようになります。



              ちなみに ESX Shell や SSH で直接ログインして確認すると、下記のように認識されています。

              [root@localhost:~] vmware -vl

              VMware ESXi 6.5.0 build-7388607

              VMware ESXi 6.5.0 Update 1

              [root@localhost:~] esxcli network nic list

              Name    PCI Device    Driver  Admin Status  Link Status  Speed  Duplex  MAC Address         MTU  Description                                                   

              ------  ------------  ------  ------------  -----------  -----  ------  -----------------  ----  -----------------------------------------------------------------------------------------

              vmnic0  0000:01:00.0  r8168   Up            Up            1000  Full    d8:9e:f3:7c:91:c6  1500  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller




              ESXi のオフライン バンドルから ISO イメージ ファイルを作成してみる。

              ESXi 5.5 を Intel NUC にインストールしてみる。

              ESXi 5.5 を Intel Haswell NUC にインストールしてみた。

              ネステッドESXi 用 VMware Tools 導入済みの ESXi 5.5 を作成してみました。(Image Builder PowerCLI)


              以上、ESXi 6.5 でカスタムイメージを作成してみる話でした。

              PowerCLI で、vSAN の情報取得や設定、操作などをすることができます。

              今回は、vSAN クラスタの概要を取得するスクリプトのサンプルを作成してみました。


              以前に VMware Hands-on Labs(HoL)で PowerCLI をためしてみる投稿をしてみましたが、


              vSAN の情報を PowerCLI 6.5 R1 で見てみる。


              手元に vSAN 環境がある場合は、コマンドラインを1行ずつ実行するよりも


              たとえば以前に HoL でためしたコマンドラインをまとめて、

              下記のリンクにあるようなスクリプトで vSAN 環境の概要を確認することができます。

              get_vsan_summary.ps1 · GitHub



              • PowerShell / PowerCLI のウインドウ幅は 140 にしてあります。
              • Connect-VIServer で vCenter に接続してから実行しています。
              • vSAN クラスタは 3つあります。(最初にクラスタ設定だけ表示)
              • vSAN Disk のデバイス名(CanonicalName)は長すぎて切れてますが、
              • 長いので2クラスタ目の途中までの表示となってますが、



              ちなみに PowerCLI のプロンプト「PowerCLI>」は、下記のように工夫しています。

              PowerCLI プロンプト文字列に vCenter への接続状態を反映してみる。


              そして今回ためした PowerCLI のバージョンは下記です。

              PowerCLI> Get-PowerCLIVersion | select UserFriendlyVersion




              VMware PowerCLI 6.5.1 build 5377412



              PowerCLI> Get-Module VMware.* | select Version,Name


              Version       Name

              -------       ----






              今回のサンプルスクリプトは、vSAN の情報を網羅的に取得しているわけではなかったり、



              以上、PowerCLI での vSAN 情報の取得例でした。

              ESXi 6.5 の「仮想マシン バージョン 13」(vmx-13)では

              対応するゲスト OS の種類が増えましたが、

              これまで対応していた OS の細分化もされています。


              たとえば Oracle Linux や CentOS などは、Red Hat Enterprise Linux 異なり

              これまで「Oracle Linux 4/5/6・・・」 のように 1つにまとめられていました。

              しかし、vmx-13 の VM では「Oracle Linux 6」「Oracle Linux 7」と、ちゃんと細分化されました。



              vCenter 6.5 の vSphere Web Client で見ると、

              vmx-11 の VM での Oracle Linux は下記のようになっています。




              vmx-13 の VM での Oracle Linux は下記のように分割されています。



              API のリファレンスを見ると、

              実際に増えた ゲスト OS の GuestId が「Since vSphere API 6.5」でわかりそうです。


              Enum - VirtualMachineGuestOsIdentifier(vim.vm.GuestOsDescriptor.GuestOsIdentifier)



              リファレンスにあるゲスト OS の NAME は VM の GuestId とおなじもので、

              PowerCLI などでゲスト OS の種類を指定するときに使用されます。

              従来だとバージョン指定がない oracleLinux64Guest だけでしたが

              oracleLinux6_64Guest、oracleLinux6_64Guest などが追加されています。

              PowerCLI> Get-VM oracle-* | select Name,GuestId | sort Name


              Name              GuestId

              ----              -------

              oracle-linux-6-vm oracleLinux6_64Guest

              oracle-linux-7-vm oracleLinux7_64Guest

              oracle-linux-vm   oracleLinux64Guest



              このように、以前より VM にゲスト OS の指定が明確にできるようになりました。


              また、.vmx ファイルの直接編集などで vmx-11 以前の仮想マシンに

              vmx-13 以降に追加された GuestId を指定してしまうと、

              vSphere Web Client での VM の設定確認・変更ができなくなることがあるので


              たとえば、vmx-11 の VM に vmx-13 で追加された「VMware Photon OS (64-bit)」


              ※実は Photon OS 2.0 GA の vmx-11 版の ova ファイルがそうなっていたりします。


              以上、ESXi 6.5 での GuestId についてでした。

              最新の ESXi を利用したい場合、たいてい ISO イメージ ファイルからインストールした後に

              オフライン バンドルとよばれるパッチを適用することになります。


              しかし PowerCLI の Image Builder を利用することで、オフライン バンドルの zip ファイルから

              そのパッチが適用された ESXi の ISO イメージ ファイルをエクスポートすることができます。


              Image Builder のドキュメントは下記のあたりです。

              vSphere ESXi Image Builder を使用したインストールのカスタマイズ


              Image Builder については、以前に下記のブログでも紹介されていました。

              VMware ESXi イメージ管理ベストプラクティス その1 - Japan Cloud Infrastructure Blog - VMware Blogs

              VMware ESXi イメージ管理ベストプラクティス その2 - Japan Cloud Infrastructure Blog - VMware Blogs


              本来であれば Image Builder は、ハードウェア特有のドライバ追加などの場合に利用されますが、

              今回はただ、パッチから最新版 ESXi のインストーラを抽出するためだけに利用してみます。


              PowerCLI をインストールしておきます。


              PowerCLI> Get-PowerCLIVersion | select User*




              VMware PowerCLI 6.5.1 build 5377412



              ESXi のオフラインバンドル(パッチ)を MyVMware からダウンロードしておきます。

              PowerCLI> ls .\ | select Name,Length


              Name                     Length

              ----                     ------




              ISO イメージ ファイルのエクスポート。

              Image Builder で、オフライン バンドルを読み込みます。

              PowerCLI> Add-EsxSoftwareDepot .\


              Depot Url





              オフラインバンドルには、複数のイメージ プロファイルが含まれています。

              今回は、VMware Tools のインストーラと、セキュリティパッチ以外の修正も含んでいる

              「ESXi-6.5.0-20171204001-standard」というイメージ プロファイルを利用します。

              PowerCLI> Get-EsxImageProfile | ft -AutoSize


              Name                             Vendor       Last Modified       Acceptance Level

              ----                             ------       -------------       ----------------

              ESXi-6.5.0-20171201001s-no-tools VMware, Inc. 2017/12/18 11:40:25 PartnerSupported

              ESXi-6.5.0-20171204001-standard  VMware, Inc. 2017/12/18 11:40:25 PartnerSupported

              ESXi-6.5.0-20171204001-no-tools  VMware, Inc. 2017/12/18 11:40:25 PartnerSupported

              ESXi-6.5.0-20171201001s-standard VMware, Inc. 2017/12/18 11:40:25 PartnerSupported



              イメージプロフィアるを指定して、ISO イメージ ファイルをエクスポートします。

              PowerCLI> Export-EsxImageProfile ESXi-6.5.0-20171204001-standard -ExportToIso -FilePath D:\work\ESXi-6.5.0-20171204001-standard.iso


              ISO イメージ ファイルがエクスポートされました。


              PowerCLI> ls D:\work\ESXi-6.5.0-20171204001-standard.iso | select Name,Length


              Name                                   Length

              ----                                   ------

              ESXi-6.5.0-20171204001-standard.iso 349245440



              PowerCLI スクリプト編。

              同様のことができる簡易的な PowerCLI スクリプトを作成してみました。




              ESXiのオフラインバンドルから、ISO イメージファイルをエクスポート。 · GitHub


              下記のように、オフライン バンドルと、スクリプト ファイルを配置して・・・

              PowerCLI> ls | select Name,Length


              Name                                     Length

              ----                                     ------


              export_esxi-offline-bundle_to_iso.ps1       502



              オフライン バンドルのファイル名とエクスポート先のディレクトリを指定して実行します。

              PowerCLI> .\export_esxi-offline-bundle_to_iso.ps1 .\ D:\work\

              Export ISO File: D:\work\ESXi-6.5.0-20171204001-standard.iso


              この ISO ファイルからブートして、ESXi をインストールすることができます。



              vSAN の動作確認などでは最新の ESXi を利用したかったりするので、



              以上、Image Builder で ESXi のパッチから ISO イメージ ファイルをエクスポートする話でした。

              Computer Design & Integration LLC (CDI LLC) is a VMware Premier Solution Provider. CDI has a very broad range of service and solution offerings to architect, deploy and manages multi-platform hybrid IT solutions, including traditional IT, public, private and hybrid clouds.


              CDI’s latest achievement is the integration of VMware NSX with its proprietary Hybrid Cloud Automation Framework (HCAF).

              Using this industry-leading tool from VMware, HCAF helps customers accelerate their journey to hybrid cloud business value.


              The HCAF framework is comprised of advisory, technology, and optimization services, solutions, and processes that seamlessly manages workload life-cycles through orchestration and automation - regardless of the cloud endpoint. The result: improved data center efficiency, satisfied business needs and greater ROI.


              Through the use of HCAF integrated with NSX, CDI's VCIX and VCDX certified experts have completed deployments across multiple verticals. VMware NSX integration include:


              • VMware NSX Multisite VXLAN Active-Active/Passive Cross-VC
              • VMware NSX Microsegmentation for Production Applications and Virtual Desktops
              • VMware NSX and 3rd Party Palo Alto Service Integration
              • VMware NSX over Cisco ACI as a physical underlay fabric


              Additional CDI / VMware partnership highlights:


              • VMware Cloud Provider Program (formerly VCAN)
              • Enterprise Partner in VMware Partner Professional Services Program
              • Americas VMware Partner Innovation Award - Software Defined Data Center
              • Leadership on VMware Partner Executive Board and Partner Technical Advisory Board
              • Full knowledge and execution for VMware SDDC assessment programs
              • Certified VMware vExperts and global evangelists
              • Formally enrolled in VMware’s Credit Transfer program


              The management team at CDI believes at knowledge sharing and help technical community by publishing white-papers, writing blogs and producing video on demand (VoD).


              For more information, please visit:

              World Wide Technology (WWT) is a leading and competent NSX delivery partner specializing in architecture, design, support, and training of VMware NSX.


              Based out of St. Louis, MO USA, WWT has trained NSX consultants and architects including 7 VCIX, 2 VCDX-NVs, and 24 VCPs whom have completed more than 2 dozen production deployments covering a wide range of NSX use cases. World Wide Technology is a part of NSBU Last Mile mentoring program and works closely with NSBU Solutions Architects to ensure successful customer deployments.


              The WWT NSX team was assembled from senior resources having background in various technologies including security, datacenter, collaboration and network services. This unique blend allowed these engineers to deliver an entire stack, solution or an individual component as required.


              Some key highlights of World Wide Technology include:


              • A leading provider of advanced virtualization and cloud computing solutions
              • A very strong VMware partner who have helped organizations optimize their IT environments from the desktop to the Data Center and into the Cloud
              • Ranked #1 National Premier Partner Solution provider in 2015 and 2016
              • VMware Premier Partner for over a decade
              • Part of the NSX Elite Program since 2014
              • Part of VMware Professional Services Partner Program (PSPP)
              • Recent wins and strong presence in healthcare and consumer products space focused on NSX Micro-Segmentation design and deployment
              • Deployed professional resources to provide end-to-end deployment tasks.
                • Services ranged from scripting/automating the creation of NSX security groups and ruleset objects to troubleshooting ESXi configuration for VVD (VMware Validated Design)
                • Providing hands-on mentorship


              WWT offers Professional Services for the entire SDDC portfolio from DC Virtualization to Automation, including capabilities to extend to products such as vRealize Automation.


              You may contact WWT at

              PowerCLI では、ESXi の「システムの詳細設定」パラメータを確認・変更することができます。

              そこで、VSAN Swap オブジェクトの Thick Provision無効化の設定を、

              PowerCLI でまとめて確認・設定してみます。



              VSAN Cormac Blog 〜VSAN 6.2 VM スワップ オブジェクトに関する新機能〜 - Japan Cloud Infrastructure Blog - VMware Blogs

              VSAN 6.2 Part 5 - New Sparse VM Swap Object -


              自宅のラボでは Swap オブジェクトの容量を確保しなくてもよいので、
              vSAN クラスタに含まれるすべての ESXi で「有効」に揃えてみます。


              PowerCLI スクリプトでの設定変更も紹介されていますが、


              Virtual SAN 6.2 & PowerCLI - Sparse Virtual Swap files - Virtual Blocks


              今回、設定変更の対象とする ESXi です。

              「vsan-cluster-01」という名前の vSAN クラスタに含まれる ESXi を対象とします。

              PowerCLI> Get-Cluster vsan-cluster-01 | Get-VMHost | sort Name | select Name,Version,Build


              Name             Version Build

              ----             ------- -----

     6.5.0   7388607

     6.5.0   7388607

     6.5.0   7388607

     6.5.0   7388607

     6.5.0   7388607

     6.5.0   7388607




              デフォルトでは VSAN.SwapThickProvisionDisabled = 0

              (「Thick プロビジョニング無効」を無効にされている状態)です。


              PowerCLI> Get-Cluster vsan-cluster-01 | Get-VMHost | sort Name | Get-AdvancedSetting VSAN.SwapThickProvisionDisabled | select Entity,Name,Value


              Entity           Name                            Value

              ------           ----                            -----

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     0

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1



              下記のように、設定変更をしていなかった ESXi だけに絞って、設定変更してみます。

              PowerCLI> Get-Cluster vsan-cluster-01 | Get-VMHost | sort Name | Get-AdvancedSetting -Name VSAN.SwapThickProvisionDisabled | where {$_.Value -ne 1} | select Entity,Name,Value


              Entity           Name                            Value

              ------           ----                            -----

     VSAN.SwapThickProvisionDisabled     0




              PowerCLI> Get-Cluster vsan-cluster-01 | Get-VMHost | sort Name | Get-AdvancedSetting -Name VSAN.SwapThickProvisionDisabled | where {$_.Value -ne 1} | Set-AdvancedSetting -Value 1 -Confirm:$false


              Name                 Value                Type                 Description

              ----                 -----                ----                 -----------

              VSAN.SwapThickPro... 1                    VMHost




              PowerCLI> Get-Cluster vsan-cluster-01 | Get-VMHost | sort Name | Get-AdvancedSetting -Name VSAN.SwapThickProvisionDisabled | select Entity,Name,Value


              Entity           Name                            Value

              ------           ----                            -----

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1

     VSAN.SwapThickProvisionDisabled     1



              PowerCLI を経由して、esxcli を実行することもできます。

              そこで、ESXi の実機で設定変更されていることを

              「esxcli system settings advanced list」でも確認してみます。

              /VSAN/SwapThickProvisionDisabled の IntValue が「1」に変更されていることがわかります。

              PowerCLI> (Get-VMHost -Name | Get-EsxCli -V2).system.settings.advanced.list.Invoke() | where {$_.Path -eq "/VSAN/SwapThickProvisionDisabled"}



              DefaultIntValue    : 0

              DefaultStringValue :

              Description        : Turn off default thick provisioning type for VM swap object and allow user to control the provisioning type using policy.

              IntValue           : 1

              MaxValue           : 1

              MinValue           : 0

              Path               : /VSAN/SwapThickProvisionDisabled

              StringValue        :

              Type               : integer

              ValidCharacters    :



              以上、PowerCLI で ESXi のパラメータを変更してみる話でした。


              Looking for a blog?

              Can't find a specific blog? Try using the Blog page to browse and search blogs.