Skip navigation

Greg's Blog

October 18, 2017 Previous day Next day

server storage I/O data infrastructure trends

 

Microsoft and Azure September 2017 Software Defined Data infrastructure Updates

 

September was a busy month for data infrastructure topics as well as Microsoft in terms of new and enhanced technologies. Wrapping up September was Microsoft Ignite  where Azure, Azure Stack, Windows, O365, AI, IoT, development tools announcements occurred, along with others from earlier in the month. As part of the September announcements, Microsoft released a new version of Windows server (e.g. 1709) that has a focus for enhanced container support. Note that if you have deployed Storage Spaces Direct (S2D) and are looking to upgrade to 1709, do your homework as there are some caveats that will cause you to wait for the next release. Note that there had been new storage related enhancements slated for the September update, however those were announced at Ignite to being pushed to the next semi-annual release. Learn more here and also here.

Azure Files and NFS

Microsoft made several Azure file storage related announcements and public previews during September including Native NFS based file sharing as companion to existing Azure Files, along with public preview of new Azure File Sync Service. Native NFS based file sharing (public preview announced, service is slated to be available in 2018) is a software defined storage deployment of NetApp OnTAP running on top of Azure data infrastructure including virtual machines and leverage Azure underlying storage.

 

Note that the new native NFS is in addition to the earlier native Azure Files accessed via HTTP REST and SMB3 enabling sharing of files inside Azure public cloud, as well as accessible externally from Windows based and Linux platforms including on premises. Learn more about Azure Storage and Azure Files here.

Azure File Sync (AFS)

Azure File Sync AFS

Azure File Sync (AFS) has now entered public preview.  While users of  Windows-based systems have been able to access and share Azure Files in the  past, AFS is something different.  I have used AFS for  some time now during several private preview iterations having seen how it has  evolved, along with how Microsoft listens incorporating feedback into the  solution.

 

Lets take a look at what is AFS, what it does, how it works, where  and when to use it among other considerations. With AFS, different and independent systems  can now synchronize file shares through Azure. Currently in the AFS preview  Windows Server 2012 and 2016 are supported including bare metal, virtual, and  cloud based. For example I have had bare metal, virtual (VMware), cloud (Azure  and AWS) as part of participating in a file sync activities using AFS.

 

Not to be confused with some other storage related AFS  including Andrew File System among others, the new Microsoft Azure File Sync service  enables files to be synchronized across different servers via Azure. This is  different then the previous available Azure File Share service that enables  files stored in Azure cloud storage to be accessed via Windows and Linux  systems within Azure, as well as natively by Windows platforms outside of  Azure. Likewise this is different from the recently announced Microsoft Azure  native NFS file sharing serving service in partnership with NetApp (e.g.  powered by OnTAP cloud).

 

https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/AFS can be used to synchronize across different on premise as well as cloud servers that can also function as cache. What this means is that for Windows work folders served via different on premise servers, those files can be synchronized across Azure to other locations. Besides providing a cache, cloud tiering and enterprise file sync share (EFSS) capabilities, AFS also has robust optimization for data movement to and from the cloud and across sites, along with management tools. Management tools including diagnostics, performance and activity monitoring among others.

Check out the AFS preview including planning for an Azure File Sync (preview) deployment (Docs Microsoft), and for those who have Yammer accounts, here is the AFS preview group link.

Microsoft Azure Blob Events via Microsoft

Azure Blob Storage Tiering and Event Triggers

Two other Azure storage features that are in public preview include blob tiering (for cold archiving) and event triggers for events. As their names imply, blob tiering enables automatic migration from active to cold inactive storage of dormant date. Event triggers are policies rules (code) that get executed when a blob is stored to do various functions or tasks. Here is an overview of blob events and a quick start from Microsoft here.

 

Keep in mind that not all blob and object storage are the same, a good example is Microsoft Azure that has page, block and append blobs. Append blobs are similar to what you might be familiar with other services objects. Here is a Microsoft overview of various Azure blobs including what to use when.

Project Honolulu and Windows Server Enhancements

Microsoft has evolved from command prompt (e.g. early MSDOS) to GUI with Windows to command line extending into PowerShell that left some thinking there is no longer need for GUI. Even though Microsoft has extended its CLI with PowerShell spanning WIndows platforms and Azure, along with adding Linux command shell, there are those who still want or need a GUI. Project Honolulu is the effort to bring GUI based management back to Windows in a simplified way for what had been headless, and desktop less deployments (e.g. Nano, Server Core). Microsoft had Server Management Tools (SMT) accessible via the Azure Portal which has been discontinued.

 

Microsoft Project Honolulu management via Microsoft.com
Project Honolulu Image via Microsoft.com

 

This is where project Honolulu comes into play for managing Windows Server platforms. What this means is that for those who dont want to rely on or have a PowerShell dependency have an alternative option. Learn more about Project Honolulu here and here, including download the public preview here.

Storage Spaces Direct (S2D) Kepler Appliance

Data Infrastructure  provider DataOn has announced a new turnkey Windows Server 2016 Storage Spaces Direct (S2D) powered Hyper-Converged Infrastructure (e.g. productization of project Kepler-47) solution with two node small form factor servers (partner with MSI). How small? Think suitcase or airplane roller board carry on luggage size.

 

What this means is that you can get into the converged, hyper-converged software defined storage game with Windows-based servers supporting Hyper-V virtual machines (Windows and Linux) including hardware for around $10,000 USD (varies by configuration and other options).

Azure and Microsoft Networking News

Speaking of Microsoft Azure public cloud, ever wonder what the network that enables the service looks like and some of the software defined networking (SDN) along with network virtualization function (NFV) objectives are, have a look at this piece from over at Data Center Knowledge.

 

In related Windows, Azure and other focus areas, Microsoft, Facebook and Telxius have completed the installation of a high-capacity subsea cable (network) to cross the atlantic ocean. Whats so interesting from a data infrastructure, cloud or legacy server storage I/O and data center focus perspective? The new network was built by the combined companies vs. in the past by a Telco provider consortium with the subsequent bandwidth sold or leased to others.

 

This new network is also 4,000 miles long including in depths of 11,000 feet, supports with current optics 160 terabits (e.g. 20 TeraBytes) per second capable of supporting 71 million HD videos streamed simultaneous. To put things into perspective, some residential Fiber Optic services can operate best case up to 1 gigabit per second (line speed) and in an asymmetrical fashion (faster download than uploads). Granted there are some 10 Gbit based services out there more common with commercial than residential. Simply put, there is a large amount of bandwidth increased across the atlantic for Microsoft and Facebook to support growing demands.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

Microsoft announced a new release of Windows Server at Ignite as part of its new semi-annual release cycle. This latest version of Windows server is optimized for containers. In addition to Windows server enhancements, Microsoft continues to extend Azure and related technologies for public, private and hybrid cloud as well as software defined data infrastructures.

 

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

 

Ok, nuff said, for now.
Gs

server storage I/O data infrastructure trends

 

Dell EMC VMware September 2017 Software Defined Data Infrastructure Updates

 

vmworld 2017

 

September was a busy month including VMworld  in Las Vegas that featured many Dell EMC VMware (among other)  software defined data infrastructure updates and announcements.

 

A summary of September VMware (and partner) related announcements include:

https://aws.amazon.com/blogs/aws/vmware-cloud-on-aws-now-available/

VMware on AWS via Amazon.com
VMware and AWS via Amazon Web Services

 

VMware and AWS

Some of you might recall VMware earlier attempt at public cloud with vCloud Air service (see Server StorageIO lab test drive here) which has since been depreciated (e.g. retired). This new approach by VMware leverages the large global presence of AWS enabling customers to set up public or hybrid vSphere, vSAN and NSX based clouds, as well as software defined data centers (SDDC) and software defined data infrastructures (SDDI).

 

VMware Cloud on AWS exists on a dedicated, single-tenant (unlike Elastic Cloud Compute (EC2) multi-tenant instances or VMs) that supports from 4 to 16 underlying host per cluster. Unlike EC2 virtual machine instances, VMware Cloud on AWS is delivered on elastic bare-metal (e.g. dedicated private servers aka DPS). Note AWS EC2 is more commonly known, AWS also has other options for server compute including Lambda micro services serverless containers, as well as Lightsail virtual private servers (VPS).

 

Besides servers with storage optimized I/O featuring low latency NVMe accessed SSDs, and applicable underlying server I/O networking, VMware Cloud on AWS leverages the VMware software stack directly on underlying host servers (e.g. there is no virtualization nesting taking place). This means more robust performance should be expected like in your on premise VMware environment. VM workloads can move between your onsite VMware systems and VMware Cloud on AWS using various tools. The VMware Cloud on AWS is delivered and managed by VMware, including pricing. Learn more about VMware Cloud on AWS here, and here (VMware PDF) and here (VMware Hands On Lab aka HOL).

 

Read more about AWS September news and related updates here in this StorageIOblog post.

 

VMware PKS
VMware and Pivotal PKS via VMware.com

Pivotal Container Service (PKS) and Google Kubernetes Partnership

During VMworld VMware, Pivotal and Google announced a partnership for enabling Kubernetes container management called PKS (Pivotal Container Service). Kubernetes is evolving as a popular open source container microservice serverless management orchestration platform that has roots within Google. What this means is that what is good for Google and others for managing containers, is now good for VMware and Pivotal. In related news, VMware has become a platinum sponsor of the Cloud Native Compute Foundation (CNCF). If you are not familiar with CNCF, add it to your vocabulary and learn more here at www.cncf.io.

Other VMworld and September VMware related announcements

Hyper converged data infrastructure provider Maxta has announced a VMware vSphere Escape Pod (parachute not included ) to facilitate migration from ESXi based  to Red Hat Linux hypervisor environments. IBM and VMware for cloud partnership, along with Dell EMC, IBM and VMware joint cloud solutions. White listing of VMware vSphere VMs for enhanced security combine with earlier announced capabilities.

 

Note that both VMware with vSphere ESXi and Microsoft with Hyper-V (Windows and Azure based) are supporting various approaches for securing Virtual Machines (VMs) and the hosts they run on. These enhancements are moving beyond simply encrypting the VMDK or VHDX virtual disks the VMs reside in or use, as well as more than password, ssh and other security measures. For example Microsoft is adding support for host guarded fabrics (and machine hosts) as well as shielded VMs. Keep an eye on how both VMware and Microsoft extend the data protection and security capabilities for software defined data infrastructures for their solutions and services.

Dell EMC Announcements

At VMworld in September Dell EMC announcements included:

  • Hyper Converged Infrastructure (HCI) and Hybrid Cloud enhancements
  • Data Protection, Goverence and Management suite updates
  • XtremIO X2 all flash array (AFA) availability optimized for vSphere and VDI

 

HCI and Hybrid Cloud enhancements include VxRail Appliance, VxRack SDDC (vSphere 6.5, vSAN 6.6, NSX 6.3) along with hybrid cloud platforms (Enterprise Hybrid Cloud and Native Hybrid Cloud) along with vSAN Ready Nodes (vSAN 6.6 and encryption) and VMware Ready System. Note that Dell EMC in addition to supporting VMware hybrid clouds also previously announced solutions for Microsoft Azure Stack back in May.

 

Software Defined Data Infrastructure Essentials at VMworld Bookstore

xxxx
Software Defined Data Infrastructure Essentials (CRC Press) at VMworld bookstore

 

My new book Software Defined Data Infrastructure Essentials (CRC Press) made its public debut in the VMware book store where I did a book signing event. You can get your copy of Software Defined Data Infrastructure Essentials which includes Software Defined Data Centers (SDDC) along with hybrid, multi-cloud, serverless, converged and related topics at Amazon among other venues. Learn more here.

 

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

A year ago at VMworld the initial conversations were started around what would become the VMware Cloud on AWS solution. Also a year ago besides VMware Integrated Containers (VIC) and some other pieces, the overall container and in particular related management story was a bit cloudy (pun intended). However, now the fog and cloud seem to be clearing with the PKS solution, along with details of VMware Cloud on AWS. Likewise vSphere, vSAN and NSX along with associated vRealize tools continue to evolve as well as customer deployment growing. All in all, VMware continues to evolve, let's see how things progress now over the year until the next VMworld.

 

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

 

Ok, nuff said, for now.
  Cheers Gs

server storage I/O data infrastructure trends

Amazon Web Service AWS September 2017 Software Defined Data Infrasture Updates

 

September was a busy month pertaining to   software defined data infrastructure including cloud and related AWS announcements. One of the announcements included VMware partnering to deliver vSphere, vSAN and NSX data infrastructure components for creating software defined data centers (SDDC) also known as multi cloud, and hybrid cloud leveraging AWS elastic bare metal servers (read more here in a companion post). Unlike traditional partner software defined solutions that relied on AWS Elastic Cloud Compute (EC2) instances, VMware is being deployed using private bare metal AWS elastic servers.

 

What this means is that VMware vSphere (e.g. ESXi) hypervisor, vCenter, software defined storage (vSAN), storage defined network (NSX) and associated vRealize tools are deployed on AWS data infrastructure that can be used for deploying hybrid software defined data centers (e.g. connecting to your existing VMware environment). Learn more about VMware on AWS here or click on the following image.

 

VMware on AWS via Amazon.com

Additional AWS Updates

Amazon Web Services (AWS) updates include, coinciding with VMworld, the initial availability of VMware on AWS (using virtual private servers e.g. think along the lines of Lightsail, not EC2 instances) was announced. Amazon Web Services (AWS) continues its expansion into database and table services with Relational Data Services (RDS) including various engines (Amazon Auora,MariaDB, MySQL, Oracle, PostgreSQL,and SQL Server along with Database Migration Service (DMS). Note that these RDS are in addition to what you can install and run your self on Elastic Cloud Compute (EC2) virtual machine instances, Lambda serverless containers, or Lightsail Virtual Private Servers (VPS).

 

AWS has published a guide to database testing on Amazon RDS for Oracle plotting latency and IOPs for OLTP workloads here using SLOB. If you are not familiar with SLOB (Silly Little Oracle Benchmark) here is a podcast with its creator Kevin Closson discussing database performance and related topics. Learn more about SLOB and step by step installation for AWS RDS Oracle here, and for those who are concerned or think that you can not run workloads to evaluate Oracle platforms, have a look at this here.

 

EC2 enhancements include charging by the second (previous by the hour) for some EC2 instances (see details here including what is or is not currently available) which is a growing trend by private cloud vendors aligning with how serverless containers have been billed. New large memory EC2 instances that for example support up to 3,904GB of DDR4 RAM have been added by AWS. Other EC2 enhancements include updated network performance for some instances, OpenCL development environment to leverage AWS F1 FPGA enabled instances, along with new Elastic GPU enabled instances. Other server and network enhancements include Network Load Balancer for Elastic Load Balancer announced, as well as application load balancer now supports load balancing to IP address as targets for AWS and on premises (e.g. hybrid) resources.

 

Other updates and announces include data protection backups to AWS via Commvault and AWS Storage Gateway VTL announced. IBM has announced their Spectrum Scale  (e.g. formerly known as SONAS aka GPFS) Scale Out Storage solution for high performance compute (HPC) quick start on AWS. Additional AWS enhancements include new edge location in Boston and a third Seattle site, while Direct Connect sites have been added in Boston and Houston along with Canberra Australia. View more AWS announcements and enhancements here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

AWS continues to grow and expand, both in terms of number of services, also the extensiveness of them. Likewise AWS continues to add more regions and data center availability zones, enhanced connectivity, along with earlier mentioned service features. The partnership with VMware should enable enterprise organizations to move towards hybrid cloud data infrastructures, while giving AWS an additional reach into those data centers. Overall a good set of enhancements by AWS who continues to evolve their cloud and software defined data infrastructure portfolio of solution offerings.

 

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

 

Ok, nuff said, for now.
Gs

server storage I/O data infrastructure trends

 

Microsoft has created an Azure and Amazon Web Service (AWS) Service Map  (corresponding services from both providers).
https://azure.microsoft.com/en-us/blog/cloud-service-map-for-aws-and-azure-available-now/

Azure AWS service map via Microsoft.com
Image via Azure.Microsoft.com

 

Note that this is an evolving work in progress from  Microsoft and use it as a tool to help position the different services from  Azure and AWS.

 

Also note that not all features or services may not be available in different regions, visit Azure and AWS sites to see current availability.

 

As with any comparison they are often dated the day they are  posted hence this is a work in progress. If you are looking for another Microsoft  created why Azure vs. AWS then check out this here. If you  are looking for an AWS vs. Azure, do a simple Google (or Bing) search and watch  all the various items appear, some sponsored, some not so sponsored among  others.

Whats In the Service Map

The following AWS and Azure services are mapped:

  • Marketplace (e.g. where you select service offerings)
  • Compute (Virtual Machines instances, Containers, Virtual Private Servers, Serverless Microservices and Management)
  • Storage (Primary, Secondary, Archive, Premium SSD and HDD, Block, File, Object/Blobs, Tables, Queues,  Import/Export, Bulk transfer, Backup, Data Protection, Disaster Recovery, Gateways)
  • Network & Content Delivery (Virtual networking, virtual private networks and virtual private cloud, domain name services (DNS), content delivery network (CDN), load balancing, direct connect, edge, alerts)
  • Database (Relational, SQL and NoSQL document and key value, caching, database migration)
  • Analytics and Big Data (data warehouse, data lake, data processing, real-time and batch, data orchestration, data platforms, analytics)
  • Intelligence and IoT (IoT hub and gateways, speech recognition, visualization, search, machine learning, AI)
  • Management and Monitoring (management, monitoring, advisor, DevOps)
  • Mobile Services (management, monitoring, administration)
  • Security, Identity and Access (Security, directory services, compliance, authorization, authentication, encryption, firewall
  • Developer Tools (workflow, messaging, email, API management, media trans coding, development tools, testing, DevOps)
  • Enterprise Integration (application integration, content management)

 

Down load a PDF version of the service map from Microsoft  here.

Where To Learn More

 

Learn more about related technology, trends, tools, techniques, and tips with the following links.

 

What this means

On one hand this can and will likely be used as a comparison however use caution as both Azure and AWS services are rapidly evolving, adding new features, extending others. Likewise the service regions and site of data centers also continue to evolve thus use the above as a general guide or tool to help map what service offerings are similar between AWS and Azure.

 

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

 

Ok, nuff said, for now.
Gs