Skip navigation
2017

server storage I/O trends

Broadcom aka Avago aka LSI announces SAS SATA NVMe Adapters with RAID

In case you missed it, Broadcom formerly known as Avago who bought the LSI adapter and RAID card business announced shipping new SAS, SATA and NVMe devices.

 

While SAS and SATA are well established continuing to be deployed for both HDD as well as flash SSD, NVMe continues to evolve with a bright future. Likewise, while there is a focus on software-defined storage (SDS), software defined data centers (SDDC) and software defined data infrastructures (SDDI) along with advanced parity RAID including erasure codes, object storage among other technologies, there is still a need for adapter cards including traditional RAID.

 

Keep in mind that while probably not meeting the definition of some software-defined aficionados, the many different variations, permutations along with derivatives of RAID from mirror and replication to basic parity to advanced erasure codes (some based on Reed Solomon aka RAID 2) rely on software. Granted, some of that software is run on regular primary server processors, some on packaged in silicon via ASICs or FPGAs, or System on Chips (SOC), RAID on Chip (RoC) as well as BIOS, firmware, drivers as well as management tools.

 

SAS, SATA and NVMe adapters

 

For some environments cards such as those announced by Broadcom are used in passthru mode effectively as adapters for attaching SAS, SATA and NVMe storage devices to servers. Those servers may be deployed as converged infrastructures (CI), hyper-converged infrastructures (HCI), Cluster or Cloud in Box (CiB) among other variations. To name names you might find the above (or in the not so distant future) in VMware vSAN or regular vSphere based environments, Microsoft Windows Server, Storage Spaces Direct (S2D) or Azure Stack, OpenStack among other deployments (check your vendors Hardware Compatibility Lists aka HCLs). In some cases these cards may be adapters in passthru mode, or using their RAID (support various by different software stacks). Meanwhile in other environments, the more traditional RAID features are still used spanning Windows to Linux among others.

 

Who Is Broadcom?

Some of you may know of Broadcom having been around for many years with a focus on networking related technologies. However some may not realize that Avago bought Broadcom and changed their name to Broadcom. Here is a history that includes more recent acquisitions such as Brocade, PLX, Emulex as well as LSI. Some of you may recall Avago buying LSI (the SAS, SATA, PCIe HBA, RAID and components) business not sold to NetApp as part of Engenio. Also recall that Avago sold the LSI flash SSD business unit to Seagate a couple of years ago as part of its streamlining. That's how we get to where we are at today with Broadcom aka formerly known as Avago who bought the LSI adapter and RAID business announcing new SAS, SATA, NVMe cards.

 

What Was Announced?

 

Broadcom has announced cards that are multi-protocol supporting Serial Attached SCSI (SAS), SATA/AHCI as well as NVM Express (NVMe) as basic adapters for attaching storage (HDD, SSD, storage systems) along with optional RAID as well as cache support. These cards can be used in application servers for traditional, as well as virtualized SDDC environments, as well as storage systems or appliances for software-defined storage among other uses. The basic functionality of these cards is to provide high performance (IOPs and other activity, as well as bandwidth) along with low latency combined with data protection as well as dense connectivity.

 

Specific features include:

  • Broadcom’s Tri-Mode SerDes Technology enables  the operation of NVMe, SAS or SATA devices in a single drive bay, allowing for  endless design flexibility.
  • Management software including LSI Storage Authority (LSA), StorCLI, HII  (UEFI)
  • Optional CacheVault(R) flash cache protection
  • Physical dimension Low Profile 6.127” x 2.712”
  • Host bus type x8 lane PCIe Express 3.1
  • Data transfer rates SAS-3 12Gbs; NVMe up to 8 GT/s PCIe Gen 3
  • Various OS and hypervisors host platform support
  • Warranty 3 yrs, free 5x8 phone support, advanced replacement option
  • RAID levels 0, 1, 5, 6, 10, 50, and 60

 

Note that some of the specific feature functionality may be available at a later date, check with your preferred vendors HCL

                                                                                                                                                                                                       

Specification

9480    8i8e

9440    8ihttps://www.broadcom.com/products/storage/raid-controllers/megaraid-9480-8i8e#specifications

9460    8ihttps://www.broadcom.com/products/storage/raid-controllers/megaraid-9440-8i

9460    16ihttps://www.broadcom.com/products/storage/raid-controllers/megaraid-9460-8i

Image

Broadcom 9480 8i83 nvme raid

Broadcom 9440 8i nvme raid

Broadcom 9460 8i nvme raid

Broadcom 9460 16i nvme raid

Internal Ports

8

 

8

16

Internal Connectors

2 x Mini-SAS HD x4 SFF-8643

2 x Mini-SAS HD x4 SFF-8643

2 x Mini-SAS HD x4 SFF-8643

4 Mini-SAS HD x4
      SFF-8643

External Ports

8

 

 

 

External Connectors

2 x Mini-SAS HD    SFF8644

 

 

 

Cache Protection

CacheVault CVPM05

 

CacheVault CVPM05

CacheVault    CVPM05

Cache Memory

2GB 2133 MHz DDR4    SDRAM

 

2GB 2133 MHz DDR4    SDRAM

4GB 2133 MHz DDR4    SDRAM

Devices Supported

SAS/SATA: 255, NVMe:    4 x4, up to 24 x2 or x4*

SAS/SATA: 63, NVMe:    4 x4, up to 24 x2 or x4*

SAS/SATA: 255, NVMe:    4 x4, up to 24 x2 or x4*

SAS/SATA: 255, NVMe:    4 x4, up to 24 x2 or x4*

I/O Processors (SAS Controller)

SAS3516 dual-core RAID-on-Chip (ROC)

SAS3408 I/O    controller (IOC)

SAS3508 dual-core    RAID-on-Chip (ROC)

SAS3516 dual-core RAID-on-Chip (ROC)

 

In case you need a refresher on SFF cable types, click on the following two images which take you to Amazon.com where you can learn more, as well as order various cable options. PC Pit Stop has a good selection of cables (See other SFF types), connectors and other accessories that I have used, along with those from Amazon.com and others.

 

Available via Amazon.com sff 8644 8643 sas mini hd cable

Left: SFF 8644 Mini SAS HD (External), Right SFF-8643 Mini SAS HD (internal) Image via Amazon.com

 

Available via Amazon.com sff 8644 8642 sas mini hd cable

Left: SFF 8643 Mini SAS HD (Internal), Right SFF-8642 SATA with power (internal) Image via Amazon.com

Wait, Doesnt NVMe use PCIe

For those who are not familiar with NVMe and in particular U.2 aka SFF 8639 based devices, physically they look the same (almost) as a SAS device connector. The slight variation is if you look at a SAS drive, there is a small tab to prevent plugging into a SATA port (recall you can plug SATA into SAS. For SAS drives that tab is blank, however on the NVMe 8639 aka U.2 drives (below left) that tab has several connectors which are PCIe x4 (single or dual path).

 

What this means is that the PCIe x4 bus electrical signals are transferred via a connector, to backplane chassis to 8639 drive slot to the drive. Those same 8639 drive slots can also have a SAS SATA connection using their traditional connectors enabling a converged or hybrid drive slot so to speak. Learn more about NVMe here (If the Answer is NVMe, then what were and are the questions?) as well as at www.thenvmeplace.com.

 

NVMe U.2 8639 driveNVMe U.2 8639 sas sata nvme drive
Left NVMe U.2 drive showing PCIe x4 connectors, right, NVMe U.2 8639 connector

Who Is This For?

These cards are applicable for general purpose IT and other data infrastructure environments in traditional servers among others uses. They are also applicable for systems builders, integrators and OEMs whom you may be buying your current systems from, or future ones.

Where to  Learn More

The following are additional resources to learn more about vSAN and related technologies.

What this  all means

Even as the industry continues to talk and move towards more software-defined focus, even for environments that are serverless, there is still need for hardware somewhere. These adapters are a good sign of the continued maturing cycle of NVMe to be well positioned into the next decade and beyond, while also being relevant today. Likewise, even though the future involves NVMe, there is a still a place for SAS along with SATA to coexist in many environments. For some environment there is a need for traditional RAID while for others simply the need for attachment of SAS, SATA and NVMe devices. Overall, a good set of updates, enhancements and new technology for today and tomorrow, now, when do I get some to play with? ;).

 

Ok, nuff said (for now...).

Cheers
Gs

server storage I/O trends

Dell EMC Azure Stack Hybrid Cloud Solution

Dell EMC have announced their Microsoft Azure Stack hybrid cloud platform solutions. This announcement builds upon earlier statements of support and intention by Dell EMC to be part of the Microsoft Azure Stack community. For those of you who are not familiar, Azure Stack is an on premise extension of Microsoft Azure public cloud.

 

What this means is that essentially you can have the Microsoft Azure experience (or a subset of it) in your own data center or data infrastructure, enabling cloud experiences and abilities at your own pace, your own way with control. Learn more about Microsoft Azure Stack including my experiences with and installing Technique Preview 3 (TP3) here.

 

software defined data infrastructures SDDI and SDDC

What Is Azure Stack

Microsoft Azure Stack is an on-premise (e.g. in your own data center) private (or hybrid when connected to Azure) cloud platform. Currently Azure Stack is in Technical Preview 3 (e.g. TP3) and available as a proof of concept (POC) download from Microsoft. You can use Azure Stack TP3 as a POC for learning, demonstrating and trying features among other activities. Here is link to a Microsoft Video providing an overview of Azure Stack, and here is a good summary of roadmap, licensing and related items.

 

In summary, Microsoft Azure Stack and this announcement is about:

  • A onsite, on-premise,  in your data center extension of Microsoft  Azure public cloud
  • Enabling private and hybrid  cloud with good integration along with shared  experiences with Azure
  • Adopt, deploy, leverage cloud on your terms and timeline  choosing what works best for you
  • Common processes,  tools, interfaces, management and user experiences
  • Leverage speed of  deployment and configuration with a purpose-built integrated  solution
  • Support existing and cloud-native  Windows, Linux, Container and other services
  • Available as a public preview via software download, as well  as vendors offering solutions

What Did Dell EMC Announce

Dell EMC announced their initial  product, platform solutions, and services  for Azure Stack. This includes a Proof of  Concept (PoC) starter kit (PE R630) for doing evaluations, prototype, training,  development test, DevOp and other initial activities with Azure Stack. Dell EMC also announced a larger for production  deployment, or large-scale development, test DevOp activity turnkey solution. The  initial production solution scales from 4 to 12 nodes, or from 80 to 336 cores  that include hardware (server compute, memory, I/O and networking, top of rack  (TOR) switches, management, Azure Stack software along with services.  Other aspects of the announcement include initial  services in support of Microsoft Azure Stack and Azure cloud offerings.


server storage I/O trends
Image via Dell EMC

 

The announcement builds on joint Dell EMC Microsoft  experience, partnerships, technologies and services spanning hardware,  software, on site data center and public cloud.

server storage I/O trends
Image via Dell EMC

 

Dell EMC along with Microsoft have engineered a hybrid cloud  platform for organizations to modernize  their data infrastructures enabling faster innovate, accelerate deployment of resources. Includes hardware  (server compute, memory, I/O networking, storage devices), software, services, and support.
server storage I/O trends
Image via Dell EMC

 

The value proposition  of Dell EMC hybrid cloud for Microsoft Azure Stack includes consistent  experience for developers and IT data infrastructure professionals. Common experience across Azure public cloud and Azure  Stack on-premise in your data center for private or hybrid. This  includes common portal, Powershell,  DevOps tools, Azure Resource Manager (ARM), Azure Infrastructure as a Service  (IaaS) and Platform as a Service (PaaS), Cloud Infrastructure and associated  experiences (management, provisioning, services).
server storage I/O trends
Image via Dell EMC

 

Secure, protect, preserve and serve applications VMs hosted  on Azure Stack with Dell EMC services along with Microsoft technologies. Dell  EMC data protection including backup and restore, Encryption as a Service, host  guard and protected VMs, AD integration among other features.
server storage I/O trends
Image via Dell EMC

 

Dell EMC services for Microsoft Azure Stack include single contact support for prepare, assessment, planning; deploy with rack  integration, delivery, configuration;  extend the platform with applicable migration,  integration with Office 365 and other applications,  build new services.
server storage I/O trends
Image via Dell EMC

 

Dell EMC Hyper-converged scale out solutions range from minimum of 4 x PowerEdge R730XD (total raw specs include 80 cores (4 x 20), 1TB RAM (4 x 256GB), 12.8TB SSD Cache, 192TB Storage, plus two top of row network switches (Dell EMC) and 1U management server node. Initial maximum configuration raw specification includes 12 x R730XD (total 336 cores), 6TB memory, 86TB SSD cache, 900TB storage along with TOR network switch and management server.

 

The above configurations initially enable HCI nodes of small (low) 20 cores, 256GB memory, 5.7TB SSD cache, 40TB storage; mid size 24 cores, 384GB memory, 11.5TB cache and 60TB storage; high-capacity with 28 cores, 512GB memory, 11.5TB cache and 80TB storage per node.
  server storage I/O trends
Image via Dell EMC

 

Dell EMC Evaluator program for Microsoft Azure Stack including the PE R630 for PoCs, development, test and training environments. The solution combines Microsoft Azure Stack software, Dell EMC server with Intel E5-2630 (10 cores, 20 threads / logical processors or LPs), or Intel E5-2650 (12 cores, 24 threads / LPs). Memory is 128GB or 256GB, storage includes flash SSD (2 x 480GB SAS) and HDD (6 x 1TB SAS). and networking.
server storage I/O trends
Image via Dell EMC

 

Collaborative support single contact between Microsoft and Dell EMC

Who Is This For

This announcement is  for any organization that is looking for an  on-premise, in your data center private or hybrid cloud turnkey solution  stack. This initial set of announcements can be for those looking to do a proof  of concept (PoC), advanced prototype,  support development test, DevOp or gain cloud-like elasticity, ease of use, rapid procurement and other experiences of public  cloud, on your terms and timeline. Naturally,  there is a strong affinity and seamless experience for those already using, or  planning to use Azure Public Cloud for Windows,  Linux, Containers and other workloads, applications,  and services.

What Does This Cost

Check with your Dell EMC representative  or partner for exact pricing which  varies for the size and configurations.  There are also various licensing models to take into consideration if you have Microsoft Enterprise  License Agreements (ELAs) that your Dell EMC representative  or business partner can address for you. Likewise being cloud based, there is also time usage-based  options to explore.

Where to learn more

What this  all means

The dust is starting to settle on last falls Dell EMC  integration, both of whom have long histories working with, and partnering  along with Microsoft on legacy, as well as virtual software-defined data centers (SDDC), software-defined data infrastructures  (SDDI), native, and hybrid clouds. Some may view the Dell EMC VMware relationship as a primary  focus, however, keep in mind that both Dell and EMC had worked with Microsoft long before VMware came into being. Likewise, Microsoft remains one of the most commonly  deployed operating systems on VMware-based  environments. Granted Dell EMC have a significant  focus on VMware, they both also sell, service and support many services for Microsoft-based solutions.

 

What about Cisco, HPE, Lenovo among others who have to announce or discussed their Microsoft  Azure Stack intentions? Good question, until we hear more about what those and  others are doing or planning, there is  not much more to do or discuss beyond speculating  for now. Another common question is if there is demand  for private and hybrid cloud, in fact,  some industry expert pundits have even said private,  or hybrid are dead which is interesting, how can something be dead if it is  just getting started. Likewise, it is  early to tell if Azure Stack will gain traction with various organizations,  some of whom may have tried or struggled with OpenStack among others.

 

Given a large number  of Microsoft Windows-based servers on VMware, OpenStack, Public cloud services  as well as other platforms, along with continued growing popularity of Azure,  having a solution such as Azure Stack provides an attractive option for many environments. That leads to the question  of if Azure Stack is essentially a replacement for Windows Servers or Hyper-V  and if only for Windows guest operating systems. At this point indeed, Windows  would be an attractive and comfortable option, however, given a large number  of Linux-based guests running on Hyper-V  as well as Azure Public, those are also primary candidates as are containers  and other services.

 

Overall, this is an excellent  and exciting move for both Microsoft  extending their public cloud software stack to be  deployed within data centers in a hybrid way, something that those  customers are familiar with doing. This  is a good example of hybrid being spanning public and private clouds, remote  and on-premise, as well as familiarity  and control of traditional procurement with the flexibility, elasticity experience  of clouds.

 

software defined data infrastructures SDDI and SDDC

 

Some will say that if OpenStack is struggling in many organizations  and being free open source, how Microsoft can have success with Azure Stack.  The answer could be that some organizations  have struggled with OpenStack while others have not due to lack of commercial  services and turnkey support. Having installed both OpenStack and Azure Stack  (as well as VMware among others), Azure Stack is at least the TP3 PoC is easy  to install, granted it is limited to one node,  unlike the production versions. Likewise,  there are easy to use appliance versions of OpenStack that are limited in  scale, as well as more involved installs that unlock full functionality.

 

OpenStack, Azure Stack, VMware and others have their places,  along, or supporting containers along with other tools. In some cases,  those technologies may exist in the same environment supporting different  workloads, as well as accessing various public clouds, after all, Hybrid is the  home run for many if not most legality IT environments.

Overall this is a good announcement from Dell EMC for those who are interested in, or should become more aware about Microsoft Azure Stack, Cloud along with hybrid clouds. Likewise look forward to hearing more about the solutions from others who will be supporting Azure Stack as well as other hybrid (and Virtual Private Clouds).

 

Ok, nuff said (for now...).

 

Cheers
Gs

server storage I/O trends

Azure Stack Technical Preview 3 (TP3) Overview Preview Review

Perhaps you are aware or use Microsoft Azure, how about  Azure Stack?

 

This is part one of a two-part series looking at Microsoft Azure Stack providing an overview, preview and review. Read part two here that looks at my experiences installing Microsoft Azure Stack Technical Preview 3 (TP3).

 

For those who are not aware, Azure Stack is a private on-premise  extension of the Azure public cloud environment. Azure Stack now in technical preview three (e.g. TP3), or  what you might also refer to as a beta (get the bits here).

 

In addition to being available via download as a preview, Microsoft is also working with vendors such as Cisco, Dell EMC, HPE, Lenovo and others who have announced Azure Stack support. Vendors such as Dell EMC have also made proof of concept kits available that you can buy including server with storage and software. Microsoft has also indicated that once launched for production versions scaling from a few to many nodes, that a single node proof of concept or development system will also remain available.

 

software defined data infrastructure SDDI and SDDC
Software-Defined Data Infrastructures (SDDI) aka Software-defined Data Centers, Cloud, Virtual and Legacy

 

Besides being an on-premise, private  cloud variant, Azure Stack is also hybrid capable being able to work with  public cloud Azure. In addition to working with public cloud Azure, Azure  Stack services and in particular workloads can also work with traditional  Microsoft, Linux and others. You can use pre built solutions from the Azure marketplace, in addition to developing your applications using Azure services and DevOps tools. Azure Stack enables hybrid deployment into public or private cloud to balance flexibility, control and your needs.

 

Azure Stack Overview

Microsoft Azure Stack is an on premise (e.g. in your own data center) private (or hybrid when connected to Azure) cloud platform. Currently Azure Stack is in Technical Preview 3 (e.g. TP3) and available as a proof of concept (POC) download from Microsoft. You can use Azure Stack TP3 as a POC for learning, demonstrating and trying features among other activities. Here is link to a Microsoft Video providing an overview of Azure Stack, and here is a good summary of roadmap, licensing and related items.

 

In summary, Microsoft Azure Stack is:

  • A onsite, on premise, in your data center extension of Microsoft Azure public cloud
  • Enabling private and hybrid cloud with strong integration along with common experiences with Azure
  • Adopt, deploy, leverage cloud on your terms and timeline choosing what works best for you
  • Common processes, tools, interfaces, management and user experiences
  • Leverage speed of deployment and configuration with a purpose-built integrate solution
  • Support existing and cloud native Windows, Linux, Container and other services
  • Available as a public preview via software download, as well as vendors offering solutions

What is Azure Stack Technical Preview 3 (TP3)

This version of Azure Stack is a single node running on a lone physical machine (PM) aka bare metal (BM). However can also be installed into a virtual machine (VM) using nesting. For example I have Azure Stack TP3 running nested on a VMware vSphere ESXi 6.5 systems with a Windows Server 2016 VM as its base operating system.

 

Microsoft Azure Stack architecture
Click here or on the above image to view list of VMs and other services (Image via Microsoft.com)

 

The TP3 POC Azure Stack is not intended for production environments, only for testing, evaluation, learning and demonstrations as part of its terms of use. This version of Azure Stack is associated with a single node identity such as Azure Active Directory (AAD) integrated with Azure, or Active Directory Federation Services (ADFS) for standalone modes. Note that since this is a single server deployment, it is not intended for performance, rather, for evaluating functionality, features, APIs and other activities. Learn more about Azure Stack TP3 details here (or click on image) including names of various virtual machines (VMs) as well as their roles.

 

Where to learn more

 

The following provide more information and insight about Azure, Azure Stack, Microsoft and Windows among related topics.

 

What this  all means

A common question is if there is demand  for private and hybrid cloud, in fact,  some industry expert pundits have even said private,  or hybrid are dead which is interesting, how can something be dead if it is  just getting started. Likewise, it is  early to tell if Azure Stack will gain traction with various organizations,  some of whom may have tried or struggled with OpenStack among others.

 

Given a large number  of Microsoft Windows-based servers on VMware, OpenStack, Public cloud services  as well as other platforms, along with continued growing popularity of Azure,  having a solution such as Azure Stack provides an attractive option for many environments. That leads to the question  of if Azure Stack is essentially a replacement for Windows Servers or Hyper-V  and if only for Windows guest operating systems. At this point indeed, Windows  would be an attractive and comfortable option, however, given a large number  of Linux-based guests running on Hyper-V  as well as Azure Public, those are also primary candidates as are containers  and other services.

 

Continue reading more in part two of this two-part series here including installing Microsoft Azure Stack TP3.

 

Ok, nuff said (for now...).

 

Cheers
Gs

server storage I/O trends

Dell EMC World 2017 Day One news announcement summary

Recap of the first day of the first combined Dell EMC World 2017 being held in Las Vegas Nevada. Last year’s  event in Las Vegas was the end of the EMC World, while this being the first of the combined Dell EMC World events that succeeded its predecessors.

 

What this means is an expanded focus because of the new Dell  EMC that has added servers among other items to the event focus. Granted, EMC had been doing servers via its VCE and converged divisions, however with the Dell EMC integration completed as of last fall,  the Dell Server group is now part of the Dell EMC organization.

 

The central theme of  this Dell EMC world is REALIZE with a focus on four pillars:

  • Digital Transformation  (Pivotal focus) of applications
  • IT Transformation (Dell  EMC, Virtustream, VMware) data center modernization
  • Workforce transformation  (Dell Client Solutions) devices from mobile to IoT
  • Information Security (RSA  and Secureworks)

software defined data infrastructures SDDI and SDDC

What Did Dell EMC Announce

Note that while there are focus areas of the different Dell Technologies  business units aligned to the pillars, there is also leveraging across those  areas and groups. For example, VMware NSX spans into security, and  PowerEdge  servers span into other pillars as a core data infrastructure building block.

 

What Dell EMC and Dell Technologies announced today.

New 14th generation  PowerEdge Servers that are core building  blocks for data infrastructures

Dell EMC has announced  the 14th generation of Intel-powered Dell EMC PowerEdge server portfolio systems. These includes servers  that get defined with software for software-defined  data centers (SDDC), software-defined data infrastructures (SDDI) for the cloud, virtual, the container as well as storage among other applications. Target  application workloads and environments range from high-performance compute (HPC), and high-productivity (or profitability)  compute (the other HPC), super compute (SC), little data and big data analytics,  legacy and emerging business applications as well as cloud and beyond. Enhancements  besides new Intel processor technology includes enhanced iDRAC, OpenManage,  REST interface, QuickSync, Secure Boot among other management, automation,  security, performance, and capacity  updates.

 

Other Dell EMC enhancements with Gen14 include support for  various NVDIMM to enable persistent memory also known as storage class memories  such as 3D Xpoint among others. Note at  this time, Dell EMC is not saying much about speeds, feeds and other details,  stay tuned for more information on these in the weeks and months to come.

 

Dell EMC has also been leaders with deploying NVMe from PCIe flash cards to 8639 U.2 devices such as  2.5” drives. Thus it makes sense to see continued adoption and deployment of  those devices along with SAS, SATA support. Note that Broadcom (formerly known  as Avago) recently announced the release of their PCIe SAS, SATA and NVMe based  adapters.

 

The reason this is worth mentioning is that in the past Dell  has OEM sourced Avago (formerly known as LSI) based adapters. Given Dell EMC  use of NVMe drives, it only makes sense to put two and two together.

 

Let’s wait a few months to see what the speeds, feeds, and specifications are to put the rest  of the puzzle together. Speaking of NVMe, also look for Dell EMC to also supporting PCIe AIC and U.2 (8639) NVMe  devices, also leverage M.2 Next Generation Form Factor (NGFF) aka Gum sticks as  boot devices.

 

While these are all Intel focused, I would expect Dell EMC not to sit back, instead, watch for what they  do with other processors and servers including ARMs among others.

 

Increased support for more GPUs to support VDI and other  graphic intensive workloads such as video rendering, imaging among others. Part  of enhanced GPU support is improvements (multi-vector cooling) to power and cooling  including sensing the type of PCIe card, and then adjusting cooling fans and subsequent  power draw accordingly. The benefit should be more proper cooling to reduce power to support more work and productivity.

 

Flexible consumption models (financing and more) from desktop to data center

Dell Technologies has announced several financing, procurement, and consumption  models with cloud-like flexible options  for different IT and data center, along  with mobile device technologies.

These range from licensing to deployment as a  service, consumption and other options  via Dell Financial Services (DFS).

 

Highlights include:

  • DFS Flex on Demand is available now in select countries  globally.
  • DFS Cloud Flex for HCI is available now for Dell EMC VxRail  and Dell EMC XC Series and has planned availability for Q3 2017 in Dell EMC  VxRack Systems.
  • PC as a Service is  available now in select countries globally.
  • Dell EMC VDI Complete Solutions are available now  in select countries globally.
  • DFS Flex on Demand is  available now in select countries globally.
  • DFS Cloud Flex for HCI is  available now for Dell EMC VxRail and Dell EMC XC Series and has planned VxRack  systems in Q3 2017.
  • PC as a Service solution is available now in select  countries globally.
  • Dell EMC VDI Complete Solutions are available now in select countries.
  • Dell Technologies  transformation license agreement (TLA) is available now in select countries

Hyper-Converged Infrastructure (HCI), Converged (CI) and Cloud like systems

Enhancements to VxRail system, VxRACK Systems, and XC Series leveraging Del EMC Gen14  PowerEdge servers along with other improvements.  Note that this also includes continued support for VMware, Microsoft as well as  Nutanix software-defined solutions.

 

New All-Flash (ADA) SSD Storage Systems (VMAX, XtremIO X2, Unity, SC, Isilon)

Storage system enhancements include from high-end (VMAX and  XtremIO) to mid-range (Unity and SC) along with scale-out NAS (Isilon) 

Highlights of the announcements include:

 

  • New VMAX 950F all flash  array (AFA)
  • New XtremIO X2 with  enhanced software, more powerful hardware
  • New Unity AFA systems
  • New SC5020 midrange hybrid  storage
  • New generation of Isilon  storage with improved performance, capacity, density

 

Integrated Data Protection Appliance  (IDPA) and Cloud Protection  solutions

Data protection enhancement highlights include:  

  • New Turnkey Integrated  Data Protection Appliance (IDPA) with four models (DP5300, DP5800, DP8300, and DP8800) starting at 34 TB  usable scaling up to 1PB usable. Data services including encryption, data  footprint reduction such as dedupe, remote monitoring, Maintenance service dispatch, along with application integration.  Application integration includes MongoDB, Hadoop, MySQL.
       
  • Enhanced cloud capabilities  powered by Data Domain virtual edition (DD VE 3.1) along with data protection suite  enable data to be protected too, and  restored from Amazon Web Services (AWS) Simple Storage Service (S3) as well as  Microsoft Azure.
       

Open Networking and software-defined  networks (SDN) with 25G

  Dell EMC Open Networking highlights include:

  • Dell  EMCs first 25GbE open networking top of  rack (TOR) switch including S5100-ON  series (With OS10 enterprise edition software)  complimenting new PowerEdge Gen14 servers with native 25GbE support. Switches  support 100GbE uplinks fabric connectivity for east-west (management) network  traffic. Also announced is the S4100-ON  series and N1100-ON  series that are in addition to recently  announce N3100-ON and N2100-ON switches.
  • Dell EMCs first optimized  Open Networking platform for unified storage network switching including  support for 16Gb/32GB Fibre Channel
  • New Network Function Virtualization  (NFV) and IoT advisory consulting services

Note that Dell EMC is announcing the availability of these networking solutions in Dell Technologies  2018 fiscal year which occurs before the traditional calendar year.

Using Gen14 servers, several Software Defined Storage (SDS) enhancements

 

Dell EMC announced enhancements to their Software Defined  Storage (SDS) portfolio that leveraging  the PowerEdge 14th generation server portfolio. These improvements include ScaleIO, Elastic Cloud Storage (ECS), IsilonSD Edge and Preview of  Project Nautilus.

 

Where to learn more

What this  all means

This is a summary of what has been announced so far on the first  morning of the first day of the first new Dell EMC world. Needless to say, there is more detail to look at for the  above announcements from speeds, feeds, functionality and related topics that  will get addressed in subsequent posts. Overall this is a good set of announcements expanding capabilities of the  combined Dell EMC while enhancing existing systems as well as well as  solutions.

 

Ok, nuff said (for now...).

 

Cheers
Gs

server storage I/O trends

GDPR  goes into effect May 25 2018 Are You Ready?

The  new European General Data Protection  Regulation (GDPR) go into effect in a year on May  25 2018 are you ready?

 

Why  Become GDPR Aware

If your initial response is that you are not in Europe and do not need to be concerned about GDPR you might want to step back and review that thought.  While it is possible that some organizations may not be affected by GDPR in  Europe directly, there might be indirect considerations. For example, GDPR, while focused on Europe, has ties to other initiatives in place or being planned for elsewhere in the world. Likewise unlike earlier regulatory compliance that  tended to focus on specific industries such as healthcare (HIPPA and HITECH) or  financial (SARBOX, Dodd/Frank among others), these new regulations can be more  far-reaching.

 

GDPR  Looking Beyond Compliance

Taking a step back, GDPR, as its name implies, is about general data protection including how information is protected, preserved, secured and served. This also includes taking safeguards to logically protect data with passwords, encryption among other techniques. Another dimension of GDPR is reporting and ability to track  who has accessed what information (including when), as well as simply knowing  what data you have.

 

What this means is that GDPR impacts users from consumers of social media such as  Facebook, Instagram, Twitter, Linkedin among others, to cloud storage and related services, as well as traditional applications. In other words, GDPR is  not just for finance, healthcare, it is more far-reaching making sure you know  what data exists, and taking adequate steps to protect.

There is a lot more to discuss of GDPR in Europe as well as what else is being done in other parts of the world. For now being aware of initiatives such as GDPR and its broader scope impact besides traditional compliance is important. With these new initiatives, the focus expands from the compliance  office or officers to the data protection office and data protection officer  whose scope is to protect, preserve, secure and serve data along with associated  information.

 

GDPR  and Microsoft Environments

As  part of generating awareness and help planning, I’m going to be presenting  a free webinar produced by Redmond Magazine sponsored by Quest (who will also be  a co-presenter) on June 22, 2017 (7AM PT). The title of the webinar is GDPR Compliance Planning for Microsoft Environments.

 

This webinar looks at the General  Data Protection Regulation (GDPR) and its impact on Microsoft environments.  Specifically, we look at how GDPR along with other future compliance directives impact Microsoft cloud, on-premise, and hybrid environments, as well as what you can do to be ready before the May 25, 2018 deadline. Join us for this  discussion of what you need to know to plan and carry out a strategy to help  address GDPR compliance regulations for Microsoft environments.
   
  What you will learn during this discussion:

  • Why GDPR and other regulations  impact your environment
  • How to assess and find  compliance risks
  • How to discover who has access to  sensitive resources
  • Importance of real-time auditing to  monitor and alert on user access activity

 

This webinar applies to  business professionals responsible for strategy, planning and policy decision-making for Microsoft environments along with associated applications. This  includes security, compliance, data protection, system admins, architects and  other IT professionals.

 

What  This All Means

Now  is the time to start planning, preparing for GDPR if you have not done so and  need to, as well as becoming more generally aware of it and other initiatives.  One of the key takeaways is that while the word compliance is involved, there  is much more to GDPR than just compliance as we have seen in the part. With GDPR  and other initiatives data protection becomes the focus including privacy,  protect, preserve, secure, serve as well as manage, have insight, awareness along  with associated reporting. Join me and Quest on June 22, 2017 7AM PT for the  webinar GDPR Compliance Planning for Microsoft Environments to learn more.

 

Ok, nuff said, for now.

Cheers
Gs

<td  style="padding:3px; font-family:Arial,Helvetica,sans-serif;" bgcolor="#00AAFF">Server StorageIO Industry Resources and Links

Volume 17, Issue V

Hello and welcome to the May 2017 issue of the Server StorageIO update newsletter.

 

Summer officially here in the northern hemisphere is still a few weeks away, however for all practical purposes it has arrived. What this means is that in addition to normal workplace activities and projects, there are plenty of outdoor things (as well as distractions) to attend to.

 

Over the past several months I have mentioned a new book that is due out this summer and which means it's getting close to announcement time. The new book title is Software Defined Data Infrastructure Essentials - Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft (CRC PRess/Taylor Francis/Auerbach) that you can learn more about here (with more details being added soon). A common question is will there be electronic versions of the book and the answer is yes (more on this in future newsletter).

Data Infrastructures

 

Another common question is what is it about, what is a data infrastructure (see this post) and what is tradecraft (see this post). Software-Defined Data Infrastructures Essentials provides  fundamental coverage of physical, cloud, converged, and virtual server storage  I/O networking technologies, trends, tools, techniques, and tradecraft skills.

 

Software-Defined  Data Infrastructures Essentials provides fundamental  coverage of physical, cloud, converged, and virtual server storage I/O  networking technologies, trends, tools, techniques, and tradecraft skills. From  webscale, software-defined, containers, database, key-value store, cloud, and  enterprise to small or medium-size business, the  book is filled with techniques,  and tips to help develop or refine your server storage I/O hardware, software,  and services skills. Whether you are new to data infrastructures or a seasoned  pro, you will find this comprehensive reference indispensable for gaining as  well as expanding experience with technologies, tools, techniques, and trends.

 

Software-Defined Data Infrastructure Essentials SDDI SDDC
ISBN-13: 978-1498738156
  ISBN-10: 149873815X
  Hardcover: 672 pages
  Publisher: Auerbach Publications; 1 edition (June 2017)
  Language: English

 

Watch for more news and insight about my new book Software-Defined Data Infrastructure Essentials soon. In the meantime,  check out the various items below in this edition of the Server StorageIO Update.

In This Issue

Enjoy this edition of the Server StorageIO update newsletter.

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Some recent Industry Activities, Trends, News and Announcements include:

 

Flackbox.com has some new independent (non NetApp produced) learning resources including NetApp simulator eBook and MetroCluster tutorial. Over in the Microsoft world, Thomas Maurer has a good piece about Windows Server build 2017 and all about containers. Microsoft also announced SQL Server 2017 CTP 2.1 is now available. Meanwhile here are some my experiences and thoughts from test driving Microsoft Azure Stack.
   
Speaking of NetApp among other announcements they released a new version of their StorageGrid object storage software. NVMe activity in the industry (and at customer sites) continues to increase with Cavium Qlogic NVMe over Fabric news, along with Broadcom recent NVMe RAID announcements. Keep in mind that if the answer is NVMe, than what are the questions.

 

Here is a good summary of the recent OpenStack Boston Summit. Storpool did a momentum announcement which for those of you into software defined storage, add Storpool to your watch list. On the VMware front, check out this vSAN 6.6 demo (video) of stretched cluster via Yellow Bricks.

 

Check out other industry news, comments, trends perspectives here.

 

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

View other recent as well as past StorageIOblog posts here

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: What to Do with Legacy Assets in a Flash Storage World
  There is still a place for hybrid arrays. A hybrid array is the home run when it comes to leveraging your existing non-flash, non-SSD based assets today.
 
  Via EnterpriseStorageForum: Where All-Flash Storage Makes No Sense
    A bit of flash in the right place can go a long way, and everybody can benefit from at least a some of flash somewhere. Some might say the more, the better. But where you have budget constraints that simply prevent you from having more flash for things such as cold, inactive, or seldom access data, you should explore other options.
   
    Via Bitpipe: Changing With the Times - Protecting VMs(PDF)

 

    Via FedTech: Storage Strategies: Agencies Optimize Data Centers by Focusing on Storage

 

    Via SearchCloudStorage: Dell EMC cloud storage strategy needs to cut through fog
   
    Via SearchStorage: Microsemi upgrades controllers based on HPE technology

 

    Via EnterpriseStorageForum: 8 Data Machine Learning and AI Storage Tips

 

    Via SiliconAngle: Dell EMC announces hybrid cloud platform for Azure Stack

View more Server, Storage and I/O trends and perspectives comments here

Events and Activities

Recent and upcoming event activities.

Sep. 13-15, 2017 - Fujifilm IT Executive Summit - Seattle WA

August 28-30, 2017 - VMworld - Las Vegas

Jully 22, 2017 - TBA

June 22, 2017 - Webinar - GDPR and Microsoft Environments

May 11, 2017 - Webinar - Email Archiving, Compliance and  Ransomware

See more webinars and activities on the Server StorageIO Events page here.

Useful links and pages:
Microsoft TechNet - Various Microsoft related from Azure to Docker to Windows
storageio.com/links - Various industry links (over 1,000 with more to be added soon)
objectstoragecenter.com - Cloud and object storage topics, tips and news items
OpenStack.org - Various OpenStack related items
storageio.com/protect - Various data protection items and topics
thenvmeplace.com - Focus on NVMe trends and technologies
thessdplace.com - NVM and Solid State Disk topics, tips and techniques
storageio.com/converge - Various CI, HCI and related SDS topics
storageio.com/performance - Various server, storage and I/O  benchmark and tools
VMware Technical Network - Various VMware related items

Ok, nuff said, for now.

Cheers
Gs

server storage I/O trends

Part II Revisiting AWS S3 Storage Gateway (Test Drive Deployment)

This Amazon Web Service (AWS) Storage Gateway Revisited posts is a follow-up to the  AWS Storage Gateway test drive and review I did a few years ago (thus why it's called revisited). As part of a two-part series, the first post looks at what AWS Storage Gateway is, how it has improved since my last review of AWS Storage Gateway  along with deployment options. The second post in the series looks at a sample test drive deployment and use.

 

What About Storage Gateway Costs?

Costs vary by region, type of storage being used (files stored in S3, Volume Storage, EBS Snapshots, Virtual Tape storage, Virtual Tape storage archive), as well as type of gateway host, along with how access and used. Request pricing varies including data written to AWS storage by gateway (up to maximum of $125.00 per month), snapshot/volume delete, virtual tape delete, (prorate fee for deletes within 90 days of being archived), virtual tape archival, virtual tape retrieval. Note that there are also various data transfer fees that also vary by region and gateway host. Learn more about pricing here.

 

What Are Some Storage Gateway Alternatives

AWS and S3 storage gateway access alternatives include those from various third-party (including that are in the AWS marketplace), as well as via data protection tools (e.g. backup/restore, archive, snapshot, replication) and more commonly storage systems. Some tools include Cloudberry, S3FS, S3 motion, S3 Browser among many others.

 

Tip is when a vendor says they support S3, ask them if that is for their back-end (e.g. they can access and store data in S3), or front-end (e.g. they can be accessed by applications that speak S3 API). Also explore what format the application, tool or storage system stores data in AWS storage, for example, are files mapped one to one to S3 objects along with corresponding directory hierarchy, or are they stored in a save set or other entity.

 

AWS Storage Gateway Deployment and Management Tips

Once you have created your AWS account (if you did not already have one) and logging into the AWS console (note the link defaults to US East 1 Region), go to the AWS Services Dashboard and select Storage Gateway (or click here which goes to US East 1). You will be presented with three options (File, Volume or VTL) modes.

 

What Does Storage Gateway and Install Look Like

The following is what installing a AWS Storage Gateway for file and then volume looks like. First, access the AWS Storage Gateway main landing page (it might change by time you read this) to get started. Scroll down and click on the Get Started with AWS Storage Gateway button or click here.

 

AWS Storage Gateway Landing Page

Select type of gateway to create, in the following example File is chosen.

Select type of AWS storage gateway

 

Next select the type of file gateway host (EC2 cloud hosted, or on-premise VMware). If you choose VMware, an OVA will be downloaded (follow the onscreen instructions) that you deploy on your ESXi system or with vCenter. Note that there is a different VMware VM gateway OAV for File Gateway and another for Volume Gateway. In the following example VMware ESXi OVA is selected and downloaded, then accessed via VMware tools such as vSphere Web Client for deployment.

 

AWS Storage Gateway select download

 

Once your VMware OVA file is downloaded from AWS, install using your preferred VMware tool, in this case I used the vSphere Web Client.

 

AWS Storage Gateway VM deploy

 

Once you have deployed the VMware VM for File Storage Gateway, it is time to connect to the gateway using the IP address assigned (static or DHCP) for the VM. Note that you may need to allocate some extra VMware storage to the VM if prompted (this mainly applies to Volume Gateway). Also follow directions about setting NTP time, using paravirtual adapters, thick vs. thin provisioning along with IP settings. Also double-check to make sure your VM and host are set for high-performance power setting. Note that the default username is sguser and password is sgpassword for the gateway.

 

AWS Storage Gateway Connect

 

Once you successfully connect to the gateway, next step will be to configure file share settings.

 

AWS Storage Gateway Configure File Share

 

 

Configure file share by selecting which gateway to use (in case you have more than one), name of an S3 bucket name to create, type of storage (S3 Standard or IA), along with Access Management security controls.

 

AWS Storage Gateway Create Share

 

 

Next step is to complete file share creation, not the commands provided for Linux and Windows for accessing the file share.

 

AWS Storage Gateway Review Share Settings

 

 

Review file share settings

 

AWS Storage Gateway access from Windows

Now lets use the file share by accessing and mounting to a Windows system, then copy some files to the file share.

AWS Storage Gateway verify Bucket Items

 

Now let's go to the AWS console (or in our example use S3 Browser or your favorite tool) and look at the S3 bucket for the file share and see what is there. Note that each file is an object, and the objects simply appear as a file. If there were sub-directory those would also exist. Note that there are other buckets that I have masked out as we are only interested in the one named awsgwydemo that is configured using S3 Standard storage.

 

AWS Storage Gateway Volume

 

Now lets look at using the S3 Storage Gateway for Volumes. Similar to deploying for File Gateway, start out at the AWS Storage Gateway page and select Volume Gateway, then select what type of host (EC2 cloud, VMware or Hyper-V (2008 R2 or 2012) for on-premise deployment). Lets use the VMware Gateway, however as mentioned above, this is a different OVA/OVF than the File Gateway.

 

AWS Storage Gateway Configure Volume

 

Download the VMware OVA/OVF from AWS, and then install using your preferred VMware tools making sure to configure the gateway per instructions. Note that the Volume Gateway needs a couple of storage devices allocated to it. This means you will need to make sure that a SCSI adapter exists (or add one) to the VM, along with the disks (HDD or SSD) for local storage. Refer to AWS documentation about how to size, for my deployment I added a couple of small 80GB drives (you can choose to put on HDD or SSD including NVMe). Note that when connecting to the gateway if you get an error similar to below, make sure that you are in fact using the Volume Gateway and not mistakenly using the File Gateway OVA (VM). Note that the default username is sguser and password is sgpassword for the gateway.

 

AWS Storage Gateway Connect To Volume

 

Now connect to the local Volume Storage Gateway and notice the two local disks allocated to it.

 

AWS Storage Gateway Cached Volume Deploy

 

Next its time to create the Gateway which are deploying a Volume Cached below.

 

AWS Storage Gateway Volume Create

 

Next up is creating a volume, along with its security and access information.

 

AWS Storage Gateway Volume Settings

 

Volume configuration continued.

 

AWS Storage Gateway Volume CHAP

 

And now some additional configuration of the volume including iSCSI CHAP security.

 

AWS Storage Gateway Windows Access

 

Which leads us up to some Windows related volume access and configuration.

 

AWS Storage Gateway Using iSCSI Volume

 

Now lets use the new iSCSI based AWS Storage Gateway Volume. On the left you can see various WIndows command line activity, along with corresponding configuration information on the right.

 

AWS Storage Gateway Being Used by Windows

 

And there you have it, a quick tour of AWS Storage Gateway, granted there are more options that you can try yourself.

 

AWS

Where To Learn More

What This All Means

Overall I like the improvements that AWS has made to the Storage Gateway along with the different options it provides. Something to keep in mind is that if you are planning to use the AWS Storage Gateway File serving sharing mode that there are caveats to multiple concurrent writers to the same bucket. I would not be surprised if some other gateway or software based tool vendors tried to throw some fud towards the Storage Gateway, however ask them then how they coordinate multiple concurrent updates to a bucket while preserving data integrity.

 

Which Storage Gateway variant from AWS to use (e.g. File, Volume, VTL) depends on what your needs are, same with where the gateway is placed (Cloud hosted or on-premise with VMware or Hyper-V). Keep an eye on your costs, and more than just the storage space capacity. This means pay attention to your access and requests fees, as well as different service levels, along with data transfer fees.

 

You might wonder what about EFS and why you would want to use AWS Storage Gateway? Good question, at the time of this post EFS has evolved from being internal (e.g. within AWS and across regions) to having an external facing end-point however there is a catch. That catch which might have changed by time you read this is that the end-point can only be accessed from AWS Direct Connect locations.

 

This means that if your servers are not in a AWS Direct Connect location, without some creative configuration, EFS is not an option. Thus Storage Gateway File mode might be an option in place of EFS as well as using AWS storage access tools from others. For example I have some of my S3 buckets mounted on Linux systems using S3FS for doing rsync or other operations from local to cloud. In addition to S3FS, I also have various backup tools that place data into S3 buckets for backup, BC and DR as well as archiving.

 

Check out AWS Storage Gateway yourself and see what it can do or if it is a fit for your environment.

 

Ok, nuff said (for now...).

 

Cheers
Gs

server storage I/O trends

AWS S3 Storage Gateway Revisited (Part I)

This Amazon Web Service (AWS) Storage Gateway Revisited posts is a follow-up to the  AWS Storage Gateway test drive and review I did a few years ago (thus why it's called revisited). As part of a two-part series, the first post looks at what AWS Storage Gateway is, how it has improved since my last review of AWS Storage Gateway  along with deployment options. The second post in the series looks at a sample test drive deployment and use.

 

If you need an AWS primer and overview of various services such as Elastic Cloud Compute (EC2), Elastic Block Storage (EBS), Elastic File Service (EFS), Simple Storage Service (S3), Availability Zones (AZ), Regions and other items check this multi-part series (Cloud conversations: AWS EBS, Glacier and S3 overview (Part I) ).

 

AWS

 

As a quick refresher, S3 is the AWS bulk, high-capacity unstructured and object storage service along with its companion deep cold (e.g. inactive) Glacier. There are various S3 storage service classes including standard, reduced redundancy storage (RRS) along with infrequent access (IA) that have different availability durability, performance, service level and cost attributes.

 

Note that S3 IA is not Glacier as your data always remains on-line accessible while Glacier data can be off-line. AWS S3 can be accessed via its API, as well as via HTTP rest calls, AWS tools along with those from third-party's. Third party tools include NAS file access such as S3FS for Linux that  I use for my Ubuntu systems to mount S3 buckets and use similar to other mount points. Other tools include Cloudberry, S3 Motion, S3 Browser as well as plug-ins available in most data protection (backup, snapshot, archive) software tools and storage systems today.

 

AWS S3 Storage Gateway and What's New

The Storage Gateway is the AWS tool that you can use for accessing S3 buckets and objects via your block volume, NAS file or tape based applications. The Storage Gateway is intended to give S3 bucket and object access to on-premise applications and data infrastructures functions including data protection (backup/restore,  business continuance (BC), business resiliency (BR), disaster recovery (DR)  and archiving), along with storage tiering to cloud.

 

Some of the things that have evolved with the S3 Storage Gateway include:

  • Easier, streamlined download, installation, deployment
  • Enhanced Virtual Tape Library (VTL) and Virtual Tape support
  • File serving and sharing (not to be confused with Elastic File Services (EFS))
  • Ability to define your own bucket and associated parameters
  • Bucket options including Infrequent Access (IA) or standard
  • Options for AWS EC2 hosted, or on-premise VMware as well as Hyper-V  gateways (file only supports VMware and EC2)

AWS Storage Gateway Three Functions

AWS Storage Gateway can be  deployed for three basic functions:

 

AWS Storage Gateway File Architecture
AWS Storage Gateway File Architecture via AWS.com

    • File Gateway (NFS NAS) - Files, folders, objects and other  items are stored in AWS S3 with a local cache for low latency access to most  recently used data. With this option, you can create folders and subdirectory similar to a regular file system or NAS device as well as configure various security, permissions, access control policies. Data is stored in S3 buckets that you specify policies such as standard or Infrequent Access (IA) among other options. AWS hosted via EC2 as well as VMware Virtual Machine (VM) for on-premise file gateway.


      Also, note that AWS cautions on multiple concurrent writers to S3 buckets with Storage Gateway so check the AWS FAQs which may have changed by the time you read this. Current file share limits (subject to change) include 1 file gateway share per S3 bucket (e.g. a one to one mapping between file share and a bucket). There can be 10 file shares per gateway (e.g. multiple shares each with its own bucket per gateway) and a maximum file size of 5TB (same as maximum S3 object size). Note that you might hear about object storage systems supporting unlimited size objects which some may do, however generally there are some constraints either on their API front-end, or what is currently tested. View current AWS Storage Gateway resource and specification limits here.

AWS Storage Non-Cached e.g. Stored Volume Gateway Architecture
AWS Storage Gateway Non-Cached Volume Architecture via AWS.com

AWS Storage Gateway cached volume Architecture
  AWS Storage Gateway Cached Volume Architecture via AWS.com

    • Volume Gateway (Block iSCSI) - Leverages S3 with a point in  time backup as an AWS EBS snapshot. Two options exist including Cached volumes  with low-latency access to most recently used data (e.g. data is stored in AWS,  with a local cache copy on disk or SSD). The other option is Stored Volumes (e.g. non-cached) where primary  copy is local and periodic snapshot backups are sent to AWS. AWS provides EC2 hosted, as well as VMs for VMware and various Hyper-V Windows Server based VMs.


      Current Storage Gateway volume limits (subject to change) include maximum  size of a cached volume 32TB, maximum size of a stored volume 16TB. Note that snapshots of cached volumes larger than 16TB can only  be restored to a storage gateway volume, they can not be restored as an EBS  volume (via EC2). There are a maximum of 32 volumes for a gateway with total size of all volumes for a gateway (cached) of 1,024TB (e.g. 1PB). The total  size of all volumes for a gateway (stored volume)  is 512TB. View current AWS Storage Gateway resource and specification limits here.

AWS Storage Gateway VTL Architecture
  AWS Storage Gateway VTL Architecture via AWS.com

  • Virtual Tape Library Gateway (VTL) - Supports saving your data for  backup/BC/DR/archiving into S3 and Glacier storage tiers. Being a Virtual Tape Library (e.g. VTL) you can  specify emulation of tapes for compatibility with your existing backup,  archiving and data protection software, management tools and processes.


    Storage Gateway limits for tape include minimum size of a virtual tape 100GB, maximum size of a virtual tape 2.5TB, maximum number of virtual tapes for a VTL is 1,500 and total size  of all tapes in a VTL is 1PB. Note that the maximum number of virtual tapes in an archive is unlimited and total size of all tapes in an archive is also unlimited. View current AWS Storage Gateway resource and specification limits here.

        AWS

Where To Learn More

What This All Means

As to which gateway function and mode (cached or non-cached for Volumes) depends on what it is that you are trying to do. Likewise choosing between EC2 (cloud hosted) or on-premise Hyper-V and VMware VMs depends on what your data infrastructure support requirements are. Overall I like the progress that AWS has put into evolving the Storage Gateway, granted it might not be applicable for all usage cases. Continue reading more and view images from the AWS Storage Gateway Revisited test drive in part two located here.

 

Ok, nuff said (for now...).

 

Cheers
Gs

server storage I/O trends

Microsoft Windows Server, Azure, Nano and life cycle Updates

Spoiler alert, Microsoft is refocusing Nano for containers only, no more Bare Metal...

 

For those of you who have an interest in Microsoft Windows  Server on-premise, on Azure, on Hyper-V or Nano life cycle here are some recently announced updates.
Microsoft Windows Server Nano Lifecycle

Microsoft has announced updates to Windows Server Core and Nano along with semi-annual channel updates (read more here). The synopsis of this new update via Microsoft (read more here) is:

In this new model, Windows Server releases are identified by  the year and month of release: for example, in 2017, a release in the 9th month  (September) would be identified as version 1709. Windows Server will release  semi-annually in fall and spring. Another release in March 2018 would be  version 1803. The support lifecycle for each release is 18 months.

Microsoft has announced that its lightweight variant of WIndows  Server 2016 (if you need a refresh on server requirements visit here) known as nano will now be focused for WIndows based containers as  opposed to being for bare metal. As part of this change, Microsoft has reiterated that Server Core the headless (aka non-desktop user interface)  version of WIndows Server 2016 will continue as the platform for BM along with other deployments where a GUI interface is not needed. Note that one of  the original premises of Nano was that it could be leveraged as a replacement  for Server Core.

 

As part of this shift, Microsoft has also stated their intention to further streamline the already slimmed down version of WIndows  Server known as Nano by reducing its size another 50%. Keep in mind that Nano is already a fraction of the footprint size of regular Windows Server (Core or  Desktop UI). The footprint of Nano includes both its capacity size on disk (HDD  or SSD), as well as its memory requirements, speed of startup boot, along with  number of components that cut the number of updates.

 

By focusing Nano for container use (e.g. Windows containers)  Microsoft is providing multiple micro services engines (e.g. Linux and  Windows) along with various management including Docker. Similar to providing  multiple container engines (e.g. Linux and Windows) Microsoft is also  supporting management from Windows along with Unix.

 

Does This Confirm Rumor FUD that Nano is Dead

IMHO the answer to the FUD rumors that are circulating around that NANO is dead are false.

 

Granted Nano is being refocused by Microsoft for containers and will not be the lightweight headless Windows Server 2016 replacement for Server Core. Instead, the Microsoft focus is two path with continued enhancements on Server Core for headless full Windows Server 2016 deployment, while Nano gets further streamlined for containers. This means that Nano is no longer bare metal or Hyper-V focused with Microsoft indicating that Server Core should be used for those types of deployments.

 

What is clear (besides no bare metal) is that Microsoft is working to slim down Nano even further by removing bare metal items, Powershell,.Net and other items instead of making those into optional items. The goal of Microsoft is to make the base Nano image on disk (or via pull) as small as possible with the initial goal of being 50% of its current uncompressed 1GB disk size. What this means is that if you need Powershell, you add it as a layer, need .Net then add as a layer instead of having the overhead of those items if you do not need tem. It will be interesting to see how much Microsoft is able to remove as standard components and make them options that you can simply add as layers if needed.

 

What About Azure and Bring Your Own License

In case you were not aware or had forgotten when you use Microsoft Azure and deploy virtual machine (aka cloud instances),  you have the option of bringing (e.g. using) your own WIndows Server licenses.  What this means is that by using your own Windows Server licenses you can cut the monthly cost of your Azure VMs. Check out the Azure site and explore  various configuration options to learn more about pricing and various virtual machine instances from Windows to Linux here as well as hybrid deployments.

 

Where To Learn More

What This All Means

Microsoft has refocused Windows Server 2016 Core and Desktop as its primary bare metal including for virtual as well as Azure OS platforms, while Nano is now focused on being optimized for Windows-based containers including Docker among other container orchestration.

 

Ok, nuff said (for now...).

 

Cheers
Gs