Digital Storage Projections For 2022, Part 3

This is my third and last blog on digital storage projections for 2022.  The prior two articles focused on storage devices (media) including magnetic tape, HDDs and SSDs.  This article focuses on developments in digital storage systems and their use in various workflows, software and other storage products.

Many companies and workers have experienced almost two years of remote work.  The tools that enabled this remote work provided, for those with good internet access, the ability to work and learn at home.  These tools have improved in their stability over this time but they still do not provide an experience that fully mimics face to face interactions.  Many of us can’t wait until we can see each other face to face again.  Nevertheless, for some workers, their productivity working remotely improved and not having to commute to work every day is a big attraction to remote work.  Many companies are providing options for remote work, which for some workers may be almost full time and for most is at least a few days a week.

In addition to the impact of remote work, there are important developments in IoT that will likely ramp up again with a slowing of the pandemic.  This combined with greater use of various Artificial Intelligence (AI) technologies in endpoints, at the edge and in the data center will drive hardware and software requirements.

The increase in remote work and possibly less travel, has increased demand for internet accessed content, from a private cloud, public cloud, hybrid (public and private) or even multi-cloud data centers.  This increase in the use of remote data centers has resulted in additional investments in hardware including digital storage and memory technology.  In 2022 and in future there are improvements and developments in memory, storage and interconnect technologies that will provide new performance levels and ways to improve efficiency and reducing data center carbon footprints.  Let’s discuss these developments as well as view some quotes from industry players on the future of storage.

PCIe is a fast computer bus that is utilized for important storage and memory interconnects, such as NVMe and CXL.  PCIe 4.0 is in mass market production and PCIe 5.0 is now available in high performance systems (it was introduced in 2019).  According to Sebastien Jean, Phison CTO, “Gen5 will usher in accelerated machine learning, modeling, and inference, and deliver value immediately to those who upgrade. Even operating systems and applications will change to take advantage of increased speed.”

The next generation PCIe 6.0 was released in January 2022.  As shown below PCIe 6.0 provides 256GB/s data rates for 16 lanes (16GB/s per lane) and is expected to be in production by about 2025. The PCI-SIG expects a doubling of PCIe bandwidth every 3 years.

PCIe 6.0 using Pulse Amplitude Modulation (PAM) with four levels and Flit (flow control unit) based encoding to more than double the available bandwidth.  PCIe 6.0 maintains backwards compatibility with all previous generations of PCIe architecture.  PCIe 6.0 will enable even faster storage and memory systems.

NVMe runs on the PCIe interface and is now the dominant interface for SSDs, both in the data center and in PCs.  NVMe looks ready to replace the SATA interface and even HDD companies, such as Seagate and controller companies such as Marvell are looking at making HDDs with NVMe interfaces.  Although SAS will survive for many enterprise data center applications, NVMe could become the standard storage interface for future data center and enterprise storage systems.  The NVMe protocol can also be carried through networking technologies such as Fibre Channel and Ethernet, creating NVMe-oF storage systems.  These have enabled networked NVMe storage systems, which are on their way to dominate network storage system hardware.

CXL also runs on the PCIe interface and provides a way to add more (higher latency, far) memory in a computer system.  CXL also enables pooling of memory and heterogeneous memory with different levels of performance because CXL is a switched network.  In addition, specialized compute accelerators can be added to the CXL network (as well as in a NVMe-oF network).  The first CXL-based systems will be available starting in 2022 and they should be mainstream by 2025.  There are other approaches that could increase near memory (low latency) capacity and performance, such as OMI.  OMI could be useful for low latency applications that require lots of memory.

Both NVMe-oF and CXL will enable pooling storage and memory respectively.  With the addition of specialized computing accelerators in these networks, computational storage and near memory computing are enabled.  Computational storage definitions and standards are being developed by a SNIA working group and should be playing an important role in off-loading functions from CPUs in data center and storage systems this year and into the future.  Note that the computer accelerators can be located in the network or in the storage device itself.   These storage and memory networks also enable composable infrastructure, where pooled compute, storage and memory can be used to create virtual machines or to support container-based workloads managed by software.  

The majority of cloud storage is object storage because its flat structure supports much greater scale in stored pieces of information than file systems can.  Object storage is often used for secondary or archived data with higher latency, but in recent years, many companies are offering object storage on NVMe SSDs that can support high performance and lower latency.  On the other hand, many common applications are made to access data as files rather than objects.  There is a struggle in the industry on whether applications should be adapted to work with objects or to create file systems on top of object storage to work with existing applications.

Some companies, such as Hammerspace, are working to create global file access, which provides users and applications with access to all of an organization’s file data, which they have permission to access, no matter where the data is stored or the user or application resides.  Hammerspace uses intelligent metadata to move data only when needed on a granular file basis.  The figure below shows the company’s Global Data Environment.

With the increase of connected things, spurred by high performance wireless networks, industries, cities, homes and devices we use every day are increasingly connected to edge and cloud services, including storage.  This enables new services and capabilities and will be a big driver of storage and memory demand, but it also raises security concerns.  Encrypting communication between these devices, frequent patching and other safeguards will be needed to prevent malware from damaging this complicated ecosystem.

Tara Holt, Senior Product Manager from Iron Mountain (and member of the Active Archive Alliance) says, that companies, “should also consider implementing a zero-trust network and multi-factor authentication to better safeguard critical systems and data.”

Bruce Kornfeld, Chief Marketing & Product Officer, StorMagic said that, “Organizations will begin to realize the performance and cost benefits of processing data at the edge versus exclusively in the cloud. Improvements in edge computing technology will allow for massive amounts of data to remain active and online while being processed, with only the most relevant data being sent to the cloud, and integrations with an assortment of third-party applications will enable advanced data analysis for industry’s that require those capabilities.”

Besides general hacking and penetration, ransomware has become a serious problem for many businesses.  Tiering of data, ransomware detection and particularly storing immutable backups should be an important element in any organization’s security measures.  Stephen Manley, CTO of Druva said, “In 2022 we will see a dramatic increase in both the volume and complexity of ransomware attacks. Cyber criminals will attack virtually any organization because ransomware has gone mainstream by becoming “as-a-service.” These attacks target users to penetrate your security and once inside, silently spread through your environment. The attack first deletes your backups, so you cannot recover from the attack without paying the ransom. Once your defenses are eliminated, they rapidly encrypt file data on servers, NAS appliances, and endpoints and demand payment for the decryption key.”

Mihir Shah, CEO of StorCentric gave some more color on the increasing danger of ransomware, “Cybercriminals and ransomware are evolving: from hitting only single organizations and/or individuals to attacking MSPs, where they could target multiple organizations with one fell swoop (e.g., Kaseya ransomware attack perpetrated by the REvil group).”

Another trend is that some organizations who moved their services to the cloud are now moving some of those services back on site to control costs or using multiple clouds for various services depending upon the price and capability of different clouds.  Many organizations are also required to store customer data in the same country as the customer, driving the need for multiple storage locations.  Hybrid and multi-cloud services are very common with many organizations and as a consequence data management between these various services is becoming more important. 

In fact, multiple public and private clouds may become the norm.  According to AB Periasamy from MinIO, “Enterprises already use nearly three public and three private clouds apiece. Hybrid cloud is still a thing, but the requirement for multi-cloud takes the conversation to the next level. Using the stats above – the average enterprise runs between five and six clouds today.”

According to Betsy Doughty, spectra Vice President of Corporate Marketing, “Organizations will continue to explore how best to integrate cloud into their IT strategies to enable low latency data availability in 2022. This will open the door to new methods of achieving this with distributed multi-cloud data management solutions capable of providing universal access and placement of data across multi-site and multi-cloud storage leading to highly efficient hybrid and multi-cloud workflows.” 

Faster storage and memory interconnects using PCIe 5 and soon, PCIe 6 will enable NVMe and CXL for storage and memory.  NVMe-oF and CXL enable pooling and composible infrastructure.  Object storage is growing fast but companies like Hammerspace seek to provide file data access.  Security and ransomware protection as well as advanced data management will be keys to enabling organizations to operate and protect their data even across multiple private and public clouds.