In a world that is fanatical about going digital, the need for serving the tremendous storage and bandwidth requirements coupled with overwhelming data rates has triggered the rapid evolution of data centers and storage technologies. Enterprise storage and server systems have been constantly trying to achieve better performance by narrowing the gap between the underlying hardware components of a storage system. Over the past five years, these components have witnessed dramatic changes in terms of the fundamental technology and the underlying data processing infrastructure.
In 2020, terminologies such as Data Analytics, Machine learning, Deep learning, and Artificial intelligence are no longer just concepts, but more of the practices that are integrated into various applications around. Data curation becomes very important for these applications as incorrect data can cause the AI engines to make erroneous decisions and hence malfunction. All these points to the impact and prominence of huge amount of data processing very accurately and efficiently.
Underlying hardware
While traditionally the challenges of a data processing infrastructure have been with compute intensive applications, with over 40 trillion gigabytes of data being stored around the world in 2020, there is a plethora of problems posed by data-intensive applications too. High performance processors were born, hardware accelerators came into being, flash storage has been on the rise, high speed network protocols were established, and the software became more sophisticated than ever before to bring out the best in data management and analytics.
Evolving processor Trends Â
The traditional method of improving processor performance has been to hitch up the clock frequency. By 2005, an alternate method to pack more processing power was unveiled- multi-core computing. Multi-core parallel processing technology has given overwhelming benefits to data processing capabilities. Obviously, the software had to change dramatically, from serial to parallel and algorithms had to be optimized for concurrent operation of the cores.
As applications turned out to be more compute intensive, the concept of computation offloading gained popularity. Storage server architectures brought in co-processors and hardware accelerators such as GPU or FPGA to offload resource-intensive compute functions.
To address the ever-growing needs of Data Analytics and Artificial intelligence, hardware-based acceleration was adopted. Acceleration is mostly done by a programmable hardware component such as FPGA or GPU. They support the implementation of complex functions such as secure hash algorithms, design modeling, and simulations, etc. which are needed for High performance computing (HPC). Â Offloading compute intensive functions to these components frees up limited processor memory and computational load on the processor.
Multi-core processing had challenges in achieving the desired application performance limited mostly by I/O and memory bandwidth. FPGAs turned out to be advantageous here in terms of providing high bandwidth, low latency connections to the network, and storage systems. FPGAs and GPUs could also interface to the high-speed storage devices directly, alleviating the need to transfer data to and fro by the processor, thus positioning them optimally for maximum performance.
What’s in store for storage and memory interconnects?
SAS and SATA used to be a popular interface choice for SSDs, which are now being taken over by NVMe. NVMe (Non-Volatile Memory express) is a low-latency, high performance protocol running over PCIe. Placed close to the CPU and delivering increasingly stellar speeds over the years, PCIe was the natural choice for designers to interface to NVMe drives, to unleash the bandwidth and performance that flash storage could offer. While SAS-3 and SATA-3 drives operate at 12 Gb/s and 6 Gb/s respectively, NVMe could deliver a whopping 40Gb/s with PCIe Gen 3.
SCM is a new solid-state technology, which was intended to combine the best of DRAMs and Flash storage. SCMs have more latency than DRAMs but lesser than a flash storage. Unlike a DRAM, they retain data even after they are powered off. A combination of a DRAM and SCM can provide a tier of storage with the highest throughput and lowest latency possible in the market today.
Trends in Network Technologies
In today’s world, computations are performed by virtualized systems formed by clusters of servers. This mandates the need to have a high-speed network infrastructure between these servers and shared storage, in the so-called Storage Area Network (SAN), which can deliver high throughput and low latency.
Various protocols were developed for SANs such as Fibre Channel, iSCSI, Infiniband, etc. When SANs were initially built, most of them used Fibre Channel protocol which transports SCSI commands over Fibre Channel network. Later iSCSI came into being, an IP-based storage networking standard, and SCSI commands are transferred over a TCP/IP network. iSCSI could use existing IP infrastructure and required no dedicated cabling. Today, most deployed SANs are based on FC followed by iSCSI.
In addition to connecting flash drives, NVMe protocol serves as a networking protocol too. The major advantage of Non-Volatile Memory Express over Fabric (NVMe-oF) is that it can enable an NVMe transport bridge, to preclude any protocol translation overhead other than those for NVMe commands, end to end. The storage I/O capabilities can now be fed directly over PCIe at a much faster rate to a powerful multicore host processor to complete more complex computations in less time.
Trends in architecture
A decade ago, most of the Directly attached storage (DAS) servers had a common architecture. The rack-mount server used to have accessible bays for plugging in SAS/SATA hard drives. The storage mid-plane is used to interface to the motherboard over a PCIe based host RAID adapter card, which houses an embedded ROC (RAID-on-chip) controller. The same architecture could cater to SAS/SATA based SSDs when they became widely used around 2012. This opened the possibility of various RAID configurations by mixing up high performance SAS drives with low-cost SATA drives in faster and slower tiers of storage. Any bay could support SAS or SATA drives, allowing the flexibility to improve performance by changing the configuration of disks. The system could scale up device connectivity by using a SAS connection to a JBOD (just a bunch of disks).
The emergence of NVMe storage based on PCIe took the storage industry by storm. NVMe drives could directly interface to the processor over the PCIe bus, thus placing it closer to the CPU and dramatically reducing the I/O latency in storage transactions compared to the HDD storage.
The need to scale up to growing demands of compute performance has led the designers to consider various system architectures. Stand-alone storage servers could have multiple processor boards that can access an array of storage devices to improve compute performance. They could add-on GPU based processor accelerator boards or FPGA based add-on modules to function as off-load engines. As modularity has become the guiding principle for scaling up, demands for highly scalable and highly configurable systems are on the rise.
 Conclusion
As the world is looking at a digital transformation era, where increasing number of AI and IoT applications are finding their space, storage servers are going to witness greater amount of data and the need for compute resources. For the latest generation of high-speed applications, the mantra would be to move the high-performance storage devices such as NAND flash SSDs closer to the faster, higher core count processors and to reduce latencies of the storage network fabric. The combination of NVMe and NVMe-oF might provide a quantum leap in terms of low latency and better performance, hence breaking the storage I/O bottleneck. In 2021, we expect to see more adoption and deployment of these technologies.
Happiest Minds has vast experience in developing Server class designs based on SAS/SATA/PCIe and NVMe storage along with customized FPGA designs for the optimized data path. We are one of the pioneers to adapt to SDN/NFV technologies and are still one of the front runners offering services in storage solutions.
is a Director of Engineering at Happiest Minds with the Hardware & Embedded group. She has over two decades of hands-on experience in adding measurable value to clients through Hardware and Embedded Systems design.
Divya’s expertise is in delivering embedded and digital transformation solutions to customers across domains such as networking, storage, telecom, computing and Industrial IoT.
She holds a master’s degree in Embedded Systems from BITS, Pilani.
