Ethernet SSDs – Hands-On with the Kioxia EM6 NVMeoF

Koixia EM6 Cover For Web

Today we have a fairly exciting piece. This is our first hands-on with a new class of SSDs designed for a different scale-out deployment model. The Kioxia EM6 is a NVMeoF SSD that uses Ethernet and RDMA instead of PCIe/ NVMe, SAS, or SATA to connect to the chassis. We get hands-on to show you why this is very cool technology.

Ethernet SSD Video Version

We have a video version of this one as well. In the video, we are able to show what the screens look like and show blinking videos and features like hot-swapping the drives.

As always, we suggest opening this in a new browser/ tab for the best viewing experience.

Kioxia EM6 NVMeoF SSD in the Ingrasys ES2000

First off, the biggest change in this entire solution is the Kioxia EM6 SSD. This may look like a normal 2.5″ SSD, but there is a big difference, its I/O is Ethernet rather than traditional PCIe/ NVMe. One can see that the Ingrasys ES2000 sled has two EDSFF connectors on it. This is to maintain a common design with traditional NVMe products, but it highlights another feature. These are dual-port NVMeoF SSDs for redundancy. That is an important feature to allow them to compete in markets traditionally serviced by dual-ported SAS SSDs.

Ingrasys ES2000 And Kioxia EM6 In Tray

We looked at the Ingrasys ES2000 for Kioxia EM6 NVMe-oF SSDs at SC21. This is the first time that we got to actually use the system though.

Ingrasys ES2000 And Kioxia EM6 24 Bay Front

In the Kioxia lab, we got to use an older revision chassis but one can see it is a 24-bay 2U that looks like many other servers and storage shelves we have seen over the years.

Ingrasys ES2000 Kioxia EM6 NVMeoF SSD Chassis Front

We show this in the video, but we can actually hot-swap the EM6 SSDs. The EDSFF connectors are designed for hot-swap, and Ethernet networks, of course, are designed to handle devices going online/ offline.

Ingrasys ES2000 Kioxia EM6 NVMeoF SSD Chassis 3

Here is a look at the backplane that the drives plug into. As one can see, we have EDSFF connectors but only one set even though the drive trays have two connectors.

Ingrasys ES2000 And Kioxia EM6 Controller EDSFF Backplane

This is because there are two in each chassis. Here is a look at the switch node under a plexiglass cover.

Ingrasys ES2000 And Kioxia EM6 Controller Node Installed

Looking at the rear of the system, we can see the two switches. This particular system only has the top node connected. Also, as a fun note, since this is an Ethernet switch node, we can have 1-6 connections and in this system the one connection being used is 100GbE.

Ingrasys ES2000 Kioxia EM6 NVMeoF SSD Chassis Rear 2

One can also see that each network switch tray has a USB port, an out-of-band management port, and a serial console port. While they may look like servers, these are switches.

Ingrasys ES2000 Kioxia EM6 NVMeoF SSD Chassis Rear 1

Inside the system, we can see the Marvell 98EX5630 switch chip covered by the big heatsink. Marvell also makes the NVMe to Ethernet controllers in the EM6 SSDs.

Ingrasys ES2000 Switch Tray 1

There is also a control plane for the switch under the black heatsink. This is powered with an Intel Atom C3538 CPU along with a M.2 SSD and 8GB of DRAM.

Ingrasys ES2000 Switch Tray 2

These switches, stacked one above the other allow each drive to access both switch networks simultaneously providing a redundant path to the drives.

For those wondering, there is a Marvell 88SN3400 adapter card that connects a traditional NVMe SSD to Ethernet so it can be used in this chassis.

Ingrasys ES2000 And Kioxia EM6 NVMe To 25GbE EDSFF Bridge

Next, let us get hands-on and see some of the fun things we can do with the Kioxia EM6 SSDs.

from Hacker News https://ift.tt/zJqBtEn