Updated: Nov 19
s Moore’s Law ends, hardware acceleration has started to take center-stage in modern cloud architecture. Cloud providers were the first to notice this decline and began experimenting with alternative methods to boost network performance, including GPUs, NPUs and ASIC chips. Today, all cloud providers run some form of SmartNIC or DPU. Amazon for example has put significant internal development effort for AWS Nitro (SmartNIC for its internal use).
Enterprises data centers (private-cloud) usually follow the architecture and solutions seen in public cloud. However, they do not have the necessary expertise to integrate complicated pieces of software/hardware into their designs yet. A problem area, which NetLOX aims to solve as part of its software offering which will ease this migration. In this post, let’s discuss what benefits SmartNICs or DPUs bring to the table:
1) Performance, Latency and Jitter
Various technologies exist today like DPDK, VPP etc which can help the CPU scale 100Gbps (with lot of tweaks and performance optimizations). But a server can’t do anything meaningful after being used for such purpose. It is not just about receiving and sending (i.e., pushing) the traffic or even switching it from port to port. It is about applying useful services to it and doing that without incurring high latency and jitter.
2) Overlay Networking
To implement a modern SDN solution, a virtual switch usually needs to to support myriad of overlay networking features, like VXLAN, VTEP, Geneve, etc. It has been seen time and again that such implementation is prone to overall server performance degradation. The performance could greatly benefit from a SmartNIC implementation.
Telemetry - Streaming telemetry measurements directly from the data plane is crucial to understand what is happening in the network. Real time insights into telemetry in conjugation with AI can provide invaluable information on networking pain points of the data-center.
Tap-network - Avoiding a separate tap-network used to monitor the primary network is possible if the SmartNIC contains tap-network functionalities, resulting in considerable cost savings. Moreover, TAP sink analyzing tapped-traffic itself can greatly benefit using SmartNIC to process high bandwidth traffic.
Firewall and micro-segmentation to protect (the workloads on) the server from external attacks and avoid that a compromised server can attack the network by moving laterally.
Encryption both symmetric and asymmetric - If performed on the main server CPU, these are the functions that will consume a lot of cores. There is no benefit of running these functions natively in CPU cores.
Containment - The SmartNIC is a separate entity from the server from a security perspective. It should have its own security posture, and the security posture must not be accessible through the PCIe interface. Like in multi-factor authentication, an attacker will need to compromise the two entities to succeed.
5) Virtual Block Storage
With the advent of NVMe over fabric the SmartNIC is perfectly positioned to act as a block storage adapter.
Let’s consider ACLs (Access Control Lists). It is easy to run a few of them in software. But, once the scale starts to grow beyond a few 1000s, there is a huge degradation in performance. Quite simply, it does not scale !
Nowadays, everyone wants to design an architecture that works for bare metal servers, virtual machines, and containers. A SmartNIC allows that since it is placed between the server and the network. A software solution running on the host is problematic in several of these environments. With virtual machines, do you implement the solution in the hypervisor or do you run over it? Can you modify the software of a bare-metal server running a database or an ERP application?
When considering all these factors, it is clear that it is not just about how many Gbps I can push in and out of the server. It is about what features and what level of security people want to achieve.
There is a good reason, why SmartNICs or DPUs will become (if not already) a must have entity in modern cloud architecture.