Updated: Dec 17, 2021
Recently, our team received Bluefield 2 DPU from NVIDIA. Our team experienced the network offload performance of this DPU "for the first time".
We documented all the steps needed to run BF2 with default distribution from Nvidia. We conducted a short test on IPSEC offload and IPSEC+VXLAN offload.
And I learned the following precious lessons.
BF2 documents are insufficient, but Nvidia provides immediate and helpful support.
I haven't found many documents about DOCA yet.
In Linux, the low-level infrastructure TC offloads the flow to the BF2 ASIC (ASAP2).
For IPSEC, the mlx5 driver is connected to the general linux ipsec offload infrastructure.
In DPDK, the low-level infrastructure rte_flow offloads the flow to (ASAP2).
We haven't found the IPSEC offload performance in DPDK yet.
Bluefield2 can improve and optimize data center redundancy several times while keeping overall TCO low. At the same time, however, we need software innovation to make the right use of off-road technology in modern data centers.
Please watch more interesting news about our work on network offloading.
Update - Nov 2021
Above steps use various open-source tools, which are actually so difficult to stabilize and make the whole thing work. NetLOX Loxilight now fully supports Mellanox BF2 IPSEC. No need for complex and non-intuitive configuration steps to get IPSEC running.