Broadcom 9400 – Should You Buy One for a Homelab?
The 9400 series is LSI/Avago/Broadcom’s first “Tri-Mode” HBA, capable of supporting SAS, SATA, and NVMe all in one adapter. There’s a few catches, but despite that, it might still be worth the buy depending on your circumstances.
U.2 Tri-Mode – Not What it Seems
First, what does “Tri-Mode” really mean in this context? It may not mean quite what you think it does.
The 9400 supports U.2 NVMe drives. The first annoyance is that you have to buy their expensive cables, although DiLinKer makes a few cheaper cables for certain connector types. This is because the pinouts of SAS/SATA and PCIe are not directly interchangeable. However, the real issue is that U.2 tri-mode really just doesn’t make much sense.
In U.2, completely disjoint sets of pins are used for SAS/SATA and NVMe. This is why, on a U.2 NVMe/SAS/SATA hybrid backplane, there will be a separate set of connectors for SAS and NVMe. For example, if we have an 8-slot backplane, where 2 of the slots also support U.2, we’ll end up with two 4-lane SAS connectors, and two 4-lane PCIe connectors, all of which need to be plugged back into an HBA. So we use the four ports on our 9400-16i – two for SAS, two for NVMe, problem solved, right?
Well, yes. But in terms of homelab secondhand prices, it would have been significantly cheaper to buy an 8-lane SAS HBA and 8-lane PCIe redriver/retimer/switch/whatever – $70+70 (or even less depending on what you buy) versus $280. This is the problem with “U.2 Tri-Mode” – in our example, two of the cables will only ever carry PCIe, and two of the cables will only ever carry SAS. So what good did the tri-mode HBA do, when there’s no single line that requires switching modes? I’ve seen it jokingly called “try-mode”, and that pretty much sums it up.
To be clear, it’s not U.2 at fault here – there is a perfectly valid reason to keep the SAS/SATA and PCIe lanes completely separate: it, quite ironically, eliminates the need for a tri-mode controller because the SAS lanes can be connected to a pure SAS controller while the PCIe lanes can be connected to the PCIe bus/HBA, and neither one of them needs to understand the other. But this also means that a tri-mode controller is in an odd position.
U.3 fixes this – it uses the same sets of pins for SAS, SATA, and NVMe (yes, theoretically, you could have a quad-lane SAS drive – whether anyone will make one remains to be seen). If you have an eight-port non-expander backplane, you’d need eight connections back to an HBA, and each of them would actually be capable of carrying SAS or NVMe at any given time, depending on what drive is inserted. Thus, a tri-mode HBA is actually warranted (assuming the backplane isn’t responsible for splitting out the connections) with U.3. But not U.2.
So Is the Card a Bad Buy?
Despite the tri-mode rant, the card itself isn’t necessarily a bad value. For one, the card is still perfectly fine as a plain SAS HBA. In fact, Broadcom doesn’t seem to be interested in making plain SAS/SATA HBAs anymore – their current HBA lineups only have tri-mode and pure NVMe solutions. I had some issues making it talk to SGPIO direct-attach backplanes, but if you’re using an expander or no backplane whatsoever, it works. It’s more expensive than the 9300-8i on a per-port basis, but it has better port density. As for 16i cards, the 9300-16i is bulky (no low-profile option), runs hot, and uses a lot of power – so much power that it requires a 6-pin auxiliary power input! The 9305-16i fixes these problems – no extra power connector, and has 16i and even 24i low-profile versions. But in terms of secondhand prices, the 9305-16i is not any cheaper than the 9400-16i – they’re both $270 or so, and the 9400 has some AliExpress generic versions for even less. The 9400 is also good if you’re tight on PCIe slots, and need your NVMe and SAS combined into one slot.
Furthermore, because the card exposes NVMe drives as SAS devices, you can get NVMe boot support on systems that don’t natively support such. It also means that you can have full NVMe hotplug support even if neither your hardware nor OS would otherwise support PCIe hotplugging. Due to the drives being isolated from the system’s PCIe topology, you’re also less likely to bork something with a surprise removal. However, all of these advantages come with a downside: potentially lower performance compared to the other NVMe options, and inability to manage drives using NVMe-specific tools.
There are other niche advantages as well. For example, this makes passing multiple NVMe drives into a VM easier, as you can simply pass the entire card in. Thus, even if you’re adding or removing drives on the fly, you won’t need to update your configuration to pass in each individual drive.
Overall, you’ll have to evaluate based on your own needs. I wouldn’t buy one if I only needed an 8i SAS controller, or wanted to use NVMe on a more modern system with good hotplug support. But if you legitimately need that many SAS lanes on a single card, or you need the excellent hotplugging support offered by the 9400, then go for it.
Tips
If you are planning on buying one and making use of NVMe, I recommend buying the DiLinKer direct attach cable. Even if you’re planning on using a backplane, having a direct cable can assist with troubleshooting, since you can cut both the backplane as well as your other cables out of the picture entirely.