This is false: Mistral small 24b at q4_K_M quantization is 15GB. q8 is 26GB. A 3090/4090/5090 with 24GB or two cards with 16GB (I recommend the 4060 Ti 16GB) will work fine with this model, and will work in a single computer. Like others have said, 10Gbe will be a huge bottleneck, plus it’s just simply not necessary to distribute a 24b model across multiple machines.

This is false when it comes to me to PCIe, as mentioned elsewhere in this thread.

Most motherboards have cutouts on one end of the PCIe x1/x4 slots, for exactly this situation. If not, and you want to be adventurous, you can cut the plastic of the slot and it’ll work fine.

If the card is PCIe 3.0 x4, and the slot is PCIe 4.0 x1, the card will run at PCIe 3.0 x1. But it’ll work.