Hot damn these are cheap!
For $600 I could get 32TB array in:
A 1-disk redundancy 32TB array sold from Newegg would be closer to $900. I could get 3-disk redundancy 32TB array from these guys for that much. π€π€π€
I've been running a 4-disk RAIDz1 on USB for 4 years now with zero failures on one machine and one failure on another where it turned out the USB controller in one WD Elements was overheating. Adhering a small heatsink on it resolved the problem and it's been stable under load for 2 years now. The USB devices have to be decent. AMD's host controllers are okay. VIA hubs are okay. ASMedia USB-to-SATA are okay. I'm using some enclosures with ASMedia and some off-the-shelf WD Elements that also use ASMedia. It's likely easier to get a reliable system if installing disks internally as the PSU and interconnects are much more regulated and any would work well, whereas with USB you have to be careful in selecting decent components.
Software, software, software! ZFS, mdraid, etc. USB is fine even with hubs, so long as your hubs and USB controllers (USB-to-SATA) are decent and not overheating.
USB hub.
The newly responsible team decides to rewrite it because "it'll be easier than adding the new feature", hopping on for another ride on the rollercoaster of sadness.
NAT π₯΄
Or you could trigger automation that turns it off for hours, then turns it back on. That way you could get around the need to physically turn it on, in case everyone is away.
They use their own TPUs instead of NVIDIA AFAIK but yeah.
If this isn't a sign for Amazon workers that their contribution is undervalued even at their current compensation, I don't know what is. For fucks sake, unionize and become millionaires!
What if you can do a 2 or 3 disk redundancy?