chaos (Offline file server)

Chaos started as a low power server to be paired with a Gigabyte Brix (NUC-like) workstation. Original specs were Asus H97I-Plus, Core i3-4370, 8GB memory, 3ware 9650se 16-port raid controller. I think the original arrays were 4x 2TB WD Red, 4x 1.5TB WD Green, 4x 500GB WD Green, and a pair of 500GB Seagate "hybrid" drives.

Drives in trays and cages

chaos without drives

chaos with drives

drives from another angle

Bought a spare PC for kids gaming, but swapped my i3-4370 for the i5-4670k CPU (4 real cores, instead of 2 + hyperthreading).

Added 2x 1TB WD Red 2.5" drives as a mirrored array.

Retired 500GB drives for a new set of 4x 4TB WD Red. Probably got rid of the 1.5TB array too, but I'm not sure when.

Chaos became a workstation too. Intel setup was replaced with Asus B350-F, Ryzen 5 2400g, 16GB DDR4, Crucial 250GB M.2 SSD.

Added a pair of 6TB WD Elements drives (shucked). Started storing optical disk images (dvd, bd) on them.

Added a 16GB Intel Optane M.2 module as cache for the main array. Not much noticeable speed difference. Maybe helps with building software/LaTeX.

Moved to Supermicro/LSI SAS controller (AOC-USAS2-L8I). Migrated to ZFS from the ext4 on top of 3ware raid.

Blurry desk testing the LSI controller

Some SSDs in a 5.25" to quad 2.5" bay adapter

Motherboard and LSI controller

Drives installed with the old 3Ware controller

Seagate SSHD drives mounted in a expansion slot holder

SSDs with power splitter hanging off the onboard SATA

Backside cable management

Drive cable management

Case Rear

3Ware controler removed

Drives wired to the new controller

Traded 6TB pair for an 8TB pair of shucked drives. 6TB drives went into the new enclosures and became offline backup.

The drives were mounted to the bottom 120mm fan space using a pair of brackets from Mountain Mods.

Extra drives mounted

Zvideo drives from above

Upgraded motherboard to Asus Prime X470-PRO for better pci-e expansion. Still found lots of limitations due to the Ryzen 2400g's internal GPU. Lots of video card hopping around this time because of kernel driver issues with older cards that I was using for extra displays. Tried FireMV 2100 (locked up I think), FirePro W2100 (worked ok, but the fan spun full speed on boot until the kernel took over), Quadro NVS420 (drawing issues, especially in terminals).

Got a Radeon Pro WX 2100. This card was generally a lot better (more modern drivers). I still had issues with it causing an Xorg crash until mid-2020. Eventually settled out between newer backports kernel, and pulling newer firmware from the kernel git repo.

Comparison with the previous FiroPro W2100:

Front

Ports

Back

Upgraded to Ryzen 5 2600, and extra 16GB DDR4.

The Seasonic Platinum power supply I bought used on ebay would die occasionally. It had boot issues, and I had to unplug to get it to reset properly. Finally gave in and picked up a Seasonic Focus 750W.

I was doing some testing of other DRAM, and wanted to be sure it wasn't my other components so I tested my original DRAM. It had a lot of failures in Memtest86+, particularly test #6 (Moving inversions, 64 bit pattern).

So I started testing pairs and individual sticks, but the issue only seemed to come with all 4 sticks installed.

Based on some comments in this thread I changed the voltage (to 1.38V), and Trfc (to 800), and even tried running the RAM at 2133 (3200 sticks). Memtest86+ continued to fail so I guess I'm back to 16GB.

This machine is primarily an offline storage machine at this point so the memory loss isn't a big hit, but I did like having a more beefy machine for heavy tasks when needed.

I was moving dreadnought to an ATX case, and since chaos is a mostly offline fileserver these days I decided to put back the old Asus B350-F motherboard.

I took the time to remove unnecessary expansion cards (firewire, USB, 1G ethernet) which helps cut down on cabling.

This machine also got a different set of RAM as a result of shifting stick around. It was the set that was in justice. After doing some memtest I'm somewhat convinced this Ryzen 2600 just has memory controller issues. Whereas before I had problems running 4 DIMMS in the X470-PRO board, dreadnought has absolutely no issue with it. Now I'm running memtest and producing errors with only 2 DIMMS, even after I swapped and reseated them. I've reduced speed to 3000 to get a reliable setup.

I got a 32GB kit (2x16GB) for review. The brand is Hirvod.

It was sold as DDR4 3200 1.35V CL18. The chips are FBGA code D9ZFW, or Micron part number MT40A1G16KD-062E:E. These chips are DDR4 3200 CL22 1.2V chips. The actual XMP profile on these is 3200 1.35V CL19.

The set did 10 passes of Memtest86+.

Good thing is I managed to get 32GB with only two sticks (see previous trouble with 4 DIMMS). Unfortunately it is overvolted which is not ideal for longevity.

I traded the Ryzen 5 2600, out for a Ryzen 5 2400g from destroy. I also removed the Radeon Pro WX 2100 graphics card. The onboard graphics is adequate since I'm not using this as a multi-monitor workstation anymore.

The 2600 from this machine is going to end up in stargazer. I was going to pull the heatsink, but chaos has the fancier wraith spire (copper slug). It is taller than the wraith stealth so I was worried it might not fit in the smaller machine. My new CPU for dreadnought should come with another wraith stealth I can use instead.

I retired justice from a gaming desktop. Since the Fractal Node 304 case can hold 6 drives, I thought it would be a nice platform to put this mostly offline storage machine in.

The basic plan was:

• Keep Asus X470-I Gaming Mini-ITX motherboard, and 550W power supply in the Node 304 case
• Put the Ryzen 2400G CPU in there
• Trade out the tower heatsink for a regular AMD one
• Mount 6 drives with the Node 304's brackets
• Somehow mount 2 extra drives in the space above the CPU

I needed some extra SATA ports, but fortunately I had a M.2 5-port SATA adapter based on JMB585. I put the dual 10Gbit ethernet adapter in the full size PCIe slot.

Motherboard and expansion cards

Motherboard installed

zstorage array cabling mess

Case with (most) drives and cards in place

I built it up, and ran into a little trouble. The M.2 slot under the board doesn't function due to lack of PCIe lanes. That's where I put the OS drive. I could move to a 2.5" SATA drive, or a processor without integrated graphics.

I found some references to other Asus X470 boards booting without any graphics if you disable the 'Wait for F1 if error" option in BIOS. I swapped the Ryzen 5 2400G for a Ryzen 5 2600 (more cores too). I did some quick tests with a GPU installed to check BIOS settings, and ensure the machine comes up for SSH.

I ran into some trouble with GRUB being extremely slow, and I noticed some errors from one of the 2TB drives. Hopefully those are related, but I decided to sort that out later. Without any GPU the machine still comes up normally. It looks like I'll have no issue running this machine headless, at least until it fails to boot, or fails to bring up the network.

That leaves future tasks:

• Remove LVM encryption to simplify headless booting
• Fabricate a double drive mount
• Sort out the drive issue (check cables, adapter port, drive)
• Install 10 GbE card and check configuration (device names will probably change)

I removed all the 2TB drives and ran SMART tests with a USB SATA enclosure. One of them is having issues. The only 2TB drive I have spare is an older WD Elements USB 2.0 device, but it is currently being used for other offline backup.

If there isn't much data on that drive I can shift it to some 1TB drives, and then it will become the fourth disk in the array.

Currently available 2TB drives seem to be around $35​/TB which is poor compared to $20-25​/TB I see even on 4TB drives. Probably a good indication this array isn't worth maintaining.