pixy (DFI ITOX ST102-CS-SVR)

[2024-04-06 Sat] A moderner frigate machine

I was looking for something to replace Yamato since it is GPU crashy. Intel 8th gen seemed like a decent compromise between cost, performance, and something that I thought would be stable. I ran across this little server with two NICs in a compact ITX case. The price seemed OK, though it was more than other capable mini PCs. I hoped to avoid the whiny fans though so I went for it.

Auction Photo - Machine + PSU

Auction Photo - Front

Auction Photo - Rear Ports

Auction Photo - BIOS

[2024-04-12 Fri] Nice drives

I got the machine and opened it up. It had a 1TB 2.5" drive. I thought this was the Windows drive so I pulled it, but Windows kept booting. I looked around, but none of the M.2, Mini PCIe, or PCIe slots were occupied. Pulling the board I found another M.2 slot on the bottom populated with a 1TB NVMe SSD. It's a bit older so the write speed isn't all that, but 2.5 GB​/s reads, and 570 MB​/s writes is pretty good.

I didn't know what drives there were going in, but after finding these two I feel pretty good about how much I paid.

Drive tray with 2.5" drive

Machine with drive tray removed

Back of motherboard

1TB NVMe Micron SSD close-up

[2024-04-14 Sun] Not so quiet, Case mod

It turned out to be pretty loud. Due to the fan configuration only the CPU fan could be controlled. I happened to have a spare Thermalright AXP90 X36 cooler which fit the 75x75mm mount of the LGA 1151 socket. I installed it, but due to component locations I had to flip the mounting bracket. With the mounting bracket flipped it no longer fit behind the motherboard. So I modded the case.

The new heatsink is more massive, but the fan is very close to the drive mounting plate. I'm probably going to remove the plate, but I sketched out a some possible holes I could cut to let air flow. The new cooler weighs 258g, while the old one was 164g. It is also a bit heavier because the larger fan which also has a plastic frame.

It's not the prettiest, but I took two measurements for each heatsink nut from the motherboard stand-offs then located the new holes. I started with a step drilled hole, then sketched in the rest of the bracket and filed it out.

I had to trade the rubber feet out for slightly taller ones, but I have an idea for a slightly better solution. The current solution leaves exposed holes, and steel bits on the bottom that might scratch things. My plan is to get a piece of metal​/wood​/plastic, and mount it under the case with some little spacers, then attach the original rubber feet to that.

The fan situation is just a single 4-pin header that was split. The CPU fan was controlled by PWM, but the chassis fan got straight 12V.

I traded the 60mm chassis fan for a Gelid Silent 6, but since I couldn't control it, I used a 3-pin adapter with a resistor in-line (10 or 12 ohm I think) to cut the voltage a bit. After tuning the BIOS fan control this machine is no louder than the workstation it sits on.

Stock chassis fan - SunOn Maglev 0.89W!

Stock cooler and fans

New heatsink bracket stick out

Initial holes test fit

Holes filed to completely clear the cooler mount

New fans and better cable management

[2024-04-16 Tue] Serial Setup

[2024-04-22 Mon] Serial Setup Again

[2024-05-03 Fri] Hiding the heatsink holes

Previously, I cut holes in the bottom of the case. They need to be covered to prevent scratching things, and to look nice.

I got some acrylic sheets. The plan was to use one with a cut-out to fit around the CPU cooler mounting bracket, then a second one to cover it up.

To cut them to size I used a steel straight edge, and a utility knife to score both sides. The first attempt failed with the break wandering. Next I scored more deeply, and clamped them to a table with a wood block, and used a second wood block to snap the piece off. The amount of force needed to snap them was still surprising. No pictures of the process.

For the middle cut out I just drilled a hole, and used the scroll saw to cut a window. The finish is rough, but that gets covered up.

The sheets are held on by the original feet with some longer M3 button head screws.

After scoring and snapping sheets

Hole cut-out using scroll saw

Spacer piece in place

Completed case bottom with rubber feet

Case front

Shiny case bottom

Not shiny edges

[2024-05-05 Sun] More memory

I wanted more memory, but didn't have it, or did I?

I had a single 16GB stick, and forbidden had a 16GB stick. I looked at them, and they are both Micron chips. The first stick was marked D9TBJ, and the other D9TZX.

ShareVDI/Vnopn single stick

Stick from forbidden (N100 laptop)

Micron has a FBGA parts decoder that will give you the part number. Then you can look up the data sheet, which lets you decode the last bit of the model number, and actually tell how fast your chips are.

FBGA Code	Part Number	Speed	CL
D9TBJ	MT40A2G4WE-083E:B	2400	16
D9TZX	MT40A2G4WE-075E:D	2666	18

Noting that these were both marked as DDR4-2666 you can see that cheap RAM sellers like to inflate numbers. Anyway, my old sticks were Transcend TS1GSH64V6B (DDR4-2666 CL19) so even the slower stick is a slight improvement.

When I loaded them up the machine chose DDR4-2666 CL19, so I ended up at the same speed, but now I've got 32GB.

• Memtest86+ bandwidth weirdness
  

  Of course I had to run a memtest to ensure these two different sticks got along. I got worried when I tested this machine before because the bandwidth Memtest86+ reports seems quite low.

  Intel says their i5-8500t has a max of 41.6 GB/s.

  Memtest86+ reports 14.2 GB/s. Meanwhile I'm also running a test on my i5-3210M Mac Mini that reports 15.7 GB/s for DDR3-1600. What gives?

  Memtest86+ testing 32GB

  What should you do when a benchmark disagrees with your expectations? Run another benchmark.

  I needed a Linux one, because Linux. A search found sysbench, and this article on how to do it. In this article a curious thing was said:

  Make sure to run sysbench with only 1 thread, when testing ram speed. If more than 1 thread is used, the reported speed will be higher. So if you use the –threads option it should always be set to 1.

  Why wouldn't I want the speed to be higher? But performance was worse than even Memtest86+ reported, only 13 GB/s. So naturally I ran it with 6 threads on my 6-core CPU. Now I get 41 GB/s.

  The mystery numbers Memtest86+ reports seem to be single threaded.

  I found a similarly concerned issue, and that person was nice enough to link to the relevant code which is indeed just some single threaded assembly.

  Mystery solved, right? Probably, I'm no expert.

[2024-05-16 Thu] Partial drive solution

Due to the limited space I needed some creative solutions for adding storage.

Initially I failed to find a low-profile NVMe + SATA combo PCIe adapter so I designed my own. I ordered some PCBs, but then I realized I forgot to adjust trace lengths. When I got the PCBs the prep by JLCPCB was a little disappointing. I made the PCB taller because it was a requirement for V-cut on the edge connector, but it looks like the cutter barely touched it.

It would have looked something like (missing M.2 connectors):

SATA side

PCIe side

Before I got around to ordering components I stumbled on Silverstone's SST-ECM28. It's a little pricy at $27, but I guess you can do that when you're the only one making such a thing. The components to complete my boards probably would have cost as much, and I might have needed to redo the board anyway. This seemed like a good compromise for now.

I stuck in a 1TB SATA SSD, and a 16GB Optane (mostly to verify functionality).

It looks like:

Optane on Silverstone PCIe adapter

Above view of installed PCIe M.2 adapter

I still need to find a suitable 2-4TB drive, which might end up being spinning rust.

This still leaves a free SATA port. If I want there to be a 2.5" drive I can cut a hole in the old drive mounting tray (for CPU fan clearance), and add it back.

This also leaves the Mini-PCIe, and M.2 E Key sockets to fill. They might get a Coral TPU, and 2.5GbE interface respectively.

[2024-08-09 Fri] Added a Coral Mini PCIe TPU

I put the Coral TPU in. There's a software install and configuration guide at Coral.ai docs. Naturally things are out of date. The docs still reference the deprecated method for adding apt keys. The pycoral in the repository is incompatible with the newer python in Debian bookworm. There has been some movement through the past year, and even past week, on getting things updated: https://github.com/google-coral/pycoral/issues/137

Since I can't do the pycoral demo I guess I'll just test it when I get around to migrating frigate.

Mini PCIe Coral Edge TPU installed

[2024-08-21 Wed] A 2.5GbE card

With the last spare slot I added a M.2 RTL8125 based 2.5GbE adapter. This gives the machine 3 NICs. They are for regular net, camera network, and storage network.

This is just to enable fast bulk transfers when it comes to updating my video library.

I cut a little hole, and drilled a couple screw holes. Ended up having to clip the cover as well.

12.11.11

12.30.07

[2025-08-15 Fri] Docker migration

I wall mounted pixy, and set up the ethernet connections in anticipation of adding some extra frigate cameras.

I was able to use the side foot mounts and a couple L brackets for a nice upside down wall mount. This keeps the fan intake at the bottom, and vent at the top for natural convection cooling. There is even enough space to sit the AC adapter on top without blocking the vents.

Mounting brackets with power supply

Ethernets + Dummy DisplayPort plug

Theoretically the migration should be kind of straightforward. I need to:

• migrate the docker containers and volumes
• export the zfs pool (external USB drive)
• shut down the old system
• import the zfs pool
• try to run the containers
• fix breakage
• migrate frigate storage to SSD
• add coral accelerator to frigate

Here is how it actually went:

• Fail to migrate the docker images
  

  My docker containers are run by docker compose, and some are probably on old images. I didn't want to upgrade during the transfer. I read that I could use docker save on the tags, then copy the archive over and import them. First, I did:

  sudo docker save -o yamato.tar ghcr.io/blakeblackshear/frigate plexinc/pms-docker homeassistant/home-assistant jellyfin/jellyfin
  

  Then on the new host:

  sudo docker import hostname.tar
  

  Unfortunately this created a massive combined image with no associated information. Completely useless.

• Migrate the docker images (kind of worked)
  

  My second try was to create one archive per image:

  sudo docker save -o yamato-frigate.tar ghcr.io/blakeblackshear/frigate
  sudo docker save -o yamato-plex.tar plexinc/pms-docker
  sudo docker save -o yamato-ha.tar homeassistant/home-assistant
  sudo docker save -o yamato-jellyfin.tar jellyfin/jellyfin
  

  Now when importing the docker image, you also need to specify the repository.

  sudo docker import yamato-frigate.tar ghcr.io/blakeblackshear/frigate
  sudo docker import yamato-plex.tar plexinc/pms-docker
  sudo docker import yamato-ha.tar homeassistant/home-assistant
  sudo docker import yamato-jellyfin.tar jellyfin/jellyfin
  

  The result isn't quite as expected. The image on the new machine is slightly larger, and it has a different image id hash.

  sudo docker image ls (old machine)Hide

  REPOSITORY                        TAG       IMAGE ID       CREATED         SIZE
  ghcr.io/blakeblackshear/frigate   stable    ​79c58d895643   4 months ago    2.71GB
  plexinc/pms-docker                latest    ​b38775eb69f9   4 months ago    353MB
  homeassistant/home-assistant      latest    ​c50a3dd3f13c   9 months ago    1.79GB
  jellyfin/jellyfin                 latest    ​49572434ce4c   13 months ago   1.04GB
  

  sudo docker image ls (new machine)Hide

  REPOSITORY                        TAG       IMAGE ID        CREATED          SIZE
  ghcr.io/blakeblackshear/frigate   latest    ​d75574ec77dc   4 minutes ago    2.75GB
  plexinc/pms-docker                latest    ​a98c75f9b6fa   26 seconds ago   359MB
  homeassistant/home-assistant      latest    ​60684c70a59e   3 seconds ago    1.93GB
  jellyfin/jellyfin                 latest    ​b34dd069e9b3   38 seconds ago   1.04GB
  

  We're going to ignore that and move on to copying the volumes.

• Migrate the docker volumes
  

  First we recreate the empty volumes. Do a docker volume ls on the old host. Then docker volume create <volume name>, once per volume, on the new host.

  Then I tarred up the whole volumes directory, copied it over, and moved each directory on top of the new empty ones.

  Filesystem resizing side questHide

  At this point I ran out of space. There was a side quest of resizing some LVM volume groups and their filesystems. Unfortunately I also needed to change the root partition. So I loaded up the Debian rescue. I learned the easy path was broken because lvreduce's --resizefs depends on fsadm which isn't included. I tried copying it over, but it is a bash script, and rescue has no bash. Lots of resize2fs, lvreduce, and lvexpand happened. Onward.
  

  The volumes I created disappeared during the reboot (probably because I deleted their directories), so I created them again. I emptied the resulting directories and copied the old files over them.

• Copy the docker compose files
  

  Just then I remembered to transfer the docker compose files over, and I read through them for any missed items. Jellyfin's had a reference to the old hostname which I updated.

• Export the ZFS pool, bring down the old system, import the ZFS pool
  

  Now it was time for the ZFS transfer. A simple zfs export, and import (after installing ZFS).

  I also had a bind mount, and NFS export, but these were done traditionally rather than through ZFS features so I copied them into the new fstab and exports file. I did a quick reboot, and everything seemed to be correct. Kerberos config can come later.

• Try running the containers
  

  Now we launch the containers and expect a little breakage.

  My plex container failed immediately. It errored like:

  sudo docker-compose -f docker-plex.ymlHide

  Creating plex ... error
  
  ERROR: for plex  Cannot create container for service plex: No command specified
  
  ERROR: for plex  Cannot create container for service plex: No command specified
  ERROR: Encountered errors while bringing up the project.
  

  I tried manually pulling an updated plex image, and it began working

  I had the same issue with jellyfin.

  The frigate container redownloaded itself, and started up normally. Cameras were running, and old content was accessible. Woohoo! There is some weirdness with two frigate images showing, and neither is getting pruned.

• Coral accelerator configuration
  

  I added the Coral device to the docker compose file. Then changed the frigate config over to the Coral PCIe TPU. The old model stanza (for OpenVINO in my case) is just deleted, no need to specify one.

  I have been getting some PCIe errors from dmesg. Haven't taken time to look into it yet.

  Coral TPU (apex) PCIe errorHide

  pcieport 0000:00:1d.4: AER: Multiple Corrected error message received from 0000:05:00.0
  apex 0000:05:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
  apex 0000:05:00.0:   device [1ac1:089a] error status/mask=00000041/00006000
  apex 0000:05:00.0:    [ 0] RxErr                  (First)
  apex 0000:05:00.0:    [ 6] BadTLP
  

• Frigate data to SSD migration
  

  I moved the frigate media over to an SSD. It involved stopping the container, copying the files (200GB at 50 MB/s), updating the docker compose file, and restarting.

• NTP setup
  

  I nearly forgot that yamato was an NTP server for the cameras. I set up OpenBSD NTP daemon again. Cameras were addressing it by IP so I went ahead and gave pixy the old yamato IP. It won't be a problem since yamato is disconnected from the camera net now.

[2025-08-27 Wed] fbdev output and system-wide pulseaudio

• Some playback testing
  
  • ffmpeg
    

    The first direction I found was to use ffmpeg, and output to the fbdev device.

    ffmpeg -i Akira\ \(1988\)\ -\ 1080p.mkv -pix_fmt bgra -f fbdev /dev/fb0
    

    This got video on the output, but there were some issues:

    • playback is very fast
    • no audio
    • video is played at native resolution

    I don't think ffmpeg can do this job. It just pipes video, and the fbdev seems to accept all frames as fast as possible, and then display what it can.

  • mplayer
    

    I also tried mplayer's fbdev2 output.

    mplayer -vo fbdev2 Akira\ \(1988\)\ -\ 1080p.mkv
    

    This started off looking reasonable. Playback is a little choppy, but it has other problems.

    • part of the console login prompty is visible
    • console text isn't cleared unless in the video display area
    • the cursor blinks through the video (probably have to disable video console)
    • no audio (fixed with system-wide pulesaudio below)
    • not fullscreen

    To fix the fullscreen issue I ran fbset, and noticed the mode was 1024x768. Then I ran fbset -fb /dev/fb0 -g 1920 1080 1920 1080 32.

    I don't know if 32-bit color is reasonable, but /sys/class/graphics/fb0/device/modes at least indicates there is a 1920x1080 mode.

    Next mplayer has to scale the output. You can pass -vf scale​=1920:-3 (use original aspect, scale to width). Other methods such as zoom, and switching with key shortcuts doesn't seem to work. Unfortunately the playback is prone to tearing.

    —

    mplayer also has SDL output that can use the fbcon device.

    sudo SDL_VIDEODRIVER=fbcon SDL_FBDEV=/dev/fb0 mplayer -vo sdl -framedrop Akira\ \(1988\)\ -\ 1080p.mkv
    

    • needs to be run as root
    • delayed start-up, about 5 seconds
    • no audio (fixed with system-wide pulesaudio below)
    • no fullscreen without scaling like above, suffers the same issues
  • mpv (mpv.io)
    

    I had an unexpected problem with mpv. It appears it was outputting to my dummy plug that I put on the DisplayPort output. Once I removed it, playback works great.

    mpv --vo=drm Akira\ \(1988\)\ -\ 1080p.mkv
    

    You can also specify the output manually like:

    mpv --vo=drm --drm-connector=HDMI-A-2 Akira\ \(1988\)\ -\ 1080p.mkv
    

    Run mpv --drm-connector=help for a list of outputs.

  • vlc
    

    There were some mentions of a DRM​/KMS backend that could be specified while using the cvlc frontend. It looks like this is unofficial.

• Fixing audio
  

  We're going to run the pulseaudio daemon system-wide. I installed pulseaudio, and followed Running PulseAudio as System-Wide Daemon.

  The general steps are:

  • disable the systemd service and socket globally
  • disable autospawn in /etc/pulse/client.conf
  • add users to pulse-access group (including root if needed)
  • make a new service to run pulseaudio with --system as root
    • systemctl edit --force --full pulseaudio.service
    • Copied from https://wiki.archlinux.org/title/PulseAudio#Starting_system-wide_on_boot

  I did a relogin, and audio was working for the mplayer methods without any command changes.

• Trying to fix other things
  

  mpv's drm output seems best so far, with mplayer's SDL output coming in second. I'd like to run mplayer as a normal user.

  I tried:

  • adding my user to tty group
  • giving the tty group write access to /dev/console
  • giving the tty group write access to /dev/tty0

  Noting that my user was already in the video group for /dev/fb0 access, this still did not work. VO_SDL reports that it is unable to open the console terminal.

  I checked with strace, but I wasn't finding any access that seemed like a problem.

• Multiple outputs
  

  It is sort of possible to run multiple outputs on a single card using mplayer's fbdev, and mpv's drm. I found out from h1x's stackexchange answer.

  mplayer's SDL mode stopped working while I was trying this. After a reboot it seems to work the same way, and isn't as prone to tearing. It appear what broke it was pressing 'f' during playback. It crashed and then SDL mode would no longer display anything.

  While you can do multiple video outputs, you will need a second sound card if you want the audio separate. You could use a HDMI audio embedder to recombine the audio, or some devices, like my modulator, have an analog input.

[2025-10-01 Wed] Debian trixie upgrade and coral gasket breakage

sed -i -e 's/no_llseek/noop_llseek/g' gasket_core.c