A recent correspondent put me in mind of the fact that, having at last settled into my new home, it was perhaps time to do a sort-of guided tour through it as far as explaining what, technologically speaking, makes my music play. I notice, for example, that just a few weeks ago I took a photo of my music 'cabinet' in which an Apple Mac Mini (vintage 2012) took pride of place as the music-producing PC. Well, that's not true these days (as regular readers will probably know, I suffer from a bad case of operating system re-installationitis, with consequent side-effects of PC moving-aroundism):
As you can see, I don't exactly do a lot of dusting! As you can also see, the Mac Mini has been replaced by my Raspberry Pi 4: though it can have a fan fitted to keep it cool, mine is housed within an aluminium case that acts as a very good passive heatsink. Temperatures are therefore seldom above 50°C, and I've accordingly not bothered to fit the fan: the Pi is completely silent in operation in consequence ...and that's a key requirement of any IT equipment that sits in my music listening room (for hopefully obvious reasons!)
So what has become of the Mac Mini? Well, it's here:
It's there, tucked into the corner of my desk in the listening room. It replaces a Lenovo ThinkCentre M93p Tiny PC, which had developed a weird symptom of being 100% utilised on one of its 4 cores, with a corresponding tendency to whine with its fan ramped-up to maximum speed. I didn't spend a huge amount of time trying to diagnose the problem, but with the need for silence paramount, it was easy to plan switching the Mac Mini for the Lenovo: the Mac Mini does have a fan, but I never hear it. If I was allowed to spend actual cash on stuff, I might well switch the Mac Mini for one of the Mele Silent PCs, which genuinely have no fan and rely entirely on passive cooling. They're not going to break world speed records for computing, with their Celeron processors, but they are probably all that I actually require. Short of being allowed to spend up for one of those, however, the almost-practically-silent Mac Mini will "do". It's vintage 2012, but sports a decent-enough Intel i5 processor and is what I do all my music ripping, cataloguing and integrity-checking on.
The Mac Mini doesn't run Apple software, I hasten to add. I tried manfully to enjoy the Apple experience a while back, but hated practically every minute of it! That's probably more to do with me and my habits than actual deficiencies in Apple's operating system, but it nevertheless means I won't be touching Apple's operating system again. So no: the Mac Mini runs Manjaro Linux, not any version of MacOS known to Mankind. The Raspberry Pi, incidentally, runs vanilla Raspberry Pi OS (which is Debian-based), but with a KDE Plasma desktop environment installed on top of the default. Let no-one tell you that the Raspberry Pi 4 is not up to running KDE/Plasma: it runs it just fine, though most of my interaction with it is via the ssh-enabled command line, Mac Mini to Pi.
So that's the heart of my classical music listening experience: an actually-silent Raspberry Pi 4B, and a practically-silent Mac Mini. In passing, I will note that it's not a great idea to have a desktop fan spinning away in your music room, but it's been hot and humid, and the English don't generally do air conditioning very well! Sometimes, one's art has to suffer for the sake of the practicalities of not sweltering to death. 🙂
So the next question is: where does the music that the Raspberry Pi plays come from? Welcome to the wonderful world of data storage, data integrity and data redundancy! Here's my 'server room' (otherwise known as the loft):
It's a bit of a mess, but in that gloomy picture, you can just about make out a couple of old HP Gen 7 servers, an iMac, a printer, and a couple-of-somethings over on the far left. Well, let's pick that apart:
So this is a close-up of that far-left bit of things. There are three PCs/Servers: the one on the far left is Lalande, a Dell PC running Windows 10: I'll come back to what he's doing shortly. The two machines next to Lalande (moving left to right) are JSB and Absbach. Absbach is the machine that runs this website: it's got a 4-disk storage array and a Xeon E3-1230 v2, running at 3.30GHz. JSB is an almost identical server (another HP Gen 8 job, purchased in 2012), with another 4-disk storage array, but using a Xeon E31260L running at 2.40GHz: it's this machine which hosts our music and movie collection and which the Raspberry Pi4 is directly connected to via an NFS mount. So when Raspberry Pi4 plays Mozart, the music is being streamed across the network from this server. JSB copies its movie and music collection to the Absbach web server once a week, so if JSB is the "primary source" of data, Absbach is the backup of it, albeit delayed by a week: if I screw something up on JSB, I have until the next week to un-screw it by copying stuff back from Absbach.
Moving on to the far-right side of the loft:
Most obviously, you'll see here an iMac (vintage 2012ish), on which I run MacOS Catalina as a test platform, to make sure that Giocoso, Semplice and Niente all run on it without drama. It otherwise has no particular use to me and so, unless a software release is imminent, is most often turned off. The more important machines are, in fact, to its right: two HP Gen 7 servers (with feeble Turion CPUs, but each with a 4-disk storage array). One is 'live', is called "Sibelius" and takes a copy from JSB every day, and synchronises with it every Friday. If I delete something important from JSB, I can recover it from Sibelius until at least the next Friday, therefore. Sibelius is thus my "immediate, online backup of JSB".
The other is called "Nielsen" and is an identical copy of Sibelius, except that it is usually switched off. It's powered on over the network (via a 'magic packet') whenever I feel like it, so that I can manually invoke a script that copies across everything Sibelius has -and having copied, it simply shuts itself down again. I usually aim to power on Nielsen once a month, but sometimes I skip a month. Whatever the specifics of its backup schedule, Nielsen represents my "offline backup": should a virus or worm start encrypting stuff on my network, they're going to have a problem encrypting Nielsen, because it's not actually switched on! Nielsen is my on-site backup of last resort, in other words.
Now: let me just back up (hah!) a bit and re-introduce you to Lalande:
It's actually just a Dell PC, with no special server hardware or resilience. What is does have, however, is an 8TB hard disk -meaning that it can comfortably house my entire movie and music collections in one place. Obviously, if that hard disk were to fail, I'd lost the lot, which is why I don't rely on this PC's continued existence: there is no redundancy here! The reason for Lalande's existence, however, is that it can run Windows 10, and thereby run Backblaze, a piece of software that allows you to backup everything your PC can 'see' to "the cloud", for the princely sum of £60 per year. So Lalande copies everything off JSB once a night and then copies across to "somewhere" in the Cloud: Lalande is, therefore, my offsite backup. Should my house burn down and take all my servers with it, the theory is that at least my music collection is safe, courtesy of Backblaze's servers somewhere in Europe.
So there you have it: Raspberry Pi plays music stored on JSB, which is thus the 'source of truth' for all things musical and movie-related. JSB copies itself to both Absbach and Sibelius at different schedules, so those two machines permit me to recover data mishaps differentially, depending on whether I notice the problem within a day or two or within a week or so. One data set, two backups.
Nielsen is a third backup that is seldom switched on, so that's my offline backup: if everything else is encrypted by ransomware, Nielsen should be safe.
Lalande is my backup of last resort: it's another copy of JSB, but held somewhere in Europe. Neither earthquake nor meteorite strike can rob me of my movies or music, provided Lalande's backups are still extant.
I now return you to my music listening room:
Tucked away on a side-table are two hard drives that I manually use to backup my movie and music collections. One is 5TB traditional hard disk; the other is a 4TB solid data hard drive. I use the traditional disk to backup the movies manually, on an entirely intermittent and 'when I feel like it' basis. The solid state disk is for taking backups of my music collection, on the same 'when I feel like it' schedule. These are two USB drives: if they're not plugged in, they can't be encrypted. They don't offer redundancy or resiliency against hardware failure, though, so they are not backups I 'care' about. They're there should everything else fail, including the cloud.
Summing up, therefore:
- I have a music and movie collection I care about, stored on JSB.
- JSB is copied to Absbach
- JSB is copied to Sibelius
- Sibelius is copied to Nielsen
- JSB is copied to Lalande
- Lalande is copied to the cloud
- JSB is infrequently copied to USB solid state and traditional hard disk storage
The loft therefore contains at least the source movie and music data (JSB) and three mostly-up-to-date backups of it (Absbach, Sibelius and Lalande). A fourth backup (Nielsen) is guaranteed to be a month or two out-of-date, but to be offline when the ransomware comes knocking. Lalande itself then creates an off-site backup. Two other manually-produced backups on assorted, portable hardware means there are at least seven copies of my music collection at hand.
Is this over the top? Probably, but I wouldn't want to be without the multi-layered redundancy my storage strategy provides. If I lost my movie collection: fine, I could re-rip everything from the source DVDs and deal with it. Nothing much would be lost, because once you've ripped a movie, you're done, pretty much. But losing my CD collection would be a disaster, since re-ripping the CDs is merely the start of the cataloguing fun! A ripped CD needs to be tagged, split into appropriate recordings and so on... lose all that work I've done over the past 25 years and I wouldn't be happy! Seven backups, online, offline and off-site seems to me the bare minimum of what's required to be comfortable in the digital music world we now inhabit!
I finish with one last tour of the loft:
It's another indication that I need to learn how to dust -as well as to not prop up printers on whatever packing boxes happen to be at hand! But here's my old Dell Workstation, sporting a 10-core, 20-thread Xeon, 192GB of RAM and 8TB of disk space. I use it as a Proxmox Server, meaning that it hosts around 27 different virtual machines, which I use to test and ensure that my Giocoso, Semplice and Niente software packages work as advertised on around 26 different Linux distros and Windows 11.
In conclusion, I thought I'd just show you the internals of the JSB server which is central to my music-listening experience:
I would recommend the HP Generation 8 microservers to anyone that can lay hands on them: those 4 disk bays are not hot-swappable, but they can accommodate anything you can throw at them, in terms of capacity. JSB has four 4TB hard drives in a ZFS 1-0 array:
hjr@jsb:~$ zpool status
pool: bulkdata
state: ONLINE
scan: scrub repaired 0B in 0 days 07:58:23 with 0 errors on Sun Aug 20 08:58:24 2023
config:
NAME STATE READ WRITE CKSUM
bulkdata ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-1ATA_ST4000VN008-2DR166_ZGYA1B4H ONLINE 0 0 0
scsi-SATA_ST4000VN008-2DR1_ZDHAQ35C ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
scsi-SATA_ST4000VN008-2DR1_ZGYA11SH ONLINE 0 0 0
scsi-SATA_ST4000VN008-2DR1_ZGYA1A41 ONLINE 0 0 0
errors: No known data errors
Thus an 8TB stripe is mirrored to another 8TB stripe, and by using the ZFS file system, I get resiliency against hardware failure (I can survive two hard disk failures without data loss, provided the right drives fail in each case!) and ZFS checksumming means I know (as the display tells me) that I have no corruption errors with my data. I wouldn't want to store my music (or movies, come to that) on any storage 'unit' that wasn't running ZFS, put it that way. It's the only way to be sure that your data isn't silently corrupting or is vulnerable to hardware failure.
One final thought: Absbach runs Ubuntu; JSB runs Debian; Sibelius runs FreeBSD and Nielsen runs OpenIndiana (a sort-of open source clone of Solaris). Why do I faff around with four different operating systems (three, if you're going to insist that Ubuntu and Debian are both versions of Linux!)? Because I never want to store my data on a bunch of servers that could all be knocked out by the same security vulnerability or operating system patch. If Debian suddenly breaks ZFS by releasing a kernel patch, my Ubuntu system should probably be safe for a week or two, at least. If both Debian and Ubuntu are knocked sideways by some intrusive security breach, the FreeBSD system should probably be safe.... and OpenIndiana is such an obscure operating system, it's likely to be safe though even the Heavens and Earth should fall 🙂
From all of which I hope you take away the idea that a good digital music strategy relies on:
- Redundancy of hardware
- Redundancy of software (especially operating systems)
- Number and Frequency of backups
- The availability of offline and off-site backups
As I said to my recent correspondent, I suspect that I am on the outlying edge of the bell-curve of people doing this sort of thing (in terms of expenditure, redundancy, refusal to use commercial software etc), but everyone doing 'digital classical music listening' needs to be on that bell-curve somewhere. If you subscribe to the idea that the music you care about is safe because it's on a single laptop's hard drive somewhere, you're riding for a fall and much gnashing of teeth, put it that way!! Ebay is currently listing HP Gen 7 Servers for around £200 and Gen 8 ones for around £150ish (always go for Gen 8 if you have the choice!): getting your music stored with hardware and software resilience doesn't cost the Earth, basically.
Also worth pointing out: pay more for your external DAC (digital to analogue converter) than your PC. The DAC is the thing which takes digital bits supplied from your PC and turns them into an analogue waveform we call sound: any old PC from even the year 1999 can supply bits. Hell: even a mobile phone can do so! No particular power or special PC construction skills is needed to be a bit supplier. The clever stuff happens turning bits into sound. That takes a DAC, and the more you spend on your DAC (within limits!), the better your music will sound. In the first photo above I mentioned a Topping E30 DAC -yours for around £150 at the time of writing.
I generally don't get involved in discussions about what this or that amplifier or speaker set can bring to the party: my ears cannot reliably tell the difference and I'm not bankrupting myself for an audio "experience" that will be mostly in my imagination! So, the short version is that my speakers are a pair of Dali Spektor 2 bookshelf speakers (£250) and my amplifier is an Auna which all the What Hifi-like reviews I read basically said sounds fantastic, given it cost peanuts (£100).
The main point of this blog piece is, however: forget about your music player (a simple phone streaming into a good DAC will do fine). Concentrate on the data backup/redundancy behind the player. That's where your money and effort is best spent, unless you are genuinely blessed with golden ears (in which case, enjoy spending your tens of thousands...)