Fix-Up Scripts – Absolutely Baching

1.0 Introduction

It is a core tenet of Niente that it only ever reads and analyzes your FLAC files: it never modifies them. However, by its nature, Niente will reveal past poor tagging decisions which, perhaps, affect many hundreds or thousands of recordings and which would be real pain to have to fix manually. In my case, for example, my first run of Niente Version 3 revealed this:

That's 17,000+ recordings of which over 8,100 have 'Album Art size issues' (see Statistic 9). Upon running the relevant detailed report, it became obvious that since I first started ripping my classical music CDs in 1999, I've been fairly cavalier about the album art I tagged it with! A lot of those 8,000+ "mistakes" were using album art that was too small: images that are 200x200 pixels were a thing back in the days of dial-up Internet (yes, I am that old!), but they make for lousy viewing on a modern 27" monitor (or any other device, come to that). Additionally, when I did finally start getting decently-sized album art, I didn't bother 'squaring it up', so I'm now left with a pile of artworks which are (say) 1401x1400 or even 1412x1398: all nicely large, but all resolutely oblong in nature!

You can see, too, that I have a 2,100+ problem with inconsistent PERFORMER/ALBUM tags (see Statistic 7). That turns out to be because I always enter my COMMENTS tags in conductor/orchestra/soloists order. Semplice (my tagging software) automatically takes the first of those elements and writes it into the PERFORMER tag for me. If the recording is of a concerto, however, I will probably use the soloist's name in the ALBUM tag as that of the 'distinguishing artist'. Thus, if Bernstein conducts Kissin, my ALBUM would read Piano Concerto No. 3 (Kissin - 1987), my COMMENTS would read Leonard Bernstein, New York Philharmonic Orchestra, Evgeny Kissin (piano). Semplice would have set PERFORMER to Leonard Bernstein... and now my ALBUM tag mentions a name that doesn't match with the PERFORMER tag. Again: this needs fixing, but re-tagging 2,100+ recordings of assorted concertos is not my idea of fun!

Anyway: Niente will not automatically fix up any issues it detects. That's an absolute given and will never change. You will not see me, ever, embed 'fixup scripts' into the Niente software itself. It is profoundly unsafe to bulk-update a music collection for starters and bulk updating anything completely ignores the nuance of a specific issue. The rules I think should apply may not be rules you're comfortable with in the first place, after all.

But I am also a realist and there's no way I'm manually fixing up nearly half of my collection's album art, or 10% of its PERFORMER tags. So: on this page, as a sort-of addendum to the Niente user manual, I'll publish some fix-it scripts which I've knocked together and run against my own music collection to fix up these sorts of bulk problems. I offer no guarantees the scripts work as intended, nor warranties (implied or otherwise) that they are fit for purpose. If you run them to fix up your own tagging issues, I'd strongly suggest (1) Take a backup of your collection before you start; and (2) Test the scripts against a small subset copy of your collection first, before unleashing it on the real thing. Check and verify the scripts really do what they claim to do, without completely screwing your metadata up!

As I say, I'm developing these in an ad hoc, as-required (by me!) manner. I'll post the scripts here when they're ready and only after I've run them successfully on my own music collection. Click on the script name to download the script somewhere (maybe to your Downloads folder), then chmod+x scriptname to make the download executable. Always read the script before running it: the first few lines of actual code will contain variables that need to be set and if they're not set (to use your particular Niente database, for example), then the script won't work at all. When you've read the script and edited it, then you can run it with the command ./scriptname. Keep in mind: they're intended to do a job, not look particularly nice or even to error-trap much (so long as any errors don't result in screwed-up metadata, I'm fine with not trapping and dealing with every last edge-case!)

Script Name	Purpose	Description/Comments
fixart.sh	To fix up Album Art sizing issues (Statistic 9).	The variables at the top of the script define the name and location of your Niente database; when album art should be thought to be too big (1600x1600 pixels by default); and when album art should be thought to be too small (under 500x500 pixels by default). There's also a parameter to set to say whether you care about album art being properly square or not (i.e., would a 1401x1400 piece of album art be thought an issue). Note that MacOS users need to set the GREPPROG to ggrep; everyone else can leave that one set to grep. When run, the script performs a fresh scan of album art properties for all recordings that Niente knows about. If you say SQUAREART_REQUIRED=0 then the only thing the script does is to find very large art and reduce it to 1400x1400; and find very small art and square it off to its largest dimension (so, a 220x195 image will become a 220x200 image). An image that was 906x900 (for example) would not be adjusted at all: it's non-square, but you've said that's not an issue; and it's neither too small nor too large to require adjusting. At this setting, in other words, good-sized, but non-square, artwork is left untouched. If you say SQUAREART_REQUIRED=1 then the script finds over- and under-sized art and any art which, although sized reasonably, is non-square. It then reduces very large art to 1400x1400; squares off very small art to its largest dimension; and then re-sizes 'reasonably sized' art to either 900x900 or 500x500, depending on what the nearest current dimension is. For example, a 1412x1401 image will be squared off to be 1400x1400; a 1035x917 image will be squared off at 900x900; a 634x599 image will end up at 500x500. At this setting, in other words, good-sized, but non-square, artwork is squared off to the closest-matching whole-number dimension. Apart from the modest increase in size that will always result from squaring off very small artwork, the effect of the script is generally to reduce the album size of 'problematic' artwork, which shouldn't affect perceived quality much. Squaring off will, of course, introduce some stretch or distortion to an image: if the existing dimensions are close, however, I think you'll barely notice the consequences. I'd defy anyone to readily notice the stretching involved in resizing a 1437x1410 image to 1400x1400, for example. If the distortion thereby introduced does turn out to be unbearable, it's likely because the original dimensions were so non-square that either you should obtain new album art entirely, or you should stick to SQUAREART_REQUIRED=0. See this blog article for some notes/comments about how this script works.
fixperformers.sh	To fix up inconsistencies between ALBUM and PERFORMER tags (Statistic 7).	Reads the ALBUM tag and finds the distinguishing artist's surname in the last pair of round brackets. Wants to set the PERFORMER tag to this value, but only in a 'full name' version. Therefore, it looks through the COMMENT tag, trying to find a full name of a performer that can be written into the PERFORMER tag. If ALBUM contains (Bernstein - 1966), for example, and COMMENT mentions "Leonard Bernstein, New York Philharmonic", then PERFORMER will be set to "Leonard Bernstein", as the "Bernstein" in ALBUM matches the first item in the COMMENT. If the ALBUM tag is itself wrong (e.g., "Boinstein - 1966"), then no automatic fix can be applied, but a text file report saved to your Desktop will alert you to this. If the COMMENT tag is wrong, that again is a problem: if ALBUM mentions "Bernstein" (spelled correctly), and PERFORMER="Evgeny Kissin", then the script will see there is an ALBUM/PERFORMER mismatch and start searching COMMENT to resolve it. If COMMENT mentions "Leonard Berrrnstein, New York Philharmonic", no match to "Bernstein" will be found. This again becomes an unfixable issue, and is included in the desktop text file report. Of course, the fix here might not just be to correct the spelling of 'Bernstein' in the COMMENT tag, but to add 'Evgeny Kissin (piano)' to the COMMENT tag as well: clearly the original PERFORMER tag was trying to name the piano soloist as being more significant than the conductor: that's something you may want to persist with, but the script cannot make those sorts of decisions for you! See this blog article for some notes/comments about how this script works.
fixtracks.sh	To create SuperFLACs in folders containnig multiple per-track FLAC files. (Statistic 12)	Visits folders containing more than 1 FLAC, as detected by Niente's Reporting Statistic 12. The script concatenates the audio signal in each FLAC found into a single 'SuperFLAC' and then embeds a cuesheet into the SuperFLAC that describes where each (now virtual) track stops and starts, so that compatible players can still play individual tracks when required, despite only one physical file existing on disk. Be sure to set the various parameters at the top of the file, particularly AUTODELETE. By default, that's set to 1, meaning that once a SuperFLAC has been created, the script will automatically delete the source per-track FLACs. You might want to set that to 0 until you're comfortable with the way the script runs and the concept of SuperFLACs. Also note the value of MAXSUPER is set to 9999 by default, meaning 'fix everything up in one go': when you're first getting to grips with SuperFLACs, you might want to set that to a really low number (say, 3 ot 5), and then the script will only generate 3 or 5 SuperFLACs before self-terminating. This gives you the chance to test things out in little 'bites', rather than rip through your entire music collection before you're entirely comfortable! The other variables at the top of the file should be self-explanatory: you need to set DBNAME to the name of your Niente database; the CONFDIR to the location of your Niente database (which shouldn't really need to be altered from its default); and the names of the various sed, grep and find programs, which may vary depending on the OS you're using. Apple Mac users, for example, need to use ggrep and gsed, not the Linux standard grep and sed programs. THe FINDPROG is usually fd and defaults to that, but on Ubuntu you may need to change it to 'fd-find' or just 'find'. The script will prompt to be run in 'r'eal or 'd'ryrun mode. A dry run produces a script on your Desktop which, with a little bit of editing, becomes a backup script, allowing you to copy music folders that would be changed by a 'real' run to a safe place before anything is actually done to them. Only a 'real' run makes alterations to your files. See this blog article for some notes/comments about how this script works.
fixnames.sh	To add 'bit-depth' identifiers to physical file names. (Statistic 11)	Reads the Niente database to discover FLACs with physical file names that do not include indicators for the bit-depth and sample-rate of the audio signal it contains. Visits each such file in turn and adds those identifiers to the end of the current file name. Thus a file called "01-Allegro.flac" would end up named "01-Allegro-16-44100.flac", for example. Before you run the script, check the five parameters at the top of the file: you need to set DBNAME to the name of your Niente database; the CONFDIR to the location of your Niente database (which shouldn't really need to be altered from its default); and the names of the various sed, grep and find programs, which may vary depending on the OS you're using. Apple Mac users, for example, need to use ggrep and gsed, not the Linux standard grep and sed programs. THe FINDPROG is usually fd and defaults to that, but on Ubuntu you may need to change it to 'fd-find' or just 'find'. Perform a fresh, full integrity check in Niente before running the script: it needs to know accurately which files do not contain the bit-depth and sample rate identifiers before it tries 'fixing' things. Perform a second fresh, full integrity check once the script has run to a conclusion before attempting to view the results of the script's work in Niente's aggregate statistics report. The script outputs 'F' for each file successfully renamed; it outputs a '.' (period or fullstop) if it thinks a file ought to be renamed but cannot be, because a file with the intended new name already exists in the relevant folder. These sorts of errors will need to be investigated manually. See this blog article for some notes/comments about how this script works."