Niente Version 4 : Reporting – Absolutely Baching

1.0 Introduction

When you perform an integrity check of your FLAC collection with Niente, not a lot appears to happen! Sure, we see the program visiting each FLAC it knows about in turn and learning things about it:

...but if it detects anything amiss during its file inspections, no red lights or sirens go off: the program will merely conclude its integrity check by re-displaying the main menu, which seems somewhat anti-climactic! So, performing an integrity check is an information gathering exercise, but the integrity check itself never shares that information with you. In order to actually see whether or not your music collection is suffering from physical corruption or logical inconsistencies, you need to use Niente's various reporting options. These all interrogate the data collected during an integrity check and list those records found to be failing one check or another.

In short: it's by running reports that you at last get to understand the health and status of your music collection.

All Niente's reports are accessible under the Reporting Menu:

You'll notice that there are three sub-menu options listed: General reports, Logical Problems reports ...and a final category unimaginitively called 'Other Reports'! You access any of these sub-menus by tapping the number next to the appropriate menu option: if you wanted to run a particular report on any logical inconsistencies in your music collection's metadata tags, you'd start by tapping '2', therefore. If you do that, you'll see this sort of thing:

The various reports that can be generated within this category are listed in the left-hand part of the screen; the three categories of reports that were previously accessed by tapping a number are now 'promoted' to appear as options on the top-bar menu. You can therefore switch between report categories easily just by pressing the left- and right-arrow cursor keys (or by tapping the initial letter of the menu required: so typing 'g' will take you to the General Reports category, for example). When you're done producing reports, just arrow over to the Main Menu item (or tap 'm') and choose from the options presented:

Option 1 takes you back to Niente's initial main menu (where you'll find the 'Reporting' option remains selected), but Option 2 is provided so that you can quit out of Niente completely with a single key-press.

When you choose to run any of Niente's reports, they will appear in a pop-up window that is independent of the window in which Niente itself is running. That window will use the font and font size you specified in the persistent configuration file, accessible via Niente's Administration menu, Option 1:

Here, you see I'm configured to display reports using the Luxi mono font, sized at 12pt. Those are actually Niente's default settings, but if you prefer your text displayed smaller (or larger), or in a different font altogether, here's where you get to choose those report characteristics. Be aware that any font you choose here should be a monospaced one: proportional fonts will completely garble the display and result in an unreadable mess. Luxi mono is a monospaced font that Niente installs for you as part of its own software installation -but you may prefer fonts such as Courier, Consolas, Andale mono and so on. It's up to you -and if you choose to type in the name of a proportional font, things will still work anyway, though reports are unlikely to be usable! Obviously, too, you should only specify a font that has actually been installed on your system: if you inadvertently type a name incorrectly and thus reference a font which doesn't exist on your system, Niente will fall back to display its reports in a ghastly, spider-thin font which is no-one's idea of a good time... but at least you'll see something!

Niente always displays its reports in a standalone 'X' window, independent of its own program window. That makes it possible to run multiple reports simultaneously: run a report, it opens in its own window; run another report, it opens in a separate window; run a third report and again a third reporting window will open. It's not necessary, in other words, to close one report before you can produce another. Sometimes, however, Niente's tendency to display its reports on-screen is not quite what you want: you may well want to save the report to disk. This is actually easy to do, however: just tap the letter s and you'll be prompted to type in a path/filename combo: press [Enter] once you've entered that and you'll find a text file version of the report is written physically to disk. Here, for example, I'm asking for the on-screen possible volume boosts report to be saved to disk:

I've already typed the 's' to initiate the 'save': now the program is prompting me for a 'log file' (which isn't the most helpful prompt I've ever seen!) and I type in a full path-and-filename combo. Note that supplied path and file name can contain spaces. I press [Enter] to submit that entry and on my Desktop I then see this file appear:

...and if I open that in any text editor of my choice, I see this:

...which is the exact same report, in exactly the same format, as it was originally displayed on-screen. All Niente reports are save-able to disk in this way.

Bear in mind that every report is produced by reading from the Niente database as it was at the time of its last integrity check. If, let's say, you produce a report that tells you four files have PERFORMER/ALBUM tag inconsistencies and you re-tag all four of those files before re-running the report once more... you're just going to see the same four files listed on the new report as they were before. The database doesn't know you've re-tagged those files until you make it realise the fact by performing a new integrity check (I'd suggest a fast check in these particular circumstances). Only after a new integrity check is performed will you see the report contents change. Similarly, if Niente tells you 3 files are physically corrupt and you restore those files from a known good backup, so that the files on disk are no longer physically corrupt, Niente will still tell you they are corrupt, because unless you do a new integrity check (in this case, a differential one would be most appropriate), its database still stores details of the original files. The new integrity check will force Niente to re-read those files and when it does so it will, at that point, notice that the internal characteristics of the files have changed and write those new, uncorrupted characteristics back to its database.

Finally, before I describe the reports in some detail, remember that when it is not reporting on the physical integrity of your FLACs, it's drawing on the Axioms of Classical Tagging to determine whether something is 'wrong' about your FLACs or not. If you do not yourself subscribe to all or some of those Axioms, the relevant reports will be of no use to you -but Niente will keep reporting them as a problem anyway! If you think recording artists should be listed in the ARTIST tag, for example, and not in the COMPOSER one, then every single FLAC you own is likely to be reported as having a 'ARTIST/COMPOSER' inconsistency. You can simply ignore such reports though, if it suits you to do so.

2.0 General Reports

2.1 Quick Aggregate Statistics (General Reports, Option 1)

Possibly the most useful report that Niente can produce, the Quick Aggregate Statistics report is a snapshot summary of the state of your music collection. It lists 15 specific statistics as follows:

Ideally, you want to see zeroes for all the rows in this report, except the first and last ones. The first row tells you how many recordings have been added to the Niente database (so hopefully that's non-zero because otherwise it means you don't have a music collection!). The last row shows the date when the last integrity check of some sort was performed and thus the date when the database was last brought up-to-date with the state of tagging and physical integrity of your collection.

Each statistic is prefixed with an identifier (such as 'L4' or 'O2'): these are the menu options you'd take to produce the report that 'explodes' that statistic out into its detailed, constituent rows. For example, the details about what 'Inconsistent PERFORMER/ALBUM tags' I have are called 'L4', which means 'Take the Logical Issues Reports menu, and run report 4 from there' (i.e., L4). The details behind the 'Recordings not yet checked' statistic, however, can be obtained by running the 'General' report number 3 (G3). Likewise, to list all the files that are missing sample rates in their file names, I'd need to run the Other report number 3, or O3.

You get the idea, I hope: each non-zero row in the Quick Aggregate Statistics report should be regarded as a trigger to produce the corresponding detailed report that will list the specifics of what's been detected as potentially going wrong.

Note that two statistics may not be listed in this Aggregate Statistics report, though they will be by default. The rows O2 ("Potential Volume Boosts") and O3 ("Missing sample rates in filenames") will only appear if you have configured the persistent configuration parameters Report on volume boosts and Report filenames without bitdepth data respectively. Both parameters default to 'yes', so those rows will appear by default. However, if you switch those parameters to 'no', because you simply aren't interested in doing volume boosts to your music files, or don't care whether their physical files contain bit-depth and sample-rate information, then those rows will disappear from the aggregate report. You can always run the equivalent detailed reports, however, even if you've turned off aggregate reporting for those particular issues.

2.2 Recordings with physical integrity problems (General Reports, Option 2)

This is probably a critical report -because even if you don't agree with one iota of how I believe classical music should be tagged, we can all agree that the audio stream of your FLAC files should not undergo any alteration from the moment it was created when you ripped the CD -and this report lists those FLAC files where that doesn't appear to be true:

As you can see, the report is not exactly big on details! That's because there's not a lot Niente can say about a possibly-corrupt file, other than that its internal MD5 hash (computed and stored with the FLAC at the very moment of its creation) no longer agrees with a new MD5 hash computed from the FLAC's present-day audio stream. That is, in fact, the very definition of 'physically corrupt'.

There is no fixing physical corruption within the file itself. The only thing you can do for files listed on this report is to replace the file -either by backup copies which you hopefully have available to you from a time before the file was first detected as corrupt; or by re-ripping the file from the original CD (or re-downloading it from whatever digital emporium you buy your music from!)

2.3 Recordings not yet checked (General Reports, Option 3)

This report details which rows in Niente database reference FLACs which have not yet been analysed to determine their physical corruption status (i.e., Niente doesn't yet know what their internal MD5 hash for their audio stream is):

Any file listed on this report will probably also be listed on the 'missing tags' report (see below). However, filling in data about missing tags requires only a logical integrity check, which can be done relatively swiftly by performing a fast integrity check. The files listed on this report are specifically missing their physical audio signal data characteristics, and that can only be rectified by performing a new full or differential integrity check (both of which will also pick up metadata tags, thus fixing the 'missing tag' report too

2.4 Report on Missing or Empty Tags (General Reports, Option 4)

You can forget to supply data for specific tags quite easily -and sometimes, physical corruption of a file will result in tags being unreadable. This report will list the files affected by such problems, along with specifics about what tag in particular is missing or empty:

As mentioned above (see Section 2.3), files listed on this report have not had logical metadata tags read; or they have been read but been found to have one or more 'compulsory' tags missing. If you were to fix up the tagging for these files, you could then run a fast integrity check to pick up the changes. It is quite likely, however, that a music file which has not had its logical metadata read will also not have had its physical MD5 hash computed for its audio signal, so the same file would be listed on the 'not yet checked' report. However, a fast integrity check will only read the now-supplied metadata tags, so though the file would then disappear off this report, it would still be listed on the 'not yet checked' report, because fast integrity checks do not analyse the audio signal in order to compute its MD5 hash. A differential or full integrity check would remove the file from both reports, though, as those sorts of checks do both MD5 hash calculations and logical metadata reads.

3.0 Logical Issues Reports

I won't detail the five reports in this section. They all work in ways similar to what you've already seen in Section 2.0 above so should be familiar in operation in that regard. The point to make about them all, collectively, is that they are fundamentally 'enforcing' the rules of tagging I set out in my article on the Axioms of Classical Tagging. If you agree with all or most of the axioms presented there, then all or most of these five reporrts will prove useful to you. For specifics about what Niente expects to be present in which tags and what logical consistency checks it will therefore perform, please see the Logical Inconsistencies section of this user manual.

If, however, you fundamentally disagree with my proposed tagging strategy, then you will probably want to ignore all these reports! If you don't write your ALBUM tags as 'composition (performer - year)', for example (see Axiom 5), then you won't have any need for a report that checks whether the performer component of ALBUM matches the PERFORMER tag, or whether the recording year component of ALBUM matches the YEAR (or DATE) tag.

That's not true of the General or Other reports, on the whole: even if you tag your FLACs in a wildly different way to me, reporting on tags which are missing, or which are possible duplicates of each other, or which are physically corrupt sound like good things to know about.

So, I guess my point is: the reports in the 'Logical Issues Reports' may not be to your taste, but the data is there, should you ever change your mind!

4.0 Other Reports

4.1 Recordings with album art issues

This report requires that you have first performed a special 'album art scan' of your music collection that checks for embedded album art, of a reasonable size. Such an analysis can be performed by using Niente's Integrity Checks Menu Option 4. The album art scan determines the horizontal and vertical pixel dimensions of every piece of embedded art in every FLAC file in your collection and this report can then show which files contain album art which is considered 'not ideal':

The criteria for something appearing on this report are as follows:

Are the horizontal and vertical pixel dimensions not equal to each other. Niente expects album art to be square by default, so non-equal dimensions are considered an issue. If non-square art is not actually a problem for you, you can turn off Niente's demand for square art by setting the 'Should album art be square' option in the persistent configuration file to 'no' (See Administration menu Option 1).
Are the horizontal and/or vertical pixel dimensions smaller than the small art threshold which defaults to 300 (but can be set to anything you liked, by setting the Threshold (pixels) for small art option in the persistent configuration file.)
Are the horizontal and/or vertical pixel dimensions larger than the large art threshold which defaults to 1600 (but which, again, can be set to anything you think more suitable by setting a value for the Threshold (pixels) for large art option in the persistent configuration file).

The report regards large album art as a problem, since anything substantially bigger than around 1400 pixels will be consuming a lot of disk space for no appreciable extra quality when it's displayed. A lot of my artwork is 3000x3000 pixels, for example -but as my monitors are only 1980x1200 at best, there's seriously no point in me storing that amount of picture!

Artwork that's too small just displays as a blurry, washed-out smudge when displayed at reasonable sizes on a reasonable monitor: as per Axiom 16, Niente would prefer album art to be larger than a metaphorical postage stamp!

For anything listed on this report, the fix is to re-acquire better album art and manually apply it to the FLACs in question (using a tool such as Semplice, for example).

4.2 Recordings with possible volume boosts

This report requires that you have first performed a special 'volume boost' scan of your music collection, which can be done by taking the Integrity Checks menu, Options 5 or 6 (option 5 checks or re-checks every FLAC for its loudness; option 6 only checks those files which haven't previously been checked for its loudness).

The Other reports, Option 2 will then list all recordings which Niente knows to have peak volumes lower than the theoretical absolute, non-distorting maximum possible volume, which is set in the persistent configuration file (See the Administration menu Option 1) as the Threshold (dB) for volume boosts. By default, that parameter is set to 2 (i.e., recordings which are 2dB or more quieter than their maximum possible non-distorting volume would be listed on the report), but you can obviously set that parameter to any value you deem appropriate.

Once that audio scan has been performed, however, the report can be produced and will look something like this:

In this example report, you'll see that one of my Alessandro Striggio choral works could be volume-boosted by a whopping 10.9 decibels, were I to use a tool like Semplice to do so. This does not mean that it's a good idea to actually perform that sort of volume boost: one organ recording I have in my collection has a peak volume some 19dB lower than its maximum possible loudness, but when you apply a volume boost to it, you quickly discover why: it was recorded in a cathedral that can be heard to be full of footsteps and people walking around! The recording engineer decided to produce the CD at its very quiet level precisely to hide that 'noise floor' under the more foreground sound of the organ. Apply a volume boost to it and that noise floor becomes so prominent that it's difficult to focus on anything but the noise! So, just because recordings are listed as boostable does not mean that they actually should be boosted! Making that sort of decision is why you have a pair of ears 🙂

You will note, too, that this report listed folders, not files. Whilst the 'check for volume boosts' integrity check does work out the peak volume in each individual file, this report aggregates and summarises by folder. It would, after all, be pointless to tell you that file 02-Lento.flac could be boosted by 8dB if file 01-Prestissimo.flac already has a peak volume that is at the theoretical, non-distorting maximim: in such a scenario, you couldn't volume-boost just file 2 without completely screwing up the relative loudness between the two tracks. You also couldn't boost both files by 8dB, because that would make file 1 sound ghastly. Therefore, confronted with two files with these loudness values, Niente will leave the entire folder off the report: nothing in that folder can be boosted without either over-boosting file 1, or changing the relative volume between files 1 and 2.

Note that Niente is not going to alter the volume levels in any file. It merely reports on those recordings which a tool like Semplice, for example, would be able to boost the volume of, as a completely separate exercise. It is therefore entirely safe to perform a volume boost check and to report on the results: nothing is actually modified in your music files by performing either action.

Finally, recall that the persistent configuration file has a setting for 'Report on volume boosts', which is 'yes' by default, but could be set to 'no'. Whatever it's set to, it makes no difference to your ability to run this report. You may not be interested in volume-boosting your music collection, so the data it contains is irrelevant to you: that's fine, but you are still allowed to generate the report should you want to out of idle curiosity!

4.3 Files missing bitdepths in names

Not everyone will approve of the idea behind this report, but if you use Semplice to tag your FLACs, that program has a feature called 'Force filenames to display bitdepth/sample rates' (sometimes called 'NAMEBITS'), as you can see from this article (and specifically Section 2.6). This clue is in the name: switching this feature on means that when Semplice is applying metadata tags to a FLAC, it will incorporate the file's sampling rate and bit-depth into the file's physical name. Thus, a file that would 'ordinarily' be called 01-Allegro.flac will be renamed to be something more like 01-Allegro-16-44100.flac. The numbers tells you the audio signal is a 16-bit one, sampled at 44.1KHz -which probably means it was sourced from a standard audio CD, since those are the 'red book' audio CD specifications. Alternatively, you might see the file be renamed to be 01-Allegro-24-88100.flac, which tells you the audio signal is 24-bit, sampled at 88.1KHz: that would tend to indicate that the audio was sourced from an SACD, since those higher sampling rates and bit-depths are characteristic of 'high resolution' audio sources. The idea in any case is to make it easy to distinguish your high-resolution FLACs from your standard-CD-audio ones using nothing more clever than your operating system's standard file manager: you'll spot the difference between them simply because the file names tell you the difference!

The Other reports, Option 3 therefore lists any FLAC that has been found not to include the sample rate and/or bit-depth figures in its physical file name:

The 'fix' for anything listed on this report is to open a terminal, 'cd' to the relevant folder, launch Semplice and then press 'q' to quit (having first ensured that that program's persistent configuration file has NAMEBITS=1!). The quitting process will re-name the file automatically. You will, of course, need to do a new differential integrity check before Niente becomes aware of the fixes you've applied.

Finally, recall that the persistent configuration file has a setting for 'Report filenames without bitdepth data'. That defaults to 'yes', but could be configured to have a 'no' value. Regardless of what that parameter is set to, though, it makes no difference to your ability to run this report. If including the sample rates and bit-depth in your physical file names is not something you actually want to do, of course, then it's fine to ignore the existence of this report: but Niente collects the base data, anyway, every time it performs an integrity check, so you are always able to generate the report should you ever fancy doing so.

4.4 Albums with multiple physical files

This is another report that anyone not using Semplice to do their FLAC tagging might not be interested in and therefore needn't ever run!

One of Semplice's options is to create 'SuperFLACs' (see this section of the Semplice user manual for an explanation). A SuperFLAC is a single FLAC file that contains within itself all the music making up a recording of a composition, along with a cuesheet that explains where each of the original tracks starts and stops within the single SuperFLAC. You might start with files 01-Allegro.flac, 02-Lento.flac, 03-Molto allegro.flac and 04-Presissimo.flac as the four movements of a symphony, for example: once you've used Semplice's SuperFLAC option to concatenate or merge them into a SuperFLAC, however, you'd be left with just a file called something like 01-Symphony No. 32.flac. Philosophically, the creation of SuperFLACs is a physical representation that a 'composition' is a single, unified whole, even if it is comprised of multiple, separate movements: their might be three movements to a piano concerto and four to a symphony, but they're still just a single concerto or a single symphony.

If you don't subscribe to that philosophy, of course, then

It's not something that everyone will want to do, but I like to do it -and I accordingly wanted Niente to report on all those recordings that I'd forgotten to do the SuperFLAC concatenation to! The report produced from Niente's Report Menu Option 12 will list all folders that contain more than one FLAC -which, by definition, means no SuperFLAC has been created there:

Each folder is listed once with a count of the number of FLAC files found within it. As ever, Niente won't fix any of these issues for you: you need to visit each row of the report's physical folder in turn and invoke Semplice there to do that. A new integrity check would then be needed to get Niente to notice that you've SuperFLAC'd a bunch of files. Remember: Niente only ever reports on things. It never actually modifies any of your FLACs directly.

4.5 Possible duplicate recordings

Once you have performed a full or differential integrity check on your music collection, Niente will have collected the 'digital fingerprints' of each FLAC that it finds, in the form of an MD5 hash value for each file. Those hash values ought to be unique -but sometimes they won't be! If you have accidentally ripped the same recording more than once, and stored it in separate folders on your hard disk, the same MD5 hash fingerprint will appear more than once in Niente's database -and that's something which Reports Menu Option 13 will list:

Here, you see that I've managed to rip Marriner's recording of Walton's As You Like It to two completely separate locations. Now, it could be that Marriner recorded the work twice and I've got rips of both recordings: think of the number of times Karajan recorded Beethoven's Symphony No. 5, for example! It would, however, be impossible for the 'Audio Fingerprint' data for two truly separate and unique recording sessions to be identical -and you can see that both my Marriner recordings have 100% matching audio fingerprints (the '0a7b9dc5f...' gibberish shown as the last column of the report: it doesn't matter what those numbers and letters actually mean. It's the fact that they appear twice on the same report that tells us these two rips are literally identical in every respect).

The fix for anything shown on this report is, somewhat obviously, to pick one of the recordings as 'the winner' and to physically delete the other off your hard disk. Looking at the different file names for my two Marriner recordings at the top of this report, I couldn't say off-hand which one I'd delete. I would expect something called 'As You Like It' to be a piece of incidental music for a theatrical performance, or maybe a film score... so that would make me want to keep the 'Film-Theatre-Radio' one, and lose the 'Orchestral' one. But the inclusion of 'A Poem for Orchestra' as part of the filename for the second instance of the FLAC makes me think that perhaps it's just an orchestral piece after all! I couldn't tell you how to fix this, therefore, without doing extra research and reading of the CD booklet... but I at least know there's a problem to be fixed and how to go about fixing it.

4.6 Display the operations log

Every time you ask Niente to create a database or perform an integrity check, it records the fact in its Operations Log. You can view this log at any time by taking Reports Menu Option 14. It is not a particularly exciting report and I doubt you'll ever really want to run it:

...but it's there if you want it and it can prove handy in reminding you the last time you did a full integrity check, for example: if it was a long time ago, it might be time to do a fresh one.

5.0 Conclusion

Running integrity checks of various types is certainly where Niente puts in the most work: but it's when you run reports that are driven off the database that's populated by integrity checks that Niente really becomes useful. The Quick Aggregate Summary should be your initial guide to what's going wrong with your music collection: use its various non-zero statistics to guide you to running the related reports which drill down to give your the specifics as to which FLACs are violating the rules, and how. A bunch of 'other reports' can then be used to view interesting information about your collection which maybe doesn't rise to the level of a physical or logical error as such, but may still be something you'd like to fix up anyway.

If you subscribe to this website's promoted ways to tag classical music, a bunch of tag-related reports will prove useful in making sure you stick to the 'rules'. If you don't, then there are still plenty of reports which you should find of use and interest.