Automated Bulk Volume Boosts

1.0 Introduction

Niente has long been able (thanks to the Integrity Checks menu, Option 6 or 7) to analyse your FLAC files for their peak loudness: this is documented in the User Manual (see Section 3.5) but by way of a quick refresher:

  • Music files reach their loudest non-distorting volume when their peak loudness is measured at 0 dB
  • Files quieter than this have negative decibel values for their peak loudness: so a -3dB file is almost as loud as it could be, but a -19dB file is very quiet and far from its maximum possible peak loudness

If you have performed an Option 6 or 7 integrity check, you might see something like this when running a Niente Aggregate Statistics report:

The "O2" statistic is a count of the folders containing FLACs that have peak loudness lower than a threshold you configure in the Niente configuration file:

Here, I've set my 'threshold' to '2': that is interpreted by Niente as "-2dB", so it means files that have peak loudness of (say) -1.3dB won't be counted as a problem, whilst one with a peak loudness of (say) -2.1dB would be. My earlier count of "187" for the aggregate statistics report therefore means that I have 187 folders containing FLACs that collectively have a  peak loudnesses quieter than 2dB.

Giving all these files a volume boost is something Semplice does, as you tag up each ripped CD in your collection, one rip at a time. Unfortunately for me, I only added volume-boosting capabilities to Semplice many years after I had ripped and catalogued most of my music collection! That's why I have rather a lot of files which are now listed as having 'potential volume boosts'. It is a pain now to have to visit each ripped CD in turn, launch Semplice and apply a volume boost: one-at-a-time when there are hundreds or even thousands of candidate recordings is a slow and tedious affair.

I have therefore written a script which will automate the job: it will query a Niente database and find out what recordings could be safely volume-boosted; it then visits each one in turn and applies the volume boost calculated to bring the peak loudness for that folder of FLACs to within 0.5dB of zero. It doesn't aim to get exactly to zero dB, because it wants to be safe and not risk introducing distortion. It also puts a hard-limit on the volume boost it will apply: even if it calculates that a folder of FLACs can have a boost of (say) 11.7dB, it won't apply a boost bigger than 6dB. The reason for that caution is that some recordings are deliberately mastered at very low volumes so as to mask imperfections in the original recording: boosting the signal now as far as it can go will make those imperfections extremely audible. See Section 3 below for an example of how 'absolutely maximum volume' can be a disaster, which my script aims to avoid!

2.0 The Bulk Booster Script

You can download my bulk booster script as follows:

wget doco.absolutelybaching.com/bulkboost

You can then run the script by opening a terminal session and typing the command:

bash bulkboost

Immediately, you'll see this:

You are prompted to supply four pieces of information. The first is the 'temporary working folder': this is simply a folder that you can access and to which the script can write various working files as needed. The second is the 'Niente database name': this needs to be the name (without the .db extension) of an existing, functioning Niente database that has already been populated with details about what music files exists and for which a volume boosting integrity check has been performed. If no prior volume boost check has taken place, the script won't find any recordings that need boosting and will therefore just bomb out without having done any work at all.

Finally, there are two fields which ask you for the loudest and quietest volumes that the script should ignore. Remember that volumes work in reverse: 1 is loud, 99 is quiet. So if you entered '6' for the Loudest volume field and '15' for the Quietest volume field, then you are saying "Ignore files which are louder than 6dB [i.e., they have volumes in the 0 - 5.99dB range]" and "Ignore files which are quieter than 15dB [i.e., they have volumes lower than -15dB]". That would mean the script will look for files with volumes between -6dB and -15dB and thus a folder with a peak loudness of (say) -3dB would be ignored. It could be volume boosted, but you said not to bother with anything that was louder than -6dB. Similarly, a folder with peak loudness of -23dB could be volume-boosted by rather a lot, but the script won't touch it at all, because you said to ''ignore files that are quieter than -15dB'.

You can pick whatever numbers you like to go into these fields but (as I explain in Section 3 below), you might not want to be messing with recordings which have been deliberately mastered at extremely quiet levels: the recording engineers are trying to hide things from you at that point and a volume boost might expose their dirty secrets! I would suggest values of 2 to 9 might be most appropriate.

A filled-in example of the screen thus might appear like so:

Here, I've made my Desktop folder the temporary work area; I've pointed it at a Niente database called 'main'; and I'm interested in bulk-boosting files that are in the 2dB to 9dB range: that would mean quite modest boosts to volume levels in recordings that aren't too far off a 'normal' recording level in the first place.

You press [Enter] to submit these parameters:

You are shown your submitted filter values, just so you can double-check them and be certain they're what you want. Please ensure you have a backup of your unmodified music files before you press [Enter] to proceed. If you tab around until the 'No' option is highlighted and then press [Enter] you'll see this:

That's just re-assuring you that you can cancel out of the bulk update process without actually altering any files in the slightest.

If you click [OK], however, then the bulk volume boosting begins in earnest, without further warning. As it progresses, you'll see this sort of thing appearing at the bottom of your terminal:

Each symbol represents a folder being visited and processed. As a folder is visited, a '.' is displayed; if the script determines that no volume boost can be applied for some reason, the '.' changes to '✗'. If the '.' changes to a '?', it means the script has not been able to find the physical folder that the Niente database was describing to it, for some reason. If the folder can be found and a volume boost can be applied, the volume boost is applied, the symbol changes to a '✓'. The above screenshot therefore shows a few folders being skipped for some reason, and then a long series of successful volume boosts.

The results of a batch volume boost will not be apparent to Niente until you have performed a fresh volume boost integrity check. Once you've done a fresh check, however, then you should be able to see the change in the Aggregate Statistics report. Here is an example of a before and after:

Before running the bulk booster script After running the script

You'll note my O2 count has been reduced from 114 to 56: about half my unusually-quiet recordings got boosted.

Bear in mind, the script will probably not reduce the number of 'potential volume boosts' to zero -and you shouldn't expect it to. Very quiet recordings won't be volume boosted to near 0dB (because the script only ever applies a maximum +6dB boost), for starters. The filters you applied for loudest and quietest files to process may also mean that some folders are skipped: if you've said "8" for the quietest volume, for example, then a folder with a maximum volume of -10.1dB will not be processed at all and will therefore remain in the 'potential volume boost' statistic.

You can 'fool the system' however: if you've bulk-boosted all files between 0 and 9 dB, for example, you could use the Niente Administration menu, Option 1 to edit the configuration file and set the threshold for declaring a file to be a candidate for volume boosts to (say) 25dB. The files with peak volumes between 9 and 25dB could be volume boosted, because your run of the bulk booster won't have touched them: but by saying you're not interested in reporting on files quieter than -25dB, you basically allow Niente to return a zero count for 'volume problems':

I know there are still 56 files that could be volume boosted to some extent or other... but I've chosen to exclude them from the bulk-booster's work, because they're too quiet to be safely or sensibly boosted. So, instead, I've just told Niente to consider very quiet files to not be an issue, and thus it reports zero O2 problems outstanding. It's a cheat, but a very pragmatic one!

3.0 Maximum Boosting is not always good!

Here are three versions of a recording in my collection, presented in its original form, after it has been volume-boosted by +6dB and, finally, after it has been given the maximum possible boost it could be given (which was +23.9dB):

Original:

With a 6DB volume boost:

With a maximum possible volume boost:

To my ears, it's difficult to tell the original and +6dB versions apart: they both sound fine, though obviously one is slightly louder than the other. The last one, however, sounds pretty awful: there's very evident hiss, people can be heard shuffling around in the background and the whole thing sounds artificially harsh, with some particularly nasty 'ringing' on the un-damped piano at the end. I don't think you'll hear any actual distortion in the recording, but it's not nice!

I have worse examples in my collection: one organ recital, for example, when boosted to its max sounds incredibly hissy and has a huge amount of background noise, as though the public were shuffling their way round the cathedral during the recording session 😮

There are, in short, very good reasons why, if Niente reports that you have some recordings that could take a volume boost, and you know that that's because the ones it's listing are those with double-digit potential boosts, you should ignore it. At the very least, you should only apply a modest and proportionate volume boost, not the maximum that you could technically apply. It is for this reason that Semplice never applies a volume boost bigger than +8dB -and this bulk booster script is even more conservative, refusing to apply a boost bigger than +6dB. This will leave especially quiet recordings short of the 'ideal' of 0dB maximum loudness... but, as the above example shows, there are recordings out there which simply don't take well to being boosted to maximum!

4.0 Summary

1. Anything with a peak volume of between 0 and 0.5dB is always ignored by the bulk booster, even if you fill in the initial filters with values of (say) 0 to 99.

2. The maximum boost that will actually be applied is always 0.5dB less than what could physically be applied. If the script calculates that the maximum possible boost to a folder of FLACs is, say, 4.3dB, then it will actually provide a boost of 3.8dB. This is to ensure there is not the slightest possibility of introducing distortion by over-boosting the volume.

3. The maximum boost that will ever be applied by the script is 6dB, even if the script initially calculates that a larger boost could be applied. If a folder is detected as having a boost potential of (say) 14.7dB, the boost applied will still only be 6dB, leaving the folder 8.7dB quieter than a theoretical maximum. This is to avoid the problem of maximising the boost on recordings which have been deliberately engineered to be quiet (see Section 3 above).

4. The progress bar indicates a period/dot when it is analysing a folder; it turns to a '✗' if a folder is determined to be already at maximum volume and no further boost can therefore be applied, or if the current volume is outside the range of volumes you've allowed for it to consider; it turns to a '✓' once a volume boost has been applied; and it turns to a '?' if, for some reason, the script cannot locate the physical folder that the Niente database has described to it.

5. The bulk booster modifies files. Don't run it if you are unsure of what volume boosting your original FLACs will do. At least make sure you have a backup of those files you expect to be modified, so you can revert the changes if necessary.

6. The script works on a per-folder basis. It doesn't boost the quietest movement of a symphony by one amount and the loudest by another! It analyses each file in a folder and determines the peak volume of the loudest file. If that's quieter than the possible maximum, it boosts the entire folder's collection of files by that amount, thereby ensuring that the relative volume differences between files remains constant, even if the absolute level at which they play has been increased.

7. You can run the script more than once: on second and subsequent runs, files which have already had a volume boost but which left them significantly short of the maximum possible non-distorting volume will have a further boost applied. See point 3 above, for example: if a first pass leaves a file 8.7dB short of 0, a second pass will leave it 2.7dB short, as the maximum +6dB is applied. A third pass would leave the file 0.5dB short of maximum, a limit beyond which the script won't ever go -so a fourth run of the script will not touch the file again. This is strongly not to be recommended, however: multiple volume boosts implies the original file was deliberately mastered at a very low volume... and there are usually good reasons for that!