1.0 What is CAO
For various reasons which I mentioned in this blog post, you might share my opinion that -at least in classical music- the concept of 'tracks' is not ideal. They encourage a fragmented and selective approach to listening, when more ideally (perhaps!), we should play a classical composition from beginning to end without being tempted to pick and play only the 'good bits'!
Thinking along those lines made me think that it would be useful to take a folder full of FLAC files and 'combine' them into a single large FLAC that contained the music contents of the separate files -and which, indeed, internally continued to store details of the individual tracks making up the whole- but which appears to anyone looking casually as a single FLAC file.
It has long been possible, of course, to rip a CD in 'disk at once' mode, so that the entire CD gets written to a single file. Usually, a separate cue sheet is produced by such rippers, detailing where the CD's individual tracks stop and start "within" the one output file. CAO is a bit subtler on this score: it doesn't rip a CD into a single file, but takes all the separate FLAC files making up a single composition and turns them into a 'master FLAC'. Hence it's name: it's not a whole 'disk-at-once' but a 'composition-at-once' generator. CAO also produces cue sheets to explain or act as an index to the contents of the 'super-FLAC' it creates... but it embeds that cue sheet within the FLAC itself. Many music players don't know how to read or interpret such embedded cue sheets: to them, CAO's super-FLAC will just look like one large piece of music (which is fine for classical music listeners!) But some players know how to read and interpret embedded cue sheets -and so they will display the super-FLAC, still, as a set of separate tracks. In these cases, you get the best of both worlds: a single file to manage and curate, together with track-specific access on-demand.
Finally, if you experiment with compositions-at-once files, you might decide they're not for you: in which case, you need a painless way of reversing the aggregation process, and splitting a 'super-FLAC' back out into its separate tracks, all the while preserving the metadata tags you associated with the music concerned. CAO will do that, too, if asked. It's "asunder" mode reverses a 'merge' and splits a super-FLAC back into component track-FLACs.
2.0 Obtaining CAO
Installing CAO is a simple process of issuing the following commands:
wget https://absolutelybaching.com/abc_installer bash abc_installer --cao
The first command downloads a generic installer script. The second command is an instruction to run that installer script, specifying (with a double-hyphen parameter) that it's CAO you want to install.
3.0 CAO Basic Operation
At its most basic, you run CAO by cd'ing to a folder full of FLACs and typing the command cao. With very little interaction, CAO will generate a 'super-FLAC' from the individual 'component FLACs' found there. It is important to realise that this process is completely non-destructive. The individual FLAC files are not destroyed, damaged or altered in any way by this 'merging' process.
CAO will give you the option of deleting the component FLACs once the merge process has finished -but unless you explicitly respond 'y' when prompted, the component FLACs will remain just as they were after the merge as they were before.
An alternative mode of operation also exists that reverses a previous merge. In this case, you run CAO by typing the command cao --asunder. CAO will then read an embedded cuesheet from the first FLAC it finds in the directory and use that to split the super-FLAC into its separate tracks. It then copies across all the metadata from the super-FLAC to the component tracks. Again, this process is non-destructive. The super-FLAC remains afterwards, along with the new component FLACs. It's up to you to audition both sets of FLACs and decide whether you want to delete one or other of them.
In summary: cao on its own merges tracks; cao --asunder splits a FLAC into tracks.
4.0 A Worked Merge Example
Take this 'album' of FLAC files:
It is a folder of a mass by Heinrich Ignaz Franz von Biber (and a very fine one, I might add!) It was ripped as 5 separate movements or 'tracks', each of which has then been tagged up (using CCDT, as it happens) and so is full of carefully-curated metadata, like so:
You'll notice, also, that there's a 'booklet.pdf' in the folder, representing a scanned copy of the CD booklet that came with this CD. So this is our starting point.
At a command prompt, I navigate to this folder and invoke CAO, like so:
Launched in this way, CAO 'merges' FLACs, which it immediately begins to do:
The spinning thing chugs away and eventually the screen changes to display this:
The merge process having completed, you are asked whether you want to delete the 'component' FLACs. Unless you explicitly type 'y' (in lower case) at this point, the component FLACs will not be deleted. So let's just press [Enter] at this point:
There's simply a confirmation that we're all done and that's it. So it's not difficult to run CAO! But let's see what it has done. Here's the folder we were looking at earlier in its post-CAO state:
So what do we notice? First: the individual tracks are still there, just as they were. Compare the file sizes with the same files listed about 6 screenshots earlier: they are the same. So, the first lesson is that CAO destroys nothing and alters nothing about the source files (unless you explicitly and specifically tell it delete the source files when prompted, of course!)
Second, we see that booklet.pdf is still there, just as it was: so the second lesson is that CAO doesn't mess with anything which isn't a FLAC file at all.
And finally, we notice a new file has been created at the bottom of the list of files: the super-FLAC. It's big: in this case 225.5MB. If you add up the displayed sizes for all the original FLACs, you'll see this is very close to being an exact match in size terms (but not quite!). You'll notice, too, that the file icon for this new file is the same 'orange square' as is displayed for the source FLACs: this tells you that CAO has copied across the album art from the source FLACs to the super-FLAC.
If we open the super-FLAC in a program that displays the metadata tags associated with it, such as Kid3 or even my own CCDT, we see this:
Here you see that all the metadata tags we associated with properly tagged FLAC files are present: ALBUM, TITLE, PERFORMER, COMPOSER, GENRE and so on. However, there's also a new one: CUESHEET. Let's open that up and see what's in it, shall we?
This is a classic 'cue sheet' piece of text. It describes the composer, the name of the work and who's performing it. It then spells out, with sub-second precision when each track making up this composition begin. It is, in short, a recipe for understanding the makeup of the super-FLAC and for being able to interpret its contents correctly.
And so that's what CAO does and how it does it. It takes (in this case) five individually-tagged-and-named FLAC files and merges them together into a single large FLAC that contains all of them, together with all their metadata, and a cue sheet that explains where the individual tracks can be 'found' within the super-FLAC.
Obviously, at this point, you should play the super-FLAC in your chosen music playing software and see whether it sounds OK to you. If it does, you can come back and manually delete the individual files making it up, since they are now mere duplicates of the contents of the super-FLAC.
4.1 Playing Super-FLACs
When you come to play the super-FLAC, you will get quite different results depending on what software you use. On Linux, for example, I generally use Strawberry or Clementine (the one was forked from the other, so what one does will generally mean the other does too). Here's Strawberry playing my new Super-FLAC:
You can see it's 5 seconds in -so it plays it just fine. It even reads the album art from it fine, and displays it nicely down in the bottom-left corner. But what do you see listed in the main playlist area on the right? One file. Strawberry does not know how to interpret the cue sheet embedded within this file and use it to dynamically display the tracks making it up. Which is fine: one of the reasons for going Super-FLAC in the first place, of course, is so that you have 'a composition', which you play or don't, rather than a bunch of tracks which you pick-and-choose from as the mood takes you (we generally don't pick and choose which bits of a mass to play in which order, for hopefully obvious reasons!) So Strawberry is not "wrong". It's doing exactly what you want it to do, given your motivation for creating super-FLACs in the first place.
But change the software, and all sorts of things change! Here's the Linux music player ludicrously called DeadBeef playing exactly the same super-FLAC as before:
Again, it clearly has no problem playing the file, as it's about half-way through the Sanctus. It also manages to display the album art just fine. But what do we see in the main playlist area: five tracks! DeadBeef is reading the embedded cue sheet and dynamically parsing the super-FLAC back into its constituent 'tracks'. So you lose the 'play everything or nothing' reason for making super-FLACs in the first place, but you now only have one physical file to backup and manage, rather than 5, so there are still administrative benefits from doing so... but individual 'tracks' remain as accessible as they always were.
The same thing happens on Windows, incidentally. Here's Foobar2000 playing the super-FLAC:
...which sorts of gets it right, but not quite! It's certainly aware there are five 'tracks' within the one file (displayed in the left panel as a single entry under Biber's name), but it calls them all 'track 01' and doesn't really know how to display their 'track titles' correctly: they all get called the same name as the entire mass. So, it's all playable and it displays all the important information, but isn't quite right. But switch software:
Ignore MusicBee's tendency to want to list thing by Performer tag on the left! That's a configuration issue related to MusicBee itself. Concentrate instead on the main panel in the middle: it's displaying five 'tracks', each correctly named and numbered, as per the contents of the cue sheet. MusicBee can dynamically interpret an embedded cue sheet, like DeadBeef... and unlike Strawberry or Foobar2000.
So your playing results after a CAO-merge will depend on the software you use to play your FLAC files in the first place. In all cases, the super-FLAC is perfectly playable. But some players may 'see' only one FLAC to play, whilst others see multiple 'tracks'.
In every case, the fact that there's only one physical file involved means (a) software which has problems doing 'gapless playback' will no longer have that problem, since there are no longer separate files to have gaps between; and (b) you'll only have to copy and manage one file now, not 5 (or nearly 100 in the case of some Baroque operas I can think of!). The administrative benefits are there; the playback simplicity is there; the question of whether you see tracks or not is up to you and your choice of playing software. For someone like me that doesn't like the existence of separate tracks... CAO provides yet another bonus!
5.0 Reversing Things
If you have used CAO to merge separate tracks into a super-FLAC, you can also use CAO to reverse the process. The end results will not be absolutely identical to what went before, but the music signal will continue to be bit-perfect, and the reversal is itself non-destructive.
So, let us start with this state of affairs:
This is the super-FLAC we created in Section 4 above. You can tell that only because the file size is large -and the file name doesn't start '01...': only per-track FLACs tend to start their file names with track numbers. Notice, as before, the folder also contains a PDF which is the CD booklet. We run CAO in 'exploding' mode like so:
It's still the 'cao' command, but this time with the run-time switch --asunder added to it. Note that, as before, I've had to physically 'cd' to the relevant directory: CAO does not work 'remotely' or against a whole batch of folders and files at once. It's way too drastic a change to your music library to allow that to happen!
Anyway, the moment I press [Enter] to submit that earlier command, this happens:
There is no interaction: the program simply locates the super-FLAC and starts work. The spinning progress indicator twirls for a few minutes and then this happens:
You are simply told that the process has completed. There's literally zero interaction when CAO is run in 'asunder' mode! Let's see what has happened as a result of its work:
Well, the first thing to notice is that the super-FLAC (at 225.5MB) is still there at the bottom of the file listing. So, lesson 1: asunder mode does not destroy your super-FLACs. It merely creates new files as splits from the super-FLAC, but it doesn't ever delete the source FLAC in the process.
Second: we see that five new 'per-track' FLACs now exist, along with a new cue file. The cue file is simply an instantiation of the cue sheet that was (and still is!) embedded in the super-FLAC. It's there for interest's sake really, not for any practical reason.
Third: the booklet PDF file is still there, so again: CAO in asunder mode is only concerned with FLAC manipulation. It doesn't touch, damage or alter anything which isn't a FLAC file.
And what of those new per-track FLACs? Well, you can see they are all marked with little orange squares: so, clearly, they have all been tagged with the same artwork that remains present in the super-FLAC. They also have file names which indicate that track numbers and titles have successfully been preserved during the split. But what of any other metadata tags that ought to be present? Are they?
They are. CAO captures all the album-wide metadata out of the source super-FLAC and applies it to all the per-track FLACs automatically. The conversion of tracks to super-FLAC and back again should result in no loss of metadata. The exception are weird things like 'tagdates' that some of my own FLAC checking software places in FLAC files -and whose loss is meaningless (since these new per-track files have never been subject to that sort of checking in the first place. Once they are, those sorts of tags will make a natural re-appearance).
The entire cycle, from source per-track FLACs, through to a super-FLAC, and back again to asundered per-track FLACs is musically lossless. But you will find that the per-track file sizes are now not quite what they started at. There are various reasons for that, to do with the fact that tracks can only be split and joined on certain 'block' boundaries, and that the splitting process involves a re-encoding of the audio signal with compression parameters that may or may not match exactly what was used when the files were originally ripped from CD. So, the process is not perfectly symmetrical: the tracks you end up with are not identical to the ones you started with. But it is musically symmetrical: the audio signal now contained in the separate post-asundered tracks is identical to the one they started with.
6.0 Later Developments on Splitting Things Apart
In version 1.08 of CAO, the underlying technology used to split super-FLACs apart was changed. Prior to that version, CAO relied on the shnsplit utility to split files apart based on the contents of an embedded cuesheet. Unfortunately, shnsplit is very old software that doesn't appear to have been significantly updated since 2009. One consequence of this is that shnsplit simply cannot split apart FLACs which store high resolution audio (in other words, FLACs which use 24-bit sample sizes or sample rates of anything greater than 44.1KHz, as used in standard CDs). Since shnsplit broke down when confronted with hi-res super-FLACs, so did CAO.
Version 1.08 therefore introduced a completely new way of splitting FLACs apart -one that worked with standard resolution FLACs and hi-resolution ones. The changes are invisible to standard-resolution FLAC users: no aspect of CAO's behaviour would noticeably change for such users.
But for anyone joining (and therefore perhaps needing to subsequently re-split apart) high resolution FLAC files, version 1.08 is the one they need to do the job.
That release also added one further capability to CAO: an ability to split apart FLACs based on the contents of an external cuesheet, rather than only on one that was embedded in the super-FLAC itself, as had been the case previously. Furthermore, CAO was able to expand this external cuesheet functionality to splitting apart files that used the APE audio codec (but only on Linux distros that themselves understood how to handle APE files: Debian- and Ubuntu-derived distros generally do not have that capability, so CAO on those distros also cannot cope with APE files).
From version 1.08 upwards, therefore, on most distros, you will be able to deal with this situation:
Here we see a single APE file -and from its size, we can guess that it is a 'super-APE', containing many individual tracks-worth of music within a single file. A single cuesheet is also present, that describes the contents of the super-APE in per-track terms (at which time each track starts, how long each one lasts and so on). Confronted with this, we can issue this new CAO invocation command:
cao --asunder --external
...and this will trigger CAO into (1) converting the super-APE into a super-FLAC; (2) parsing the super-FLAC according to its cuesheet; and (3) splitting the super-FLAC apart into its constituent tracks, resulting in this final outcome:
You will notice that the original cuesheet file has apparently disappeared from this directory listing! Be aware that it is actually still embedded in the super-FLAC that is the largest file listed. A tool such as Kid3 or CCDT can be used to display it, should its contents still be needed.
Note that when CAO is told to process an external cuesheet, it makes no attempt to tag up the resulting per-track files with information contained within that cuesheet, as it would do if the cuesheet had been stored internally. It simply splits the original file apart into component per-track files: tagging them up is left as a separate exercise, perhaps using this site's own CCDT software. The reason for this is simply that a cuesheet which CAO itself created and embedded in a FLAC is something CAO can reliably parse when splitting the file apart again; but parsing someone else's idea of a cuesheet is problematic at best, and CAO simply doesn't bother trying to do so, given the likely tagging errors that would cause.
Also note that when confronted with an APE file, CAO can only split it apart by invoking this site's own AUAC utility, which needs to be separately installed beforehand. FLACs can be handled 'natively' however, without needing AUAC to be installed first.
When splitting FLACs (or APEs), CAO will always ensure that the resulting per-track output files share the same audio resolution characteristics as the source input file. That is, if you are splitting a hi-res FLAC or APE, the individual tracks will also be hi-res FLACs. Similarly, if you are splitting a standard CD Audio resolution FLAC or APE, the resulting per-track files will also be standard CD audio resolution.
Finally, be aware that the switch --external used to tell CAO to process a FLAC or APE with an external cuesheet can be abbreviated as -x. Whether spelled out in full or used in its abbreviated form, however, the switch must be accompanied by the --asunder switch before it can do anything useful. Without both switches present, nothing will happen. When splitting a FLAC known to contain an embedded cuesheet, --asunder on its own is sufficient, of course. In short, --asunder on its own is fine in some contexts; --asunder and --external (or --x) are also fine, in some contexts; but --external (or --x) on its own is simply ignored and will probably trigger an error as CAO fails to work out what it is you want it to do.
7.0 Summary
In conclusion, I hope you can see that combining tracks into a super-FLAC is painless and non-destructive. It is also reversible, if at some later date you think you made a horrible mistake in turning your music collection into a set of super-FLACs!
Should you use CAO to merge tracks? It's entirely up to you. If you would prefer to listen to music without tracks most of the time, but have playing software that can display tracks from an embedded cue sheet when you want it to, then there's not a lot to lose by doing so -and a whole new way of listening to classical music if you do.
If you are using playback software that can't do proper gapless playback, then CAO is certainly a way to get around that software's limitations (though that's not why I'd chose to use CAO: I'd chose to get better playing software in the first place, as gapless playback is a long-solved problem by now).
And if you've got a library of nearly 70,000 "tracks" but only 9,000 or so "compositions", then you may well find that backing up much fewer but substantially larger files is something your backup software thanks you for. (Guess how I know!) Rsync, in particular, likes it when I feed it fewer but bigger files: its network copying capabilities respond to that very well and my backups (of the same data, remember, just differently-configured) are now substantially faster than they used to be.
But ultimately: the reason I wrote CAO is because I think you shouldn't really pick-and-choose which bits of a classical music work you want to listen to, and should instead learn to listen to it from beginning to end. And that's why I wanted to "abolish tracks", which is what CAO does as far as many software music players are concerned. But you may not share those opinions -and that's perfectly fine: I don't mind if you want to be wrong!- and therefore they may not sway you.
I suppose my final point is simply that you can use CAO to merge and then de-merge your files: so try it out on a copy of a few folders of your music collection and see how you get on, for you have nothing thereby to lose.
One word of warning I will end with: CAO only works if and when the folder of FLACs it is merging into a super-FLAC are "coherent". That is, all belonging to the same composition. It actually reads the metadata from the first per-track FLAC and assumes that will apply to the super-FLAC it ends up creating. It would not be sensible, therefore, to have (say) 8 tracks in one folder that represent 4 movements of a Beethoven symphony and 4 movements of a Mahler one. For then the super-FLAC will be declared to belong to Beethoven in its entirety, which will be wrong. It's not that CAO won't work in these situations, in other words: it's that if you are reducing things to a single set of metadata tags, they can only meaningfully describe a single set of 'circumstances' -and multiple compositions by multiple composers simply don't fit that bill.
So CAO is a tool that has a good role to play in a well-managed, well-tagged, well-curated music collection. It isn't a substitute for getting any of those aspects of music management right!
And a final parting shot: the original technical specification of the cue sheet permitted only a maximum 99 tracks per sheet. That limit still survives to this day. Accordingly, CAO will not attempt to merge more than 99 tracks into a super-FLAC: if it finds more than that number in a folder, it will warn you and quit. It's not a limitation of CAO as such, but of the cue sheet technology it has to use.
8.0 Housekeeping and Legal Stuff
Frequently Asked Questions (FAQ)
The CAO FAQ can be viewed here.
Author
CAO was devised and written by Howard Rogers ([email protected]). However, it uses a lot of opensource software to actually do anything useful, so full credit to the developers of (for example) ffmpeg.
License
CAO is copyright © Howard Rogers 2021 but is made available freely under the GPL v2.0 only. That license may be downloaded here.
Bugs Tracking, Feature Requests, Comments
There is no formal mechanism for reporting and tracking bugs, feature requests or general comments. But you are very welcome to email your comments, complaints or suggestions to [email protected].