Database Play Mode

1.0 Introduction

Giocoso runs in two basic 'playback modes': Direct Play Mode, where you tell Giocoso what music exists and where it can be found on disk; and Database Play Mode, where a database persistently stores details of what music exists and where it can be found on disk.

The advantage of using a database as the 'source' of information about what can be played is that Giocoso can select from its data in all sorts of different ways. If you want to listen to music that lasts longer than 20 minutes but less than 30 minutes, Giocoso can do that -because the database stores the duration of each piece of music it knows about. If you want to listen to 'some Beethoven', that's fine too: the database stores the composer name associated with each recording it knows about. A database, in other words, lets you filter and select music to be played via all sorts of qualities intrinsic to the music files themselves. Giocoso can even go one step further: it is able to pick something from the database entirely at random. That will mean the music you listen to will be unexpected -and hopefully, more 'fresh' and enjoyable as a result.

For these advantages of Database Play Mode to be possible, however, there must first be a database! That is, you have to create one (which is a one-off process) and then you have to periodically 'refresh' it, to make sure that it accurately describes your ever-changing music collection. A Giocoso database is relatively small, no matter how large your music collection might be, because it doesn't actually store any of the music, merely attribute data about the music.

In the following sections, I'll take you through the process of creating, refreshing and backing up your Giocoso database.

2.0 Creating a Giocoso Database

You create a new Giocoso database by running Giocoso with the --createdb runtime parameter. You also have to give the database a name, so the --dbname runtime parameter is required, too. Finally, the purpose of creating a database is to store within it details of where your music files can be found on disk, so the --musicdir parameter is required, to point to the 'root' or parent folder of your music collection. A complete command to create a database would therefore end up looking something like this:

giocoso --dbname=mymusic --createdb --musicdir=/music/classical

The order of the runtime parameters is irrelevant. If your musicdir path contains spaces, wrap it in double quotation marks. It is not recommended to put spaces into the names of your database, but if you insisted on one being called (for example) My Music, then you will again need to wrap the parameter value in double quotation marks. This would work, for example, though wouldn't be recommended:

giocoso --dbname="My Music" --musicdir="/music/classical/Orchestral Music"

Note that database names are case-sensitive. The database name 'Music' is distinctly different from one named 'music': if you were to try to subsequently run Giocoso with 'Music' when you actually created 'music', you'll be told via an error message that no such database exists.

There is no limit to the number of databases that you can create, though only one can be used as a source of playback at a time. It would thus be perfectly reasonable to create a database for your classical music collection that is quite separate from the one you create for your jazz music collection, for example. There's also nothing to stop you simply creating one large database to handle both collections, too.

If you try to create a database that already exists, an error message will warn you that this is not possible to do: Giocoso never over-writes an existing database, for hopefully obvious reasons. You will be warned that a 'refresh' operation might be more suitable for a database that already exists. Databases are determined to exist (or not) by looking at the physical files which exist in the $HOME/.local/share/giocoso folder. If there is a file in there called (for example) "bach.db", then a database called "bach" was previously created. If you delete a .db file from that folder using your operating system's standard file management tools, that database no longer exists as far as Giocoso is concerned and all data it may have contained will have been lost. On the other hand, if you use your operating system's standard file management tools to copy a .db file from that folder to somewhere else, you've just performed a backup of your Giocoso database!

Here are some screenshots of me creating a new Giocoso database using this command:

giocoso --dbname=main --createdb --musicdir=/sourcedata/music/classical

First, the database is almost immediately created (it's only around 50KB small at this point). That is to say, all the database structures are created at this point, but no actual data has been loaded into them. That requires (as the message tells you) to press a key. As soon as you do, this happens:

At this point, Giocoso is scanning the folder you specified with the --musicdir runtime parameter. It is creating a list of FLAC files found anywhere within that folder structure. Giocoso has no special limits on how deep with the folder hierarchy it will go -but it cannot cross 'out' of it! That is, if I had folders:

  • /sourcedata/music/classical/Benjamin Britten
  • /sourcedata/music/classical/Orchestral/Franz Schubert
  • /sourcedata/music/classical/Renaissance/Choral/Cambridge Choirs/William Byrd
  • /sourcedata/music/Wolfgang Amadeus Mozart

...and specified --musicdir=/sourcedata/music/classical then my Giocoso database will contain music by Britten, Schubert and Byrd -despite the wildly inconsistent sub-folder hierarchies for each composer! All three nevertheless lie within the /sourcedata/music/classical parent folder and therefore Giocoso's scan will find them. On the other hand, I will be left with no Mozart in my Giocoso database at all, since Wolfgang has /sourcedata/music as his parent folder, and Giocoso has already gone down to the classical folder to start searching.

The 'deep scan' is parallelised as much as possible, to make it go faster: Giocoso will use all available threads on your CPU, if they are available. Nevertheless, the deep scan can take quite a long time to complete (for my 15,000 recordings, 2TB music collection, this stage of proceedings usually takes at least 20 minutes). There is no real indicator of progress: just a spinning red 'thing' tells you the program hasn't hung, at least.

When the list of all possible music files has been created, it is sorted into alphabetical order (of folder name), and then each folder is visited in turn:

Here, you see Giocoso has actually visited the 'BWV 024' sub-folder in the Johann Sebastian Bach folder and has found a FLAC there. It scans every FLAC in turn, determining the following information:

  • Duration (measured and computed)
  • Composer (from the ARTIST metadata tag)
  • Composition name (from the ALBUM metadata tag)
  • Genre (from the GENRE metadata tag)
  • Comment (from the COMMENT metadata tag)
  • Principal performer (from the PERFORMER metadata tag)

Once these six bits of information have been worked out for all discovered FLACs, they are loaded into the Giocoso database in one fell swoop. The database size grows in consequence -for my music collection, it ended up around the 32MB mark. So not nothing, but not exactly Gigabyte-sized, either!

At the end of the data loading process, Giocoso performs its first query on the completed database to tell you about it:

As a general rule, Giocoso always capitalizes database names for display purposes, so they stand out clearly from the surrounding text. Don't let this fool you into thinking that main=MAIN=Main, however. It doesn't, and when you want to use the database you've just created, you need to refer to it by whatever actual capitalisation/case you created it with.

As you can probably tell, creating a Giocoso database is basically the same as 'taking a snapshot' of your music collection: the database contents are 'right' only so long as you don't go off and add new recordings to your collection or modify the tag data associated with existing ones. If your music collection is fairly static, you may well not need to worry about this 'static' nature of the Giocoso database... but chances are, you add and edit your collection all the time, so you will have to worry about it. Fortunately, the problem is fixed by a database refresh.

3.0 Refresh Database

Refreshing an existing Giocoso database merely makes Giocoso re-scan a folder-full of music, specified with the --musicdir runtime parameter. The database to be refreshed has to be explicitly named with the --dbname runtime parameter. The runtime parameter that actually triggers the refresh process is the --refreshdb runtime parameter. Put all those three parameters together, therefore, and you end up with a command such as:

giocoso --dbname=main --musicdir=/sourcedata/music/classical --refreshdb

The command to refresh a database is therefore practically the same as the command to create it in the first place, apart from 'refreshdb' replacing 'createdb'. As ever, the order of the runtime parameters make no difference, and parameter values must be enclosed within double quotation marks if they contain spaces within them.

When a Giocoso database is refreshed, its complete library of existing recordings is wiped and re-populated from scratch. However, the history of 'plays' (i.e., a record of which pieces of music have been played, and when) is never wiped. The play history therefore carries forward, no matter how many times you refresh the database.

A technical consequence of the fact that the existing recordings are wiped is that the unique ID each recording is assigned at the time it is added to the database changes over time. The recording ID is therefore not immutable, and only the extended composition name (as recorded in the COMPOSITION column and derived from the ALBUM metadata tag) is actually an immutable primary key of the Giocoso database.

The screenshots of a database refresh would look identical to those for the database creation process, since that's what's really going on anyway: the database is practically being re-created from scratch (except for the play history).

The Giocoso database uses several 'indexes' to make accessing the data in the database faster. These are automatically maintained by a database refresh operation. It should never be necessary to perform manual maintenance operations on those indexes, but in rare cases of severe internal corruption, it might be advisable to run Giocoso with the --reindex runtime parameter. That drops and re-creates the internal indexes without performing a full music scan. A rebuilt index should be free from corruption (unless the underlying table is corrupt, in which case you have much more serious problems than a reindex can fix!) I again emphasise that you shouldn't really ever have to use the --reindex parameter, as Giocoso's database is self-administering in all but the rarest cases.

4.0 Backing Up the Database

You may have gathered that the Giocoso database is -mostly!- disposable, in that its store of what music files exists can always be re-created by performing a --refreshdb operation. However, the history of plays it contains is not so lightly lost! Should anything happen to that particular table within the database, you would lose the history of maybe thousands of plays having taken place over years.

As previously mentioned, the database exists as single file (<something.db>) within the $HOME/.local/share/giocoso folder (though there will be many other files stored within that folder which are not the database). Note the full-stop (or period) in front of the ".local" part of the name: it means that folder is a hidden folder. You may need to turn on 'show hidden files' in your operating system's file manager to be able to see it. Backing the database literally means using standard file manager or operating system commands to copy the .db file to somewhere other than your local hard disk. This command would work for me, for example:

cp $HOME/.local/share/giocoso/main.db /backup/

You may wish to be more comprehensive than that, though, and grab everything in the Giocoso folder (since all of it can be useful and none of it is very big):

cp $HOME/.local/share/giocoso/* /backup/

Those commands are only really useful, too, if my /backup folder actually maps to a network storage folder of some sort: there's little point in copying something from part of your hard disk that might fail to another part of the same hard disk, after all!

You don't need to use the 'cp' command to copy your database, of course: any other tool with equivalent functionality will do the job. I tend to use rsync, though it can be complex to use and is really overkill for something as straightforward as a mere copy of a file or folder. Use whatever tools you're comfortable with, basically.

Personally, I have over 10,000 plays recorded in my Giocoso database and I cannot afford to lose that data: so I make sure I back up my database every night without fail, by using a crontab entry as follows:

# Nightly backup of Giocoso at 4AM
# --------------------------------------------------------------
0 4 * * * /usr/bin/rsync -avh --delete $HOME/.local/share/giocoso/ /home/hjr/Documents/Backups/Giocoso_Backups/

Now, this seems to violate the rule of 'backup to some disk that isn't the source', because it's copying from my home folder to ...another part of my home folder! However, I have other scripts which copy my entire home folder to various remote servers and hard disks scattered about the house, one of which also backs itself up to the cloud... so in the event of complete domestic disaster, I should still be able to recover my play history from somewhere!

5.0 Recovering the Plays history

Talking of recovering your database brings me to the question of how you actually do the recovering, should it ever prove necessary!

Let's say that my $HOME/.local/share/giocoso/main.db file has become corrupted somehow. How do I go about getting it and its precious history of previous plays back? Simply by copying a backup copy into that folder, on top of the existing (and broken) file. These commands would do the job for me, for example:

rm -f $HOME/.local/share/giocoso/main.db
cp /home/hjr/Documents/Backups/Giocoso_Backups/main.db $HOME/.local/share/giocoso

Once it's back in its true 'home', the file can be used just as it always could.

An alternative option is possible, however, if you've access to a previous backup of a database but merely wish to pull its play history out and stuff it into a completely new database. This can be done with the --importplays runtime parameter, which takes as its value the full path and filename of a database containing a populated PLAYS history table. Here is a worked example:

I start with a newly-refreshed database:

As you can see, it's a database called 'main' that has thousands of recordings within it. But it's also pretty much brand new and has no plays in it:

That the --stats report for this database: if you look in the lower half, there are zero plays recorded for anything at all. But now I restore a backup by copying the database file to my desktop:

Somehow I've managed to recover a copy of my old database and it's been restored onto my desktop under the name "oldmain.db". Let's now import the plays from that into my freshly-built database. The command is as follows:

That's to say: you point the --importplays runtime parameter at the file containing the copy of the PLAYS table you want imported, and the --dbname runtime parameter specifies the database into which you want the imported. Press [Enter] to submit that command:

It takes fractions of a second for it to do its work, but this tells you that something has happened. Can we discern the effects of the import? Sure:

See how the bottom half of the stats report, on exactly the same system as displayed earlier, now reports thousands of plays. That can only happen if your PLAYS table is full of details concerning what music you've played and when -and they can only have come from that 'oldmain' backup.

5.0 Making practical use of the Database

So now you know how to create and keep fresh a Giocoso database. You also know how to protect it from loss and recover its PLAYS data in the event something happens to it. But how do you make use of it in day-to-day music playing activities?

There are really two ways of doing that: (1) random play mode; and (2) guided random play mode! Let's start at the beginning...

5.1 Random Play

If you launch Giocoso with the --dbname runtime parameter set to anything other than 'none', Giocoso will look for a database with the same name as the dbname parameter's value in $HOME/.local/share/giocoso. If it finds one, it opens it and selects something at random from the RECORDINGS table (populated when the database was first created and re-populated every time you do a --refreshdb). This is 'true random' playback: Giocoso picks something entirely at random and then plays it from beginning to end. Summing up, this is how I would do random play:

giocoso --dbname=main

I should mention at this point that the way Giocoso selects things to play at random is significant. First, Giocoso makes a random choice of a composer from a list of all the unique composer names you have in your music collection. In my case, I've got 578 unique composers, so Giocoso picks 1 of them, completely at random. Second, Giocoso then picks a composition attributed to that composer. The significance of this approach is that the size of a composer's contribution to your music collection is completely irrelevant to the final outcome. Say you have 100 recordings of Bach, 100 recordings of Wagner and 2 recordings of Mozart. If Giocoso simply picked 1 out of 202 recordings, the chances are pretty good you'd end up hearing some Bach or Wagner. The chances of hearing any Mozart would be 2/202, which is about 1%. Fortunately, that's not the way Giocoso does it! Instead, the first pass sees Giocoso select one composer from 3 at random. So Mozart stands a 1/3 chance of selection, precisely the same as Bach or Wagner. The fact that you own hardly any Mozart doesn't reduce Mozart's chances of random selection. Only once we have a composer do we then pick a composition -and by then the disparity in size of contribution to the entire collection doesn't matter.

There are two extra runtime parameters you can add to proceedings to slightly alter the outcome. First, --selections=n tells Giocoso to randomly select something, play it, randomly select something else, play it and so on until n random selections have been made and played. The default number of selections is 1, so by default Giocoso plays one random selection and then quits (but could be re-launched, of course). Each new selection is made entirely oblivious to the previous selections and on the same basis as previously described: pick a composer at random, pick something attributed to that composer at random. The fact that a Beethoven symphony just got played doesn't in any way preclude a Beethoven piano concerto from being selected next. Random sometimes happens like that: strange 'bunching' or coincidences... but that's all it would be.

The second extra parameter you might want to use is --timebar=n (where 'n' is a number of hours). The timebar is a way of stopping what I just described might happen by chance: it tells Giocoso that, if you happened to randomly select Beethoven last time, he's not to be re-selected until n hours have elapsed. The number of hours starts ticking from the time the original play completes. So, say you have a timebar of 3 hours, and you start playing a Beethoven symphony at 9AM: if that symphony concludes at 9:40AM, then Beethoven cannot possibly be randomly selected for another play of anything until 12:40PM. There is a default timebar of 6 hours. The timebar can be disabled by setting the parameter to a value of 0.

There is one other factor to mention that can prevent Giocoso's random choices from being completely, 100% random: the excludes. If you create a text file in the $HOME/.local/share/giocoso folder called excludes.txt, you can then write the exact name of a composer, one composer per line, and the presence of a composer's name in that folder completely prevents Giocoso from selecting that composer (and thus anything by that composer) from being played. The excludes file is a sort-of 'permanent timebar': the timebar prevents selection of a composer for a specified number of hours; the excludes file prevents the selection of a composer for as long as the entry in the excludes file persists. I should mention that the names in the exclude file must be precisely as those you used tagged your music with. If you tagged your Beethoven as being by 'Ludwig van Beethoven', then an entry in the excludes file of 'Ludwig Van Beethoven' will not stop Beethoven's music being selected and played: the capital 'v' on 'Van' in the one means there won't be a match to the data actually stored in the database.

Finally, if you launch Giocoso with the command giocoso --editx, you will be able to launch your system's default text editor directly to create, edit and amend entries in the excludes.txt file. You can edit it with any text editor, of course: it's just a text file, after all... but the --editx runtime parameter makes it potentially a bit more convenient to edit it.

5.2 Modified Random Play

There are a grand total of ten runtime parameters which allow you to 'guide' Giocoso in its choice of music to play. It will still generally be randomised, but you'll be 'weighting the dice' heavily, and gently prodding it to select things that might be of particular interest to you. You trigger this 'modified random' playback by appending various runtime parameters to the basic Giocoso launch command. Let's just first list the things you can use to 'guide' Giocoso:

  • Composer (--composer)
  • Genre (--genre)
  • Performer (--performer)
  • Composition Name (--composition)
  • Comment (--comment)
  • Unplayed Status of the Composer (--unplayed)
  • Unplayed Status of the Composition (--unplayedworks)
  • Minimum Duration (--minduration)
  • Maximum Duration (--maxdration)
  • Recording-specific ID Number (--recordnumber)

These are all discussed at length in the article on Selection and Filtering, which you should read to understand all their subtleties. What follows is a massive simplification of the information provided in that article!

Apart from the last one, each of these parameters can be combined with any of the others in ways that make logical sense, but no-one will check your logic! If you say giocoso --dbname=main --composer=britten --composition="Symphony No. 5" then Giocoso will happily accept the combination of filters... but nothing will match, since Britten didn't write numbered symphonies. Aside from such logical absurdities, though, there's nothing to stop you asking Giocoso to play 'masses by Haydn that last between 20 and 40 minutes that were conducted by Karajan that I haven't played before':

giocoso --dbname=main --composition=missa --minduration=20 --maxduration=40 --performer=karajan --unplayedworks

That at least is logically plausible, even if I don't think Karajan ever actually conducted any Haydn masses (and if he did, I don't think I've got any of them in my collection)!

As ever, the order of parameters in the launch command makes no difference to their effectiveness; and if any parameter value contains spaces or punctuation characters, wrap that parameter value in double quotes. If you want to play some Vincent d'Indy, for example, you'll have to ask for --composer="d'Indy".

Selection parameters are not case-sensitive (so 'britten' matches 'BRITTEN') and they are generally wild-carded, so 'bri' matches 'Britten' and 'Bridge' equally well.

Selection parameters can have their sense inverted by adding an '@' (at symbol) to the end of the parameter value. That is --composer=britten means 'play me some Britten', but [email protected] means 'play me anything so long as it's not Britten'.

By default, Giocoso will search through your entire music collection trying to find anything that matches your particular combination of selective runtime parameters. For a large music collection, this can take many, many minutes and so is probably an unacceptably high limit, despite it meaning that Giocoso is at least comprehensive in its searches for things to play. To speed things up, you can set the MAXSEARCH=n parameter in the persistent configuration file, where 'n' can be any positive number you like. I have mine set to 500: Giocoso then tries 500 times to find something that will match my choice of filters and if it gets lucky, fine: I hear music. If if doesn't get lucky, I see this sort of thing:

The correct response to seeing this error message is to relax your filtering criteria a bit: maybe demanding the mass be conducted by Karajan isn't such an important factor to you after all? Maybe you could back off on the duration restrictions in either or both directions a little? Also, bear in mind that you may have added Joseph Haydn to the excludes.txt file -which means Giocoso is now fighting against your own explicitly-expressed wish not to hear any more Haydn! Finally, remember the timebar: it's possible there's lots of Haydn masses that would fit the original selection bill, but you played his Symphony No. 94 two hours ago... if your timebar is set to the default of 6 (hours) then Giocoso is again caught between a rock (your desire to hear Haydn) and a hard place (your desire not to keep hearing the same composer within too short a time span): and the timebar will always win!

6.0 Source and Destination Databases

When you run Giocoso in Direct Play Mode, the concept of separate source and destination databases is something you have to grapple with. The source database is a database which tells Giocoso where to find music files on disk. The destination database is the database in which Giocoso records its plays. Of course, in Direct play mode, there is no source database -by definition, if there were, then you'd be doing Database Play Mode. However, Direct Play Mode can (optionally) use a destination dataabase, so that a non-database sourced play can nevertheless be recorded as having happened in a database somewhere.

Database Play Mode doesn't really have this same split concept of source and destination, however, because, always and irrevocably, in Database Play Mode the source and destination databases are the same database.

If I have a database called 'main' that contains details of 15000 recordings, and I run Giocoso with the command giocoso --dbname=main then I have just triggered fully random Database Play Mode where the database called 'main' will be the source of my plays. Without exception, every single play Giocoso then performs will be stored in the PLAYS table of the 'main' database, too. 'Main' is therefore both source and destination database.

You cannot change this behaviour and if you try to do so (by adding a --destdb=myplays parameter to the launch options) the destdb parameter is silently ignored anyway.

However, because Direct Play Mode does make the distinction between a source of recordings and a destination for plays, the runtime parameter --dbname can be specified as --sourcedb, if you prefer that nomenclature. The two are exact synonyms for each other. The command giocoso --sourcedb=main has precisely the same effect as giocoso --dbname=main and is functionally exactly equivalent to it. This documentation generally uses --dbname for now, but over time, it is thought that --sourcedb might become the more common way of invoking the various options that depend on it. For now, you can continue to use --dbname as the documentation (and past practice back in Giocoso Version 1 days!) suggests. Note that the persistent configuration file parameter that means 'dbname' is already called SOURCEDB.

7.0 Conclusion

Summing all of the above up, then:

  • Database Play Mode requires the creation of a Giocoso Database using the --dbname, --musicdir and --createdb runtime parameters (in any order)
  • The Giocoso dataabase is a single file found in $HOME/.local/share/giocoso
  • The Giocoso database stores details of what music file exists and where they can be found on disk; and details of what music has been played. It is therefore both a source and a destination database
  • The store of what music exists can be periodically refreshed (or re-created from scratch) by running Giocoso with the --dbname, --musicdir and --refreshdb runtime parameters
  • The store of what music has been played cannot be re-created, but can only be restored from backup or imported from a backup copy of the database: backing up the Giocoso database is therefore an important job
  • Database Play Mode is triggered when --dbname=xxxx is present on the command line when launching Giocoso (and --refreshdb or --createdb are not present)
  • Database Play Mode triggers random selections of music to play from a database
  • Randomness can be 'guided' or modified by use of 10 selective runtime parameters

Most other music players out there tend to talk about a 'music library'. Giocoso calls it a database (because it is, and its author used to be a Database Administrator!), but it's essentially the same thing: a record of what you own and what you have played. Being able to interact with the database, however, by specifying multiple selection criteria as you launch Giocoso rather sets Giocoso apart in the music player stakes, I think!


[Back to Front Page] | [Selective Runtime Parameters]