Giocoso 3 - The Database Management Menu : Full Scan all recordings

Fast scans are great for picking up the existence of new recordings. They do not work to remove old, redundant data out of a Giocoso database, however; nor does a Fast Scan ever pick up modifications to metadata. For those sorts of things, you need to completely re-scan your music collection, starting with a totally blank slate -and that's called a Full Scan and is what Database Management menu Option 3 will do for you.

When you first take this menu option, you are required to specify which database you want to refresh:

The form will simply list every file with a .db extension that's found in the $HOME/.local/share/giocoso3/db folder. Whilst there may be many such files, you can only select one of them at a time with this form: you simply arrow up-and-down the list of file names until the correct one is highlighted, then press the Space Bar to select it. Doing so causes an asterisk to appear in the brackets next to the selected database's name. You can then press [Enter] to complete the selection.

Giocoso then prompts you for the 'root' folder of your music collection: the one folder that is at the 'base' of your folder structures in which all FLACs can be found, no matter how many sub-folders they may be buried in:

You can navigate your on-disk folder structures by arrowing around the entries in the top pane and pressing the Space Bar when a correct folder name is highlighted. As you 'descend' into a folder, its name is appended to the full path that appears underneath the folder list, in the second panel of the form. Alternatively, you can simply type the path in that second panel, and the top panel will automatically adjust itself to display the last folder on the path you've supplied.

Once you've got the right 'root' folder mentioned in the second panel, press [Enter] and Giocoso will run off to that folder and start searching for FLACs:

The 'Scanning file system...' message will let you know that Giocoso is at least thinking about things, but there won't be any other evidence of it actually doing any work (potentially for many minutes, if your music collection is large). Eventually, however, that one, slightly cryptic message will give way to a proper progress dialog:

You can see the specific folder being investigated at each moment, and an overall progress bar clicks slowly up to the 100% mark. Eventually, you'll see a message to say the refresh has completed: click [Enter] on that and you'll be returned to the main menu.

The Full Scan completely wipes Giocoso's existing RECORDINGS table (though it never touches the PLAYS table: Giocoso's play history contained in PLAYS is far too precious to ever be deleted, truncated or modified in any way, shape or form). It re-populates it from scatch, which means the process is exhaustive and will accordingly take quite a lot of time, depending on the size of your music collection. With 15,000+ unique recordings, for example, my i5 from 2012 takes about an hour to complete a Full Scan. Functionally, the Full Scan is exactly what happens when you create a database in the first place: every FLAC is analysed and its metadata tags read; that metadata is then loaded into an empty RECORDINGS table.

Practically, this means that a Full Scan will not only add new recordings into a Giocoso music database, but will pick up any metadata changes to existing recordings and will also notice (and record the fact) that this or that recording no longer exists on disk, because you chose to delete it.

By way of an example, consider this situation I discussed at length in the documentation about doing a fast scan. I've got one file on disk:

Despite the folder name it finds itself in, the metadata for this file says:

...that this is a String Quartet written by Benjamin Britten and played by the Amadeus Quartet, even though the COMMENT tag mentions the Utrecht String Quartet! Anyway, after a fast scan, Giocoso is thoroughly confused:

It thinks two files exist, and both are of music by Grachaninov, not Britten. And both are by the Utrecht quartet, with no mention at all of the Amadeus quartet. This is what fast scans get you: new recordings added to a database, but old ones are not removed nor do they ever have their metadata refreshed.

So let me now perform a full scan for this same database:

When I report on the database after a full scan, Giocoso now correctly reports a single FLAC exists; it's correctly by Benjamin Britten; it now also correctly mentions the Amadeus Quartet. A full scan, therefore, causes the metadata for all recordings to be re-read from scratch, so all metadata changes are automatically picked up. Old data that was in Giocoso's database, referencing recordings that no longer exist, is purged, so that the database only mentions FLACs which are known to physically exist.

In other words, if you are actively curating your music collection -adding new stuff to it, deleting obsolete recordings from it, whilst also modifying the metadata associated with particular recordings, you will want to periodically perform a full scan of the collection, so as to ensure that Giocoso's 'picture' of it is as accurate and up-to-date as it can be.

In fact, you will probably want to run fast scans on a daily basis (so that new additions to your collection are picked up in a timely fashion), whilst reserving full scans to be performed maybe once a week (so that metadata edits and recording disposals are taken notice of in a reasonable amount of time).

To this end, it's possible to schedule fast and full scans (using a tool such as cron). Here are my own crontab entries:

# Refresh the Giocoso Database every night at 11pm
# ----------------------------------------------------------------
0 23 * * *     /usr/bin/giocoso3.sh --fastrefresh main /sourcedata/music/classical

# Full Refresh Giocoso every 1st and 15th of month at 4am
# ----------------------------------------------------------------
0 4 1,15 * *     /usr/bin/giocoso3.sh --fullrefresh main /sourcedata/music/classical

Notice that when you want to automate the unattended running of Giocoso in database maintenance mode, you have to launch Giocoso with one of its few remaining runtime switches: --fullrefresh or --fastrefresh, depending on what type of database maintenance you're wanting to perform. You will also note that it's necessary to pass two additional parameters to the program when launched in this manner: the database name you want to refresh and the 'root' music folder in which Giocoso will find all its FLAC files.

From the above crontab entries, you can see that whilst I do a nightly fastrefresh, I only do a fullrefresh once a fortnight. That's a mix of maintenance functions which is about right for me: I do add new recordings to my collection quite often, so I want Giocoso to take notice of those very qickly. However, I try to get my tagging correct at the point of adding the new recordings to my collection, so I don't do a lot of metadata correction or modification after they've been added. I am also notoriously reluctant to dispose of old recordings, no matter how ghastly the music they contain might be... so I don't do a lot of deleting of FLACs off my hard disk! Accordingly, I personally don't need to do very frequent full refreshes. So this particular mix of nightly fast scans and only infrequent full scans makes sense for me and the way I interact with my music collection. You will obviously want to strike your own balance, depending on what you do with yours!


[ User Manual Home ] | [ Back to Database Management Menu ]