Giocoso 3 - The Database Management Menu : Full Scan all recordings

1.0 Performing a Full Scan

Fast scans are great for picking up the existence of new recordings. From Version 3.30 onwards, they also clear out old recordings that no longer exist on disk. A Fast Scan, however, never picks up modifications to the metadata inside existing FLAC files. For that, you'd need to completely re-scan your music collection, starting with a totally blank slate -and that's called a Full Scan and is what Database Management menu Option 3 will do for you.

When you first take this menu option, you are required to specify which database you want to refresh:

The form will simply list every file with a .db extension that's found in the $HOME/.local/share/giocoso3/db folder. Whilst there may be many such files, you can only select one of them at a time with this form: you simply arrow up-and-down the list of file names until the correct one is highlighted, then press the Space Bar to select it. Doing so causes an asterisk to appear in the brackets next to the selected database's name. You can then press [Enter] to complete the selection.

Giocoso then prompts you for the 'root' folder of your music collection: the one folder that is at the 'base' of your folder structures in which all FLACs can be found, no matter how many sub-folders they may be buried in:

You can navigate your on-disk folder structures by arrowing around the entries in the top pane and pressing the Space Bar when a correct folder name is highlighted. As you 'descend' into a folder, its name is appended to the full path that appears underneath the folder list, in the second panel of the form. Alternatively, you can simply type the path in that second panel, and the top panel will automatically adjust itself to display the last folder on the path you've supplied.

Once you've got the right 'root' folder mentioned in the second panel, press [Enter] and Giocoso will run off to that folder and start searching for FLACs:

The 'Scanning file system...' message will let you know that Giocoso is at least thinking about things, but there won't be any other evidence of it actually doing any work (potentially for many minutes, if your music collection is large). Eventually, however, that one, slightly cryptic message will give way to a proper progress dialog:

You can see the specific folder being investigated at each moment, and an overall progress bar clicks slowly up to the 100% mark. Eventually, you'll see a message to say the refresh has completed: click [Enter] on that and you'll be returned to the main menu.

The Full Scan completely wipes Giocoso's existing RECORDINGS table (though it never touches the PLAYS table: Giocoso's play history contained in PLAYS is far too precious to ever be deleted, truncated or modified in any way, shape or form). It re-populates it from scatch, which means the process is exhaustive and will accordingly take quite a lot of time, depending on the size of your music collection. With 15,000+ unique recordings, for example, my i5 from 2012 takes about an hour to complete a Full Scan. Functionally, the Full Scan is exactly what happens when you create a database in the first place: every FLAC is analysed and its metadata tags read; that metadata is then loaded into an empty RECORDINGS table.

Practically, this means that a Full Scan will not only add new recordings into a Giocoso music database, but will pick up any metadata changes to existing recordings and will also notice (and record the fact) that this or that recording no longer exists on disk, because you chose to delete it. In other words, if you are actively curating your music collection -adding new stuff to it, deleting obsolete recordings from it, whilst also modifying the metadata associated with particular recordings- you will want to periodically perform a full scan of the collection, so as to ensure that Giocoso's 'picture' of it is as accurate and up-to-date as it can be.

2.0 Scheduling Full Scans

In fact, you will probably want to run fast scans on a daily basis (so that new additions to your collection are picked up in a timely fashion), whilst reserving full scans to be performed maybe once a week (so that metadata edits and recording disposals are taken notice of in a reasonable amount of time).

To this end, it's possible to schedule fast and full scans (using a tool such as cron). Here are my own crontab entries:

# Refresh the Giocoso Database every night at 11pm
# ----------------------------------------------------------------
0 23 * * *     /usr/bin/giocoso3.sh main /sourcedata/music/classical --fastrefresh

# Full Refresh Giocoso every 1st and 15th of month at 4am
# ----------------------------------------------------------------
0 4 1,15 * *     /usr/bin/giocoso3.sh main /sourcedata/music/classical --fullrefresh

Notice that when you want to automate the unattended running of Giocoso in database maintenance mode, you have to launch Giocoso with one of its few remaining runtime switches: --fullrefresh or --fastrefresh, depending on what type of database maintenance you're wanting to perform. You will also note that it's necessary to pass two additional parameters to the program when launched in this manner: the database name you want to refresh and the 'root' music folder in which Giocoso will find all its FLAC files. The three variables you add to the basic call to run Giocoso are, therefore, and in order: database name (without the .db extension of the file name associated with that database), the root of the music folder containing the FLACs you want to scan and, finally, the full or fast refresh runtime switch.

From the above crontab entries, you can see that whilst I do a nightly fastrefresh, I only do a fullrefresh once a fortnight. That's a mix of maintenance functions which is about right for me: I do add new recordings to my collection quite often, so I want Giocoso to take notice of those very qickly. However, I try to get my tagging correct at the point of adding the new recordings to my collection, so I don't do a lot of metadata correction or modification after they've been added. Accordingly, I personally don't need to do very frequent full refreshes. So this particular mix of nightly fast scans and only infrequent full scans makes sense for me and the way I interact with my music collection. You will obviously want to strike your own balance, depending on what you do with yours!

3.0 Full Scans and Giocoso Pro

All fast (and full) refreshes of Giocoso are updates to the Giocoso local database only. Each installation of Giocoso on different music-playing computing devices will have its own, unique database, periodically refreshed and kept accurate by fast or full refreshes.

However, the release of Version 3.30 introduced the concept of 'Giocoso Pro', whereby a device's local database contents can be 'pushed' to a remote MySQL/MariaDB database that is shared amongst all music-playing devices. Accordingly, from that version onwards, and if Pro functionality has been enabled by setting a remote database server's IP address in the Giocoso configuration file, a fast refresh will complete its local work by going on to copy the refreshed local RECORDINGS table to the remote shared database. Essentially, the local RECORDINGS table's data is inserted into the remote database's GLOBAL_RECORDINGS table.

This data copying process is always a complete replacement of the GLOBAL_RECORDINGS table contents. It starts by deleting all rows from that table and then re-loading it from scratch. In this way, the GLOBAL_RECORDINGS table always contains up-to-date  data about the recordings that the device doing the full refresh can (and can no longer) see.

Note the implication here, however: if Device A refreshes itself, it wipes and replaces the global_recordings table that all devices see and use. If Device B then comes along and refreshes, it will wipe and replace the global_recordings table... and so on for every other device. The assumption underlying Giocoso Pro, however, is that all devices see the same digital music collection, so Device B really didn't need to do a refresh at all if Device A had already done it: they'd both be scanning for the same files and both would discover the exact-same number of music files. In other words, in a Giocoso Pro environment, pick one device to be the one that does routine refreshes: all other devices just need to be pointed to the global database to get their source of plays. They themselves do not need to perform routine refreshes to pick up the latest arrivals to the collections, provided only that one designated device has done so.


[ User Manual Home ] | [ Back to Database Management Menu ]