Merlin's Database

Note

If running in local mode, results will be stored in a sqlite file in your Merlin home directory. This file will be named ~/.merlin/merlin.db. To interact with this file while using the merlin database command, use the -l (or --local) option. For example,

merlin database -l info

When a Merlin study is run with the merlin run and merlin run-workers commands, information about the study is stored as data in the results backend database.

See the below sections for how the database works and why it's needed. Additional info about Merlin's database can be found at the below pages:

• Understanding Merlin's Database Entities
• The Database Command - Overview of the database command
• Info - Using the info subcommand
• Retrieving Data - Using the get subcommand
• Deleting Data - Using the delete subcommand
• Garbage Collection - Using the gc subcommand

How it Works

Unless running locally, Merlin requires a connection to a results backend database by default. Therefore, Merlin's database will automatically link to this results backend connection, using it as a store for study-related information.

When a study is converted to Celery tasks and sent to the broker with the merlin run command, Merlin will create the following entities in your database: a StudyEntity (if one does not yet exist), a RunEntity, and one or more LogicalWorkerEntity instances (if they don't already exist).

Similarly, when workers are started on your allocation with the merlin run-workers command, Merlin will create one or more LogicalWorkerEntity instances (if they don't already exist) and one or more PhysicalWorkerEntity instances.

Why is this Needed?

The entities that are created in the database are used by the merlin monitor command to help keep your workflow alive during the lifetime of your allocation. You can read more about how this process works on the Monitoring Studies for Persistent Allocations page.