Skip to content

utils

Utility functions to support Merlin CLI command handlers.

This module provides common helper functions used by various CLI commands in the Merlin application. These utilities focus on parsing and validating command-line arguments related to specification files and variable overrides, as well as loading and expanding Merlin YAML specifications.

get_merlin_spec_with_override(args)

Shared command to retrieve a MerlinSpec object and an expanded filepath.

This function processes parsed command-line interface (CLI) arguments to validate and expand the specified filepath and any associated variables. It then constructs and returns a MerlinSpec object based on the provided specification.

Parameters:

Name Type Description Default
args Namespace

Parsed CLI arguments containing:

  • specification: the path to the specification file
  • variables: optional variable overrides to customize the spec.
 required

Returns:

Name Type Description
spec MerlinSpec

An instance of the MerlinSpec class with the expanded configuration based on the provided filepath and variables.
filepath str

The expanded filepath derived from the specification.
Source code in merlin/cli/utils.py 
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def get_merlin_spec_with_override(args: Namespace) -> Tuple[MerlinSpec, str]:
    """
    Shared command to retrieve a [`MerlinSpec`][spec.specification.MerlinSpec] object
    and an expanded filepath.

    This function processes parsed command-line interface (CLI) arguments to validate
    and expand the specified filepath and any associated variables. It then constructs
    and returns a [`MerlinSpec`][spec.specification.MerlinSpec] object based on the
    provided specification.

    Args:
        args: Parsed CLI arguments containing:\n
            - `specification`: the path to the specification file
            - `variables`: optional variable overrides to customize the spec.

    Returns:
        spec (spec.specification.MerlinSpec): An instance of the
            [`MerlinSpec`][spec.specification.MerlinSpec] class with the expanded
            configuration based on the provided filepath and variables.
        filepath: The expanded filepath derived from the specification.
    """
    filepath = verify_filepath(args.specification)
    variables_dict = parse_override_vars(args.variables)
    spec = get_spec_with_expansion(filepath, override_vars=variables_dict)
    return spec, filepath

parse_override_vars(variables_list)

Parse a list of command-line variables into a dictionary of key-value pairs.

This function takes an optional list of strings following the syntax "KEY=val" and converts them into a dictionary. It validates the format of the variables and ensures that keys are valid according to specified rules.

Parameters:

Name Type Description Default
variables_list Optional[List[str]]

An optional list of strings, where each string should be in the format "KEY=val", e.g., ["KEY1=value1", "KEY2=42"].

 required

Returns:

Type Description
Optional[Dict[str, Union[str, int]]]

A dictionary where the keys are variable names (str) and the values are either strings or integers. If variables_list is None or empty, returns None.

Raises:

Type Description
ValueError

If the input format is incorrect, including:

  • Missing '=' operator.
  • Excess '=' operators in a variable assignment.
  • Invalid variable names (must be alphanumeric and underscores).
  • Attempting to override reserved variable names.
Source code in merlin/cli/utils.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def parse_override_vars(
    variables_list: Optional[List[str]],
) -> Optional[Dict[str, Union[str, int]]]:
    """
    Parse a list of command-line variables into a dictionary of key-value pairs.

    This function takes an optional list of strings following the syntax
    "KEY=val" and converts them into a dictionary. It validates the format
    of the variables and ensures that keys are valid according to specified rules.

    Args:
        variables_list: An optional list of strings, where each string should be in the
            format "KEY=val", e.g., ["KEY1=value1", "KEY2=42"].

    Returns:
        A dictionary where the keys are variable names (str) and the
            values are either strings or integers. If `variables_list` is
            None or empty, returns None.

    Raises:
        ValueError: If the input format is incorrect, including:\n
            - Missing '=' operator.
            - Excess '=' operators in a variable assignment.
            - Invalid variable names (must be alphanumeric and underscores).
            - Attempting to override reserved variable names.
    """
    if variables_list is None:
        return None
    LOG.debug(f"Command line override variables = {variables_list}")
    result: Dict[str, Union[str, int]] = {}
    arg: str
    for arg in variables_list:
        try:
            if "=" not in arg:
                raise ValueError("--vars requires '=' operator. See 'merlin run --help' for an example.")
            entry: str = arg.split("=")
            if len(entry) != 2:
                raise ValueError("--vars requires ONE '=' operator (without spaces) per variable assignment.")
            key: str = entry[0]
            if key is None or key == "" or "$" in key:
                raise ValueError("--vars requires valid variable names comprised of alphanumeric characters and underscores.")
            if key in RESERVED:
                raise ValueError(f"Cannot override reserved word '{key}'! Reserved words are: {RESERVED}.")

            val: Union[str, int] = entry[1]
            with suppress(ValueError):
                int(val)
                val = int(val)
            result[key] = val

        except Exception as excpt:
            raise ValueError(
                f"{excpt} Bad '--vars' formatting on command line. See 'merlin run --help' for an example."
            ) from excpt
    return result