============================= Plugin Oriented Configuration ============================= Applications need to be able to be configured. This presents a number of unique challenges to Plugin Oriented Programming. For the concepts around app merging to work, the loading up of configuration data at the startup of an application must be taken into full account. The process of loading up that data must also be pluggable. Addressing these needs creates a difficult point of intersection. As these points get evaluated it becomes necessary to have systems in place that can make configuration of the applications not only easy, but accessible and extensible. The configuration loading system used by `pop` is called `pop-config`. Extensive documentation on how to use `pop-config` is presented by the `pop-config` project. Configuration Merging ===================== Since apps can merge, configuration can merge. When defining configuration that can be merged between multiple applications it becomes critical that the loaded configuration is consistent across applications even after they have been merged. To do this, the `conf.py` files of each project are read, parsed, namespaced, and allowed to be loaded via several mediums. The options for a specific application are stored on the `hub` in a predictable path that is made available to the application, whether loaded standalone or merged into another application. Configuration Priority ====================== When loading a robust configuration system for an application, it needs to load configuration options in a specific order. Configuration of an application, as applied during the startup of the application, can be loaded from several sources. There are four main sources where configuration loading can occur: defaults, environment variables, configuration files, and command-line flags. These then need to be applied in the correct priority. - Command-line flags overwrite environment variables - Environment variables overwrite configuration file options - Configuration file options overwrite defaults The `pop-config` system allows you to simply place the configuration values, defaults, accepted sources, and documentation, into a single location. This makes it easy to add configuration options to a project and automatically makes those configuration options app-mergeable! The conf.py Dictionaries ======================== Plugin Oriented Programming presents a single file to specify the configuration of a given project. This file is used to set up all configuration options and to load said options onto the hub. We have already introduced this file for the use of the *DYNE* dictionary. But there are 3 additional dictionaries found therein. The *CLI_CONFIG* dictionary, the *CONFIG* dictionary, and the *SUBCOMMANDS* dictionary. Each dictionary holds data to present configuration options to the end user. CONFIG Dictionary ================= The *CONFIG* dictionary is a very powerful system for applying configuration values and where the root of configuration should be applied. The *CONFIG* dictionary is primarily used for non command-line flags, those should be reserved for the *CLI_CONFIG* dictionary. Basic Settings -------------- Nearly every config setting needs to have 2 basic options, `default` and `help`. These are very self explanatory. `default` sets the default value of the option if no option is passed and `help` presents, not only the command-line help, but is also the single source of documentation for the option. Here is a simple example: .. code-block:: python CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } This establishes the basic data for the setting and is all that is needed for settings in the *CONFIG* dictionary. Destination ----------- When the argument is named "test" it will appear on the option namespace as "test". This may not always be desirable. If the name of the option and where it needs to be stored differs, then use the `dest` option: .. code-block:: python CONFIG = { "test": { "default": "Red", "dest": "cheese", "help": "What color to test", }, } In this example the option will be stored under the name "cheese". Location -------- Once the config system has been run, all configuration data will appear in the `hub.OPT` namespace. This means that in our first example, if the system in question is part of an app named `myapp`, then the option data will be present at `hub.OPT["myapp"]["test"]`. In the second example, the option data will be present at `hub.OPT["myapp"]["cheese"]`. CLI_CONFIG ========== The heaviest configuration options will be in the *CLI_CONFIG* dictionary. All options that appear on the CLI need to be activated in the *CLI_CONFIG*, but the basic configuration needs to be in the *CONFIG* dictionary. Pop-config uses Python's venerable `argparse` under the hood to present and process the arguments. Pop-config will also transparently pass options from the dictionary into `argparse`. This makes Pop-config transparently compatible with new `argparse` options that are made available. This document is intended, therefore, to present the most commonly used options. Please see the `argparse` doc for more in depth data on available options. If the cli option is very simple it can be as simple as just doing this: .. code-block:: python CLI_CONFIG = { "test": {}, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } This way the options presented in the cli are all explicitly set. But additional options about how the cli is presented to the end user is easy to do. This document will cover all available options for the *CLI_CONFIG*. Remember that the `default` and `help` values should always be in the *CONFIG* section to facilitate easy app-merging of configuration data and settings. Source ------ By default, the *CLI_CONFIG* references the local *CONFIG* setting. The `source` option allows you to reference a documented configuration from a separate project configuration. This powerful option allows you to manage the arguments and flags in a namespace of an app that is being merged into this app. The benefit here is that the *CONFIG* values do not need to be rewritten and you maintain a single authoritative source of documentation. When using `source` in the *CLI_CONFIG* the namespace that defined the option in the *CONFIG* dictionary will own the option. This makes it easy for an application that uses its own config namespace to be app merged into another application that can then transparently manage the configuration of the merged app. Dyne ---- A powerful option in the *CLI_CONFIG* is `dyne`. This uses vertical app merging to modify another application's cli options. This allows a vertical app merge repo to define cli arguments that will be made available when the plugins are installed to extend an external app. Options ------- By default the options presented on the command-line are identical to the name of the value. So for the above example the presented option would be `--test`. If alternative options are desired, they can be easily added: .. code-block:: python CLI_CONFIG = { "test": { "options": ["-t", "--testy-mc-tester", "-Q"], }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } Pop-config automatically determines between short options and long options. Positional Arguments -------------------- Positional arguments are very common and can create a much more user friendly experience for users. Adding positional arguments is easy. Just use the `positional` argument: .. code-block:: python CLI_CONFIG = { "test": { "positional": True, }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } When working with multiple positional arguments the `display_priority` flag can be used: .. code-block:: python CLI_CONFIG = { "test": { "positional": True, "display_priority": 2, }, "run": { "positional": True, "display_priority": 1, }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, "run": { "default": "green", "help": "What color to run", }, } In the above example the first argument will be `run` and the second will be `test`. Accepting Environment Variables ------------------------------- Operating systems allow for configuration options to be passed in via specific means. In Unix based systems like Linux and MacOS, environment variables can be used. In Windows based systems the registry can be used. To allow for an os variable to be used just add the os option: .. code-block:: python CLI_CONFIG = { "test": { "os": "MYAPP_TEST", }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } Now the flag can be set by setting the environment variable `MYAPP_TEST` to the desired configuration value. Actions ------- Actions allow a command-line argument to perform an action, or flip a switch. The `action` option passes through to `argparse`. If the examples in this document do not make sense you can also check the `argsparse` section on `action`. A number of actions are supported by `argparse`. Arguably the most frequently used actions are `store_true` and `store_false`: .. code-block:: python CLI_CONFIG = { "test": { "action": "store_true", }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } A few other useful actions are `append` and `count`. If `append` is used then every time the argument is used the option passed to the argument is appended to the final list. The `count` option allows for the number of times that the argument is passed to be counted up. This is useful for situations where you want to specify what the verbosity of the output should be, so that you can pass `-vvv` in a similar fashion to ssh. Number of Arguments ------------------- The number of arguments that should be expected can also be set using the `nargs` option. This allows for a specific or fluid number of options to be passed into a single cli option. The `nargs` option passes through to `argparse`. If the examples in this document do not make sense you can also check the `argsparse` section on `nargs`. Integer (1) ``````````` Specifying an integer defines the explicit number of options to require: .. code-block:: python CLI_CONFIG = { "test": { "nargs": 3, }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } The above example will require that exactly 3 options are passed to `--test`. Question Mark (?) ````````````````` One argument will be consumed from the command-line, if possible, and produced as a single item. If no command-line argument is present, the value from default will be produced. Asterisk (*) ```````````` All command-line arguments present are gathered into a list. .. code-block:: python CLI_CONFIG = { "test": { "nargs": "*", }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } Plus (+) ```````` Just like '*', all command-line args present are gathered into a list. Additionally, an error message will be generated if there wasn't at least one command-line argument present. Type ---- The value type can be enforced with the `type` option. A type can be passed in that will be enforced, such as `int` or `str`. .. code-block:: python CLI_CONFIG = { "test": { "type": int, }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } Render ------ Sometimes it is desirable to load up complex data structures from the command-line. This can be done with the `render` option. The `render` option allows you to specify that the argument passed will be rendered using a data serialization medium such as json or yaml. .. code-block:: python CLI_CONFIG = { "test": { "render": "yaml", }, } CONFIG = { "test": { "default": "Red", "help": "What color to test", }, } This cli could then look like this: .. code-block:: bash myapp --test "Food: true" Then the resulting value would be: `{"Food": True}` Subcommands ----------- Sometimes it is desirable to have subcommands. Subcommands allow your CLI to work in a way similar to the git cli, where you have multiple routines that all can be called from a single command. This example shows how multiple subcommands can be defined and utilized. .. code-block:: python CLI_CONFIG = { "name": { "subcommands": ["test", "apply"], }, "weight": {}, "power": { "subcommands": ["apply"], }, } CONFIG = { "name": { "default": "frank", "help": "Enter the name to use", }, "weight": { "default": "150", "help": "Enter how heavy it should be", }, "power": { "default": "100", "help": "Enter how powerful it should be", }, } SUBCOMMANDS = { "test": { "help": "Used to test", "desc": "When running in test mode, things will be tested", }, "apply": { "help": "Used to apply", "desc": "When running in apply mode, things will be applied", }, } In this example we see that the option `name` will be available under the subcommands `test` and `apply`. The option `power` will be available only under the subcommand `apply` and the option `weight` is globally available. Conclusion ========== In the end, `pop-config` gives you everything that you need to create not only beautiful command-line applications, but also deeply robust configuration loading systems that facilitate app merging.