Getting Started¶
The following tutorial demonstrates basic usage of ConfigTree. It covers all features, but does not explain them in details. It is enough to start using ConfigTree within your project, however more detailed explanation will be given in the next section.
Warming up¶
Let’s create a separate directory, for our experiments:
$ mkdir configs
$ cd configs
Then create some dummy data:
$ touch test.yaml
$ echo "x.y: 1" >> test.yaml
$ echo "x:" >> test.yaml
$ echo " z: 2" >> test.yaml
The test file should look like this:
x.y: 1
x:
z: 2
Now, let’s run ctdump shell command to see how it loads our file:
$ ctdump json
{"x.z": 2, "x.y": 1}
Let’s play with its arguments:
$ ctdump json --json-indent=4 --json-sort
{
"x.y": 1,
"x.z": 2
}
$ ctdump json --json-indent=4 --json-sort --json-rare
{
"x": {
"y": 1,
"z": 2
}
}
And finally, let’s load the file within Python code and fiddle with the result:
>>> from configtree import Loader
>>> load = Loader()
>>> tree = load('.')
>>> tree
Tree({'x.y': 1, 'x.z': 2})
>>> tree['x']
BranchProxy('x'): {'z': 2, 'y': 1}
>>> tree['x.y']
1
>>> tree == {'x.y': 1, 'x.z': 2}
True
>>> tree['x'] == {'y': 1, 'z': 2}
True
You can see that:
ConfigTree flattens the file on loading, i.e. there is no difference between dot-separated keys and nested mappings:
# This is identical... x.y: 1 x.z: 2 # ...to this x: y: 1 z: 2
See
configtree.tree.flatten()
for details.ConfigTree uses
configtree.tree.Tree
to store the result. This class provides dictionary interface and can be used wherever built-indict
is expected. It also provides ability to get branches, i.e. expose intermediate keys. That is why it named “Tree”.configtree.loader.Loader
is used to loadconfigtree.tree.Tree
object from files. The following tutorial is devoted to its features.ctdump shell command can be used to dump tree into JSON, so it can be useful to build configuration for programs written in other programming languages.
Now remove the test file and move on to the real world example.
Safe defaults¶
Let’s imagine that we develop a web service, which consists of two web applications: frontend and REST API. First of all, we need simple configurations for development and production environments. These two configurations will have lots of common parameters. So it will be better to create a default configuration, that should be updated by environment-specific options.
However, the default configuration must contain safe default parameters. Because it is always possible that someone forget to override default value in the production environment. Nobody wants to go live with weak cryptographic keys, for instance.
Using ConfigTree it is possible to mark keys as required. So the loader will raise an error, if such keys have not been overridden.
Create default.yaml
with the following content:
api: # API configuration
host: "!!! API host name"
port: 80
db:
driver: "mysql"
user: "!!!"
password: "!!!"
name: "demo_db"
secret: "!!! Web tokens encryption key"
logging: "error"
frontend: # Frontend configuration
host: "!!! Frontend host name"
port: 80
js:
merge: yes
minify: yes
css:
merge: yes
minify: yes
templates:
reload: no
cache: yes
logging: "error"
Now let’s test it:
$ ctdump json
configtree [ERROR]: Undefined required key <api.db.password>
configtree [ERROR]: Undefined required key <api.db.user>
configtree [ERROR]: Undefined required key <api.host>: API host name
configtree [ERROR]: Undefined required key <api.secret>: Web tokens encryption key
configtree [ERROR]: Undefined required key <frontend.host>: Frontend host name
As you can see, the loader reports error for each key, marked with “!!!”.
If you run loader programmatically, an exception of configtree.loader.ProcessingError
will be raised.
Move on and see how to override the values in the environment-specific configuration.
Loading environment-specific configuration¶
Let’s create production configuration in file env-prod.yaml
with the following
content:
api:
host: "api.example.com"
db:
user: "demo_user"
password: "pa$$w0rd" # Password must be strong
secret: "s3cre7" # As well as cryptographic key :)
frontend:
host: "www.example.com"
Now we should “say” to the loader to load this file only in the production environment.
The part of loader that responds to get list of files to load is Walker.
To change its default behavior, we should manually create configtree.loader.Walker
object and pass it into configtree.loader.Loader
:
>>> from configtree import Loader, Walker
>>> walk = Walker(env='prod')
>>> load = Loader(walk=walk)
To make it work in ctdump shell command, create loaderconf.py file with the following content:
import os
from configtree import Walker
walk = Walker(env=os.environ['ENV_NAME'])
And test it:
$ ENV_NAME=prod ctdump json
{...}
Using hierarchical environments¶
Now let’s think about development environments. Our imaginable project consists of two parts: API and frontend. So our imaginable team should consist of two sub-teams: API developers and frontend developers.
The frontend team does not care about backend logs, but they want to have debug logging level on frontend. They also work on templates, and want to switch off caching and switch on reloading options, and so on. While the backend team needs slightly different configuration.
So let’s create a directory for development configuration with three files:
env-dev/ # Development configuration directory
common.yaml # Common development options
env-api.yaml # API team development options
env-frontend.yaml # Frontend team development options
And play with ENV_NAME
. Here we use --verbose
option of ctdump shell command
to get list of loaded files:
$ ENV_NAME=dev ctdump json --verbose
configtree [INFO]: Walking over "/path/to/configs"
configtree [INFO]: Loading "defaults.yaml"
configtree [INFO]: Loading "env-dev/common.yaml"
$ ENV_NAME=dev.api ctdump json --verbose
configtree [INFO]: Walking over "/path/to/configs"
configtree [INFO]: Loading "defaults.yaml"
configtree [INFO]: Loading "env-dev/common.yaml"
configtree [INFO]: Loading "env-dev/env-api.yaml"
$ ENV_NAME=dev.frontend ctdump json --verbose
configtree [INFO]: Walking over "/path/to/configs"
configtree [INFO]: Loading "defaults.yaml"
configtree [INFO]: Loading "env-dev/common.yaml"
configtree [INFO]: Loading "env-dev/env-frontend.yaml"
As you can see, environments can be organized in hierarchy, where the most common configuration options are defined at the root, and the most specific—at the leafs.
Templates and evaluable expressions¶
Sometimes you need to calculate some values in your configuration.
For example, let’s add some endpoint URLs to the API configuration.
Edit default.yaml
file and add the following:
api:
# Previously added API configuration goes here
endpoints:
index: "%>> http://%(api.host)s:%(api.port)s"
login: "%>> %(api.endpoints.index)s/login"
logout: "%>> %(api.endpoints.index)s/logout"
frontend:
# Previously added frontend configuration goes here
In the result production configuration it will look like this:
{
"api": {
"endpoints": {
"index": "http://api.example.com:80",
"login": "http://api.example.com:80/login",
"logout": "http://api.example.com:80/logout"
}
}
}
Such expressions are calculated after whole configuration has been loaded. So you can use values that are defined after the expression, or even defined in another file.
You can also use expressions similar to standard Python console. And even add your own syntactic sugar. See Updater and Post-processor sections of the manual for details.