`configtree.loader`¶

The module provides utility functions to load tree object from files

class configtree.loader.Loader(walk=None, update=None, postprocess=None, tree=None)¶

Configuration tree loader

Parameters:	walk (Walker) – Walk actor that generates list of files to load update (Updater) – Update actor that implements syntactic sugar postprocess (PostProcessor) – Result tree post processor tree (Tree) – Tree object that should contain result of loading

classmethod fromconf(path)¶

Creates loader using configuration module loaderconf

Parameters:	path (str) – Path to a directory that contains `loaderconf`
Returns:	Ready to use loader object
Return type:	Loader

__call__(path)¶

Loads configuration

Parameters:	path (str) – Path to a directory that contains configuration files.
Returns:	Result tree object
Return type:	Tree

Utilities¶

class configtree.loader.Pipeline¶

Utility class that helps to build pipelines

__pipeline__¶: List of workers that includes each method of the class that marked by worker() decorator. The list is sorted by worker priority. Inactive workers are not included in the list.

static worker(priority, enabled=True)¶

Decorator that marks method as a worker

Parameters:	priority (int) – Priority of the worker enabled (bool) – Whether worker is active or not

Walker¶

class configtree.loader.Walker(**params)¶

File walker is used by Loader to get list of files to load.

params¶

Dictionary that contains all keyword arguments that are passed into constructor. The dictionary is copied into each File object into File.params attribute. This attribute can be used by workers from __pipeline__ to make decisions about the file priority.

Only the env parameter makes sense for environment() worker. All other parameters are simply ignored, but could be used in extensions.

__pipeline__¶

File processing pipeline. Each File object is passed through the following methods until some method returns int value. The special value -1 means that the passed file should be ignored. Other values mean file priority. For instance, regular files (see regular()) have priorities equal to 30 or 31, and final ones (see final()) have 100 or 101. That means that final files will be at the end of result list of files, and regular files will be at the start of the list.

The list of workers is: [ignored(), final(), environment(), regular()]

__call__(path)¶

Walks over the path and yields files to load

Parameters:	path (str) – Path to walk over

walk(current)¶

Processes current traversing file

If current is regular file, it will be yielded as is. If it is directory, the list of its files will be prioritized using __pipeline__. Then the list will be sorted using given priorities and each file will be processed using this method recursively.

The method is low level implementation of __call__() and should not be used directly.

Parameters:	current (File) – Current traversing file

ignored(fileobj)¶

Worker that filters out ignored files and directories

The file will be ignored, if its name starts with dot char or underscore, or the file format is not supported by loader.

Parameters:	fileobj (File) – Current traversing file
Returns:	`-1` when the file is ignored one; `None` when the file is not ignored one.

__priority__ = 10

Examples:

.hidden             # returns -1 (file name starts with dot char)
_ignored            # returns -1 (file name starts with underscore)
unsupported.txt     # returns -1 (txt files are not supported)
other.yaml          # returns None

final(fileobj)¶

Worker that checks whether current traversing file is final or not.

Final files are processed at the end of current list of files. If the file name starts with “final”, it will be treated as final one.

Parameters:	fileobj (File) – Current traversing file
Returns:	`100` when the file is final one and it is directory; `101` when the file is final one and it is regular file; `None` when the file is not final one.

__priority__ = 30

Examples:

final/           # returns 100
final-dir/       # returns 100
other-dir/       # returns None
final.yaml       # returns 101
final-file.json  # returns 101
other.yaml       # returns None

environment(fileobj)¶

Worker that checks whether current traversing file is environment specific or not.

The file will be treated as environment specific, if its name starts with “env-” string. The rest part of the name (without extension) is treated as environment name. If the environment name does not match to env parameter (see params), then the file will be ignored.

Parameters:	fileobj (File) – Current traversing file
Returns:	`-1` when the file is environment specific, but environment name is not match; `50` when the file is environment specific and it is regular file; `51` when the file is environment specific and it is directory; `None` when the file is not environment specific one.

__priority__ = 50

Examples:

# params['env'] == "foo.bar"

env-foo.yaml      # returns 50
env-bar.yaml      # returns -1 (environment name is not match)
env-foo/          # returns 51
    env-bar.yaml  # returns 50
    env-baz.yaml  # returns -1
    other.yaml    # returns None

regular(fileobj)¶

Worker that treats any file as a regular one. The worker should be the last in the __pipeline__, because it does not make any check.

Parameters:	fileobj (File) – Current traversing file
Returns:	`30` when the file is regular file; `31` when the file is directory.

__priority__ = 1000

class configtree.loader.File(path, name, params)¶

Represents current traversing file within Walker routine

path¶: Path of parent directory containing the file

name¶: File name itself

params¶: The copy of Walker.params that could be used and transformed by workers from Walker.__pipeline__. See Walker.environment().

fullpath¶: Full path to the file

isdir¶: Boolean value that means whether the file is directory or not

isfile¶: Boolean value that means whether the file is regular file or not

ext¶: Extension of the file (with leading dot char)

cleanname¶: Name of the file without its extension

Updater¶

class configtree.loader.Updater(**params)¶

Updater is used by Loader to set up key-value pairs into updating tree object. The object extends default updating mechanism adding some syntactic sugar.

params¶

Dictionary that contains all keyword arguments that are passed into constructor. The attribute can be used by workers.

Only the namespace parameter makes sense for eval_value() worker. All other parameters are simply ignored, but could be used in extensions.

__pipeline__¶

Transforms UpdateAction object that created by __call__().

Each UpdateAction object is passed through the following methods. Each method can transform UpdateAction.key, UpdateAction.value, or UpdateAction.update, attributes. So that the default behavior of UpdateAction can be changed.

The list of workers is: [set_default(), call_method(), format_value(), printf_value(), eval_value(), required_value()]

__call__(tree, key, value, source)¶

Updates tree

It creates UpdateAction object. Then pass the object through the __pipeline__. And finally calls the action.

Parameters:	tree (Tree) – Updating tree object key (str) – Setting up key value – Setting up value source (str) – Full path to a source file

set_default(action)¶

Worker that changes default UpdateAction.update from __setitem__ to setdefault if key ends with “?” char.

It also transforms key, i.e. strips the last char.

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 20

Example:

x: 1
x?: 2           # x == 1
y?: 3           # y == 3

call_method(action)¶

Worker that changes default UpdateAction.update if key contains “#” char.

It splits UpdateAction.key by the char. The left part is set up as the key itself. The right part is used as a method name. It gets value from UpdateAction.tree by the new key and call its method using UpdateAction.value as an argument. If any of the values is instance of Promise, then it will be wrapped by another Promise object. See PostProcessor.resolve_promise().

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 30

Example:

foo: [1, 2]
bar: ">>> self['foo'][:]"        # Get copy of the latest `foo`
bar#extend: [5, 6]               # bar == [1, 2, 3, 4, 5, 6]
foo#extend: [3, 4]               # foo == [1, 2, 3, 4]

format_value(action)¶

Worker that transforms UpdateAction.value that starts with "$>> " (with trailing space char) into formatting expression and wraps it into Promise. See PostProcessor.resolve_promise().

The expression uses str.format(). Current tree and current branch are passed as self and branch names into template. Both are wrapped by ResolverProxy.

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 50

Example:

a: "foo"
b:
    x: "bar"
    y: "a = {self[a]!r}, b.x = {branch[x]!r}"
       # == "a = 'foo', b.x = 'bar'"

printf_value(action)¶

Worker that transform UpdateAction.value that starts with "%>> " (with trailing space char) into formatting expression and wraps it into Promise. See PostProcessor.resolve_promise().

The expression uses printf style, i.e. % operator. UpdateAction.tree wrapped by ResolverProxy is used as a formatting value.

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 60

Example:

name: "World"
hello: "%>> Hello %(name)s"     # == "Hello World"

eval_value(action)¶

Worker that transform UpdateAction.value that starts with ">>> " (with trailing space char) into expression and wraps it into Promise. See PostProcessor.resolve_promise().

The expression uses built-in function eval(). The value of namespace key from params is passed as gloabls argument of eval. UpdateAction.tree is passed as self and UpdateAction.branch is passed as branch names via locals argument of eval. Both are wrapped by ResolverProxy.

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 70

Example:

>>> from math import floor
>>> update = Updater(namespace={'floor': floor})

a: ">>> 1 + 2"                         # == 3
b:
    x: 3
    y: ">>> self['a'] * branch['x']"   # == 9
    z: ">>> floor(3.0 / 2)"            # == 1

required_value(action)¶

Worker that transform UpdateAction.value that starts with "!!!" into an instance of Required. See PostProcessor.check_required().

Parameters:	action (UpdateAction) – Current update action object

__priority__ = 80

Example:

foo: "!!!"                              # without comment
bar: "!!! This should be redefined"     # with comment

class configtree.loader.UpdateAction(tree, key, value, source)¶

Helper object that is used within Updater routine. It represents current update context.

tree¶: Current updating configtree.tree.Tree object

branch¶

Property that is used to get current branch from the tree

>>> tree = Tree({'a.x': 1, 'a.y': 2})
>>> action = UpdateAction(tree, 'a.z', 3, '/path/to/src.yaml')
>>> action.branch == tree['a']
True

key¶: Current setting up key

value¶: Current setting up value

source¶

Path to a file processing by Loader. Is used as a part of debug information.

>>> UpdateAction(Tree(), 'foo', 'bar', '/path/to/src.yaml')
<tree['foo'] = 'bar' from '/path/to/src.yaml'>

update¶: Callable object that represent current update action. By default is equal to default_update().

__call__()¶: Calls update, i.e. performs update action

promise(deferred)¶

Helper method that wraps deferred callable by try-except block. It adds self as a first argument to any exception that might be raised from deferred. So that the exception will contain information of what expression from which file is caused it.

Parameters:	deferred (callable) – Callable object that should be wrapped by `Promise`

static default_update(action)¶

Default value of update. Literally performs:

action.tree[action.key] = action.value

Parameters:	action (UpdateAction) – Current action object

class configtree.loader.Promise(deferred)¶

Represents deferred expression that should be calculated at the end of loading process. See resolve(), Updater.eval_value(), Updater.format_value(), Updater.printf_value(), and PostProcessor.resolve_promise().

Parameters:	deferred (callable) – Deferred expression

__call__()¶: Resolves deferred value, i.e. calls it and returns its result

static resolve(value)¶

Helper method that resolves passed promises and returns their results. Other values are returned as is.

Parameters:	value – Value to resolve
Returns:	Resolved promise or value as it is.

>>> Promise.resolve(Promise(lambda: 1))
1
>>> Promise.resolve(2)
2

class configtree.loader.ResolverProxy(tree, source=None)¶

Helper object that wraps configtree.tree.Tree objects.

It pass each extracted value through resolve(), so that one deferred expression (see Promise) can use another.

If source argument is not None, there will be __file__ and __dir__ keys available.

Parameters:	tree (Tree) – Tree object to wrap source (str) – Path to source file

>>> tree = Tree()
>>> proxy = ResolverProxy(tree, '/path/to/src.yaml')
>>> tree['foo'] = Promise(lambda: 1)
>>> tree['bar'] = Promise(lambda: proxy['foo'] + 1)
>>> proxy['foo']
1
>>> proxy['bar']
2
>>> proxy['__file__']
'/path/to/src.yaml'
>>> proxy['__dir__']
'/path/to'

class configtree.loader.Required(key, comment='')¶

Helper object that indicates undefined required key.

Values of the type are set up by Updater.required_value() and treated as error by PostProcessor.check_required().

Post processor¶

class configtree.loader.PostProcessor¶

Post processor is used by Loader to perform final transformations of its result tree object after loading process is finished.

Post processor iterates over passed configtree.tree.Tree object and pass its keys and values through __pipeline__. If any worker of the pipeline returns non None value, this value will be treated as an error. Such errors are accumulated and raised within ProcessingError exception at the end of processing.

__pipeline__¶: The list of workers is: [resolve_promise(), check_required()]

__call__(tree)¶

Runs post processor

Parameters:	tree (Tree) – A tree object to process

resolve_promise(tree, key, value)¶

Worker that resolves Promise objects.

Any exception raised within promise expression will not be caught.

Parameters:	tree (Tree) – Current processing tree key (str) – Current traversing key value – Current traversing value

__priority__ = 30

check_required(tree, key, value)¶

Worker that checks tree for raw Required values.

Parameters:

tree (Tree) – Current processing tree
key (str) – Current traversing key
value – Current traversing value

Returns:

passed value, if it is an an instance of Required;
None for other values.

__priority__ = 50

class configtree.loader.ProcessingError¶: Exception that will be raised, if post processor gets any error

`configtree.loader`¶

Utilities¶

Walker¶

Updater¶

Post processor¶

Table of Contents

Previous topic

Next topic

This Page

configtree.loader¶

Utilities¶

Walker¶

Updater¶

Post processor¶

`configtree.loader`¶