django-confit¶
django-confit eases Django configuration management.
As a Django user, in order to configure a project:
- django-confit helps you load the settings wherever they are, whatever the format: Python modules, environment variables, JSON, YAML...
- django-confit validates the settings, i.e. it tells you if some directive is missing, has wrong format...
As a Django library developer, in order to help your application’s users:
- you write configuration schemas for your application, using django-confit‘s toolkit and conventions.
- django-confit helps you document your application’s specific configuration.
As a non Django user, in order to deploy and run a Django-powered project:
- you write the configuration as you like, depending on your workflow and your provisioning toolkit. You know the project can load them using django-confit.
- you expect applications to validate the configuration before they actually use it, and report errors with a readable output.
Example¶
In a project’s settings.py
file, let’s load configuration from various
locations:
import os
import django_confit
# Load settings.
raw_settings = {}
raw_settings.update(django_confit.load_module('myproject.default_settings'))
raw_settings.update(django_confit.load_file(open('/etc/myproject.json')))
raw_settings.update(django_confit.load_mapping(os.environ, prefix='MYPROJECT_')
# Update globals, because that's the way Django uses DJANGO_SETTINGS_MODULE.
globals().update(raw_settings)
Optionally, you can use builtin schemas to validate settings:
# Validate and clean settings.
cleaned_settings = django_confit.validate_settings(raw_settings)
# Update globals, because that's the way Django uses DJANGO_SETTINGS_MODULE.
globals().update(cleaned_settings)
Warning
At the moment, there is no builtin schema for latest Django>=1.7.
Project status¶
Today, django-confit is a proof of concept:
- loading settings is nice and easy.
- validating configuration is easy... provided you have the schemas.
- creating configuration schemas is verbose. It uses colander [1] which has nice features, but may not be the definitive option.
- generating documentation from schemas is not implemented.
The main limitation is that, when you use validation, schemas are mandatory. If some configuration directive is not registered in a schema, it will not be present in validation output. It means that, if you install a new third-party Django application, you need the configuration schema for this application, else its settings will not pass validation. So the most-wanted contribution is submitting configuration schemas for third-party applications.
Notice that this behaviour is a wanted feature. As django-confit author, I think libraries should always provide a schema for the settings they use. I do not pretend django-confit should be THE answer. I just bet that, if schemas were widely adopted by the Django community, configuration would be easier to manage.
django-confit does not pretend to be the ultimate configuration management app for Django. Its goal is to show how some issues could be resolved, and to highlight the benefits. django-confit is a proposal. If you like its concepts, then you can:
- use django-confit of course!
- discuss, spread the word, send feedback.
- improve code. Help around configuration schemas of third-party apps would be appreciated.
Ressources¶
- Documentation: https://django-confit.readthedocs.org
- PyPI page: https://pypi.python.org/pypi/django-confit/
- Code repository: https://github.com/benoitbryon/django-confit
- Bugtracker: https://github.com/benoitbryon/django-confit/issues
- Continuous integration: https://travis-ci.org/benoitbryon/django-confit
- Roadmap: https://github.com/benoitbryon/django-confit/milestones
Contents¶
Install¶
django-confit is open-source, published under BSD license. See License for details.
If you want to install a development environment, you should go to Contributing to django-confit documentation.
Prerequisites¶
Warning
Django>=1.8 no longer supports Python 3.3, and Django<1.8 does not support Python 3.5. As an example, django-confit can’t work with Django 1.7 and Python 3.5.
As a library¶
In most cases, you will use django-confit as a dependency of another project
(typically a Django project or a Django application).
In such a case, you should add django-confit
in your main project’s
requirements. Typically in setup.py
:
from setuptools import setup
setup(
install_requires=[
'django-confit',
#...
]
# ...
)
Then when you install your main project with your favorite package manager (like pip [2]), django-confit will automatically be installed.
Standalone¶
You can install django-confit with your favorite Python package manager. As an example with pip [2]:
pip install django-confit
Check¶
Check django-confit has been installed:
python -c "import django_confit;print(django_confit.__version__)"
You should get django_confit‘s version.
Notes & references
See also
[1] | https://www.python.org/ |
[2] | (1, 2) https://pypi.python.org/pypi/pip/ |
Configure¶
Once django-confit is installed, let’s configure it. This section describes directives related to django-confit in your Django settings.
CONFIT_SCHEMAS¶
A dictionary of <APP>: <SCHEMA>
where <APP>
is in
INSTALLED_APPS
and <SCHEMA>
is the Python dotted path to a Schema class
(or factory).
This configuration directive is mandatory only when you need configuration schemas that are not builtin django-confit. See also the list of builtin schemas.
validate_settings()
automatically loads schemas
from INSTALLED_APPS
setting. For each <APP> in INSTALLED_APPS
, it
looks for:
CONFIT_SCHEMAS[<APP>]
,django_confit.schemas.<APP>.ConfigurationSchema
.
Simple example from default settings in django-confit demo project [1]:
CONFIT_SCHEMAS = {
'django_confit_demo': 'django_confit_demo.settings_schemas'
'.DjangoConfitDemoConfigurationSchema'
}
Note
Early versions of django_confit tried to automatically load schemas, without having to register them:
settings_schemas.<APP>.ConfigurationSchema
in current package, to allow local overrides<APP>.settings_schemas.ConfigurationSchema
, to allow third-party applications to manage their own schema.django_confit.schemas.<APP>.ConfigurationSchema
to load django-confit‘s builtins.
The idea was nice. But it did not work. Because while trying to import
<APP>.settings_schemas.ConfigurationSchema
, if import <APP>
imports
Django stuff, then a circular import can occur. That is a pain to debug and
a pain to fix.
With a registry, we import only things that are expected to work.
Notes & references
[1] | https://github.com/benoitbryon/django-confit/blob/master/demo/django_confit_demo/default_settings.py |
Load configuration¶
As a Django user, you manage configuration as you like: as Python modules, as JSON or YAML files, as environment variables... Do whatever integrates best with your deployment workflow and your team.
As a non Django user (such as a member of the Ops team), you may appreciate putting configuration in configuration files or in environment variables. Not in Python code.
This document explains how django-confit can help Django users load configuration from various locations and formats.
load_mapping¶
-
django_confit.loaders.
load_mapping
(input, prefix='')¶ Convert mapping of {key: string} to {key: complex type}.
This function makes it possible (and easy) to load complex types from single-level key-value stores, such as environment variables or INI files.
Of course, both flat and nested mappings are supported:
>>> from django_confit import load_mapping >>> flat_mapping = {'DEBUG': 'True', 'SECRET_KEY': 'not a secret'} >>> output = load_mapping(flat_mapping) >>> output == flat_mapping True
>>> nested_mapping = {'DATABASES': {'USER': 'me', 'HOST': 'localhost'}} >>> output = load_mapping(nested_mapping) >>> output == nested_mapping True
Values can be complex types (sequences, mappings) using JSON or YAML. Keys using ”.json” or ”.yaml” suffix are automatically decoded:
>>> nested_mapping = { ... 'DATABASES.yaml': 'ENGINE: sqlite3', ... } >>> output = load_mapping(nested_mapping) >>> output['DATABASES'] == {'ENGINE': 'sqlite3'} True
You can use optional
prefix
argument to load only a subset of mapping:>>> mapping = {'YES_ONE': '1', 'NO_TWO': '2'} >>> load_mapping(mapping, prefix='YES_') {'ONE': '1'}
load_module¶
-
django_confit.loaders.
load_module
(module_path)¶ Return module’s globals as a dict.
>>> from django_confit import load_module >>> settings = load_module('django.conf.global_settings') >>> settings['DATABASES'] {}
It does not load “protected” and “private” attributes (those with underscores).
>>> '__name__' in settings False
load_file¶
-
django_confit.loaders.
load_file
(file_obj)¶ Return mapping from file object, using
name
attr to guess format.Supported file formats are JSON and YAML. The lowercase extension is used to guess the file type.
>>> from django_confit import load_file >>> from six.moves import StringIO >>> file_obj = StringIO('SOME_LIST: [a, b, c]') >>> file_obj.name = 'something.yaml' >>> load_file(file_obj) == { ... 'SOME_LIST': ['a', 'b', 'c'], ... } True
Validate schemas¶
As a Django user, once you have loaded configuration, you want to make sure it is valid. It is a must-have feature when the configuration comes from various locations. Or when it has been written by several users.
As a non Django user (such as a member of the Ops team, who deploys and configure services), you do not want to be prosecuted because you made a typo in configuration. You think applications should validate the configuration before they actually use it, and report errors with a readable output.
This section explains how django-confit helps Django users validate configuration.
validate_settings¶
-
django_confit.schemas.
validate_settings
(raw_settings)¶ Return cleaned settings using schemas collected from INSTALLED_APPS.
Low-level API¶
Use deserialize()
method of Colander schemas:
http://docs.pylonsproject.org/projects/colander/en/latest/basics.html#deserialization
Write configuration schemas¶
As a Django application developer, you want to expose configuration directives. You want to make sure your application is well configured. As an example, you appreciate to mark some directives as required, and to provide default values for optional ones. But you do not want to re-invent the wheel about validation.
As a Django user, you do not want to bother about application configuration
schemas. You just want it loaded when you register the application in
INSTALLED_APPS
.
This sections explains how to create with django-confit.
Create settings_schemas.ConfigurationSchema¶
See:
Simple example from custom schemas in django-confit demo project [3]:
"""Custom configuration schema(s)."""
import colander
class DjangoConfitDemoConfigurationSchema(colander.MappingSchema):
CONFITDEMO_HELLO = colander.SchemaNode(
colander.String(),
missing=colander.required,
default='',
)
Warning
In django-confit code, you will see some hacks around versions of applications. Yes, they are hacks. They are necessary because from the django-confit point of view, we do not know which version of the application is actually installed. So we have to write conditionals in order to support them as well as possible... But it is a true pain.
Just imagine the schema lives within the third-party application itself... As a consequence:
- one schema is enough. No need to manage concurrent schemas. The one for the current version is enough.
- the schema is updated along the application. It is part of the application’s release process.
- as an user, you do not have to find or write a schema for the application. There is an official one! You install the app, you get the schema!
Register your schema¶
In a project or a third-party application, register schemas in
settings.CONFIT_SCHEMAS
. See Configure.
In django-confit itself, make sure the schema can be loaded with
django_confit.schemas.<APP>.ConfigurationSchema
.
Notes & references
[1] | http://docs.pylonsproject.org/projects/colander |
[2] | https://github.com/benoitbryon/django-confit/tree/master/django_confit/schemas |
[3] | https://github.com/benoitbryon/django-confit/blob/master/demo/django_confit_demo/settings_schemas.py |
Builtin schemas¶
Here is the list of configuration schemas implemented as part of django-confit. They are schemas that will automatically be loaded.
- Django 1.5 and Django 1.6 are supported
- django-debug-toolbar
- django-nose
- django-pimpmytheme
- django-pipeline
- raven
Note
Maintaining schemas for third-party applications is hard! Any help would be welcome, as an example:
- report bugs ;
- improve existing schemas (smarter validation) ;
- watch upgrades, support several versions.
If you are a library developer and you like django-confit, the best thing you can do is to distribute your app’s schema inside your app! So that you maintain the schema along with your application, and django-confit does not have to deal with multiple versions (given a version of your app, there is only one valid schema for your app, and it lives within your app).
Demo project¶
Demo folder in project’s repository [1] contains a Django project to illustrate django-confit usage.
Notes & references
[1] | https://github.com/benoitbryon/django-confit/tree/master/demo/ |
About django-confit¶
This section is about django-confit project itself.
Vision¶
django-confit has been created with the following concepts in mind:
- Local configuration should not live in code. Altering
sys.path
or putting local settings module within code is not a good practice. Moreover, code should not vary from one environment to another. - Environment variables are not enough to manage settings. You can load settings from environment variables, but you can also do it from files. Files have many benefits (backup, version control...).
- Settings imported from external locations should (always) be validated.
- Application should expose the full list of settings they use.
- Documentation about applications settings should be auto-generated.
License¶
Copyright (c) 2014, Benoît Bryon. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
- Neither the name of django-confit nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Authors & contributors¶
Maintainer: Benoît Bryon <benoit@marmelune.net>
Developers: https://github.com/benoitbryon/django-confit/graphs/contributors
Changelog¶
This document describes changes between past releases. For information about future releases, check milestones [1] and Vision.
0.5 (2015-12-15)¶
- Features #3, #31, #32 and bug #30 - Introduced support for Django 1.7 to 1.9 and Python 3.5.
0.4 (2015-12-11)¶
- Bug #28 - Configuration loaders work with unsupported versions, whereas validation doesn’t (was requiring supported versions of Django).
0.3 (2015-03-23)¶
Upgrades: Django and Sphinx.
- Feature #26 - Added support for Django up to versions 1.5.12 and 1.6.11.
- Bug #25 - Fixed support of latest Sphinx version.
0.2 (2014-06-30)¶
- Feature #13 - Extended support for Django from 1.5 to 1.5.8 and from 1.6 to 1.6.5.
- Feature #16 - Added schema for django-pimpmytheme settings.
- Bug #15 - DjangoSettingsSchema.TEMPLATE_LOADERS now accepts either strings or tuples (was “accepts only strings”), as specified in Django’s settings reference documentation.
0.1 (2014-04-21)¶
Proof of concept.
- Introduced configuration schemas, using Colander.
- Implemented configuration schema for Django 1.5.5 and Django 1.6.2.
- Implemented configuration schema for some third-party libraries: django-pipeline, django-debug-toolbar, django-nose and raven.
- Introduced configuration loading utilities.
- Feature #7 - Code repository contains a demo project, used for documentation and tests.
- Feature #9 - Demo project shows how to register a custom configuration schema.
- Feature #10 - Validate_settings() emits a warning if some directives used in raw settings input are not in cleaned settings output. Helps figure out which schemas are missing.
Notes & references
[1] | https://github.com/benoitbryon/django-confit/milestones |
Contributing to django-confit¶
This document provides guidelines for people who want to contribute to the project.
Create tickets¶
Please use django-confit bugtracker [1] before starting some work:
- check if the bug or feature request has already been filed. It may have been answered too!
- else create a new ticket.
- if you plan to contribute, tell us, so that we are given an opportunity to give feedback as soon as possible.
- Then, in your commit messages, reference the ticket with some
refs #TICKET-ID
syntax.
Use topic branches¶
- Work in branches.
- Prefix your branch with the ticket ID corresponding to the issue. As an
example, if you are working on ticket #23 which is about contribute
documentation, name your branch like
23-contribute-doc
. - If you work in a development branch and want to refresh it with changes from master, please rebase [2] or merge-based rebase [3], i.e. do not merge master.
Fork, clone¶
Clone django-confit repository (adapt to use your own fork):
git clone git@github.com:benoitbryon/django-confit.git
cd django-confit/
Usual actions¶
The Makefile is the reference card for usual actions in development environment:
- Install development toolkit with pip [4]:
make develop
. - Run tests with tox [5]:
make test
. - Build documentation:
make documentation
. It builds Sphinx [6] documentation in var/docs/html/index.html. - Release django-confit project with zest.releaser [7]:
make release
. - Cleanup local repository:
make clean
,make distclean
andmake maintainer-clean
.
See also make help
.
Notes & references
[1] | https://github.com/benoitbryon/django-confit/issues |
[2] | http://git-scm.com/book/en/v2/Git-Branching-Rebasing |
[3] | http://tech.novapost.fr/psycho-rebasing-en.html |
[4] | https://pypi.python.org/pypi/pip/ |
[5] | https://pypi.python.org/pypi/tox/ |
[6] | https://pypi.python.org/pypi/Sphinx/ |
[7] | https://pypi.python.org/pypi/zest.releaser/ |