Python and Django functions, classes and settings re-used across different OpenWISP modules, stored here with the aim of avoiding code duplication and ease maintenance.
Don't repeat yourself!
- Configurable admin theme
- Configurable navigation menu
- OpenAPI / Swagger documentation
- Model utilities
- Admin utilities
- Code utilities
- REST API utilities
- Test utilities
- Quality assurance checks
Table of Contents:
- Current features
- Install stable version from pypi
- Install development version
- Using the
admin_theme - Main navigation menu
- Model utilities
- Admin utilities
- Code utilities
- REST API utilities
- Test utilities
- Quality Assurance Checks
- Settings
- Installing for development
- Contributing
- Support
- Changelog
- License
Install from pypi:
pip install openwisp-utils
# install optional dependencies for REST framework
pip install openwisp-utils[rest]
# install optional dependencies for tests (flake8, black and isort)
pip install openwisp-utils[qa]
# or install everything
pip install openwisp-utils[rest,qa]Install tarball:
pip install https://github.com/openwisp/openwisp-utils/tarball/masterAlternatively you can install via pip using git:
pip install -e git+git://github.com/openwisp/openwisp-utils#egg=openwisp-utilsThe admin theme requires Django >= 2.2..
Add openwisp_utils.admin_theme to INSTALLED_APPS in settings.py:
INSTALLED_APPS = [
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'openwisp_utils.admin_theme', # <----- add this
'django.contrib.sites',
# admin
'django.contrib.admin',
]Add the list of all packages extended to EXTENDED_APPS in settings.py.
For example, if you've extended django_x509:
EXTENDED_APPS = ['django_x509']This is a static finder which looks for static files in the static
directory of the apps listed in settings.EXTENDED_APPS.
Add openwisp_utils.staticfiles.DependencyFinder to STATICFILES_FINDERS
in settings.py.
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'openwisp_utils.staticfiles.DependencyFinder', # <----- add this
]This is a template loader which looks for templates in the templates
directory of the apps listed in settings.EXTENDED_APPS.
Add openwisp_utils.loaders.DependencyLoader to
template loaders in settings.py as shown below.
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'OPTIONS': {
'loaders': [
# ... other loaders ...
'openwisp_utils.loaders.DependencyLoader', # <----- add this
],
'context_processors': [
# ... omitted ...
],
},
},
]Add openwisp_utils.admin_theme.context_processor.admin_theme_settings to
template context_processors in settings.py as shown below.
This will allow to set OPENWISP_ADMIN_THEME_LINKS
and OPENWISP_ADMIN_THEME_JS settings
to provide CSS and JS files to customise admin theme.
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'OPTIONS': {
'loaders': [
# ... omitted ...
],
'context_processors': [
# ... other context processors ...
'openwisp_utils.admin_theme.context_processor.admin_theme_settings' # <----- add this
],
},
},
]Note
You will have to deploy these static files on your own.
In order to make django able to find and load these files
you may want to use the STATICFILES_DIR setting in settings.py.
You can learn more in the Django documentation.
Add openwisp_utils.admin_theme.context_processor.menu_items to
template context_processors in settings.py as shown below.
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'OPTIONS': {
'loaders': [
# ... omitted ...
],
'context_processors': [
# ... other context processors ...
'openwisp_utils.admin_theme.context_processor.menu_items' # <----- add this
],
},
},
]If you need to define custom menu items, see: OPENWISP_ADMIN_MENU_ITEMS.
Users will only be able to see menu items for objects they have permission to either view or edit.
Model class which provides a UUID4 primary key.
Model class inheriting UUIDModel which provides two additional fields:
createdmodified
Which use respectively AutoCreatedField, AutoLastModifiedField from model_utils.fields
(self-updating fields providing the creation date-time and the last modified date-time).
A model field whic provides a random key or token, widely used across openwisp modules.
Admin mixin which adds two readonly fields created and modified.
This is an admin mixin for models inheriting TimeStampedEditableModel
which adds the fields created and modified to the database.
A read-only ModelAdmin base class.
Will include the id field by default, which can be excluded by supplying
the exclude attribute, eg:
from openwisp_utils.admin import ReadOnlyAdmin
class PostAuthReadOnlyAdmin(ReadOnlyAdmin):
exclude = ['id']A mixin designed for inline items and model forms, ensures the item is created even if the default values are unchanged.
Without this, when creating new objects, inline items won't be saved unless users change the default values.
An admin class that provides the UUID of the object as a read-only input field (to make it easy and quick to copy/paste).
An admin class that provides an URL as a read-only input field (to make it easy and quick to copy/paste).
Generates an random string of 32 characters.
Takes input of a list of models name and settings_menu name and adds them to the side navigation bar in the admin dashboard interface.
Usage:
from openwisp_utils.utils import register_menu_items
from openwisp_utils.api.apps import ApiAppConfig
class YourAwesomeAppConfig(ApiAppConfig):
def ready(self, *args, **kwargs):
super().ready(*args, **kwargs)
items = [{'model': 'your_project.your_model_name'}]
# register_menu_items(items[, name_menu=YOUR_SETTINGS_MENU_NAME])
register_menu_items(items, name_menu='OPENWISP_DEFAULT_ADMIN_MENU_ITEMS')Returns a new dict which is the result of the merge of the two dictionaries,
all elements are deep-copied to avoid modifying the original data structures.
Usage:
from openwisp_utils.utils import deep_merge_dicts
mergd_dict = deep_merge_dicts(dict1, dict2)If the program is being executed during automated tests the value supplied in
the test argument will be returned, otherwise the one supplied in the
value argument is returned.
from openwisp_utils.utils import default_or_test
THROTTLE_RATE = getattr(
settings,
'THROTTLE_RATE',
default_or_test(value='20/day', test=None),
)default colors: ['white_bold', 'green_bold', 'yellow_bold', 'red_bold']
If you want to print a string in Red Bold, you can do it as below.
from openwisp_utils.utils import print_color
print_color('This is the printed in Red Bold', color_name='red_bold')You may also provide the end arguement similar to built-in print method.
A model serializer which calls the model instance full_clean().
If you're creating an OpenWISP module which provides a REST API built with Django REST Framework, chances is that you may need to define some default settings to control its throttling or other aspects.
Here's how to easily do it:
from django.conf import settings
from django.utils.translation import ugettext_lazy as _
from openwisp_utils.api.apps import ApiAppConfig
class MyModuleConfig(ApiAppConfig):
name = 'my_openwisp_module'
label = 'my_module'
verbose_name = _('My OpenWISP Module')
# assumes API is enabled by default
API_ENABLED = getattr(settings, 'MY_OPENWISP_MODULE_API_ENABLED', True)
# set throttling rates for your module here
REST_FRAMEWORK_SETTINGS = {
'DEFAULT_THROTTLE_RATES': {'my_module': '400/hour'},
}Every openwisp module which has an API should use this class to configure its own default settings, which will be merged with the settings of the other modules.
This method can be used to mock a signal call inorder to easily verify that the signal has been called.
Usage example as a context-manager:
from openwisp_utils.tests import catch_signal
with catch_signal(openwisp_signal) as handler:
model_instance.trigger_signal()
handler.assert_called_once_with(
arg1='value1',
arg2='value2',
sender=ModelName,
signal=openwisp_signal,
)
This class extends the default test runner provided by Django and logs the time spent by each test, making it easier to spot slow tests by highlighting time taken by it in yellow (time shall be highlighted in red if it crosses the second threshold).
By default tests are considered slow if they take more than 0.3 seconds but you can control this with OPENWISP_SLOW_TEST_THRESHOLD.
In order to switch to this test runner you have set the following in your settings.py:
TEST_RUNNER = 'openwisp_utils.tests.TimeLoggingTestRunner'This decorator can be used to capture standard output produced by tests, either to silence it or to write assertions.
Example usage:
from openwisp_utils.tests import capture_stdout
@capture_stdout()
def test_something(self):
function_generating_output() # pseudo code
@capture_stdout()
def test_something_again(self, captured_ouput):
# pseudo code
function_generating_output()
# now you can create assertions on the captured output
self.assertIn('expected stdout', captured_ouput.getvalue())
# if there are more than one assertions, clear the captured output first
captured_error.truncate(0)
captured_error.seek(0)
# you can create new assertion now
self.assertIn('another output', captured_ouput.getvalue())Notes:
- If assertions need to be made on the captured output, an additional argument
(in the example above is named
captured_output) can be passed as an argument to the decorated test method, alternatively it can be omitted. - A
StingIOinstance is used for capturing output by default but if needed it's possible to pass a customStringIOinstance to the decorator function.
Equivalent to capture_stdout, but for standard error.
Example usage:
from openwisp_utils.tests import capture_stderr
@capture_stderr()
def test_error(self):
function_generating_error() # pseudo code
@capture_stderr()
def test_error_again(self, captured_error):
# pseudo code
function_generating_error()
# now you can create assertions on captured error
self.assertIn('expected error', captured_error.getvalue())
# if there are more than one assertions, clear the captured error first
captured_error.truncate(0)
captured_error.seek(0)
# you can create new assertion now
self.assertIn('another expected error', captured_error.getvalue())Equivalent to capture_stdout and capture_stderr, but captures both types of
output (standard output and standard error).
Example usage:
from openwisp_utils.tests import capture_any_output
@capture_any_output()
def test_something_out(self):
function_generating_output() # pseudo code
@capture_any_output()
def test_out_again(self, captured_output, captured_error):
# pseudo code
function_generating_output_and_errors()
# now you can create assertions on captured error
self.assertIn('expected stdout', captured_output.getvalue())
self.assertIn('expected stderr', captured_error.getvalue())This package contains some common QA checks that are used in the automated builds of different OpenWISP modules.
Shell script to automatically format Python code. It runs isort and black.
Shell script to run the following quality assurance checks:
- checkmigrations
- checkcommit
- checkendline
- checkpendingmigrations
- checkrst
flake8- Python code linterisort- Sorts python imports alphabetically, and seperated into sectionsblack- Formats python code using a common standard
If a check requires a flag, it can be passed forward in the same way.
Usage example:
openwisp-qa-check --migration-path <path> --message <commit-message>
Any unneeded checks can be skipped by passing --skip-<check-name>
Usage example:
openwisp-qa-check --skip-isort
You can do multiple checkmigrations by passing the arguments with space-delimited string.
For example, this multiple checkmigrations:
checkmigrations --migrations-to-ignore 3 \
--migration-path ./openwisp_users/migrations/ || exit 1
checkmigrations --migrations-to-ignore 2 \
--migration-path ./tests/testapp/migrations/ || exit 1
Can be changed with:
openwisp-qa-check --migrations-to-ignore "3 2" \
--migration-path "./openwisp_users/migrations/ ./tests/testapp/migrations/"
Ensures the latest migrations created have a human readable name.
We want to avoid having many migrations named like 0003_auto_20150410_3242.py.
This way we can reconstruct the evolution of our database schemas faster, with less efforts and hence less costs.
Usage example:
checkmigrations --migration-path ./django_freeradius/migrations/
Ensures the last commit message follows our commit message style guidelines.
We want to keep the commit log readable, consistent and easy to scan in order to make it easy to analyze the history of our modules, which is also a very important activity when performing maintenance.
Usage example:
checkcommit --message "$(git log --format=%B -n 1)"
If, for some reason, you wish to skip this QA check for a specific commit message
you can add #noqa to the end of your commit message.
Usage example:
[qa] Improved #20 Simulation of a special unplanned case #noqa
Ensures that a blank line is kept at the end of each file.
Ensures there django migrations are up to date and no new migrations need to be created.
It accepts an optional --migration-module flag indicating the django app
name that should be passed to ./manage.py makemigrations, eg:
./manage.py makemigrations $MIGRATION_MODULE.
Checks the syntax of all ReStructuredText files to ensure they can be published on pypi or using python-sphinx.
default: openwisp_utils.admin_theme.admin.OpenwispAdminSite
If you need to use a customized admin site class, you can use this setting.
default: OpenWISP Admin
Title value used in the <title> HTML tag of the admin site.
default: OpenWISP
Heading text used in the main <h1> HTML tag (the logo) of the admin site.
default: Network administration
Title shown to users in the index page of the admin site.
default: []
Allows to pass a custom list of menu items to display in the admin menu.
If passed, overrides the default menu which is built by different openwisp modules.
The list should not include "home", "change password" and "log out", because those are automatically added and cannot be removed.
Example usage:
OPENWISP_ADMIN_MENU_ITEMS = [
{'model': 'config.Device'},
{'model': 'config.Template'},
{'model': 'openwisp_users.User'},
{
'model': 'openwisp_radius.Accounting',
'label': 'Radius sessions' # custom label
}
]default: []
Note: this setting requires the admin_theme_settings context processor in order to work.
Allows to override the default CSS and favicon, as well as add extra <link> HTML elements if needed.
This setting overrides the default theme, you can reuse the default CSS or replace it entirely.
The following example shows how to keep using the default CSS, supply an additional CSS and replace the favicon.
Example usage:
OPENWISP_ADMIN_THEME_LINKS = [
{'type': 'text/css', 'href': '/static/admin/css/openwisp.css', 'rel': 'stylesheet', 'media': 'all'},
{'type': 'text/css', 'href': '/static/admin/css/custom-theme.css', 'rel': 'stylesheet', 'media': 'all'},
{'type': 'image/x-icon', 'href': '/static/favicon.png', 'rel': 'icon'}
]default: []
Allows to pass a list of strings representing URLs of custom JS files to load.
Example usage:
OPENWISP_ADMIN_THEME_JS = [
'/static/custom-admin-theme.js',
]default: False
When True, enables Django user links on the admin site.
i.e. (USER NAME/ VIEW SITE / CHANGE PASSWORD / LOG OUT).
These links are already shown in the main navigation menu and for this reason are hidden by default.
default: True
Whether the OpenAPI documentation is enabled.
When enabled, you can view the available documentation using the
Swagger endpoint at /api/v1/docs/.
You also need to add the following url to your project urls.py:
urlpatterns += [
url(r'^api/v1/', include('openwisp_utils.api.urls')),
]default:
{
'title': 'OpenWISP API',
'default_version': 'v1',
'description': 'OpenWISP REST API',
}Define OpenAPI general information.
NOTE: This setting requires OPENWISP_API_DOCS = True to take effect.
For more information about optional parameters check the drf-yasg documentation.
default: [0.3, 1] (seconds)
It can be used to change the thresholds used by TimeLoggingTestRunner to detect slow tests (0.3s by default) and highlight the slowest ones (1s by default) amongst them.
Install sqlite:
sudo apt-get install sqlite3 libsqlite3-devInstall your forked repo:
git clone git://github.com/<your_fork>/openwisp-utils
cd openwisp-utils/
pip install -e .[qa,rest]Install test requirements:
pip install -r requirements-test.txtCreate database:
cd tests/
./manage.py migrate
./manage.py createsuperuserRun development server:
cd tests/
./manage.py runserverYou can access the admin interface of the test project at http://127.0.0.1:8000/admin/.
Run tests with:
./runtests.py --parallelPlease refer to the OpenWISP contributing guidelines.
See OpenWISP Support Channels.
See CHANGES.
See LICENSE.