2 - Semaphore NetBox
Deploy, operate, and upgrade NetBox from within a purpose built Ansible Semaphore management wrapper. Get started in less than 15 minutes.
- Deploy & Upgrade NetBox
- NetBox Quick Start Installation Guide
- Install NetBox
- Upgrade NetBox
- Remove NetBox
- NetBox CSRF Settings
- Manage & Operate NetBox
- Deploy NetBox Community Plugins
- Deploy Netos Enterprise
Deploy & Upgrade NetBox
NetBox Quick Start Installation Guide
Deploy a NetBox Dev/Test Environment in 20 Minutes
To get NetBox up and running in the fastest possible way, follow this guide. This framework will allow you to install, backup, remove, upgrade, customize, and deploy NetBox plugins, all in minutes.
The blue info boxes flag What You've Missed, which are stages you should consider in a production, or in "more than just a NetBox lab" environment.
5 Minute Demo
Installation Guide
1. Install Ubuntu Server
You're going to need to get a server with Ubuntu 24.04 LTS up and running. Download an image like ubuntu-24.04-live-server-amd64.iso
or spin up an instance on your favorite cloud provider, like AWS.
2. Install Semaphore and the Netos NetOps Project
You can follow the full Install Semaphore Guide, however, in summary, once you install Ubuntu, paste this command to pull the repo and run the install script, and you shouldn't need to do anything else:
sudo wget https://raw.githubusercontent.com/netos-networks/netos-netops/refs/heads/main/easy-deploy.sh && sudo chmod +x easy-deploy.sh && sudo ./easy-deploy.sh
The Key Semaphore Principles guide explains more about Semaphore.
Check the Users, Accounts, and Tokens guide for more information about credentials and variables.
3. Configure Semaphore
These steps are optional, so you can skip this for now.
Change the DOMAIN
and SEMAPHORE_HOSTNAME
variables in the NetBox Global Settings Semaphore environment. Follow the Deploying NGINX & Certificates guide.
Deploy self-signed certificates, so you don't get "your connection isn't private" browser warnings. Follow the Deploying NGINX & Certificates guide.
Enable Semaphore Backups.
4. Deploy NGINX and Self-Signed Cert for NetBox
You need to deploy NGINX to connect to NetBox. To fast track, you can just create a self-signed certificate. The default hostname will be netbox.netos.dev
.
Change the DOMAIN
and NETBOX_HOSTNAME
variables in the NetBox Global Settings Semaphore environment. Follow the Deploying NGINX & Certificates guide.
Deploy self-signed certificates, so you don't get "Your connection isn't private" browser warnings. Follow the Deploying NGINX & Certificates guide.
5. Install NetBox
Browse to the NETBOX DEPLOY view in Semaphore to start the installation. The full NetBox Installation Guide is here.
If you want to install a specific version, set it here, matching the numeric version from Releases · netbox-community/netbox (github.com). Don't add the v
, i.e. v4.1.2
would be 4.1.2
.
Configure scheduled daily NetBox Backups or set your own custom frequency.
Upgrade NetBox. For example, maybe you've used the Restore process to restore to an older version and want to upgrade to a newer version.
Enable the NetBox Maintenance Playbook on a daily schedule as per the developer's guidance.
6. Connect to NetBox
Once the playbook is complete, you should be able to connect using HTTPS and the IP address to NetBox. Check the Users, Accounts, and Tokens guide for full details on account and passwords. The default credentials are as follows:
- Username:
admin
- Password:
ohp8toef7Jee
If you experience any problems, you can run the Health Check Playbook to quickly diagnose common issues. For example, check the logs to catch the reason why NetBox failed to start.
7. Install NetBox Plugins
You can Install and Upgrade popular NetBox plugins from within Semaphore, and also Disable NetBox Plugins. This feature is a work-in-progress, and probably will be for a while...!
If things go wrong in your test environment, you can quickly Wipe NetBox and start again.
Install NetBox
Introduction
This guide details how to deploy a new instance of NetBox. For guidance on upgrading NetBox, please refer to this Upgrade NetBox.
NetBox is installed from the public GitHub repository according to the version set in Semaphore (or proposed during an upgrade). Netos has not forked NetBox. Any customizations made by Netos are optional and controlled from within the NETOS view in the NetBox project in Semaphore.
Configuration Guide
1. Configure the Semaphore Environment
You can set the SUPER_USER
username, email, and password in the NetBox Global Settings environment settings, or just use the defaults for testing:
Please don't change the install directory from /opt/netbox
as it is hard coded in configuration files and variables.
There is a bug in Ansible Semaphore that requires you to delete and recreate the secret value. Save doesn't work.
2. Deploy NGINX and the Cert
Refer to the following guide Deploying NGINX & Certificates for a detailed guide on deploying NGINX and certs. Once you've deployed, you'll see a screen like this and should be able to connect to a URL like https://uk-lab-nb05-netbox.core.netos.io.
You need to deploy NGINX because it configures the webserver to mount alias /opt/netbox/netbox/static/;
correctly to avoid the Static Media Failure error.
3. Install NetBox
We have included pre-defined NetBox templates (which will be updated from time-to-time), for example, you can just click play on the Semaphore template to deploy NetBox v3.7.8
or v4.0.11
.
If you want to install a custom version of NetBox, you can do so by editing the tags in the Semaphore Template. For example, change --tags=X.X.X
to --tags=3.6.5
. This feature is useful for a migration, for example, if you wanted to migrate from a legacy NetBox instance, you could take a backup and Restore, then Upgrade.
Note that the netbox-deploy.yml
Ansible playbook has logic that matches the tags referenced in Semaphore. The tags must be set for this playbook to execute. The tags are used to set variables which are passed to the netbox-deploy
Ansible role.
- hosts: 127.0.0.1
connection: local
become: yes
vars_files:
- ./vars/env-netbox-settings.yml
tasks:
- name: Set NetBox release version to v4.1.1
set_fact:
NETBOX_RELEASE: 4.1.1
tags: 4.1.0
- name: Set NetBox release version to v4.1.0
set_fact:
NETBOX_RELEASE: 4.1.0
tags: 4.1.0
4. Connect to NetBox
Once you've deployed NetBox you can connect to the application based on the NETBOX_HOSTNAME
and DOMAIN
variables configured in the NetBox Global Setting environment.
Check the Users, Accounts, and Tokens guide for full details on account and passwords. The default credentials are as follows:
- Username:
admin
- Password:
ohp8toef7Jee
a. NetBox 3.7.8
b. NetBox 4.0.11
Thoughts About Docker
We (Netos) used and tested the NetBox Docker image for over 6 months. When we started to load test to simulate a large production enterprise environment, we experienced stability issues, especially when hitting the API hard, and also had to make a number of changes to the Docker NGINX configuration.
Others may have had success; however, we were unable to achieve stability (outside of development environments) with NetBox in Docker. The playbooks here install NetBox natively on the Linux operating system, and we've had no issues whatsoever.
Upgrade NetBox
Introduction
This guide will walk you through the process of upgrading NetBox, including a suggested pre-staging strategy. The approach we take to upgrading NetBox is as follows:
- Upgrade to the final patch of the current release, e.g. from
3.7.4
to3.7.8
- Upgrade to the final patch version of the next major release, e.g. from
3.7.8
to4.0.11
The official NetBox upgrade strategy can be found here: netbox/docs/installation/upgrading.md at develop · netbox-community/netbox (github.com)
This feature is a work-in-progress. There are many other considerations to take into account, such as plugin configurations and customizations, as well as sequencing.
Upgrade Playbooks and Templates
Within the NetBox Deploy view in Semaphore, you'll see these playbooks.
- NetBox Upgrade Check Path - checks the current running version with the mapping table to suggest the next version.
- NetBox Upgrade to Next Suitable Version - this will take the result from the "Check Path" playbook and perform an upgrade.
Checking the Upgrade Path
The following screenshot shows that NetBox 3.7.8
has been detected, and based on the upgrade path mapping table, the next suggested version is 4.0.11
.
When the playbook runs it checks a mapping table that is structured like this. The logic is:
- If
current_version
andpatch
are the different, then upgrade to the nextpatch
in the same release - If
current_version
andpatch
are the same, then upgrade tonext_major
netbox_upgrade_paths:
3.7.0:
patch: 3.7.8
next_major: 4.0.11
3.7.1:
patch: 3.7.8
next_major: 4.0.11
3.7.2:
patch: 3.7.8
next_major: 4.0.11
3.7.3:
patch: 3.7.8
next_major: 4.0.11
3.7.4:
patch: 3.7.8
next_major: 4.0.11
3.7.5:
patch: 3.7.8
next_major: 4.0.11
3.7.6:
patch: 3.7.8
next_major: 4.0.11
3.7.7:
patch: 3.7.8
next_major: 4.0.11
3.7.8:
patch: 3.7.8
next_major: 4.0.11
4.0.0:
patch: 4.0.11
next_major: 4.1.0
In a production environment you should not upgrade past the final release in the last release. For example, once 4.0.0
was released, 3.7.8
was considered the final stable release. Once 4.1.0
was released, we would consider 4.0.11
stable, and so on.
Running the Upgrade
Here you can see that the suggested upgrade version from the mapping table has been passed to the netbox-deploy
playbook to facilitate the upgrade.
You must take a full backup before starting the upgrade. If possible, we would suggest stopping all services on the server and taking a snapshot (disk image) before proceeding.
Ensure you check compatibility with any plugins that you have installed and have a clear upgrade path for them.
Ensure you check breaking changes in the NetBox Release notes that could impact you, for example, if you have integrated your systems using the NetBox Rest API, endpoints do change from time-to-time, which could break your integrations.
Have a clear test plan.
Pre-Stage Your Upgrades
It's fast and simple to spin up a staging NetBox instance for testing. A "belt and braces" approach to an upgrade is as follows:
- Install Semaphore
- Install NetBox, ensuring that you match the NetBox version
- Backup your current production NetBox instance
- Restore to the new staging NetBox instance
- Test the upgrade process in safety!
Remove NetBox
Introduction
In a test or lab environment it is useful to be able to quickly wipe NetBox so you can re-install and restore.
This process will drop the NetBox database from PostgreSQL and delete the /opt/netbox
folder. It's probably not a good idea to run this on production system. In fact, delete the template on production systems!
NetBox CSRF Settings
Introduction
This playbook netbox-csrf
will add your domains in the following format to the NetBox settings.py
file based on the DOMAIN
variable in Semaphore.
http://*.yourdomain.com
https://*.yourdomain.com
http://yourdomain.com
https://yourdomain.com
When you run the playbook, it will look like this:
Use Case
If your server name is uk-lab-nb01.core.netos.io
, and you configured a DNS CNAME to give it a more friendly name like demo2.netos.io
via a reverse proxy, then you would see this error and won't be able to login:
To fix this, change the domain to in the NetBox Global Settings:
Which will create a settings.py
file entry like this and fix the error:
CSRF_TRUSTED_ORIGINS = getattr(configuration, 'CSRF_TRUSTED_ORIGINS', ['http://netos.io', 'https://netos.io', 'http://*.netos.io', 'https://*.netos.io'])
Manage & Operate NetBox
Backup, restore, and perform health checks and maintenance on your NetBox instance
NetBox Backup Guide
Introduction
The NetBox backup process provides regular secure backups. In summary the Ansible playbook does the following:
- Manages retention based on your preferences, e.g. deletes local backups older than 14 days old.
- Backs up and encrypts the media directory.
- Backs up and encrypts the
netbox
PostgreSQL database. - Captures the current state of NetBox from the
/api/status
endpoint and saves the output in a JSON file that will be compared agains the target state in a restore event. - Optionally SFTP's the three backup files to a secure remote SFTP server.
The NetBox process is stopped during the backup to ensure no changes are made to the database during the backup, and to ensure data integrity. This is typically for less than a minute, but take this into account if you anything connecting to the NetBox, e.g. automation tools. Netos Pod (Airflow) has a retry mechanism that handles broken connectivity to NetBox.
Cron Scheduling
The cron scheduler in Semaphore can be configured to backup the database at regular intervals, for example, at 02:30 every day.
Note that there is a bug in Ansible Semaphore UI that causes the same task to run many times. The solution is to toggle the "Show cron format' button and use UNIX formatting like here.
On the server, you can see the daily 02:00 encrypted backups in the red boxes. Below that, you can see an example of the JSON status that was captured when the backup was executed, so you know the exact version of NetBox and installed plugins for a restore.
Backup File Rotation
You can set the retention period for backup files stored in /netos/backups/netbox
in the NetBox Backup Settings Semaphore environment.
Ensure you use the exact values of days
or weeks
, i.e. no capitals.
Remote SFTP
To enable remote SFTP, change the No
value in the SFTP_ENABLED
variable to Yes
, and set the SFTP_HOST/USER/PASS
values accordingly.
Restoring from Backup
The NetBox Restore Process is one way to restore. If you want to manually decrypt the enc
files on your local workstation, use the following commands.
openssl enc -aes-256-cbc -d -in BACKUP_FILE.enc -out OUTPUT.sql.gz/.tar.gz
You will need the password set in the NetBox Backup Settings Semaphore environment variable ENCRYPTION_KEY
. For example, by running this command and entering the password, we decrypt the NetBox media directory:
openssl enc -aes-256-cbc -d -in netbox_media_09_25_2024_02_00.tar.gz.enc -out netbox_media_09_25_2024_02_00.tar.gz
NetBox Development Snapshots
Use Backups for Labs
The Netos team use the tools outlined in these guides to deploy, restore, develop, and manage dozens of NetBox development environments, typically with many changes and deployments each day.
The original backup below used the standard timestamp format mm_dd_yyyy_HH_MM
. However, we renamed it so we can snapshot different useful development environments pre-populated with data. We can then have many backups and images deployed to different NetBox instances on Semaphore to test and stage.
As long as you only replace the timestamp in the filename with some text, you can set this in the RESTORE_FILE_STAMP
, variable in the NetBox Backup Settings Semaphore environment. Here are some examples:
Before | After |
netbox_db_10_01_2024_15_24.sql.gz.enc |
netbox_db_01_Data_Feeds_and_Excel.sql.gz.enc |
netbox_media_10_01_2024_15_24.tar.gz.enc |
netbox_media_01_Data_Feeds_and_Excel.tar.gz.enc |
netbox_status_10_01_2024_15_24.json |
netbox_status_01_Data_Feeds_and_Excel.json |
Additionally, during the backup process the generated timestamp is displayed, for example; 10_01_2024_18_35
. You could use this as per the screenshot above to quickly roll back.
NetBox Restoration Guide
Introduction
The restore process works by taking a backup generated by the NetBox Backup, decrypting it, and restoring it. Check the NetBox Development Snapshots section in the backup guide for an approach on using this feature for dev/test.
The ENCRYPTION_KEY
value in the NetBox Backup Settings Semaphore environment is used in both the backup and restore process. Be sure to keep track as if you change it on the target system, all future backups will use the new key.
Configure the Restore
The NetBox backup process will create three timestamped (mm_dd_yyyy_HH_MM
) files like these. The enc
files are encrypted using the ENCRYPTION_KEY
variable.
netbox_db_03_18_2024_01_00.sql.gz.enc
netbox_media_03_18_2024_01_00.tar.gz.enc
netbox_status_03_18_2024_01_00.json
You can manually decrypt the files using the command openssl enc -aes-256-cbc -d -in BACKUP_FILE.enc -out OUTPUT.sql.gz.tar.gz
The restore file prefix is set in the NetBox Backup Settings environment and is the timestamp used for all three files. Set this to match the timestamp of the files that you place in the /netos/netbox/backups
directory.
Plugin Version Validation Logic
By default, when the backup runs it will check the NetBox version and plugin versions of the target system by comparing the output of the /api/status
endpoint collected from the backup with the version of the target system.
This is an example of the relevant JSON:
"netbox-version": "3.7.8",
"plugins": {
"netos": "1.4.6",
"netos_fabric": "1.3.3",
"netos_model_builder": "1.2.37",
"netos_reporting": "1.3.16"
}
In the event of a mismatch, you will see a warning like this that details the discrepancies between the backup and target system.
To disable this check, change BYPASS_PLUGIN_CHECK
to Yes
. Disabling may be acceptable when restoring to a slightly different target system, for example, where the Netos Insights plugin has increased from minor version 1.2.24
to 1.2.26
.
When BYPASS_PLUGIN_CHECK
is set to Yes
the restore process will look like this:
Any NetBox Open Source Plugins or Netos Enterprise NetBox Plugins installed using Semaphore will be named according to their version and saved in /netos/working-dir/netbox-plugins
and /netos/working-dir/netos-plugins
. The /netos/working-dir
directory is backed up each day, assuming you enable the schedule in Semaphore.
NetBox Health Checks & Maintenance
Introduction
In order to keep your NetBox instance operating smoothly you can run housekeeping and maintenance scripts.
NetBox Housekeeping Script
NetBox includes a housekeeping
management command that should be run nightly. This command handles:
- Clearing expired authentication sessions from the database
- Deleting changelog records older than the configured retention time
- Deleting job result records older than the configured retention time
- Check for new NetBox releases (if
RELEASE_CHECK_URL
is set)
The official documentation is here netbox/docs/administration/housekeeping.md at develop · netbox-community/netbox (github.com)
As per the NetBox developer suggestions, you can schedule the housekeeping script to run each day by enabling the schedule job in Semaphore.
NetBox Health Check
The NetBox Health Check script runs some useful commands to quickly capture the state of your NetBox instance, for example, checking processes are running, checking system logs, etc.
systemctl is-active netbox
systemctl is-active netbox-rq
python3 /opt/netbox/netbox/manage.py check
grep -i 'error' /var/log/nginx/netbox.error.log || true
grep -i 'warning' /var/log/nginx/netbox.error.log || true
journalctl -u netbox --since '24 hours ago' -n 250 | grep -i 'error' || true
journalctl -u netbox-rq --since '24 hours ago' -n 250 | grep -i 'error' || true
When you run the script, you'll see output like this:
In this scenario we can quickly see there is an error when starting NetBox, which pointed us in the right direction for troubleshooting a syntax issue in settings.py
(using the commands detailed above).
Deploy NetBox Community Plugins
Deploy NetBox Community Plugins
This feature is a work-in-progress. Testing has been completed deploying the plugins in the screenshot to NetBox 3.7.8
, and NetBox 4.0.11
, and also upgrading NetBox, and then re-deploying using this tool.
Introduction
Deploy, manage, and upgrade NetBox community plugins from within Semaphore:
You can queue up tasks (providing concurrency=1
in the Semaphore project/dashboard settings, in order to avoid having multiple tasks running at the same time):
Plugin Mapping Tables
Version Selection
Similar to the mapping logic during the NetBox installation process, each plugin has a defined mapping table that maps the plugin version to the supported NetBox version. For example, in this playbook, the logic is as follows:
- Get the current NetBox version from the NetBox API
/api/status/
- Query the plugin-specific mapping table, in this case for
netbox_topology_views
- Check the NetBox Version to Plugin Version mapping, and if there is a match, proceed with the installation.
Here you can see that we are running NetBox 3.7.8
and that the playbook has detected that the suitable plugin version is 3.9.0
.
The script then proceeds to download the plugin (to /netos/working-dir/netbox-plugins
) and install it.
If a match isn't found, i.e. the plugin is not supported on that version of NetBox (according to the mapping table) the installation will fail like this:
Mapping File Example
Each plugin has a mapping file similar to this in the netos-netbox
repository. They are manually constructed based on the individual plugin recommendations to ensure that for each NetBox version the latest plugin is installed.
The git_url
, version_prefix
, and file_type
variables are used to construct the URL to pull the plugin from the repo, as URL structures vary between plugins.
netbox_topology_views:
name: "NetBox Topology Views"
description: "A NetBox plugin that provides a graphical topology view for devices in NetBox."
git_url: "https://github.com/netbox-community/netbox-topology-views"
version_prefix: "v"
file_type: "zip"
version_mapping:
- netbox_version: "4.1.0"
plugin_version: "4.1.0"
- netbox_version: "4.0.0"
plugin_version: "4.0.0"
- netbox_version: "3.7.8"
plugin_version: "3.9.0"
Every NetBox version is mapped to a specific plugin version based on the plugin documentation.
Updating Configuration.py
Plugins must be registered in configuration.py
, for example; PLUGINS = ['netbox_topology_views']
. The playbook will check if the plugin name is included, and if not, add it.
If there is an issue with the plugin, it can cause NetBox to fail to start, i.e. you'll get an NGINX 502 error. If this happens you can use the Health Check script to see if there is anything obvious in the logs. You can also Disable NetBox Plugins to remove the plugin entry from configuration.py
.
Post Installation
After running the playbooks in the screenshot at the top of this guide, we can see that the plugins have been installed in NetBox:
Add New Plugins to Semaphore
The process for adding a plugin to deploy through Semaphore is as follows:
- The respective templates must be configured in Semaphore, pointing to the correct playbooks, i.e.
netbox-plugin-deploy.yml
andnetbox-plugin-disable.yml
- Within the template, the tag must align with the plugin name, i.e.
--tags=netbox_topology_views
- The plugin name must be added to
netbox-plugin-deploy.yml
andnetbox-plugin-disable.yml
- The mapping file must be configured, e.g. in Ansible
/mappings/netbox-plugins/netbox_topology_views.yml
Some plugins, i.e. netbox_data_flows
, caused issues with the venv
, and broke the installation of other plugins. Further investigation is required.
To Do
- Define the optimal sequencing, i.e. upgrade plugin, then NetBox, then migrate, etc.
- Knit everything together, e.g. upgrading NetBox along with 7 different plugins
- Ensuring a critical mass of mapping tables and data to make it useful
- Adding / supporting external/local/private plugins
- Create combined playbooks / checks to test the installation of all NetBox plugins on all NetBox versions
Disable NetBox Community Plugins
Introduction
Each community plugin can be disabled, however, note that all this script will do is remove the plugin name from the PLUGINS = ['plugin_name']
string in the configuration.py
file. It will not uninstall the plugin or drop any database tables.
To re-enable the plugin, follow the Deploy NetBox Community Plugins guide.
Deploy Netos Enterprise
Deploy Netos Enterprise Plugins
Introduction
This guide summarizes how to deploy licensed Netos enterprise plugins, which are compiled and licensed per-customer. The Netos Plugin architecture is as follows, and you can find out more here.
License Status
You can check the license status for each plugin in Netos Core.
Securely Deploy with Cloudflare Zero Trust
Typically you will deploy via Cloudflare which is a two-stage authentication process built for Netos NetBox and Netos Pod deployments not directly connected to the Netos internal network. Tokens and enterprise settings are configured in the Netos Enterprise Plugin environment in Semaphore.
- Using the following ID and secret the deployment playbook will authenticate to Cloudflare Zero Trust to enable pass through to our secure GitLab server.
CF_ACCESS_CLIENT_ID
CF_ACCESS_CLIENT_SECRET
- The playbook will then authenticate to your dedicated private repository using your customer ID and token:
CUSTOMER_ID
CUSTOMER_TOKEN
- The plugins are downloaded and installed.
Deployment Process
First the Python dependencies will be checked and if missing installed using pip
.
Once the plugins are deployed, the lates plugin versions are displayed.
Enterprise Support
Please speak with Netos Support for any assistance with this process. This is just a high-level guide.
Deploy Netos User Preferences & Field Choices
Introduction
Netos enriches the standard NetBox dataset, and this playbook imports settings and views to align the Netos plugins with NetBox. When running the playbook, it will insert markers in the configuration.py
file to ensure that duplicates are not added.
# BEGIN netos-field-choices ANSIBLE MANAGED BLOCK
FIELD_CHOICES = {
'dcim.Device.status+': (
('imported', 'Imported', 'purple'),
('detected', 'Detected', 'blue'),
('discovery_failed', 'Discovery Failed', 'orange'),
('polling_error', 'Polling Error', 'red'),
)
}
# END netos-field-choices ANSIBLE MANAGED BLOCK
# BEGIN netos-preferences ANSIBLE MANAGED BLOCK
DEFAULT_USER_PREFERENCES = {
"ui": {
"colormode": "light"
},
"tables": {
"ASNTable": {
"columns": [
"asn",
"rir",
"site_count",
"provider_count",
"sites",
"description"
]
},
# END netos-preferences ANSIBLE MANAGED BLOCK
If you need to add custom FIELD_CHOICES or user preferences either include the standard Netos settings or make changes between the markers.
Run the Playbook
Running the playbook checks and deploys the settings.
Default User Preferences
Each user can set their profile preferences, for example, from the screenshot below. These "default user preferences" can also be set globally, i.e. for all users, which is what this playbook does.
The primary Netos use case is to customise table layouts outside of the Netos plugins (like DCIM Devices). For example, where Netos enriches and add useful data, that can be included in the default user settings for all users.
Field Choices
Certain functions in the Netos Fabric plugin change the status of devices. In order to expand the field choices, Netos uses the NetBox standard Data & Validation Field Choices settings to add additional status values:
Netos Reporting Cache Pre-Load
Introduction
Some reports require large database queries to prepare the data, which can take 10-30 seconds to run. From a user perspective this is too slow.
Netos has developed a caching engine that pre-loads the report caches so the charts in reports, like above, loads in an acceptable time frame (usually 2-3 seconds).
Pre-Load Job
At present, there is a playbook that first flushes the current cache (stored in redis
) and then pre-loads the cache. This task is also initiated after the Netos plugins are deployed.
The task in Semaphore will probably be depreciated in the future once https://github.com/netbox-community/netbox/issues/16971 is resolved.
Scheduling
Depending on the frequency of updates on the NetBox instance, the scheduler in Semaphore can be configured to automate the cache refresh. By default, this is every hour.
Create NetBox API Token for Airflow
Introduction
This task will provision an API key in NetBox to allow Netos Pod (Airflow) to pull and push data into NetBox. The playbook will perform the following steps:
- Create a NetBox user called
airflow_api
with asuperuser
role. - Generate an API key for the airflow_api user (meaning that any updates from Airflow are clearly attributed to Airflow, i.e. Netos Pod).
- Configure the
NETBOX_API_KEY
variable in Airflow.
This solution assumes that Airflow is running on the local server. For remotely connected NetBox <> Netos Pod configurations, please set the API manually in Airflow.
The Playbook
The "Create API User" task failed because the user exists. In this case it will refresh the token used by Airflow.
Airflow
The NetBox API key is created. If the key is wrong, the Global Orchestration DAG will not start and will show an authentication error in the logs.