The blue info boxes flag **What You've Missed**, which are stages you should consider in a production, or in "more than just a NetBox lab" environment.
### 5 Minute Demo ### Installation Guide #### 1. Install Ubuntu Server You're going to need to get a server with **Ubuntu 24.04 LTS** up and running. [Download an image](https://ubuntu.com/download/server) like `ubuntu-24.04-live-server-amd64.iso` or spin up an instance on your favorite cloud provider, like AWS. #### 2. Install Semaphore and the Netos NetOps Project You can follow the full [Install Semaphore Guide](https://docs.netos.io/books/semaphore-netops/page/deploy-semaphore-netos-projects), however, in summary, once you install Ubuntu, paste this command to pull the repo and run the install script, and you shouldn't need to do anything else: `sudo wget https://raw.githubusercontent.com/netos-networks/netos-netops/refs/heads/main/easy-deploy.sh && sudo chmod +x easy-deploy.sh && sudo ./easy-deploy.sh` [](https://docs.netos.io/uploads/images/gallery/2024-09/wjPnxkbt4Z0ogrTF-image.png)The [Key Semaphore Principles](https://docs.netos.io/books/semaphore-netops/page/key-semaphore-principles) guide explains more about Semaphore.
Check the [Users, Accounts, and Tokens](https://docs.netos.io/books/semaphore-netops/page/users-accounts-tokens) guide for more information about credentials and variables.
#### 3. Configure Semaphore These steps are optional, so you can skip this for now.Change the `DOMAIN` and `SEMAPHORE_HOSTNAME` variables in the **NetBox Global Settings** Semaphore environment. Follow the [Deploying NGINX & Certificates](https://docs.netos.io/books/semaphore-netops/page/deploying-nginx-certificates) guide.
Deploy self-signed certificates, so you don't get "your connection isn't private" browser warnings. Follow the [Deploying NGINX & Certificates](https://docs.netos.io/books/semaphore-netops/page/deploying-nginx-certificates) guide.
Enable [Semaphore Backups](https://docs.netos.io/books/semaphore-netops/page/semaphore-backup-guide).
#### 4. Deploy NGINX and Self-Signed Cert for NetBox You need to deploy NGINX to connect to NetBox. To fast track, you can just create a self-signed certificate. The default hostname will be `netbox.netos.dev`. [](https://docs.netos.io/uploads/images/gallery/2024-09/UZ5mahAPnD3cl951-image.png)Change the `DOMAIN` and `NETBOX_HOSTNAME` variables in the **NetBox Global Settings** Semaphore environment. Follow the [Deploying NGINX & Certificates](https://docs.netos.io/books/semaphore-netops/page/deploying-nginx-certificates) guide.
Deploy self-signed certificates, so you don't get "Your connection isn't private" browser warnings. Follow the [Deploying NGINX & Certificates](https://docs.netos.io/books/semaphore-netops/page/deploying-nginx-certificates) guide.
#### 5. Install NetBox Browse to the **NETBOX DEPLOY** view in Semaphore to start the installation. The full [NetBox Installation Guide](https://docs.netos.io/books/semaphore-netbox/page/install-netbox) is here. [](https://docs.netos.io/uploads/images/gallery/2024-09/gkzWMYios59nJyix-image.png) If you want to install a specific version, set it here, matching the numeric version from [Releases · netbox-community/netbox (github.com)](https://github.com/netbox-community/netbox/releases). Don't add the `v`, i.e. `v4.1.2` would be `4.1.2`. [](https://docs.netos.io/uploads/images/gallery/2024-09/fIACUdKPF7wErapl-image.png)Configure scheduled daily [NetBox ](https://docs.netos.io/books/semaphore-netbox/page/netbox-backup-guide)Backups or set your own custom frequency.
[Upgrade NetBox](https://docs.netos.io/books/semaphore-netbox/page/upgrade-netbox). For example, maybe you've used the [Restore](https://docs.netos.io/books/semaphore-netbox/page/netbox-restoration-guide) process to restore to an older version and want to upgrade to a newer version.
Enable the [NetBox Maintenance Playbook](https://docs.netos.io/books/semaphore-netbox/page/netbox-health-checks-maintenance) on a daily schedule as per the developer's guidance.
#### 6. Connect to NetBox Once the playbook is complete, you should be able to connect using HTTPS and the IP address to NetBox. Check the [Users, Accounts, and Tokens](https://docs.netos.io/books/semaphore-netops/page/users-accounts-tokens) guide for full details on account and passwords. The default credentials are as follows: - **Username**: `admin` - **Password**: `ohp8toef7Jee` [](https://docs.netos.io/uploads/images/gallery/2024-09/K1H1uubECrlQwMCS-image.png)If you experience any problems, you can run the [Health Check Playbook](https://docs.netos.io/books/semaphore-netbox/page/netbox-health-checks-maintenance) to quickly diagnose common issues. For example, check the logs to catch the reason why NetBox failed to start.
#### 7. Install NetBox Plugins You can [Install and Upgrade](https://docs.netos.io/books/semaphore-netbox/page/deploy-netbox-community-plugins) popular NetBox plugins from within Semaphore, and also [Disable NetBox Plugins](https://docs.netos.io/books/semaphore-netbox/page/disable-netbox-community-plugins). This feature is a work-in-progress, and probably will be for a while...! [](https://docs.netos.io/uploads/images/gallery/2024-09/cZ0FC3AH3LYr2m9U-image.png)If things go wrong in your test environment, you can quickly [Wipe NetBox](https://docs.netos.io/books/semaphore-netbox/page/remove-netbox) and start again.
# Install NetBox ### Introduction This guide details how to deploy a new instance of NetBox. For guidance on upgrading NetBox, please refer to this [Upgrade NetBox](https://docs.netos.io/books/semaphore-netbox/page/upgrade-netbox).NetBox is installed from the [public GitHub repository](https://github.com/netbox-community/netbox) according to the version set in Semaphore (or proposed during an upgrade). Netos **has not forked** NetBox. Any customizations made by Netos are optional and controlled from within the NETOS view in the NetBox project in Semaphore.
### Configuration Guide #### 1. Configure the Semaphore Environment You can set the `SUPER_USER` username, email, and password in the **NetBox Global Settings** environment settings, or just use the defaults for testing: [](https://docs.netos.io/uploads/images/gallery/2024-09/sEXwOttqGjSC4bdR-image.png)Please don't change the install directory from `/opt/netbox` as it is hard coded in configuration files and variables.
There is a [bug in Ansible Semaphore](https://github.com/semaphoreui/semaphore/issues/2293) that requires you to delete and recreate the secret value. Save doesn't work.
#### 2. Deploy NGINX and the Cert Refer to the following guide [Deploying NGINX & Certificates](https://docs.netos.io/books/semaphore-netops/page/deploying-nginx-certificates) for a detailed guide on deploying NGINX and certs. Once you've deployed, you'll see a screen like this and should be able to connect to a URL like https://uk-lab-nb05-netbox.core.netos.io. [](https://docs.netos.io/uploads/images/gallery/2024-09/najqx4QkhBTa2lpY-image.png)You need to deploy NGINX because it configures the webserver to mount `alias /opt/netbox/netbox/static/;` correctly to avoid the [Static Media Failure](https://stackoverflow.com/questions/62102664/static-media-failure-with-nginx-and-gunicorn) error.
#### 3. Install NetBox We have included pre-defined NetBox templates (which will be updated from time-to-time), for example, you can just click play on the Semaphore template to deploy NetBox `v3.7.8` or `v4.0.11`. [](https://docs.netos.io/uploads/images/gallery/2024-09/A5BzjtElionPicRv-image.png) If you want to install a custom version of NetBox, you can do so by editing the tags in the Semaphore Template. For example, change `--tags=X.X.X` to `--tags=3.6.5`. This feature is useful for a migration, for example, if you wanted to migrate from a legacy NetBox instance, you could take a backup and [Restore](https://docs.netos.io/books/semaphore-netbox/page/netbox-restoration-guide), then [Upgrade](https://docs.netos.io/books/semaphore-netbox/page/upgrade-netbox). [](https://docs.netos.io/uploads/images/gallery/2024-09/FsLJHLmRh2vXLvXz-image.png) Note that the `netbox-deploy.yml` Ansible playbook has logic that matches the tags referenced in Semaphore. The tags must be set for this playbook to execute. The tags are used to set variables which are passed to the `netbox-deploy` Ansible role. ```yaml - hosts: 127.0.0.1 connection: local become: yes vars_files: - ./vars/env-netbox-settings.yml tasks: - name: Set NetBox release version to v4.1.1 set_fact: NETBOX_RELEASE: 4.1.1 tags: 4.1.0 - name: Set NetBox release version to v4.1.0 set_fact: NETBOX_RELEASE: 4.1.0 tags: 4.1.0 ``` #### 4. Connect to NetBox Once you've deployed NetBox you can connect to the application based on the `NETBOX_HOSTNAME` and `DOMAIN` variables configured in the NetBox Global Setting environment. Check the [Users, Accounts, and Tokens](https://docs.netos.io/books/semaphore-netops/page/users-accounts-tokens) guide for full details on account and passwords. The default credentials are as follows: - **Username**: `admin` - **Password**: `ohp8toef7Jee` ##### a. NetBox 3.7.8 [](https://docs.netos.io/uploads/images/gallery/2024-09/lwiS3RbQZsEhpi87-image.png) ##### b. NetBox 4.0.11 [](https://docs.netos.io/uploads/images/gallery/2024-10/7MVITL9DIZzOn5Al-image.png) ### Thoughts About DockerWe (Netos) used and tested the [NetBox Docker](https://github.com/netbox-community/netbox-docker) image for over 6 months. When we started to load test to simulate a large production enterprise environment, we experienced stability issues, especially when hitting the API hard, and also had to make a number of changes to the Docker NGINX configuration.
Others may have had success; however, we were unable to achieve stability (outside of development environments) with NetBox in Docker. The playbooks here install NetBox natively on the Linux operating system, and we've had no issues whatsoever.
# Upgrade NetBox ### Introduction This guide will walk you through the process of upgrading NetBox, including a suggested pre-staging strategy. The approach we take to upgrading NetBox is as follows: 1. Upgrade to the final patch of the current release, e.g. from `3.7.4` to `3.7.8` 2. Upgrade to the final patch version of the next major release, e.g. from `3.7.8` to `4.0.11`The official NetBox upgrade strategy can be found here: [netbox/docs/installation/upgrading.md at develop · netbox-community/netbox (github.com)](https://github.com/netbox-community/netbox/blob/develop/docs/installation/upgrading.md)
This feature is a work-in-progress. There are many other considerations to take into account, such as plugin configurations and customizations, as well as sequencing.
### Upgrade Playbooks and Templates Within the NetBox Deploy view in Semaphore, you'll see these playbooks. - **NetBox Upgrade Check Path** - checks the current running version with the mapping table to suggest the next version. - **NetBox Upgrade to Next Suitable Version** - this will take the result from the "Check Path" playbook and perform an upgrade. [](https://docs.netos.io/uploads/images/gallery/2024-09/DppUnx16QZeKRlCS-image.png) #### Checking the Upgrade Path The following screenshot shows that NetBox `3.7.8` has been detected, and based on the upgrade path mapping table, the next suggested version is `4.0.11`. [](https://docs.netos.io/uploads/images/gallery/2024-09/r7POsdI4tVftAd9f-image.png) When the playbook runs it checks a mapping table that is structured like this. The logic is: 1. If `current_version` and `patch` are the different, then upgrade to the next `patch` in the same release 2. If `current_version` and `patch` are the same, then upgrade to `next_major` ```yaml netbox_upgrade_paths: 3.7.0: patch: 3.7.8 next_major: 4.0.11 3.7.1: patch: 3.7.8 next_major: 4.0.11 3.7.2: patch: 3.7.8 next_major: 4.0.11 3.7.3: patch: 3.7.8 next_major: 4.0.11 3.7.4: patch: 3.7.8 next_major: 4.0.11 3.7.5: patch: 3.7.8 next_major: 4.0.11 3.7.6: patch: 3.7.8 next_major: 4.0.11 3.7.7: patch: 3.7.8 next_major: 4.0.11 3.7.8: patch: 3.7.8 next_major: 4.0.11 4.0.0: patch: 4.0.11 next_major: 4.1.0 ```In a production environment you should not upgrade past the final release in the last release. For example, once `4.0.0` was released, `3.7.8` was considered the final stable release. Once `4.1.0` was released, we would consider `4.0.11` stable, and so on.
#### Running the Upgrade Here you can see that the suggested upgrade version from the mapping table has been passed to the `netbox-deploy` playbook to facilitate the upgrade. [](https://docs.netos.io/uploads/images/gallery/2024-09/X9FOmLW5G2XvevSC-image.png)You must take a [full backup](https://docs.netos.io/books/semaphore-netbox/page/netbox-backup-guide) before starting the upgrade. If possible, we would suggest stopping all services on the server and taking a snapshot (disk image) before proceeding.
Ensure you check compatibility with any plugins that you have installed and have a [clear upgrade path for them](https://docs.netos.io/books/semaphore-netbox/page/deploy-netbox-community-plugins).
Ensure you check breaking changes in the [NetBox Release notes](https://github.com/netbox-community/netbox/releases) that could impact you, for example, if you have integrated your systems using the [NetBox Rest API](https://netbox.readthedocs.io/en/stable/integrations/rest-api/), endpoints do change from time-to-time, which could break your integrations.
Have a clear test plan.
### Pre-Stage Your Upgrades It's fast and simple to spin up a staging NetBox instance for testing. A "belt and braces" approach to an upgrade is as follows: 1. [Install Semaphore](https://docs.netos.io/books/semaphore-netops/page/deploy-semaphore-netos-projects) 2. [Install NetBox, ensuring that you match the NetBox version](https://docs.netos.io/books/semaphore-netbox/page/install-netbox) 3. [Backup your current production NetBox instance](https://docs.netos.io/books/semaphore-netbox/page/netbox-backup-guide) 4. [Restore to the new staging NetBox instance](https://docs.netos.io/books/semaphore-netbox/page/netbox-restoration-guide) 5. Test the upgrade process in safety! # Remove NetBox ### Introduction In a test or lab environment it is useful to be able to quickly wipe NetBox so you can [re-install](https://docs.netos.io/books/semaphore-netbox/page/install-netbox) and [restore](https://docs.netos.io/books/semaphore-netbox/page/netbox-restoration-guide). [](https://docs.netos.io/uploads/images/gallery/2024-09/CExlcQOhNpdbi7Nl-image.png)This process will drop the NetBox database from PostgreSQL and delete the `/opt/netbox` folder. It's probably not a good idea to run this on production system. In fact, delete the template on production systems!
# NetBox CSRF Settings ### Introduction This playbook `netbox-csrf` will add your domains in the following format to the NetBox `settings.py` file based on the `DOMAIN` variable in Semaphore. - `http://*.yourdomain.com` - `https://*.yourdomain.com` - `http://yourdomain.com` - `https://yourdomain.com` When you run the playbook, it will look like this: [](https://docs.netos.io/uploads/images/gallery/2024-09/FIFPzvJXKrPhjm3J-image.png) ### Use Case If your server name is `uk-lab-nb01.core.netos.io`, and you configured a DNS CNAME to give it a more friendly name like `demo2.netos.io` via a reverse proxy, then you would see this error and won't be able to login: [](https://docs.netos.io/uploads/images/gallery/2024-09/Gek6Giygy04COwqM-image.png) To fix this, change the domain to in the NetBox Global Settings: [](https://docs.netos.io/uploads/images/gallery/2024-09/ExXDkAV0PMPqSzYi-image.png) Which will create a `settings.py` file entry like this and fix the error: `CSRF_TRUSTED_ORIGINS = getattr(configuration, 'CSRF_TRUSTED_ORIGINS', ['http://netos.io', 'https://netos.io', 'http://*.netos.io', 'https://*.netos.io'])` # Manage & Operate NetBox Backup, restore, and perform health checks and maintenance on your NetBox instance # NetBox Backup Guide ### Introduction The NetBox backup process provides regular secure backups. In summary the Ansible playbook does the following: 1. Manages retention based on your preferences, e.g. deletes local backups older than 14 days old. 2. Backs up and encrypts the media directory. 3. Backs up and encrypts the `netbox` PostgreSQL database. 4. Captures the current state of NetBox from the `/api/status` endpoint and saves the output in a JSON file that will be compared agains the target state in a restore event. 5. Optionally SFTP's the three backup files to a secure remote SFTP server.The NetBox process is **stopped** during the backup to ensure no changes are made to the database during the backup, and to ensure data integrity. This is typically for less than a minute, but take this into account if you anything connecting to the NetBox, e.g. automation tools. Netos Pod (Airflow) has a retry mechanism that handles broken connectivity to NetBox.
[](https://docs.netos.io/uploads/images/gallery/2024-09/G4UKByPY2Fo9R5ZR-image.png) #### Cron Scheduling The cron scheduler in Semaphore can be configured to backup the database at regular intervals, for example, at 02:30 every day. [](https://docs.netos.io/uploads/images/gallery/2024-09/zGK0JFzkP9aHRrjq-image.png)Note that there is a [bug in Ansible Semaphore UI](https://github.com/semaphoreui/semaphore/issues/2294) that causes the same task to run many times. The solution is to toggle the "Show cron format' button and use UNIX formatting like [here](https://crontab.guru/examples.html).
On the server, you can see the daily 02:00 encrypted backups in the red boxes. Below that, you can see an example of the JSON status that was captured when the backup was executed, so you know the exact version of NetBox and installed plugins for a restore. [](https://docs.netos.io/uploads/images/gallery/2024-09/Gz50nAMkDRfUO5hY-image.png) #### Backup File Rotation You can set the retention period for backup files stored in `/netos/backups/netbox` in the **NetBox Backup Settings** Semaphore environment. [](https://docs.netos.io/uploads/images/gallery/2024-09/X44Bpmbd52wn9qkb-image.png)Ensure you use the exact values of `days` or `weeks`, i.e. no capitals.
#### Remote SFTP To enable remote SFTP, change the `No` value in the `SFTP_ENABLED` variable to `Yes`, and set the `SFTP_HOST/USER/PASS` values accordingly. [](https://docs.netos.io/uploads/images/gallery/2024-09/nHyM898RhDS7qZBt-image.png) ### Restoring from Backup The [NetBox Restore Process](https://docs.netos.io/books/semaphore-netbox/page/netbox-restoration-guide) is one way to restore. If you want to manually decrypt the `enc` files on your local workstation, use the following commands. `openssl enc -aes-256-cbc -d -in BACKUP_FILE.enc -out OUTPUT.sql.gz/.tar.gz` You will need the password set in the NetBox Backup Settings Semaphore environment variable `ENCRYPTION_KEY`. For example, by running this command and entering the password, we decrypt the NetBox media directory: `openssl enc -aes-256-cbc -d -in netbox_media_09_25_2024_02_00.tar.gz.enc -out netbox_media_09_25_2024_02_00.tar.gz` ### NetBox Development Snapshots #### Use Backups for Labs The Netos team use the tools outlined in these guides to deploy, restore, develop, and manage dozens of NetBox development environments, typically with many changes and deployments each day.The original backup below used the standard timestamp format `mm_dd_yyyy_HH_MM`. However, we renamed it so we can snapshot different useful development environments pre-populated with data. We can then have many backups and images deployed to different NetBox instances on Semaphore to test and stage.
[](https://docs.netos.io/uploads/images/gallery/2024-10/LQ5C5kPTyFrJHqgv-image.png) As long as you only replace the timestamp in the filename with some text, you can set this in the `RESTORE_FILE_STAMP`, variable in the **NetBox Backup Settings** Semaphore environment. Here are some examples:| **Before** | **After** |
| `netbox_db_10_01_2024_15_24.sql.gz.enc` | `netbox_db_01_Data_Feeds_and_Excel.sql.gz.enc` |
| `netbox_media_10_01_2024_15_24.tar.gz.enc` | `netbox_media_01_Data_Feeds_and_Excel.tar.gz.enc` |
| `netbox_status_10_01_2024_15_24.json` | `netbox_status_01_Data_Feeds_and_Excel.json` |
The `ENCRYPTION_KEY` value in the **NetBox Backup Settings** Semaphore environment is used in both the backup and restore process. Be sure to keep track as if you change it on the target system, all future backups will use the new key.
#### Configure the Restore The NetBox backup process will create three timestamped (`mm_dd_yyyy_HH_MM`) files like these. The `enc` files are encrypted using the `ENCRYPTION_KEY` variable. ``` netbox_db_03_18_2024_01_00.sql.gz.enc netbox_media_03_18_2024_01_00.tar.gz.enc netbox_status_03_18_2024_01_00.json ```You can manually decrypt the files using the command openssl `enc -aes-256-cbc -d -in BACKUP_FILE.enc -out OUTPUT.sql.gz.tar.gz`
The restore file prefix is set in the NetBox Backup Settings environment and is the timestamp used for all three files. Set this to match the timestamp of the files that you place in the `/netos/netbox/backups` directory. [](https://docs.netos.io/uploads/images/gallery/2024-09/8vSSlDZFoJuvEKbR-image.png) #### Plugin Version Validation Logic By default, when the backup runs it will check the **NetBox version** and **plugin versions** of the target system by comparing the output of the `/api/status` endpoint collected from the backup with the version of the target system. This is an example of the relevant JSON: ```json "netbox-version": "3.7.8", "plugins": { "netos": "1.4.6", "netos_fabric": "1.3.3", "netos_model_builder": "1.2.37", "netos_reporting": "1.3.16" } ``` In the event of a mismatch, you will see a warning like this that details the discrepancies between the backup and target system. [](https://docs.netos.io/uploads/images/gallery/2024-09/xmR4A2UOYzZ0pyhi-image.png) To disable this check, change `BYPASS_PLUGIN_CHECK` to `Yes`. Disabling may be acceptable when restoring to a slightly different target system, for example, where the Netos Insights plugin has increased from minor version `1.2.24` to `1.2.26`. [](https://docs.netos.io/uploads/images/gallery/2024-09/xoCDes2oDTzVv8OJ-image.png) When `BYPASS_PLUGIN_CHECK` is set to `Yes` the restore process will look like this: [](https://docs.netos.io/uploads/images/gallery/2024-09/QxHxdV47PHudNbfu-image.png)Any [NetBox Open Source Plugins](https://docs.netos.io/books/semaphore-netbox/page/deploy-netbox-community-plugins) or [Netos Enterprise NetBox Plugins](https://docs.netos.io/books/semaphore-netbox/page/deploy-netos-enterprise-plugins) installed using Semaphore will be named according to their version and saved in `/netos/working-dir/netbox-plugins` and `/netos/working-dir/netos-plugins`. The `/netos/working-dir` directory is [backed up each day](https://docs.netos.io/books/semaphore-netbox/page/netbox-backup-guide), assuming you enable the schedule in Semaphore.
# NetBox Health Checks & Maintenance ### Introduction In order to keep your NetBox instance operating smoothly you can run housekeeping and maintenance scripts. #### NetBox Housekeeping Script NetBox includes a `housekeeping` management command that should be run nightly. This command handles: - Clearing expired authentication sessions from the database - Deleting changelog records older than the configured [retention time](https://github.com/netbox-community/netbox/blob/develop/docs/configuration/miscellaneous.md#changelog_retention) - Deleting job result records older than the configured [retention time](https://github.com/netbox-community/netbox/blob/develop/docs/configuration/miscellaneous.md#job_retention) - Check for new NetBox releases (if [`RELEASE_CHECK_URL`](https://github.com/netbox-community/netbox/blob/develop/docs/configuration/miscellaneous.md#release_check_url) is set)The official documentation is here [netbox/docs/administration/housekeeping.md at develop · netbox-community/netbox (github.com)](https://github.com/netbox-community/netbox/blob/develop/docs/administration/housekeeping.md)
[](https://docs.netos.io/uploads/images/gallery/2024-09/DitpDAZIRwrYqr2c-image.png) As per the NetBox developer suggestions, you can schedule the housekeeping script to run each day by enabling the schedule job in Semaphore. [](https://docs.netos.io/uploads/images/gallery/2024-09/UrqaXFpImdDFfgRo-image.png) #### NetBox Health Check The NetBox Health Check script runs some useful commands to quickly capture the state of your NetBox instance, for example, checking processes are running, checking system logs, etc. ```bash systemctl is-active netbox systemctl is-active netbox-rq python3 /opt/netbox/netbox/manage.py check grep -i 'error' /var/log/nginx/netbox.error.log || true grep -i 'warning' /var/log/nginx/netbox.error.log || true journalctl -u netbox --since '24 hours ago' -n 250 | grep -i 'error' || true journalctl -u netbox-rq --since '24 hours ago' -n 250 | grep -i 'error' || true ``` When you run the script, you'll see output like this: [](https://docs.netos.io/uploads/images/gallery/2024-09/txUB7gBOkzMrXtHx-image.png) In this scenario we can quickly see there is an error when starting NetBox, which pointed us in the right direction for troubleshooting a syntax issue in `settings.py` (using the commands detailed above). [](https://docs.netos.io/uploads/images/gallery/2024-09/0UFTf6lwIIicFwzZ-image.png) # Deploy NetBox Community Plugins # Deploy NetBox Community PluginsThis feature is a work-in-progress. Testing has been completed deploying the plugins in the screenshot to NetBox `3.7.8`, and NetBox `4.0.11`, and also [upgrading NetBox](https://docs.netos.io/books/semaphore-netbox/page/upgrade-netbox), and then re-deploying using this tool.
### Introduction Deploy, manage, and upgrade NetBox community plugins from within Semaphore: [](https://docs.netos.io/uploads/images/gallery/2024-09/bEuiZYJZRZttXP2T-image.png) You can queue up tasks (providing `concurrency=1` in the Semaphore project/dashboard settings, in order to avoid having multiple tasks running at the same time): [](https://docs.netos.io/uploads/images/gallery/2024-10/f93QzqYQS4tzrj7y-image.png) ### Plugin Mapping Tables #### Version Selection Similar to the mapping logic during the [NetBox installation process](https://docs.netos.io/books/semaphore-netbox/page/install-netbox), each plugin has a defined mapping table that maps the plugin version to the supported NetBox version. For example, in this playbook, the logic is as follows: 1. Get the current NetBox version from the NetBox API `/api/status/` 2. Query the plugin-specific mapping table, in this case for `netbox_topology_views` 3. Check the **NetBox Version** to **Plugin Version** mapping, and if there is a match, proceed with the installation. Here you can see that we are running NetBox `3.7.8` and that the playbook has detected that the suitable plugin version is `3.9.0`. [](https://docs.netos.io/uploads/images/gallery/2024-09/bvk6JVQDvrqBy8cQ-image.png) The script then proceeds to download the plugin (to `/netos/working-dir/netbox-plugins`) and install it. [](https://docs.netos.io/uploads/images/gallery/2024-09/HN1ZbbN3fjeSTh38-image.png) If a match isn't found, i.e. the plugin is not supported on that version of NetBox (according to the mapping table) the installation will fail like this: [](https://docs.netos.io/uploads/images/gallery/2024-09/yzXdBOryt3Fxg1mg-image.png) #### Mapping File Example Each plugin has a mapping file similar to this in the `netos-netbox` repository. They are manually constructed based on the individual plugin recommendations to ensure that for each NetBox version the latest plugin is installed. The `git_url`, `version_prefix`, and `file_type` variables are used to construct the URL to pull the plugin from the repo, as URL structures vary between plugins. ```json netbox_topology_views: name: "NetBox Topology Views" description: "A NetBox plugin that provides a graphical topology view for devices in NetBox." git_url: "https://github.com/netbox-community/netbox-topology-views" version_prefix: "v" file_type: "zip" version_mapping: - netbox_version: "4.1.0" plugin_version: "4.1.0" - netbox_version: "4.0.0" plugin_version: "4.0.0" - netbox_version: "3.7.8" plugin_version: "3.9.0" ```Every NetBox version is mapped to a specific plugin version based on the plugin documentation.
#### Updating Configuration.py Plugins must be registered in `configuration.py`, for example; `PLUGINS = ['netbox_topology_views']`. The playbook will check if the plugin name is included, and if not, add it.If there is an issue with the plugin, it can cause NetBox to fail to start, i.e. you'll get an NGINX 502 error. If this happens you can use the [Health Check](https://docs.netos.io/books/semaphore-netbox/page/netbox-health-checks-maintenance) script to see if there is anything obvious in the logs. You can also [Disable NetBox Plugins](https://docs.netos.io/books/semaphore-netbox/page/disable-netbox-community-plugins) to remove the plugin entry from `configuration.py`.
### Post Installation After running the playbooks in the screenshot at the top of this guide, we can see that the plugins have been installed in NetBox: [](https://docs.netos.io/uploads/images/gallery/2024-09/FuOQmutLBMwS0rNt-image.png) ### Add New Plugins to Semaphore The process for adding a plugin to deploy through Semaphore is as follows: 1. The respective templates must be configured in Semaphore, pointing to the correct playbooks, i.e. `netbox-plugin-deploy.yml` and `netbox-plugin-disable.yml` 2. Within the template, the tag must align with the plugin name, i.e. `--tags=netbox_topology_views` 3. The plugin name must be added to `netbox-plugin-deploy.yml` and `netbox-plugin-disable.yml` 4. The mapping file must be configured, e.g. in Ansible `/mappings/netbox-plugins/netbox_topology_views.yml` [](https://docs.netos.io/uploads/images/gallery/2024-09/gvr56MRmqQu8YNvN-image.png)Some plugins, i.e. `netbox_data_flows`, caused issues with the `venv`, and broke the installation of other plugins. Further investigation is required.
### To Do - Define the optimal sequencing, i.e. upgrade plugin, then NetBox, then migrate, etc. - Knit everything together, e.g. upgrading NetBox along with 7 different plugins - Ensuring a critical mass of mapping tables and data to make it useful - Adding / supporting external/local/private plugins - Create combined playbooks / checks to test the installation of all NetBox plugins on all NetBox versions # Disable NetBox Community Plugins ### Introduction Each community plugin can be disabled, however, note that all this script will do is remove the plugin name from the `PLUGINS = ['plugin_name']` string in the `configuration.py` file. It will not uninstall the plugin or drop any database tables. To re-enable the plugin, follow the [Deploy NetBox Community Plugins](https://docs.netos.io/books/semaphore-netbox/page/deploy-netbox-community-plugins) guide. [](https://docs.netos.io/uploads/images/gallery/2024-09/bZvBmcFt8F6mHTZm-image.png) # Deploy Netos Enterprise # Deploy Netos Enterprise Plugins ### Introduction This guide summarizes how to deploy licensed Netos enterprise plugins, which are compiled and licensed per-customer. The Netos Plugin architecture is as follows, and you can find out more [here](https://netos.io/deploy-netos/how-it-works/). [](https://docs.netos.io/uploads/images/gallery/2024-09/mxDFOvpJcRl5Am1M-image.png) ### License Status You can check the license status for each plugin in Netos Core. [](https://docs.netos.io/uploads/images/gallery/2024-09/3cZgB2oQ8w3Pa9dy-image.png) ### Securely Deploy with Cloudflare Zero Trust Typically you will deploy via Cloudflare which is a two-stage authentication process built for Netos NetBox and Netos Pod deployments not directly connected to the Netos internal network. Tokens and enterprise settings are configured in the **Netos Enterprise Plugin** environment in Semaphore. 1. Using the following ID and secret the deployment playbook will authenticate to Cloudflare Zero Trust to enable pass through to our secure GitLab server. 1. `CF_ACCESS_CLIENT_ID` 2. `CF_ACCESS_CLIENT_SECRET` 2. The playbook will then authenticate to your dedicated private repository using your customer ID and token: 1. `CUSTOMER_ID` 2. `CUSTOMER_TOKEN` 3. The plugins are downloaded and installed. [](https://docs.netos.io/uploads/images/gallery/2024-09/oib9dP376oWFUGQL-image.png) ### Deployment Process First the Python dependencies will be checked and if missing installed using `pip`. [](https://docs.netos.io/uploads/images/gallery/2024-09/oflkGDuFd7HGZg9S-image.png) Once the plugins are deployed, the lates plugin versions are displayed. [](https://docs.netos.io/uploads/images/gallery/2024-09/ju8Ulk35JU440esJ-image.png) ### Enterprise Support Please speak with Netos Support for any assistance with this process. This is just a high-level guide. # Deploy Netos User Preferences & Field Choices ### Introduction Netos enriches the standard NetBox dataset, and this playbook imports settings and views to align the Netos plugins with NetBox. When running the playbook, it will insert markers in the `configuration.py` file to ensure that duplicates are not added. ```python # BEGIN netos-field-choices ANSIBLE MANAGED BLOCK FIELD_CHOICES = { 'dcim.Device.status+': ( ('imported', 'Imported', 'purple'), ('detected', 'Detected', 'blue'), ('discovery_failed', 'Discovery Failed', 'orange'), ('polling_error', 'Polling Error', 'red'), ) } # END netos-field-choices ANSIBLE MANAGED BLOCK # BEGIN netos-preferences ANSIBLE MANAGED BLOCK DEFAULT_USER_PREFERENCES = { "ui": { "colormode": "light" }, "tables": { "ASNTable": { "columns": [ "asn", "rir", "site_count", "provider_count", "sites", "description" ] }, # END netos-preferences ANSIBLE MANAGED BLOCK ```If you need to add custom FIELD\_CHOICES or user preferences either include the standard Netos settings or make changes between the markers.
#### Run the Playbook Running the playbook checks and deploys the settings. [](https://docs.netos.io/uploads/images/gallery/2024-09/QVarr3t7q7TCuIAA-image.png) #### Default User Preferences Each user can set their profile preferences, for example, from the screenshot below. These "default user preferences" can also be set globally, i.e. for all users, which is what this playbook does. [](https://docs.netos.io/uploads/images/gallery/2024-09/zO4DwdIMshhPmP5T-image.png) The primary Netos use case is to customise table layouts outside of the Netos plugins (like DCIM Devices). For example, where Netos enriches and add useful data, that can be included in the default user settings for all users. [](https://docs.netos.io/uploads/images/gallery/2024-09/2A2ZHtBvmA4BrY2M-image.png) #### Field Choices Certain functions in the Netos Fabric plugin change the status of devices. In order to expand the field choices, Netos uses the NetBox standard [Data & Validation Field Choices](https://netbox.readthedocs.io/en/stable/configuration/data-validation/#field_choices) settings to add additional status values: [](https://docs.netos.io/uploads/images/gallery/2024-09/12PircWNa2rYLuQU-image.png) # Netos Reporting Cache Pre-Load ### Introduction Some reports require large database queries to prepare the data, which can take 10-30 seconds to run. From a user perspective this is too slow. [](https://docs.netos.io/uploads/images/gallery/2024-09/rl9usGAgDmCjQelE-image.png) Netos has developed a caching engine that pre-loads the report caches so the charts in reports, like above, loads in an acceptable time frame (usually 2-3 seconds). #### Pre-Load Job At present, there is a playbook that first flushes the current cache (stored in `redis`) and then pre-loads the cache. This task is also initiated after the Netos plugins are deployed. [](https://docs.netos.io/uploads/images/gallery/2024-09/mbfhMgHYMqv615fs-image.png)The task in Semaphore will probably be depreciated in the future once [https://github.com/netbox-community/netbox/issues/16971](https://github.com/netbox-community/netbox/issues/16971) is resolved.
#### Scheduling Depending on the frequency of updates on the NetBox instance, the scheduler in Semaphore can be configured to automate the cache refresh. By default, this is every hour. [](https://docs.netos.io/uploads/images/gallery/2024-09/2dPLzohUEXPM39kQ-image.png) # Create NetBox API Token for Airflow ### Introduction This task will provision an API key in NetBox to allow Netos Pod (Airflow) to pull and push data into NetBox. The playbook will perform the following steps: 1. Create a NetBox user called `airflow_api` with a `superuser` role. 2. Generate an API key for the airflow\_api user (meaning that any updates from Airflow are clearly attributed to Airflow, i.e. Netos Pod). 3. Configure the `NETBOX_API_KEY` variable in Airflow.This solution assumes that Airflow is running on the local server. For remotely connected **NetBox <> Netos Pod** configurations, please set the API manually in Airflow.
#### The Playbook The "Create API User" task failed because the user exists. In this case it will refresh the token used by Airflow. [](https://docs.netos.io/uploads/images/gallery/2024-10/9vNXNE36YcfV2sTI-image.png) #### Airflow The NetBox API key is created. If the key is wrong, the Global Orchestration DAG will not start and will show an authentication error in the logs. [](https://docs.netos.io/uploads/images/gallery/2024-10/780s9ZPg6OWOmeO2-image.png)