Stale reports remain in Netbox after modifying the Python code

[llamafilm]

Deployment Type

Self-hosted

NetBox Version

v3.7.0

Python Version

3.10

Steps to Reproduce

1. Create a local data source with URL like file:///etc/netbox/config/reports
2. Create a file there with contents from this example: https://github.com/netbox-community/customizations/blob/master/reports/ipam-reports/ip-duplicate.py
3. Add a new report module from this file via the local data source
4. Delete the report module
5. Modify the Python code to remove the UniquePrefixReport class
6. Resync the data source
7. Add a new report module from the same Python file

Expected Behavior

It should show only one report for UniqueIPReport

Observed Behavior

It still shows both reports.

Trying to view the source results in an error. /extras/reports/ip_duplicate/UniquePrefixReport/source/

[markkuleinio]

The same happens with scripts: if the class name is changed, then there are two scripts listed, but only the "current one" works.

Just to be sure, on 3.7.1 as well:

1. Create data source:

• Name = Local scripts
• Type = Local
• URL = file:///opt/netbox/netbox/scripts/

2. Sync the data source
3. Create this script file "Tunnel.py" in the scripts directory (with owner netbox:netbox):

from extras.scripts import Script

class NewTunnelScript(Script):

    class Meta:
        name = "New VPN tunnel"

    def run(self, data, commit):
        self.log_success("OK")

4. Go to Customization - Scripts - Add:

• Data source = Local scripts
• File = Tunnel.py
• select Auto sync enabled
• Create

5. See that there is script "Tunnel", with name "New VPN tunnel" in it, linking to [...]/NewTunnelScript/
6. Edit the script file (in the filesystem): change class NewTunnelScript to NewTunnel
7. Refresh the Scripts page
8. See that now you have two "New VPN tunnel" names in the list, one links to /NewTunnel/ (correct), another links to /NewTunnelScript/ (does not exist anymore)
9. Click Delete
10. Recreate the same script: use steps 3 and 4 above BUT use the class name NewTunnel this time right from the start
11. See that there are still two names called "New VPN tunnel", one links to /NewTunnel/ (correct), another links to /NewTunnelScript/ (does not exist anywhere)

--> Looks like the script classes are saved to some other memory as well, not just in the script files.

[markkuleinio]

Apparently restarting NetBox services after deleting the script helps: then only the correct class is shown in the list when the script is added again.

[kkthxbye-code]

@markkuleinio - This was one of the silent removals in 3.5 with the new script system. Pre-3.5 there was code present to ensure that the sys.modules cache was cleared on script load, however this code was removed. So I assume it's intentional not supporting reloading scripts without restarting netbox anymore.

netbox/netbox/extras/scripts.py

Lines 545 to 554 in 72767fb

	# Deleting from sys.modules needs to done behind a lock to prevent race conditions where a module is
	# removed from sys.modules while another thread is importing
	with lock:
	for module_name in list(sys.modules.keys()):
	# Everything sharing a base module path with a module in the script folder is removed.
	# We also remove all modules with a base module called "scripts". This allows modifying imported
	# non-script modules without having to reload the RQ worker.
	module_base = module_name.split(".")[0]
	if module_base == "scripts" or module_base in modules_bases:
	del sys.modules[module_name]

[jeremystretch]

So I assume it's intentional not supporting reloading scripts without restarting netbox anymore.

No, this is not intentional, and you can very easily see that uploading new reports/scripts works without needing to restart any services. @kkthxbye-code since you've already tracked this down perhaps you'd like to put in a PR?