"can't pickle module objects" - Referencing a plugin as a module object breaks caching

[m90]

• I have read the Filing Issues and subsequent “How to Get Help” sections of the documentation.
• I have searched the issues (including closed ones) and believe that this is not a duplicate.

• OS version and name: python:3.7 Docker image
• Python version: 3.7
• Pelican version: 4.5.3
• Links to plugins: https://github.com/offen/pelican-decorate-content
• Link to your source: https://github.com/offen/website

Issue

When importing a Pelican plugin as a Python module and passing its reference to the PLUGINS setting like this

from pelican_decorate_content import decorate_content

PLUGIN_PATHS = ['./plugins']
PLUGINS = [decorate_content, 'assets']

and using CACHE_CONTENT

CACHE_CONTENT = True

Pelican fails building my site with the following error:

Traceback (most recent call last):
  File "/root/.local/bin/pelican", line 8, in <module>
    sys.exit(main())
  File "/root/.local/lib/python3.7/site-packages/pelican/__init__.py", line 516, in main
    autoreload(args)
  File "/root/.local/lib/python3.7/site-packages/pelican/__init__.py", line 440, in autoreload
    pelican.run()
  File "/root/.local/lib/python3.7/site-packages/pelican/__init__.py", line 109, in run
    p.generate_context()
  File "/root/.local/lib/python3.7/site-packages/pelican/generators.py", line 681, in generate_context
    self.readers.save_cache()
  File "/root/.local/lib/python3.7/site-packages/pelican/cache.py", line 67, in save_cache
    pickle.dump(self._cache, fhandle)
TypeError: can't pickle module objects
make: *** [Makefile:47: regenerate] Error 1

which I would assume is due to the fact that caching tries to pickle the settings file, but fails to pickle the imported plugin. If I either remove the plugin or disable caching, building the site works as expected.

Is this:

• a problem with the plugin itself?
• wrong configuration on my end?
• a bug in pelican that got introduced when allowing to import plugins like this?

[avaris]

Well, neither. Placing imported plugins directly into PLUGINS was supported way before cache was introduced. Looks more like an oversight in the caching.

Edit: Oh, I forgot to add. I suspect fixing this won't be easy and might require reworking caching considerably.

[m90]

I suspect fixing this won't be easy and might require reworking caching considerably.

Yeah, this sounds like a tricky situation.

In the meantime would it be an option to document this limitation or catch such errors when trying to build a cache in order to print a message explaining what's going on?

[m90]

For the record, we currently work around this by vendoring the plugin into the repository and passing a string reference to PLUGINS instead:

PLUGIN_PATHS = ['./plugins']
PLUGINS = ['decorate_content', 'assets']

which has caching work as expected.

[avaris]

Hmm, actually it might be possible to "cheat" a little and replace module object with a string after registering.