Upgrade to Chameleon 4.6.0.

[mauritsvanrees]

We should use latest Chameleon (only possible on Python 3.9+) to avoid deprecation warnings for some changes in the ast module. In Python 3.14 these will become errors.
See chameleon PR.

Zope itself also uses some deprecated ast code in Products.PageTemplates. And z3c.pt does it as well. Maybe it needs to be fixed there first:

...Zope/src/Products/PageTemplates/engine.py:312: DeprecationWarning:
  ast.Str is deprecated and will be removed in Python 3.14;
  use ast.Constant instead
    type=ast.Str(self.type),
.../Zope/src/Products/PageTemplates/engine.py:313: DeprecationWarning:
  ast.Str is deprecated and will be removed in Python 3.14;
  use ast.Constant instead
    expression=ast.Str(self.expression),
...z3c/pt/expressions.py:178: DeprecationWarning:
  ast.Str is deprecated and will be removed in Python 3.14;
  use ast.Constant instead
    component = ast.Str(part)

At any rate the Zope tests currently fail with latest Chameleon:

Failure in test testBatchIteration (Products.PageTemplates.tests.testChameleonTalesExpressions.ChameleonTalesExpressionTests.testBatchIteration)
Traceback (most recent call last):
  File "/Users/maurits/.pyenv/versions/3.13.1/lib/python3.13/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/Users/maurits/.pyenv/versions/3.13.1/lib/python3.13/unittest/case.py", line 651, in run
    self._callTestMethod(testMethod)
  File "/Users/maurits/.pyenv/versions/3.13.1/lib/python3.13/unittest/case.py", line 606, in _callTestMethod
    if method() is not None:
  File "/Users/maurits/community/plone-coredev/6.2/src/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 177, in testBatchIteration
    self.assert_expected(self.folder.t, 'CheckBatchIteration.html')
  File "/Users/maurits/community/plone-coredev/6.2/src/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 96, in assert_expected
    assert not t._v_errors, 'Template errors: %s' % t._v_errors
AssertionError: Template errors: ['Compilation failed', 'builtins.IndentationError: unexpected indent (85fd4b83cc53a097d3f8a169b0279653.py, line 5)']

I did not yet manage to get that generated python file and see what the source code is.

[icemac]

According to https://chameleon.readthedocs.io/en/latest/configuration.html either CHAMELEON_CACHE and/or CHAMELEON_DEBUG should do the trick to inspect the contents of that file.

[dataflake]

I am working in a sandbox with CHAMELEON_DEBUG and CHAMELEON_CACHE pointing to a folder where the compiled sources are saved.

The errors are all the same it seems. There are supposedly indentation errors towards the top of the file where the import statements are. But when I look at the compiled source they look fine, see example below. The only oddity I can see is that every one of the supposedly faulty source files has a whole bunch of identical from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER imports. At the bottom I have copied part of a diff comparing the Python output for the failing file between Chameleon 4.4.4 (the last working version) and 4.6.0 -

Example error:

Sorry: IndentationError: unexpected indent (167832ff5f17ce388c229d45458f835d.py, line 8)


Failure in test testBatchIteration (Products.PageTemplates.tests.testChameleonTalesExpressions.ChameleonTalesExpressionTests.testBatchIteration)
Traceback (most recent call last):
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 651, in run
    self._callTestMethod(testMethod)
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 606, in _callTestMethod
    if method() is not None:
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 177, in testBatchIteration
    self.assert_expected(self.folder.t, 'CheckBatchIteration.html')
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 96, in assert_expected
    assert not t._v_errors, 'Template errors: %s' % t._v_errors
AssertionError: Template errors: ['Compilation failed', 'builtins.IndentationError: unexpected indent (167832ff5f17ce388c229d45458f835d.py, line 8)']

The top of the mentioned file 167832ff5f17ce388c229d45458f835d.py looks like this:

# -*- coding: utf-8 -*-
# template: <string>
#
__filename = '<string>'

__tokens = {31: ('modules/ZTUtils', 2, 24), 48: (' python:ztu.Batch(range(10), 5', 2, 41), 103: ('b', 3, 21), 146: ('n', 4, 39), 188: ('b/next', 6, 21), 236: ('n', 7, 39)}
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
from sys import exc_info as _exc_info
from Products.PageTemplates.engine import _compile_zt_expr as __compile_zt_expr
from Products.PageTemplates.engine import _C2ZContextWrapper as __C2ZContextWrapper
_static_4589148944 = {}
...

Partial diff comparing Python output between Chameleon 4.4.4 and 4.6.0 (diff between 4.4.4 and 4.5.0 as the first failing version is similar):

--- chameleon444/38eaa4e4a325ebb3b85247942ea63989.py	2025-02-21 09:04:11
+++ chameleon460/167832ff5f17ce388c229d45458f835d.py	2025-02-21 09:03:36
@@ -4,15 +4,31 @@
 __filename = '<string>'

 __tokens = {31: ('modules/ZTUtils', 2, 24), 48: (' python:ztu.Batch(range(10), 5', 2, 41), 103: ('b', 3, 21), 146: ('n', 4, 39), 188: ('b/next', 6, 21), 236: ('n', 7, 39)}
-
-from Products.PageTemplates.expression import BoboAwareZopeTraverse as _BoboAwareZopeTraverse
-from sys import exc_info as _exc_info
 from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
-from AccessControl.cAccessControl import guarded_getattr as _guarded_getattr
-
-_static_4432424208 = _BoboAwareZopeTraverse()
-_static_4419559312 = {}
-
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER
+from sys import exc_info as _exc_info
+from Products.PageTemplates.engine import _compile_zt_expr as __compile_zt_expr
+from Products.PageTemplates.engine import _C2ZContextWrapper as __C2ZContextWrapper
+_static_4416692816 = {}

[dataflake]

One more data point: When I unset CHAMELEON_DEBUG so that the Python output is not re-generated and then edit the file to remove the duplicated import statements from chameleon.tales import DEFAULT_MARKER as _DEFAULT_MARKER I am getting an entirely different traceback:

Error in test testBatchIteration (Products.PageTemplates.tests.testChameleonTalesExpressions.ChameleonTalesExpressionTests.testBatchIteration)
Traceback (most recent call last):
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 651, in run
    self._callTestMethod(testMethod)
  File "/Users/jens/src/python/Python-3.13.2/lib/python3.13/unittest/case.py", line 606, in _callTestMethod
    if method() is not None:
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 177, in testBatchIteration
    self.assert_expected(self.folder.t, 'CheckBatchIteration.html')
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/tests/testHTMLTests.py", line 101, in assert_expected
    out = t(*args, **kwargs)
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/PageTemplate.py", line 101, in __call__
    return self.pt_render(extra_context={'options': kwargs})
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/PageTemplate.py", line 81, in pt_render
    return super().pt_render(
  File "/Users/jens/src/.eggs/zope.pagetemplate-5.1-py3.13.egg/zope/pagetemplate/pagetemplate.py", line 134, in pt_render
    return self._v_program(
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/engine.py", line 368, in __call__
    return template.render(**kwargs)
  File "/Users/jens/src/.eggs/z3c.pt-4.4-py3.13.egg/z3c/pt/pagetemplate.py", line 198, in render
    return base_renderer(**context)
  File "/Users/jens/src/.eggs/Chameleon-4.6.0-py3.13.egg/chameleon/zpt/template.py", line 359, in render
    return super().render(**_kw)
  File "/Users/jens/src/.eggs/Chameleon-4.6.0-py3.13.egg/chameleon/template.py", line 268, in render
    raise_with_traceback(exc, tb)
  File "/Users/jens/src/.eggs/Chameleon-4.6.0-py3.13.egg/chameleon/utils.py", line 52, in raise_with_traceback
    raise exc
  File "/Users/jens/src/.eggs/Chameleon-4.6.0-py3.13.egg/chameleon/template.py", line 238, in render
    self._render(
  File "/Users/jens/Desktop/chameleon460/167832ff5f17ce388c229d45458f835d.py", line 118, in render
    __value = _static_4400666448('path', 'modules/ZTUtils', econtext=econtext)(_static_4401111888(econtext, __zt_tmp))
  File "/Users/jens/src/zope/Zope/src/Products/PageTemplates/engine.py", line 142, in _compile_zt_expr
    expr = engine.types[type](type, expression, engine)
TypeError: TalesExpr.__init__() takes 2 positional arguments but 4 were given

 - Expression: "modules/ZTUtils"
 - Filename:   <string>
 - Location:   (line 2: col 24)
 - Arguments:  template: <Products.PageTemplates.tests.testChameleonTalesExpressions.ChameleonAqPageTemplate object at 0x108215e50>
               options: {'args': ()}
               nothing: None
               request: None
               modules: <zope.tales.expressions.SimpleModuleImporter object at 0x108216210>
               here: <Products.PageTemplates.tests.testHTMLTests.Folder object at 0x1072ef8c0>
               context: <Products.PageTemplates.tests.testHTMLTests.Folder object at 0x1072ef8c0>
               container: <Products.PageTemplates.tests.testHTMLTests.Folder object at 0x1072ef8c0>
               root: <Products.PageTemplates.tests.testHTMLTests.Folder object at 0x1072ef8c0>
               default: <DEFAULT>
               repeat: <Products.PageTemplates.engine.RepeatDictWrapper object at 0x10829bad0>
               loop: {}
               target_language: None
               translate: <function BaseTemplate.render.<locals>.translate at 0x1082c51c0>

[dataflake]

Another data point about the "new" traceback:

Under Chameleon 4.6.0 the expression __value = _static_4400666448('path', 'modules/ZTUtils', econtext=econtext)(_static_4401111888(econtext, __zt_tmp)) resolves to a call Products.PageTemplates.engine._compile_zt_expr('path', 'modules/ZTUtils', econtext=econtext)(_static_4401111888(econtext, __zt_tmp)). Following the call chain all the way down it ends up instantiating an object of type chameleon.tales.TalesExpr, where the __init__ only expects two positional parameters, hence the TypeError.

Under Chameleon 4.4.4 this expression looks like __value = _static_4353259472(get('modules', modules), econtext, True, ('ZTUtils', )), where _static_4353259472 is an instance of type Products.PageTemplates.expression.BoboAwareZopeTraverse, so it's doing Products.PageTemplates.expression.BoboAwareZopeTraverse()(get('modules', modules), econtext, True, ('ZTUtils', )). This works fine.

I don't know why that expression is so different between Chameleon 4.4.4 and Chameleon 4.(5|6).0.