Wrong line number for UndefinedError exceptions

[terminalmage]

When using jinja2.StrictUndefined and rendering a template, the line numbers seem to be off when an UndefinedError exception is triggered:

Python 2.7.5 (default, Sep  6 2013, 09:55:21)
[GCC 4.8.1 20130725 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> template = '''Hello world!
... {{ missing }}
... {{ exists }}
... '''
>>> jinja_env = jinja2.Environment(undefined=jinja2.StrictUndefined)
>>> print jinja_env.from_string(template).render(exists='foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 3, in top-level template code
jinja2.exceptions.UndefinedError: 'missing' is undefined

The missing variable is on line 2, not line 3. However, when the variables are properly defined, the value for the missing variable is on line 2 like you would expect.

>>> print jinja_env.from_string(template).render(exists='foo', missing='bar')
Hello world!
bar
foo
>>>

The same line number mismatch is seen when loading the template from a file, as well:

Python 2.7.5 (default, Sep  6 2013, 09:55:21)
[GCC 4.8.1 20130725 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> jinja_env = jinja2.Environment(loader=jinja2.FileSystemLoader('/home/erik'), undefined=jinja2.StrictUndefined)
>>> print jinja_env.get_template('template.txt').render(exists='foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/erik/template.txt", line 3, in top-level template code
    {{ exists }}
jinja2.exceptions.UndefinedError: 'missing' is undefined

So, missing is undefined, yet the traceback shows the exists line.

The above testing was done using version 2.7.1.

[terminalmage]

Any ideas?

[Naddiseo]

I use something like this:

class UndefinedVar(jinja2.Undefined):
    def __getattribute__(self, name, *args, **kwargs):

        try:
            return super(UndefinedVar, self).__getattribute__(name, *args, **kwargs)
        except AttributeError:
            if settings.TEMPLATE_DEBUG:
                import inspect
                f = inspect.currentframe().f_back.f_back.f_code
                file_name = f.co_filename
                lineno = f.co_firstlineno
                warnings.warn("[{}:{}] Trying to access attribute '{}' on undefined variable '{}'".format(file_name, lineno, name, self._undefined_name))
            return UndefinedVar()

        return None

[terminalmage]

@Naddiseo Can you provide a more complete example? I don't see how this fits into my example.

[terminalmage]

Nevermind, after looking at the jinja2 source code it seems like when I instantiate the class I just need to use undefined=UndefinedVar when instantiating a jinja2.Environment class instance.

[terminalmage]

Has there been any progress in resolving this bug?

[Naddiseo]

The reason this issue is so hard to fix is because jinja compiles the templates to python code, so when there's an exception you get the traceback of the compiled code. One (ugly) approach I can think of is to use the backtrace data to get the python line number, then use the linemapping data that jinja puts in the same file to try and calculate what the template line number would be.

[rbjorklin]

@terminalmage I'm not sure I understand this issue and actually stumbled over here from this saltstack issue where I seem to get the line number as can be seen in my comment. Is this what you were looking for?

[terminalmage]

@rbjorklin I think you misread this issue. The problem isn't that jinja doesn't give a line number (a perusal of my original post will show you that it does), the problem is that it gives the wrong line number, for reasons upon which others have elaborated.

[quantus]

I think I hit this same issue with Raven where the stack information collection fails if the call stack contains jinja templates. The wrong line numbers cause inspect module functions raise unexpected exceptions. I reported the issue there: getsentry/raven-python#1133

[terminalmage]

@davidism Thanks!