airflow should show where it's looking for templated file when jinja raises the exception jinja2.exceptions.TemplateNotFound:

[tronlightracer]

Apache Airflow version

2.2.5 (latest released)

What happened

When running this dag:

from airflow.models import DAG
from airflow.operators.python import PythonOperator
from airflow.models.renderedtifields import RenderedTaskInstanceFields as rtif
from airflow import settings

import datetime


def template_ext_func(**context):
        file = context['templates_dict']['filename']
        print(file)

with DAG(
    dag_id="python_template_searchpath_bug",
    start_date=datetime.datetime(2021, 1, 1),
    schedule_interval=datetime.timedelta(days=365),
    render_template_as_native_obj=True,
    tags=["core", "python-operator"]
) as dag:

    py0 = PythonOperator(
        task_id="templates_dict_test",
        python_callable=template_ext_func,
        templates_dict={"filename": "/usr/local/airflow/include/python_template_exts.txt"},
        templates_exts=[".txt"]
    )

I get the exception of jinja2.exceptions.TemplateNotFound: /usr/local/airflow/include/python_template_exts.txt
However when I add a template searchpath of '/' to the dag's parameters the template is found by airflow. It seems like airflow is prepending the default location of the template_searchpath to the absolute path of the filename.

What you think should happen instead

I think if an absolute path is provided as a filename with templates_exts=[".txt"] then airflow should either be able to find the templated file or it should recognize that an absolute path was passed in as the filename or it should show where airflow is looking for the templated file.

How to reproduce

No response

Operating System

Docker (debian:buster)

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

Using the Astro CLI with this image:
FROM quay.io/astronomer/ap-airflow:2.2.5-onbuild

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

jinja doesn't include searchpath in the error message. The exception probably needs to be caught at Airflow side to reraise with searchpath appended.

diff --git a/airflow/models/abstractoperator.py b/airflow/models/abstractoperator.py
index 2c53a8b75..e70b37001 100644
--- a/airflow/models/abstractoperator.py
+++ b/airflow/models/abstractoperator.py
@@ -382,7 +382,13 @@ class AbstractOperator(LoggingMixin, DAGNode):
 
         if isinstance(value, str):
             if any(value.endswith(ext) for ext in self.template_ext):  # A filepath.
-                template = jinja_env.get_template(value)
+                from jinja2.exceptions import TemplateNotFound
+
+                try:
+                    template = jinja_env.get_template(value)
+                except TemplateNotFound as e:
+                    message = f"{value} not found in search path {jinja_env.loader.searchpath}"
+                    raise TemplateNotFound(message)
             else:
                 template = jinja_env.from_string(value)
             dag = self.get_dag()

jinja2.exceptions.TemplateNotFound: /tmp/a/python_template_exts.txt not found in search path ['/opt/airflow/tests/operators', '/']

[github-actions]

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.