EmailOperator is not working on Mac with Postfix : expecting TLS

[r39132]

Why do I need TLS? This requires setting it up for any smtp server, even one I run locally.

2015-06-19 13:20:09,793 - root - INFO - Executing <Task(EmailOperator): email_pipeline_start> for 2010-10-10 00:00:00
2015-06-19 13:20:10,459 - root - ERROR - STARTTLS extension not supported by server.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 746, in run
    task_copy.execute(context=self.get_template_context())
  File "/usr/local/lib/python2.7/site-packages/airflow/operators/email_operator.py", line 36, in execute
    send_email(self.to, self.subject, self.html_content)
  File "/usr/local/lib/python2.7/site-packages/airflow/utils.py", line 339, in send_email
    send_MIME_email(SMTP_MAIL_FROM, to, msg)
  File "/usr/local/lib/python2.7/site-packages/airflow/utils.py", line 349, in send_MIME_email
    s.starttls()
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 644, in starttls
    raise SMTPException("STARTTLS extension not supported by server.")
SMTPException: STARTTLS extension not supported by server.
2015-06-19 13:20:10,503 - root - ERROR - Failed to send email to: ['sanand@agari.com']
2015-06-19 13:20:10,503 - root - ERROR - STARTTLS extension not supported by server.
2015-06-19 13:20:10,503 - root - ERROR - STARTTLS extension not supported by server.
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 10, in <module>
    args.func(args)
  File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 195, in test
    ti.run(force=True, ignore_dependencies=True, test_mode=True)
  File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 749, in run
    raise e
smtplib.SMTPException: STARTTLS extension not supported by server.

I also tested this out with our production postfix server, and that didn't work for a different reason, which is not apparent yet. Which email SMTP servers have you tried this out with? Examples would be very helpful.

[artwr]

@r39132, we mostly use Amazon Simple Email. We might introduce a way not to force STARTTLS while making sure it still works with our current setup. That said, messages sent with local postfixes usually end up in spam folders when sent to a more mainstream mail server...

You prod server should probably support STARTTLS (http://switch.richard5.net/isp-in-a-box-v2/configuring-the-mail-server-components/configure-postfix-to-use-tls/).

[martingrayson]

How hard would it be to switch to using SNS rather than SES? We currently use SNS for our internal alerting so that users can subscribe / maintain their own alerts.

[r39132]

Thanks @artwr SES should work, though now that @martingrayson mentions it, SNS would also work, though probably with its own operator, e.g. SNSOperator. Do you folks have a simple version with some simple templated html content? That would be very helpful.

[mistercrunch]

I don't know much about smtp authentication protocols, but it should be very easy to fix make work with other standard configuration supported by the python smtp lib. There's already an [smtp] section in the cfg file, you just need to add support for your protocol. Squeeze in an AUTHENTICATION_PROTOCOL entry in the cfg that takes different relevant values and that behaves accordingly, the email sending function in utils.py is 10 lines of code:
https://github.com/mistercrunch/Airflow/blob/master/airflow/utils.py#L346

That's the kind of things that we have to rely on the community to come up with since we can't set this up and test it.

[artwr]

We also already rely on boto for our s3 hooks and operators, and it seems boto has the ability to talk to SNS. This could be something to consider. Unfortunately, I have little experience with this service, and I don't have access to that infra at work to test an operator...

[r39132]

Boto makes communicating with AWS so easy. Here are examples using both SES and SNS. I send email via SES and SNS as steps 4a and 4b, respectively, once the "wait_for_empty_queue" task completes.

def _send_flow_complete_email_internal(ds, **kwargs):
    conn = boto.ses.connect_to_region("us-west-2")
    conn.send_email(
                    'your@email.com',
                    'Airflow Notification : EP Data Load Completed Successfully',
                    'This is to inform you of successful completion of the EP Data Load',
                    ['your@email.com'])
    print '------- SENT FLOW COMPLETION EMAIL'
    return

def _send_sns_notification_internal(ds, **kwargs):
    conn = boto.sns.connect_to_region("us-west-2")
    conn.publish(message='Airflow flow complete', subject='Airflow Flow Complete', target_arn='<your SNS topic ARN here>',
                )
    print '----- Sent SNS about FLOW Completion'
    return



# Operator 4a : Send email when flow completes successfully
send_flow_complete_email = PythonOperator(
    task_id='send_flow_complete_email',
    provide_context=True,
    python_callable=_send_flow_complete_email_internal,
    dag=dag)
send_flow_complete_email.set_upstream(wait_for_empty_queue)

# Operator 4b : Send SNS when flow completes successfully
send_sns_notification = PythonOperator(
    task_id='send_sns_notification',
    provide_context=True,
    python_callable=_send_sns_notification_internal,
    dag=dag)
send_sns_notification.set_upstream(wait_for_empty_queue)