Inability to utilise azure functions

[Richyyl]

Problem Description

The recommended method for scheduling batch shipyard jobs appears to be via azure functions, as per the following documentation https://batch-shipyard.readthedocs.io/en/latest/60-batch-shipyard-site-extension/. However, the windows function apps do not support python apps and the linux function apps do not support site extensions, as such how can azure function apps be used to schedule batch jobs?

Batch Shipyard Version

3.8.2 release

Steps to Reproduce

Follow batch shipyard instructions for creation of azure app environment with batch shipyard

Expected Results

Azure app environment that can be used to deploy python apps to schedule batch shipyard jobs

Actual Results

Windows app environment that cannot run python apps

[alfpark]

@Richyyl It looks like the set up for Azure Function Apps has changed quite a bit since I made the guide. I've made some updates to the guide which you can view in the develop branch (you may need to force-reload the images/page): https://github.com/Azure/batch-shipyard/blob/develop/docs/60-batch-shipyard-site-extension.md

You can effectively ignore the Runtime Stack selection, as Python is installed as a site extension in a later step. The important thing it to select Windows as the OS.

[Richyyl]

@alfpark Thank you for the response, I've managed to get the function app built with the two extensions, however when I try to push code from VScode into the app to run as a function it states that python code is not supported. As such how would you recommend adding python code into the apps? Is there something I'm missing / getting wrong?

[alfpark]

Unfortunately that path may be blocked as you've found (sorry I've never used VScode to push code into an app service).

You can try to use other methods as detailed in the Deployment Center (under Platform Features).

In the next minor (or major) version of Batch Shipyard, there will be an effort to showcase using the Azure Functions 2.0 runtime (#298). You can do this yourself right now if you understand how to package Batch Shipyard into a container image on top of the function runtime image.

[alfpark]

@Richyyl One other comment - I re-read your issue and you stated:

The recommended method for scheduling batch shipyard jobs appears to be via azure functions

You do not need to use Azure Functions to schedule Batch Shipyard workloads. You can simply use the tooling locally on a command line or via Azure Cloud Shell. Please see the introductory materials: Installation, Quickstart, and Step-by-step.

[Richyyl]

@alfpark ahhh that may be poor english on my part, what I'm referring to is orchestrating when batch shipyard jobs will start and stop based on timings / triggers, rather than deploying locally. Hence trying to use azure function apps, or is there a better method?

[alfpark]

In that case, that's the recommended path (assuming you need a trigger via Azure Functions). If you just need to schedule jobs on static recurrences, you can do that locally via job schedules. Search this doc for the word recurrence: https://github.com/Azure/batch-shipyard/blob/master/docs/14-batch-shipyard-configuration-jobs.md

[Richyyl]

@alfpark thanks I'll take a look! I'm assuming that the recurrence method will still mean I'll have to use an az function to delete the pool and jobs once its complete so as not to incur additional costs for the vm etc? Also thank you so much for the help, I'll make sure to write some of this up as docs if that helps?

[alfpark]

You can utilize one of two methods to remove or scale down pools when you're done without having the assistance of an Azure Function trigger:

1. Use autopools. Search for auto_pool in this doc: https://github.com/Azure/batch-shipyard/blob/master/docs/14-batch-shipyard-configuration-jobs.md. You'll want to make sure you select the correct pool_lifetime here so your resource is reclaimed after each scheduled job instance, or for the entire job schedule lifetime.
2. Use pool autoscale. See this guide: https://github.com/Azure/batch-shipyard/blob/master/docs/30-batch-shipyard-autoscale.md