Scheduling Python Scripts using Azure Function App and Task Scheduler — Part 1
So recently I was working on a utility to migrate the documents from Azure Data Lake to a document repository via API. This utility was supposed to run after every 60 mins where it would pick the documents from the Azure Data Lake and upload it to the Document Repository. The challenge was to come up with the deployment strategy where the utility would run indefinitely after every 60 mins.
The proposed solutions were:
- Deploying it to Azure VM using window’s task scheduler
- Deploying it to Azure using Azure Function App
Let’s understand how to implement these options.
Deploying Python Scripts to Azure VM using window’s Task Scheduler
Azure Virtual Machines (VM) is one of several types of on-demand, scalable computing resources that Azure offers. An Azure VM gives you the flexibility of virtualization without having to buy and maintain the physical hardware that runs it. However, you still need to maintain the VM by performing tasks, such as configuring, patching, and installing the software that runs on it.
Since it was a windows virtual machine in Azure, it had built-in windows task scheduler. What is Windows Task Scheduler?
Windows Task Scheduler is a component that gives the ability to schedule and automate tasks in Windows by running scripts or programs automatically at a given moment.
You can either convert the .py file to batch file or directly schedule the .py file to run on a scheduled basis.
- To create a task, search for “Task Scheduler” in windows search box.
- Click on “Task Scheduler” and select “Task scheduler Library” to see the list of scheduled tasks.
- To create a new task, click on “Create Task” in the right menu. Under ‘General’ tab, add name for you task and check ‘Run with highest privileges’ and click ‘OK’
- Under ‘Triggers’ tab, create a new trigger by clicking ‘New’ and select ‘Begin the task on a schedule’. Select start time and settings as per the requirement. If you want to repeat the task after certain interval, go to ‘Advanced settings’ and select duration and time. Click ‘OK’
- Under ‘Action’ tab, create a new action by clicking on ‘New’ button. Select ‘Action’ as ‘start a program’. Under ‘Program/script’, add the path to python executable. To get the path for python executable, open cmd prompt and type ‘where python’.
- Under ‘Program/script’, add the python executable path. Add the file name under ‘Arguments’ and file path under ‘Start in’. Click ‘OK’ to continue.
- Again press ‘OK’ and your script is scheduled to run at the given date and time.
Hope this was helpful! Kindly let me know your views in the comment section and suggestions if any.
Also, in the next blog we will see how to deploy Python Scripts to Azure using Azure function App.