[SOLVED] How long can an Azure Durable Function run for?

Issue

I have multiple ETL kind of tasks, that I plan to perform serverless. The execution time for the tasks vary from 5 to 30 minutes (depending on the amount of data coming at an instance). Since functions have a timeout of 10 minutes, these tasks cannot be performed together in one single function. I recently came across Durable Functions in Azure for orchestration of different functions. I wanted to know if Durable functions altogether also have a timeout of 10 minutes, or I can have multiple functions in it (which run from 3-5mins each).

For example, task 1 takes 3mins, task 2 takes 5 mins, task 3 takes 7 minutes, task 4 takes 3minutes and task 5 takes 2mins. Can I have all these tasks orchestrated in a single durable function?

My current approach is to have a queue trigger function for each of the tasks separately, but this kind of workflow is quiet a mess. I feel durable functions will be best for making a streamlined workflow.

Solution

Can I have all these tasks orchestrated in a single durable function?

In short: yes. But here’s some context.

Durable Functions is an extension of Azure Functions that lets you write stateful functions in a serverless compute environment. The extension lets you define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model. Behind the scenes, the extension manages state, checkpoints, and restarts for you, allowing you to focus on your business logic.

It sounds like your scenario would best fit the Function Chaining pattern.

Function chaining

In the function chaining pattern, a sequence of functions executes in a specific order. In this pattern, the output of one function is applied to the input of another function.

A diagram of the function chaining pattern

The code for this is relatively simple:

[FunctionName("Chaining")]
public static async Task<object> Run(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    try
    {
        var x = await context.CallActivityAsync<object>("F1", null);
        var y = await context.CallActivityAsync<object>("F2", x);
        var z = await context.CallActivityAsync<object>("F3", y);
        return  await context.CallActivityAsync<object>("F4", z);
    }
    catch (Exception)
    {
        // Error handling or compensation goes here.
    }
}

Each separate piece of functionality in this example (F1 to F4) is it’s own Function. Which means it has its own timeout.

Please take into account that a different type of hosting plan might be a better suited solution for your problem. Have a look at Azure Functions hosting options – Service limits to see what the options and their respective (timeout) limits are.

Answered By – rickvdbosch

Answer Checked By – Senaida (BugsFixing Volunteer)

Leave a Reply

Your email address will not be published.