[Infrastructure] Timeout-Free Asynchronous Processing: Implementation Patterns for Long-Running Batch Jobs using Vercel Workflows

The Wall of "Execution Time" Plaguing Serverless

Serverless architectures (such as Vercel Functions and AWS Lambda) possess the tremendous advantages of high scalability and millisecond-based billing, but they also have a major design constraint: "They can only perform short-running processes."

For example, Vercel's standard plans have the following limits:

• Hobby (Free) Plan: API timeout max 10 seconds
• Pro Plan: API timeout max 60 seconds (Can be extended up to 30 minutes via configuration)

This restriction becomes a massive wall when executing "heavy" B2B tasks such as:

• Large-Scale Data Migrations: Transferring tens of thousands of customer records to a different DB.
• AI Batch Summarization: Sequentially reading hundreds of documents, summarizing them, and creating PDFs.
• Sequential Synchronization with External APIs: Slowly polling data while adhering to a partner system's connection limits.

If these are run with standard APIs, timeouts occur mid-process, stopping the system with data in an incomplete state. Fundamentally clearing this "execution time limit" requires step execution and state management via Vercel Workflows.

---

2. Before & After: Resolving the Challenges

Item	Traditional Serverless Functions (Before)	Vercel Workflows (After)
Allowable Execution Time	Forced shutdown in 10 seconds to a few minutes	Capable of executing workflows lasting hours to days by dividing into steps
Retry Control	Must restart entirely from the beginning when an error occurs	Automatically retries only the failed steps (with backoff)
Temporary Wait (Sleep)	Server uptime costs accrue even while waiting	context.sleep temporarily suspends execution (zero cost)
Progress Visualization	It is unclear what is happening in the background	Real-time visibility of completed steps from the dashboard

---

3. Concrete Implementation Example: Data Migration Batch Divided into Steps

Below is a TypeScript implementation example of a workflow that sequentially migrates a massive amount of data in small chunks, automatically retrying upon failure.

import { workflow } from "@vercel/workflows";

export const dataMigrationWorkflow = workflow(async (context) => {
  // Fetch the list of target item IDs to migrate (Step 1)
  const itemIds = await context.run("fetch-target-items", async () => {
    const response = await fetch("https://api.source-db.com/items");
    const data = await response.json();
    return data.ids as string[];
  });

  const batchSize = 50;
  
  // Divide the item list into batches of 50 and process sequentially
  for (let i = 0; i < itemIds.length; i += batchSize) {
    const chunk = itemIds.slice(i, i + batchSize);
    const chunkIndex = Math.floor(i / batchSize);

    // Execute each batch's processing as an independent step (Step 2–N)
    await context.run(`process-batch-${chunkIndex}`, async () => {
      const response = await fetch("https://api.destination-db.com/batch-import", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ ids: chunk }),
      });

      if (!response.ok) {
        // Errors thrown here are automatically retried based on Vercel Workflows settings
        throw new Error(`Batch ${chunkIndex} failed with status: ${response.status}`);
      }
    });

    // Wait for 10 seconds per batch, respecting the destination API's rate limits (wait time is not billed)
    await context.sleep(`sleep-after-batch-${chunkIndex}`, 10);
  }

  // Notify the system administrator after migration completion (Final Step)
  await context.run("notify-admin", async () => {
    await sendEmail (Resend)Notification(`🎉 The data migration workflow for a total of ${itemIds.length} items has successfully completed.`);
  });
});

async function sendEmail (Resend)Notification(message: string) {
  await fetch(process.env.Email (Resend)_WEBHOOK_URL!, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ text: message }),
  });
}

---

4. HadayaLab's Long-Running Processing Backend Construction

At HadayaLab, we specialize in developing long-running batches in serverless environments and highly fault-tolerant backends that arise when advancing "Business Process Re-engineering (BPR)" for mid-sized enterprises.

• Design and Introduction of Vercel Workflows: Investigating existing heavy processes and breaking them down into optimal workflow steps.
• Highly Fault-Tolerant External API Connections: Queue designs to avoid rate limits and prevent webhook loss.
• Building State Monitoring Dashboards: Visualization support to track whether running automated tasks are progressing normally at a glance from a dashboard.

If you are an IT leader troubled by issues like "existing batch processes frequently interrupt due to errors" or "automation is struggling due to API limits," please take advantage of HadayaLab's serverless infrastructure construction support.