Bounded Concurrency

JavaScript allows for launching unlimited async tasks, which can cause system failures by overwhelming resources. One example is when promises queue up faster than they resolve, which can lead to memory exhaustion. "Bounded concurrency" is one way to limit the number of tasks you do concurrently, to avoid severe performance degradation and system instability due to resource exhaustion.

Worker Pool

Why this works

• i is shared state across workers.
• Each worker loops: grab next task index, await it, repeat.
• At most n tasks are pending because you only have n workers actively awaiting (the n that are created from your array)

var promisePool = async function(functions, n) {
    let i = 0;

async function worker() {
        while (i < functions.length) {
            const work = functions[i++];
            await work();
        }
    }

return await Promise.all(
      Array.from({length: n}, worker)
    )
};

Notice the use of an index i that moves along the queue.

This is better than cloning functions into a local workPool (which takes up extra space and time), then shift() from that pool (which is an O(n) operation).

Potential weaknesses

• n edge case: if n <= 0, you create 0 workers and Promise.all([]) resolves immediately (wrong). In real code you’d guard: n = Math.max(1, n).

• Fairness/ordering misconception: it starts tasks in index order, but completion order is naturally whatever finishes first. That’s fine for this problem.

• Busy workers are “serialized”: each worker runs tasks sequentially (by design). This is correct, but if your tasks internally spawn work and resolve immediately, it can appear like “more than n started quickly” depending on what “pending” means to you.

• Shared mutable state: it’s safe in JS because the i++ happens synchronously per tick, but some people get nervous about shared state; still fine here.

Alternative approach

Instead of creating fixed workers, we:

• Start up to n promises.
• Whenever one finishes, immediately start the next.
• Repeat until all functions are executed.
• Resolve once nothing is left running.

It’s a “fill the pool → wait for one to finish → refill” loop.

var promisePool = async function(functions, n) {
  let i = 0;
  const inFlight = new Set();

const launch = () => {
    if (i >= functions.length) return;
    const p = functions[i++]().then(() => inFlight.delete(p));
    inFlight.add(p);
  };

while (i < functions.length || inFlight.size) {
    while (i < functions.length && inFlight.size < n) launch();
    if (inFlight.size) await Promise.race(inFlight);
  }
};

When we start a promise p we immediately store it with inFlight.add(p) while it executes. then() executes when the promise has resolved and deletes the promise from inFlight

Potential weaknesses

• Same n <= 0 issue: if n is 0, you deadlock / infinite loop-ish (you’ll never launch, inFlight stays empty, loop condition may still be true). Guarding n matters more here.
• Slightly more overhead: Promise.race(inFlight) each cycle + maintaining a Set is a bit heavier than the worker approach (usually negligible).
• Memory retention risk if you forget cleanup: the pattern relies on then(() => inFlight.delete(p)). If you omit it (common bug), the set grows and you can hang forever.
• If rejections exist (not in this problem): Promise.race would reject as soon as the first reject happens unless you wrap tasks to always resolve. The worker version would also reject via Promise.all, but the failure mode differs.

Limiting concurrent operations

p-limit can help limit the number of concurrent operations.

concurrency defines how many tasks can be executed simultaneously.

• os.cpus().length * 2 assumes 2 threads per core
• -1 is sometimes used to leave room for other tasks
• os.availableParallelism() is recommended, as os.cpus().length might report more cores than the process can actually use (esp eith containerized environments)

import pLimit from 'p-limit';

// assumes 2 threads per core
const concurrency = Math.min(8, os.availableParallelism());

const limit = pLimit(concurrency);

const input = [
	limit(() => fetchSomething('foo')),
	limit(() => fetchSomething('bar')),
	limit(() => doSomething())
];

// Run and wait for tasks to complete
const result = await Promise.all(input);
console.log(result);

limit accepts a function that executes an async function. input is an array of limits, which is passed into Promise.all. But instead of executing all the tasks at once, pLimit manages task execution.

Or, if you want, you can roll your own simple semaphore.

Producer/Consumer Model

function* jobGenerator(...) {
  for(...of...) {
    yield { jobArgs }
  }
}

const jobs = jobGenerator(...);

async function worker() {
  while (true) {
    const { value, done } = jobs.next();
    if (done) return;

const { jobArgs } = value;
    await job(jobArgs);
  }
}

await Promise.all(Array.from({ length; concurrency }, worker));