JavaScript Iterators and Generators: Controlling Asynchronous and Lazy Evaluation

1. Introduction: Why Do We Need Iterators and Generators

In JavaScript, we deal with "iterable things" every day: arrays, strings, Map, Set, arguments, even NodeList. for...of can uniformly iterate over them, and behind the scenes it relies on the iterable protocol and iterator protocol.

When we face scenarios like "data is too large to load at once" or "asynchronous data streams in continuously", ordinary array traversal is insufficient. This is where generators and async generators provide an elegant solution: they allow functions to "pause-resume", enabling lazy evaluation and asynchronous flow control.

This article starts from the protocols, then dives into generators, yield, async generators, and provides runnable examples.

2. Iterable Protocol and Iterator Protocol

JavaScript's traversal ability is built on two protocols. Understanding them is the foundation for what follows.

The relationship between the two protocols can be understood as follows:

flowchart LR
  A[Iterable object] --> B[Returns iterator]
  B --> C[value done result]
  C --> D[Continue taking values]
  C --> E[Iteration ends]
  D --> B

2.1 Iterable Protocol

An object is an iterable as long as it implements the Symbol.iterator method and that method returns an iterator.

const arr = [1, 2, 3];
console.log(typeof arr[Symbol.iterator]); // "function"

const iter = arr[Symbol.iterator]();
console.log(iter.next()); // { value: 1, done: false }
console.log(iter.next()); // { value: 2, done: false }
console.log(iter.next()); // { value: 3, done: false }
console.log(iter.next()); // { value: undefined, done: true }

for...of, the spread operator ..., destructuring assignment, Array.from, and other syntax all essentially call the object's Symbol.iterator.

2.2 Iterator Protocol

An iterator is an object that must implement a next() method, returning a result shaped like { value, done }:

• value: the current yielded value;
• done: a boolean, true indicates iteration has finished.

function createRangeIterator(start, end) {
  let current = start;
  return {
    next() {
      if (current <= end) {
        return { value: current++, done: false };
      }
      return { value: undefined, done: true };
    },
    // Make the iterator itself iterable, so for...of can use it
    [Symbol.iterator]() {
      return this;
    },
  };
}

for (const n of createRangeIterator(1, 5)) {
  console.log(n); // 1 2 3 4 5
}

2.3 Custom Iterable Object

Combining the two protocols allows any object to support for...of:

const range = {
  from: 1,
  to: 5,
  [Symbol.iterator]() {
    let current = this.from;
    const last = this.to;
    return {
      next() {
        return current <= last
          ? { value: current++, done: false }
          : { value: undefined, done: true };
      },
    };
  },
};

console.log([...range]); // [1, 2, 3, 4, 5]

3. Generator Functions: A Cleaner Iterator Factory

Hand-writing iterators is intuitive but state management is tedious. Generator functions are declared with function*, yield values via yield internally, and return an iterator when called. The code reads as naturally as a synchronous function.

function* rangeGenerator(from, to) {
  for (let i = from; i <= to; i++) {
    yield i;
  }
}

const gen = rangeGenerator(1, 5);
console.log(gen.next()); // { value: 1, done: false }
console.log(gen.next()); // { value: 2, done: false }
console.log([...rangeGenerator(1, 5)]); // [1, 2, 3, 4, 5]

Key characteristics of generator functions:

• Calling a generator function does not immediately execute the function body; it returns an iterator;
• Each call to next() executes up to the next yield and pauses;
• After the function body completes, done becomes true.

3.1 Generator Execution Flow

flowchart TD
  A[Call gen] --> B[Return iterator object]
  B --> C[gen next]
  C --> D[Execute to yield and pause]
  D --> E{More yields?}
  E --> C
  E --> F[Return done true]

4. Deep Dive into yield: Two-Way Communication and Delegation

yield is not just "producing a value"; it is also a two-way channel: the argument passed to next(value) becomes the return value of the previous yield expression.

4.1 yield as an Expression

function* dialog() {
  const name = yield "What is your name?";
  const age = yield `Hello, ${name}! How old are you?`;
  return `${name}, ${age} years old, nice to meet you.`;
}

const it = dialog();
console.log(it.next());        // { value: "What is your name?", done: false }
console.log(it.next("Ming"));  // { value: "Hello, Ming! How old are you?", done: false }
console.log(it.next(18));      // { value: "Ming, 18 years old, nice to meet you.", done: true }

Note: the argument of the first next() is ignored, because there is no yield waiting to receive it yet.

4.2 yield* Delegation

yield* delegates iteration control to another iterable object:

function* inner() {
  yield 1;
  yield 2;
}

function* outer() {
  yield 0;
  yield* inner();
  yield 3;
}

console.log([...outer()]); // [0, 1, 2, 3]

4.3 return and throw

Generator iterators also support return() and throw() for early termination or throwing errors into the generator:

function* task() {
  try {
    yield 1;
    yield 2;
  } finally {
    console.log("Cleaning resources");
  }
}

const t = task();
console.log(t.next()); // { value: 1, done: false }
console.log(t.return(99)); // triggers finally, logs "Cleaning resources", returns { value: 99, done: true }

5. Lazy Evaluation: The Core Value of Generators

The biggest advantage of generators is compute on demand. They don't compute all results at once; each next() computes one value. This is friendly to infinite sequences and large data processing.

5.1 Infinite Sequences

function* naturals() {
  let n = 0;
  while (true) {
    yield n++;
  }
}

const nums = naturals();
console.log(nums.next().value); // 0
console.log(nums.next().value); // 1
console.log(nums.next().value); // 2

5.2 Lazy Pipelines

Chaining multiple generators builds lazy data pipelines where computation only happens when values are actually consumed:

function* map(iterable, fn) {
  for (const item of iterable) {
    yield fn(item);
  }
}

function* filter(iterable, predicate) {
  for (const item of iterable) {
    if (predicate(item)) yield item;
  }
}

function* take(iterable, n) {
  let count = 0;
  for (const item of iterable) {
    if (count++ >= n) return;
    yield item;
  }
}

const pipeline = take(
  filter(map(naturals(), (x) => x * x), (x) => x % 2 === 0),
  5
);

console.log([...pipeline]); // [0, 4, 16, 36, 64]

The example above squares infinite natural numbers, filters for evens, and takes the first 5, without any extra computation along the way.

The lazy pipeline consumption flow looks like this; data flows level by level only when consumed:

flowchart LR
  A[naturals infinite] --> B[map square]
  B --> C[filter evens]
  C --> D[take first five]
  D --> E[Consumer spreads pipeline]
  E --> A

6. Async Generators: Handling Streaming Asynchronous Data

When data arrives asynchronously (paginated APIs, WebSocket messages, file streams), ordinary generators are insufficient. Async generators are declared with async function* and used with for await...of.

The difference between synchronous and asynchronous iteration can be compared in this diagram:

flowchart TD
  A[Consumer for await of] --> B[Call asyncIterator next]
  B --> C[Returns Promise]
  C --> D[await waits for async data readiness]
  D --> E{done is true?}
  E --> F[Yield value and process]
  F --> B
  E --> G[End iteration]

6.1 Basic Usage

async function* fetchPages() {
  for (let page = 1; page <= 3; page++) {
    const data = await fakeFetch(page);
    yield data;
  }
}

function fakeFetch(page) {
  return new Promise((resolve) =>
    setTimeout(() => resolve(`Page ${page} data`), 300)
  );
}

async function main() {
  for await (const page of fetchPages()) {
    console.log(page);
  }
}

main();
// Page 1 data
// Page 2 data
// Page 3 data

6.2 Async Iterator Protocol

An async generator returns an async iterator. Its next() returns Promise<{ value, done }> and it is identified by Symbol.asyncIterator:

async function* gen() {
  yield 1;
  yield 2;
}

const asyncIter = gen();
console.log(typeof asyncIter[Symbol.asyncIterator]); // "function"

asyncIter.next().then((res) => console.log(res)); // { value: 1, done: false }

6.3 Async Lazy Pipelines

Async generators can also be composed into lazy pipelines, which is great for streaming data:

async function* asyncMap(iterable, fn) {
  for await (const item of iterable) {
    yield fn(item);
  }
}

async function* asyncFilter(iterable, predicate) {
  for await (const item of iterable) {
    if (predicate(item)) yield item;
  }
}

async function* source() {
  for (let i = 1; i <= 10; i++) {
    await new Promise((r) => setTimeout(r, 100));
    yield i;
  }
}

async function run() {
  const result = asyncFilter(
    asyncMap(source(), (x) => x * 10),
    (x) => x % 20 === 0
  );
  for await (const v of result) {
    console.log(v); // 20 40 60 80 100
  }
}

run();

7. Practical Example: Pagination with Async Generators

Here is a more realistic business example: fetching pages until there is no more data, with lazy handling on the consumer side.

async function* paginate(fetchPage, pageSize = 10) {
  let page = 1;
  while (true) {
    const list = await fetchPage(page, pageSize);
    if (!list || list.length === 0) return;
    for (const item of list) {
      yield item;
    }
    if (list.length < pageSize) return;
    page++;
  }
}

// Mock API
async function fetchPage(page, size) {
  await new Promise((r) => setTimeout(r, 200));
  if (page > 3) return [];
  return Array.from({ length: size }, (_, i) => `item-${(page - 1) * size + i + 1}`);
}

async function consume() {
  let count = 0;
  for await (const item of paginate(fetchPage, 5)) {
    console.log(item);
    if (++count >= 8) break; // Consumer can interrupt anytime to avoid extra requests
  }
}

consume();

The benefit of this pattern: the consumer pulls on demand. Once break happens, the generator's internal loop also terminates, so it won't keep issuing meaningless requests.

The collaboration between pagination fetching and consumer interruption is shown here:

sequenceDiagram
  participant C as Consumer consume
  participant G as paginate generator
  participant API as fetchPage API
  C->>G: for await requests next item
  G->>API: Fetch first page
  API-->>G: Return list
  G-->>C: yield single item
  C->>G: Continue requesting
  G->>API: Fetch next page
  API-->>G: Return list
  G-->>C: yield item
  C->>G: count reaches 8 triggers break
  G->>G: Terminate loop, no more requests

8. Common Pitfalls and Best Practices

• Generators are single-use: once consumed, an iterator cannot be reset; you must call the generator function again.
• Don't forget cleanup logic in return: release resources in try/finally. Both return() and break trigger finally.
• Avoid heavy side effects inside generators: lazy evaluation means execution timing is uncertain; be careful with side effects.
• Error handling in async generators: wrap for await...of in try/catch, or let the generator throw internally.
• Performance considerations: generators have extra call overhead; don't force them on very small data volumes.

async function safeConsume() {
  try {
    for await (const item of paginate(fetchPage, 5)) {
      console.log(item);
    }
  } catch (err) {
    console.error("Fetch failed:", err);
  }
}

9. Summary

Concept	Key Point	Typical Scenario
Iterable protocol	Implements Symbol.iterator	Custom object supports for...of
Iterator protocol	Implements next() returning { value, done }	Manual control of traversal
Generator function	function* + yield	Simplify iterator writing
yield	Two-way communication, yield* delegation	State machines, coroutine-style flows
Async generator	async function* + for await...of	Pagination, streaming data

Iterators and generators turn "traversal" from a one-shot calculation into a controllable, pausable, composable process. Mastering them lets you write more elegant lazy evaluation code and handle asynchronous data flows with ease.