Iterator Helpers: Laziness, Single Use and Early Stop

BotBrowser supports replaying an authorized same-origin fixture under a declared profile and keeping a bounded record of what a page actually pulled from a stream, but it cannot make a consumed iterator produce values a second time, cannot turn take into a drain, and cannot stop a drop from walking the values it skips. This review runs one probe over eleven fixed cases and records when each helper pulls its first value, why the result of a helper cannot be looped over twice while the source array can, how far take and drop advance the source, what a partially consumed helper leaves behind, and how a throw inside the source propagates through the chain.

TL;DR

• A helper is lazy: after source().map(...) the probe prints lazy-before -> pulled=0, and only the first next() pulls one value (lazy-after-one -> first=10 pulled=1).
• A helper returns an iterator, not an array: the probe prints type -> isArray=false isIterator=true.
• A helper is single use: spreading it twice prints consume-twice -> first=2,4,6 second= while the array it came from prints array-reiterable -> a=1,2,3 b=1,2,3.
• take(2) stops the source early, so the probe prints take -> values=1,2 pulled=1,2 and never pulls values three to five.
• drop(3) does not drain the source up front, but its first next() walks the three skipped values: drop-after-one -> value=4 pulled=1,2,3,4.
• flatMap needs an iterable result; returning a number throws a TypeError.
• find and some stop pulling as soon as the answer is known, while reduce drains the whole source.
• A throw inside the source reaches the consumer after the values already yielded, and the chain ends there.

Contents

• When a helper pulls its first value
• Why the result is single use
• How far take and drop advance the source
• What a throw inside the source does
• Record an iterator-helpers reading

When a helper pulls its first value

An iterator helper is a wrapper that produces values on demand, and the whole point is that nothing runs until a consumer asks. The probe builds a generator that records every value it yields, wraps it in map, and checks the counter before anything asks for a value:

node=v26.9.0
lazy-before -> pulled=0
lazy-after-one -> first=10 pulled=1
type -> isArray=false isIterator=true
consume-twice -> first=2,4,6 second=
array-reiterable -> a=1,2,3 b=1,2,3
helper-reiterable -> a=2,3,4 b=
take -> values=1,2 pulled=1,2
take-stopped -> pulledCount=2
drop -> values=4,5 pulled=1,2,3,4,5
drop-before-next -> pulled=0
drop-after-one -> value=4 pulled=1,2,3,4
partial -> head=1 rest=4,9,16
flatMap -> 1,10,2,20
flatMap-non-iterable -> TypeError
find -> value=2 pulled=1,2
some -> result=true pulled=1,2,3
reduce -> total=6 pulled=1,2,3
exhausted-source -> length=0
thrown -> seen=2 error=Error:source failed
from-before -> pulled=0 hasMap=true
from-take -> values=1,2,3 pulled=1,2,3
toArray -> 3,6,9 thenNext=true
break -> seen=1,2 nextDone=true

The first two lines are the definition of the design. lazy-before -> pulled=0 means the map call itself did no work: it bound a callback to a source and returned. lazy-after-one -> first=10 pulled=1 shows that the first next() pulled exactly one value from the generator, ran the callback on it, and returned the mapped result. A chain of helpers stays lazy the same way, because each link pulls from the one before it only when the consumer pulls from the chain.

A second entry point is worth mentioning because it changes what a helper can wrap. Iterator.from takes any iterable or iterator and returns a helper-carrying iterator, which means an object that only implements Symbol.iterator can be given the same helpers a native iterator has. The probe's cases all start from a generator or an array, and both already carry the helpers, so the value of Iterator.from shows up with a custom iterable such as a cursor over a database result. A page that wraps its own cursor in Iterator.from can then cap it with take without writing a manual loop, and the same laziness applies: nothing is read from the cursor until the first pull. The probe prints from-before -> pulled=0 hasMap=true, then wraps a five-value custom iterable in take(3) and prints from-take -> values=1,2,3 pulled=1,2,3, so the cursor advanced exactly three times.

That matters for a page that reads from a large or unbounded source. A map over a generator that would take minutes to finish costs nothing to construct, and a take(10) later in the chain means only ten values are ever produced. The alternative, building an intermediate array, forces the whole source to run before the first value is available, which is the difference between a stream that starts delivering in a fraction of a second and one that blocks until the end.

The laziness is per pull, not per item in a batch. Each next() runs the source until it has one more value, so the callback for value n runs only when the consumer asks for value n. A page that interleaves its own work between pulls, such as reading from a network stream or writing a file, therefore spreads the source's cost across the consumer's turns instead of paying it in one blocking call. That interleaving is the same turn boundary microtasks and timers describes, where a callback queued as a microtask runs before the next task arrives.

The cost is that a helper carries no control over how many values the source has already produced. There is no peek, no rewind and no count, so a consumer that needs those has to build them, typically by wrapping the generator itself with a counter. The helper only adds the mapping step, and it is deliberately the thinnest possible wrapper because anything heavier would break the laziness that makes it worth using over an array method.

Why the result is single use

The second half of the design is that a helper returns an iterator rather than a collection, and the probe shows the exact consequence. It prints type -> isArray=false isIterator=true, then spreads the same helper twice and gets consume-twice -> first=2,4,6 second=.

The first spread walked the iterator to its end, and the second found it exhausted and produced nothing, with no error. That is the behaviour of every iterator in the language, and a helper inherits it because it is built on the iteration protocol rather than on a copy. Contrast the source array on the next line: array-reiterable -> a=1,2,3 b=1,2,3, because array.values() returns a fresh iterator each time it is called.

The line after that is the sharp end of the same rule. helper-reiterable -> a=2,3,4 b= shows the helper itself is not re-iterable: it keeps the single iterator it was built with, the first spread consumed it, and the second saw nothing. This is the mistake that turns a lazy pipeline into a silent empty result. A function that returns a helper and is called twice gets a dead iterator on the second call, and the failure is an empty sequence rather than an exception.

The fix is to return a function that builds a helper on each call, or to return the source and build the pipeline at the call site. A helper that is meant to be reusable has to be re-derived from its source every time, because the language gives an iterator no reset. That is the same reason a generator object cannot be looped over twice, and a page that mixes the two styles gets the empty second result whenever it assumes otherwise.

A partially filled batch of pulled values has the same shape problem as a sparse array, where a missing index and a stored undefined read the same but iterate differently. Partial consumption of a helper is the softer version of the same rule. The probe pulls one value, then spreads the rest, and prints partial -> head=1 rest=4,9,16 for a map that squares its input. The single next() returned 1 (the square of 1), and the later spread continued from where the pull stopped, returning 4, 9, 16 for the inputs 2, 3, 4. Nothing was lost and nothing was repeated, because the iterator remembers its position; a second spread would have returned nothing. A terminal helper finishes the iterator just as a spread does: the probe prints toArray -> 3,6,9 thenNext=true, so after toArray collected every value the next next() reported done.

A source that is already exhausted before a helper is built is the quiet case. The probe drains an array iterator, then builds a map over the same object and spreads it, printing exhausted-source -> length=0. There is no error and no warning: the helper was built over an iterator that had already reported done, so every pull after that returns done immediately. A page that caches an iterator and builds a helper over it later gets an empty result whenever something else consumed the iterator in between, and the fix is the same as for the double-spread case, namely to build the helper from a source that can produce a fresh iterator each time.

A loop that stops early behaves the opposite way from what the position model suggests. The probe runs a for...of over a helper, breaks at the value 2, and then asks the helper for one more value: it prints break -> seen=1,2 nextDone=true. So a break does not park the iterator for a later read; leaving the loop closes it, and the values after the break are gone. A consumer that wants to read a header, stop, and continue from the body therefore cannot use a break inside for...of on a single iterator; it has to pull with next() so the iterator is not closed, or build a fresh iterator over the source.

That matters most for the reason this whole design exists, which is bounding cost. A break is the one early exit that discards the rest, so a consumer that reads until a condition and then leaves pays only for what it read but gains nothing more from that iterator. The pull-based form keeps the option open, and the take form bounds the read without ever needing a break at all, so a pipeline that composes take is the version that both stops cleanly and leaves no work half-done.

How far take and drop advance the source

take and drop both decide how much of a source a consumer sees, and the probe shows that they advance it in opposite ways. take stops early, and drop walks the values it skips.

The take case prints take -> values=1,2 pulled=1,2 and then take-stopped -> pulledCount=2. The source generated five values in total, and after taking two, the recorded pulls are exactly the first two. take(2) did not read the remaining three to discard them; it asked for two and then stopped, which is what lets an infinite source be bounded. A page that takes a fixed number of items off an unbounded stream therefore pays for exactly that many, and the rest of the stream is never touched.

The drop case is the mirror image. The eager spread prints drop -> values=4,5 pulled=1,2,3,4,5, because spreading to the end pulled everything, and the dropped values had to be generated to be skipped. The more revealing line is the lazy one: drop-before-next -> pulled=0 shows a fresh drop(3) has pulled nothing at construction, and drop-after-one -> value=4 pulled=1,2,3,4 shows its first next() walked four values to return the fourth. So drop is lazy at construction like every helper, but the work it defers is the work of skipping, and a consumer that asks for one value after a drop(3) has already caused four pulls.

That asymmetry decides how to bound a stream. take is the safe cap because it never over-reads. drop is a filter on position, and a drop with a large count over a slow source does the slow work before the first value reaches the consumer, so a page that composes a big drop with a take still pays for the drop. A consumer that wants to skip a header and then read a body therefore pays for the header as soon as it asks for the first body value, not before it started, but not for free either.

The interaction with flatMap is worth one line of the same reading. The probe prints flatMap -> 1,10,2,20 for a flatMap that returns two values per input, so each pull from the source became two pulls to the consumer, and the flattening is one level deep rather than recursive. A flatMap that returned a nested array would hand the consumer an array as a value, not its elements, which is the same one-level rule as the array method of the same name.

flatMap also refuses to flatten a value that is not iterable. The probe's attempt to return a number prints flatMap-non-iterable -> TypeError, so a callback that sometimes returns a scalar and sometimes a list fails on the scalar instead of skipping it. A page that builds a flatMap from a callback that can return undefined therefore needs to normalize the result to an empty array or a single-element array before the helper sees it, because the helper will not treat a non-iterable as "nothing to flatten".

What a throw inside the source does

An iterator that fails partway has to do something with the values it already produced, and the probe pins the behaviour down. Its source yields 1 and then throws an Error('source failed'), and a map over it is consumed with a for...of loop:

thrown -> seen=2 error=Error:source failed

The consumer received the mapped first value, 2, recorded it, and then the next pull ran the source into the throw, which propagated out of the loop as the original error. The values before the throw are not withdrawn, and the values after it never existed. That is the contract a consumer relies on when it processes items as they arrive: everything it already handled stays handled, and the failure surfaces at the point where the source failed rather than at the start.

The error is not wrapped. The probe prints Error:source failed, which is the same constructor and message the source threw, so a chain of helpers passes the thrown value through unchanged. A page that wants to add context has to wrap the error at the point it processes items, because the helper will not add a layer that names which step in the chain failed.

The iterator is finished after the throw, so a retry on the same object produces nothing rather than re-running the source. That is the same single-use rule as the spread case, applied to a failure: the position is past the throw, and there is no way to rewind to it. A page that needs to retry has to build a new iterator from a new source, which usually means a function that constructs the generator rather than a stored generator object.

A reduce over the same source is eager, and that changes when the throw happens relative to the caller. The probe prints reduce -> total=6 pulled=1,2,3 for a three-value source, with no partial result available mid-way, so a reduce that hits a throw loses the accumulator it had built. The non-reducing helpers let a consumer see the values one at a time and act on each; a reduce collapses them into one value at the end, and a failure means that value never arrives. Choosing between the two is therefore a choice about how much partial progress the consumer needs to keep.

Record an iterator-helpers reading

An iterator-helpers reading supports a narrow claim: this runtime, this source, these helpers, produced these pulls. Four fields keep it checkable:

1. The pull count before the first next(), because laziness is the whole point and a constructed helper that has already pulled something is not lazy.
2. Whether the consumer held an array or an iterator, because a second iteration over an iterator returns nothing and over an array returns the values again.
3. The pull count at each stop, because take and drop advance the source by different amounts and only the count shows which.
4. Whether a failure arrived after the values already yielded, because a consumer that keeps its partial results depends on the earlier values surviving the throw.

Public sources

• MDN: Iterator
• MDN: Iterator.prototype.map()
• MDN: Iterator.prototype.take()
• MDN: Iterator.prototype.drop()
• MDN: Iterator.prototype.flatMap()
• ECMAScript: Iterator Helpers
• BotBrowser: automation consistency reference

Practical conclusion

Build a pipeline of helpers and pay only for the values you pull: construct the chain, cap it with take when the source is unbounded, and remember that drop walks what it skips on the first pull rather than up front. Return a function that builds the pipeline rather than a stored iterator when a caller may iterate twice, because a helper is single use and a second spread is silently empty. Normalize a flatMap callback's result to an array so a scalar cannot throw, and decide whether a consumer needs to keep the values it already handled, because a throw ends the iterator and a reduce loses its accumulator. For the ordering a microtask-based stream delivers in, read microtasks and timers; for the hole a partially filled batch leaves, read sparse array holes.