TaskPool.asyncBuf

Given a source range that is expensive to iterate over, returns an input range that asynchronously buffers the contents of source into a buffer of bufSize elements in a worker thread, while making previously buffered elements from a second buffer, also of size bufSize, available via the range interface of the returned object. The returned range has a length iff hasLength!S. asyncBuf is useful, for example, when performing expensive operations on the elements of ranges that represent data on a disk or network.

auto asyncBuf(S source, size_t bufSize)
class TaskPool
auto
asyncBuf
(
S
)
(
S source
,
size_t bufSize = 100
)
if (
isInputRange!S
)
auto asyncBuf(C1 next, C2 empty, size_t initialBufSize, size_t nBuffers)

Examples

import std.conv, std.stdio;

void main()
{
    // Fetch lines of a file in a background thread
    // while processing previously fetched lines,
    // dealing with byLine's buffer recycling by
    // eagerly duplicating every line.
    auto lines = File("foo.txt").byLine();
    auto duped = std.algorithm.map!"a.idup"(lines);

    // Fetch more lines in the background while we
    // process the lines already read into memory
    // into a matrix of doubles.
    double[][] matrix;
    auto asyncReader = taskPool.asyncBuf(duped);

    foreach (line; asyncReader)
    {
        auto ls = line.split("\t");
        matrix ~= to!(double[])(ls);
    }
}

Exception Handling:

Any exceptions thrown while iterating over source are re-thrown on a call to popFront or, if thrown during construction, simply allowed to propagate to the caller.

TaskPool.asyncBuf

Examples

Meta

Source