Python Async Comprehension

Asynchronous comprehensions extend standard Python comprehensions by allowing iteration over asynchronous iterables using async for, and the evaluation of awaitables using the await keyword, within list, set, dictionary comprehensions, and generator expressions. Introduced in PEP 530, they enable concise syntax for consuming and transforming asynchronous data streams.

Syntax Variations

Asynchronous comprehensions manifest in two primary forms, which can also be combined: 1. Asynchronous Iteration (async for) Used when the source data is an asynchronous iterable (an object implementing __aiter__ and __anext__).

[expression async for item in async_iterable if condition]

2. Awaiting Expressions (await) Used when the transformation expression yields an awaitable (like a coroutine or Task), even if the underlying iterable is synchronous.

[await expression for item in iterable if condition]

3. Combined Used when both the iterable is asynchronous and the transformation requires awaiting.

[await expression async for item in async_iterable if condition]

Supported Data Structures

The asynchronous syntax applies to all native comprehension types:

List Comprehension: [x async for x in async_iterable]
Set Comprehension: {x async for x in async_iterable}
Dict Comprehension: {k: v async for k, v in async_iterable}
Generator Expression: (x async for x in async_iterable)

Note: An asynchronous generator expression returns an asynchronous iterator, not a standard generator. It must be consumed using async for or anext().

Execution Rules and Constraints

Context Requirement: Asynchronous comprehensions and generator expressions containing async for or await can only be defined and executed within an async def function or an asynchronous context.
Evaluation: While the comprehension builds the collection synchronously in memory, it yields control back to the event loop at each async for step or await expression.
Sequential Execution: The iterations and awaits inside the comprehension are evaluated sequentially, not concurrently. [await coro(x) for x in iterable] will await each coroutine one after the other. To execute them concurrently, asyncio.gather() must be used instead of a comprehension.

Mechanical Demonstration

The following code illustrates the mechanics of asynchronous comprehensions without relying on external I/O libraries.

import asyncio


# Asynchronous iterable (Async Generator)
async def async_number_generator():
    for i in range(3):
        yield i


# Awaitable (Coroutine)
async def async_transform(x):
    return x * 10

async def main():
    # 1. Consuming an async iterable into a list
    list_comp = [x async for x in async_number_generator()]
    # Result: [0, 1, 2]

    # 2. Awaiting a coroutine within a synchronous iteration
    await_comp = [await async_transform(x) for x in range(3)]
    # Result: [0, 10, 20]

    # 3. Combined: Consuming an async iterable and awaiting a coroutine
    combined_comp = [await async_transform(x) async for x in async_number_generator()]
    # Result: [0, 10, 20]

    # 4. Asynchronous Dictionary Comprehension
    dict_comp = {x: await async_transform(x) async for x in async_number_generator()}
    # Result: {0: 0, 1: 10, 2: 20}

    # 5. Asynchronous Generator Expression
    async_gen_expr = (x ** 2 async for x in async_number_generator())
    
    # Consuming the resulting asynchronous iterator
    gen_results = [x async for x in async_gen_expr]
    # Result: [0, 1, 4]

asyncio.run(main())