Python Async/Await Deep Dive 2026: The Complete Problem-Solving Guide
By Huzi
File #536 · 3 min read
Your script fetches a hundred API endpoints one by one, each requests.get() politely waiting for the network before the next line runs. Fifty seconds later you have your data and a chai break you never asked for.
The quick answer: with asyncio and an async HTTP client, the same loop finishes in roughly two seconds — same machine, same bandwidth. Learn asyncio.run(), TaskGroup, httpx.AsyncClient and asyncio.to_thread(), keep every coroutine non-blocking, and remember async only pays for I/O-bound work.
One loop, many waits
The event loop is a patient waiter: a while-loop asking the operating system, thousands of times a second, which of your sockets are ready, then resuming whichever coroutine was waiting on that one. While one coroutine waits on the network, the loop moves to the next — single-threaded concurrency.
You rarely touch the loop directly: asyncio.run() creates it, runs your main coroutine, and tears it down. The one rule that matters: never block it — a time.sleep(5) inside a coroutine freezes every other coroutine for five seconds.
import asyncio
async def fetch(url: str) -> str:
await asyncio.sleep(0.5) # standing in for network I/O
return f"data from {url}"
async def main() -> None:
sequential = [await fetch(f"url-{i}") for i in range(5)] # 2.5s
concurrent = await asyncio.gather(*(fetch(f"url-{i}") for i in range(5))) # ~0.5s
print(len(sequential), len(concurrent))
asyncio.run(main())
async def defines a coroutine; await suspends it. asyncio.gather runs them concurrently and returns results in order — the workhorse.
TaskGroup is the modern default
For anything serious, prefer asyncio.TaskGroup (Python 3.11 onward). A bare asyncio.create_task() returns a Task you must reference yourself, or the garbage collector may reap it mid-flight.
TaskGroup holds the references, and when one task raises, it cancels the siblings instead of letting them limp on — structured concurrency with honest tracebacks.
async def main() -> None:
async with asyncio.TaskGroup() as tg:
t1 = tg.create_task(fetch("a"))
t2 = tg.create_task(fetch("b"))
print(t1.result(), t2.result())
Pro tip: Forgetting the
awaitis the classic beginner bug — Python warns "coroutine was never awaited" and your code silently does nothing. Treat that warning as a red light, not noise.
HTTP, databases, and the escape hatch
For HTTP, httpx is the sensible default: one API for sync and async, HTTP/2 built in, the same client FastAPI uses in tests. Keep aiohttp on the shelf for its mature WebSocket client.
The session is the performance win — one pooled set of connections reused across requests. A scraper should wrap it in asyncio.Semaphore(10) so a single host never gets hammered. For PostgreSQL, asyncpg is the fast raw driver; SQLAlchemy 2.0's AsyncSession adds the ORM on top.
async with httpx.AsyncClient(http2=True, timeout=10) as client:
responses = await asyncio.gather(*(client.get(u) for u in urls))
resp = await asyncio.to_thread(requests.get, "https://slow.api/")
If one flaky endpoint must not poison the batch, pass return_exceptions=True and log the exceptions while keeping the survivors.
Picking the right pattern
| Pattern | Reach for it when | Watch out for |
|---|---|---|
asyncio.gather |
N independent waits, ordered results | Exceptions poison the batch unless return_exceptions=True |
TaskGroup |
Anything serious on Python 3.11+ | Cancels siblings on failure — usually what you want |
Semaphore(10) |
Scraping one host | Set it too high and you are the incident |
to_thread |
Stuck with a blocking library | Threads cost memory — pool them |
| Multiprocessing | Pure CPU-crunching loops | The GIL means async never touches the second core |
Async wins for thousands of I/O-bound waits on one thread; threads rescue blocking libraries; multiprocessing is the only model that uses extra CPU cores.
Concurrency is a travel desk's whole life — one agent, forty files in the air, nobody on hold. We run HTG Travels like an event loop: visa files, Umrah groups and rebookings all moving at once, each picked up when its turn comes.
Async patience has another face: developers in Gaza queueing work for the hours the electricity returns, coding around power cuts my slowest API has never rivalled.
Want a remote month somewhere greener? HTG Travels sorts the flights, visa file and guesthouse with load-bearing WiFi — message +92 317 5054959 and everything moves at once.




