[{"data":1,"prerenderedAt":3025},["ShallowReactive",2],{"page-\u002Fpython\u002F21-concurrency-threading-and-multiprocessing":3},{"id":4,"title":5,"body":6,"description":30,"extension":3019,"meta":3020,"navigation":52,"path":3021,"seo":3022,"stem":3023,"__hash__":3024},"content\u002Fpython\u002F21-concurrency-threading-and-multiprocessing.md","21 — Concurrency: Threading & Multiprocessing",{"type":7,"value":8,"toc":2988},"minimark",[9,13,227,370,597,695,769,1014,1178,1349,1586,1771,1984,2149,2327,2377,2577,2636,2823,2984],[10,11,5],"h1",{"id":12},"_21-concurrency-threading-multiprocessing",[14,15,17,22,169,202],"question-wrapper",{"language":16},"python",[18,19,21],"h3",{"id":20},"q1-what-does-the-cpython-global-interpreter-lock-gil-actually-restrict","Q1. What does the CPython Global Interpreter Lock (GIL) actually restrict?",[23,24,25],"code-wrapper",{"language":16},[26,27,31],"pre",{"className":28,"code":29,"language":16,"meta":30,"style":30},"language-python shiki shiki-themes github-light github-dark","import threading\n\ndef cpu_heavy():\n    total = 0\n    for i in range(50_000_000):\n        total += i\n\nt1 = threading.Thread(target=cpu_heavy)\nt2 = threading.Thread(target=cpu_heavy)\nt1.start(); t2.start()\nt1.join(); t2.join()\n","",[32,33,34,47,54,67,80,104,116,121,141,157,163],"code",{"__ignoreMap":30},[35,36,39,43],"span",{"class":37,"line":38},"line",1,[35,40,42],{"class":41},"svdQ7","import",[35,44,46],{"class":45},"ssxIu"," threading\n",[35,48,50],{"class":37,"line":49},2,[35,51,53],{"emptyLinePlaceholder":52},true,"\n",[35,55,57,60,64],{"class":37,"line":56},3,[35,58,59],{"class":41},"def",[35,61,63],{"class":62},"sIsaT"," cpu_heavy",[35,65,66],{"class":45},"():\n",[35,68,70,73,76],{"class":37,"line":69},4,[35,71,72],{"class":45},"    total ",[35,74,75],{"class":41},"=",[35,77,79],{"class":78},"snvgF"," 0\n",[35,81,83,86,89,92,95,98,101],{"class":37,"line":82},5,[35,84,85],{"class":41},"    for",[35,87,88],{"class":45}," i ",[35,90,91],{"class":41},"in",[35,93,94],{"class":78}," range",[35,96,97],{"class":45},"(",[35,99,100],{"class":78},"50_000_000",[35,102,103],{"class":45},"):\n",[35,105,107,110,113],{"class":37,"line":106},6,[35,108,109],{"class":45},"        total ",[35,111,112],{"class":41},"+=",[35,114,115],{"class":45}," i\n",[35,117,119],{"class":37,"line":118},7,[35,120,53],{"emptyLinePlaceholder":52},[35,122,124,127,129,132,136,138],{"class":37,"line":123},8,[35,125,126],{"class":45},"t1 ",[35,128,75],{"class":41},[35,130,131],{"class":45}," threading.Thread(",[35,133,135],{"class":134},"sCrzJ","target",[35,137,75],{"class":41},[35,139,140],{"class":45},"cpu_heavy)\n",[35,142,144,147,149,151,153,155],{"class":37,"line":143},9,[35,145,146],{"class":45},"t2 ",[35,148,75],{"class":41},[35,150,131],{"class":45},[35,152,135],{"class":134},[35,154,75],{"class":41},[35,156,140],{"class":45},[35,158,160],{"class":37,"line":159},10,[35,161,162],{"class":45},"t1.start(); t2.start()\n",[35,164,166],{"class":37,"line":165},11,[35,167,168],{"class":45},"t1.join(); t2.join()\n",[170,171,174,184,190,196],"ul",{"className":172},[173],"contains-task-list",[175,176,179,183],"li",{"className":177},[178],"task-list-item",[180,181],"input",{"disabled":52,"type":182},"checkbox"," It prevents more than one thread from existing in a process at all",[175,185,187,189],{"className":186},[178],[180,188],{"disabled":52,"type":182}," It allows only one thread to execute Python bytecode at a time, so this two-thread CPU-bound loop runs no faster (often slightly slower) than doing the work in one thread sequentially",[175,191,193,195],{"className":192},[178],[180,194],{"disabled":52,"type":182}," It only restricts I\u002FO operations, so this CPU-bound example runs fully in parallel",[175,197,199,201],{"className":198},[178],[180,200],{"disabled":52,"type":182}," It was removed in Python 3.12, so this now runs on two cores",[203,204,205,209,217],"details",{},[206,207,208],"summary",{},"Show Answer",[210,211,212,216],"p",{},[213,214,215],"strong",{},"Answer:"," B — Only one thread executes Python bytecode at a time; this CPU-bound loop gets no real speedup from two threads",[210,218,219,222,223,226],{},[213,220,221],{},"Explanation:"," ",[213,224,225],{},"Performance:"," The GIL ensures only one OS thread runs Python bytecode at any instant, even on a multi-core machine. Threads still exist and get scheduled (option A is wrong), but for a pure-Python CPU-bound loop like this, the two threads take turns on a single core, so total wall-clock time is roughly the same as — or worse than, due to context-switch overhead — running both loops sequentially. As of the environment's cutoff, the GIL is still the default in mainline CPython 3.12; an optional free-threaded (\"no-GIL\") build exists as of 3.13 but is not the default, so option D's blanket claim is wrong for standard installs.",[14,228,229,237,313,347],{"language":16},[18,230,232,233,236],{"id":231},"q2-for-which-of-these-workloads-does-threading-typically-give-a-real-measurable-speedup-in-cpython","Q2. For which of these workloads does ",[32,234,235],{},"threading"," typically give a real, measurable speedup in CPython?",[23,238,239],{"language":16},[26,240,242],{"className":28,"code":241,"language":16,"meta":30,"style":30},"import threading, requests\n\ndef fetch(url):\n    return requests.get(url).text\n\nthreads = [threading.Thread(target=fetch, args=(u,)) for u in urls]\n",[32,243,244,251,255,265,273,277],{"__ignoreMap":30},[35,245,246,248],{"class":37,"line":38},[35,247,42],{"class":41},[35,249,250],{"class":45}," threading, requests\n",[35,252,253],{"class":37,"line":49},[35,254,53],{"emptyLinePlaceholder":52},[35,256,257,259,262],{"class":37,"line":56},[35,258,59],{"class":41},[35,260,261],{"class":62}," fetch",[35,263,264],{"class":45},"(url):\n",[35,266,267,270],{"class":37,"line":69},[35,268,269],{"class":41},"    return",[35,271,272],{"class":45}," requests.get(url).text\n",[35,274,275],{"class":37,"line":82},[35,276,53],{"emptyLinePlaceholder":52},[35,278,279,282,284,287,289,291,294,297,299,302,305,308,310],{"class":37,"line":106},[35,280,281],{"class":45},"threads ",[35,283,75],{"class":41},[35,285,286],{"class":45}," [threading.Thread(",[35,288,135],{"class":134},[35,290,75],{"class":41},[35,292,293],{"class":45},"fetch, ",[35,295,296],{"class":134},"args",[35,298,75],{"class":41},[35,300,301],{"class":45},"(u,)) ",[35,303,304],{"class":41},"for",[35,306,307],{"class":45}," u ",[35,309,91],{"class":41},[35,311,312],{"class":45}," urls]\n",[170,314,316,322,328,337],{"className":315},[173],[175,317,319,321],{"className":318},[178],[180,320],{"disabled":52,"type":182}," Computing the SHA-256 hash of a 10 GB file in pure Python",[175,323,325,327],{"className":324},[178],[180,326],{"disabled":52,"type":182}," Fetching 50 URLs over the network concurrently, as shown above",[175,329,331,333,334,336],{"className":330},[178],[180,332],{"disabled":52,"type":182}," Multiplying two large matrices with nested Python ",[32,335,304],{}," loops",[175,338,340,342,343,346],{"className":339},[178],[180,341],{"disabled":52,"type":182}," Parsing a large JSON file with the pure-Python ",[32,344,345],{},"json"," module",[203,348,349,351,356],{},[206,350,208],{},[210,352,353,355],{},[213,354,215],{}," B — Fetching 50 URLs over the network concurrently",[210,357,358,222,360,362,363,366,367,369],{},[213,359,221],{},[213,361,225],{}," Threading shines for I\u002FO-bound work because CPython releases the GIL around blocking I\u002FO calls (socket reads, file I\u002FO, ",[32,364,365],{},"time.sleep","), letting other threads run while one waits on the network. The other three options are CPU-bound pure-Python work, where the GIL prevents any of the threads from running Python bytecode simultaneously — you'd see little to no speedup, and possibly a slowdown from thread-switching overhead. The common beginner mistake is reaching for ",[32,368,235],{}," for CPU-heavy loops expecting linear speedup with thread count.",[14,371,372,383,527,570],{"language":16},[18,373,375,376,379,380,382],{"id":374},"q3-why-does-multiprocessing-achieve-real-parallel-speedup-for-cpu-bound-work-where-threading-does-not","Q3. Why does ",[32,377,378],{},"multiprocessing"," achieve real parallel speedup for CPU-bound work where ",[32,381,235],{}," does not?",[23,384,385],{"language":16},[26,386,388],{"className":28,"code":387,"language":16,"meta":30,"style":30},"from multiprocessing import Process\n\ndef cpu_heavy():\n    total = sum(i * i for i in range(20_000_000))\n\nif __name__ == \"__main__\":\n    procs = [Process(target=cpu_heavy) for _ in range(4)]\n    for p in procs: p.start()\n    for p in procs: p.join()\n",[32,389,390,403,407,415,448,452,470,504,516],{"__ignoreMap":30},[35,391,392,395,398,400],{"class":37,"line":38},[35,393,394],{"class":41},"from",[35,396,397],{"class":45}," multiprocessing ",[35,399,42],{"class":41},[35,401,402],{"class":45}," Process\n",[35,404,405],{"class":37,"line":49},[35,406,53],{"emptyLinePlaceholder":52},[35,408,409,411,413],{"class":37,"line":56},[35,410,59],{"class":41},[35,412,63],{"class":62},[35,414,66],{"class":45},[35,416,417,419,421,424,427,430,432,434,436,438,440,442,445],{"class":37,"line":69},[35,418,72],{"class":45},[35,420,75],{"class":41},[35,422,423],{"class":78}," sum",[35,425,426],{"class":45},"(i ",[35,428,429],{"class":41},"*",[35,431,88],{"class":45},[35,433,304],{"class":41},[35,435,88],{"class":45},[35,437,91],{"class":41},[35,439,94],{"class":78},[35,441,97],{"class":45},[35,443,444],{"class":78},"20_000_000",[35,446,447],{"class":45},"))\n",[35,449,450],{"class":37,"line":82},[35,451,53],{"emptyLinePlaceholder":52},[35,453,454,457,460,463,467],{"class":37,"line":106},[35,455,456],{"class":41},"if",[35,458,459],{"class":78}," __name__",[35,461,462],{"class":41}," ==",[35,464,466],{"class":465},"sJ6F3"," \"__main__\"",[35,468,469],{"class":45},":\n",[35,471,472,475,477,480,482,484,487,489,492,494,496,498,501],{"class":37,"line":118},[35,473,474],{"class":45},"    procs ",[35,476,75],{"class":41},[35,478,479],{"class":45}," [Process(",[35,481,135],{"class":134},[35,483,75],{"class":41},[35,485,486],{"class":45},"cpu_heavy) ",[35,488,304],{"class":41},[35,490,491],{"class":45}," _ ",[35,493,91],{"class":41},[35,495,94],{"class":78},[35,497,97],{"class":45},[35,499,500],{"class":78},"4",[35,502,503],{"class":45},")]\n",[35,505,506,508,511,513],{"class":37,"line":123},[35,507,85],{"class":41},[35,509,510],{"class":45}," p ",[35,512,91],{"class":41},[35,514,515],{"class":45}," procs: p.start()\n",[35,517,518,520,522,524],{"class":37,"line":143},[35,519,85],{"class":41},[35,521,510],{"class":45},[35,523,91],{"class":41},[35,525,526],{"class":45}," procs: p.join()\n",[170,528,530,540,548,556],{"className":529},[173],[175,531,533,535,536,539],{"className":532},[178],[180,534],{"disabled":52,"type":182}," Each ",[32,537,538],{},"Process"," is a separate OS process with its own Python interpreter and its own GIL, so four processes can genuinely run on four cores simultaneously",[175,541,543,222,545,547],{"className":542},[178],[180,544],{"disabled":52,"type":182},[32,546,378],{}," disables the GIL globally for the whole machine",[175,549,551,222,553,555],{"className":550},[178],[180,552],{"disabled":52,"type":182},[32,554,538],{}," objects share the same GIL but are scheduled with higher OS priority than threads",[175,557,559,561,562,565,566,569],{"className":558},[178],[180,560],{"disabled":52,"type":182}," There's no real difference; ",[32,563,564],{},"multiprocessing.Process"," is just an alias for ",[32,567,568],{},"threading.Thread"," under the hood",[203,571,572,574,582],{},[206,573,208],{},[210,575,576,578,579,581],{},[213,577,215],{}," A — Each ",[32,580,538],{}," is a separate OS process with its own interpreter and its own GIL, so multiple processes can run truly in parallel",[210,583,584,222,586,222,588,590,591,593,594,596],{},[213,585,221],{},[213,587,225],{},[32,589,378],{}," sidesteps the GIL entirely by spawning independent OS processes, each with its own Python interpreter, memory space, and GIL — there is no single lock shared across processes, so four processes really can use four CPU cores concurrently. This is the fundamental reason ",[32,592,378],{}," is the standard recommendation for CPU-bound parallelism in pure Python, unlike ",[32,595,235],{},". The trade-off (covered later in this quiz) is the overhead of inter-process communication, since memory is not shared by default.",[14,598,599,607,626,666],{"language":16},[18,600,602,603,606],{"id":601},"q4-given-concurrentfutures-which-executor-should-you-pick-for-a-downloading-200-files-over-http-and-b-computing-200-independent-cpu-heavy-fibonacci-like-calculations-in-pure-python","Q4. Given ",[32,604,605],{},"concurrent.futures",", which executor should you pick for (a) downloading 200 files over HTTP and (b) computing 200 independent CPU-heavy Fibonacci-like calculations in pure Python?",[23,608,609],{"language":16},[26,610,612],{"className":28,"code":611,"language":16,"meta":30,"style":30},"from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor\n",[32,613,614],{"__ignoreMap":30},[35,615,616,618,621,623],{"class":37,"line":38},[35,617,394],{"class":41},[35,619,620],{"class":45}," concurrent.futures ",[35,622,42],{"class":41},[35,624,625],{"class":45}," ThreadPoolExecutor, ProcessPoolExecutor\n",[170,627,629,641,652,660],{"className":628},[173],[175,630,632,222,634,637,638],{"className":631},[178],[180,633],{"disabled":52,"type":182},[32,635,636],{},"ProcessPoolExecutor"," for both, since it's always faster than ",[32,639,640],{},"ThreadPoolExecutor",[175,642,644,222,646,648,649,651],{"className":643},[178],[180,645],{"disabled":52,"type":182},[32,647,640],{}," for the downloads (I\u002FO-bound), ",[32,650,636],{}," for the CPU-heavy computation (bypasses the GIL for true parallelism)",[175,653,655,222,657,659],{"className":654},[178],[180,656],{"disabled":52,"type":182},[32,658,640],{}," for both, since threads are lighter-weight and that outweighs GIL contention",[175,661,663,665],{"className":662},[178],[180,664],{"disabled":52,"type":182}," It doesn't matter — both executors use the same underlying worker model",[203,667,668,670,681],{},[206,669,208],{},[210,671,672,674,675,677,678,680],{},[213,673,215],{}," B — ",[32,676,640],{}," for I\u002FO-bound downloads, ",[32,679,636],{}," for CPU-bound computation",[210,682,683,222,685,688,689,691,692,694],{},[213,684,221],{},[213,686,687],{},"Idiom:"," This is the standard rule of thumb: I\u002FO-bound tasks spend most of their time waiting (network, disk), so lightweight threads are sufficient and avoid process-spawn\u002FIPC overhead. CPU-bound pure-Python tasks are limited by the GIL under threads, so ",[32,690,636],{}," is needed to actually use multiple cores. Defaulting to ",[32,693,636],{}," \"because it's always faster\" (option A) is wrong — for I\u002FO-bound work, process overhead (spawning, pickling task arguments\u002Fresults) usually makes it slower than threads for the same job.",[14,696,697,709,743],{},[18,698,700,701,704,705,708],{"id":699},"q5-why-does-a-network-call-like-socketrecv-or-requestsget-allow-other-python-threads-to-make-progress-even-though-the-gil-normally-lets-only-one-thread-run-at-a-time","Q5. Why does a network call like ",[32,702,703],{},"socket.recv()"," or ",[32,706,707],{},"requests.get()"," allow other Python threads to make progress, even though the GIL normally lets only one thread run at a time?",[170,710,712,718,724,730],{"className":711},[173],[175,713,715,717],{"className":714},[178],[180,716],{"disabled":52,"type":182}," It doesn't — network calls block all threads until they return",[175,719,721,723],{"className":720},[178],[180,722],{"disabled":52,"type":182}," Blocking I\u002FO calls in CPython release the GIL while waiting on the OS\u002Fkernel for data, allowing other threads to acquire it and run Python bytecode in the meantime",[175,725,727,729],{"className":726},[178],[180,728],{"disabled":52,"type":182}," Network calls are implemented entirely in a separate process automatically",[175,731,733,735,736,739,740,742],{"className":732},[178],[180,734],{"disabled":52,"type":182}," Only ",[32,737,738],{},"asyncio",", not ",[32,741,235],{},", gets this benefit",[203,744,745,747,752],{},[206,746,208],{},[210,748,749,751],{},[213,750,215],{}," B — Blocking I\u002FO calls release the GIL while waiting on the OS, letting other threads run meanwhile",[210,753,754,756,757,759,760,762,763,765,766,768],{},[213,755,221],{}," CPython's C implementations of blocking I\u002FO operations explicitly release the GIL before making the blocking system call and re-acquire it afterward. This is precisely why ",[32,758,235],{}," is effective for I\u002FO-bound concurrency: while one thread is parked waiting on the kernel for socket data, the GIL is free for another thread to run actual Python code. This release\u002Freacquire dance is unrelated to ",[32,761,738],{}," (option D) — it's a ",[32,764,235],{},"-level mechanism that predates ",[32,767,738],{}," entirely.",[14,770,771,783,929,969],{"language":16},[18,772,774,775,778,779,782],{"id":773},"q6-despite-the-gil-serializing-bytecode-execution-this-code-can-still-print-a-final-counter-value-less-than-200000-why","Q6. Despite the GIL serializing bytecode execution, this code can still print a final ",[32,776,777],{},"counter"," value less than ",[32,780,781],{},"200000",". Why?",[23,784,785],{"language":16},[26,786,788],{"className":28,"code":787,"language":16,"meta":30,"style":30},"import threading\n\ncounter = 0\n\ndef increment():\n    global counter\n    for _ in range(100_000):\n        counter += 1\n\nthreads = [threading.Thread(target=increment) for _ in range(2)]\nfor t in threads: t.start()\nfor t in threads: t.join()\n\nprint(counter)\n",[32,789,790,796,800,809,813,822,830,847,857,861,891,903,915,920],{"__ignoreMap":30},[35,791,792,794],{"class":37,"line":38},[35,793,42],{"class":41},[35,795,46],{"class":45},[35,797,798],{"class":37,"line":49},[35,799,53],{"emptyLinePlaceholder":52},[35,801,802,805,807],{"class":37,"line":56},[35,803,804],{"class":45},"counter ",[35,806,75],{"class":41},[35,808,79],{"class":78},[35,810,811],{"class":37,"line":69},[35,812,53],{"emptyLinePlaceholder":52},[35,814,815,817,820],{"class":37,"line":82},[35,816,59],{"class":41},[35,818,819],{"class":62}," increment",[35,821,66],{"class":45},[35,823,824,827],{"class":37,"line":106},[35,825,826],{"class":41},"    global",[35,828,829],{"class":45}," counter\n",[35,831,832,834,836,838,840,842,845],{"class":37,"line":118},[35,833,85],{"class":41},[35,835,491],{"class":45},[35,837,91],{"class":41},[35,839,94],{"class":78},[35,841,97],{"class":45},[35,843,844],{"class":78},"100_000",[35,846,103],{"class":45},[35,848,849,852,854],{"class":37,"line":123},[35,850,851],{"class":45},"        counter ",[35,853,112],{"class":41},[35,855,856],{"class":78}," 1\n",[35,858,859],{"class":37,"line":143},[35,860,53],{"emptyLinePlaceholder":52},[35,862,863,865,867,869,871,873,876,878,880,882,884,886,889],{"class":37,"line":159},[35,864,281],{"class":45},[35,866,75],{"class":41},[35,868,286],{"class":45},[35,870,135],{"class":134},[35,872,75],{"class":41},[35,874,875],{"class":45},"increment) ",[35,877,304],{"class":41},[35,879,491],{"class":45},[35,881,91],{"class":41},[35,883,94],{"class":78},[35,885,97],{"class":45},[35,887,888],{"class":78},"2",[35,890,503],{"class":45},[35,892,893,895,898,900],{"class":37,"line":165},[35,894,304],{"class":41},[35,896,897],{"class":45}," t ",[35,899,91],{"class":41},[35,901,902],{"class":45}," threads: t.start()\n",[35,904,906,908,910,912],{"class":37,"line":905},12,[35,907,304],{"class":41},[35,909,897],{"class":45},[35,911,91],{"class":41},[35,913,914],{"class":45}," threads: t.join()\n",[35,916,918],{"class":37,"line":917},13,[35,919,53],{"emptyLinePlaceholder":52},[35,921,923,926],{"class":37,"line":922},14,[35,924,925],{"class":78},"print",[35,927,928],{"class":45},"(counter)\n",[170,930,932,943,952,960],{"className":931},[173],[175,933,935,937,938,940,941],{"className":934},[178],[180,936],{"disabled":52,"type":182}," It's impossible — the GIL guarantees ",[32,939,777],{}," is always exactly ",[32,942,781],{},[175,944,946,222,948,951],{"className":945},[178],[180,947],{"disabled":52,"type":182},[32,949,950],{},"counter += 1"," is not a single atomic bytecode operation; the GIL can switch threads mid-sequence (between the read, add, and store steps), so increments from the two threads can be lost to a race condition",[175,953,955,222,957,959],{"className":954},[178],[180,956],{"disabled":52,"type":182},[32,958,568],{}," silently drops some loop iterations for performance",[175,961,963,222,965,968],{"className":962},[178],[180,964],{"disabled":52,"type":182},[32,966,967],{},"global counter"," desynchronizes the variable between threads",[203,970,971,973,980],{},[206,972,208],{},[210,974,975,674,977,979],{},[213,976,215],{},[32,978,950],{}," isn't atomic; the GIL can switch threads between the read\u002Fadd\u002Fstore steps, losing increments to a race condition",[210,981,982,222,984,987,988,990,991,994,995,998,999,1002,1003,1005,1006,1009,1010,1013],{},[213,983,221],{},[213,985,986],{},"Safety:"," A very common misconception is that the GIL makes all Python code thread-safe — it does not. ",[32,989,950],{}," compiles to multiple bytecode instructions (load, add, store), and CPython's scheduler can switch to another thread between any of them. If thread A reads ",[32,992,993],{},"counter=5",", thread B also reads ",[32,996,997],{},"5"," before A writes back ",[32,1000,1001],{},"6",", both compute ",[32,1004,1001],{},", and one increment is lost. The correct fix is a ",[32,1007,1008],{},"threading.Lock"," (or an atomic structure) around the read-modify-write: ",[32,1011,1012],{},"with lock: counter += 1",".",[14,1015,1016,1026,1110,1145],{"language":16},[18,1017,1019,1020,1022,1023,1025],{"id":1018},"q7-what-overhead-does-multiprocessing-introduce-that-threading-does-not-when-passing-data-to-and-from-worker-processes","Q7. What overhead does ",[32,1021,378],{}," introduce that ",[32,1024,235],{}," does not, when passing data to and from worker processes?",[23,1027,1028],{"language":16},[26,1029,1031],{"className":28,"code":1030,"language":16,"meta":30,"style":30},"from multiprocessing import Pool\n\ndef process_item(item):\n    return item.upper()\n\nif __name__ == \"__main__\":\n    with Pool(4) as pool:\n        results = pool.map(process_item, large_list_of_strings)\n",[32,1032,1033,1044,1048,1058,1065,1069,1081,1100],{"__ignoreMap":30},[35,1034,1035,1037,1039,1041],{"class":37,"line":38},[35,1036,394],{"class":41},[35,1038,397],{"class":45},[35,1040,42],{"class":41},[35,1042,1043],{"class":45}," Pool\n",[35,1045,1046],{"class":37,"line":49},[35,1047,53],{"emptyLinePlaceholder":52},[35,1049,1050,1052,1055],{"class":37,"line":56},[35,1051,59],{"class":41},[35,1053,1054],{"class":62}," process_item",[35,1056,1057],{"class":45},"(item):\n",[35,1059,1060,1062],{"class":37,"line":69},[35,1061,269],{"class":41},[35,1063,1064],{"class":45}," item.upper()\n",[35,1066,1067],{"class":37,"line":82},[35,1068,53],{"emptyLinePlaceholder":52},[35,1070,1071,1073,1075,1077,1079],{"class":37,"line":106},[35,1072,456],{"class":41},[35,1074,459],{"class":78},[35,1076,462],{"class":41},[35,1078,466],{"class":465},[35,1080,469],{"class":45},[35,1082,1083,1086,1089,1091,1094,1097],{"class":37,"line":118},[35,1084,1085],{"class":41},"    with",[35,1087,1088],{"class":45}," Pool(",[35,1090,500],{"class":78},[35,1092,1093],{"class":45},") ",[35,1095,1096],{"class":41},"as",[35,1098,1099],{"class":45}," pool:\n",[35,1101,1102,1105,1107],{"class":37,"line":123},[35,1103,1104],{"class":45},"        results ",[35,1106,75],{"class":41},[35,1108,1109],{"class":45}," pool.map(process_item, large_list_of_strings)\n",[170,1111,1113,1119,1125,1139],{"className":1112},[173],[175,1114,1116,1118],{"className":1115},[178],[180,1117],{"disabled":52,"type":182}," None — memory is automatically shared between processes just like threads",[175,1120,1122,1124],{"className":1121},[178],[180,1123],{"disabled":52,"type":182}," Arguments and return values must be pickled (serialized) to cross the process boundary and unpickled on the other side, which costs CPU time and can dominate runtime for large or many small objects",[175,1126,1128,222,1130,1133,1134,739,1136],{"className":1127},[178],[180,1129],{"disabled":52,"type":182},[32,1131,1132],{},"multiprocessing.Pool"," uses shared memory exclusively, so this overhead only applies to ",[32,1135,538],{},[32,1137,1138],{},"Pool",[175,1140,1142,1144],{"className":1141},[178],[180,1143],{"disabled":52,"type":182}," The overhead only applies to the return values, not the input arguments",[203,1146,1147,1149,1154],{},[206,1148,208],{},[210,1150,1151,1153],{},[213,1152,215],{}," B — Arguments and return values must be pickled\u002Funpickled to cross the process boundary, costing CPU time",[210,1155,1156,222,1158,1160,1161,1164,1165,1168,1169,1171,1172,1174,1175,1177],{},[213,1157,221],{},[213,1159,225],{}," Unlike threads, which share the same process memory space, separate processes do not share memory by default. ",[32,1162,1163],{},"Pool.map"," serializes each argument with ",[32,1166,1167],{},"pickle",", sends it through an OS pipe to a worker process, and pickles the result back. For workloads with many small, fast tasks, this serialization overhead can exceed the actual computation time, sometimes making ",[32,1170,378],{}," slower than a single-threaded loop — a frequent surprise for people expecting free parallelism. ",[32,1173,1138],{}," uses the same pickling mechanism as ",[32,1176,538],{}," (option C is wrong); it does not get free shared memory.",[14,1179,1180,1187,1273,1304],{"language":16},[18,1181,1183,1184,1186],{"id":1182},"q8-what-happens-when-you-try-to-pass-this-to-a-multiprocessingpool","Q8. What happens when you try to pass this to a ",[32,1185,1132],{},"?",[23,1188,1189],{"language":16},[26,1190,1192],{"className":28,"code":1191,"language":16,"meta":30,"style":30},"from multiprocessing import Pool\n\nif __name__ == \"__main__\":\n    with Pool(4) as pool:\n        results = pool.map(lambda x: x * 2, [1, 2, 3])\n",[32,1193,1194,1204,1208,1220,1234],{"__ignoreMap":30},[35,1195,1196,1198,1200,1202],{"class":37,"line":38},[35,1197,394],{"class":41},[35,1199,397],{"class":45},[35,1201,42],{"class":41},[35,1203,1043],{"class":45},[35,1205,1206],{"class":37,"line":49},[35,1207,53],{"emptyLinePlaceholder":52},[35,1209,1210,1212,1214,1216,1218],{"class":37,"line":56},[35,1211,456],{"class":41},[35,1213,459],{"class":78},[35,1215,462],{"class":41},[35,1217,466],{"class":465},[35,1219,469],{"class":45},[35,1221,1222,1224,1226,1228,1230,1232],{"class":37,"line":69},[35,1223,1085],{"class":41},[35,1225,1088],{"class":45},[35,1227,500],{"class":78},[35,1229,1093],{"class":45},[35,1231,1096],{"class":41},[35,1233,1099],{"class":45},[35,1235,1236,1238,1240,1243,1246,1249,1251,1254,1257,1260,1263,1265,1267,1270],{"class":37,"line":82},[35,1237,1104],{"class":45},[35,1239,75],{"class":41},[35,1241,1242],{"class":45}," pool.map(",[35,1244,1245],{"class":41},"lambda",[35,1247,1248],{"class":45}," x: x ",[35,1250,429],{"class":41},[35,1252,1253],{"class":78}," 2",[35,1255,1256],{"class":45},", [",[35,1258,1259],{"class":78},"1",[35,1261,1262],{"class":45},", ",[35,1264,888],{"class":78},[35,1266,1262],{"class":45},[35,1268,1269],{"class":78},"3",[35,1271,1272],{"class":45},"])\n",[170,1274,1276,1282,1292,1298],{"className":1275},[173],[175,1277,1279,1281],{"className":1278},[178],[180,1280],{"disabled":52,"type":182}," It runs fine — lambdas are pickled by value automatically",[175,1283,1285,1287,1288,1291],{"className":1284},[178],[180,1286],{"disabled":52,"type":182}," Raises ",[32,1289,1290],{},"PicklingError"," (or similar) because lambda functions cannot be pickled — only module-level (importable) functions and objects can cross the process boundary this way",[175,1293,1295,1297],{"className":1294},[178],[180,1296],{"disabled":52,"type":182}," It silently falls back to running single-threaded",[175,1299,1301,1303],{"className":1300},[178],[180,1302],{"disabled":52,"type":182}," It works, but only on Linux, never on any other OS",[203,1305,1306,1308,1313],{},[206,1307,208],{},[210,1309,1310,1312],{},[213,1311,215],{}," B — Raises a pickling error because lambdas can't be pickled; only importable, module-level functions\u002Fobjects can be sent to worker processes",[210,1314,1315,222,1317,222,1320,1322,1323,1325,1326,1328,1329,1332,1333,1336,1337,1339,1340,1342,1343,704,1345,1348],{},[213,1316,221],{},[213,1318,1319],{},"Debug:",[32,1321,1167],{}," (the mechanism ",[32,1324,378],{}," uses to send work to worker processes) serializes callables by reference — it stores the module and qualified name and re-imports them in the worker. A ",[32,1327,1245],{}," has no importable name (",[32,1330,1331],{},"\u003Clambda>","), so pickling fails with something like ",[32,1334,1335],{},"AttributeError: Can't pickle local object",". The fix is to define the function at module level with ",[32,1338,59],{}," instead of as a ",[32,1341,1245],{},". This is a frequent gotcha for people used to ",[32,1344,235],{},[32,1346,1347],{},"concurrent.futures.ThreadPoolExecutor",", where lambdas work fine since no pickling ever happens.",[14,1350,1351,1355,1475,1536],{"language":16},[18,1352,1354],{"id":1353},"q9-two-worker-processes-need-to-update-a-shared-counter-why-wont-this-work-as-expected-and-whats-the-fix","Q9. Two worker processes need to update a shared counter. Why won't this work as expected, and what's the fix?",[23,1356,1357],{"language":16},[26,1358,1360],{"className":28,"code":1359,"language":16,"meta":30,"style":30},"from multiprocessing import Process\n\ncounter = 0\n\ndef increment():\n    global counter\n    counter += 1\n\nprocs = [Process(target=increment) for _ in range(10)]\nfor p in procs: p.start()\nfor p in procs: p.join()\n\nprint(counter)\n",[32,1361,1362,1372,1376,1384,1388,1396,1402,1411,1415,1445,1455,1465,1469],{"__ignoreMap":30},[35,1363,1364,1366,1368,1370],{"class":37,"line":38},[35,1365,394],{"class":41},[35,1367,397],{"class":45},[35,1369,42],{"class":41},[35,1371,402],{"class":45},[35,1373,1374],{"class":37,"line":49},[35,1375,53],{"emptyLinePlaceholder":52},[35,1377,1378,1380,1382],{"class":37,"line":56},[35,1379,804],{"class":45},[35,1381,75],{"class":41},[35,1383,79],{"class":78},[35,1385,1386],{"class":37,"line":69},[35,1387,53],{"emptyLinePlaceholder":52},[35,1389,1390,1392,1394],{"class":37,"line":82},[35,1391,59],{"class":41},[35,1393,819],{"class":62},[35,1395,66],{"class":45},[35,1397,1398,1400],{"class":37,"line":106},[35,1399,826],{"class":41},[35,1401,829],{"class":45},[35,1403,1404,1407,1409],{"class":37,"line":118},[35,1405,1406],{"class":45},"    counter ",[35,1408,112],{"class":41},[35,1410,856],{"class":78},[35,1412,1413],{"class":37,"line":123},[35,1414,53],{"emptyLinePlaceholder":52},[35,1416,1417,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1443],{"class":37,"line":143},[35,1418,1419],{"class":45},"procs ",[35,1421,75],{"class":41},[35,1423,479],{"class":45},[35,1425,135],{"class":134},[35,1427,75],{"class":41},[35,1429,875],{"class":45},[35,1431,304],{"class":41},[35,1433,491],{"class":45},[35,1435,91],{"class":41},[35,1437,94],{"class":78},[35,1439,97],{"class":45},[35,1441,1442],{"class":78},"10",[35,1444,503],{"class":45},[35,1446,1447,1449,1451,1453],{"class":37,"line":159},[35,1448,304],{"class":41},[35,1450,510],{"class":45},[35,1452,91],{"class":41},[35,1454,515],{"class":45},[35,1456,1457,1459,1461,1463],{"class":37,"line":165},[35,1458,304],{"class":41},[35,1460,510],{"class":45},[35,1462,91],{"class":41},[35,1464,526],{"class":45},[35,1466,1467],{"class":37,"line":905},[35,1468,53],{"emptyLinePlaceholder":52},[35,1470,1471,1473],{"class":37,"line":917},[35,1472,925],{"class":78},[35,1474,928],{"class":45},[170,1476,1478,1490,1518,1530],{"className":1477},[173],[175,1479,1481,1483,1484,1486,1487,1489],{"className":1480},[178],[180,1482],{"disabled":52,"type":182}," It prints ",[32,1485,1442],{}," correctly because ",[32,1488,378],{}," synchronizes global variables automatically",[175,1491,1493,1495,1496,1498,1499,1501,1502,1505,1506,1509,1510,1513,1514,1517],{"className":1492},[178],[180,1494],{"disabled":52,"type":182}," Each process gets its own copy of the module (and thus ",[32,1497,777],{},") at fork\u002Fspawn time; changes in a child are invisible to the parent and other children, so the parent's ",[32,1500,777],{}," stays ",[32,1503,1504],{},"0",". The fix is ",[32,1507,1508],{},"multiprocessing.Value","\u002F",[32,1511,1512],{},"Array"," or a ",[32,1515,1516],{},"Manager"," for real shared state",[175,1519,1521,1523,1524,1527,1528],{"className":1520},[178],[180,1522],{"disabled":52,"type":182}," It raises ",[32,1525,1526],{},"RuntimeError"," because global variables are forbidden with ",[32,1529,378],{},[175,1531,1533,1535],{"className":1532},[178],[180,1534],{"disabled":52,"type":182}," It prints a random number between 0 and 10 due to race conditions, same as with threads",[203,1537,1538,1540,1555],{},[206,1539,208],{},[210,1541,1542,1544,1545,1547,1548,1509,1551,1509,1553,1517],{},[213,1543,215],{}," B — Each process gets its own independent copy of ",[32,1546,777],{},"; the parent's value never changes, so the fix is ",[32,1549,1550],{},"Value",[32,1552,1512],{},[32,1554,1516],{},[210,1556,1557,222,1559,1561,1562,1564,1565,1567,1568,1570,1571,1573,1574,1262,1576,1578,1579,1509,1582,1585],{},[213,1558,221],{},[213,1560,986],{}," Unlike threads, which share one address space, each ",[32,1563,564],{}," has an entirely separate memory space (a copy of the parent's state at fork time, or a fresh reimport under spawn). Incrementing ",[32,1566,777],{}," inside a child mutates that child's private copy — the parent's ",[32,1569,777],{}," never sees it, so the parent prints ",[32,1572,1504],{},", not a race-condition-corrupted partial count (that's the threading failure mode from Q6, not this one). To actually share mutable state across processes you need explicit IPC-aware primitives: ",[32,1575,1508],{},[32,1577,1512],{},", or a ",[32,1580,1581],{},"Manager().dict()",[32,1583,1584],{},"list()",", all backed by shared memory or a proxy process.",[14,1587,1588,1592,1685,1732],{"language":16},[18,1589,1591],{"id":1590},"q10-whats-wrong-with-this-locking-pattern-and-what-does-it-cause","Q10. What's wrong with this locking pattern, and what does it cause?",[23,1593,1594],{"language":16},[26,1595,1597],{"className":28,"code":1596,"language":16,"meta":30,"style":30},"import threading\n\nlock = threading.Lock()\n\ndef outer():\n    with lock:\n        inner()\n\ndef inner():\n    with lock:\n        print(\"done\")\n\nouter()\n",[32,1598,1599,1605,1609,1619,1623,1632,1639,1644,1648,1657,1663,1676,1680],{"__ignoreMap":30},[35,1600,1601,1603],{"class":37,"line":38},[35,1602,42],{"class":41},[35,1604,46],{"class":45},[35,1606,1607],{"class":37,"line":49},[35,1608,53],{"emptyLinePlaceholder":52},[35,1610,1611,1614,1616],{"class":37,"line":56},[35,1612,1613],{"class":45},"lock ",[35,1615,75],{"class":41},[35,1617,1618],{"class":45}," threading.Lock()\n",[35,1620,1621],{"class":37,"line":69},[35,1622,53],{"emptyLinePlaceholder":52},[35,1624,1625,1627,1630],{"class":37,"line":82},[35,1626,59],{"class":41},[35,1628,1629],{"class":62}," outer",[35,1631,66],{"class":45},[35,1633,1634,1636],{"class":37,"line":106},[35,1635,1085],{"class":41},[35,1637,1638],{"class":45}," lock:\n",[35,1640,1641],{"class":37,"line":118},[35,1642,1643],{"class":45},"        inner()\n",[35,1645,1646],{"class":37,"line":123},[35,1647,53],{"emptyLinePlaceholder":52},[35,1649,1650,1652,1655],{"class":37,"line":143},[35,1651,59],{"class":41},[35,1653,1654],{"class":62}," inner",[35,1656,66],{"class":45},[35,1658,1659,1661],{"class":37,"line":159},[35,1660,1085],{"class":41},[35,1662,1638],{"class":45},[35,1664,1665,1668,1670,1673],{"class":37,"line":165},[35,1666,1667],{"class":78},"        print",[35,1669,97],{"class":45},[35,1671,1672],{"class":465},"\"done\"",[35,1674,1675],{"class":45},")\n",[35,1677,1678],{"class":37,"line":905},[35,1679,53],{"emptyLinePlaceholder":52},[35,1681,1682],{"class":37,"line":917},[35,1683,1684],{"class":45},"outer()\n",[170,1686,1688,1697,1714,1723],{"className":1687},[173],[175,1689,1691,1693,1694,1696],{"className":1690},[178],[180,1692],{"disabled":52,"type":182}," Nothing — ",[32,1695,1008],{}," allows the same thread to reacquire it any number of times",[175,1698,1700,1702,1703,1705,1706,1709,1710,1713],{"className":1699},[178],[180,1701],{"disabled":52,"type":182}," The thread deadlocks itself: ",[32,1704,1008],{}," is non-reentrant, so ",[32,1707,1708],{},"inner()","'s attempt to acquire an already-held lock blocks forever. Using ",[32,1711,1712],{},"threading.RLock"," instead would allow the same thread to reacquire it",[175,1715,1717,1523,1719,1722],{"className":1716},[178],[180,1718],{"disabled":52,"type":182},[32,1720,1721],{},"RuntimeError: lock already held"," immediately instead of blocking",[175,1724,1726,222,1728,1731],{"className":1725},[178],[180,1727],{"disabled":52,"type":182},[32,1729,1730],{},"with lock:"," silently no-ops on the second acquisition",[203,1733,1734,1736,1747],{},[206,1735,208],{},[210,1737,1738,1740,1741,1743,1744,1746],{},[213,1739,215],{}," B — The thread deadlocks itself; ",[32,1742,1008],{}," is non-reentrant, so use ",[32,1745,1712],{}," for nested acquisition by the same thread",[210,1748,1749,222,1751,1753,1754,1756,1757,1760,1761,1763,1764,1767,1768,1770],{},[213,1750,221],{},[213,1752,986],{}," A plain ",[32,1755,1008],{}," is not reentrant — once a thread holds it, that same thread blocks (rather than passing through) on a second ",[32,1758,1759],{},"acquire()",", because the lock has no concept of \"owner.\" Since ",[32,1762,1708],{}," runs in the same thread that already holds the lock via ",[32,1765,1766],{},"outer()",", the program hangs forever with no exception (option C is wrong — no error is ever raised; it just blocks). ",[32,1769,1712],{}," tracks the owning thread and an acquisition count, letting the same thread re-enter safely, which is the standard fix for recursive or nested locking within one thread's call stack.",[14,1772,1773,1777,1914,1946],{"language":16},[18,1774,1776],{"id":1775},"q11-a-cpu-bound-function-calls-into-a-c-extension-eg-numpys-matrix-multiply-that-explicitly-releases-the-gil-during-its-computation-what-does-this-enable","Q11. A CPU-bound function calls into a C extension (e.g., NumPy's matrix multiply) that explicitly releases the GIL during its computation. What does this enable?",[23,1778,1779],{"language":16},[26,1780,1782],{"className":28,"code":1781,"language":16,"meta":30,"style":30},"import threading\nimport numpy as np\n\ndef matmul_heavy():\n    a = np.random.rand(2000, 2000)\n    b = np.random.rand(2000, 2000)\n    a @ b\n\nthreads = [threading.Thread(target=matmul_heavy) for _ in range(4)]\nfor t in threads: t.start()\nfor t in threads: t.join()\n",[32,1783,1784,1790,1802,1806,1815,1834,1851,1861,1865,1894,1904],{"__ignoreMap":30},[35,1785,1786,1788],{"class":37,"line":38},[35,1787,42],{"class":41},[35,1789,46],{"class":45},[35,1791,1792,1794,1797,1799],{"class":37,"line":49},[35,1793,42],{"class":41},[35,1795,1796],{"class":45}," numpy ",[35,1798,1096],{"class":41},[35,1800,1801],{"class":45}," np\n",[35,1803,1804],{"class":37,"line":56},[35,1805,53],{"emptyLinePlaceholder":52},[35,1807,1808,1810,1813],{"class":37,"line":69},[35,1809,59],{"class":41},[35,1811,1812],{"class":62}," matmul_heavy",[35,1814,66],{"class":45},[35,1816,1817,1820,1822,1825,1828,1830,1832],{"class":37,"line":82},[35,1818,1819],{"class":45},"    a ",[35,1821,75],{"class":41},[35,1823,1824],{"class":45}," np.random.rand(",[35,1826,1827],{"class":78},"2000",[35,1829,1262],{"class":45},[35,1831,1827],{"class":78},[35,1833,1675],{"class":45},[35,1835,1836,1839,1841,1843,1845,1847,1849],{"class":37,"line":106},[35,1837,1838],{"class":45},"    b ",[35,1840,75],{"class":41},[35,1842,1824],{"class":45},[35,1844,1827],{"class":78},[35,1846,1262],{"class":45},[35,1848,1827],{"class":78},[35,1850,1675],{"class":45},[35,1852,1853,1855,1858],{"class":37,"line":118},[35,1854,1819],{"class":45},[35,1856,1857],{"class":41},"@",[35,1859,1860],{"class":45}," b\n",[35,1862,1863],{"class":37,"line":123},[35,1864,53],{"emptyLinePlaceholder":52},[35,1866,1867,1869,1871,1873,1875,1877,1880,1882,1884,1886,1888,1890,1892],{"class":37,"line":143},[35,1868,281],{"class":45},[35,1870,75],{"class":41},[35,1872,286],{"class":45},[35,1874,135],{"class":134},[35,1876,75],{"class":41},[35,1878,1879],{"class":45},"matmul_heavy) ",[35,1881,304],{"class":41},[35,1883,491],{"class":45},[35,1885,91],{"class":41},[35,1887,94],{"class":78},[35,1889,97],{"class":45},[35,1891,500],{"class":78},[35,1893,503],{"class":45},[35,1895,1896,1898,1900,1902],{"class":37,"line":159},[35,1897,304],{"class":41},[35,1899,897],{"class":45},[35,1901,91],{"class":41},[35,1903,902],{"class":45},[35,1905,1906,1908,1910,1912],{"class":37,"line":165},[35,1907,304],{"class":41},[35,1909,897],{"class":45},[35,1911,91],{"class":41},[35,1913,914],{"class":45},[170,1915,1917,1923,1929,1935],{"className":1916},[173],[175,1918,1920,1922],{"className":1919},[178],[180,1921],{"disabled":52,"type":182}," Nothing — pure computation always requires the GIL, so this is no different from a pure-Python loop",[175,1924,1926,1928],{"className":1925},[178],[180,1927],{"disabled":52,"type":182}," These threads can achieve genuine multi-core parallelism, because NumPy's underlying C code releases the GIL for the duration of the heavy computation, unlike pure-Python bytecode loops",[175,1930,1932,1934],{"className":1931},[178],[180,1933],{"disabled":52,"type":182}," NumPy always uses multiprocessing internally regardless of the GIL",[175,1936,1938,1940,1941,1943,1944],{"className":1937},[178],[180,1939],{"disabled":52,"type":182}," This only works if ",[32,1942,235],{}," is replaced with ",[32,1945,378],{},[203,1947,1948,1950,1955],{},[206,1949,208],{},[210,1951,1952,1954],{},[213,1953,215],{}," B — These threads can achieve genuine multi-core parallelism because NumPy's C code releases the GIL during the heavy computation",[210,1956,1957,222,1959,1961,1962,1965,1966,1969,1970,1973,1974,1509,1977,1980,1981,1983],{},[213,1958,221],{},[213,1960,225],{}," The GIL restriction applies specifically to executing Python bytecode, not to C code. Well-written C extensions (NumPy, many ",[32,1963,1964],{},"hashlib"," and ",[32,1967,1968],{},"zlib"," operations, some ",[32,1971,1972],{},"re"," operations) explicitly release the GIL around long-running C-level work using ",[32,1975,1976],{},"Py_BEGIN_ALLOW_THREADS",[32,1978,1979],{},"Py_END_ALLOW_THREADS",", letting other Python threads run truly concurrently on other cores during that window. This is an important exception to the general \"threading doesn't help CPU-bound work\" rule (Q1\u002FQ2) and explains why numeric\u002Fscientific Python libraries can benefit meaningfully from ",[32,1982,235],{},", unlike hand-written pure-Python loops.",[14,1985,1986,1994,2086,2124],{"language":16},[18,1987,1989,1990,1993],{"id":1988},"q12-what-is-the-effect-of-daemontrue-on-a-thread-and-what-happens-without-it","Q12. What is the effect of ",[32,1991,1992],{},"daemon=True"," on a thread, and what happens without it?",[23,1995,1996],{"language":16},[26,1997,1999],{"className":28,"code":1998,"language":16,"meta":30,"style":30},"import threading, time\n\ndef background_task():\n    while True:\n        time.sleep(1)\n\nt = threading.Thread(target=background_task, daemon=True)\nt.start()\nprint(\"main thread finished\")\n",[32,2000,2001,2008,2012,2021,2031,2040,2044,2070,2075],{"__ignoreMap":30},[35,2002,2003,2005],{"class":37,"line":38},[35,2004,42],{"class":41},[35,2006,2007],{"class":45}," threading, time\n",[35,2009,2010],{"class":37,"line":49},[35,2011,53],{"emptyLinePlaceholder":52},[35,2013,2014,2016,2019],{"class":37,"line":56},[35,2015,59],{"class":41},[35,2017,2018],{"class":62}," background_task",[35,2020,66],{"class":45},[35,2022,2023,2026,2029],{"class":37,"line":69},[35,2024,2025],{"class":41},"    while",[35,2027,2028],{"class":78}," True",[35,2030,469],{"class":45},[35,2032,2033,2036,2038],{"class":37,"line":82},[35,2034,2035],{"class":45},"        time.sleep(",[35,2037,1259],{"class":78},[35,2039,1675],{"class":45},[35,2041,2042],{"class":37,"line":106},[35,2043,53],{"emptyLinePlaceholder":52},[35,2045,2046,2049,2051,2053,2055,2057,2060,2063,2065,2068],{"class":37,"line":118},[35,2047,2048],{"class":45},"t ",[35,2050,75],{"class":41},[35,2052,131],{"class":45},[35,2054,135],{"class":134},[35,2056,75],{"class":41},[35,2058,2059],{"class":45},"background_task, ",[35,2061,2062],{"class":134},"daemon",[35,2064,75],{"class":41},[35,2066,2067],{"class":78},"True",[35,2069,1675],{"class":45},[35,2071,2072],{"class":37,"line":123},[35,2073,2074],{"class":45},"t.start()\n",[35,2076,2077,2079,2081,2084],{"class":37,"line":143},[35,2078,925],{"class":78},[35,2080,97],{"class":45},[35,2082,2083],{"class":465},"\"main thread finished\"",[35,2085,1675],{"class":45},[170,2087,2089,2101,2110,2118],{"className":2088},[173],[175,2090,2092,222,2094,2096,2097,2100],{"className":2091},[178],[180,2093],{"disabled":52,"type":182},[32,2095,1992],{}," has no effect; the program always exits once ",[32,2098,2099],{},"main()"," finishes regardless of running threads",[175,2102,2104,2106,2107,2109],{"className":2103},[178],[180,2105],{"disabled":52,"type":182}," A daemon thread is killed abruptly when the main program exits; without ",[32,2108,1992],{},", this infinite-loop thread would keep the whole process alive forever since Python waits for all non-daemon threads to finish",[175,2111,2113,222,2115,2117],{"className":2112},[178],[180,2114],{"disabled":52,"type":182},[32,2116,1992],{}," makes the thread run with elevated OS privileges",[175,2119,2121,2123],{"className":2120},[178],[180,2122],{"disabled":52,"type":182}," Daemon threads run on a separate GIL, so they never block other threads",[203,2125,2126,2128,2133],{},[206,2127,208],{},[210,2129,2130,2132],{},[213,2131,215],{}," B — A daemon thread is killed when the main program exits; without it, this infinite loop would keep the process alive forever",[210,2134,2135,222,2137,2139,2140,2142,2143,2145,2146,2148],{},[213,2136,221],{},[213,2138,1319],{}," By default, CPython will not exit the process until every non-daemon thread finishes, since threads represent real OS-level work that might need to complete. An infinite-loop background thread without ",[32,2141,1992],{}," would hang the program indefinitely after ",[32,2144,925],{}," runs — the process never terminates. Marking it ",[32,2147,1992],{}," tells the interpreter it's safe to abruptly kill that thread on exit, which is standard practice for background workers (polling loops, heartbeat threads) that shouldn't block shutdown.",[14,2150,2151,2163,2236,2281],{"language":16},[18,2152,2154,2155,2158,2159,2162],{"id":2153},"q13-what-is-likely-to-go-wrong-if-this-script-using-the-default-spawn-start-method-eg-on-windows-or-macos-omits-the-if-__name__-__main__-guard","Q13. What is likely to go wrong if this script (using the default ",[32,2156,2157],{},"spawn"," start method, e.g. on Windows or macOS) omits the ",[32,2160,2161],{},"if __name__ == \"__main__\":"," guard?",[23,2164,2165],{"language":16},[26,2166,2168],{"className":28,"code":2167,"language":16,"meta":30,"style":30},"from multiprocessing import Process\n\ndef worker():\n    print(\"working\")\n\np = Process(target=worker)\np.start()\np.join()\n",[32,2169,2170,2180,2184,2193,2205,2209,2226,2231],{"__ignoreMap":30},[35,2171,2172,2174,2176,2178],{"class":37,"line":38},[35,2173,394],{"class":41},[35,2175,397],{"class":45},[35,2177,42],{"class":41},[35,2179,402],{"class":45},[35,2181,2182],{"class":37,"line":49},[35,2183,53],{"emptyLinePlaceholder":52},[35,2185,2186,2188,2191],{"class":37,"line":56},[35,2187,59],{"class":41},[35,2189,2190],{"class":62}," worker",[35,2192,66],{"class":45},[35,2194,2195,2198,2200,2203],{"class":37,"line":69},[35,2196,2197],{"class":78},"    print",[35,2199,97],{"class":45},[35,2201,2202],{"class":465},"\"working\"",[35,2204,1675],{"class":45},[35,2206,2207],{"class":37,"line":82},[35,2208,53],{"emptyLinePlaceholder":52},[35,2210,2211,2214,2216,2219,2221,2223],{"class":37,"line":106},[35,2212,2213],{"class":45},"p ",[35,2215,75],{"class":41},[35,2217,2218],{"class":45}," Process(",[35,2220,135],{"class":134},[35,2222,75],{"class":41},[35,2224,2225],{"class":45},"worker)\n",[35,2227,2228],{"class":37,"line":118},[35,2229,2230],{"class":45},"p.start()\n",[35,2232,2233],{"class":37,"line":123},[35,2234,2235],{"class":45},"p.join()\n",[170,2237,2239,2245,2260,2270],{"className":2238},[173],[175,2240,2242,2244],{"className":2241},[178],[180,2243],{"disabled":52,"type":182}," Nothing — the guard is only a stylistic convention, never functionally required",[175,2246,2248,2250,2251,2253,2254,2257,2258],{"className":2247},[178],[180,2249],{"disabled":52,"type":182}," On ",[32,2252,2157],{},"-based platforms, each child process re-imports the main module to set itself up; without the guard, the child re-executes the top-level ",[32,2255,2256],{},"Process(...).start()"," call too, spawning another child recursively — leading to a runaway process bomb or ",[32,2259,1526],{},[175,2261,2263,2265,2266,2269],{"className":2262},[178],[180,2264],{"disabled":52,"type":182}," It causes a ",[32,2267,2268],{},"SyntaxError"," at parse time",[175,2271,2273,2275,2276,2278,2279],{"className":2272},[178],[180,2274],{"disabled":52,"type":182}," It only affects ",[32,2277,1138],{},", never plain ",[32,2280,538],{},[203,2282,2283,2285,2295],{},[206,2284,208],{},[210,2286,2287,2289,2290,2292,2293],{},[213,2288,215],{}," B — Without the guard, ",[32,2291,2157],{},"-based child processes re-execute the top-level code, recursively spawning more children — a process bomb or ",[32,2294,1526],{},[210,2296,2297,222,2299,2302,2303,2305,2306,2309,2310,2312,2313,2315,2316,2319,2320,2322,2323,2326],{},[213,2298,221],{},[213,2300,2301],{},"Portability:"," On platforms using the ",[32,2304,2157],{}," start method (the default on Windows and, since Python 3.8, macOS), a new child process starts a fresh Python interpreter and re-imports ",[32,2307,2308],{},"__main__"," to reconstruct what it needs to run the target function — it does not fork the parent's already-running memory. If the module-level code that creates and starts the ",[32,2311,538],{}," isn't guarded by ",[32,2314,2161],{},", each child re-runs that same top-level code on import, spawning yet another child, recursively. CPython actually detects this specific pattern and raises ",[32,2317,2318],{},"RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase..."," with guidance to add the guard. This is a genuine functional requirement on ",[32,2321,2157],{},"\u002FWindows, not just a style preference (option A) — though on Linux's default ",[32,2324,2325],{},"fork"," method the same code often \"works\" without the guard, which is precisely why the bug is a portability trap.",[14,2328,2329,2333,2360],{},[18,2330,2332],{"id":2331},"q14-is-the-gil-a-single-global-lock-shared-across-an-entire-machine-or-something-else","Q14. Is the GIL a single global lock shared across an entire machine, or something else?",[170,2334,2336,2342,2348,2354],{"className":2335},[173],[175,2337,2339,2341],{"className":2338},[178],[180,2340],{"disabled":52,"type":182}," It's one lock per machine — even unrelated Python processes contend for the same GIL",[175,2343,2345,2347],{"className":2344},[178],[180,2346],{"disabled":52,"type":182}," It's one lock per Python process (per interpreter instance); separate processes each have their own independent GIL and don't contend with each other",[175,2349,2351,2353],{"className":2350},[178],[180,2352],{"disabled":52,"type":182}," It's one lock per CPU core",[175,2355,2357,2359],{"className":2356},[178],[180,2358],{"disabled":52,"type":182}," It's one lock per thread, meaning it provides no serialization at all",[203,2361,2362,2364,2369],{},[206,2363,208],{},[210,2365,2366,2368],{},[213,2367,215],{}," B — One GIL per Python process\u002Finterpreter; separate processes have entirely independent GILs",[210,2370,2371,2373,2374,2376],{},[213,2372,221],{}," The name \"Global Interpreter Lock\" refers to global-within-one-interpreter scope, not global-across-the-machine — each Python process runs its own interpreter with its own GIL, completely independent of any other process's GIL. This is exactly why ",[32,2375,378],{}," sidesteps the limitation: spawning multiple processes means multiple independent GILs, each free to run bytecode on a different core simultaneously, whereas multiple threads inside one process must all share that one process's single GIL.",[14,2378,2379,2383,2489,2539],{"language":16},[18,2380,2382],{"id":2381},"q15-two-threads-need-to-hand-off-work-items-safely-without-manual-locking-whats-the-idiomatic-tool","Q15. Two threads need to hand off work items safely without manual locking. What's the idiomatic tool?",[23,2384,2385],{"language":16},[26,2386,2388],{"className":28,"code":2387,"language":16,"meta":30,"style":30},"import threading, queue\n\ndef producer(q):\n    for i in range(10):\n        q.put(i)\n\ndef consumer(q):\n    while True:\n        item = q.get()\n        print(item)\n        q.task_done()\n\nq = queue.Queue()\n",[32,2389,2390,2397,2401,2411,2427,2432,2436,2445,2453,2463,2470,2475,2479],{"__ignoreMap":30},[35,2391,2392,2394],{"class":37,"line":38},[35,2393,42],{"class":41},[35,2395,2396],{"class":45}," threading, queue\n",[35,2398,2399],{"class":37,"line":49},[35,2400,53],{"emptyLinePlaceholder":52},[35,2402,2403,2405,2408],{"class":37,"line":56},[35,2404,59],{"class":41},[35,2406,2407],{"class":62}," producer",[35,2409,2410],{"class":45},"(q):\n",[35,2412,2413,2415,2417,2419,2421,2423,2425],{"class":37,"line":69},[35,2414,85],{"class":41},[35,2416,88],{"class":45},[35,2418,91],{"class":41},[35,2420,94],{"class":78},[35,2422,97],{"class":45},[35,2424,1442],{"class":78},[35,2426,103],{"class":45},[35,2428,2429],{"class":37,"line":82},[35,2430,2431],{"class":45},"        q.put(i)\n",[35,2433,2434],{"class":37,"line":106},[35,2435,53],{"emptyLinePlaceholder":52},[35,2437,2438,2440,2443],{"class":37,"line":118},[35,2439,59],{"class":41},[35,2441,2442],{"class":62}," consumer",[35,2444,2410],{"class":45},[35,2446,2447,2449,2451],{"class":37,"line":123},[35,2448,2025],{"class":41},[35,2450,2028],{"class":78},[35,2452,469],{"class":45},[35,2454,2455,2458,2460],{"class":37,"line":143},[35,2456,2457],{"class":45},"        item ",[35,2459,75],{"class":41},[35,2461,2462],{"class":45}," q.get()\n",[35,2464,2465,2467],{"class":37,"line":159},[35,2466,1667],{"class":78},[35,2468,2469],{"class":45},"(item)\n",[35,2471,2472],{"class":37,"line":165},[35,2473,2474],{"class":45},"        q.task_done()\n",[35,2476,2477],{"class":37,"line":905},[35,2478,53],{"emptyLinePlaceholder":52},[35,2480,2481,2484,2486],{"class":37,"line":917},[35,2482,2483],{"class":45},"q ",[35,2485,75],{"class":41},[35,2487,2488],{"class":45}," queue.Queue()\n",[170,2490,2492,2502,2518,2529],{"className":2491},[173],[175,2493,2495,2497,2498,2501],{"className":2494},[178],[180,2496],{"disabled":52,"type":182}," Manually appending to and popping from a shared ",[32,2499,2500],{},"list"," with no synchronization, since list operations are \"atomic enough\"",[175,2503,2505,222,2507,2510,2511,1509,2514,2517],{"className":2504},[178],[180,2506],{"disabled":52,"type":182},[32,2508,2509],{},"queue.Queue",", which is internally synchronized with its own lock\u002Fcondition variables, making ",[32,2512,2513],{},"put",[32,2515,2516],{},"get"," safe to call from multiple threads without any extra locking code",[175,2519,2521,222,2523,2526,2527],{"className":2520},[178],[180,2522],{"disabled":52,"type":182},[32,2524,2525],{},"multiprocessing.Queue",", since it's a strict superset of ",[32,2528,2509],{},[175,2530,2532,2534,2535,2538],{"className":2531},[178],[180,2533],{"disabled":52,"type":182}," A plain Python ",[32,2536,2537],{},"dict"," keyed by timestamp",[203,2540,2541,2543,2550],{},[206,2542,208],{},[210,2544,2545,674,2547,2549],{},[213,2546,215],{},[32,2548,2509],{}," is internally synchronized, making it safe for multi-threaded producer\u002Fconsumer patterns without manual locking",[210,2551,2552,222,2554,222,2556,2558,2559,1509,2561,2563,2564,2566,2567,1509,2570,2573,2574,2576],{},[213,2553,221],{},[213,2555,687],{},[32,2557,2509],{}," is specifically designed for thread-safe communication — its ",[32,2560,2513],{},[32,2562,2516],{}," methods handle all locking internally, and it additionally supports blocking with optional timeouts, making it the standard building block for producer\u002Fconsumer patterns in ",[32,2565,235],{}," code. Relying on raw ",[32,2568,2569],{},"list.append",[32,2571,2572],{},"pop"," (option A) is risky: while individual list methods are atomic due to the GIL, compound patterns (check-then-act, popping under certain conditions) are not, and it lacks the queue's blocking\u002Fwake-up semantics. ",[32,2575,2525],{}," (option C) is a different, heavier implementation meant for cross-process communication with its own pickling overhead — not a strict superset for thread use.",[14,2578,2579,2583,2613],{},[18,2580,2582],{"id":2581},"q16-a-teams-data-pipeline-spawns-500-threads-to-process-500-small-cpu-bound-number-crunching-tasks-in-pure-python-expecting-a-500x-speedup-on-a-16-core-machine-what-actually-happens-and-whats-the-better-approach","Q16. A team's data pipeline spawns 500 threads to process 500 small CPU-bound number-crunching tasks in pure Python, expecting a 500x speedup on a 16-core machine. What actually happens, and what's the better approach?",[170,2584,2586,2592,2601,2607],{"className":2585},[173],[175,2587,2589,2591],{"className":2588},[178],[180,2590],{"disabled":52,"type":182}," It gets the expected roughly-500x speedup since threads scale linearly with count",[175,2593,2595,2597,2598,2600],{"className":2594},[178],[180,2596],{"disabled":52,"type":182}," It gets little to no speedup — likely worse than sequential due to GIL contention and context-switching overhead among 500 threads; a ",[32,2599,636],{}," sized to the CPU count (e.g., 16) would actually use the available cores",[175,2602,2604,2606],{"className":2603},[178],[180,2605],{"disabled":52,"type":182}," It crashes because Python enforces a hard limit of 100 threads",[175,2608,2610,2612],{"className":2609},[178],[180,2611],{"disabled":52,"type":182}," It gets exactly a 16x speedup automatically because the GIL load-balances across cores",[203,2614,2615,2617,2625],{},[206,2616,208],{},[210,2618,2619,2621,2622,2624],{},[213,2620,215],{}," B — Little to no speedup, likely worse than sequential due to GIL contention\u002Fcontext-switching; a right-sized ",[32,2623,636],{}," would use the cores",[210,2626,2627,222,2629,2631,2632,2635],{},[213,2628,221],{},[213,2630,225],{}," Since only one thread executes Python bytecode at a time regardless of thread count, adding more threads for CPU-bound work doesn't add parallelism — it adds scheduling and context-switch overhead, which can make total throughput worse than a single thread doing the work sequentially. The fix is ",[32,2633,2634],{},"ProcessPoolExecutor(max_workers=16)"," (roughly matching core count) so the OS can schedule genuinely parallel processes, each with its own GIL. Over-threading for CPU work is a common performance anti-pattern from people assuming \"more threads = more parallel,\" which only holds once I\u002FO is involved.",[14,2637,2638,2645,2738,2783],{"language":16},[18,2639,2641,2642,2644],{"id":2640},"q17-a-web-server-handler-stores-per-request-state-in-a-variable-at-module-scope-so-different-helper-functions-can-access-it-without-passing-it-explicitly-under-threading-why-is-this-dangerous-and-whats-the-fix","Q17. A web server handler stores per-request state in a variable at module scope so different helper functions can access it without passing it explicitly. Under ",[32,2643,235],{},", why is this dangerous, and what's the fix?",[23,2646,2647],{"language":16},[26,2648,2650],{"className":28,"code":2649,"language":16,"meta":30,"style":30},"current_user = None\n\ndef handle_request(user, request):\n    global current_user\n    current_user = user\n    process(request)\n\ndef process(request):\n    print(f\"Processing for {current_user}\")\n",[32,2651,2652,2662,2666,2676,2683,2693,2698,2702,2712],{"__ignoreMap":30},[35,2653,2654,2657,2659],{"class":37,"line":38},[35,2655,2656],{"class":45},"current_user ",[35,2658,75],{"class":41},[35,2660,2661],{"class":78}," None\n",[35,2663,2664],{"class":37,"line":49},[35,2665,53],{"emptyLinePlaceholder":52},[35,2667,2668,2670,2673],{"class":37,"line":56},[35,2669,59],{"class":41},[35,2671,2672],{"class":62}," handle_request",[35,2674,2675],{"class":45},"(user, request):\n",[35,2677,2678,2680],{"class":37,"line":69},[35,2679,826],{"class":41},[35,2681,2682],{"class":45}," current_user\n",[35,2684,2685,2688,2690],{"class":37,"line":82},[35,2686,2687],{"class":45},"    current_user ",[35,2689,75],{"class":41},[35,2691,2692],{"class":45}," user\n",[35,2694,2695],{"class":37,"line":106},[35,2696,2697],{"class":45},"    process(request)\n",[35,2699,2700],{"class":37,"line":118},[35,2701,53],{"emptyLinePlaceholder":52},[35,2703,2704,2706,2709],{"class":37,"line":123},[35,2705,59],{"class":41},[35,2707,2708],{"class":62}," process",[35,2710,2711],{"class":45},"(request):\n",[35,2713,2714,2716,2718,2721,2724,2727,2730,2733,2736],{"class":37,"line":143},[35,2715,2197],{"class":78},[35,2717,97],{"class":45},[35,2719,2720],{"class":41},"f",[35,2722,2723],{"class":465},"\"Processing for ",[35,2725,2726],{"class":78},"{",[35,2728,2729],{"class":45},"current_user",[35,2731,2732],{"class":78},"}",[35,2734,2735],{"class":465},"\"",[35,2737,1675],{"class":45},[170,2739,2741,2749,2762,2774],{"className":2740},[173],[175,2742,2744,2746,2747],{"className":2743},[178],[180,2745],{"disabled":52,"type":182}," It's safe — the GIL ensures each request sees only its own ",[32,2748,2729],{},[175,2750,2752,2754,2755,2757,2758,2761],{"className":2751},[178],[180,2753],{"disabled":52,"type":182}," Concurrent requests handled by different threads share the same module-level ",[32,2756,2729],{},", so one thread can overwrite it while another is mid-request, leaking one user's identity into another's processing; the fix is ",[32,2759,2760],{},"threading.local()"," to give each thread its own isolated copy",[175,2763,2765,2767,2768,2771,2772],{"className":2764},[178],[180,2766],{"disabled":52,"type":182}," It's safe as long as ",[32,2769,2770],{},"process"," is called immediately after setting ",[32,2773,2729],{},[175,2775,2777,222,2779,2782],{"className":2776},[178],[180,2778],{"disabled":52,"type":182},[32,2780,2781],{},"global"," automatically makes the variable thread-safe by serializing access",[203,2784,2785,2787,2798],{},[206,2786,208],{},[210,2788,2789,2791,2792,2794,2795,2797],{},[213,2790,215],{}," B — Module-level ",[32,2793,2729],{}," is shared across all threads, so concurrent requests can clobber each other's value; use ",[32,2796,2760],{}," for per-thread isolation",[210,2799,2800,222,2802,2804,2805,2808,2809,2812,2813,2816,2817,2819,2820,2822],{},[213,2801,221],{},[213,2803,986],{}," A module-level variable lives in one shared namespace regardless of which thread touches it — there's no per-thread isolation just because the GIL exists. If Thread A sets ",[32,2806,2807],{},"current_user = alice"," and, before ",[32,2810,2811],{},"process()"," runs, the GIL switches to Thread B which sets ",[32,2814,2815],{},"current_user = bob",", Thread A's ",[32,2818,2811],{}," call could print \"Processing for bob,\" a serious correctness (and potentially security) bug. ",[32,2821,2760],{}," creates an object where each thread transparently sees its own independent attribute values — the standard fix for this exact per-request-state pattern, used internally by frameworks like Flask for request-context globals.",[14,2824,2825,2836],{"language":16},[18,2826,2828,2829,2831,2832,2835],{"id":2827},"q18-when-submitting-many-quick-tasks-to-a-processpoolexecutor-why-might-poolmap-significantly-underperform-compared-to-running-them-in-a-single-process-even-though-the-tasks-are-cpu-bound","Q18. When submitting many quick tasks to a ",[32,2830,636],{},", why might ",[32,2833,2834],{},"pool.map()"," significantly underperform compared to running them in a single process, even though the tasks are CPU-bound?",[23,2837,2838,2841,2844,2847,2855,2858,2921,2983],{"language":16},[210,2839,2840],{},"from concurrent.futures import ProcessPoolExecutor",[210,2842,2843],{},"def square(x):\nreturn x * x",[210,2845,2846],{},"with ProcessPoolExecutor() as pool:\nresults = list(pool.map(square, range(1_000_000)))",[26,2848,2853],{"className":2849,"code":2851,"language":2852},[2850],"language-text","::\n\n- [ ] `ProcessPoolExecutor` never actually parallelizes anything; it's identical to a loop\n- [ ] For very cheap per-item work, the fixed cost of pickling each argument\u002Fresult and IPC round-trips per task can dwarf the actual computation time, making the overhead outweigh the parallelism gained; batching items (e.g., `chunksize`) or doing the work in one process is often faster\n- [ ] `pool.map()` silently caps at processing 1,000 items maximum\n- [ ] `ProcessPoolExecutor` requires manual chunking or it raises `MemoryError`\n\n\u003Cdetails>\n\u003Csummary>Show Answer\u003C\u002Fsummary>\n\n**Answer:** B — For cheap per-item work, pickling\u002FIPC overhead per task can dwarf actual computation, so overhead can outweigh parallelism gains; batching (`chunksize`) or single-process execution is often faster\n\n**Explanation:** **Performance:** Each call to `square(x)` here does trivial work, but by default `Pool.map`\u002F`ProcessPoolExecutor.map` may send items to workers with a small chunk size, meaning the per-task pickling\u002Funpickling and inter-process messaging cost can exceed the actual `x * x` computation many times over. The fix is to increase `chunksize` (grouping many items per IPC round-trip) or to recognize that sub-millisecond per-item tasks generally aren't good candidates for process-based parallelism at all — the overhead needs to be amortized over meaningfully larger units of work.\n\n\u003C\u002Fdetails>\n::\n\n::question-wrapper\n### Q19. For a long-running batch of independent CPU-bound tasks where results should be consumed as soon as each one finishes (not necessarily in submission order), which is more appropriate: `Pool.map()` or `Pool.imap_unordered()`?\n\n- [ ] `Pool.map()`, because it always finishes faster regardless of use case\n- [ ] `Pool.imap_unordered()`, because it yields each result as soon as any worker finishes it, rather than `map()`'s behavior of collecting all results and returning them (in original order) only once the entire batch completes\n- [ ] They are functionally identical; the name difference is purely cosmetic\n- [ ] `imap_unordered()` runs tasks sequentially in one process, unlike `map()`\n\n\u003Cdetails>\n\u003Csummary>Show Answer\u003C\u002Fsummary>\n\n**Answer:** B — `imap_unordered()` yields each result as soon as it's ready, rather than waiting for the whole batch like `map()`\n\n**Explanation:** **Idiom:** `Pool.map()` blocks until every task in the iterable is complete and then returns a list in the original submission order — fine when you need all results together, but wasteful if you want to start processing\u002Fstreaming results as they arrive. `imap_unordered()` returns an iterator that yields each finished result the moment its worker completes, in whatever order they finish, which better suits streaming pipelines or progress reporting, especially when task durations vary. Choosing `map()` by default even when order doesn't matter needlessly delays consuming the earliest-finished results.\n\n\u003C\u002Fdetails>\n::\n\n::question-wrapper{language=\"python\"}\n### Q20. A codebase mixes `threading.Lock` for protecting a shared cache with `multiprocessing.Process` workers that also read\u002Fwrite that same Python-level cache object. Why is this fundamentally broken, and what's the fix?\n\n::code-wrapper{language=\"python\"}\n```python\nimport threading\nfrom multiprocessing import Process\n\ncache = {}\nlock = threading.Lock()\n\ndef worker(key, value):\n    with lock:\n        cache[key] = value\n\nprocs = [Process(target=worker, args=(i, i * i)) for i in range(5)]\n","text",[32,2854,2851],{"__ignoreMap":30},[210,2856,2857],{},"::",[170,2859,2861,2870,2898,2908],{"className":2860},[173],[175,2862,2864,2866,2867,2869],{"className":2863},[178],[180,2865],{"disabled":52,"type":182}," It works correctly — ",[32,2868,1008],{}," synchronizes across processes just like within a single process",[175,2871,2873,222,2875,2877,2878,2880,2881,1965,2884,2887,2888,1505,2890,2893,2894,2897],{"className":2872},[178],[180,2874],{"disabled":52,"type":182},[32,2876,1008],{}," only synchronizes threads within the same process's memory; each ",[32,2879,538],{}," gets its own independent copy of ",[32,2882,2883],{},"lock",[32,2885,2886],{},"cache",", so the lock provides no cross-process protection at all, and updates in workers never appear in the parent's ",[32,2889,2886],{},[32,2891,2892],{},"multiprocessing.Manager().dict()"," with a ",[32,2895,2896],{},"multiprocessing.Lock()",", or another IPC-aware shared structure",[175,2899,2901,2903,2904,2907],{"className":2900},[178],[180,2902],{"disabled":52,"type":182}," The code raises ",[32,2905,2906],{},"TypeError"," at process-start time because locks can't be used in worker functions",[175,2909,2911,2913,2914,2916,2917,2920],{"className":2910},[178],[180,2912],{"disabled":52,"type":182}," It's correct, but only if ",[32,2915,2883],{}," is created inside ",[32,2918,2919],{},"worker()"," instead of at module scope",[203,2922,2923,2925,2943],{},[206,2924,208],{},[210,2926,2927,674,2929,2931,2932,2934,2935,1965,2937,2939,2940,2942],{},[213,2928,215],{},[32,2930,1008],{}," only works within one process's shared memory; each ",[32,2933,538],{}," gets an independent copy of ",[32,2936,2883],{},[32,2938,2886],{},", so it gives zero cross-process protection, and the parent's ",[32,2941,2886],{}," never updates",[210,2944,2945,222,2947,222,2949,2951,2952,2954,2955,2957,2958,2960,2961,2963,2964,2966,2967,1501,2969,2972,2973,2975,2976,2978,2979,2982],{},[213,2946,221],{},[213,2948,986],{},[32,2950,1008],{}," is implemented against in-process memory primitives — when a ",[32,2953,538],{}," is spawned\u002Fforked, it receives its own copy of the ",[32,2956,2883],{}," object (a distinct OS-level mutex, not a shared one) and its own copy of ",[32,2959,2886],{},", exactly like the ",[32,2962,777],{}," example in Q9. Acquiring \"the same\" lock in different processes acquires two entirely different locks, providing no real mutual exclusion across processes, and any writes to ",[32,2965,2886],{}," inside a worker only affect that worker's private copy — the parent's ",[32,2968,2886],{},[32,2970,2971],{},"{}",". The correct fix uses cross-process-aware primitives: ",[32,2974,2892],{}," for a proxy-backed shared dict, combined with ",[32,2977,2896],{}," (not ",[32,2980,2981],{},"threading.Lock()",") for true cross-process mutual exclusion.","\n::",[2985,2986,2987],"style",{},"html pre.shiki code .svdQ7, html code.shiki .svdQ7{--shiki-default:#D73A49;--shiki-github-dark:#F97583}html pre.shiki code .ssxIu, html code.shiki .ssxIu{--shiki-default:#24292E;--shiki-github-dark:#E1E4E8}html pre.shiki code .sIsaT, html code.shiki .sIsaT{--shiki-default:#6F42C1;--shiki-github-dark:#B392F0}html pre.shiki code .snvgF, html code.shiki .snvgF{--shiki-default:#005CC5;--shiki-github-dark:#79B8FF}html pre.shiki code .sCrzJ, html code.shiki .sCrzJ{--shiki-default:#E36209;--shiki-github-dark:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .github-dark .shiki span {color: var(--shiki-github-dark);background: var(--shiki-github-dark-bg);font-style: var(--shiki-github-dark-font-style);font-weight: var(--shiki-github-dark-font-weight);text-decoration: var(--shiki-github-dark-text-decoration);}html.github-dark .shiki span {color: var(--shiki-github-dark);background: var(--shiki-github-dark-bg);font-style: var(--shiki-github-dark-font-style);font-weight: var(--shiki-github-dark-font-weight);text-decoration: var(--shiki-github-dark-text-decoration);}html pre.shiki code .sJ6F3, html code.shiki .sJ6F3{--shiki-default:#032F62;--shiki-github-dark:#9ECBFF}",{"title":30,"searchDepth":49,"depth":49,"links":2989},[2990,2991,2993,2995,2997,2999,3001,3003,3005,3006,3007,3008,3010,3012,3013,3014,3015,3017],{"id":20,"depth":56,"text":21},{"id":231,"depth":56,"text":2992},"Q2. For which of these workloads does threading typically give a real, measurable speedup in CPython?",{"id":374,"depth":56,"text":2994},"Q3. Why does multiprocessing achieve real parallel speedup for CPU-bound work where threading does not?",{"id":601,"depth":56,"text":2996},"Q4. Given concurrent.futures, which executor should you pick for (a) downloading 200 files over HTTP and (b) computing 200 independent CPU-heavy Fibonacci-like calculations in pure Python?",{"id":699,"depth":56,"text":2998},"Q5. Why does a network call like socket.recv() or requests.get() allow other Python threads to make progress, even though the GIL normally lets only one thread run at a time?",{"id":773,"depth":56,"text":3000},"Q6. Despite the GIL serializing bytecode execution, this code can still print a final counter value less than 200000. Why?",{"id":1018,"depth":56,"text":3002},"Q7. What overhead does multiprocessing introduce that threading does not, when passing data to and from worker processes?",{"id":1182,"depth":56,"text":3004},"Q8. What happens when you try to pass this to a multiprocessing.Pool?",{"id":1353,"depth":56,"text":1354},{"id":1590,"depth":56,"text":1591},{"id":1775,"depth":56,"text":1776},{"id":1988,"depth":56,"text":3009},"Q12. What is the effect of daemon=True on a thread, and what happens without it?",{"id":2153,"depth":56,"text":3011},"Q13. What is likely to go wrong if this script (using the default spawn start method, e.g. on Windows or macOS) omits the if __name__ == \"__main__\": guard?",{"id":2331,"depth":56,"text":2332},{"id":2381,"depth":56,"text":2382},{"id":2581,"depth":56,"text":2582},{"id":2640,"depth":56,"text":3016},"Q17. A web server handler stores per-request state in a variable at module scope so different helper functions can access it without passing it explicitly. Under threading, why is this dangerous, and what's the fix?",{"id":2827,"depth":56,"text":3018},"Q18. When submitting many quick tasks to a ProcessPoolExecutor, why might pool.map() significantly underperform compared to running them in a single process, even though the tasks are CPU-bound?","md",{},"\u002Fpython\u002F21-concurrency-threading-and-multiprocessing",{"title":5,"description":30},"python\u002F21-concurrency-threading-and-multiprocessing","rPQDNWVoKzJNAMU3UvInJxSrR33EKbOtHDEqplJVe5o",1787335398315]