Differences

This shows you the differences between two versions of the page.

--- python [2025/08/21 09:57] – carl
+++ python [2026/01/12 10:13] (current) – carl
@@ Line 26: / Line 26: @@
+==== Problems with Multiprocessing ====
+Caching comes in many forms, but one surprising way that it shows up is in connection caching (a.k.a. pooling).  Without special handling, connections cannot be shared between processes.   When a Python program spawns another process, the new process does not inherit the file descriptors (fds) and thus has to re create them itself.  This has many problems:
+  * For socket fds, reconnecting is slow. for the caller
+  * For socket fds, re-establing an SSL connection is very slow and CPU intensive
+  * For socket fds, reconnecting causes the remote endpoint to burn resources as well.
+  * For file fds, its not possible to synchronize access to them.  (if Python had better shared memory support and the correct atomics to modify the memory, this may be possible).
+Multiprocessing skips the ''atexit'' hooks, meaning it's not possible to do process-wide cleanup work.  They are silently skipped.
+==== Problems with Descriptors ====
+Evaluation order matters.  For example:
+<code python>
+a: Descriptor
+b: Descriptor
+foo(a, b)
+</code>
+In this, if a or b have side-effects on evaluation, like throwing an exception, the order in which a and b are invoked matters.
+==== Futures ====
+  * Futures don't have a getstate() method.  This means trying to find out the running/finished/cancel/pending states of a future are racy.   The lock (condition) on the future is also private, so it's not safe to lock and query each of the state methods.