Extending and Embedding Python, Release 3.9.5 Chapter 2. Creating extensions without third party tools 16 While Python uses the traditional reference counting implementation, it also offers a cycle detector that works to detect reference cycles. This allows applications to not worry about creating direct or indirect circular references; these are the weakness of garbage collection implemented using only reference counting. Reference cycles consist of objects which contain (possibly indirect) references to themselves, so that each object in the cycle has a reference count which is non-zero. Typical reference counting implementations are not able to reclaim the memory belonging An alternative strategy is called automatic garbage collection . (Sometimes, reference counting is also referred to as a garbage collection strategy, hence my use of “automatic” to distinguish the two.) The big advantage of automatic garbage collection is that the user doesn’t need to call$_{free()}$ explicitly. (Another claimed advantage is an improve- ment in speed or memory usage — this is no hard fact however.) The disadvantage is that for C, there is no truly portable automatic garbage collector, while reference counting can be implemented portably (as long as the functions malloc() and$_{free()}$ are available — which the C Standard guarantees). Maybe some day a sufficiently portable automatic garbage collector will be available for C. Until then, we’ll have to live with reference counts. Since Python makes heavy use of$_{malloc()}$ and$_{free()}$ , it needs a strategy to avoid memory leaks as well as the use of freed memory. The chosen method is called reference counting . The principle is simple: every object contains a counter, which is incremented when a reference to the object is stored somewhere, and which is decremented when a reference to it is deleted. When the counter reaches zero, the last reference to the object has been deleted and the object is freed. Common causes of memory leaks are unusual paths through the code. For instance, a function may allocate a block of memory, do some calculation, and then free the block again. Now a change in the requirements for the function may add a test to the calculation that detects an error condition and can return prematurely from the function. It’s easy to forget to free the allocated memory block when taking this premature exit, especially when it is added later to the code. Such leaks, once introduced, often go undetected for a long time: the error exit is taken only in a small fraction of all calls, and most modern machines have plenty of virtual memory, so the leak only becomes apparent in a long-running process that uses the leaking function frequently. Therefore, it’s important to prevent leaks from happening by having a coding convention or strategy that minimizes this kind of errors. Every block of memory allocated with$_{malloc()}$ should eventually be returned to the pool of available memory by exactly one call to$_{free()}$ . It is important to call$_{free()}$ at the right time. If a block’s address is forgotten but free() is not called for it, the memory it occupies cannot be reused until the program terminates. This is called a memory leak . On the other hand, if a program calls$_{free()}$ for a block and then continues to use the block, it creates a conflict with re-use of the block through another$_{malloc()}$ call. This is called using freed memory . It has the same bad consequences as referencing uninitialized data — core dumps, wrong results, mysterious crashes. In languages like C or C++, the programmer is responsible for dynamic allocation and deallocation of memory on the heap. In C, this is done using the functions$_{malloc()}$ and$_{free()}$ . In C++, the operators$_{new}$and$_{delete}$ are used with essentially the same meaning and we’ll restrict the following discussion to the C case. 2.1.10 Reference Counts Py_BuildValue("") None Py_BuildValue("i", 123) 123 Py_BuildValue("iii", 123, 456, 789) (123, 456, 789) Py_BuildValue("s", "hello") 'hello' Py_BuildValue("y", "hello") b'hello' Py_BuildValue("ss", "hello", "world") ('hello', 'world') Py_BuildValue("s#", "hello", 4)'hell' Py_BuildValue("y#", "hello", 4)b'hell' Py_BuildValue("()") () Py_BuildValue("(i)", 123) (123,) Py_BuildValue("(ii)", 123, 456) (123, 456) Py_BuildValue("(i,i)", 123, 456) (123, 456) Py_BuildValue("[i,i]", 123, 456) [123, 456] Py_BuildValue("{s:i,s:i}", "abc", 123, "def", 456) {'abc': 123, 'def': Py_BuildValue("((ii)(ii)) (ii)", 1, 2, 3, 4, 5, 6)(((1, 2), (3, 4)), (5, 6))