Python Object Graph

This is a great way to know the object graph 

objgraph is a module that lets you visually explore Python object graphs.

You’ll need graphviz if you want to draw the pretty graphs.

I recommend xdot for interactive use. pip install xdot should suffice; objgraph will automatically look for it in your PATH.

nstallation and Documentation

pip install objgraph or download it from PyPI.

Documentation lives at https://mg.pov.lt/objgraph.

Below is a simple object graph 

x = []

>>> y = [x, [x], dict(x=x)]

>>> import objgraph

>>> objgraph.show_refs([y], filename='sample-graph.png')

Graph written to ....dot (... nodes)

Image generated as sample-graph.png

To get the back references, just need to try the below 

objgraph.show_backrefs([x], filename='sample-backref-graph.png')

... 

Graph written to ....dot (8 nodes)

Image generated as sample-backref-graph.png

Below is a memory leak example 

objgraph.show_most_common_types() 

tuple                      5224

function                   1329

wrapper_descriptor         967

dict                       790

builtin_function_or_method 658

method_descriptor          340

weakref                    322

list                       168

member_descriptor          167

type 

But that’s looking for a small needle in a large haystack. Can we limit our haystack to objects that were created recently? Perhaps.

Let’s define a function that “leaks” memory

class MyBigFatObject(object):

...     pass

...

>>> def computate_something(_cache={}):

...     _cache[42] = dict(foo=MyBigFatObject(),

...                       bar=MyBigFatObject())

...     # a very explicit and easy-to-find "leak" but oh well

...     x = MyBigFatObject() # this one doesn't leak

We take a snapshot of all the objects counts that are alive before we call our function

>>> 

objgraph.show_growth(limit=3) 

tuple                  5228     +5228

function               1330     +1330

wrapper_descriptor      967      +967

and see what changes after we call it

computate_something()

>>> objgraph.show_growth() 

MyBigFatObject        2        +2

dict                797        +1

It’s easy to see MyBigFatObject instances that appeared and were not freed. I can pick one of them at random and trace the reference chain back to one of the garbage collector’s roots.

For simplicity’s sake let’s assume all of the roots are modules. objgraph provides a function, is_proper_module(), to check this. If you’ve any examples where that isn’t true, I’d love to hear about them (although see Reference counting bugs).

import random

>>> objgraph.show_chain(

...     objgraph.find_backref_chain(

...         random.choice(objgraph.by_type('MyBigFatObject')),

...         objgraph.is_proper_module),

...     filename='chain.png')

Graph written to ...dot (13 nodes)

Image generated as chain.png

References:

https://mg.pov.lt/objgraph/