Inspecting Call Stacks

In this book, for many purposes, we need to lookup a function's location, source code, or simply definition. The class StackInspector provides a number of convenience methods for this purpose.

Prerequisites

• This is an internal helper class.
• Understanding how frames and local variables are represented in Python helps.

Inspecting Call Stacks

StackInspector is a class that provides a number of utility functions to inspect a call stack, notably to identify caller functions.

When tracing or instrumenting functions, a common issue is to identify the currently active functions. A typical situation is depicted below, where my_inspector() currently traces a function called function_under_test().

Function	Class
...	StackInspector
caller_frame()	StackInspector	invokes $\uparrow$
caller_function()	StackInspector	invokes $\uparrow$
my_inspector()	some inspector; a subclass of StackInspector	invokes $\uparrow$
function_under_test()	(any)	is traced by $\uparrow$
-/-	(any)	invokes $\uparrow$

To determine the calling function, my_inspector() could check the current frame and retrieve the frame of the caller. However, this caller could be some tracing function again invoking my_inspector(). Therefore, StackInspector provides a method caller_function() that returns the first caller outside of a StackInspector class. This way, a subclass of StackInspector can define an arbitrary set of functions (and call stack); caller_function() will always return a function outside of the StackInspector subclass.

# ignore
from typing import cast, Dict, Any, Tuple, Callable, Optional, Type

The method caller_frame() walks the current call stack from the current frame towards callers (using the f_back attribute of the current frame) and returns the first frame that is not a method or function from the current StackInspector class or its subclass. To determine this, the method our_frame() determines whether the given execution frame refers to one of the methods of StackInspector or one of its subclasses.

class StackInspector:
    """Provide functions to inspect the stack"""

    def caller_frame(self) -> FrameType:
        """Return the frame of the caller."""

        # Walk up the call tree until we leave the current class
        frame = cast(FrameType, inspect.currentframe())

        while self.our_frame(frame):
            frame = cast(FrameType, frame.f_back)

        return frame

    def our_frame(self, frame: FrameType) -> bool:
        """Return true if `frame` is in the current (inspecting) class."""
        return isinstance(frame.f_locals.get('self'), self.__class__)

When we access program state or execute functions, we do so in the caller's environment, not ours. The caller_globals() method acts as replacement for globals(), using caller_frame().

class StackInspector(StackInspector):
    def caller_globals(self) -> Dict[str, Any]:
        """Return the globals() environment of the caller."""
        return self.caller_frame().f_globals

    def caller_locals(self) -> Dict[str, Any]:
        """Return the locals() environment of the caller."""
        return self.caller_frame().f_locals

The method caller_location() returns the caller's function and its location. It does a fair bit of magic to retrieve nested functions, by looking through global and local variables until a match is found. This may be simplified in the future.

Location = Tuple[Callable, int]

class StackInspector(StackInspector):
    def caller_location(self) -> Location:
        """Return the location (func, lineno) of the caller."""
        return self.caller_function(), self.caller_frame().f_lineno

The function search_frame() allows to search for an item named name, walking up the call stack. This is handy when trying to find local functions during tracing, for whom typically only the name is provided.

class StackInspector(StackInspector):
    def search_frame(self, name: str, frame: Optional[FrameType] = None) -> \
        Tuple[Optional[FrameType], Optional[Callable]]:
        """
        Return a pair (`frame`, `item`) 
        in which the function `name` is defined as `item`.
        """
        if frame is None:
            frame = self.caller_frame()

        while frame:
            item = None
            if name in frame.f_globals:
                item = frame.f_globals[name]
            if name in frame.f_locals:
                item = frame.f_locals[name]
            if item and callable(item):
                return frame, item

            frame = cast(FrameType, frame.f_back)

        return None, None

    def search_func(self, name: str, frame: Optional[FrameType] = None) -> \
        Optional[Callable]:
        """Search in callers for a definition of the function `name`"""
        frame, func = self.search_frame(name, frame)
        return func

If we cannot find a function by name, we can create one, using create_function().

class StackInspector(StackInspector):
    # Avoid generating functions more than once
    _generated_function_cache: Dict[Tuple[str, int], Callable] = {}

    def create_function(self, frame: FrameType) -> Callable:
        """Create function for given frame"""
        name = frame.f_code.co_name
        cache_key = (name, frame.f_lineno)
        if cache_key in self._generated_function_cache:
            return self._generated_function_cache[cache_key]

        try:
            # Create new function from given code
            generated_function = cast(Callable,
                                      FunctionType(frame.f_code,
                                                   globals=frame.f_globals,
                                                   name=name))
        except TypeError:
            # Unsuitable code for creating a function
            # Last resort: Return some function
            generated_function = self.unknown

        except Exception as exc:
            # Any other exception
            warnings.warn(f"Couldn't create function for {name} "
                          f" ({type(exc).__name__}: {exc})")
            generated_function = self.unknown

        self._generated_function_cache[cache_key] = generated_function
        return generated_function

The method caller_function() puts all of these together, simply looking up and returning the currently calling function – and creating one if it cannot be found.

class StackInspector(StackInspector):
    def caller_function(self) -> Callable:
        """Return the calling function"""
        frame = self.caller_frame()
        name = frame.f_code.co_name
        func = self.search_func(name)
        if func:
            return func

        if not name.startswith('<'):
            warnings.warn(f"Couldn't find {name} in caller")

        return self.create_function(frame)

    def unknown(self) -> None:  # Placeholder for unknown functions
        pass

The method is_internal_error() allows us to differentiate whether some exception was raised by StackInspector (or a subclass) – or whether it was raised by the inspected code.

class StackInspector(StackInspector):
    def is_internal_error(self, exc_tp: Type, 
                          exc_value: BaseException, 
                          exc_traceback: TracebackType) -> bool:
        """Return True if exception was raised from `StackInspector` or a subclass."""
        if not exc_tp:
            return False

        for frame, lineno in traceback.walk_tb(exc_traceback):
            if self.our_frame(frame):
                return True

        return False

Synopsis

StackInspector is typically used as superclass, providing its functionality to subclasses.

Here is an example of how to use caller_function(). The test() function invokes an internal method caller() of StackInspectorDemo, which in turn invokes callee():

Function	Class
callee()	StackInspectorDemo
caller()	StackInspectorDemo	invokes $\uparrow$
test()	(main)	invokes $\uparrow$
-/-	(main)	invokes $\uparrow$

Using caller_function(), callee() determines the first caller outside of a StackInspector class and prints it out – i.e., <function test>.

class StackInspectorDemo(StackInspector):
    def callee(self) -> None:
        func = self.caller_function()
        assert func.__name__ == 'test'
        print(func)

    def caller(self) -> None:
        self.callee()

def test() -> None:
    demo = StackInspectorDemo()
    demo.caller()

test()

<function test at 0x7fe5cfcc07b8>

Here are all methods defined in this chapter:

# ignore
from ClassDiagram import display_class_hierarchy, class_tree

# ignore
display_class_hierarchy([StackInspector],
                        abstract_classes=[
                            StackInspector,
                        ],
                        public_methods=[
                            StackInspector.caller_frame,
                            StackInspector.caller_function,
                            StackInspector.caller_globals,
                            StackInspector.caller_locals,
                            StackInspector.caller_location,
                            StackInspector.search_frame,
                            StackInspector.search_func,
                            StackInspector.is_internal_error,
                            StackInspector.our_frame,
                        ],
                        project='debuggingbook')

Lessons Learned

• In Python, inspecting objects at runtime is easy.