modulegraph2 reference documentation

A module import dependency graph for Python projects.

This package defines a class representing the dependency graph between a collection of python modules and scripts, as well as supporting functions and classes.

The graph itself is an subclass of objectgraph.ObjectGraph.

This module provides annotation for use with Mypy.

Graph

class modulegraph2.ModuleGraph(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)

Class representing the dependency graph between a collection of python modules and scripts.

The roots of the graph are those nodes that are added to the graph using add_script() and add_module().

Parameters:
  • use_stdlib_implies (*) – Use the built-in implied actions for the stdlib.
  • use_builtin_hooks (*) – Use the built-in extension hooks

Creating the graph

ModuleGraph.__init__(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)

Create a new empty graph

ModuleGraph.add_module(module_name: str) → modulegraph2._nodes.BaseNode

Add a module to the graph and process imports.

Will not raise an exception for non-existing modules.

Parameters:module_name – Name of the module to import.
ModuleGraph.add_script(script_path: os.PathLike) → modulegraph2._nodes.Script

Add a script to the module graph and process imports

Parameters:

script_path – Filesystem path for the script to be added

Returns:

The script node for the just added script

Raises:
  • ValueError – If the script is already part of the graph
  • OSError – If the script cannot be opened.
  • SyntaxError – If the script is invalid
ModuleGraph.add_distribution(distribution: Union[modulegraph2._distributions.PyPIDistribution, str])

Add a package distribution to the graph, with references to all importable names in that distribution

Parameters:distribution – A distribution or distribution name
Returns
The node added to the graph

Affecting building the graph

ModuleGraph.add_excludes(excluded_names: Iterator[str]) → None

Exclude the names in “excludeded_names” from the graph

Excluded names can end up as ExcludedNode nodes in the graph, but the dependencies of the actual module are not gathered.

  • excluded_names: An interator yielding names to exclude.
ModuleGraph.add_implies(implies: Dict[str, Union[None, modulegraph2._implies.Alias, modulegraph2._implies.Virtual, Sequence[str]]]) → None

Add implied actions for the graph.

An implied action can be used for three purposes: * A list of dependencies.

Commonly used to add module dependencies for modules that modulegraph2 cannot scan, such as extensions and modules using __import__().
  • An Alias for another node

    Used to mark an importable name as an alias for some other module. An example of this is os.path, which is an alias to a platform specific path module (such as posixpath.

  • A Virtual module

    Used to mark an importable name as a virtual module that is added to sys.modules by some other module.

Parameters:implies – A mapping from module names to implied actions

Callbacks

ModuleGraph.add_missing_hook(hook: Callable[[ModuleGraph, Optional[modulegraph2._nodes.BaseNode], str], Optional[modulegraph2._nodes.BaseNode]]) → None

Add a hook function that’s used to try to resolve a missing module.

The hook functions are called in reverse order of addition, and the result of the first hook that doesn’t return None is used in the graph.

Parameters:hook – The hook function. Run hook(self, importing_module, module_name) before creating a MissingModule node for module_name.
ModuleGraph.add_post_processing_hook(hook: Callable[[ModuleGraph, modulegraph2._nodes.BaseNode], None]) → None

Add a hook function to be ran whenever a node is fully processed.

It is possible to add multip hooks by calling this method multiple times.

Parameters:
  • hook – The post processing hook. Run hook(self, node)
  • node is fully processed. (when) –

Reporting

ModuleGraph.distributions(reachable: bool = True) → Iterator[modulegraph2._distributions.PyPIDistribution]

Yield all distributions used in the graph.

If reachable is True (default) this reports only on distributions used by nodes reachable from one of the graph roots, otherwise this reports on distributions used by any root.

This will not report PyPIDistributions that are nodes in the graph, unless they are also the distribution attribute of a node.

Parameters:reacable – IF true only report on nodes that are reachable from a graph root, otherwise report on all nodes.
ModuleGraph.report(file: TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) → None

Print information about nodes reachable from the graph roots to the given file.

Parameters:file – Stream to write to

Graph Nodes

The graph can contain nodes of a number of classes, as described below. All these classes are dataclasses.

class modulegraph2.BaseNode(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict)

Bases: object

Base class for all module nodes in the dependency graph.

name

Name of the module.

loader

Importlib loader for the module when available.

distribution

Package distribution that contains this module. None for the stdlib and uninstalled modules.

filename

Filesystem path to the module when available

extension_attributes

A dictionary for use by users for the modulegraph2 library, not used by modulegraph2 itself.

identifier

Graph identifier for use with objectgraph.ObjectGraph.

class modulegraph2.Module(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.BaseNode

Information about a Module

globals_written

Set of global names written to by the module

globals_read

Set of global names read by the module

uses_dunder_file

True if the module appears to use the __file__ attribute.

uses_dunder_import

True if the module appears to use the __import__() function.

class modulegraph2.BuiltinModule(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.Module

Node representing a built-in extension module.

class modulegraph2.BytecodeModule(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.Module

Node representing a python module for which only byte code is available.

class modulegraph2.ExtensionModule(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.Module

Node representing a native extension module.

class modulegraph2.FrozenModule(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.Module

Note representing a python module that is frozen into an executable. This has source code available, but not on the filesystem.

class modulegraph2.SourceModule(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str])

Bases: modulegraph2._nodes.Module

Node representing a python module for which the source code is available.

class modulegraph2.NamespacePackage(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, search_path: List[pathlib.Path], has_data_files: bool)

Bases: modulegraph2._nodes.BaseNode

Node representing an implicit namespace package (PEP 420).

search_path

The search path for modules in this package.

globals_read

Always an empty set

globals_written

Always an empty set

class modulegraph2.Package(name: str, loader: Optional[importlib.abc.Loader], distribution: Optional[modulegraph2._distributions.PyPIDistribution], filename: Optional[pathlib.Path], extension_attributes: dict, init_module: modulegraph2._nodes.BaseNode, search_path: List[pathlib.Path], has_data_files: bool, namespace_type: Optional[str])

Bases: modulegraph2._nodes.BaseNode

Node representing a namespace package with an __init__ module.

init_module

Node representing the __init__ module for this package.

search_path

The search path for modules in this package.

has_data_files

True if this package contains data files (other than empty directories and python files).

namespace_type

None, “pkgutil” or “pkg_resources” for regular packages, namespace packages using pkgutil and namespace packages using pkg_resources.

globals_read

The globals read from by the module __init__

globals_written

The globals written to by the module __init__

class modulegraph2.Script(filename: os.PathLike)

Bases: modulegraph2._nodes.BaseNode

Not representing a Python script.

The name of the node is the string representation of the full filename of the script.

Special nodes

class modulegraph2.VirtualNode(module_name, providing_module)

Bases: modulegraph2._nodes.BaseNode

Node representing a virtual module, that is added to sys.modules by some other module.

Attributes
providing_module
The module that creates this module in sys.modules.
class modulegraph2.AliasNode(module_name, actual_module)

Bases: modulegraph2._nodes.BaseNode

Node representing a module alias, that is an imported name that refers to some other module.

Attributes
actual_module
The module that this name aliases to.
class modulegraph2.ExcludedModule(module_name)

Bases: modulegraph2._nodes.BaseNode

Node representing a module that is explicitly excluded by the user.

class modulegraph2.MissingModule(module_name)

Bases: modulegraph2._nodes.BaseNode

Node representing a name that is imported, but could not be located.

class modulegraph2.InvalidRelativeImport(module_name)

Bases: modulegraph2._nodes.BaseNode

Node representing a relative import that refers to a location outside of a toplevel package.

The name is a name starting with one or more dots.

Edge attributes

class modulegraph2.DependencyInfo(is_optional: bool, is_global: bool, in_fromlist: bool, imported_as: Optional[str])

A frozen dataclass representing information about the dependency edge between two graph nodes.

is_optional

True if the import appears to be optional

is_global

True if the import affects global names in the module

in_fromlist

True if the name is imported in the name list of an from ... import ... statement

imported_as

Rename for this module (import ... as impoted_as), None when there is no as clause.

Distributions

class modulegraph2.PyPIDistribution(identifier: str, name: str, version: str, files: Set[str], import_names: Set[str])

Information about a package distribution

identifier

Unique identifier fot the distribution for use with modulegraph2.ObjectGraph

Type:str
name

Name of the distribution (as it is found on PyPI)

Type:str
files

Files that are part of this distribution

Type:Set[str]
import_names

The importable names in this distribution (modules and packages)

Type:Set[str]

Note

The information about distributions is fairly minimal at this point, and will be enhanced as needed.

contains_file(filename: Union[str, os.PathLike])

Check if a file is part of this distribution.

Parameters:filename – The filename to look for
Returns:True if filename is part of this distribution, otherwise False.
modulegraph2.distribution_named(name: str, path: Optional[Iterable[str]] = None) → Optional[modulegraph2._distributions.PyPIDistribution]

Find a named distribution on the search path.

Parameters:
  • name – Distribution name to look for.
  • path – Module search path (defaults to sys.path)
Returns:

The distribution, or None

modulegraph2.all_distributions(path: Optional[Iterable[str]] = None) → Iterator[modulegraph2._distributions.PyPIDistribution]

Yield all distributions found on the search path.

Parameters:path – Module search path (defaults to sys.path).

Utilities

modulegraph2.saved_sys_path()

Contextmanager that will restore the value of sys.path when leaving the with block.

class modulegraph2.Alias
class modulegraph2.Virtual