modulegraph2 reference documentation ¶

A module import dependency graph for Python projects.

This package defines a class representing the dependency graph between a collection of python modules and scripts, as well as supporting functions and classes.

The graph itself is an subclass of objectgraph.ObjectGraph.

This module provides annotation for use with Mypy.

Graph ¶

class modulegraph2.ModuleGraph(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)¶

Class representing the dependency graph between a collection of python modules and scripts.

The roots of the graph are those nodes that are added to the graph using add_script(), add_module() and add_dependencies_for_source().

Parameters:

use_stdlib_implies (*) – Use the built-in implied actions for the stdlib.
use_builtin_hooks (*) – Use the built-in extension hooks

Creating the graph ¶

ModuleGraph.__init__(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)¶: Create a new empty graph

ModuleGraph.add_module(module_name: str) → BaseNode¶

Add a module to the graph and process imports.

Will not raise an exception for non-existing modules.

Parameters:: module_name – Name of the module to import.

ModuleGraph.add_script(script_path: PathLike) → Script¶

Add a script to the module graph and process imports

Parameters:

script_path – Filesystem path for the script to be added

Returns:

The script node for the just added script

Raises:

ValueError – If the script is already part of the graph
OSError – If the script cannot be opened.
SyntaxError – If the script is invalid

ModuleGraph.add_distribution(distribution: PyPIDistribution | str) → PyPIDistribution | str¶

Add a package distribution to the graph, with references to all importable names in that distribution

Parameters:: distribution – A distribution or distribution name

Returns: The node added to the graph

ModuleGraph.add_dependencies_for_source(source_code: str) → None¶

Add the modules imported by the Python code in source_code to the graph as roots.

Parameters:: source_code – Source code for a python script or module

Added in version 2.3.

Affecting building the graph ¶

ModuleGraph.add_excludes(excluded_names: Iterable[str]) → None¶

Exclude the names in “excludeded_names” from the graph

Excluded names can end up as ExcludedNode nodes in the graph, but the dependencies of the actual module are not gathered.

excluded_names: An interator yielding names to exclude.

ModuleGraph.add_implies(implies: dict[str, None | Alias | Virtual | Sequence[str]]) → None¶

Add implied actions for the graph.

An implied action can be used for three purposes: * A list of dependencies.

Commonly used to add module dependencies for modules that modulegraph2 cannot scan, such as extensions and modules using __import__().

An Alias for another node

Used to mark an importable name as an alias for some other module. An example of this is os.path, which is an alias to a platform specific path module (such as posixpath.
A Virtual module

Used to mark an importable name as a virtual module that is added to sys.modules by some other module.

Parameters:: implies – A mapping from module names to implied actions

Callbacks ¶

ModuleGraph.add_missing_hook(hook: Callable[[ModuleGraph, BaseNode | None, str], BaseNode | None]) → None¶

Add a hook function that’s used to try to resolve a missing module.

The hook functions are called in reverse order of addition, and the result of the first hook that doesn’t return None is used in the graph.

Parameters:: hook – The hook function. Run hook(self, importing_module, module_name) before creating a MissingModule node for module_name.

ModuleGraph.add_post_processing_hook(hook: Callable[[ModuleGraph, BaseNode], None]) → None¶

Add a hook function to be ran whenever a node is fully processed.

It is possible to add multip hooks by calling this method multiple times.

Parameters:

hook – The post processing hook. Run hook(self, node)
at (when node is fully processed. The hook will be run)
node (most once per)
cause (even if recipes or hooks)
node. (re-processing of a)

ModuleGraph.hook_context() → Iterator[None]¶

Contextmanger that allows using hook APIs outside of a hook callback.

Usage:

with mg.hook_context():
    mg.import_module(...)

Added in version 2.4.

ModuleGraph.import_module(importing_module: BaseNode, import_name: str) → BaseNode¶

Import ‘import_name’ and add an edge from ‘module’ to ‘import_name’

This is an API to be used by hooks. The graph is not fully updated after calling this method.

Parameters:

importing_module – The module that triggers this import
import_name – The name that should be imported

Returns: The graph node for import_name.

ModuleGraph.import_package(importing_module: BaseNode, package_name: str) → BaseNode¶

Add all modules in a package to the graph and process imports.

Will not raise an exception for non-existing packages or when the package_name refers to a module instead of a package.

This is an API to be used by hooks. The graph is not fully updated after calling this method.

This method only supports packages that are found in the filesystem.

Parameters:

importing_module – The module that triggers this import
package_name – Name of the package to import

Added in version 2.3.

Reporting ¶

ModuleGraph.distributions(reachable: bool = True) → Iterator[PyPIDistribution]¶

Yield all distributions used in the graph.

If reachable is True (default) this reports only on distributions used by nodes reachable from one of the graph roots, otherwise this reports on distributions used by any root.

This will not report PyPIDistributions that are nodes in the graph, unless they are also the distribution attribute of a node.

Parameters:: reachable – IF true only report on nodes that are reachable from a graph root, otherwise report on all nodes.

ModuleGraph.report(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None¶

Print information about nodes reachable from the graph roots to the given file.

Parameters:: file – Stream to write to

Graph Nodes ¶

The graph can contain nodes of a number of classes, as described below. All these classes are dataclasses.

class modulegraph2.BaseNode(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict)¶

Bases: object

Base class for all module nodes in the dependency graph.

name¶

Name of the module.

Type:: str

loader¶

Importlib loader for the module when available.

Type:: importlib._abc.Loader | None

distribution¶

Package distribution that contains this module. None for the stdlib and uninstalled modules.

Type:: modulegraph2._distributions.PyPIDistribution | None

filename¶

Filesystem path to the module when available

Type:: pathlib.Path | None

extension_attributes¶

A dictionary for use by users for the modulegraph2 library, not used by modulegraph2 itself.

Type:: dict

property identifier: str¶: Graph identifier for use with objectgraph.ObjectGraph.

class modulegraph2.Module(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: BaseNode

Information about a Module

globals_written¶

Set of global names written to by the module

Type:: set[str]

globals_read¶

Set of global names read by the module

Type:: set[str]

code¶

Code for the module

Type:: types.CodeType | None

Added in version 2.2: The code attribute

property uses_dunder_file: bool¶: True if the module appears to use the __file__ attribute.

property uses_dunder_import: bool¶: True if the module appears to use the __import__() function.

class modulegraph2.BuiltinModule(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: Module

Node representing a built-in extension module.

class modulegraph2.BytecodeModule(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: Module

Node representing a python module for which only byte code is available.

class modulegraph2.ExtensionModule(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: Module

Node representing a native extension module.

class modulegraph2.FrozenModule(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: Module

Node representing a python module that is frozen into an executable. This has source code available, but not on the filesystem.

class modulegraph2.SourceModule(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, globals_written: set[str], globals_read: set[str], code: CodeType | None)¶

Bases: Module

Node representing a python module for which the source code is available.

class modulegraph2.NamespacePackage(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, search_path: list[Path], has_data_files: bool)¶

Bases: BaseNode

Node representing an implicit namespace package (PEP 420).

search_path¶

The search path for modules in this package.

Type:: list[pathlib.Path]

property globals_read¶: Always an empty set

property globals_written¶: Always an empty set

class modulegraph2.Package(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, init_module: BaseNode, search_path: list[Path], has_data_files: bool, namespace_type: str | None)¶

Bases: BaseNode

Node representing a namespace package with an __init__ module.

init_module¶

Node representing the __init__ module for this package.

Type:: modulegraph2._nodes.BaseNode

search_path¶

The search path for modules in this package.

Type:: list[pathlib.Path]

has_data_files¶

True if this package contains data files (other than empty directories and python files).

Type:: bool

namespace_type¶

None, “pkgutil” or “pkg_resources” for regular packages, namespace packages using pkgutil and namespace packages using pkg_resources.

Type:: str | None

property globals_read¶: The globals read from by the module __init__

property globals_written¶: The globals written to by the module __init__

class modulegraph2.FrozenPackage(name: str, loader: Loader | None, distribution: PyPIDistribution | None, filename: Path | None, extension_attributes: dict, init_module: BaseNode, search_path: list[Path], has_data_files: bool, namespace_type: str | None)¶

Bases: Package

Node representing a python package that is frozen into an executable. This has source code available, but not on the filesystem.

Added in version 2.4.

class modulegraph2.Script(filename: PathLike, code: CodeType | None)¶

Bases: BaseNode

Node representing a Python script.

The name of the node is the string representation of the full filename of the script.

globals_written¶

Global varialbles written to

Type:: set[str]

globals_read¶

Global variables read from

Type:: set[str]

code¶

Code object for the script

Type:: types.CodeType | None

Added in version 2.2: The code attribute

Special nodes ¶

class modulegraph2.VirtualNode(module_name, providing_module)¶

Bases: BaseNode

Node representing a virtual module, that is added to sys.modules by some other module.

Attributes

providing_module: The module that creates this module in sys.modules.

class modulegraph2.AliasNode(module_name, actual_module)¶

Bases: BaseNode

Node representing a module alias, that is an imported name that refers to some other module.

Attributes

actual_module: The module that this name aliases to.

class modulegraph2.ExcludedModule(module_name)¶

Bases: BaseNode

Node representing a module that is explicitly excluded by the user.

class modulegraph2.MissingModule(module_name)¶

Bases: BaseNode

Node representing a name that is imported, but could not be located.

class modulegraph2.InvalidRelativeImport(module_name)¶

Bases: BaseNode

Node representing a relative import that refers to a location outside of a toplevel package.

The name is a name starting with one or more dots.

Edge attributes ¶

class modulegraph2.DependencyInfo(is_optional: bool, is_global: bool, in_fromlist: bool, imported_as: str | None)¶

A frozen dataclass representing information about the dependency edge between two graph nodes.

is_optional¶

True if the import appears to be optional

Type:: bool

is_global¶

True if the import affects global names in the module

Type:: bool

in_fromlist¶

True if the name is imported in the name list of an from ... import ... statement

Type:: bool

imported_as¶

Rename for this module (import ... as impoted_as), None when there is no as clause.

Type:: str | None

Distributions ¶

class modulegraph2.PyPIDistribution(identifier: str, name: str, version: str, files: set[str], import_names: set[str], extension_attributes: dict)¶

Information about a package distribution

identifier¶

Unique identifier fot the distribution for use with modulegraph2.ObjectGraph

Type:: str

name¶

Name of the distribution (as it is found on PyPI)

Type:: str

files¶

Files that are part of this distribution

Type:: Set[str]

import_names¶

The importable names in this distribution (modules and packages)

Type:: Set[str]

extension_attributes¶

A dictionary for use by users for the modulegraph2 library, not used by modulegraph2 itself.

Type:: dict

Note

The information about distributions is fairly minimal at this point, and will be enhanced as needed.

contains_file(filename: str | PathLike)¶

Check if a file is part of this distribution.

Parameters:: filename – The filename to look for
Returns:: True if filename is part of this distribution, otherwise False.

modulegraph2.distribution_named(name: str, path: Iterable[str] | None = None) → PyPIDistribution | None¶

Find a named distribution on the search path.

Parameters:

name – Distribution name to look for.
path – Module search path (defaults to sys.path)

Returns:

The distribution, or None

modulegraph2.all_distributions(path: Iterable[str] | None = None) → Iterator[PyPIDistribution]¶

Yield all distributions found on the search path.

Parameters:: path – Module search path (defaults to sys.path).

Utilities ¶

modulegraph2.saved_sys_path()¶: Contextmanager that will restore the value of sys.path when leaving the with block.

modulegraph2.stdlib_module_names() → list[str]¶: Return a list of modules in the standard library

class modulegraph2.Alias¶

class modulegraph2.Virtual¶