modulegraph2 reference documentation

A module import dependency graph for Python projects.

This package defines a class representing the dependency graph between a collection of python modules and scripts, as well as supporting functions and classes.

The graph itself is an subclass of objectgraph.ObjectGraph.

This module provides annotation for use with Mypy.

Graph

class modulegraph2.ModuleGraph(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)

Class representing the dependency graph between a collection of python modules and scripts.

The roots of the graph are those nodes that are added to the graph using add_script() and add_module().

Parameters

  • use_stdlib_implies (*) – Use the built-in implied actions for the stdlib.

  • use_builtin_hooks (*) – Use the built-in extension hooks

Creating the graph

ModuleGraph.__init__(*, use_stdlib_implies: bool = True, use_builtin_hooks: bool = True)

Create a new empty graph

ModuleGraph.add_module(module_name: str) → BaseNode

Add a module to the graph and process imports.

Will not raise an exception for non-existing modules.

Parameters

module_name – Name of the module to import.

ModuleGraph.add_script(script_path: PathLike) → Script

Add a script to the module graph and process imports

Parameters

script_path – Filesystem path for the script to be added

Returns

The script node for the just added script

Raises

  • ValueError – If the script is already part of the graph

  • OSError – If the script cannot be opened.

  • SyntaxError – If the script is invalid

ModuleGraph.add_distribution(distribution: Union[PyPIDistribution, str])

Add a package distribution to the graph, with references to all importable names in that distribution

Parameters

distribution – A distribution or distribution name

Returns

The node added to the graph

Affecting building the graph

ModuleGraph.add_excludes(excluded_names: Iterator[str]) → None

Exclude the names in “excludeded_names” from the graph

Excluded names can end up as ExcludedNode nodes in the graph, but the dependencies of the actual module are not gathered.

  • excluded_names: An interator yielding names to exclude.

ModuleGraph.add_implies(implies: Dict[str, Union[None, Alias, Virtual, Sequence[str]]]) → None

Add implied actions for the graph.

An implied action can be used for three purposes: * A list of dependencies.

Commonly used to add module dependencies for modules that modulegraph2 cannot scan, such as extensions and modules using __import__().

  • An Alias for another node

    Used to mark an importable name as an alias for some other module. An example of this is os.path, which is an alias to a platform specific path module (such as posixpath.

  • A Virtual module

    Used to mark an importable name as a virtual module that is added to sys.modules by some other module.

Parameters

implies – A mapping from module names to implied actions

Callbacks

ModuleGraph.add_missing_hook(hook: Callable[[ModuleGraph, Optional[BaseNode], str], Optional[BaseNode]]) → None

Add a hook function that’s used to try to resolve a missing module.

The hook functions are called in reverse order of addition, and the result of the first hook that doesn’t return None is used in the graph.

Parameters

hook – The hook function. Run hook(self, importing_module, module_name) before creating a MissingModule node for module_name.

ModuleGraph.add_post_processing_hook(hook: Callable[[ModuleGraph, BaseNode], None]) → None

Add a hook function to be ran whenever a node is fully processed.

It is possible to add multip hooks by calling this method multiple times.

Parameters

  • hook – The post processing hook. Run hook(self, node)

  • at (when node is fully processed. The hook will be run) –

  • node (most once per) –

  • cause (even if recipes or hooks) –

  • node. (re-processing of a) –

Reporting

ModuleGraph.distributions(reachable: bool = True) → Iterator[PyPIDistribution]

Yield all distributions used in the graph.

If reachable is True (default) this reports only on distributions used by nodes reachable from one of the graph roots, otherwise this reports on distributions used by any root.

This will not report PyPIDistributions that are nodes in the graph, unless they are also the distribution attribute of a node.

Parameters

reacable – IF true only report on nodes that are reachable from a graph root, otherwise report on all nodes.

ModuleGraph.report(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) → None

Print information about nodes reachable from the graph roots to the given file.

Parameters

file – Stream to write to

Graph Nodes

The graph can contain nodes of a number of classes, as described below. All these classes are dataclasses.

class modulegraph2.BaseNode(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict)

Bases: object

Base class for all module nodes in the dependency graph.

name

Name of the module.

Type

str

loader

Importlib loader for the module when available.

Type

Optional[importlib.abc.Loader]

distribution

Package distribution that contains this module. None for the stdlib and uninstalled modules.

Type

Optional[modulegraph2._distributions.PyPIDistribution]

filename

Filesystem path to the module when available

Type

Optional[pathlib.Path]

extension_attributes

A dictionary for use by users for the modulegraph2 library, not used by modulegraph2 itself.

Type

dict

property identifier: str

Graph identifier for use with objectgraph.ObjectGraph.

class modulegraph2.Module(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: BaseNode

Information about a Module

globals_written

Set of global names written to by the module

Type

Set[str]

globals_read

Set of global names read by the module

Type

Set[str]

code

Code for the module

Type

Optional[code]

property uses_dunder_file: bool

True if the module appears to use the __file__ attribute.

property uses_dunder_import: bool

True if the module appears to use the __import__() function.

class modulegraph2.BuiltinModule(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: Module

Node representing a built-in extension module.

class modulegraph2.BytecodeModule(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: Module

Node representing a python module for which only byte code is available.

class modulegraph2.ExtensionModule(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: Module

Node representing a native extension module.

class modulegraph2.FrozenModule(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: Module

Note representing a python module that is frozen into an executable. This has source code available, but not on the filesystem.

class modulegraph2.SourceModule(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, globals_written: Set[str], globals_read: Set[str], code: Optional[code])

Bases: Module

Node representing a python module for which the source code is available.

class modulegraph2.NamespacePackage(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, search_path: List[Path], has_data_files: bool)

Bases: BaseNode

Node representing an implicit namespace package (PEP 420).

search_path

The search path for modules in this package.

Type

List[pathlib.Path]

property globals_read

Always an empty set

property globals_written

Always an empty set

class modulegraph2.Package(name: str, loader: Optional[Loader], distribution: Optional[PyPIDistribution], filename: Optional[Path], extension_attributes: dict, init_module: BaseNode, search_path: List[Path], has_data_files: bool, namespace_type: Optional[str])

Bases: BaseNode

Node representing a namespace package with an __init__ module.

init_module

Node representing the __init__ module for this package.

Type

modulegraph2._nodes.BaseNode

search_path

The search path for modules in this package.

Type

List[pathlib.Path]

has_data_files

True if this package contains data files (other than empty directories and python files).

Type

bool

namespace_type

None, “pkgutil” or “pkg_resources” for regular packages, namespace packages using pkgutil and namespace packages using pkg_resources.

Type

Optional[str]

property globals_read

The globals read from by the module __init__

property globals_written

The globals written to by the module __init__

class modulegraph2.Script(filename: PathLike, code: Optional[code])

Bases: BaseNode

Node representing a Python script.

The name of the node is the string representation of the full filename of the script.

globals_written

Global varialbles written to

Type

Set[str]

globals_read

Global variables read from

Type

Set[str]

code

Code object for the script

Type

Optional[code]

Special nodes

class modulegraph2.VirtualNode(module_name, providing_module)

Bases: BaseNode

Node representing a virtual module, that is added to sys.modules by some other module.

Attributes
providing_module

The module that creates this module in sys.modules.

class modulegraph2.AliasNode(module_name, actual_module)

Bases: BaseNode

Node representing a module alias, that is an imported name that refers to some other module.

Attributes
actual_module

The module that this name aliases to.

class modulegraph2.ExcludedModule(module_name)

Bases: BaseNode

Node representing a module that is explicitly excluded by the user.

class modulegraph2.MissingModule(module_name)

Bases: BaseNode

Node representing a name that is imported, but could not be located.

class modulegraph2.InvalidRelativeImport(module_name)

Bases: BaseNode

Node representing a relative import that refers to a location outside of a toplevel package.

The name is a name starting with one or more dots.

Edge attributes

class modulegraph2.DependencyInfo(is_optional: bool, is_global: bool, in_fromlist: bool, imported_as: Optional[str])

A frozen dataclass representing information about the dependency edge between two graph nodes.

is_optional

True if the import appears to be optional

Type

bool

is_global

True if the import affects global names in the module

Type

bool

in_fromlist

True if the name is imported in the name list of an from ... import ... statement

Type

bool

imported_as

Rename for this module (import ... as impoted_as), None when there is no as clause.

Type

Optional[str]

Distributions

class modulegraph2.PyPIDistribution(identifier: str, name: str, version: str, files: Set[str], import_names: Set[str])

Information about a package distribution

identifier

Unique identifier fot the distribution for use with modulegraph2.ObjectGraph

Type

str

name

Name of the distribution (as it is found on PyPI)

Type

str

files

Files that are part of this distribution

Type

Set[str]

import_names

The importable names in this distribution (modules and packages)

Type

Set[str]

Note

The information about distributions is fairly minimal at this point, and will be enhanced as needed.

contains_file(filename: Union[str, PathLike])

Check if a file is part of this distribution.

Parameters

filename – The filename to look for

Returns

True if filename is part of this distribution, otherwise False.

modulegraph2.distribution_named(name: str, path: Optional[Iterable[str]] = None) → Optional[PyPIDistribution]

Find a named distribution on the search path.

Parameters

  • name – Distribution name to look for.

  • path – Module search path (defaults to sys.path)

Returns

The distribution, or None

modulegraph2.all_distributions(path: Optional[Iterable[str]] = None) → Iterator[PyPIDistribution]

Yield all distributions found on the search path.

Parameters

path – Module search path (defaults to sys.path).

Utilities

modulegraph2.saved_sys_path()

Contextmanager that will restore the value of sys.path when leaving the with block.

modulegraph2.stdlib_module_names() → List[str]

Return a list of modules in the standard library

class modulegraph2.Alias
class modulegraph2.Virtual