Python Modules and Packages

2023-12-09

Modules

As your program gets longer, you can split it into several files for easier maintenance, you may also want to share a handy function across several programs.

To support this, Python has a way to put definitions in a file, such a file is called a module, the file name is the module name with the suffix .py appended.

fibo.py:

print("In fibo module.")

def fib(n):
    """write Fibonacci series up to n."""
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

print(f"module name: {__name__}")

A module can contain executable statements as well as function definitions. They are executed only the first time the module name is encountered in an import statement.

For efficiency reasons, each module is only imported once per interpreter session.

Within a module, the module’s name (as a string) is available as the value of the global variable __name__.

# In a Python interpreter

>>> import fibo
In fibo module.
module name: fibo

# In Shell

> python fibo.py
In fibo module.
module name: __main__

python -c 'import fibo' outputs the module name fibo.

python -m fibo outputs the module name __main__.

You may see some Python files that have the following code:

if __name__ == "__main__":
    # some code

__name__ is used to distinguish whether the file is running directly or imported as a module.

Each module has its own private namespace.

# fibo is a namespace

>>> fibo.fib
<function fib at 0x7f89fc505400>
>>> fib=fibo.fib
>>> fib(100)
0 1 1 2 3 5 8 13 21 34 55 89

Different ways to use import:

# Import the module, fibo is just a namespace
# To call the fib function within the fibo module: fibo.fib(100)
import fibo
# fib is an alias of fibo
import fibo as fib

# Only import the fib function
from fibo import fib
# fibonacci is an alias of fib
from fibo import fib as fibonacci

# Import all names except those beginning with an underscore (_)
from fibo import *

The module search path:

built-in modules

# check them

import sys
print(sys.builtin_module_names)

A list of directories given by the variable sys.path
```
# check it

import sys
print(sys.path)
```

sys.path is initialized from these locations:

The directory containing the .py file or current working directory
fibo.py is in /opt/py-modules.
When running the module directly:
```
python fibo.py
```
The directory is /opt/py-modules.
If /opt/py-modules is a symlink to /opt/third-party/py-modules, the directory is /opt/third-party/py-modules, not /opt/py-modules, Python follows the symlink.
When using a Python interpreter:
```
>>> import sys
>>> print(sys.path)
```
You will see the directory (the first one) is current working directory.
The directory is placed at the beginning of the search path, ahead of the standard library path. This means that modules in the directory will be loaded instead of modules of the same name in the library directory.
Directories listed in the shell environment variable PYTHONPATH
The default value of PYTHONPATH is an empty string, but Python always append the directories containing standard Python modules (e.g, /usr/local/lib/python311.zip, /usr/local/lib/python3.11) as well as any extension modules that these modules depend on (e.g, /usr/local/lib/python3.11/lib-dynload).
The installation-dependent default directories
It includes the user-specific site-packages directory and all global site-packages directories.
```
>>> import site
>>> print(site.getusersitepackages())
>>> print(site.getsitepackages())
```

In a directory, you may see a __pycache__ directory, Python caches the compiled version of each module in it.

The only advantage of .pyc files is that they can be loaded faster, not run faster.

Python does not check the cache in two circumstances:

Load a module from the command line
No corresponding .py source modules

Packages

run.py
sound/                          Top-level package
      __init__.py               Initialize the sound package
      formats/                  Subpackage for file format conversions
              __init__.py
              wavread.py
              wavwrite.py
              ...
      effects/                  Subpackage for sound effects
              __init__.py
              echo.py
              surround.py
              ...
      filters/                  Subpackage for filters
              __init__.py
              equalizer.py
              vocoder.py
              ...

# Load the submodule sound.effects.echo

import sound.effects.echo

# OR

from sound.effects import echo

# Import the desired function or variable directly
from sound.effects.echo import echofilter

Packages are a way of structuring Python’s module namespace by using “dotted module names”.

The __init__.py files are required to make Python treat directories containing the file as packages.

In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable.

# sound/effects/__init__.py

__all__ = ["echo"]

When using from sound.effects import *, only import echo submodule from the sound.effects package. __all__ likes a filter.

If you define a function or variable named echo in sound/effects/__init__.py, it will shadow the echo.py module.

If no __all__, Python only ensures that the package sound.effects has been imported, not submodules.

Using import * is a bad practice in production code.

# absolute imports
from sound.effects import echo

# relative imports
# leading dots indicate the current and parent packages
from . import echo
from .. import formats
from ..filters import equalizer

Relative imports are based on the name of the current module.

python run.py runs the program, run.py is the main module, its __name__ is __main__.

The directory containing the main module run.py is not a package, sound is the top-level package, we can’t use relative imports in run.py and other modules at the same directory.

When using from .sound.effects import echo in run.py, the error message:

ImportError: attempted relative import with no known parent package