Dynamic, Lazy Dependency Injection in Python

Dependency Injection (DI) solves many problems by improving testability, decoupling, maintainability and readability. However, managing dependencies can sometimes introduce inefficiencies. When do we initialize them? How do we initialize? Can they be reused effectively?

In this article, we'll explore how you can take DI a step further using FastInject: a Python package that simplifies dependency management with just a few decorators. FastInject automatically handles dependency instantiation and injection, taking much of the manual work off your plate while offering:

• improved performance: Only create dependencies when actually needed
• simpler initialization: Dependencies are resolved dynamically
• avoid circular imports: Dependency resolution is deferred until runtime
• improved flexibility: Dependencies can be influenced by runtime information or configuration

Let's code!

Contents

1. DI refresher: A comparison of code that uses DI versus code that doesn't
2. FastInject: Automatically managing injectable dependencies:
   Learn how to automatically declare and inject dependencies into your functions
3. Conclusion: Why FastInject simplifies your development process

Turn Your Python Function into a Decorator with One Line of Code

A new way to write shorter, clearer, and more readable Python decorators that also act as a context manager

MikeHuls.comMike Huls

Refresher: DI vs No-DI

Dependency injection is a design pattern that allows you to decouple components in your application by injecting dependencies rather than hardcoding them. Instead of your class instantiating its dependencies, they are provided externally.

Let's compare two pieces of code: without DI and with DI.

Without DI: A Tightly Coupled Example

Here's a simple class, DatabaseHelper, that is tightly coupled with PostgresConnection to interact with a database. It's tightly coupled because DatabaseHelper instantiates PostgresConnection in its constructor:

class PostgresConnection:
  def __init__(self, constring:str):
    self.constring = constring
  
  def execute(self, stmt:str) -> List[Dict]:
    print(f"simulating query executing '{stmt}' on {self.constring}..")
    return [{'id': 1, 'data': 'xxx'}]

class DatabaseHelper:
  dbcon:PostgresConnection
  
  def __init__(self, constring:str):
    self.dbcon = PostgresConnection(constring=constring)
  
  def get_users(self):
    return self.dbcon.execute("select * from users")

Usage:

dbhelper = DatabaseHelper(constring="user:passs@mydatabase")
users:List[Dict] = dbhelper.get_users()
print(users)

Problems with this approach:

• Tightly coupled classes: DatabaseHelper must know about the connection string and how to create a PostgresConnection.
• Inflexible: It's impossible to swap PostgresConnection for, say, a SqlServerConnection since it's hardcoded in the DatabaseHelper class.
• Testing difficulties: Due to the tight coupling it's quite difficult to test DatabaseHelper. You'll need to use patches and mocks in your tests, making testing quite cumbersome.
• Reduced readability: all these interlinked dependencies make the code harder to maintain.

Python: __init__ is NOT a constructor: a deep dive in Python object creation

Tinkering with Python’s constructor to create fast, memory-efficient classes

MikeHuls.comMike Huls

With DI: A Loosely Coupled Example

We'll refactor using DI. First, we'll create a generic Connection interface using an Abstract Base Class (ABC).

import abc
from typing import Dict, List


class Connection(abc.ABC):
    @abc.abstractmethod
    def execute(self, stmt: str) -> List[Dict]:
        pass


class PostgresConnection(Connection):
    def __init__(self, constring:str):
        self.constring = constring
   
    def execute(self, stmt:str) -> List[Dict]:
        print(f"simulating query executing '{stmt}' on {self.constring}..")
        return [{'id': 1, 'data': 'xxx'}]

Now, we rewrite DatabaseHelper to accept any Connection instance:

class DatabaseHelper:
    dbcon:Connection
    def __init__(self, dbcon:Connection):
        self.dbcon = dbcon
    def get_users(self):
        return self.dbcon.execute("select * from users")

Usage (notice that we inject PostgresConnection into DatabaseHelper):

  dbcon_postgres = PostgresConnection(constring="user:passs@mydatabase")
  dbhelper = DatabaseHelper(dbcon=dbcon_postgres)
  users:List[Dict] = dbhelper.get_users()
  print(users)

Benefits:

• Loosely coupled classes: DatabaseHelper accepts PostgresConnection and any other class that implements the Connection ABC
• Testability: You can instantiate DatabaseHelper with a mock connection for unit tests
• Runtime Flexibility: We can swap connection types at runtime:

if os.getenv("DB_TYPE") == "sqlserver":
    dbcon = SqlServerConnection(constring="user:pass@sqlserverhost")
else:
    dbcon = PostgresConnection(constring="user:pass@postgreshost")
dbhelper = DatabaseHelper(dbcon=dbcon)

Cython for absolute beginners: 30x faster code in two simple steps

Easy Python code compilation for blazingly fast applications

MikeHuls.comMike Huls

FastInject: managing injectable dependencies

While DI solves many problems, it introduces challenges:

• When and how should dependencies be initialized?
• How do we manage circular imports or dependency graphs?

Enter FastInject: a package that handles these concerns. Just declare dependencies as injectable, and they’ll be instantiated and injected automatically.

You don't need to manually instantiate dependencies yourself and import them throughout your app. Dependencies are resolved at runtime instead of at import time, reducing the likelihood of circular dependencies.

Minimal Example: Lazy Injection

Here's a simple service. With the injectable decorator we'll mark it as injectable:

import time, datetime
from fastinject import injectable

@injectable()           # <-- Declares TimeStamp to be injectable
class TimeStamp:
    ts: float

    def __init__(self) -> None:
        self.ts = time.time()

    @property
    def datetime_str(self) -> str:
        return datetime.datetime.fromtimestamp(self.ts).strftime("%Y-%m-%d %H:%M:%S")

We want to inject TimeStamp into a function; just add the inject decorator:

from fastinject import inject
from services import TimeStamp

@inject()               # <-- Injects required services into function
def function_with_injection(ts: TimeStamp):
    print(f"In the injected function, the current time is {ts.datetime_str}.")

if __name__ == "__main__":
    function_with_injection()

These two decorators are enough to inject instances of TimeStamp into the function! Key points:

• the function is called without any arguments: FastInject injected them automatically.
• No more import errors or circular dependencies since we don't need to import instances of TimeStamp. When FastInject recognizes the TimeStamp type hint in the function, it will create and provide an instance.

Python args, kwargs, and All Other Ways to Pass Arguments to Your Function

Expertly design your function parameters in 6 examples

MikeHuls.comMike Huls

Singleton Dependencies

FastInject can manage singleton services, ensuring only one instance is created accross your application's lifetime. Singleton services are usefull for ensuring that shared resources, such as database connections or API clients, are not recreated unnecessarily.

By declaring the scope of the injectable to be a singleton, no more than one instance will be created by FastInject in the lifetime of your app:

from typing import Dict, List
from src.fastinject import inject, injectable, singleton

@injectable(scope=singleton)           # <-- set scope to singleton
class ApiClient:
    def __init__(self) -> None:
        pass

    def get_users(self) -> List[Dict]:
        """retrieves users from the database"""
        return [{"id": 1, "name": "mike"}]

Usage:

@inject()
def function_1(api_client: ApiClient):
    print(f"fn1: Get users with api-client {id(api_client)}")
    return api_client.get_users()

@inject()
def function_2(api_client: ApiClient):
    print(f"fn2: Get users with api-client {id(api_client)}")
    return api_client.get_users()

Both functions will receive the same instance of ApiClient.

Nested Dependencies

With FastInject you can specify how dependencies are created. This is especially convenient when some of our dependencies rely on each other, like in our previous example where DatabaseHelper requires an instance of DatabaseConnection.

A ServiceConfig details how certain services need to be instantiated. In this case DatabaseConnection depends on AppConfiguration; FastInject automatically resolves this dependency graph.

@injectable()
class MyServiceConfig(ServiceConfig):
    @provider
    def provide_app_config(self) -> AppConfiguration:
        return AppConfiguration("my_db_config_string")

    @singleton
    @provider
    def provide_database_connection(self) -> DatabaseConnection:
        return DatabaseConnection(connection_string=self.provide_app_config().connection_string)

Key points:

• MyServiceConfig is marked as injectable with the injectable decorator
• MyServiceConfig must inherit from ServiceConfig
• We can decorate Services within the config; e.g. by declaring that the database-connection is a singleton

Duplicate types, other options and demo's

The list below contains some features that FastInject offers. You can check out the full list with demo's here.

• Using Singletons
• Registering multiple services of the same type
• Using the registry to add services at runtime
• Using the registry to add service config at runtime
• Using multiple registries
• Validate all registered services. This is especially convenient for bootstrapping your application

Applying Python multiprocessing in 2 lines of code

When and how to use multiple cores to execute many times faster

MikeHuls.comMike Huls

Conclusion

FastInject takes dependency injection to the next level making it very easy to automatically instantiate and inject instances of your services. By dynamically resolving dependencies we avoid avoiding circular imports, simplifying the development process. Additionally, lazy initialization ensures that services are only created when needed, enhancing performance and resource efficiency. All in all

By using FastInject, your code becomes easier to maintain, more flexible to extend, and simpler to test. Try it out in your next project or check it out on PyPI or GitHub!

I hope this article was as clear as I intended it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics.

Happy coding!

— Mike

P.s: like what I'm doing? Follow me!