Dynamic, Lazy Dependency Injection in Python
Dependency Injection (DI) solves many problems by improving testability, decoupling, maintainability and readability. However, managing dependencies can sometimes introduce new problems. When do we initialize them? How do we initialize? Can they be reused effectively?
In order to take DI to the next level I've created FastInject: a Python package that simplifies dependency management with just a few decorators. FastInject automatically handles dependency instantiation and injection so that you can focus on your project. Features:
- improved performance: Only create dependencies when they are actually needed
- simpler initialization: Dependencies are resolved dynamically
- avoid circular imports: Dependency resolution is deferred until runtime
- improved flexibility: Dependencies can be influenced by runtime information or configuration
Let's code!
Contents
- DI refresher: A comparison of code that uses DI versus code that doesn't
-
Dependency management with FastInject: Learn how to automatically declare and inject dependencies into your functions
- Conclusion: Why FastInject simplifies your development process
Turn Your Python Function into a Decorator with One Line of Code
Refresher: DI vs No-DI
Dependency injection is a design pattern that allows you to decouple components in your application by injecting dependencies rather than hardcoding them. Instead of your class instantiating its dependencies, they are provided externally.
Let's compare two pieces of code: without DI and with DI.
Without DI: A Tightly Coupled Example
Here's a simple class, DatabaseHelper, that is tightly coupled with PostgresConnection to interact with a database. It's tightly coupled because DatabaseHelper instantiates PostgresConnection in its constructor:
class PostgresConnection:
def __init__(self, constring:str):
self.constring = constring
def execute(self, stmt:str) -> List[Dict]:
print(f"simulating query executing '{stmt}' on {self.constring}..")
return [{'id': 1, 'data': 'xxx'}]
class DatabaseHelper:
dbcon:PostgresConnection
def __init__(self, constring:str):
self.dbcon = PostgresConnection(constring=constring)
def get_users(self):
return self.dbcon.execute("select * from users")Usage:
dbhelper = DatabaseHelper(constring="user:passs@mydatabase")
users:List[Dict] = dbhelper.get_users()
print(users)Problems with this approach:
- Tightly coupled classes:
DatabaseHelpermust know about the connection string and how to create aPostgresConnection. - Inflexible: It's impossible to swap
PostgresConnectionfor, say, aSqlServerConnectionsince it's hardcoded in theDatabaseHelperclass. - Testing difficulties: Due to the tight coupling it's quite difficult to test
DatabaseHelper. You'll need to use patches and mocks in your tests, making testing quite cumbersome. - Reduced readability: all these interlinked dependencies make the code harder to maintain.
Python: init is NOT a constructor: a deep dive in Python object creation
With DI: A Loosely Coupled Example
We'll refactor using DI. First, we'll create a generic Connection interface using an Abstract Base Class (ABC).
import abc
from typing import Dict, List
class Connection(abc.ABC):
@abc.abstractmethod
def execute(self, stmt: str) -> List[Dict]:
pass
class PostgresConnection(Connection):
def __init__(self, constring:str):
self.constring = constring
def execute(self, stmt:str) -> List[Dict]:
print(f"simulating query executing '{stmt}' on {self.constring}..")
return [{'id': 1, 'data': 'xxx'}]Now, we rewrite DatabaseHelper to accept any Connection instance:
class DatabaseHelper:
dbcon:Connection
def __init__(self, dbcon:Connection):
self.dbcon = dbcon
def get_users(self):
return self.dbcon.execute("select * from users")Usage (notice that we inject PostgresConnection into DatabaseHelper):
dbcon_postgres = PostgresConnection(constring="user:passs@mydatabase")
dbhelper = DatabaseHelper(dbcon=dbcon_postgres)
users:List[Dict] = dbhelper.get_users()
print(users)Benefits:
- Loosely coupled classes:
DatabaseHelperacceptsPostgresConnectionand any other class that implements theConnectionABC - Testability: You can instantiate
DatabaseHelperwith a mock connection for unit tests - Runtime Flexibility: We can swap connection types at runtime:
if os.getenv("DB_TYPE") == "sqlserver":
dbcon = SqlServerConnection(constring="user:pass@sqlserverhost")
else:
dbcon = PostgresConnection(constring="user:pass@postgreshost")
dbhelper = DatabaseHelper(dbcon=dbcon)Cython for absolute beginners: 30x faster code in two simple steps
FastInject: managing injectable dependencies
While DI solves many problems, it introduces challenges:
- When and how should dependencies be initialized?
- How do we manage circular imports or dependency graphs?
We'll use FastInject to handles these concerns. Just declare dependencies as injectable, and they'll be instantiated and injected automatically.
You don't need to manually instantiate dependencies yourself and import them throughout your app. Dependencies are resolved at runtime instead of at import time, reducing the likelihood of circular dependencies.
Minimal Example: Lazy Injection
Here's a simple service. With the injectable decorator we'll mark it as injectable:
import time, datetime
from fastinject import injectable
@injectable() # <-- Declares TimeStamp to be injectable
class TimeStamp:
ts: float
def __init__(self) -> None:
self.ts = time.time()
@property
def datetime_str(self) -> str:
return datetime.datetime.fromtimestamp(self.ts).strftime("%Y-%m-%d %H:%M:%S")We want to inject TimeStamp into a function; just add the inject decorator:
from fastinject import inject
from services import TimeStamp
@inject() # <-- Injects required services into function
def function_with_injection(ts: TimeStamp):
print(f"In the injected function, the current time is {ts.datetime_str}.")
if __name__ == "__main__":
function_with_injection()These two decorators are enough to inject instances of TimeStamp into the function! Key points:
- the function is called without any arguments: they are injected automatically at runtime. This is what the
injectdecorator does. - No more import errors or circular dependencies since we don't need to import instances of
TimeStamp. Once theinjectdecorator recognizes theTimeStamptype hint in the function, it will create and provide an instance.
Args vs kwargs: which is the fastest way to call a function in Python?
Singleton Dependencies
Some services only require one instance across your application's lifetime. These so-called singletons are useful for ensuring that shared resources, such as database connections or API clients, are not recreated unnecessarily.
By declaring the scope of the injectable to be a singleton, no more than one instance will be created in you app's lifetime, saving the time and resources for recreating instances:
from typing import Dict, List
from src.fastinject import inject, injectable, singleton
@injectable(scope=singleton) # <-- set scope to singleton
class ApiClient:
def __init__(self) -> None:
pass
def get_users(self) -> List[Dict]:
"""retrieves users from the database"""
return [{"id": 1, "name": "mike"}]Usage:
@inject()
def function_1(api_client: ApiClient):
print(f"fn1: Get users with api-client {id(api_client)}")
return api_client.get_users()
@inject()
def function_2(api_client: ApiClient):
print(f"fn2: Get users with api-client {id(api_client)}")
return api_client.get_users()Both functions will receive the same instance of ApiClient.
Nested Dependencies
Sometimes it's required how to create dependencies, especially when some rely on each other, like in our previous example where DatabaseHelper requires an instance of DatabaseConnection.
In order to specify how certain services need to be instantiated, FastInject provides a ServiceConfig class. This allows you to create a class with methods that detail how certain services need to be instantiated.
In the example below we provide two services: AppConfiguration and DatabaseConnection (which depends on AppConfiguration):
@injectable()
class MyServiceConfig(ServiceConfig):
@provider
def provide_app_config(self) -> AppConfiguration:
return AppConfiguration("my_db_config_string")
@singleton
@provider
def provide_database_connection(self) -> DatabaseConnection:
return DatabaseConnection(
connection_string=self.provide_app_config().connection_string
)Key points:
MyServiceConfigmust inherit fromServiceConfig- Make all services specified in the service config injectable with the
@injectabledecorator. - all methods decorated with
providerprovide injecatble types - We can decorate Services within the config; e.g. by declaring that the database-connection is a singleton
- FastInject automatically resolves the dependency graph
Duplicate types, other options and demos
The list below contains some additional features that FastInject offers. You can check out the full list with demos here.
- Using Singletons
- Registering multiple services of the same type
- Using the registry to add services at runtime
- Using the registry to add service config at runtime
- Using multiple registries
- Validate all registered services. This is especially convenient for bootstrapping your application
Conclusion
To make full use of the benefits of Dependency Injection, it's important to manage your dependencies. Controlling when and how instances of your services are created and injected is essential for an uncomplicated project that is more flexible to extend, easier to maintain and simpler to test.
With FastInject I hope to have demonstrated one beginner-friendly, straight-forward way to to automatically instantiate and inject instances of your services. By dynamically resolving dependencies we avoid avoiding circular imports, simplifying the development process. Additionally, lazy initialization ensures that services are only created when needed, enhancing performance and resource efficiency.
I hope this article was as clear as I intended it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of Programming-related topics.
Happy coding!
— Mike
P.s: like what I'm doing? Follow me!
