Thread safe lazy property caching for python.
F.Y.I: This article assumes you have idea about property
decorator in python.
class Math:
@property
def tao(self):
pass# way you use it is
math = Math()
# access the property
tao = math.tao
property
decorator are cool in a sense that it turns your function call into intuitive property access. For example:
class App: @property
def name(self):
# some computation
_name = getattr(sys.modules['__main__'], '__file__', None)
if _name is None:
return '__main__'
return custom_name
You can simply call this function as a property access
app = App()# this will call the method and assign the result to `app_id`
app_id = app.name
However property
decorator comes with the cost which is, it re-runs the computation every time we access the property of object and also is not thread safe.
I think we can improve upon what standard python provides by adding caching and thread safety to property access.
The idea is to convert method into lazy property, running the computation only once and storing the result of computation such that access to property returns the result from cache instead of re-running the computation.
Time for code meal:
from threading import RLock# senitel
_missing = object()class cached_property(object): def __init__(self, func):
self.__name__ = func.__name__
self.__module__ = func.__module__
self.__doc__ = func.__doc__
self.func = func
self.lock = RLock() def __get__(self, obj, type=None):
if obj is None:
return self
with self.lock:
value = obj.__dict__.get(self.__name__, _missing)
if value is _missing:
value = self.func(obj)
obj.__dict__[self.__name__] = value
return value
The way you use it is as follows:
class App: @cached_property
def logger(self):
return create_logger(self) @cached_property
def name(self):
_name = getattr(sys.modules['__main__'], '__file__', None)
if _name is None:
return '__main__'
return custom_name
And finally:
app = App()
# runs app.name for the first time and caches
app.name
# uses the cached value
app.name + '_PRODUCTION'
My final thoughts are:
- Avoid eager object instantiation in
constructor/__init__
and offload object creation only when it is needed. For example: creating logging instance only when program starts to log, connecting to database only when program needs to persist data. - Use
cached property
if you don’t require to re-run the computation every time you need to access the property - Use
cached_property
if you need to lazily load/create objects during runtime only when it is accessed so that your initial startup time can be minimized.
Thank you for your time. Cheers.