Question

I have two questions about creating thread safe types in python, and one related question about multiple inheritance.

1) Are there any problematic implications with using the following subclasses in my threaded application as a sort of “lazy” thread-safe type? I realize that whomever sets values which may be altered by other threads bears the responsibility to ensure those values are thread safe as well.

2) Another question I have is if there exists more prudent alternatives to these types within python in a typical installation.

Example:

from threading import Lock
from __future__ import with_statement

class safedict(dict):
    def __init__(self,*args,**kwargs):
        self.mylock=Lock(); 
        super(safedict, self).__init__(*args, **kwargs)
    def __setitem__(self,*args,**kwargs):
        with self.mylock:
            print " DEBUG: Overloaded __setitem__ has the lock now."
            super(safedict,self).__setitem__(*args,**kwargs)


class safeobject(object):
    mylock = Lock(); # a temporary useless lock, until we have a proper instance.
    def __init__(self,*args,**kwargs):
        self.mylock=Lock(); 
        super(safeobject, self).__init__(*args, **kwargs)
    def __setattr__(self,*args,**kwargs):
        with self.mylock:
            print " DEBUG: Overloaded __setattr__ has the lock now."
            super(safeobject,self).__setattr__(*args,**kwargs)

3) If both of the types defined above could be considered reasonably safe, what negative implications would be faced by using multiple inheritance to create a type that supported a mixture of both of these modifications, and does my example inherit those classes in the optimal order?

Example:

class safedict2(safeobject,dict):
    def __setitem__(self,*args,**kwargs):
        with self.mylock:
            print " DEBUG: Overloaded __setitem__ has the lock now."
            super(safedict2,self).__setitem__(*args,**kwargs)

Edit:
Just another example of another type inheriting both of the former types, and testing using ipython.

In [304]: class safedict3(safeobject,safedict):
   .....:         pass
   .....: 

In [305]: d3 = safedict3()
 DEBUG: Overloaded __setattr__ has the lock now.
 DEBUG: Overloaded __setattr__ has the lock now.

In [306]: d3.a=1
 DEBUG: Overloaded __setattr__ has the lock now.

In [307]: d3['b']=2
 DEBUG: Overloaded __setitem__ has the lock now.

In [308]: d3
Out[308]: {'b': 2}

Answer 1

As to your first and second questions, the dict, list, etc. types are already thread-safe. You do not have to add thread safety to them. However you may find this useful. It’s a decorator that basically implements the synchronized keyword from Java, using function scope to define a critical section. Using a similar approach it is possible to author a threading.Condition oriented decorator also.

import threading

def tryfinally(finallyf):
  u"returns a decorator that adds try/finally behavior with given no-argument call in the finally"
  def decorator(callable):
    def execute(*args, **kwargs):
      try: result = callable(*args, **kwargs)
      finally: finallyf()
      return result
    return execute
  return decorator

def usinglock(lock):
  u"returns a decorator whose argument will acquire the given lock while executing"
  def decorator(function):
    body = tryfinally(lock.release)(function)
    def execute(*args, **kwargs):
      lock.acquire()
      return body(*args, **kwargs)
    return execute
  return decorator

def synchronized(function):
  u"decorator; only one thread can enter the decorated function at a time; recursion is OK"
  return usinglock(threading.RLock())(function)

Use it like this (and beware deadlocks if you overuse it):

@synchronized
def foo(*args):
  print 'Only one thread can enter this function at a time'

On the third question, the Python tutorial states that the search order for inherited attributes is depth-first, left-first. So if you inherit (myclass, dict) then the __setitem__ method from myclass should be used. (In older versions of Python, this same section in the tutorial implied that this choice was arbitrary, but nowadays it appears to be quite deliberate.)

I’m guessing from the Freudian slip of a semicolon in the posted source that you are new to Python but experienced in either Java or C#. If so you will need to keep in mind that attribute (method) resolution occurs at run time in Python, and that the classes as well as the instance are first-class objects that can be inspected/explored at run time.

First the instance attribute dictionary is searched, then the class attributes, and then the parent class search algorithm starts. This is done with (conceptually) the equivalent of repeated hasattr(class_or_instance, attribute) calls.

The below confirms that for “new-style” classes (classes that inherit from object, which in the 2.x language specification is optional), this resolution occurs each time the attribute is looked up. It is not done when the class (or subclass) is created, or when instances are created. (This was done in release 2.7.2.)

>>> class Foo(object):
...   def baz(self):
...     print 'Original Foo.baz'
...
>>> class Bar(Foo): pass
...
>>> def newprint(self):
...   print 'New Foo.baz'
...
>>> x = Foo()
>>> y = Bar()
>>> Foo.baz = newprint
>>> a = Foo()
>>> b = Bar()
>>> map(lambda k: k.baz(), (x, y, a, b))
New Foo.baz
New Foo.baz
New Foo.baz
New Foo.baz
[None, None, None, None]

Replacing the method of class Foo changes the behavior of subclasses already defined and of instances already created.