Question

I’m a C++ developer who has primarily programmed on Solaris and Linux until recently, when I was forced to create an application targeted to Windows.

I’ve been using a communication design based on C++ I/O stream backed by TCP socket. The design is based on a single thread reading continuously from the stream (most of the time blocked in the socket read waiting for data) while other threads send through the same stream (synchronized by mutex).

When moving to windows, I elected to use the boost::asio::ip::tcp::iostream to implement the socket stream. I was dismayed to find that the above multithreaded design resulted in deadlock on Windows. It appears that the operator<<(std::basic_ostream<...>,std::basic_string<...>) declares a ‘Sentry’ that locks the entire stream for both input and output operations. Since my read thread is always waiting on the stream, send operations from other threads deadlock when this Sentry is created.

Here is the relevant part of the call stack during operator<< and Sentry construction:

    ...
    ntdll.dll!7c901046()    
    CAF.exe!_Mtxlock(_RTL_CRITICAL_SECTION * _Mtx=0x00397ad0)  Line 45  C
    CAF.exe!std::_Mutex::_Lock()  Line 24 + 0xb bytes   C++
    CAF.exe!std::basic_streambuf<char,std::char_traits<char> >::_Lock()  Line 174   C++
    CAF.exe!std::basic_ostream<char,std::char_traits<char> >::_Sentry_base::_Sentry_base(std::basic_ostream<char,std::char_traits<char> > & _Ostr={...})  Line 78   C++
    CAF.exe!std::basic_ostream<char,std::char_traits<char> >::sentry::sentry(std::basic_ostream<char,std::char_traits<char> > & _Ostr={...})  Line 95 + 0x4e bytes  C++
>   CAF.exe!std::operator<<<char,std::char_traits<char>,std::allocator<char> >(std::basic_ostream<char,std::char_traits<char> > & _Ostr={...}, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & _Str="###")  Line 549 + 0xc bytes   C++
    ...

I would be fine if the istream and ostream components were locked separately, but that is not the case.

Is there an alternate implementation of the stream operators that I can use? Can I direct it not to lock? Should I implement my own (not sure how to do this)?

Any suggestions would be appreciated.

(Platform is Windows 32- and 64-bit. Behavior observed with Visual Studio 2003 Pro and 2008 Express)

Answer 1

This question has languished for long enough. I’m going to report what I ended up doing even though there’s a chance I’ll be derided.

I had already determined that the problem was that two threads were coming to a deadlock while trying to access an iostream object in separate read and write operations. I could see that the Visual Studio implementation of string stream insertion and extraction operators both declared a Sentry, which locked the stream buffer associated with the stream being operated on.

I knew that, for the stream in question for this deadlock, the stream buffer implementation was boost::asio::basic_socket_streambuf. I inspected the implementation to see that read and write operations (underflow and overflow) actually operate on different buffers (get vs. put).

With the above verified, I chose to simply circumvent the locking for this application. To do that, I used project-specific pre-processor definitions to exclude the locking code in the basic_istream implementation of the locking sentry:

    class _Sentry_base
        {   // stores thread lock and reference to input stream
    public:
        __CLR_OR_THIS_CALL _Sentry_base(_Myt& _Istr)
            : _Myistr(_Istr)
            {   // lock the stream buffer, if there
#ifndef MY_PROJECT
            if (_Myistr.rdbuf() != 0)
                _Myistr.rdbuf()->_Lock();
#endif
            }

        __CLR_OR_THIS_CALL ~_Sentry_base()
            {   // destroy after unlocking
#ifndef MY_PROJECT
            if (_Myistr.rdbuf() != 0)
                _Myistr.rdbuf()->_Unlock();
#endif
            }

Upside:

• It works
• Only my project (with the appropriate defines) is affected

Downside:

• Feels a little hacky
• Each platform where this is built will need this modification

I plan to mitigate the latter point by loudly documenting this in the code and project documentation.

I realize that there may be a more elegant solution to this, but in the interest of expediency I chose a direct solution after due diligence to understand the impacts.