Question

I’m writing a small utility which is supposed to launch several commands in parallel using system() and wait for their results for logging purposes. However, even though I’m calling system() on different threads, by looking at my Activity Monitor I only see one instance of each command at a time. It looks like system is internally synchronized on a mutex, and only one execution is allowed at each time, however this looks like a huge limitation, can someone confirm this behavior? Do you have any ideas on how to solve it?

Update by looking at the threads execution flow, it looks like they’re effectively synchronizing on a mutex. Is there an alternative system() which doesn’t do that?

I should mention I’m using C++11 (w/ clang and libc++) on Mac OS 10.7.5.

Update code is:

void Batch::run()
{
    done.clear();
    generator->resetGeneration();

    while(generator->hasMoreParameters())
    {
        // Lock for accessing active
        unique_lock<mutex> lock(q_mutex, adopt_lock);

        // If we've less experiments than threads
        if (active.size() < threads)
        {
            Configuration conf = generator->generateParameters();
            Experiment e(executable, conf);

            thread t(&Experiment::run, e, reference_wrapper<Batch>(*this));
            thread::id id = t.get_id();
            active.insert(id);
            t.detach();
        }

        // Condition variable
        q_control.wait(lock, [this] { return active.size() < threads; } );

    }
}

void Batch::experimentFinished(std::thread::id pos)
{
    unique_lock<mutex> lock(q_mutex, adopt_lock);
    active.erase(pos);
    lock.unlock();
    q_control.notify_all();
}

void Experiment::run(Batch& caller)
{    
    // Generate run command
    stringstream run_command;
    run_command << executable + " ";
    ParameterExpression::printCommandLine(run_command, config);

    if (system(run_command.str().c_str()))
        stats["success"] = "true";
    else
        stats["success"] = "false";

    caller.experimentFinished(this_thread::get_id());
}

Just to be clear: the spawning and handling of threads works fine and does what it needs to do, but it looks like you can have just one instance of system() running at a time.

Thanks

Answer 1

To whoever comes later, popen did the trick, as it doesn’t internally keep a mutex. The code to make it work is

FILE* proc;
char buff[1024];

// Keep track of the success or insuccess of execution
if (!(proc = popen(run_command.str().c_str(), "r")))
    stats["success"] = "false";
else
    stats["success"] = "true";

// Exhaust output
while(fgets(buff, sizeof(buff), proc) != nullptr);

pclose(proc);