Question

I’m new to multithreaded programming, and have a question. How do I get each thread to iterate through all the elements in a list being added to by a different thread?

Here’s a simple program to demonstrate. I have a single list of integers, and 10 threads, numbered 1 through 10, that are working on it. Each thread is to write all the values in the list to a StringBuilder. After a thread writes all the values in the list, it adds its number to the list, then terminates.

I’m trying to get each thread to keep checking the list for elements until the list is no longer being modified by any other thread, but am having trouble with the locking on it. If successful, the program would have output that might look like:

3: 1,
8: 1,3,2,4,5,7,
6: 1,3,2,4,5,7,8,
9: 1,3,2,4,5,7,8,6,
7: 1,3,2,4,5,
10: 1,3,2,4,5,7,8,6,9,
5: 1,3,2,4,
4: 1,3,2,
2: 1,3,
1: 

Which happens sometimes, but often two or more of the threads finish before the lock is set, so the iteration ends prematurely:

1: 
2: 1,5,4,8,7,3,10,
10: 1,5,4,8,7,3,
9: 1,5,4,8,7,3,10,2,
3: 1,5,4,8,7,
7: 1,5,4,8,
5: 1,    <<one of these threads didn't wait to stop iterating. 
4: 1,    <<
8: 1,5,4,
6: 1,5,4,8,7,3,10,2,

Does anyone have any ideas?

===========

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.locks.ReentrantLock;

public class ListChecker implements Runnable{

static List<Integer> list = new ArrayList<Integer>();
static ReentrantLock lock = new ReentrantLock();

int id;
StringBuilder result = new StringBuilder(); 

public ListChecker(int id){
    this.id = id;
}

@Override
public void run() {
    int i=0;

    do{
        while (i < list.size()){            
            result.append(list.get(i++)).append(',');
        }   
        if (!lock.isLocked()){
            break;
        }
    }while (true);                  
    addElement(id);
    System.out.println(id + ": " + result.toString());
}

public void addElement(int element){
    try{
        lock.lock();
        list.add(element);  
    }finally{
        lock.unlock();
    }
}


public static void main(String[] args){
    for(int i=1; i<=10; i++){
        ListChecker checker = new ListChecker(i);
        new Thread(checker).start();
    }   
}   
}

---

Edit: Thanks for the help so far. I should clarify, that I would like for each thread to be iterating through the list at the same time. In my case, there’s a lot of processing that needs to be performed on each element of the list by each thread (instead of appending to a StringBuffer, I’m doing a lot of comparisons of a candidate item to a list of finalists). So, having each thread be able to work on the same list at the same time is required for the multithreading to improve my performance. So, I don’t think locking around the entire iteration, or putting the whole iteration is a synchronized(list) block, will work.

---

Edit 2: I think I got it. The trick was not just to synchronize on the list when adding elements to it, but also when determining if there are any more elements. This prevents thread 2 from stopping its iteration before thread 1 finishes adding to the list. It looks a bit kludgy, but this keeps the code that I need to run in multiple threads outside the synchronize block, so my real case should get the performance increase I need.

Thanks to everyone who helped!

import java.util.ArrayList;
import java.util.List;

public class ListChecker2 implements Runnable{

    static List<Integer> list = new ArrayList<Integer>();

    int id;
    StringBuilder result = new StringBuilder(); 

    public ListChecker2(int id){
        this.id = id;
    }

    @Override
    public void run() {
        int i = 0;
        do{
            synchronized (list) {           
                if (i >= list.size()){
                    list.add(id);
                    System.out.println(id + ": " + result.toString());
                    return;
                }
            }
            result.append(list.get(i++)).append(',');
            System.out.println("running " + id);

        }while(true);
    }


    public static void main(String[] args){
        for(int i=1; i<=30; i++){
            ListChecker2 checker = new ListChecker2(i);
            new Thread(checker).start();
        }   
    }   
}

Answer 1

You can do this much more simply by synchronizing on the list.

public class ListChecker implements Runnable {
  // Number of threads.
  static final int THREADS = 10;
  // The list.
  static final List<Integer> list = new ArrayList<Integer>(THREADS);

  // My ID
  int id;

  public ListChecker(int id) {
    // Remember my ID.
    this.id = id;
  }

  @Override
  public void run() {
    // My string.
    StringBuilder result = new StringBuilder ();
    // Wait for exclusive access to the list.
    synchronized (list) {
      // Build my string.
      for ( Integer i : list ) {
        result.append(i).append(",");
      }
      // Add me to the list.
      list.add(id);
    }
    System.out.println(id + ": " + result.toString());
  }

  public static void main(String[] args) {
    for (int i = 1; i <= THREADS; i++) {
      ListChecker checker = new ListChecker(i);
      new Thread(checker).start();
    }
  }
}

This is my output. A bit boring I’m afraid but it proves it works.

1: 
2: 1,
3: 1,2,
4: 1,2,3,
5: 1,2,3,4,
6: 1,2,3,4,5,
7: 1,2,3,4,5,6,
8: 1,2,3,4,5,6,7,
9: 1,2,3,4,5,6,7,8,
10: 1,2,3,4,5,6,7,8,9,

Added

If you need to avoid locking the whole list (as your edit suggests) you could try a special list that locks itself whenever it delivers the last entry. You then need to specifically unlock it of course.

Sadly, this technique does not handle the empty list situation very well. Perhaps you can think of an apt solution to that.

public class ListChecker implements Runnable {
  // Number of threads.
  static final int THREADS = 20;
  // The list.
  static final EndLockedList<Integer> list = new EndLockedList<Integer>();
  // My ID
  int id;

  public ListChecker(int id) {
    // Remember my ID.
    this.id = id;
  }

  @Override
  public void run() {
    // My string.
    StringBuilder result = new StringBuilder();
    // Build my string.
    for (int i = 0; i < list.size(); i++) {
      result.append(i).append(",");
    }
    // Add me to the list.
    list.add(id);
    // Unlock the end lock.
    list.unlock();
    System.out.println(id + ": " + result.toString());
  }

  public static void main(String[] args) {
    for (int i = 0; i < THREADS; i++) {
      ListChecker checker = new ListChecker(i + 1);
      new Thread(checker).start();
    }
  }

  private static class EndLockedList<T> extends ArrayList<T> {
    // My lock.
    private final Lock lock = new ReentrantLock();
    // Were we locked?
    private volatile boolean locked = false;

    @Override
    public boolean add(T it) {
      lock.lock();
      try {
        return super.add(it);
      } finally {
        lock.unlock();
      }
    }

    // Special get that locks the list when the last entry is got.
    @Override
    public T get(int i) {
      // Get it.
      T it = super.get(i);
      // If we are at the end.
      if (i == size() - 1) {
        // Speculative lock.
        lock.lock();
        // Still at the end?
        if (i < size() - 1) {
          // Release the lock!
          lock.unlock();
        } else {
          // Remember we were locked.
          locked = true;
          // It is now locked for putting until specifically unlocked.
        }
      }
      // That's the one.
      return it;
    }

    // Unlock.
    void unlock() {
      if (locked) {
        locked = false;
        lock.unlock();
      }
    }
  }
}

Output (notice the mishandling of an empty list):

2: 
8: 
5: 
1: 
4: 
7: 
6: 
9: 
3: 
10: 0,1,2,3,4,5,6,7,8,
11: 0,1,2,3,4,5,6,7,8,9,
12: 0,1,2,3,4,5,6,7,8,9,10,
13: 0,1,2,3,4,5,6,7,8,9,10,11,
14: 0,1,2,3,4,5,6,7,8,9,10,11,12,
15: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,
16: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,
17: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,
18: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,
19: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,
20: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,