Question

i am writing an app which does file downloading from a web service.

The way how files are downloaded is described as follows:

1. From the web service, download a list of files. If the number of files is too big, download the first group of files, the maximum number in each group is unknown. Downloaded files are saved in system temp directory.
2. For each file, construct a FileDescriptor (customed) class which contains the files filename and the temporary filename in the system temp folder and some other attribute.b The FileDescriptor is saved in a list.
3. Do some business logic here.
4. Download the next group of file if any.

The collection I used to save the list of FileDescriptor is LinkedList. However, for each group of files, I am going to create a brand new LinkedList to hold their FileDescriptors. The Psudocode is something like:

do {
   List<FileDescriptor> list = new LinkedList<FileDescriptor>();
   GroupOfFiles group = webService.getGroupOfFiles();
   Iterator<File> itr = group.iterator();
   while(itr.hasNext) {
       list.add(new(FileDescriptor(itr.next()));
   }

   <My Business Logic here>

} while(group.hasMoreGroups());

As can be seem, everytime a new group of files is retrieved, I am going to create a new LinkedList. After the group of files are processed, the LinkedList is no longer needed. I cannot create one list and reuse it because the maximum number each group can contain is unknown.

If I use the code like this and there are millions of files, each group can only up to 1000 file for example, it will end up with a lot of LinkedList (and its element) as Garbage. Is this something good to do? I am sure there will be better ways to handle it in this scenario.

Please give your opinion.

Many thanks.

Answer 1

Creating 1000 LinkedList objects (and garbage-collecting them) has a negligible cost compared to downloading millions of files. We’re talking about 1 milli-second vs. hours here. Some meters vs. the distance from the earth to the moon.

You’re pre-optimizing, and as is usually the case when pre-optimizing, you’re doing it at the wrong place.

That said, a list is a dynamically sized data structure, and you don’t need to know its maximum size to be able to reuse it. It will grow dynamically:

List<FileDescriptor> list = new LinkedList<FileDescriptor>();
do {
    list.clear(); // remove everything from the list
    ...
}
while (...);