Question

I’m implementing a loop in Ruby, but it looks ugly and I wonder if there’s a neater, more Ruby-like way of writing it:

def get_all_items
  items = []; page = 1; page_items = nil
  while page_items != [] # Loop runs until no more items are received
    items += (page_items = get_page_items(page))
    page += 1
  end
  items
end

Note that the get_page_items method runs a HTTP request to get the items for the page, and there is no way of knowing the number of pages, or the total number of items, or the number of items for any page before actually executing the requests in order until one of them returns an empty item set.

Imagine leafing through a catalog and writing down all the products, without knowing in advance how many pages it has, or how many products there are.

Answer 1

I think that this particular problem is compounded because A) there’s no API for getting the total number of items and B) the response from get_page_items is always truthy. Further, it doesn’t make sense for you to iteratively call a method that is surely making individual requests to your DB with an arbitrary limit, only to concatenate them together. You should, at the risk of repeating yourself, implement this method to prompt a DB query (i.e. model.all).

Normally when you are defining an empty collection, iterating and transforming a set, and then returning a result, you should be using reduce (a.k.a inject):

array.reduce(0) { |result, item| result + item } # a quick sum

Your need to do a form of streaming in this same process makes this difficult without tapping into Enumerable. I find this to be a good compromise that is much more readable, even if a bit distasteful in fondling this items variable too much:

items = []
begin
  items << page_items = get_page_items(page ||= 1)
  page += 1
end until page_items.empty?
items.flatten