Question

I have a tiff file and the text on it, which has been OCR’d at an earlier stage. The words have their exact positions as information (upper left, lower right). I now need to read the text within a user-drawn rectangle.

Normal paragraphs are no problem, but I don’t know how I should handle text columns. If there are two paragraphs next to each other, simply taking the row as a single line would make the result unusable.

Are there algorithms to help me put the words in the right order? I’m guessing that I have to examine the spaces between words to detect patterns that identify columns. I would like to avoid processing the image directly, although it should be possible (but no OCR).

I am also unsure about the influence of lists/tables, e.g. in orders & bills. A line-orientated approach would probably be better here.

I am developing in Delphi, but adaptable algorithms in other languages would also be appreciated.

edit: I will try to post sample data tomorrow, but basically I have an Array of Words, with their respective coordinates on the image (I could easily draw a rectangle around them, for example).

Answer 1

Suppose your original text is in two columns like this:

Aaaa bb ccc ddddd     mmmm nn oooo pp
eee fff ggggg hh      qqq rrrrrrrrr
i jjjj kkk lll        sss tttt uu.

From your description, it sounds like your OCR has given you the individual words and their bounding rectangles. If the original page is scanned orthogonally, then all of the words on a given line should have the same (or very close) y values. If they’re not exactly the same, you can do an integer division on the vertical positions with some fraction of a typical box height. That should cluster the y values. You can do similar processing on the x coordinates to ensure that words at the edge of a column also have identical x values.

To detect the separate columns, I’d try making a histogram of all the “left” values of all the words (or right edges if your text runs right-to-left). You should see a peak at the beginning of each column.

You can probably rule out any false positives by ensuring that, on every line, there is a gap between the right coordinate of the last box before the candidate start of a column. The gap should probably be at least as large as the smallest width of any word.

You can then partition your words up into column groups by checking which horizontal range their left and right coordinates fall in to. In our example, the words from Aaaa through lll would end up in the first partition and the words from mmmm through uu. would end up in the second partition.

Within each partition, you can then partition on line by sorting on the y coordinates. Finally, for each line, you sort on the x coordinate. (Whether you sort on ascending or descending depends on your coordinate system and the direction your text flows.)

The same basic idea could be applied to tables as well as columns of text, but you might need some tweaks to deal with things like right-justified cells.