Home/ Questions/Q 6060067
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T08:46:59+00:00

Summary I am trying to design a heuristic for matching up sentences in a

  • 0

Summary

I am trying to design a heuristic for matching up sentences in a translation (from the original language to the translated language) and would like guidance and tips. Perhaps there is a heuristic that already does something similar? So given two text files, I would like to be able to match up the sentences (so I can pick out a sentence and say this is the translation of that sentence).

Details

The input text would be translated novels. So I do not expect the translations to be literal, although, using something like google translate might be a good way to test the accuracy of the heuristic.

To help me, I have a library that will gloss the contents of the translated text and give me the definitions of the words in the sentence. Other things I know:

  • Chapters and order are preserved; I know that the first sentence in chapter three will match with the first sentence in chapter three of the translation (Note, this is not strictly true; the first sentence might match up with the first two sentences, or even the second sentence)
  • I can calculate the overall size (characters, sentences, paragraphs); which could give me an idea of the average difference in sentence size (for example, the translation might be 30% longer).

Looking at the some books I have, the translated version has about 30% more sentences than the original text.

Implementation

(if it matters)

  • I am planning to do this in Java – but I am not that fussed – any language will do.
  • I am not greatly concerned about speed.

I guess to to be sure of the matches, some user feedback might be required. Like saying “Yes, this sentence definitely matches with that sentence.” This would give the heuristic some more ground to stand on. This would mean that the user would need a little proficiency in the languages.

Background

(for those interested)

The reason I want to make this is that I want it to assist with my foreign language study. I am studying Japanese and find it hard to find “good” material (where “good” is defined by what I like). There are already tools to do something similar with subtitles from videos (an easier task – using the timing information of the video). But nothing, as far as I know, for texts.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    There are tools called “sentence aligners” used in NLP research that does exactly what you want.

    I advise hunalign:

    http://mokk.bme.hu/resources/hunalign/

    and MS sentence aligner:

    http://research.microsoft.com/en-us/downloads/aafd5dcf-4dcc-49b2-8a22-f7055113e656/

    Both are quite OK, but remember that nothing is perfect. Sentences that are too hard to be aligned will be dropped and some sentences may be wrongly aligned.

    • 0