Question

Let’s say there is a very large SQL database – hundreds of tables, thousands of fields, millions of records. This database is in Dutch. I would like to translate all the values of certain fields for test purposes in English. It doesn’t have to be completely correct, it has to be readable for the testers.

I know that most of the texts are stored in fields, named “name” and “description” throughout the database. Or basically all fields with type NVARCHAR(any length) in all tables would be the candidates for translation. Enumerating all the tables and fields is really too much work, and I would like to avoid it, so translating only these fields would be enough.

Is there a way to walk through all the tables and for all the records retrieve the values from the specific fields and replace them with their English translations? Can this be done with SQL only?

The db server doesn’t matter – I can mount the database on MSSQL, Oracle or any server of a choice. Translating the texts using google or some other automatic tool is good enough for the purpose. Of course this tool must have an API in order to be used automatically.

Does anybody have an experience with similar operations?

Answer 1

In Pseudocode, here is what I would do, using Oracle :

Query AllFieldsQuery = new Query(connection,"Select table_name, column_name 
from user_tab_columns where column_name='name' OR column_name='description'");

AllFieldsQuery.ExecuteReader();

It gives you something like this :

TABLE_NAME | COLUMN_NAME

Table1 | Name

Table2 | Name

Table2 | Description

……….. | ………..

Foreach TableColumnLine

Query FieldQuery = new Query("Select DISTINCT "+COLUMN_NAME+" AS ToTranslate 
from +"TABLE_NAME);

Create a new parametrized query :
MyParamQuery = new ParamQuery(connection, UPDATE TABLE_NAME SET COLUMN_NAME =
@Translated WHERE COLUMN_NAME = @ToTranslate);

Foreach ToTranslateLine

@Translated = GetTranslationFromGoogle(@ToTranslate);

MyParamQuery.Execute(ToTranslate = @ToTranslate, Translated = @Translated);

End Foreach ToTranslateLine

End Foreach TableColumnLine

The AllFieldsQuery gives you all the fields in all the tables you need to update.

The FieldQuery gives you all the texts you need to translate.

The MyParamQuery updates each one of these texts.

This may be a very long process, it depends also on the time you spend to get the translation. What I would recommend is to do at least a commit by table, and print a report to tell you for each table if everything went ok, because you could in your API have an exclude function that excludes fiels and/or tables from the first query because they were already translated before.

.

Enhancement : Writing a File and executing it at the end of the translation instead of executing sql updates.

If it’s too long, and it will possibly be too long, instead of executing the sql UPDATES directly, you’d rather write a file with all your updates, and after you foreach, go get the file and use it in SQL.

.

Enhancement : Multi Threaded Version

A multi threaded version of your API could let you win some translation time, you put all your results to translate in a synchronized queue, and you use a pool of threads to translate the queue, and put all the translated messages in another synchronized Queue, which is used to write the SQL output file or launching the updates.