Question

I want to extract text contents from GUI apps,here are 2 examples::

example 1:

Suppose I opened firefox, and input url : http://www.google.com

how can I extract the string “www.google.com” from firefox using my own app ?

example 2:

open calculator(using gcalctool),then input 1+1

How can I extract the string “1+1” of calculator from my own program?

in brief ,what I want is to find out whether there is a way to extract the text contents from any widget of an GUI application

Thanks

Answer 1

I don’t think there’s a generic way to do this, at least not a very elegant one.

Some inelegant ideas:

You might be able to modify the X window system or even some toolkit framework to extract what is being displayed in specific window elements as text.

You could take a screenshot and use an OCR library to convert the pixels back into text for the interesting areas.

You could recompile the apps of interest to add some kind of mechanism for asking them questions.

You could use something like xtest to inject events highlighting the region of interest and copying it to the clipboard.