Home/ Questions/Q 4043524
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-20T13:08:16+00:00

I am working with bidirectional text (mixed English and Hebrew) for a project. The

  • 0

I am working with bidirectional text (mixed English and Hebrew) for a project. The text is displayed in HTML, so sometimes a LTR or RTL mark (‎ or ‏) is required to make ‘weak characters’ like punctuation display properly. These marks are not present in the source text due to technical limitations, so we need to add them in order for the final displayed text to appear correct.

For instance, the following text: (example: מדגם) sample renders as sample (מדגם :example) in right-to-left mode. The corrected string would look like ‎(example:‎ מדגם) sample and would render as sample (מדגם (example:.

We’d like to do on-the-fly insertion of these marks rather than re-authoring all the text. At first this seems simple: just append an ‎ to each instance of punctuation. However, some of the text that needs to get modified on-the-fly contains HTML and CSS. The reasons for this are unfortunate and unavoidable.

Short of parsing HTML/CSS, is there a known algorithm for on-the-fly insertion of Unicode directional marks (pseudo-strong characters)?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I don’t know of an algorithm to insert directional marks into an HTML string safely without parsing it. Parsing the HTML into a DOM and manipulating the text nodes is the safest way of ensuring you don’t accidentally add directional marks to text inside <script> and <style> tags.

    Here is a short Python script which might help you transform your files automatically. The logic should be easy to translate into other languages if necessary. I’m not familiar enough with the RTL rules you’re trying to encode, but you can tweak the regexp '(\W([^\W]+)(\W)' and substituion pattern ur"\u200e\1\2\3\u200e" to get your expected result:

    import re
import lxml.html

_RE_REPLACE = re.compile('(\W)([^\W]+)(\W)', re.M)

def _replace(text):
    if not text:
        return text
    return _RE_REPLACE.sub(ur'\u200e\1\2\3\u200e', text)

text = u'''
<html><body>
    <div>sample (\u05de\u05d3\u05d2\u05dd :example)</div>
    <script type="text/javascript">var foo = "ignore this";</script>
    <style type="text/css">div { font-size: 18px; }</style>
</body></html>
'''

# convert the text into an html dom
tree = lxml.html.fromstring(text)
body = tree.find('body')
# iterate over all children of <body> tag
for node in body.iterdescendants():
    # transform text with trails after the current html tag
    node.tail = _replace(node.tail)
    # ignore text inside script and style tags
    if node.tag in ('script','style'):
        continue
    # transform text inside the current html tag
    node.text = _replace(node.text)

# render the modified tree back to html
print lxml.html.tostring(tree)

    Output:

    python convert.py

<html><body>
    <div>sample (&#1502;&#1491;&#1490;&#1501; &#8206;:example)&#8206;</div>
    <script type="text/javascript">var foo = "ignore this";</script>
    <style type="text/css">div { font-size: 18px; }</style>
</body></html>
    • 0