Home/ Questions/Q 9260319
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-18T12:51:41+00:00

I read a webpage which contains hebrew characters, using: response = (”).join(opener.open(url).readlines()) The result

  • 0

I read a webpage which contains hebrew characters, using:

response = ('').join(opener.open(url).readlines())

The result I get is mixed, some of the characters come back as unicode, which I can handle.

Some of the response seems garbled. In a format I cant recognize.
An example of the recieved text is:
שלך

More precisely, it looks like this (only a snippet…):

<h3 class="_52r al aps">About ‎גדי‎</h3><div>&#x5e9;&#x5dc;&#x5da; ....</div>

The text between the divs seems scrambled. Can I convert it to unicode?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You are looking at HTML entities; use the HTMLParser library to decode these:

    >>> import HTMLParser
>>> h = HTMLParser.HTMLParser()
>>> print h.unescape('&#x5e9;&#x5dc;&#x5da;')
שלך
>>> h.unescape('&#x5e9;&#x5dc;&#x5da;')
u'\u05e9\u05dc\u05da'

    To read a full urllib2 response, just use .read():

    response = opener.open(url).read()
    • 0