Home/ Questions/Q 6785121
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T17:05:14+00:00

Please try this yourself :) ! curl http://www.windowsphone.com/en-US/apps?list=free the result is: <html><head><title>Object moved</title></head><body> <h2>Object

  • 0

Please try this yourself 🙂 !

curl http://www.windowsphone.com/en-US/apps?list=free

the result is:

    <html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="https://login.live.com/login.srf?wa=wsignin1.0&amp;rpsnv=11&amp;checkda=1&amp;ct=1320735308&amp;rver=6.1.6195.0&amp;wp=MBI&amp;wreply=http:%2F%2Fwww.windowsphone.com%2Fen-US%2Fapps%3Flist%3Dfree&amp;lc=1033&amp;id=268289">here</a>.</h2>
</body></html>

or

def download(source_url):
    try:
        socket.setdefaulttimeout(10)        
        agents = ['Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)','Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1)','Microsoft Internet Explorer/4.0b1 (Windows 95)','Opera/8.00 (Windows NT 5.1; U; en)']
        ree = urllib2.Request(source_url)
        ree.add_header('User-Agent',random.choice(agents))
        resp = urllib2.urlopen(ree)
        htmlSource = resp.read()
        return htmlSource
    except Exception, e:
        print e
        return ""

download('http://www.windowsphone.com/en-US/apps?list=free')

the result is:

<html><head><meta http-equiv="REFRESH" content="0; URL=http://www.windowsphone.com/en-US/apps?list=free"><script type="text/javascript">function OnBack(){}</script></head></html>

I want to download the actual source of the webpage.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    The reason it fails is because http://www.windowsphone.com attempts to set a cookie, which is checked on https://login.live.com which creates another cookie and redirects back to windowsphone.com if successful.

    You should look into http://docs.python.org/library/cookielib.html

    If you want to use curl, allow it to create a cookie-file like so:

    curl -so /dev/null 'http://www.windowsphone.com/en-US/apps?list=free' -c 'myCookieJar'

    Run more myCookieJar in your shell and you’ll see something like this:

    # Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

.www.windowsphone.com   TRUE    /       FALSE   0       WPMSLSS SLSS=1
login.live.com  FALSE   /       FALSE   0       MSPRequ lt=1320738008&co=1&id=268289

    Run (notice the -b option before ‘myCookieJar’):

    curl -so 'windowsphone.html' 'http://www.windowsphone.com/en-US/apps?list=free' -b 'myCookieJar'

    and you’ll get the contents of the page in the file windowsphone.html as you see it in your browser.

    • 0