Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I am trying to write a crawler in a bash to download images from bing. Everything works fine except the fact that bing loads only 30 images in the beginning (uses infinite scrolling) and hence I am not able to download more than 30 images using wget.
Any workaround to download more than 30 images from bing images at once.?
—
Thanks for any help. 🙂
Bing has an API, perhaps that would work better than screen scraping? It would certainly be more allowed/supported by Bing than screen scraping, and less prone to breaking changes as they change their offerings over time.
If you’re really set on the screen scraping for whatever reason, you might try loading the page with debugging (such as with Firebug) and watch what happens when the infinite scroll is triggered. Then try to manually trigger it with code.