Question

I was wondering what would be the most ethical way to consume some bytes (386 precisely) of content from a given Site A, with an application (e.g. Google App Engine) in some Site B, but doing it right, no scraping intended, I really just need to check the status of a public service and they’re currently not providing any API. So the markup in Site A has a JavaScript array with the info I need and being able to access that let’s say once every five minutes would suffice.

Any advice will be much appreciated.

UPDATE:

First all thanks much for the feedback. Site A is basically the website of the company that currently runs our public subway network, so I’m planning to develop a tiny free Android app for anyone to have not only a map with the whole network and its stations but also updated information about the availability of the service (and those are the bytes I will eventually be consuming), etcétera.

Answer 1

There will be some very differents points of view, but hopefully here is some food for thought:

1. Ask the site owner first, if they know ahead of time they are less likely to be annoyed.
2. Is the content on Site A accessible on a public part of the site, e.g. without the need to log in?
3. If the answer to #2 is that it is public content, then I wouldn’t see an issue, as scraping the site for that information is really no different then pointing your browser at the site and reading it for yourself.
4. Of course, the answer to #3 is dependent on how the site is monetised. If Site A provides advertistment for generating revenue for the site, then it might not be an idea to start scraping content, as you would be bypassing how the site makes money.

I think the most important thing to do, is talk to the site owner first, and determine straight from them if:

1. Is it ok for me to be scraping content from their site.
2. Do they have an API in the pipeline (simply highlighting the desire may prompt them to consider it).

Just my point of view…