Question

does it have access to the database or maybe it sits in the middle and crawls the results, others generate by their queries? anyway is it possible to write a crawler to crawl scientific databases like elsevier and pull information and store it in a database like just article title and publish year and auther … etc.?

Answer 1

The question title says “scientific database,” which suggests a repository of experimental quantitative data. That’s a rather broad question, because there’s no structure that you could impose on the universe of all experimental results. Furthermore, researchers have an interest in restricting access to their data in advance of publication. But if that’s what your’re interested in, you might start with the Wikipedia entry for Open notebook science: http://en.wikipedia.org/wiki/Open_notebook_science.

However, the question’s examples (“Elsevier,” “article title”) describe a repository of journal articles on scientific subjects. Generally, some article metadata is publicly available, hence crawlable (in principle), information like abstract, keywords, author names. But an article’s contents and its citations are protected by a paywall. (An exception, becoming increasingly important, is the trend to open access journals.)

Here’s an example article, selected at random, from Nature: http://www.nature.com/nature/journal/v485/n7396/full/nature10912.html