Question

I am looking to incorporate a loop in R which goes through every game’s boxscore data on the NFL statistics website here: http://www.pro-football-reference.com/years/2012/games.htm

At the moment I am having to manually click on the “boxscore” link for every game every week; is there any way to automate this in R? My code works with the Full play-by-play dataset within each link; it’s taking me ages at the moment!

Answer 1

Web scraping may be against the terms of use of some websites. The enforceability of these terms is unclear. While outright duplication of original expression will in many cases be illegal, in the United States the courts ruled in Feist Publications v. Rural Telephone Service that duplication of facts is allowable.

require(RCurl)
require(XML)
bdata<-getURL('http://www.pro-football-reference.com/years/2012/games.htm')
bdata<-htmlParse(bdata)
boxdata<-xpathSApply(bdata,'//a[contains(@href,"boxscore")]',xmlAttrs)[-1]

The above will get the boxscore stem for the various games.