I wrote some codes in PHP to extract content of some elements from other websites. These elements are addressed by XPath. These codes worked for one website successfully but failed for the other one. Therefore, I am sure that not the whole code is incorrect.
by the way: I extracted the element’s XPath address by using ‘Inspect Element’ in Firefox and Right Click on the element and choosing ‘Copy XPath’.
What is wrong for the second website?
thanks
Here is the code:
//MyCode.PHP
<html>
<head>
<title>This is the title</title>
</head>
<body>
<?php
class EmDIV
{
public $url="";
public $content="";
public $name="";
public $query="";
public function EmDIV($CdivName,$Curl,$CQuery)
{
$this->name=$CdivName;
$this->url=$Curl;
$this->query=$CQuery;
$html = new DOMDocument();
@$html->loadHtmlFile($this->url);
$bodies = $html->getElementsByTagName('body');
assert($bodies->length === 1);
$body = $bodies->item(0);
$xpath = new DOMXPath( $html );
$nodelist = $xpath->query($this->query);
//echo @$body->saveHTML();
if($nodelist->length==1)
{
$this->content=$nodelist->item(0)->textContent;
//sanitizing
//$this->content=Jsoup.clean($this->content, Whitelist.basic());
}
//echo $nodelist->item(0)->nodeName;
foreach ($nodelist as $node)
echo $node->getNodePath()."\n";
}
}
$emdiv=array(
//new EmDIV('parsmalaysia','http://www.parsmalaysia.com/exchange.html','/html/body/div/div[5]/div/div/div/div/div/div/div/div/div/div/table/tbody/tr/td[4]/text()'),
new EmDIV('atlas-exchange','http://atlas-exchange.com/','/html/body/div/div/table/tr/td/table/tr/td/div/div[10]/div/div/div/table/tr[3]/td[2]/text()'),
new EmDIV('usunmalaysia','http://www.usunmalaysia.com/Home.aspx','/html/body/form/table/tbody/tr[3]/td/table/tbody/tr/td/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr/td/table/tbody/tr/td[2]/div/table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[2]/td/table/tbody/tr[3]/td/table/tbody/tr[4]/td[2]/text()'),
);
?>
<table border="1">
<tr>
<td>Site</td>
<td>RM Price</td>
</tr>
<?php
foreach ($emdiv as $ed)
{
echo "<tr>";
echo "<td>".$ed->name."</td>";
echo "<td>".$ed->content."</td>";
echo "</tr>";
}
?>
</table>
</body>
</html>
In the second page there are no tbody elements. You have to remove all tbody elementes from the path. Firefox show the tbody elements because they are part of the internal structure of the tables, but they are not in the markup you are retrieving.