Question

we have a DITA XML application that produces on the fly xhtml and when viewed in the browser it looks fine.

An example url is: http://livecontent.jordanpublishing.co.uk/content/en/FAMILY-201103311115/Family_FLJONLINE_FLJ_2009_07_4

If I try to load the url using curl I get the following error:

Error checking function parameter 3 in call transform:transform($fDoc, LiveContent-UI:get_xsl("ui/ui_skin.xsl", ""), LiveContent-UI:get_xsl_params(untyped-value-check[xs:string, $skin], $extra_params)): The actual cardinality for parameter 1 does not match the cardinality declared in the function's signature: LiveContent-Util:browser_from_user_agent($a as xs:string) xs:string. Expected cardinality: exactly one, got 0.

XQuery Stack TraceLiveContent-Util:browser_from_user_agent(xs:string)   161:55
LiveContent-UI:get_xsl_params(xs:string, node())    145:25
LiveContent-UI:get_html(xs:string, xs:string, node(), node())   313:25
LiveContent-Pub:home(xs:string, xs:string, xs:string)   65:17
Java Stack Trace:Class Name Method Name File Name   Line
org.exist.xquery.DynamicCardinalityCheck    eval    DynamicCardinalityCheck.java    80
org.exist.xquery.Atomize    eval    Atomize.java    66
org.exist.xquery.UntypedValueCheck  eval    UntypedValueCheck.java  75
org.exist.xquery.DynamicTypeCheck   eval    DynamicTypeCheck.java   61
org.exist.xquery.FunctionCall   eval    FunctionCall.java   185
org.exist.xquery.AbstractExpression eval    AbstractExpression.java 61
org.exist.xquery.PathExpr   eval    PathExpr.java   241
org.exist.xquery.AttributeConstructor   eval    AttributeConstructor.java   95
org.exist.xquery.ElementConstructor eval    ElementConstructor.java 212
org.exist.xquery.AbstractExpression eval    AbstractExpression.java 61
org.exist.xquery.PathExpr   eval    PathExpr.java   241
org.exist.xquery.ElementConstructor eval    ElementConstructor.java 271
org.exist.xquery.AbstractExpression eval    AbstractExpression.java 61
org.exist.xquery.PathExpr   eval    PathExpr.java   241
org.exist.xquery.DebuggableExpression   eval    DebuggableExpression.java   56
org.exist.xquery.DebuggableExpression   eval    DebuggableExpression.java   63
org.exist.xquery.LetExpr    eval    LetExpr.java    208
org.exist.xquery.BindingExpression  eval    BindingExpression.java  158
org.exist.xquery.AbstractExpression eval    AbstractExpression.java 61
org.exist.xquery.PathExpr   eval    PathExpr.java   241

and I am completely at a loss as to what is going wrong.

The PHP Curl code is as follows:

$ch = curl_init();
/**
* Set the URL of the page or file to download.
*/
curl_setopt($ch, CURLOPT_URL, 'http://onlineservices.letterpart.com/sitemap.xml;jsessionid=1j1agloz5ke7l?id=1j1agloz5ke7l');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec ($ch);
curl_close ($ch);

$xml = new SimpleXMLElement($data);
foreach ($xml->url as $url_list) {
    $url = $url_list->loc;
    echo $url ."<br>"; 
    //file_get_contents($url);




    echo $url ."<br>";   
    $ch = curl_init($url); //load the urls
                    echo $url ."<br>";  
            curl_setopt($ch, CURLOPT_TIMEOUT_m2, 20); //No need to wait for it to load. Execute it and go.
            curl_exec($ch); //Execute
            curl_close($ch); //Close it off 

Can anyone help? I’m at a bit of a loss as this is way outside of my skill set.

Thanks,

EDIT:

It was suggested that I added a User Agent so I added the following:

curl_setopt($ch, CURLOPT_USERAGENT, “Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)”);

I now get the following error in my logs:

[13-May-2011 16:30:14] PHP Notice:  Use of undefined constant CURLOPT_TIMEOUT_m2 - assumed 'CURLOPT_TIMEOUT_m2' in /home/digital1/public_html/dev/sitemap.php on line 43
[13-May-2011 16:30:14] PHP Warning:  curl_setopt() [<a href='function.curl-setopt'>function.curl-setopt</a>]: Invalid curl configuration option in /home/digital1/public_html/dev/sitemap.php on line 43

The two line in question here (43) is:

curl_setopt($ch, CURLOPT_TIMEOUT_m2, 20); //No need to wait for it to load. Execute it and go.

I do seem to be having better luck using the Googlebot agent:

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

as in I am actually getting content written to the screen but my logs are still showing these errors.

Answer 1

My immediate guess is that your Java app is expecting the User-Agent header to be set. Since Curl does not send a User-Agent header by default, you will need to set one. Try adding this above your option to set CURL_TIMEOUT_m2.

curl_setopt($ch, CURLOPT_USERAGENT, "PHP/".PHP_VERSION );

If for some reason it does not like that User-Agent string, you might want to try using one from an actual browser.

EDIT:

Per your edit, this is because you’ve typoed the curl timeout constant. It should be CURLOPT_TIMEOUT_MS not CURLOPT_TIMEOUT_m2.