Question

I’m stuck on a problem and I’m looking for ideas regarding the best way to tackle it.

I’ve taken over developing a site in which the backend is written in Perl and the front end makes extensive use of javascript.

The client regularly receives an update of a few hundred tracked objects from the backend. The objects are mapped onto a google map via javascript. The object hash (parsed by the javascript) contains lots of information about the object e.g. location, description and various state variables.
Some of the data is in string form and some numeric.

The problem is, that in the process of pushing the data out to the client side javascript all values are becoming strings. And so, in the javascript, if I test for example, to see if a value is positive the test is succeeding even if the value is 0 because the value is actually “0” not 0.

On the server side, data is encoded with JSON::XS as follows;

sub process_json {
my $self = shift;

return if( $self->{processed} );
$self->{processed} = 1;
if( $self->{requestrec}->status =~ /^3/ )
{
    return $self->{requestrec}->status;
}

$self->{data}->{session} = {
    id   => $self->session->id,
    user => $self->session->{data}->{__user},
};

use utf8;
my $output;
eval { $output = JSON::XS->new->utf8->encode( $self->{data} ); };
if($@)
{
    use Carp;
    confess($@);
}

$self->discard_request_body();
$self->req->content_type('application/json; charset=utf-8');
$self->req->headers_out->set( 'Expires' => 'Thu, 1 Jan 1970 00:00:00 GMT' );

my $out_bytes = encode( 'UTF-8', $output );
$self->req->headers_out->set( 'Content-Length' => length $output );
$self->req->print($output) unless( $self->req->header_only() );

delete $self->{session}->{data}->{errors}
    if( exists $self->{session}->{data}->{errors} );
delete $self->{session}->{data}->{info}
    if( exists $self->{session}->{data}->{info} );
} ## end of: sub process_json

On the client side, the data is decoded with JSON.parse as follows;

function http_process (req, callback, errorfn) {
if (req.readyState == 4) {
    this.activerequest = null;
    if (req.status != 200 && req.status != 0 && req.status != 304 && req.status != 403) {
        if (errorfn) { return errorfn(req); } else { dialogue("Server error", "Server error " + req.status); }
        if (req.status == 400) { dialogue("ERROR","Session expired"); this.failed = 1;
        } else if (req.status == 404) { dialogue("ERROR", "Server error (404)"); this.failed = 1;
        } else if (req.status == 500) { dialogue("ERROR", "Server error (500)"); this.failed = 1; }
    } else {
        if (callback) {
            if (req.responseText) {
                lstatus("Loaded (" + req.responseText.length + " bytes)");
                warn("Received " + req.responseText.length + " bytes");
                try {
                    var obj = JSON.parse(req.responseText);
                } catch(e) {
                    warn(e);
                    warn(req.responseText);
                }
                callback( obj );
            } else {
                callback({});
            }
        }
    }
}

}

Can anyone see a useful way of tackling this? Perl isn’t strongly typed and JSON::XS is simply parsing the numeric data as a string rather then a number. I have tried to add a reviver to to JSON.parse (see code below) – however it doesn’t work (it processes most of the data correctly but keeps failing with a variety of messages such as non_object_property_call or undefined_method.

Regardless it would be preferable if the work of parsing the data to ensure the correct type was handled on the server side. Can anyone see how that might be achieved efficiently?

function toNum(key, value) {
/*make sure value contains an integer or a float not a string */
/*TODO: This seems VERY inefficient - is there a better way */
if (value instanceof Object) return;
if (value instanceof Array) return;
var intRE = /^\d+$/;
var floatRE = /^\d*\.\d+$/;
if(value.match(intRE)) {
    value = parseInt(value);
    return value;
}
if(value.match(floatRE)) {
    value = parseFloat(value);
    return value;
}   

}

Answer 1

Simple Perl scalars (any scalar that is not a reference) are the most difficult objects to encode: JSON::XS and JSON::PP will encode undefined scalars as JSON null values, scalars that have last been used in a string context before encoding as JSON strings, and anything else as number value …

The trick is to “numify” any value you want to encode as a number right before you encode it.

$ perl -MJSON -e '$x="5"; print JSON->new->encode( [ $x, "$x", 0+$x ] )'
["5","5",5]