Question

I have recently noticed that our product has incorrect postcode locations (lat & long coordinates) for a number of “short” postcodes – i.e “AB10” rather than “ABD 1PT” etc.

The postcode database/table is used to generate pins on a Google map and I have now worked out that at some point when we merged the short postcodes to the table with full postcodes, some (around 2200) were incorrectly entered with the longitude and latitude around the wrong way.

Obviously this is a simple fix and so I decided to write a little script to take care of the incorrect values (basically swap them around).

Here is what I have:

<cfscript>

  /** Fetch the wrong postcodes data **/
  db  = "postcodes";
  sql = "
    SELECT
      postcode, longitude, latitude
    FROM
      postcodes
    WHERE
      longitude > latitude
  ";
  q    = new Query(sql = trim(sql), datasource = db);
  data = q.execute().getResult();

  if (structKeyExists(form, "execute")) {
    if (isQuery(data) && data.recordCount > 0) {
      transaction action="begin" 
      { 
        try {
          qUpdate = new Query(
            datasource = db, 
            sql = "UPDATE postcodes SET longitude = :longitude, latitude = :latitude WHERE postcode = :postcode"
          );
          for (x = 1; x <= data.recordCount; x++) {
            writeOutput("<p>" & data["postcode"][x] & "</p>");

            qUpdate.addParam(name = "longitude", value = data["latitude"][x], cfsqltype = "CF_SQL_DOUBLE");
            qUpdate.addParam(name = "latitude", value  = data["longitude"][x], cfsqltype = "CF_SQL_DOUBLE");
            qUpdate.addParam(name = "postcode", value  = data["postcode"][x], cfsqltype = "CF_SQL_VARCHAR");

            qUpdate.execute();
            qUpdate.clearParams();
          }
          transactionCommit();
        } catch (any e) {
          transactionRollback();
          writeOutput("<p>The database transaction failed, rolling back changes</p>");
          writeDump(e);
        }
      }
      writeOutput("#data.recordCount# postcodes have been updated");  
    } else {
      writeOutput("There were no incorrect postcodes found in the database");
    }
  }
</cfscript>
<cfoutput>
  <form name="update" action="" method="post">
    <input type="hidden" name="execute" value="1"/>
    <input type="submit" name="update" value="Update #val(data.recordCount)# Postcodes"/>
  </form>
</cfoutput>

<!--- <cfdump var="#data#"/> --->

The script is wrapped in a transaction as I had planned to run it on the live server however after testing the script locally it has continued to run for over an hour!

The postcode database contains almost 1.7 million records and has just three columns all correctly indexed postcode, longitude, latitude the first query returns the correct 2,200 results.

I have checked the component cache setting in ColdFusion admin to see if this was missing on my local, however it is turned on!

So my question – Why is this taking so long to execute?

We are using mysql and ACF 9.

Answer 1

Why do it in CF at all? Just do it all in SQL, it’ll be much faster. I don’t use mysql, but something vaguely like this:

UPDATE postcodes 
SET longitude = newlongitude,
latitude = newlatitude 
FROM (SELECT latitude AS newlongitude, longitude AS newlatitude FROM postcodes 
         WHERE longitude > latitude)