My queryScenario.php file is loading too slowly, avg 28sec. What can I do to decrease this with queries and loops?
I have no idea how to deal with it, since I know almost nothing about optimizing PHP.
Thanks.
This is my queryScenario.php file:
<?php
include("database.php");
include("dataModel.php");
if(!isset($_SESSION))
session_start();
//$userId = 1; // This variable will be obtained from the session
$userId = $_SESSION['userIdNumber'];
//$selectedUserLandId = $_GET['id']; // This variable may be defined as a session variable
$selectedUserLandId = $_SESSION['selectedUserLandId'];
// load the related user based model tables (dynamic data)
$landLiteral = dbGetUserLandByLandId($userId, $selectedUserLandId);
if (count($landLiteral)< 1){
// TODO: Throw an exception here
}
$landLiteralSelected = $landLiteral[0];
$userLand = new Land();
$userLand->id = $landLiteralSelected["id"];
$userLand->name = $landLiteralSelected["name"];
$userLand->desc = $landLiteralSelected["description"];
$landmap = dbGetLandmapById($landLiteralSelected["id"]);
$userLand->mapimage = $landmap[0]["mapimage"];
$geopos = dbGetGeoposById($landLiteralSelected["geopos_id"]);
$userLand->lat = $geopos[0]["lat"];
$userLand->lng = $geopos[0]["lng"] ;
$userLand->type = $landLiteralSelected["landenum_id"];
$sectorsLiteral = dbGetSectorsByLandId($userLand->id);
foreach ($sectorsLiteral as $sectorLiteral){
$sector = new Sector();
$sector->id = $sectorLiteral["id"];
$sector->name = $sectorLiteral["name"];
$sector->desc = $sectorLiteral["description"];
$geoposSector = dbGetGeoposById($sectorLiteral["geopos_id"]);
$sector->lat = $geoposSector[0]["lat"];
$sector->lng = $geoposSector[0]["lng"] ;
$userLand->sectionList[] = $sector;
$devicesLiteral = dbGetDevicesBySectorId($sector->id);
foreach ($devicesLiteral as $deviceLiteral){
$device = new Device();
$device->id = $deviceLiteral["id"];
$device->sceneName = $deviceLiteral["scenename"];
$device->sceneDesc = $deviceLiteral["scenedescription"];
$mapinfoDevice = dbGetMapInfoById($deviceLiteral["mapitem_id"]);
$device->mapInfo[] = $mapinfoDevice[0];
$geoposDevice = dbGetGeoposById($deviceLiteral["geopos_id"]);
$device->lat = $geoposDevice[0]["lat"];
$device->lng = $geoposDevice[0]["lng"];
$device->type = $deviceLiteral["deviceenum_id"];
$device->serialNo = $deviceLiteral["serialNo"];
$sector->deviceList[] = $device;
// NOW THE MODULES
$modulesLiteral = dbGetDeviceModulesByDeviceType($device->type);
foreach($modulesLiteral as $moduleLiteralList){
foreach ($moduleLiteralList as $moduleLiteral){
$module = new Module();
$module->id = $moduleLiteral["id"];
$module->type = $moduleLiteral["moduleenum_id"];
$module->unit = $moduleLiteral["unit"];
$module->subid = dbGetSubIdOfModule($module->id);
$deviceStateLiteral=dbGetDeviceStates($device->id);
if (count($deviceStateLiteral) > 0){
$moduleState=new ModuleState();
$moduleState->pendingState=$deviceStateLiteral[0]["pending_state"];
$moduleState->desiredState=$deviceStateLiteral[0]["desired_state"];
$moduleState->currentState=$deviceStateLiteral[0]["current_state"];
$module->moduleState=$moduleState;
}
$moduleDataLiteral=dbGetLastDeviceModuleData($device->id,$module->id);
$moduleData=new ModuleData();
if (count($moduleDataLiteral) > 0){
$moduleData->data=$moduleDataLiteral[0]["value"]." ".$module->unit;
$moduleData->timeStamp=$moduleDataLiteral[0]["timestamp"];
}else{
$moduleData->data="Veri Yok";
$moduleData->timeStamp="Veri Yok";
}
$module->moduleData=$moduleData;
$device->moduleList[] = $module;
// NOW THE MODULE SPECS
// First, get the default specs
$specsLiteral = dbGetDefaultSpecsByModuleId($module->id);
foreach($specsLiteral as $specLiteral){
$spec = new Spec();
$spec->type = $specLiteral["specenum_id"];
$spec->editable = $specLiteral["editable"];
$spec->value = $specLiteral["default"];
// NOW LOOK FOR UPDATED SPEC FOR THIS DEVICE MODULE AND SPEC
$updatedSpecLiteral = dbGetUpdatedDeviceSpec($device->id, $module->id, $spec->type);
if ($updatedSpecLiteral){
$spec->value = $updatedSpecLiteral[0]["specvalue"];
}
$module->specList[] = $spec;
}
}
}
}
}
if(!(isset($_GET["nonEcho"]) && $_GET["nonEcho"]=="yes"))
echo json_encode($userLand);
?>
These are data models:
<?php
class Node {
public $id;
}
class SpatialNode extends Node {
public $lat = 0;
public $lng = 0;
}
class Land extends SpatialNode{
public $name;
public $desc = "";
public $type;
public $sectionList = array();
public $mapimage;
}
class Sector extends SpatialNode{
public $name;
public $desc = "";
public $deviceList = array();
}
class Device extends SpatialNode{
public $sceneName;
public $sceneDesc;
public $type;
public $desc = "";
public $serialNo;
public $installationDate = "";
public $moduleList = array();
public $mapInfo = array();
}
class Module extends Node{
public $type;
public $unit = "";
public $subid;
public $specList = array();
public $moduleData;
public $moduleState;
}
class ModuleData{
public $data;
public $timeStamp;
}
class ModuleState{
public $pendingState;
public $desiredState;
public $currentState;
}
class Spec{
public $type;
public $value;
public $editable;
}
class DeviceStatus{
public $name;
public $address;
public $pendingState;
public $desiredState;
}
?>
These are queries:
<?php
//include("dataModel.php");
function dbConnect() {
$dbhost = 'XXXXXXXX';
$dbuser = 'XXXXX';
$dbpass = 'XXXXXXXXX';
$conn = mysql_connect($dbhost, $dbuser, $dbpass);
if (!$conn) {
die('Could not connect: ' . mysql_error());
}
mysql_select_db('XXXXXXXXX', $conn);
mysql_query("SET NAMES UTF8");
return $conn;
}
function executeSQL($sql) {
$conn = dbConnect();
$values = array();
$retval = mysql_query($sql, $conn);
if (!$retval) {
die('Could not get data: ' . mysql_error());
}
while ($row = mysql_fetch_array($retval, MYSQL_ASSOC)) {
array_push($values, $row);
}
return $values;
}
function executeSQLForPersist($sql){
$conn = dbConnect();
$retval = mysql_query($sql, $conn);
return $retval;
}
function dbGetLandsByUserId($userId){
$sql = "SELECT * FROM land WHERE user_id=$userId or user_id=1 ";
return executeSQL($sql);
}
function dbGetUserLandByLandId($userId, $landId){
$sql = "SELECT * FROM land WHERE user_id=$userId AND id=$landId UNION SELECT * FROM land WHERE user_id=1 AND id=1";
return executeSQL($sql);
}
function dbGetGeoposById($geoId){
$sql = "SELECT * FROM geoposition WHERE id=$geoId";
return executeSQL($sql);
}
function dbGetLandmapById($id){
$sql = "SELECT * FROM landimages WHERE land_id=$id";
return executeSQL($sql);
}
function dbGetMapInfoById($id) {
$sql = "SELECT * FROM mapitems WHERE id=$id";
return executeSQL($sql);
}
function dbGetEnums(){
$sql = "SELECT * FROM enums";
return executeSQL($sql);
}
function dbGetSceneDevices(){
$sql = "SELECT * FROM scenedevices";
return executeSQL($sql);
}
function dbGetSectorsByLandId($landId){
$sql = "SELECT * FROM sector WHERE land_id=$landId";
return executeSQL($sql);
}
function dbGetDevicesBySectorId($sectorId){
$sql = "SELECT * FROM scenedevices WHERE sector_id=$sectorId";
return executeSQL($sql);
}
function dbGetDeviceModulesByDeviceType($deviceTypeId){
$sql = "SELECT module_id FROM devicedef WHERE deviceenum_id=$deviceTypeId";
$moduleIdsLiteral = executeSQL($sql);
$modulesLiteralList = array();
foreach ($moduleIdsLiteral as $moduleIdLiteral){
$moduleId = $moduleIdLiteral["module_id"];
$sqlModuleQuery = "SELECT * FROM module WHERE id=$moduleId";
$modulesLiteralList[] = executeSQL($sqlModuleQuery);
}
return $modulesLiteralList;
}
function dbGetDefaultSpecsByModuleId($moduleId){
$sql = "SELECT * FROM module_specs WHERE module_id=$moduleId";
return executeSQL($sql);
}
function dbGetUpdatedDeviceSpec($deviceId, $moduleId, $specId){
$sql = "SELECT * FROM deviceprefs WHERE scenedevice_id=$deviceId AND module_id=$moduleId AND specenum_id=$specId";
return executeSQL($sql);
}
function dbGetProfileLandImageByLandId($landId){
$sql = "SELECT * FROM landimages WHERE land_id=$landId AND profile=1";
return executeSQL($sql);
}
function dbGetSubIdOfModule($moduleId){
$sql = "SELECT * FROM devicedef WHERE module_id='$moduleId'";
$idLiteral = executeSQL($sql);
if(count($idLiteral) != 0) {
return $idLiteral[0]["subid"];
}
else {
return -1;
}
}
function dbGetModuleIdByModuleEnumId($moduleEnumId){
$sql = "SELECT * FROM module WHERE moduleenum_id=$moduleEnumId";
$idLiteral = executeSQL($sql);
if (count($idLiteral) == 1){
return $idLiteral[0]['id'];
}else{
return -1;
}
}
function dbGetAllDeviceModuleData($devId, $modId){
$sql = "SELECT * FROM realtimedevicedata WHERE scenedevice_id=$devId AND module_id=$modId ORDER BY timestamp LIMIT 0, 500"; //TODO.
return executeSQL($sql);
}
function dbGetLastDeviceModuleData($devId, $modId){
$sql = "SELECT * FROM realtimedevicedata WHERE scenedevice_id=$devId AND module_id=$modId ORDER BY timestamp desc limit 1";
return executeSQL($sql);
}
function dbGetLastDayDeviceModuleData($devId, $modId){
$sql = "SELECT * FROM realtimedevicedata WHERE scenedevice_id=$devId AND module_id=$modId AND timestamp>=SYSDATE()-INTERVAL 1 DAY ORDER BY timestamp";
return executeSQL($sql);
}
function dbGetIntervalDeviceModuleData($devId, $modId, $startDate, $endDate){
$sql = "SELECT * FROM realtimedevicedata WHERE scenedevice_id=$devId AND module_id=$modId AND timestamp>='$startDate' AND timestamp<='$endDate' ORDER BY timestamp";
return executeSQL($sql);
}
function dbGetDeviceStates($devId) {
$sql = "SELECT * FROM deviceprefs WHERE scenedevice_id=$devId";
return executeSQL($sql);
}
function dbGetAllDeviceStates($imei) {
$sql = "SELECT scenedevices.scenename, scenedevices.serialNo, deviceprefs.pending_state, ".
"deviceprefs.desired_state FROM scenedevices, deviceprefs ".
"WHERE scenedevices.parentId = '".$imei."' AND scenedevices.id = deviceprefs.scenedevice_id ";
return executeSQL($sql);
}
?>
I realize that the reason of my code is running slow is that realtimedevicedata table is too large(approx 120MB and has 50K records) and I try to order this table before select no records. To avoid this, I change my code to fetch first 500 records from that table and then order the table. This breaks the chains.
Thanks to all who wanted to help, and sorry for the time loose.
Have a good day.
SQL changes :
Old one :
New one:
Proof of realtimedevicedata is a bad girl:
Now, she promise me to be a nice girl 🙂