Question

I have a client-server application that send user data to the cloud (Amazon EC2 + RDS + S3).

1. Every user can have multiple devices connecting to the cloud & sending data at the same time
2. Client application installed on each device is multi-threaded and eventually upload multiple data snippets at the same time.

I’d like to reliably track disk usage used in this context and I wonder how to do this in this context?

I have two ideas so far, but I’m not even sure they are correct:

Option 1: Add a trigger to mysql table? ie.

CREATE TRIGGER DiskUsage AFTER UPDATE OF Fully_Updated_File_Flag ON Files
BEGIN
    for each row
    begin
        UPDATE Users SET SpaceUsed = SpaceUsed + new.Size WHERE (new.Fully_Updated_File_Flag = 1) And UserID=
    end
END;

If I opt to use triggers, how am I supposed to dynamically inject the user id?

---

Option 2: Update mysql table via PHP? ie.

<?php

  SendFileToS3($file_name);
  mysql_query('UPDATE Stats SET Value = Value + ' . filesize($file_name) . ' WHERE user_id=' . $user_id);

?>

What if two instances are trying to update the same record? (I’m using Mysql 5.5.27-log / MyISAM), would this still work.

---

Note #1 Although I didn’t yet release my application, I still need something that scales well. Even if it means changing db engine all together.

Note #2 DB-related code is encapsulated in modular functions (ie. InsertIntoDB(), UpdateDB() & DeleteFromDB()). Plus all of these routines relies on CodeIgniter 2.1 with active record class.

This is to say that I could always make the switch if I have to (although I’d like to avoid that)

Answer 1

You should use MySQL Triggers instead of PHP code and you have to store the related user_id into diskusage table.

I use InnoDB engine because of the CONSTRAINT. You can also use MyISAM, but you should remove the CONSTRAINT.

REMARK

I would use InnoDB because of Transactions and (more important here) Row-Locking.

Table Structure (InnoDB)

-- ----------------------------
--  Table structure for `users`
-- ----------------------------
DROP TABLE IF EXISTS `users`;
CREATE TABLE `users` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `Name` VARCHAR(10) NOT NULL DEFAULT '',
  `SpaceUsed` BIGINT(20) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8;

-- ----------------------------
--  Table structure for `diskusage`
-- ----------------------------
DROP TABLE IF EXISTS `diskusage`;
CREATE TABLE `diskusage` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `Filename` VARCHAR(50) NOT NULL DEFAULT '',
  `Size` BIGINT(20) NOT NULL,
  `user_id` INT(11) UNSIGNED DEFAULT NULL,
  `Fully_Updated_File_Flag` TINYINT(4) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `fk_diskusage_user` (`user_id`),
  CONSTRAINT `fk_diskusage_user` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=INNODB DEFAULT CHARSET=utf8;

Table Structure (MyISAM)

-- ----------------------------
--  Table structure for `users`
-- ----------------------------
DROP TABLE IF EXISTS `users`;
CREATE TABLE `users` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `Name` VARCHAR(10) NOT NULL DEFAULT '',
  `SpaceUsed` BIGINT(20) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

-- ----------------------------
--  Table structure for `diskusage`
-- ----------------------------
DROP TABLE IF EXISTS `diskusage`;
CREATE TABLE `diskusage` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `Filename` VARCHAR(50) NOT NULL DEFAULT '',
  `Size` BIGINT(20) NOT NULL,
  `user_id` INT(11) UNSIGNED DEFAULT NULL,
  `Fully_Updated_File_Flag` TINYINT(4) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `fk_diskusage_user` (`user_id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

Thats all, together with some triggers on table diskusage.

INSERT TRIGGER

-- ----------------------------
--  AFTER INSERT TRIGGER for `diskusage`
-- ----------------------------
delimiter ;;
CREATE TRIGGER `diskusage_after_insert` AFTER INSERT ON `diskusage` FOR EACH ROW BEGIN
  IF NEW.Fully_Updated_File_Flag = 1 THEN
    UPDATE users
    SET
      SpaceUsed = SpaceUsed + NEW.Size
    WHERE
      id = NEW.user_id;
  END IF;
END;
 ;;
delimiter ;

UPDATE TRIGGER

-- ----------------------------
--  AFTER UPDATE TRIGGER for `diskusage`
-- ----------------------------
delimiter ;;
CREATE TRIGGER `diskusage_after_update` AFTER UPDATE ON `diskusage` FOR EACH ROW BEGIN

  -- same to DELETE TRIGGER

  -- decrease SpaceUsed with OLD Size for OLD user

  IF OLD.Fully_Updated_File_Flag = 1 THEN
    UPDATE users
    SET
      SpaceUsed = SpaceUsed - OLD.Size
    WHERE
      id = OLD.user_id;
  END IF;

  -- same to INSERT TRIGGER

  -- increase SpaceUsed with NEW Size for NEW user

  IF NEW.Fully_Updated_File_Flag = 1 THEN
    UPDATE users
    SET
      SpaceUsed = SpaceUsed + NEW.Size
    WHERE
      id = NEW.user_id;
  END IF;

END;
 ;;
delimiter ;

DELETE TRIGGER

-- ----------------------------
--  AFTER DELETE TRIGGER for `diskusage`
-- ----------------------------
delimiter ;;
CREATE TRIGGER `diskusage_after_delete` AFTER DELETE ON `diskusage` FOR EACH ROW BEGIN

  IF OLD.Fully_Updated_File_Flag = 1 THEN
    UPDATE users
    SET
      SpaceUsed = SpaceUsed - OLD.Size
    WHERE
      id = OLD.user_id;
  END IF;

END;
 ;;
delimiter ;