Question

I have problem SIMILAR to preventing form data reposting, but not quite the same. That question has been answered many times. The advice given, like header redirects and sessions, didn’t work for me. Whether they failed because of my incorrect implementation, lack of experience solving this problem, or both, I don’t know.

I have an intranet web app that, when launched, extracts data from a different server, transforms it, and then loads it to a database local to the application. What happens is that if the user refreshes the main page, the ETL scripts run again and duplicate the data.

How can prevent the ETL scripts from re-running? For this app, there is no reason to refresh the page. Should I simply prevent refreshes altogether on that page, if that’s even possible? I’m probably overlooking something simple. So, I’m open to suggestions.

UPDATE:

I got this test code to work on the index.php page. So, I’m going to build on that to craft a solution. I’ll keep you posted.

session_start();
$_SESSION['views'] = 1; // store session data
echo "Pageviews = ". $_SESSION['views']; //retrieve data

Answer 1

I’ve outlined a very basic example for you of how you could use a session to make sure your scripts are only run once per visit (per user). The key point here is the script would still run more than once in the event of more than 1 user using the system.

To get around this I would simply use your local db to keep track of when the scripts were last run. So you could have a datetime column in your db that would keep track of when the last ‘run’ was performed.

1. So in your PHP code (in replace of checking the $_SESSION) you would query the db to retrieve the datetime value
2. Once you have the last run date/time you would check if the current date/time is past the allowable ‘update time’ (for example 24hrs)
3. If the time is past your alloted ‘update time’ say 25hrs the code would run and you would update the datetime to now()
4. If the time is within your alloted time, nothing would happen, no update of db and your scripts would not run.
5. Repeat 3 – 4.

This above way would make the number of users irrelevant as all checks will be performed against a central data set (i.e. your db). It would also probably be a bit more robust as it will be independent of browser and user.

Anyway here is a session (per user) method:

<?php
/* 
 * Set PHP sessions to suit:
 * 
 * session.gc_maxlifetime is the number of seconds after which the data 
 * will be seen as garbage and cleaned up (session life) - 
 * 
 * Set to 86400 to in theory make the session last 24 hours - adjust this 
 * number to suit.
 * 
 * If this doesn't work you may have to try and adjust the php session lifetime
 * in either the php.ini file directly or htaccess file.
 * 
 */
ini_set('session.gc_maxlifetime',86400);
/* 
 * not strictly neccessary as this is default setting (but just to make sure)
 * This will keep the cookie alive (the link to your session) until the user
 * closes their browser. 
 * 
 * You could set it to > 86400 if you want that extra bit of certainity that 
 * the session will get renewed @ 24 hrs / the time you wanted to.
 */ 
ini_set('session.cookie_lifetime',0);
// Start PHP session
session_start(); 

// Check if the session is empty / exists (which it will be on first page entry)
if ( empty($_SESSION['hasVisited']) ) {

  /* 
   * Add your ETL script logic here:
   * 
   * This will only run on the first visit to the site
   */   
}

// Now set the session as the scripts have run once
$_SESSION['hasVisited'] = true;

// Continue on with your app...................................................

?>

Let me know if I missed what you were trying to do or you want further explanations on anything?