Question

I wrote a simple powershell script that recursively walks a file tree and returns the paths of each node along with the time of its creation in tab-separated form, so that I can write it out to a text file and use it to do statistical analysis:

echo "PATH  CREATEDATE"
get-childitem -recurse | foreach-object {
$filepath = $_.FullName
$datecreated = $_.CreationTime
echo "$filepath $datecreated"
} 

Once I had done this, however, I noticed that the CreationDate times that get produced by the script are exactly one hour ahead of what Windows Explorer says when I look at the same attribute of the same files. Based on inspecting the rest of my dataset (which recorded surrounding events in a different format), it’s clear that the results I get from explorer are the only ones that fit the overall narrative, which leads me to believe that there’s something wrong with the Powershell script that makes it write out the incorrect time. Does anyone have a sense for why that might be?

Problem background:

I’m trying to correct for a problem in the design of some XML log files, which logged when the users started and stopped using an application when it was actually supposed to log how long it took the users to get through different stages of the workflow. I found a possible way to overcome this problem, by pulling date information from some backup files that the users sent along with the XML logs. The backups are generated by our end-user application at the exact moment when a user transitions between stages in the workflow, so I’m trying to bring information from those files’ timestamps together with the contents of the original XML log to figure out what I wanted to know about the workflow steps.

Summary of points that have come out in comment discussion:

• The files are located on the same machine as the script I’m running (not a network store)
• Correcting for daylight savings and time zones has improved the data quality, but not for the specific issue posed in the original question.

Answer 1

I never found the ultimate technical reason for the discrepancy between the timestamps from powershell vs. explorer, but I was able to correct for it by just subtracting an hour off all the timestamps I got from the powershell script. After doing that, however, there was still a large amount of disagreement between the time stamps I got from out of my XML log files and the ones I pulled from the filesystem using the powershell script. Reasoning that the end-users probably stayed in the same time zone when they were generating the files, I wrote a little algorithm to estimate the time zone of each user by evaluating the median amount of time between steps 1 and 2 in the workflow and steps 2 and 3. If there was a problem with the user’s time zone, one of those two timespans would be negative (since the time of the step 2 event was estimated and the times of the steps 1 and 3 events were known from the XML logs.) I then rounded the positive value down to the nearest hour and applied that number of hours as an offset to that user’s step 2 times. Overall, this took the amount of bad data in my dataset from 20% down to 0.01%, so I’m happy with the results.

In case anyone needs it, here’s the code I used to make the hour offset in the timestamps (not powershell code, this was in a C# script that handled another part of data processing):

    DateTime step2time = DateTime.Parse(LastModifyDate);
    TimeSpan shenanigansCorrection = new TimeSpan(step2time.Hour-1,step2time.Minute,step2time.Second); 
    step2time= step2time.Date + shenanigansCorrection;

The reason for redefining the step2time variable is that DateTimes aren’t mutable in .NET.