Question

I got high frequency data from a limit order book in Stata. Time does not have a regular interval, and some observations are at the same time (in milliseconds). For each observation I need to get the midpoint 5 minutes later in a separate column. So for observation 1 the midpoint would be 10.49, because the last midpoint closest to 09:05:02.579 would be 10.49.

How to do this in Stata?

datetime                         midpoint
12/02/2012 09:00:02.579          10.5125
12/02/2012 09:00:03.471          10.5125
12/02/2012 09:00:03.471          10.5125
12/02/2012 09:00:03.471          10.51
12/02/2012 09:00:03.471          10.51
12/02/2012 09:00:03.549          10.505
12/02/2012 09:00:03.549          10.5075
   ......
12/02/2012 09:04:59.785          10.495
12/02/2012 09:05:00.829          10.4925
12/02/2012 09:05:01.209          10.49
12/02/2012 09:05:03.057          10.4875
12/02/2012 09:05:05.055          10.485
 .....

Answer 1

My approach would be

1. generate a new data set shifted by five minutes
2. append this shifter data set
3. find closest before and after observations to your five minute delta
4. use some criteria to pick the better of these two values

You specified closest, but you might want to add some other criteria depending on your book. Also, you mentioned more than one value at a given ms tick, but without more information I’m not sure how to handle that. Do you want to combine those midpoints first? Or are they different stocks?

Here’s some code that implements the basics of the approach above.

clear
version 11.2
set seed 2001

* generate some data
set obs 100000
generate double dt = ///
    tc(02dec2012 09:00:00.000) + 1000*_n + int(100*rnormal())
format dt %tcDDmonCCYY_HH:MM:SS.sss
sort dt
generate midpt = 100
replace midpt = ///
    round(midpt[_n - 1] + 0.1*rnormal(), 0.005) if (_n != 1)

* add back future midpts
preserve
tempfile future
rename midpt fmidpt
rename dt fdt
generate double dt = fdt - tc(00:05:00.000)
save `future'
restore
append using `future'

* generate midpoints before and after 5 minutes in the future
sort dt
foreach v of varlist fdt fmidpt {
    clonevar `v'_b = `v'
    replace `v'_b = `v'_b[_n - 1] if missing(`v'_b)
}

gsort -dt
foreach v of varlist fdt fmidpt {
    clonevar `v'_a = `v'
    replace `v'_a = `v'_a[_n - 1] if missing(`v'_a)
}

format fdt* %tcDDmonCCYY_HH:MM:SS.sss

* use some algorithm to pick correct value
sort dt    
generate choose_b = ///
    ((dt + tc(00:05:00.000)) - fdt_b) < (fdt_a - (dt + tc(00:05:00.000))) 
generate fdt_c = cond(choose_b, fdt_b, fdt_a)
generate fmidpt_c = cond(choose_b, fmidpt_b, fmidpt_a)
format fdt_c %tcDDmonCCYY_HH:MM:SS.sss