Home/ Questions/Q 7574549
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-30T16:27:45+00:00

I have an irregular time series and am using xts ‘s endpoints to get

  • 0

I have an irregular time series and am using xts‘s endpoints to get hourly indices of my time series. 

endpoints(data, on="hours")

I am using this in order to calculate hourly in such fashion

period.apply(data, INDEX=endpoints(data, on="hours"), FUN=mean)

The problem, however, is that function endpoints returns two consecutive indices (thus for the same hour).

> endpoints(data, on="hours")[7201:7220]
[1] 87077 87078 87089 87101 87113 87125 87137 87149 87162 87175 87187 87199 87211 87223 87235 87247 87259 87271 87283 87295

If we take a look which datetimes they represent:

data[endpoints(data, on="hours")[7201:7220]]

we notice that these are

> data[endpoints(data, on="hours")[7201:7220]]
                 jstimestamp X61757 X61754 X61760 X61753 X61758 X61762 X61756 X61759 X61761 X61755 X61752
2007-10-28 01:55:00 1.193529e+12  938.7 1339.6  450.8  799.4  850.0 1653.6  622.3  159.6 4415.4  681.6 1421.0
2007-10-28 02:00:00 1.193530e+12  946.0 1326.3  437.8  799.9  829.3 1644.1  629.0  182.8 4413.7  688.5 1397.2
2007-10-28 02:55:00 1.193533e+12  916.4 1337.0  432.3  778.0  838.6 1581.5  616.8  166.0 4282.8  670.9 1361.8
2007-10-28 03:55:00 1.193540e+12  909.1 1273.8  446.9  765.4  836.2 1559.7  599.5  163.8 4191.2  667.9 1373.3
2007-10-28 04:55:00 1.193544e+12  930.8 1320.3  426.4  758.3  834.8 1567.5  594.0  152.7 4130.2  688.4 1377.3
2007-10-28 05:55:00 1.193547e+12  943.5 1355.1  447.7  784.6  856.9 1592.4  629.0  163.8 4150.3  686.2 1391.5
2007-10-28 06:55:00 1.193551e+12 1018.3 1443.2  463.7  841.0  877.1 1677.3  670.8  161.8 4310.8  708.9 1441.3
2007-10-28 07:55:00 1.193554e+12 1052.2 1525.7  472.5  887.7  903.6 1734.9  716.1  199.5 4390.9  722.7 1504.3
2007-10-28 08:52:34 1.193558e+12 1167.1 1570.3  519.2  933.0  957.5 1919.9  795.0  225.5 4706.4  733.0 1561.1
2007-10-28 09:55:00 1.193562e+12 1224.1 1653.4  547.2  992.1 1039.9 2053.5  797.2  217.9 4952.1  739.4 1610.6
2007-10-28 10:55:00 1.193565e+12 1233.4 1745.9  569.8 1038.1 1060.8 2145.3  778.0  231.6 5182.4  759.1 1621.5
2007-10-28 11:55:00 1.193569e+12 1217.8 1751.6  581.3 1056.6 1084.2 2177.6  791.0  246.6 5296.4  758.6 1642.0
2007-10-28 12:55:00 1.193572e+12 1212.5 1786.3  589.2 1034.4 1069.2 2191.2  784.4  242.2 5357.5  728.5 1670.8
2007-10-28 13:55:00 1.193576e+12 1200.1 1694.8  586.1 1059.3 1063.2 2174.2  773.3  248.6 5336.7  747.8 1650.6
2007-10-28 14:55:00 1.193580e+12 1188.1 1736.7  577.7 1049.9 1041.1 2168.4  771.5  233.6 5332.9  746.9 1651.5
2007-10-28 15:55:00 1.193583e+12 1187.9 1696.8  574.1 1056.9 1060.4 2152.6  790.4  255.8 5326.9  740.6 1653.0
2007-10-28 16:55:00 1.193587e+12 1250.3 1793.2  580.8 1048.1 1116.8 2232.6  810.2  257.1 5360.4  765.6 1688.4
2007-10-28 17:55:00 1.193590e+12 1325.9 1796.0  614.7 1148.7 1134.2 2368.6  816.9  301.6 5530.3  772.7 1673.1
2007-10-28 18:55:00 1.193594e+12 1433.2 1966.3  697.8 1183.6 1276.2 2615.3  928.2  324.1 5805.4  762.7 1853.9
2007-10-28 19:55:00 1.193598e+12 1436.2 1906.2  678.5 1196.9 1217.9 2575.4  882.2  337.1 5809.5  789.7 1852.5

The problem is that hour 2007-10-28 02 is represented twice. My understanding of hourly endpoints is that this should not happen. Am I doing something wrong here?

Edit: This was indeed daylight savings in effect as per Dirk Eddelbuettel’s answer below. To resolve the problem, I needed to:

  1. convert to timezone UTC when parsing the data (the default timezone that was applied was my machine’s -> CET)

  2. after calculating hourly means as:

    data.hourly = period.apply(data, INDEX=endpoints(data, on=”hours”), FUN=mean)

I needed to manually override data.hourly’s time zone as well (again, default computer’s time zone was used):

indexTZ(data.hourly) <- "UTC"
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Just a hunch but this may be yet another manifestation of TZ and a switch from standard time to daylight savings time. Try to convert to UTC and see what happens.

    Alternatively, move your data by a week (or month or …) and see if the same issue arises.

    And if that’s the issue. then this is not really a bug as the ‘fall back’ night has indeed two hours starting at 02:00am.

    Edit: And while you didn’t say what your timezone was, and while it doesn’t work in mine (US Central), we can exhibit the issue for the European continent:

    R> ISOdate(2007, 10, 28, 0, 30, 0, tz="Europe/Berlin") + seq(0,4)*60*60
[1] "2007-10-28 00:30:00 CEST" "2007-10-28 01:30:00 CEST"
[3] "2007-10-28 02:30:00 CEST" "2007-10-28 02:30:00 CET" 
[5] "2007-10-28 03:30:00 CET" 
R>

    See how

    1. the TZ attribute switches from CEST to CET

    2. there are in fact two 02:30:00 hours, one for each of the two timezones.

    So no bug in xts but a feature in the data.

    • 0