Question

Is there a sensible way to group Long UTC dates by Day?

I would Mod them by 86400 but that doesn’t take leap seconds into account.

Does anyone have any other ideas, I’m using java so I could parse them into Date Objects, but I’m a little worried of the performance overhead of using the date class.

Also is there a more efficient way than comparing the year month and day parts of a Date object?

Answer 1

Does your source data definitely include leap seconds to start with? Some APIs do, and some don’t – Joda Time (which I’d recommend over the built-in date/time APIs) doesn’t use leap seconds, for example. The Java date and time APIs ‘sometimes’ do – they support 60 and 61 as values of ‘second in minute’ but support depends on the operating system (see below). If you have some good sample values, I’d check that first if I were you. Obviously just dividing is rather simpler than anything else.

If you do need to create Date objects (or DateTime in Joda) I would benchmark it before doing anything else. You may well find that the performance is actually perfectly adequate. There’s no point in wasting time optimizing something which is okay for your data. Of course, you’ll need to decide data size you need to support, and how quick it needs to be first 🙂

Even the java.util.Date support for leap seconds is somewhat indeterminate. From the docs:

Although the Date class is intended to reflect coordinated universal time (UTC), it may not do so exactly, depending on the host environment of the Java Virtual Machine. Nearly all modern operating systems assume that 1 day = 24 × 60 × 60 = 86400 seconds in all cases. In UTC, however, about once every year or two there is an extra second, called a ‘leap second.’ The leap second is always added as the last second of the day, and always on December 31 or June 30. For example, the last minute of the year 1995 was 61 seconds long, thanks to an added leap second. Most computer clocks are not accurate enough to be able to reflect the leap-second distinction.

There’s a rather good blog post about the mess with Java and leap seconds which you may want to read too.