Repository navigation
Fix Unix time table calculation to maintain file day - #38
Conversation
balarsen
left a comment
There was a problem hiding this comment.
These look good to me, interesting issue to kind and sort
|
@jtniehof can you poke this? |
* This fixes the problem where day with record at 23:59:59.6 would appear to have data in the next day.
* If update the conversion in one place, need to update it everywhere for consistency.
21cff0c to
3dcb192
Compare
|
Few things going on here.
I'm going to turn off the "require up to date" and we'll just have to manually enforce "do a rebase and force push before merge" if the history looks like I think it does now...might have to do a reversion. |
OK, please do provide guidance, the button I hit was the only option. Seems like it is all good now. |
|
Looks like our history stayed linear, putting further discussion in #40. No reversion required, though. |
The Unix time table truncated start times for files to integer and used ceiling for end times. This means that a file with a timestamp late in the day (e.g. 23:59:59.600) could potentially be treated as if it had data in the next day, so for instance it would be passed in as an input for any processes that were using the next day. This PR does a truncate-to-integer in all cases, in both calculating the values for storage in the database and in converting datetimes to Unix time for lookup in the database.
Using an integer here is reasonable for speed (and not having to change database format yet again), and in general we don't operate with sub-second accuracy. We just need to do the Right Thing (or the Justifiable Thing) when there are sub-second timestamps. Because the beginning of a second is considered to be part of the second but the "end" of a second is not (i.e. second 5 includes 5.000000 but excludes 6.00000), this treatment guarantees that all moments within a given second are treated the same. (And thus, by induction, all moments within a minute, hour, day, etc.).
PR Checklist
See issue #orCloses #)