String-Date Conversion with Nanoseconds

String-Date conversion with nanoseconds

The result you are getting is absolutely right.

Let's analyze this:

17.08.2012 05:35:19:7600000
  • 17: Day of month (17th)
  • 08: Month of year (August)
  • 2012: Year (2012)
  • 05: Hour of day (5am)
  • 35: Minute of hour (:35)
  • 19: Second of minute (:19)
  • 7600000: Milliseconds of second (7,600,000)

Now, the way the VM sees this is that you are declaring the time of day as 5:35:19am, then adding 7,600,000 milliseconds to it. 7,600,000 milliseconds = 7,600 seconds = 2 hours, 6 minutes, 40 seconds. 5:35:19am + 02:06:40 = 7:41:59am (and 0 milliseconds). This is the result you are getting. (It also appears that you are not setting the timezone properly, so the GMT string is 3 hours behind your result.)

If you want to retain the :7600000, to my knowledge this is not possible. As this can be simplified into seconds, the VM will automatically reduce it into the other time increments. The milliseconds (the SSSS) should be for storing values <1000.

I'd suggest you create a new SimpleDateFormat for your output; but remember that the milliseconds will be absorbed into the other times (since they are all stored as a single long in the Date object).

Convert date with nanoseconds in java

You are using the troublesome old legacy date-time classes that are now supplanted by the java.time classes. These legacy classes are limited to milliseconds which means three decimal places in the fraction of a second. So the legacy classes cannot handle the nine digits you desire.

The nine digits of decimal fraction you want means a resolution of nanoseconds rather than milliseconds. Fortunately, the java.time classes have a resolution of nanoseconds.

Parse your input as a LocalDateTime object as it lacks any indication of time zone or offset-from-UTC. Replace the SPACE in middle with a "T" to comply with standard ISO 8601 format.

String input = "2017-05-17 00:31:16.227".replace( " " , "T" ) ;
LocalDateTime ldt = LocalDateTime.parse( input );

You should be able to force the trailing zeros you desire in the decimal fraction in a string representing the value of this LocalDateTime object. Use either a DateTimeFormatter or a DateTimeFormatterBuilder.

Java date parsing with microsecond or nanosecond accuracy

tl;dr

LocalDateTime.parse(                 // With resolution of nanoseconds, represent the idea of a date and time somewhere, unspecified. Does *not* represent a moment, is *not* a point on the timeline. To determine an actual moment, place this date+time into context of a time zone (apply a `ZoneId` to get a `ZonedDateTime`). 
"2015-05-09 00:10:23.999750900" // A `String` nearly in standard ISO 8601 format.
.replace( " " , "T" ) // Replace SPACE in middle with `T` to comply with ISO 8601 standard format.
) // Returns a `LocalDateTime` object.

Nope

No, you cannot use SimpleDateFormat to handle nanoseconds.

But your premise that…

Java does not support time granularity above milliseconds in its date patterns

…is no longer true as of Java 8, 9, 10 and later with java.time classes built-in. And not really true of Java 6 and Java 7 either, as most of the java.time functionality is back-ported.

java.time

SimpleDateFormat, and the related java.util.Date/.Calendar classes are now outmoded by the new java.time package found in Java 8 (Tutorial).

The new java.time classes support nanosecond resolution. That support includes parsing and generating nine digits of fractional second. For example, when you use the java.time.format DateTimeFormatter API, the S pattern letter denotes a "fraction of the second" rather than "milliseconds", and it can cope with nanosecond values.

Instant

As an example, the Instant class represents a moment in UTC. Its toString method generates a String object using the standard ISO 8601 format. The Z on the end means UTC, pronounced “Zulu”.

instant.toString()  // Generate a `String` representing this moment, using standard ISO 8601 format.

2013-08-20T12:34:56.123456789Z

Note that capturing the current moment in Java 8 is limited to millisecond resolution. The java.time classes can hold a value in nanoseconds, but can only determine the current time with milliseconds. This limitation is due to the implementation of Clock. In Java 9 and later, a new Clock implementation can grab the current moment in finer resolution, depending on the limits of your host hardware and operating system, usually microseconds in my experience.

Instant instant = Instant.now() ;  // Capture the current moment. May be in milliseconds or microseconds rather than the maximum resolution of nanoseconds.

LocalDateTime

Your example input string of 2015-05-09 00:10:23.999750900 lacks an indicator of time zone or offset-from-UTC. That means it does not represent a moment, is not a point on the timeline. Instead, it represents potential moments along a range of about 26-27 hours, the range of time zones around the globe.

Pares such an input as a LocalDateTime object. First, replace the SPACE in the middle with a T to comply with ISO 8601 format, used by default when parsing/generating strings. So no need to specify a formatting pattern.

LocalDateTime ldt = 
LocalDateTime.parse(
"2015-05-09 00:10:23.999750900".replace( " " , "T" ) // Replace SPACE in middle with `T` to comply with ISO 8601 standard format.
)
;

java.sql.Timestamp

The java.sql.Timestamp class also handles nanosecond resolution, but in an awkward way. Generally best to do your work inside java.time classes. No need to ever use Timestamp again as of JDBC 4.2 and later.

myPreparedStatement.setObject( … , instant ) ;

And retrieval.

Instant instant = myResultSet.getObject( … , Instant.class ) ;

OffsetDateTime

Support for Instant is not mandated by the JDBC specification, but OffsetDateTime is. So if the above code fails with your JDBC driver, use the following.

OffsetDateTime odt = instant.atOffset( ZoneOffset.UTC ) ; 
myPreparedStatement.setObject( … , odt ) ;

And retrieval.

Instant instant = myResultSet.getObject( … , OffsetDateTime.class ).toInstant() ;

If using an older pre-4.2 JDBC driver, you can use toInstant and from methods to go back and forth between java.sql.Timestamp and java.time. These new conversion methods were added to the old legacy classes.

Table of date-time types in Java (both legacy and modern) and in standard SQL


About java.time

The java.time framework is built into Java 8 and later. These classes supplant the troublesome old legacy date-time classes such as java.util.Date, Calendar, & SimpleDateFormat.

The Joda-Time project, now in maintenance mode, advises migration to the java.time classes.

To learn more, see the Oracle Tutorial. And search Stack Overflow for many examples and explanations. Specification is JSR 310.

You may exchange java.time objects directly with your database. Use a JDBC driver compliant with JDBC 4.2 or later. No need for strings, no need for java.sql.* classes.

Where to obtain the java.time classes?

  • Java SE 8, Java SE 9, Java SE 10, and later

    • Built-in.
    • Part of the standard Java API with a bundled implementation.
    • Java 9 adds some minor features and fixes.
  • Java SE 6 and Java SE 7
    • Much of the java.time functionality is back-ported to Java 6 & 7 in ThreeTen-Backport.
  • Android
    • Later versions of Android bundle implementations of the java.time classes.
    • For earlier Android (<26), the ThreeTenABP project adapts ThreeTen-Backport (mentioned above). See How to use ThreeTenABP….

The ThreeTen-Extra project extends java.time with additional classes. This project is a proving ground for possible future additions to java.time. You may find some useful classes here such as Interval, YearWeek, YearQuarter, and more.

Parse timestamp string in Java with variable number of nanoseconds

You'd better use Java Time API1, from the package java.time.

Date, SimpleDateFormatter and Calendar classes are flawed and obsolete.

The DateTimeFormatter class provides numerous options, so you can configure all you need. Note that by using the method appendFraction, the nanos are right-padded.

String[] dateStrs = {
"2018-11-02 11:39:03.4-04",
"2018-11-02 11:45:22.71-04",
"2018-11-03 14:59:17.503-04"
};

DateTimeFormatter f = new DateTimeFormatterBuilder()
.appendPattern("yyyy-MM-dd HH:mm:ss.")
.appendFraction(ChronoField.NANO_OF_SECOND, 1, 9, false)
.appendPattern("X")
.toFormatter();

// Single item:
LocalDateTime date = LocalDateTime.parse("2018-11-02 11:39:03.7356562-04", f);

// Multiple items:
List<LocalDateTime> dates = Arrays.asList(dateStrs).stream()
.map(t -> LocalDateTime.parse(t, f))
.collect(Collectors.toList());

1 Java 8 new Date and Time API is heavily influenced by Joda Time. In fact the main author is Stephen Colebourne, the author of Joda Time.

Convert String to Date with Milliseconds

The Date class stores the time as milliseconds, and if you look into your date object you will see that it actually has a time of 1598515567413 milliseconds.

You are fooled by the System.out.println() which uses Date's toString() method. This method is using the "EEE MMM dd HH:mm:ss zzz yyyy" format to display the date and simply omits all milliseconds.

If you use your formatter, which has milliseconds in its format string, you will see that the milliseconds are correct:

System.out.println(formatter.format(dateFormatter));

outputs 2020-08-27T10:06:07.413

How to convert a string to a DatetimeWithNanoseconds format with python?

I learned that from a datetime format it is easy to extract hours for example just by calling date.hour (same for year, month, etc).

Knowing this, the way to convert a string to a DatetimeWithNanoseconds format takes these 2 easy steps:

  1. Convert the string to a datetime format:
date = '19551231'
date = datetime.datetime.strptime(date, '%Y%m%d')

  1. Convert to DatetimeWithNanoseconds:
nano = DatetimeWithNanoseconds(date.year, date.month, date.day, date.hour, date.minute, date.second, nanosecond=0)

Parsing datetime strings containing nanoseconds

You can see from the source that datetime objects don't support anything more fine than microseconds. As pointed out by Mike Pennington in the comments, this is likely because computer hardware clocks aren't nearly that precise. Wikipedia says that HPET has frequency "at least 10 MHz," which means one tick per 100 nanoseconds.

If you can live with throwing out the last three digits (which probably aren't too meaningful anyway), you could parse this by just slicing the input string to have only six digits after the decimal point and parsing with %f. Otherwise, it looks like you'll have to implement the subtraction yourself.


Much later update: numpy and pandas now each have (somewhat different) support for timestamps that includes the possibility of tracking nanoseconds, which are often good solutions. See the other answers for how.

Python 3.7+ also has time.time_ns and related functions in time (PEP 564), but still no support for nanoseconds in datetime.

using simpledateformat to parse string date with 6 digit of milliseconds

Instead of the long deprecated SimpleDateFormat and java.util.Date, you should use the new DateTimeFormatter and LocalDateTime in the java.time and java.time.format packages. If for no other reason than SimpleDateFormat does not offer nanosecond resolution (and your input appears to have nanoseconds).

Something like,

String date = "2019-07-04 00:32:08:627158";
DateTimeFormatter formatter = DateTimeFormatter.ofPattern(
"yyyy-MM-dd HH:m:ss:nnnnnn");
LocalDateTime ldt = LocalDateTime.parse(date, formatter);
DateTimeFormatter outFormatter = DateTimeFormatter.ofPattern(
"dd-MMM-yy hh.mm.ss.nnnnnn a");
System.out.println(outFormatter.format(ldt));

Outputs

04-Jul-19 12.32.08.627158 AM

If you want JUL add a toUpperCase() call, and if you need a literal extra three zeros add them in the outFormatter. Like,

DateTimeFormatter outFormatter = DateTimeFormatter.ofPattern(
"dd-MMM-yy hh.mm.ss.nnnnnn000 a");
System.out.println(outFormatter.format(ldt).toUpperCase());

Outputs

04-JUL-19 12.32.08.627158000 AM

It isn't clear where those three zeros came from, if you wanted more precision I would have used

DateTimeFormatter outFormatter = DateTimeFormatter.ofPattern(
"dd-MMM-yy hh.mm.ss.nnnnnnnnn a");
System.out.println(outFormatter.format(ldt).toUpperCase());

But that outputs (as I would expect)

04-JUL-19 12.32.08.000627158 AM


Related Topics



Leave a reply



Submit