Date and Time

The following notebook contains examples for the most important date and time functions in SQL:

Date and Time with SQL - Databrickss3.us-east-1.amazonaws.com

Date and time patterns in Spark 3.x

With Spark 3, new date time patterns were introduced. The current version is documented here:

Datetime patterns - Spark 4.0.1 Documentationspark.apache.org

You can always switch to the date time patterns from Spark 2.x with the following Scala command:

spark.conf.set("spark.sql.legacy.timeParserPolicy","LEGACY")