Check out example codes for "spark write partitionby". It will help you in understanding the concepts better.

Code Example 1

import spark.implicits._
df.repartition($"entity", $"year", $"month", $"day", $"status").write.partitionBy("entity", "year", "month", "day", "status").mode(SaveMode.Append).parquet(s"$location")

Learn ReactJs, React Native from akashmittal.com