I don't know
python
. But
withColumn
is a function so you're missing something before that. Probably you need to
query. before first
withColumn
. So make required changes.
query = query.withColumn("contains_km", col("def").rlike("2km")).withColumn("ts", to_timestamp(col("timestamp"))).withColumn("week_num", weekofyear(col("ts"))).withColumn("month", month(col("ts"))).withColumn("year", year(col("ts"))).withColumn("hour", hour(col("ts"))).groupby("id", "contains_km", "month", "def", "ts").agg(count("contains_km").alias("countresult")).select("id", "def", "countresult", "month", "contains_km", "ts")