query data with spark paging. Ordinary sql () does not support paging sql statements.
it is said that you can add a sequence to realize
, but basically it is scala
/ / add a list of "id" information to the original Schema information
val schema: StructType = dataframe.schema.add(StructField("id", LongType))
// DataFrameRDD zipWithIndex
val dfRDD: RDD[(Row, Long)] = dataframe.rdd.zipWithIndex()
val rowRDD: RDD[Row] = dfRDD.map(tp => Row.merge(tp._1, Row(tp._2)))
// RDD DataFrame
val df2 = spark.createDataFrame(rowRDD, schema)
df2.show()
< hr >
how to implement it with java