如何在 PySpark 中转换数据帧列中的 JSON 字符串？

我有一个 pyspark 数据框，如下所示

+--------------------+---+
|            _c0|_c1|
+--------------------+---+
|{"object":"F...|  0|
|{"object":"F...|  1|
|{"object":"F...|  2|
|{"object":"E...|  3|
|{"object":"F...|  4|
|{"object":"F...|  5|
|{"object":"F...|  6|
|{"object":"S...|  7|
|{"object":"F...|  8|

专栏_c0包含字典形式的字符串。

'{"object":"F","time":"2019-07-18T15:08:16.143Z","values":[0.22124142944812775,0.2147877812385559,0.16713131964206696,0.3102800250053406,0.31872493028640747,0.3366488814353943,0.25324496626853943,0.14537988603115082,0.12684473395347595,0.13864757120609283,0.15222792327404022,0.238663449883461,0.22896413505077362,0.237777978181839]}'

如何将上面的字符串转换为字典形式并获取每个键值对并将其存储到变量中？我不想将其转换为 pandas，因为它很昂贵。

您应该使用 Scala 的 Spark API 的等效项数据集.withColumn http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Dataset and 来自_json http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.functions%24标准功能。