[SEDONA-713] add OSM PBF reader

[Imbruced]

Did you read the Contributor Guide?

• Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

• Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-713. The PR name follows the format [SEDONA-713] my subject.

• No:

  • this is a documentation update. The PR name follows the format [DOCS] my subject
  • this is a CI update. The PR name follows the format [CI] my subject

What changes were proposed in this PR?

How was this patch tested?

Did this PR include necessary documentation updates?

• Yes, I am adding a new API. I am using the current SNAPSHOT version number in vX.Y.Z format.
• Yes, I have updated the documentation.
• No, this PR does not affect any public API so no need to change the documentation.

[jiayuasu]

Why is the code put in spark-3.5 folder? instead of common? Does it only work with spark-3.5?

[Imbruced]

Hi, I added an example to discuss the correctness of the code. We need to either copy the code for spark 3.4 and 3.3 or abstract the

  val path = new Path(new URI(file.filePath.toString()))

As I think it might fail for spark 3.4 and 3.3 due to this, I need to confirm it

[jiayuasu]

@zhangfengcdt @Kontinuation I think we probably ran into this val path = new Path(new URI(file.filePath.toString())) issue?

[zhangfengcdt]

@zhangfengcdt @Kontinuation I think we probably ran into this val path = new Path(new URI(file.filePath.toString())) issue?

I think 3.3 and >3.4.0 should be good bacause the changes are:

https://javadoc.io/static/org.apache.spark/spark-sql_2.12/3.3.2/org/apache/spark/sql/execution/datasources/PartitionedFile.html
https://javadoc.io/static/org.apache.spark/spark-sql_2.12/3.5.0/org/apache/spark/sql/execution/datasources/PartitionedFile.html

The filePath.toString() shild all return the current URI string.

[jiayuasu]

If the same OSM reader code works for all versions (i.e., in spark/common), we should move the test to spark/common as well.

[Imbruced]

yeah let's what happens :D

[Imbruced]

@jiayuasu looks like everything passed