Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMORO-2032]: Spark3.2 version does not require the class ResolveProcedures #2035

Merged
merged 1 commit into from
Sep 26, 2023

Conversation

bdxmq
Copy link
Contributor

@bdxmq bdxmq commented Sep 26, 2023

Why are the changes needed?

When using amoro spark-3.2-runtime-0.5.0.jar, an error was reported and the class ResolveProcedures could not be found. Through comparison, it was found that changing the class is no longer necessary. Simply remove the dependency of this class.
Close #2032 .

Brief change log

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? ( no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@github-actions github-actions bot added the module:mixed-spark Spark module for Mixed Format label Sep 26, 2023
@wangtaohz wangtaohz requested a review from baiyangtx September 26, 2023 02:18
@codecov
Copy link

codecov bot commented Sep 26, 2023

Codecov Report

All modified lines are covered by tests ✅

see 3 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

@baiyangtx
Copy link
Contributor

Can you provide a screenshot of using the packaged runtime.jar in the Spark engine? It's because unit tests for the Spark module don't test whether shade is reasonable or not.

@wangtaohz
Copy link
Contributor

Thanks for your contribution! 👍 @bdxmq

It looks like a miss for Spark 3.2.

If you verify it successfully in your enviromnent, we can merge it and consider including it in version 0.5.1.

@bdxmq
Copy link
Contributor Author

bdxmq commented Sep 26, 2023

I removed this line of code locally, packaged it, and used Spark SQL to use queries normally
image

Copy link
Contributor

@wangtaohz wangtaohz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@baiyangtx baiyangtx changed the title [WIP][AMORO-2032]fix:Spark3.2 version does not require the class ResolveProcedures [AMORO-2032]: Spark3.2 version does not require the class ResolveProcedures Sep 26, 2023
Copy link
Contributor

@baiyangtx baiyangtx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for you contribution.

@baiyangtx baiyangtx merged commit 30ce7c0 into apache:master Sep 26, 2023
wangtaohz pushed a commit that referenced this pull request Sep 26, 2023
…edures (#2035)

fix:Spark3.2 version does not require the class ResolveProcedures
ShawHee pushed a commit to ShawHee/arctic that referenced this pull request Dec 29, 2023
…edures (apache#2035)

fix:Spark3.2 version does not require the class ResolveProcedures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:mixed-spark Spark module for Mixed Format
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Spark SQL using amoro spark runtime error class cannot find ResolveProcedures
3 participants