[Improvement]: Separate local-optimizer and flink-optimizer into different modules and package them independently.

[baiyangtx]

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

Currently, the Optimizing job is a fat-jar that is applicable for both standalone mode and Flink job mode. This is not conducive to future expansion of more deployment methods and may cause conflicts during dependency packaging.

A typical example is that when deploying independently, Hadoop-related dependencies need to be packaged, but in Flink job mode, there may be conflicts with the Hadoop dependencies that come with the Flink engine.

Therefore, independent packaging for each type is a better choice.

How should we improve?

1. The optimizer module will be split into three sub-modules: optimizer/common, optimizer/jvm, and optimizer/flink.
2. The optimizer/jvm sub-module, as a fatjar, will include various dependencies such as Hadoop and AWS.
3. The optimizer/flink sub-module will only be used as a Flink job and will not include lower-level dependencies.
4. The optimizer/jvm and optimizer/flink sub-modules will depend on the optimizer/common sub-module. The logic related to optimizer/task execution will be implemented in this module.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• I agree to follow this project's Code of Conduct