[Improvement]: Support list partitions via flink catalog #1940
Assignees
Labels
Comments
YesOrNo828
added
type:improvement
module:mixed-flink
|
I can complete this part of the work, but from my practice, listing all partitions will definitely scan all metadata. For some large tables, this may take a long time, which will put a certain amount of pressure on the flinkclient that runs the statement. |
@huyuanfeng2018 Thanks for volunteering to fix it. I've assigned it to you. I think your consideration is reasonable, and we also need to scan two tables including the change and base table, if the mixed-format table is a Keyed Table. |
3 tasks
czy006
added a commit
to czy006/amoro
that referenced
this issue
Oct 11, 2023
zhoujinsong
added a commit
that referenced
this issue
Oct 11, 2023
* [AMORO-1983][Feature] Copy From Flink 1.15 Code to Flink 1.17 * [AMORO-1983][Feature] Flink 1.17 API compatible * [AMORO-1983][Feature] Copy From Flink 1.15 Test Code to Flink 1.17 Test Code * [AMORO-1983][fix] fix package rename to planner * [AMORO-1983][Feature] Flink 1.17 Mode Test Fix * [AMORO-1983][Feature] fix pom * [AMORO-1983] FlinkSplitPlanner Remove SequenceNumber * [AMORO-1983] TestUnkeyed Replace to KafkaContainer Test * Add KafkaContainerTest CountAllRecords * Remove KafkaTestBase * spotless code * fix TestCatalog use default after drop catalog * fix LookupITCase set streaming true * fix checkstyle * fix test timeout thread commit failed * remove bridge to public common * fix pom problem * patch #1940 --------- Co-authored-by: ZhouJinsong <[email protected]>
ShawHee
pushed a commit
to ShawHee/arctic
that referenced
this issue
Dec 29, 2023
…#2001) * [AMORO-1983][Feature] Copy From Flink 1.15 Code to Flink 1.17 * [AMORO-1983][Feature] Flink 1.17 API compatible * [AMORO-1983][Feature] Copy From Flink 1.15 Test Code to Flink 1.17 Test Code * [AMORO-1983][fix] fix package rename to planner * [AMORO-1983][Feature] Flink 1.17 Mode Test Fix * [AMORO-1983][Feature] fix pom * [AMORO-1983] FlinkSplitPlanner Remove SequenceNumber * [AMORO-1983] TestUnkeyed Replace to KafkaContainer Test * Add KafkaContainerTest CountAllRecords * Remove KafkaTestBase * spotless code * fix TestCatalog use default after drop catalog * fix LookupITCase set streaming true * fix checkstyle * fix test timeout thread commit failed * remove bridge to public common * fix pom problem * patch apache#1940 --------- Co-authored-by: ZhouJinsong <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Search before asking
What would you like to be improved?
Now the flink catalog is not implemented
public List<CatalogPartitionSpec> listPartitions(ObjectPath tablePath)interface; soSHOW PARTITIONSstatement is not supported.How should we improve?
No response
Are you willing to submit PR?
Subtasks
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: