Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow non offset based timezones #1078

Open
Fokko opened this issue Mar 12, 2025 · 1 comment
Open

Allow non offset based timezones #1078

Fokko opened this issue Mar 12, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@Fokko
Copy link
Contributor

Fokko commented Mar 12, 2025

Apache Iceberg Rust version

None

Describe the bug

I was trying to make PyIceberg rely solely on Iceberg-rust for the partition transforms, but I ran into the following:

    def _transform(array: "ArrayLike") -> "ArrayLike":
        if isinstance(array, pa.Array):
            return transform_func(array, *args)
        elif isinstance(array, pa.ChunkedArray):
            result_chunks = []
            for arr in array.iterchunks():
>               result_chunks.append(transform_func(arr, *args))
E               ValueError: Unexpected => Parser error: Invalid timezone "UTC": only offset based timezones supported without chrono-tz feature

We initialize the datetime as:

datetime(2023, 12, 31, 0, 0, 0, tzinfo=timezone.utc)

To Reproduce

In test_transform.py:

def test_year_transform_datetime():
    arr = pa.array([
        datetime(1970, 1, 1, 0, 0, 0, tzinfo=timezone.utc),
        datetime(2000, 1, 1, 0, 0, 0, tzinfo=timezone.utc)
    ])
    result = transform.year(arr)
    expected = pa.array([0, 30], type=pa.int32())
    assert result == expected

Expected behavior

Able to just parse these. I'm not sure if we need to fix this in Arrow, or if we can import chrono-tz package.

Willingness to contribute

I cannot contribute a fix for this bug at this time

@liurenjie1024
Copy link
Contributor

Seems this is cause by arrow, see https://github.com/apache/arrow-rs/blob/d5339f31a60a4bd8a4256e7120fe32603249d88e/Cargo.toml#L98C51-L98C56 .

The temporal transforms are using arrow to do the transform:

let array = date_part(&input, DatePart::Year)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants