0.3.0

What's Changed

• add python ci by @crwen in #201
• chore bump to 0.2.0 by @ethe in #203
• fix: resolve correct dep by @ethe in #204
• refactor: update fusio & use DynWrite in wal instead by @ethe in #210
• chore: detach SnapShot from Transaction by @KKould in #207
• Minor refactor: replace Ulid to FileId by @LMJW in #221
• Isolate range param lifetime from scan lifetime by @LMJW in #220
• make tonbo build under wasm32 arch by @crwen in #216
• refactor: replace all &FileId to FileId by @sollhui in #222
• chore: fix dead link in README.md by @sollhui in #224
• test: add benchmark for tonbo on s3 by @KKould in #226
• feat: support parquet lru reader by @ethe in #215
• chore: remove parquet default features required by @ethe in #229
• chore: make ra happy by @ethe in #231
• chore: new issue template by @ethe in #232
• fix: issue template error by @ethe in #233
• chore: rename record::str to record::test by @ethe in #236
• fix transaction projection on mutable by @crwen in #235
• refactor: remove needless unsafe Send Sync implementations by @ethe in #237
• chore: replace FileId::new by @ClSlaid in #245
• chore(deps): update pprof requirement from 0.13 to 0.14 by @dependabot in #244
• chore(deps): update datafusion requirement from 42 to 43 by @dependabot in #243
• add s3 test by @crwen in #248
• flush all in-memory table by @crwen in #246
• fix: wrong delete gen level by @crwen in #250
• fix: change TokioExecutor::default to TokioExecutor::current by @ethe in #252
• refactor: seperate schema trait from record by @KKould in #241
• feat: re-export fusio::path::Path by @ethe in #256
• feat: derive Clone on TokioExecutor by @ethe in #257
• chore: bump fusio to 0.3.4 by @crwen in #263
• refactor: remove reduant boxing in trigger by @ethe in #253
• docs: init user documentation by @crwen in #254
• add doc ci by @crwen in #267
• refactor: replace log with fusio-log by @crwen in #266
• Refactor: remove log in VersionInner by @crwen in #269
• feat: support destroy by @crwen in #271
• fix: wrong primary key index by @crwen in #272
• chore:bump arrow and parquet to 54 by @crwen in #274
• chore(deps): update rocksdb requirement from 0.22 to 0.23 by @dependabot in #260
• chore(deps): update foyer requirement from 0.12.2 to 0.14.1 by @dependabot in #275
• chore: export array in record module by @crwen in #276
• refactor: rename Datatype to DataType by @ethe in #278
• refactor: rename InternalRecordRef to OptionRecordRef by @ethe in #277
• fix: replace null with default in record builder by @crwen in #279
• chore: remove unused crates by @crwen in #280
• refactor: define Value with ValueDesc by @crwen in #281
• feat: wasm bindings by @crwen in #249
• docs: fix a typo by @SteveLauC in #286
• fix: compile error by @crwen in #285
• chore: bump to 0.3.0 by @ethe in #284
• chore: bump fusio-log and fusio-dispatch by @ethe in #289
• chore: bump parquet-lru to 0.3.0 by @ethe in #290

New Contributors

• @LMJW made their first contribution in #221
• @sollhui made their first contribution in #222
• @ClSlaid made their first contribution in #245
• @SteveLauC made their first contribution in #286

Full Changelog: 0.2.0...0.3.0

Tonbo release 0.2.0

Tonbo Now Supports S3 as Remote Storage

With the integration of Fusio, Tonbo now supports storing data on S3, making cloud storage integration easier and reducing costs. Users can store either parts of stale data or the full dataset on S3. We believe object storage is key for the next generation of data systems, and supporting S3 is crucial for Tonbo's goal of providing a unified storage solution for both local and cloud environments, enabling databases like SQLite and PostgreSQL to act as stateless query engines on top of it. Configuration example:

let options = DbOption::from(Path::from_filesystem_path("./db_path/users").unwrap())
    // tonbo only stores 3 and beyond level stale data of lsm tree on S3
    // 1 and 2 level fresh data are on local disk as default
    .level_path(
        3,
        "/remote-storage".into(),
        FsOptions::S3 {
            bucket: "bucket".into(),
            credential: None,
            region: None,
            sign_payload: None,
            checksum: None,
        },
    )
    .unwrap();
let db = DB::new(options, TokioExecutor::default()).await.unwrap();

Python Binding Now Supported

Starting from version 0.2.0, Tonbo can be used in Python with minimal setup. Given Python's wide adoption in scientific computing, AI, and data analysis, this support helps developers easily build data-intensive applications for local/edge-first environments.

The Python binding offers an ORM-like experience, simplifying data modeling and reducing boilerplate code, allowing developers to focus on building features. Quick preview:

from tonbo import DbOption, Column, DataType, Record, TonboDB, Bound
import asyncio
tempfile

@Record
class User:
    id = Column(DataType.Int64, name="id", primary_key=True)
    age = Column(DataType.Int16, name="age", nullable=True)
    name = Column(DataType.String, name="name", nullable=False)
    email = Column(DataType.String, name="email", nullable=True)
    data = Column(DataType.Bytes, name="data", nullable=True)


async def main():
    temp_dir = tempfile.TemporaryDirectory()

    db = TonboDB(DbOption(temp_dir.name), User())
    await db.insert(User(id=18, age=175, name="Alice"))

    record = await db.get(18)
    assert record == {
        "id": 18,
        "age": 175,
        "name": "Alice",
        "email": None,
        "data": None,
    }

    txn = await db.transaction()

    txn.insert(
        User(
            id=19,
            age=195,
            name="Bob",
            data=b"Hello Tonbo!",
            email="[email protected]",
        )
    )

    await txn.commit()
    txn = await db.transaction()
    scan = await txn.scan(
        Bound.Excluded(18),
        None,
        limit=100,
        projection=["id", "email", "data"],
    )
    async for record in scan:
        assert record["age"] is None
        print(record)
    await txn.commit()


asyncio.run(main())