Skip to content
View kszucs's full-sized avatar
😊
😊

Organizations

@apache @conda-forge @daskos @huggingface @ibis-project @datafusion-contrib

Block or report kszucs

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
kszucs/README.md

Krisztián Szűcs

For over a decade I've built the open-source infrastructure that data runs on — Apache Arrow, Parquet, Iceberg — with a focus on moving and storing data more efficiently.

Open Source Engineer at Hugging Face and an Apache Arrow PMC member. I'm one of the authors of Ibis, an Apache OpenDAL committer, and a regular contributor across the data ecosystem — hundreds of merged pull requests to Arrow, DataFusion, Iceberg, and more.

Some things I've built

  • Apache Arrow — years of core work as a PMC member: the cross-platform CI/CD that ships packages at 100M+ downloads/month, PyArrow, and the Arrow C++ internals much of the columnar ecosystem is built on.
  • Ibis — one of the authors of the portable Python dataframe library: its core, plus koerce, the pattern-matching and rewrite engine underneath it.
  • marrow — a ground-up Apache Arrow implementation in Mojo.
  • Parquet Content-Defined Chunking — a technique I designed and shipped: placing Parquet page boundaries by content instead of byte offset, so a barely-changed file re-uploads only the chunks that moved. It landed in PyArrow and pandas and powers Xet content-addressable storage, cutting data transfer by 90%+ in common rewrite patterns. → writeup

Across the ecosystem · Arrow · arrow-rs · DataFusion · iceberg-rust · OpenDAL · Ibis · xet-core

Elsewhere · kszucs.dev · LinkedIn · X @kszucs_

Pinned Loading

  1. apache/arrow apache/arrow Public

    Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

    C++ 16.9k 4.2k

  2. ibis-project/ibis ibis-project/ibis Public

    the portable Python dataframe library

    Python 6.6k 736

  3. marrow marrow Public

    Arrow implementation in Mojo

    Mojo 89 5

  4. koerce koerce Public

    Reusable Pattern Matching on Python Objects

    Python 20 2

  5. lensacom/sparkit-learn lensacom/sparkit-learn Public

    PySpark + Scikit-learn = Sparkit-learn

    Python 1.2k 254

  6. pandahouse pandahouse Public

    Pandas interface for Clickhouse database

    Python 239 68