vaex

Vaex Overview Vaex is a high-performance Python library designed for lazy, out-of-core DataFrames to process and visualize tabular datasets that are too large to fit into RAM. Vaex can process over a billion rows per second, enabling interactive data exploration and analysis on datasets with billions of rows. When to Use This Skill Use Vaex when: Processing tabular datasets larger than available RAM (gigabytes to terabytes) Performing fast statistical aggregations on massive datasets Creating visualizations and heatmaps of large datasets Building machine learning pipelines on big data Converting between data formats (CSV, HDF5, Arrow, Parquet) Needing lazy evaluation and virtual columns to avoid memory overhead Working with astronomical data, financial time series, or other large-scale scientific datasets Core Capabilities Show more