LiquidAI releases Pipette, an evaluation suite for on-device AI models
JosephJacks_ · x · 2026-08-25
LiquidAI, in partnership with Artificial Analysis, has released Pipette, an open-source model evaluation suite tailored for on-device intelligence.
Key Features::
- On-device Focus: Unlike cloud-optimized platforms, Pipette provides a common, reproducible way to measure the quality, speed, latency, and memory usage of AI models on devices like phones, laptops, and embedded hardware.
- Holistic Comparison: Allows comparing models as a combination of model + quantization + runtime + device from a single interface.
- Results Warehouse: Includes a repository of verified benchmark results, currently with over 10k results spanning 35 model classes, 7 quantization types, and llama.cpp runtimes.
More from Infra
- Nvidia MGX Power Rack session packed to standing room on final OCP day — firstadopter · 2026-10-16
- Intel hosts 816B-param Muse Spark Open air-cooled on four 160GB D-series GPUs — BenBajarin · 2026-10-16
- Running an 816B-parameter model on two Mac Studios: $40k instead of $500k of compute — charles_irl · 2026-10-16
- Filecoin marks 6 years of mainnet, pivots storage stack to AI workloads with new S3 offering — juanbenet · 2026-10-16
- PrismML ships Bonsai 2 ternary models: 9B fits in 2GB for local AI — gajesh · 2026-10-16
- Artificial Analysis to add 4-bit formats and Splash Metal engine to agent benchmark — ArtificialAnlys · 2026-10-16