Miller
A command-line tool that does to CSV, TSV and JSON what awk and sed do to plain text — filter, reshape and transform tabular data without spinning up a Python script.
🔗 Visit Miller
Description
The classic Unix tools awk, sed and cut are fast and universally available, but they treat everything as raw text with no idea what a column header is, which makes them clumsy for structured data like CSV or JSON. Writing a small pandas script instead works, but it's overkill for a quick transformation and pulls in Python plus its dependencies just to reshape a file. Miller sits in between: it's a single, fast command-line binary that understands CSV, TSV and JSON structure natively, so operations like filtering, sorting and joining just work with column names instead of fragile positional indexes.
Miller supports multi-format processing (CSV, TSV, JSON, JSON Lines) with named-field access, format conversion between them, filtering, sorting, aggregation and streaming processing that can handle files larger than available RAM using single-pass algorithms. It ships as a single portable Go binary with zero runtime dependencies. It's free, open source, mature (10,000+ GitHub stars, active commits as recently as this week) and documented at miller.readthedocs.io.
💬 Our review
The short version: if you're stitching together awk, sed, cut and a bit of Python just to filter and reshape a CSV or JSON file, Miller replaces that whole chain with one tool that actually understands column headers and record structure.
What separates Miller from classic Unix text tools is format awareness: instead of treating a CSV as lines of comma-separated text, it operates on named fields, so a filter or sort just references a column name rather than a fragile positional index that breaks if a column gets added. Against pandas or R, Miller's advantage is deployment simplicity — it's a single dependency-free binary you can drop on any server or CI pipeline, with a streaming architecture that can process files bigger than RAM, something pandas struggles with by default. The tradeoff is Miller's own domain-specific language has a real learning curve, and it's not a substitute for a full data-analysis environment when you need statistical modeling, visualization or the breadth of pandas' ecosystem. Excellent for DevOps engineers, data engineers and anyone doing ETL, log processing or database post-processing from the command line; less suited to exploratory data analysis or work needing a rich plotting/statistics ecosystem.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Projet open source, aucune offre payante
Pros
Comprend nativement la structure CSV/TSV/JSON (colonnes nommées, pas d'index fragile)
Binaire unique sans dépendance, portable sur tout serveur ou pipeline CI
Architecture en streaming, traite des fichiers plus gros que la RAM disponible
Combine les usages d'awk, sed, cut, join et sort en un seul outil
Développement actif (10k+ étoiles, commits récents)
Cons
Langage propre à Miller (DSL) avec une vraie courbe d'apprentissage
Moins riche qu'un environnement complet comme pandas ou R pour l'analyse exploratoire
Pas de calcul distribué natif, traitement sur une seule machine