Data Engineering ACID
Subscribe
Sign in
Home
Notes
Archive
About
Latest
Top
XTX's 500PB Open Source Lessons, Enterprise RAG Reality, Cachey, and the Art of Performance Engineering
When algorithmic trading meets open source, RAG systems hit enterprise reality, and custom optimization beats general-purpose excellence.
Sep 19
•
Data Engineering ACID | e6data
4
Rust's Perf Survey'25, AI eating code, Apache Wayang, and Data Engineering's Wall Street Moment
When performance matters more than preferences: the evolving reality of data engineering work with AI.
Sep 12
•
Data Engineering ACID | e6data
Fluss Fast-Tracks, Rust's Learning Gains, Parquet's Duality, Kafka-Iceberg Paths, and Snowflake Costs
This week we explore the latest emerging formats and projects in data engineering with their real-life implications at scale.
Sep 5
•
Data Engineering ACID | e6data
1
Microsoft Fabric SQL Mirroring, BI Chaos, Medallion Architecture, Wide Tables for Warehouses, and AI for Enterprises
This week's reality checks from the data engineering frontlines on Text-to-SQL, wide tables, Medallion architecture, Microsoft Fabric, and more (plus…
Sep 2
•
Data Engineering ACID | e6data
August 2025
1.4T-Event Spotify Dashboard Machine, Perfect Query Plans, Databricks Pro Playbook, and Agents for Data Analysts
Real-world experience to ace Databricks DE Pro, How Spotify handles their dashboards, resilient scraping tactics, why “optimal” query plans mislead, and…
Aug 22
•
Data Engineering ACID | e6data
1
Cursor & GPT-5 took over our newsletter!
Skewed joins are quiet cost multipliers. One hot key creates massive shuffle imbalance, long-tail tasks, and 2–5x cost. Here's how to fix it.
Aug 8
•
Data Engineering ACID | e6data
1
Embeddings Exposed, Kafka in 300 Lines, Redundant SQL, and Lakehouse Costs Breakdown
Deep dives into vectors, minimalist Kafka builds, primary keys, SQL as a language, and transparent lakehouse pricing,
Aug 1
•
Data Engineering ACID | e6data
July 2025
LLMs Stall, Queries Spill, and e6data Bridges Delta, Hudi, Iceberg & Polaris
Tactics to tame runaway memory, curb Bronze disasters, refactor tests,& AI drama. Learn how to build a modern data pipeline in Snowflake and our product…
Jul 25
•
Data Engineering ACID | e6data
How to Rust CGP, Iceberg's the wrong spec?, Spark's testimony, Froid-inspired UDF engine, and more
Today we talk about Rust again, why Iceberg might have a metadata issue, Spark's future, testing frameworks, and building a better UDF-engine inspired…
Jul 18
•
Data Engineering ACID | e6data
How to Rust, Cursor's vector search dump, Supabase MCP's leak (?), Apache Arrow Summit, and more
Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…
Jul 11
•
Data Engineering ACID | e6data
Agent-Built JUnit Suites, CDC vs Daily Snapshots, Ducklake vs Iceberg, Queries, Costs, and more
Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…
Jul 4
•
Data Engineering ACID | e6data
1
June 2025
How to check Snowflake costs, My slowing Spark pipeline, IndiCS CFPs, NL2SQL MCP, and more
Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…
Jun 27
•
Data Engineering ACID | e6data
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
