Archive - Data Engineering ACID

Embeddings Exposed, Kafka in 300 Lines, Redundant SQL, and Lakehouse Costs Breakdown

Deep dives into vectors, minimalist Kafka builds, primary keys, SQL as a language, and transparent lakehouse pricing,

Aug 1 •

Data Engineering ACID | e6data

July 2025

LLMs Stall, Queries Spill, and e6data Bridges Delta, Hudi, Iceberg & Polaris

Tactics to tame runaway memory, curb Bronze disasters, refactor tests,& AI drama. Learn how to build a modern data pipeline in Snowflake and our product…

Jul 25 •

Data Engineering ACID | e6data

How to Rust CGP, Iceberg's the wrong spec?, Spark's testimony, Froid-inspired UDF engine, and more

Today we talk about Rust again, why Iceberg might have a metadata issue, Spark's future, testing frameworks, and building a better UDF-engine inspired…

Jul 18 •

Data Engineering ACID | e6data

How to Rust, Cursor's vector search dump, Supabase MCP's leak (?), Apache Arrow Summit, and more

Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…

Jul 11 •

Data Engineering ACID | e6data

Agent-Built JUnit Suites, CDC vs Daily Snapshots, Ducklake vs Iceberg, Queries, Costs, and more

Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…

Jul 4 •

Data Engineering ACID | e6data

June 2025

How to check Snowflake costs, My slowing Spark pipeline, IndiCS CFPs, NL2SQL MCP, and more

Every Friday, we deliver your weekend win: copy-paste tutorial, cost-optimisation technique, CFPs worth your pitch, and fresh ideas from the field. Stop…

Jun 27 •

Data Engineering ACID | e6data

Deep-Fake Interviews, ETL Bottlenecks, and Shift-Left Showdowns

Your weekly sweep of top 3 trends and hacks in data engineering by leading tech companies a peek at what we’re building at e6data.

Jun 20 •

Data Engineering ACID | e6data

Spark’s Crossroads, Polars Momentum, and YouTube’s Guardrails for Petabytes

Your weekly sweep of top 3 trends and hacks in data engineering by leading tech companies a peek at what we’re building at e6data.

Jun 13 •

Data Engineering ACID | e6data

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts