Also in v3.2.0: OTLP receiver concurrency caps, operator reconciles 4 pipelines in parallel, and full backpressure signals from every component to the control plane.
Before v3.1.0, when a downstream stage couldn't keep up, GlassFlow had no coordinated way to slow down. Events would pile up, NATS memory would fill, pipeline would fail. That's fixed now. 🧵
If you're sending #OTel data to ClickHouse without a processing layer in between, you're probably storing 15–30% duplicate logs and fighting nested JSON schema issues.
We built GlassFlow to fix this:dedup, enrichment, & schema mapping for #ClickHouse.
→ glassflow.dev/use-cases/real…
Still looking for a Founding Engineer 🚀🚀
Not a "senior dev" role. Not a backlog ticket role. A "here's the problem, let's build for it" role.
→ Low-latency #eventstreaming infra
→ Billions of events
→ GlassFlow stack: #Go, #Kafka, #NATS
Apply here: join.com/companies/getg…
Your Kafka → #ClickHouse pipeline shouldn't be a 2 AM nightmare.
ReplacingMergeTree doesn't deduplicate on insert. It merges "eventually", leading in messy count()s
GlassFlow fixes this by handling dedup, PII masking, and more upstream
➡️ Details: glassflow.dev#Kafka
Stop writing glue code to connect #OTel to #ClickHouse. 🛑
GlassFlow now ingests OTLP data natively: deduplicating, masking PII, and delivering traces/logs/metrics query-ready.
✅ No #Kafka
✅ No custom transforms
✅ Just clean data
See how: glassflow.dev/blog/clean-enr…#OpenTelemetry
ClickHouse for #observability is great, until you have to deal with:
• Duplicate spans from collector retries
• PII leaking into trace attributes
New guide: #OTel Collector → GlassFlow → #ClickHouse
Dedup, PII masking, tail sampling.
🔗 Link in replies
#OpenTelemetry
We just shipped native #OTel ingestion in GlassFlow. 🚀
The OTel Collector exports. ClickHouse stores. Nobody handles what happens in between.
Now GlassFlow does: deduplication, enrichment, schema mapping, before the first row hits your DB.
glassflow.dev/blog/clean-enr…#OpenTelemetry
Moving to #ClickHouse? Don't let denormalization slow you down!
While it’s a standard hack for row-based DBs, in ClickHouse it causes:
📦 Bloated storage
✍️ Write amplification
⏳ Ingestion friction
The solution?
glassflow.dev/blog/denormali…#DataEngineering
Backfilling PB-scale #AI data is usually where the "standard" approach breaks.
If you're managing real-time embeddings or feature stores in #ClickHouse, you need a strategy that won't kill your production uptime.
Here\s how to handle the switch:
glassflow.dev/blog/clickhous…#AIInfra
ICYMI: GlassFlow just broke the 500k events/sec barrier. 🚀
We’ve achieved linear horizontal scaling within a single pipeline. No architectural rework, no complexity—just add resources and go.
Read the benchmark breakdown:
glassflow.dev/blog/glassflow…#DataEngineering#ClickHouse
301 Followers 2K FollowingA #DataScience enthusiast who uses #MachineLearning to train models and generate #PredictiveAnalytics. Fascinated by all things #Data - the #science & #art🛴
4K Followers 4K Followingsde 3 @maersk | gsoc | gsod | lfx | icpc regionalist | just sharing whats on my mind this is my life the wins the mess the mindset the data opinions mine alone
330 Followers 967 Following🔮Quest to Cloud Native computing ☁⚓☸
https://t.co/oJOt4LoRBp and https://t.co/HKm4MKzMrv
Retweets, likes and follow do not mean endorsement. Opinions are my own
3K Followers 2K FollowingGP @UpfrontVC, focused on healthcare, gaming (check out The Gaming Founders Podcast), dev tools & infra. Appreciate co-op game recs.
198 Followers 2K FollowingPronouns: Don't care about mine, sure as hell will respect yours.
Berlin-based software developer. Loving: Code, learning, reinforcement learning...
Pro Choice.
394 Followers 41 Following🥑 Developer Advocate 🧑💻 Software Engineer 🧑🏫 Speaker
Follow me to learn more about AI, Data Engineering, and stream processing tools.
330 Followers 967 Following🔮Quest to Cloud Native computing ☁⚓☸
https://t.co/oJOt4LoRBp and https://t.co/HKm4MKzMrv
Retweets, likes and follow do not mean endorsement. Opinions are my own
652K Followers 113 FollowingLets Learn #Python with tips and tricks. Free Python Course: https://t.co/l9NKxZWrh7 biz : [email protected] AI Community Partner. DM for Everything.