Kumar Sowri Banglore @kbanglore

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better: Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better: Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better: Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work! In summary: - As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding. - Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

This is INSANE. Boris Cherny, the head of Claude Code just revealed his exact workflow. This is a goldmine, must repost & bookmark:

You’re in a backend interview. They ask: “Design a globally distributed configuration propagation service that pushes config updates to tens of thousands of servers within seconds, with versioning, rollback, and strong delivery guarantees.” Here’s how to approach it:

📝 Blogged: "On Idempotency Keys" Discussing several options for ensuring exactly-once processing in distributed systems using idempotency keys, from UUIDs to monotonically increasing sequences. 👉 morling.dev/blog/on-idempo…

What Does Write Skew Look Like? justinjaffray.com/what-does-writ… This post is about gaining intuition for Write Skew, and, by extension, Snapshot Isolation. Snapshot Isolation is billed as a transaction isolation level that offers a good mix between performance and correctness.

API Authorization Frameworks Controlling What Users Are Allowed to Do in Your API 1. Role-Based Access Control (RBAC) → Definition ✓ Access is granted based on predefined roles such as admin, editor, or viewer. → How It Works ✓ Users are assigned roles. ✓ Roles determine permissions. → Use Cases ✓ Admin dashboards ✓ Content management systems ✓ SaaS platforms with tiered access 2. Attribute-Based Access Control (ABAC) → Definition ✓ Permissions are granted based on user attributes, resource attributes, and environment conditions. → Examples of Attributes ✓ User: age, department, subscription ✓ Resource: sensitivity, owner ✓ Environment: time, location → Use Cases ✓ Enterprise systems ✓ Highly dynamic permissions ✓ Large organizations with complex structures 3. OAuth 2.0 Authorization Framework → Definition ✓ Enables third-party apps to access resources without sharing user passwords. → Key Concept ✓ Authorization is delegated using access tokens. → Flows ✓ Authorization Code ✓ Client Credentials ✓ Device Authorization → Use Cases ✓ Social logins ✓ Payment integrations ✓ Enterprise apps 4. OpenID Connect (OIDC) → Definition ✓ An identity layer built on top of OAuth 2.0. ✓ Adds user authentication to OAuth. → What It Provides ✓ Verified user identity ✓ Standardized ID token → Use Cases ✓ Single Sign-On (SSO) ✓ Identity federation 5. JSON Web Token (JWT) Authorization → Definition ✓ Authorization through signed tokens containing user roles or permissions. → How It Works ✓ User logs in → Server issues JWT ✓ JWT includes claims about user authorization → Use Cases ✓ Microservices ✓ Mobile and web applications ✓ Stateless API authorization 6. Policy-Based Access Control (PBAC) → Definition ✓ Uses centralized policies to determine access. → How It Works ✓ Policies define who can do what ✓ Enforcement points check policies before granting access → Use Cases ✓ Government systems ✓ Compliance-heavy industries 7. Access Control Lists (ACLs) → Definition ✓ Permissions are attached specifically to resources. → How It Works ✓ Each resource lists who is allowed to access it and what actions they can perform. → Use Cases ✓ File systems ✓ Resource-based permission models 8. Tip ✓ RBAC → Role-driven access ✓ ABAC → Attribute-driven access ✓ OAuth 2.0 → Delegated authorization ✓ OIDC → Identity + OAuth ✓ JWT → Stateless authorization ✓ PBAC → Policy-driven ✓ ACLs → Resource-specific permissions →Grab the API Mastery Ebook: codewithdhanian.gumroad.com/l/vrzagk

Sep, the world's fastest #dotnet csv parser, infographic by Nano Banana Pro 🍌🚀 github.com/nietras/Sep PS: Take info with grain of salt, read the readme for accurate info.

Understanding the C4 Model 👌 alirezafarokhi.medium.com/understanding-… c4 by @simonbrown is my fav diagramming approach❤️

📺 "Distributed Systems 6.2: Raft" Enjoyed watching this intro to the Raft consensus algorithm, part of a larger DistSys lecture series by Martin Kleppmann. The pseudo-code makes it actually digestible really well. youtube.com/watch?v=uXEYuD…

Sharing the best DSA resources for free in one place 👇 A free, well organized collection of top DSA materials for both beginners and advanced learners who want to level up without spending a single rupee. From clear roadmaps to practice sets and real interview questions, everything you need to master DSA and prepare confidently is right here. To get the doc link ✅ 1- Follow( so that i can dm you) 2- Like & Repost to help others 3- Reply "DSA" ✅ Note: I’ll add the link later in the thread, so you can grab it from there!

Most people don’t understand the difference between Message Queues and Pub Sub. They are different systems meant for different use cases powered by different data structures. Pub-Sub systems like Kafka are powered by the Log data structure. Message Queues like RabbitMQ and SQS are powered by... the Queue. 🔶 The Queue data structure is well known to most engineers. You append items to the end and pop items off of front. The key word is pop - once an item is popped, it no longer exists in the queue. 🔶 The Log data structure is similar - items are appended to the end. It is also read from front to end. The big difference? Elements are NOT deleted once read. This inherently enables read-fanout - the act of reading the same message multiple times. Since the same app doesn't need to read data twice, it's usually different apps that make use of the same message. Both MQs and PubSub are used for asynchronous processing. But their technical differences lead them to optimize for different use cases: 🛑 MESSAGE QUEUES Queues work one item at a time. Consumers in these systems read one message, process it, mark it as processed, the message gets deleted forever and the consumer reads another one. Queues are therefore meant for point-to-point communication. Only one destination gets to ever read and process that message. This maps perfectly to the mental model of a "job" - a discrete unit of work that needs to get executed once and marked as complete. Message Queues are therefore best suited for long-running tasks like: • run a CI job (minutes to hours) • generate a report (seconds) • send an e-mail or notification (milliseconds) Such jobs don’t have uniform processing times either. One CI job can take hours and another can take minutes. Because of the long and unpredictable processing times, MQs enable parallel processing by allowing multiple readers to read from the same queue. 👌 This avoids head-of-line blocking. Otherwise a single slow CI job would block every subsequent one from getting processed in parallel. ⏳ This single-message processing means a Queue doesn't scale as much as a Log, but it can still easily handle thousands of jobs a second. And that's ok - it doesn't need more. By not focusing on scalability at all costs, queues can provide a lot of rich functionality like payload routing, message TTLs, per-message priority, scheduled delivery, error handling and schema validation. A common complaint about message queues is their propensity to crash and lose data. Older MQ implementations stored messages exclusively in memory. This meant that they could run out of memory, crash and lose all the unprocessed data. ❌ Worse off, this flaw indirectly coupled producers to consumers. If your consumers were slow/broken, your queue would eventually crash. That lead to downtime in your producers too, because they couldn't write to the queue. Newer message queues don't do this. 🛑 PUB SUB Log-based systems work with many messages (batches) at a time. This is because in pub subs, messages usually represent data points, not tasks. They aren’t singular jobs that need processing but rather events which get processed in aggregate. Examples include: • process website IP visits and detect bots 🤖 • store website analytics data (clicks, views) and compute counts in real-time 📊 • detect video game cheating in real-time by analyzing the delta between character location data (are they using a speed hack?) 👾 A single event doesn’t have much value, but the collection of events in sequence do. This is why Pub-Sub systems optimize for scale and strict ordering. Events with the same key deterministically go to the same partition. Head of line blocking exists here - each partition is read by only one consumer. 👌 This makes stateful processing per entity dead-easy. 👉 Take the video game cheat detection as an example: if messages are keyed by user id, the consumer knows that it will see every event for that player in the order it was produced. This lets the consumer compute the location coordinate deltas properly. Log-based systems do not delete data once it’s read. This unlocks: • read fanout - the same data used for cheat detection can be read for game analytics purposes • replay - the whole stream of data can be reprocessed from the beginning Replay is useful when you need to rebuild the latest state or re-process the data after fixing a bug which broke the way the consumer processed data. As mentioned earlier, pub-sub systems allow read-fanout. LinkedIn originally created Kafka because they had multiple destinations per event. Unlike point-to-point, this is a one-to-many communication model. One produces writes one piece of data that gets read by many consumers. 💡 Read fanout is handled by consumer groups. A single log (partition) is read from only one consumer at a time. To achieve fanout and read the same partition multiple times, multiple consumer groups are created. Each group has its own consumer clients. Consumers from different groups process the log independently. They read the exact same set of data but keep completely separate markers to denote the progress they've made in reading the log. Log-based pub sub systems can scale massively. To achieve that, they sacrifice a lot of features. Kafka most famously is implemented as a dumb pipe - the server is completely agnostic to what data is being sent to it. It doesn't even support schemas on the server-side, not to mention things like message priority, re-ordering or schema validation. This trades off a lot of usability for the ability to scale to many millions of messages a second. 👌 🛑 SUMMARY Both systems are similar in that they enable asynchronous processing of messages. Certain use cases like one-off task handling is best done with Message Queues, and other higher-volume aggregation jobs are best done with Pub Sub. The most important thing is to not confuse both, and choose the right tool for the job.

I finally finished watching the entire Data Structures & Algorithms playlist by @pmavrin 61 videos ~90 hours of content Presented by an ICPC World champion I already attended lectures with the same content 14 years ago. Yet, it was a nice refresher. I was happy to see how much simpler things became. Back at ITMO University, I struggled most of the time to understand the concepts behind some sophisticated data structures. Turns out, the skills I gained through more than a decade of work experience really helped. Specifically, 1. Decomposing a big problem into smaller ones 2. Thinking about edge cases 3. General reasoning skills

keyboard_arrow_left Previous keyboard_arrow_right Next

Improve API Responsiveness With Streaming JSON Deserialization by @zoranh75 youtube.com/watch?v=hlhAZg… #aspnetcore

At #MSIgnite this week, we announced a huge upgrade for the AI Toolkit for #VSCode! #Copilot‑guided agent creation, eval, tracing, workflow visualization, 1‑click deploy to #MicrosoftFoundry, and access to #Anthropic #Claude models! Full post: aka.ms/AAyq5xo

Nice video on streaming JSON in #aspnetcore Minimal APIs and how it impacts scaling & memory use: youtu.be/hlhAZgFp9xg

Orchestration vs Choreography Forget the clumsy analogies Orchestration The platform manages the continuation Choreography The application manages the continuation Everything else is noise

The best tool to reason about Distributed Executions is the one we already know: Recursive Abstraction a.k.a Functions If we collapse local function calls, we are left with the local parts of a distributed execution and can reason about its mapping onto a distributed system

Dominik Tornow @DominikTornow

11 months ago

There is a difference between a Distributed System and a Distributed Execution Distributed System A set of components that communicate by exchanging messages Distributed Execution An execution that is spread over the components, each performing a part of the execution

0 3 24 7K 18

Confession: I can't stop collecting GitHub repos. Here are 7 popular GitHub repos on software architecture: (for .NET developers) 1. Evolutionary architecture by example - github.com/evolutionary-a… 2. Modular monolith application with DDD - github.com/kgrzybek/modul… 3. .NET 9 starter kit with multitenancy support - github.com/fullstackhero/… 4. eCommerce microservice .NET application - github.com/dotnet/eShop 5. Vertical slice architecture example - github.com/jbogard/Contos… 6. Clean architecture template for .NET apps - github.com/jasontaylordev… 7. Hexagonal application example - github.com/ivanpaulovich/… *** P.S. I break down 5 of those in my ".NET Blueprints" guide. You can download it here: lp.devsecrets.net/blueprints

Koen Claessen did not only invent async await but also worked on formally defining distributed programming model semantics, elevating location to a first class citizen Both ideas a big inspiration for @resonatehqio's durable programming model Distributed Async Await

Dominik Tornow @DominikTornow

a year ago

TIL Koen Claessen, the inventor of async await is also the inventor of property-based testing Legend

keyboard_arrow_left Previous keyboard_arrow_right Next

2 36 297 23K 171

TIL Koen Claessen, the inventor of async await is also the inventor of property-based testing Legend

keyboard_arrow_left Previous keyboard_arrow_right Next