Milla @milla_webdev

How old were you when you found out that MySQL, MariaDB and MaxDB were named after Michael Widenius' three children?

Self attention clearly explained! With LLMs (Language Models) taking the world by storm, it's important to understand what makes them so powerful. Today, we will focus on understanding the three key ingredients of self-attention: - Keys - Queries - Values Background: Neural networks are good with number❗️ In NLP we convert the sequence of words into token & then token to embeddings. Token: Picture a sentence: "The cat sat on the mat." You can think of each word in this sentence as a token. Embeddings: Each token is then transformed into a numerical representation called an embedding. Think of this as assigning unique characteristics to each word/token, helping us identify and relate them. Crafting Keys, Queries, and Values: For each token, we derive three elements: a key (K), a query (Q), and a value (V). Let’s relate this to a library analogy: 🔹 Keys (K): Imagine each book in a library has a unique identifier. This identifier, or key, helps us locate the book on the shelf. 🔸 Query (Q): Suppose you’re looking for a specific book. The information you have about this book acts as the query, which is used to search through the library. 🔹 Value (V): Once you find your book, the content inside it is the value. It holds the actual information you’re looking for. So as we derive K, Q & V for each token, we know what a token contains, what else it is looking for & what it will provide to other tokens. Check this out👇 Building a context aware embedding: A 3-Step process! With our keys, queries, and values in place, it’s time for the tokens to start interacting! (refer the image below as you read ahead) 🔹 Step 1: Attention score calculation: Each token’s query (Q) interacts with all the keys (K) in the sentence, including its own. This interaction is a dot product, followed by scaling. The result? An attention score, representing the relationship between our token and every other token in the sentence. Example: Consider the sentence "I love tennis". Let's say our token is "I". The query for "I" interacts with the key for every word, revealing how closely "I" should attend to "love" & "tennis". 🔸 Step 2: Softmax and Weighting With attention scores in hand, the next step is normalization. We apply the softmax function to these scores, ensuring that they sum up to 1. This gives us the weights – indicating the level of attention each token should pay to every other token. For example, "I" might pay most of its attention to "love" and less to "tennis". 🔹 Step 3: Constructing the Context Now that we have the weights, we use them to compute a weighted sum of the values (V). This sum is the context representation of the token, including information from other tokens based on the calculated attention weights. So, the context of "I" would be a combination of the information from "love" and "tennis", with more emphasis on "love" as per our example. Here's an illustration of what we discussed so far👇 Understanding LLMs & LLMOps is going to be a high leverage skill in future! @LightningAI provides state of the art tutorials on LLMs & LLMOps, I have personally learnt a lot from there. I have shared a link to their LLM learning lab in next tweet! ______________ If you enjoyed reading this & interested in more content like this: Find me → @akshay_pachaar ✔️ Everyday, I share tutorials around LLMs, MLOps & ML Engineering! ______________ Thanks for reading! 🥂

keyboard_arrow_left Previous keyboard_arrow_right Next

Complimentary PDF Book: Learn Python, JavaScript, Git and Excel. Learn these for completely free. You will get: → Python PDF → JavaScript PDF → Git PDF → Excel PDF To get it: 1. Like & Reply “book” 2. Retweet (much appreciated) 3. Follow me (so that I can DM)

What is Event Sourcing? How is it different from normal CRUD design? The diagram below shows a comparison of normal CRUD system design and event sourcing system design. We use an order service as an example. The event sourcing paradigm is used to design a system with determinism. This changes the philosophy of normal system designs. How does this work? Instead of recording the order states in the database, the event sourcing design persists the events that lead to the state changes in the event store. The event store is an append-only log. The events must be sequenced with incremental numbers to guarantee their ordering. The order states can be rebuilt from the events and maintained in OrderView. If the OrderView is down, we can always rely on the event store which is the source of truth to recover the order states. Let's look at the detailed steps. 🔹Non-Event Sourcing Steps 1 and 2: Bob wants to buy a product. The order is created and inserted into the database. Steps 3 and 4: Bob wants to change the quantity from 5 to 6. The order is modified with a new state. 🔹Event Sourcing Steps 1 and 2: Bob wants to buy a product. A NewOrderEvent is created, sequenced, and stored in the event store with eventID=321. Steps 3 and 4: Bob wants to change the quantity from 5 to 6. A ModifyOrderEvent is created, sequenced, and persisted in the event store with eventID=322. Step 5: The order view is rebuilt from the order events, showing the latest state of an order. Over to you: Which type of system is suitable for event sourcing design? Have you used this paradigm in your work? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

What is GraphQL? Is it a replacement for the REST API? The diagram below shows the quick comparison between REST and GraphQL. 🔹GraphQL is a query language for APIs developed by Meta. It provides a complete description of the data in the API and gives clients the power to ask for exactly what they need. 🔹GraphQL servers sit in between the client and the backend services. 🔹GraphQL can aggregate multiple REST requests into one query. GraphQL server organizes the resources in a graph. 🔹GraphQL supports queries, mutations (applying data modifications to resources), and subscriptions (receiving notifications on schema modifications). We talked about the REST API in last week’s video and will compare REST vs. GraphQL vs. gRPC in a separate post/video. Over to you: 1). Is GraphQL a database technology? 2). Do you recommend GraphQL? Why/why not? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

🤣🤣

40 years ago today, Lt. Col. Stanislav Petrov prevented nuclear war by disobeying orders when the USSR’s early warning system erroneously detected nuclear missile launches from the United States. He believed it was a false alarm and chose not to follow Soviet military protocol.

Wishing RMS a speedy recovery

nixCraft 🐧 @nixcraft

3 years ago

Richard Stallman reveals he has cancer in the GNU 40 Hacker Meeting talk audio-video.gnu.org/video/gnu40/rm… It's disheartening to hear the news. I sincerely hope for his speedy recovery and wish him all the best. He is a crucial figure in the current era of SaaS and cloud computing,

18 130 424 97K 47

I've seen many people who learned a skill from YouTube, built something based on it, and made over $100K. In fact, I’m one of them. In 2020, I started learning web development from YouTube, creating content around it, selling digital content. The process is simple. • Learn a skill that excites you • Talk about it on X • Show up daily • Build • Earn

As Mitnick said in The art of deception “…the human factor is truly security's weakest link.”

CSS Gradients:

A person can read 150-300 words per minute. It's time-consuming. Why just only read when you can chat with any document? Let me show you how:

A lot of people ask me if I know of any good FREE courses on Ethical Hacking. Cisco offers great 70-hour course with 34 Labs included. Link in comment below.

sudo in a nutshell

People don't care about your hard work. They care about whether you can get the job done or not.

If you use JSON files, you'll probably like this tool Nested JSON files are hard to read. JsonCrack generates graph diagrams from JSON files and makes them easy to read. Additionally, the generated diagrams can be downloaded as images. – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq

Operating systems turn ugly hardware into beautiful abstractions

30 AI Tools to Start your Online Business in 2023: 𝟭 𝗙𝗔𝗖𝗘𝗟𝗘𝗦𝗦 𝗬𝗢𝗨𝗧𝗨𝗕𝗘 𝗖𝗛𝗔𝗡𝗡𝗘𝗟 Klap Heygen Synthesia 𝟮. 𝗘-𝗕𝗢𝗢𝗞𝗦 Jasper ChatGPT Writesonic 𝟯. 𝗥𝗘𝗦𝗨𝗠𝗘𝗦 Rezi TealHq Kickresume 𝟰. 𝗪𝗘𝗕 𝗗𝗘𝗩𝗘𝗟𝗢𝗣𝗠𝗘𝗡𝗧 10Web Hocoos Landingsite 𝟱. CODING CodeT5 Codeium CodeMate 6 𝗕𝗨𝗦𝗜𝗡𝗘𝗦𝗦𝗘𝗦 Dante ChatBase ChatShape 𝟳. 𝗢𝗡𝗟𝗜𝗡𝗘 𝗖𝗢𝗨𝗥𝗦𝗘𝗦 Courseai Coursebox Learnworlds 𝟴. 𝗜𝗠𝗔𝗚𝗘𝗦 Dall-E Midjourney Adobe Firefly 𝟵. 𝗔𝗗𝗦 AdCopy AdGenai AdCreative 𝟭𝟬. 𝗖𝗢𝗣𝗬𝗪𝗥𝗜𝗧𝗜𝗡𝗚 Copy Al Wordtune Copysmith

Developers, check this out! Refine — a React framework for building CRUD apps quickly. Here's an example website: example.crm.refine.dev Complete React CRM Application built with Refine. It's Open-source and the project code is downloadable. Here is the source code: github.com/refinedev/refi…

Using the same password for everything: