Posts

Links : http://www.gooath.org/blog/2022/12/05/how-do-computers-really-work-part-1-basics/ Transistor Diode Logic gates from transistor Basics Atom made up of electron , proton , nucleus conductor -> free electrons in valence shell insulator -> no electrons in valence shell semiconductor -> few electrons in valence neither a good conductor or insulator semiconductor in pure form is useless , dope it to make n / p type Diode Electric field: start from positive end to negative end electrons accelerate in the opposite direction of the field ...

Google doc link Async Programming Basics of Async Programming At any single time, a single CPU thread executes only a single function ( no multiprocessing happens ) atleast in the case of python !! Its just smart switching Pick up a event loop / create a event loop Create / Add tasks in it Check for the status and if done return the result Coroutines : A coroutine is a special function that can give up control to its caller without losing its state. ...

Basics of Networking Google doc link SSH explained ssh is a secure way to connect to a machine Logic A 2 layer key is setup , a public and private key you Setup Lets say a machine in London wants to talk to a machine in Delhi , london machine is connected to wifi and has a fixed internal IP ( 192.168.1.200 ) , the wifi IP is Dynamic , so in the router’s setting we need to create a port forwarding rule to internal IP (192.168.1.200) on port 22 ...

Learning about AI agents and how they are built in real world

C concepts LIBRARIES #include <stdio.h> : standard input - output , printf , stdin , stdout #include <string> : For string operations string a = “this is a string” #include <stdlib.h> : (https://www.tutorialspoint.com/c_standard_library/stdlib_h.htm) standard libraries , that contains functions, size_t , free , allocate , malloc , realloc , exit , rand , srand , abort , exit , rand , RAND_MAX, EXIT_SUCCESS , EXIT_FAILURE #include <time.h> : time() , use ...

Doc link REPLIT Trying to Learn React once again Delta to learn react is high, you need JS, css, html Importing / Exporting Named export ( export function App ) Default export ( export default ) Default import ( import App from ‘./App’ ) Named import ( import {App} from ‘./App’ ) the { } must match the function name exactly Multiple Exports ** File is : app.js export const Test1 = (()=>{ }) export const Test2 = function(){ } export const Test3 = () => { } // call it as following import {Test, Test2 , Test3} from './app.js' Components : Different parts Like parts of a motorcycle , fan .. these are components and a site is made up many such components Component name should always start with a Capital letter Inside JSX, we can write javascript also Can only return only one parent element ...

Activation / Non-linear function A non-linear function is one in which cannot be expressed from y = mx + c lets suppose a function with y = x * W1 * W2 .. Wl Linearity with respect to x, this means rest of the variable are constant and only x is changing , so linearity with respect to x is linear Linearity with respect to Wi, this means rest of the variable (x here) is constant and only weights are changing , so linearity with respect to W is non-linear ...

Deep dive into the whole required stastistics , that would be a required to learn stable diffusion from the very scratch Probability and Distributions Probability : It tells the chances of an event to happen. And is calculated as the ratio of no. of outcomes of a particular event to the total no. of outcomes. Likelihood : It measures how well a statistical model fits the observed data. Expectation : so E(x) is states what is the average value of x , now let say we have E(tau ~ P(Theta)) = {R(tau)}, this means what is the average reward value,given my tau comes from the distribution P(theta) so its expanded as SUM P(tau | theta) * R(tau) = Expectation {R(tau}, this is how this all plays out ! ...

Training Loop demystified Forward Pass: Compute predictions (The code where you pass the input to the model to get the output) Loss Calculation: Compute the loss (The code where you calculate the loss between the predicted output and the target output) Backward Pass: Compute the gradients for all the parameters where we have requires_grad = True and is stored in the .grad attribute of the parameter (The code where you calculate the gradients of the loss with respect to the model parameters). In the loss.backward() function, it does not update the .grad attribute of the parameter. It just computes the gradient and stores it in the .grad attribute of the parameter. Parameter Update aka optimization: Update the parameters of the model for which we have .grad attribute / requires grad attributes (The code where you update the parameters using the gradients and a learning rate) Zero_grad(): The default behavior of the .grad attribute is to accumulate the gradients of the loss with respect to the parameters (The .grad is a tensor and it accumulates the grad for all the parameters). So we need to set it to zero before doing the backward pass. Training Loop in Pytorch for epoch in range(num_epochs): for i, (images, labels) in enumerate(train_loader): optimizer.zero_grad() # Forward pass outputs = model(images) loss = criterion(outputs, labels) # Backward pass (calculate the gradients and stored in .grad attribute of the parameter) loss.backward() # Updates these parameters based on the defined optimizer and learning rate optimizer.step() The alias codes and removing the abstraction loss.backward() translates to ...

Docker Commands end to end docker run - Run a container from an image docker start - Start a container that has been stopped docker stop - Stop a running container docker kill - Kill a running container docker rm - Remove a container Dockerfile CMD ["/app/start.sh"] : This CMD command runs as soon as you run the container ( not at the build time ) , only at the run time this command works and easily gets overwritten by something like this docker run -it <image_name> ./bin/bash now bin/bash will run as the entry point !! `` : ...

KL-Divergence flow-matching Generative models are very good / best in class approximators of a complex probabilistic equation/distribution that we most of the times have no idea of !! Images modality : All images in the world are from a very complex distribution of pixels that gives direction , based on the prompts , and are dependent of what the user wants , and as it very complex to model that distribution we rely on NN to predict it , hence we have diffusion models ...

The pre-requisites for this blog post is Journey till diffusion model Generative image models We start with some image dataset, lets say x ~ X (where x is a sample and X is the dataset) this x belongs to R_(1024 x 1024 x 3) space and doing any operations in this space is very hard so we try to do these in a lower dimension (z) z belongs to R_(1 x 512) ...