Natural Gradient Descent without the Tears

Photo Link To set the scene, suppose we have a model with weights \(w\) and some loss function \(L(w)\) that we want to minimize. Then our objective is to find \(w^*\) where: \[w^* = \underset{w}{\text{argmin}} L(w)\] Suppose we want to minimize how much our weights change between optimization iterations. To do this, with \(w^{(k)}\) denotin... Read more 02 Jul 2021 - 19 minute read

Video TFRecords: How to Efficiently Load Video Data

Photo Link Compared to images, loading video data is expensive due to the I/O bottleneck and increased decoding time. This reduces efficiency leading to significantly longer training times. Reading online, there are generally two solutions for data loading videos: Decode the video and save its matrix as is With this approach, we improv... Read more 16 Nov 2020 - 2 minute read

Generative Models: Recursive Edition

Photo Link Generative Adversarial Networks (GANs) have shown great results in computer vision but how do they perform when applied to time-series data? Following this, do Convolutional Neural Networks (CNNs) or do Recursive Neural Networks (RNNs) achieve the best results? In this post, we discuss GAN implementations which aim to generate time-... Read more 23 Jul 2020 - 5 minute read

Convex Optimization: Algorithms and their Rate of Convergence

Photo Link Get the presentation here! Get the handout presentation here! Get the Beautiful Latex here! Thank you! Read more 19 Apr 2020 - less than 1 minute read

Encapsulating Capsule Networks: Everything You Need To Know

Photo Link When applying Convolutional Neural Networks (CNNs) (LeCun et al., 1990) to a computer vision task, a change in viewpoint (change in orientation, position, shear, etc.) is likely to lead to drastically different network activations, hindering the model’s ability to generalize. To solve this problem, current CNNs require a large number... Read more 03 Aug 2019 - 27 minute read

Going with the Flow: An Introduction to Normalizing Flows

Photo Link Normalizing Flows (NFs) (Rezende & Mohamed, 2015) learn an invertible mapping \(f: X \rightarrow Z\), where \(X\) is our data distribution and \(Z\) is a chosen latent-distribution. Normalizing Flows are part of the generative model family, which includes Variational Autoencoders (VAEs) (Kingma & Welling, 2013), and Genera... Read more 17 Jul 2019 - 8 minute read

Older Newer