2018 coming to an end

It is December again. Another wonderful year is coming to an end. It’s been a long time since I posted my last blog here. Was extremely busy with my job at Oracle. Even though I was very busy I managed to do some good work in Deep learning.

Courses

In the last 5 months, I completed some  great courses from Coursera and Youtube on Deep learning.

Deeplearning.ai is one of the courses I did in the last 5 months. This is a package of 5 courses starting from Basics of the Neural net to Advanced concepts like CNN, RNN, GRU, LSTM etc. It costs about 3000Rs per month for subscribing to the course and will take nearly 3-4 months to complete. Working through this course I am able to understand and implement most of the latest concepts in Deep learning. This is an extremely recommended course for everyone who is interested to learn advanced concepts.

Deep learning Frameworks

I am able to work on some deep learning frameworks including TensorFlow, PyTorch and Keras. I also did a course in datacamp to learn more about Numpy.

Kaggle

I finally started to participate in Kaggle competitions. I heard about kaggle for the first time about 2 years back. I recently participated in the PUBG kernel competition to predict the chance of a player to win the game. Kaggle is more than just participating in competitions. There is a section known as Kaggle learn where there are free tutorials that teach from the basics of machine learning to Advanced concepts like CNN implementation. Many people share their Kernel which has extremely valuable information relating to the techniques, algorithms,implementation etc.

Learn Maths

I started to relearn the mathematical concepts like Probability ,Linear algebra and Vector calculus. I am currently going through my old college textbooks. Knowing these concepts really helps in understanding research papers with some heavy math.

Code

I have also worked on some code for ML algorithms that I learned and implemented. I have uploaded some of them in my GitHub repo

The following is an interview video with an ML PhD student. I particularly liked this because of the quality of discussion on GAN.So I am putting it here.

GAN -Generative Adversarial Network

“(GANs), and the variations that are now being proposed is the most interesting idea in the last 10 years in ML”

These are the words of Yann LeCun, one of the very famous researchers in Deep Learning. GAN or Generative Adversarial Network is considered to be one the most promising developments in ML last year.

What is GAN?

If you have some idea about deep learning networks you will know about using them for various tasks like NLP, image recognition, object detection etc. Neural networks work extremely well in all these tasks. But does it mean it can do well in all areas? Imagine this situation. An artist is creating a masterpiece drawing. A neural network might be able to recognize objects in the image, but can we create a network which can create a great painting like that. Can a machine write beautiful articles and publish them. This is considered to be of great difficulty since we train the networks with existing data. The problem here is to make the network generate its own data. This is a Generative task.

The issue with the Generative task.

Imagine you are learning chess. If you want to improve your game you need a powerful opponent who does innovative moves. A chess player improves his game skills by analyzing his moves and finding the mistakes. If we are creating a model for chess we train it using many moves we manually prepare.

Image Generation Problem.

Image generation problem is where the machine learning model generates an image by itself. Imagine a machine creating a drawing as beautiful as Picasso. We give a set of images for training and the model generates the image while testing that it is similar to the training images, but are not same.

Problem

Here the issue is that we need a way to score the output image. If there are 2 output images, how can we say which one is better?

GAN proposes to use a neural network for this process. So in addition to the model, there is another neural network that scores the image output. The neural net that generates the image is called Generator and the one that scores the image is called the Discriminator.

 

Working of GAN?

As I said earlier there are 2 neural nets basically. Generator and Discriminator.

Generator

The generator has the duty of generating a novel image based on the patterns in the training samples.It uses the probability distribution to generate the similar patterns of the test set in the output image. We can see Generator as the function G(z). It takes the input z which is the sample from the probability distribution p(z).

Discriminator

Discriminator takes input from two sources. Real data and generated data from Generator. Its duty is to score the images into classes – Generated, Real. i.e it tells whether the input image is a real image or a generated image. It uses a sigmoid classifier to score the images

Training GAN

Training of GAN is done similar to two agents, playing with each other in a reinforcement learning setup. Here, Generator is trying to create an image that is indistinguishable by Discriminator and Discriminator is trying to discriminate the image from the real image. Like a game right?

Mathematically it is represented as

In the function V(D, G), the first term is the entropy of log likelihood of sample data from Probability Distribution of real data. Discriminator tries to maximize this value to 1. The second term is the entropy of generated image(fake image). The discriminator tries to maximize this to 0.

 

Applications of GAN

1. Increasing resolution of Image

2. Text to image generation

This is implemented with some modifications called Conditional GAN. Paper

3. Interactive Image Generation

Conclusion

Here I have tried to give a simple and brief idea about GAN.In the next post, let’s look at training a GAN more practically and let’s implement one in tensorflow.