In a previous introductory tutorial on neural networks, a three layer neural network was developed to classify the hand-written digits of the MNIST dataset. In the end, it was able to achieve a classification accuracy around 86%. For a simple data set such as MNIST, this is actually quite poor. Further optimizations can bring densely… Read More »Convolutional Neural Networks Tutorial in PyTorch
In the late 80’s and 90’s, neural network research stalled due to a lack of good performance. There were a number of reasons for this, outlined by the prominent AI researcher Geoffrey Hinton – these reasons included poor computing speeds, lack of data, using the wrong type of non-linear activation functions and poor initialization of the… Read More »Weight initialization tutorial in TensorFlow
Deep learning is huge in machine learning at the moment, and no wonder – it is making large and important strides in solving problems in computer vision, natural language and reinforcement learning and problems in many other areas. Deep learning neural networks are neural networks which are characterized by many layers – making them deep instead of wide. Deep networks have been… Read More »The vanishing gradient problem and ReLUs – a TensorFlow investigation
Consuming data efficiently becomes really paramount to training performance in deep learning. In a previous post I discussed the TensorFlow data queuing framework. However, TensorFlow development is always on the move and they have now created a more streamlined and efficient way of setting up data input pipelines. This TensorFlow Dataset tutorial will show you… Read More »TensorFlow Dataset API tutorial – build high performance data pipelines
In this post, I’m going to introduce the concept of reinforcement learning, and show you how to build an autonomous agent that can successfully play a simple game. Reinforcement learning is an active and interesting area of machine learning research, and has been spurred on by recent successes such as the AlphaGo system, which has… Read More »Reinforcement learning tutorial using Python and Keras
In previous posts, I introduced Keras for building convolutional neural networks and performing word embedding. The next natural step is to talk about implementing recurrent neural networks in Keras. In a previous tutorial of mine, I gave a very comprehensive introduction to recurrent neural networks and long short term memory (LSTM) networks, implemented in TensorFlow.… Read More »Keras LSTM tutorial – How to easily build a powerful deep learning language model
In my previous tutorial on recurrent neural networks and LSTM networks in TensorFlow, we weren’t able to get fantastic results. This is because I was running the code on my little ol’ laptop CPU – not exactly the ideal setup for big deep learning networks. So what to do? I could fork out thousands on… Read More »How to create a TensorFlow deep learning powerhouse on Amazon AWS
So – if you’re a follower of this blog and you’ve been trying out your own deep learning networks in TensorFlow and Keras, you’ve probably come across the somewhat frustrating business of debugging these deep learning libraries. Sure, they have Python APIs, but it’s kinda hard to figure out what exactly is happening when something… Read More »A PyTorch tutorial – deep learning in Python
In the deep learning journey so far on this website, I’ve introduced dense neural networks and convolutional neural networks (CNNs) which explain how to perform classification tasks on static images. We’ve seen good results, especially with CNN’s. However, what happens if we want to analyze dynamic data? What about videos, voice recognition or sequences of text?… Read More »Recurrent neural networks and LSTM tutorial in Python and TensorFlow
I’ve been dedicating quite a bit of time recently to Word2Vec tutorials because of the importance of the Word2Vec concept for natural language processing (NLP) and also because I’ll soon be presenting some tutorials on recurrent neural networks and LSTMs for sequence prediction/NLP (UPDATE: I’ve completed a comprehensive tutorial on these topics – Recurrent neural… Read More »Python gensim Word2Vec tutorial with TensorFlow and Keras