Solution to Kaggle’s Dogs vs. Cats Challenge using Convolutional Neural Networks

Dogs vs. cats challenge [1] from Kaggle ended in Jan 2014 but it is still extremely popular for getting started in deep learning. This is because of two main reasons: the data set is small (25,000 images taking up about 600MB), and it is relatively easy to get a good score.

There are many many online articles discussing on how pre-process data , design a CNN model and finally train the model. So, in this post I am not going to discuss the implementation details. Instead, I am simply going to report my results using a custom designed model and transfer learning. I used Tensorflow and tf.keras with Python and it is available from my Exploring Deep Learning repository [2] at Github.

Learning using a Custom Model

Note that this is my best attempt and not the first attempt. I used four blocks of 2D convolution layers followed by max pooling. In the end, I used two dense layers and a softmax layer as output. I also used dropout layers and image augmentation. The exact command line for training this model is:

TrainCNN.py --cnnArch Custom --classMode Categorical --optimizer Adam --learningRate 0.0001 --imageSize 224 --numEpochs 30 --batchSize 16 --dropout --augmentation --augMultiplier 3

The CNN model is given below:

--------------------------------------------------------------- 
Model: "Custom"
---------------------------------------------------------------   
Layer (type)                 Output Shape              Param #

conv2d (Conv2D)              (None, 224, 224, 32)      896
max_pooling2d (MaxPooling2D) (None, 112, 112, 32)      0

conv2d_1 (Conv2D)            (None, 112, 112, 64)      18496
max_pooling2d_1 (MaxPooling2 (None, 56, 56, 64)        0
 
conv2d_2 (Conv2D)            (None, 56, 56, 128)       73856
max_pooling2d_2 (MaxPooling2 (None, 28, 28, 128)       0
 
conv2d_3 (Conv2D)            (None, 28, 28, 256)       295168
max_pooling2d_3 (MaxPooling2 (None, 14, 14, 256)       0
 
flatten (Flatten)            (None, 50176)             0
dense (Dense)                (None, 512)               25690624
dense_1 (Dense)              (None, 256)               131328
dense_2 (Dense)              (None, 2)                 514
=============================================================== 
Total params: 26,210,882
Trainable params: 26,210,882
Non-trainable params: 0
--------------------------------------------------------------- 

The above model was trained on 15,000 (7,500 each for dogs and cats) randomly chosen images from the Kaggle data set and validated with a separate 5,000 (2,500 each for dogs and cats) images. The model achieved 94% accuracy after 24 epochs. It took about 4 hours of training on my PC with NVidia GeForce GTX 1050 with 2GB of RAM.

Cross-entropy loss for training and validation and the classification
accuracy for training and validation using a custom CNN model.

Transfer Learning using VGG16 Model

For the second part, I used the VGG16 model with imagenet weights without the top layer and a custom denser layers at the end. Similar to the previous step, I used dropout layers and image augmentation. The exact command line for training this model is:

TrainCNN.py --cnnArch VGG16 --classMode Categorical --optimizer Adam --learningRate 1e-5 --imageSize 224 --numEpochs 30 --batchSize 25 --dropout --augmentation --augMultiplier 3

The CNN model is given below:

---------------------------------------------------------------
Model: "VGG16"
---------------------------------------------------------------   
Layer (type)                 Output Shape              Param #
===============================================================  
vgg16 (Model)                (None, 7, 7, 512)         14714688
flatten (Flatten)            (None, 25088)             0
dense (Dense)                (None, 512)               12845568
dense_1 (Dense)              (None, 256)               131328
dense_3 (Dense)              (None, 2)                 514
===============================================================
Total params: 27,692,104
Trainable params: 12,977,410
Non-trainable params: 14,714,688
---------------------------------------------------------------

The above model was trained on the same dataset as the custom model above and it achieved an accuracy of 98% after 11 epochs. Clearly, this model is far more efficient and more accurate then the custom designed model.

Cross-entropy loss for training and validation and the classification accuracy
for training and validation using a transfer learning from VGG16 model.

References:

  1. https://www.kaggle.com/c/dogs-vs-cats
  2. https://github.com/saurabhg17/ExploringDeepLearning

String Selection Widget for Qt5

Some time back, I developed a data entry application in Qt5. One of the requirements was to let the user select a single string from a predefined list of string. I developed a custom widget called SStringSelector for this purpose. SStringSelector has two views: display and selection. The display view presents the currently selected string (blank if no string is selected), and a push button. To select a string, the user clicks on the button which presents the user with the selection dialog. The selection dialog consists of a list of string in an QListWidget and the user can select one of them by double-clicking a string. If the list of strings are long, the user can filter them using a filter QLineEdit present above the QListWidget.

SStringSelector is distributed as a part of QtUtils repository hosted on Github. The SStringSelector widget is really simple to use. Simple add the SStringSelector.h and SStringSelector.cpp files in your project and add an instance of SStringSelector in the layout of your app.

Below are some screenshots of the widget under Windows:

The SStringSelector Widget.
Selection Dialog of the SStringSelector Widget.
Filtering Strings in the Selection Dialog.

Color Picker Widget for Qt5

Qt5 support standard dialogs such as QFileDialog, QFontDialog, and QColorDialog, however, it does not provide a color picker to allow a user to pick a color. Recently, I need a color picker for one of my projects and I implemented a simple color picker widget.

SColorPicker is available from Github as a part of QtUtils repository. To use SColorPicker, add the header and cpp files directly in your project. Then, simply add an instance of SColorPicker in a layout. SColorPicker will appear as 16×16 pixels colored square in the layout. If you need a different size, change it in the SColorPicker's constructor. When a user double-clicks on the colored square, the system’s color dialog will appear allowing the user to choose a color. The selected color can be obtained from color() function or by connecting to colorPicked() signal.

Below are the screenshots of the SColorPicker_Demo and system color dialog present to the user on Windows 10 computer.


Fill Disk Partition

Recently, I had to give away a computer with couple of disks in it. I wanted to securely erase data on these disks as I stored personal sensitive information on them. Using a program such as DBAN was not an option as I was not allowed to remove the operating system from the computer. My goal was to simply overwrite free space from all the partitions. I couldn’t find anything I liked so I ended up writing a simple tool called FillPartition in python.

FillPartition is hosted on Github at https://github.com/saurabhg17/FillPartition. It is really easy to use with just one mandatory argument (the path of the partition) and one optional argument (–outputDir, -od) the directory in the partition where files should be written. FillPartition writes 1GB files filled with 0 bytes until the free space is less than 1GB and then write one final file of the size equal to the remaining free space.

Below is a screenshot of a run of FillPartition on Windows


Tomato Cells under Microscope

To see tomato cells under microscope, simply squeeze a bit of tomato juice on a clean glass slide and gently place a cover slip over it.

Micrographs

Below is the micrograph of the tomato cells:

Tomato cells magnified 40 times

Tomato cells are floating in the juice and hence are not connected to each other. The thick black circles are air bubbles that got trapped between slide and cover slip. I was not able to get rid of them after couple of tries.

Tomato cells are very big compared to onion skin cells. In fact, they are more than 25 times bigger than onion skin cells! Below is the micrograph of onion skin cells for comparison:

Onion cells magnified 40 times

Search Box using QLineEdit

This week, at work I had to implement a search box for a software I am working on. The search box is to filter some data dynamically as user types a query. I wanted to show a clear (cross) icon at the right side of the search box so that user can clear the results instead of selecting the current query and deleting it manually. Lastly, for clarity I wanted to show a search icon on the left side of search box. The search box looks like this:

Screenshot of the Search box implemented using QLineEdit

After the user enters a query a clear icon appears on the right. The clear icon is in fact a button and clicking it will clear the current search.
Screenshot of the Search box with keywords implemented using QLineEdit

It is really easy to make this search box using QLineEdit. We need only the following three lines of code:

QLineEdit* _lineEdit = new QLineEdit();
_lineEdit->setClearButtonEnabled(true);
_lineEdit->addAction(":/resources/search.ico", QLineEdit::LeadingPosition);
_lineEdit->setPlaceHolderText("Search...");

// add _lineEdit to your widget

Line 2 enables the clear button which adds the clear action and cross icon to the right. Line 3 adds another action with a search icon to the left of the QLineEdit. We don’t listen to this action as it is merely decorative. Line 4 adds a placeholder text which is shown in the QLineEdit but is cleared as soon as user starts typing.

We only connect textChanged(const QString&)  signal which is emitted both when a user clicks on the cross icon and when he enters a search query.

 




Markdown to PDF Converter

Few weeks ago, I published SLogLib (a cross-platform logging library) on GitHub. I wrote the user manual in a readme.md file as is the standard practice at GitHub. However, since most of users don’t have markdown viewers installed on their machines they would either need to access GitHub repository or would have to convert it to more popular format such as PDF or perhaps HTML. For many users going online is becoming standard practice to access documentation but I prefer offline manuals. Thus, I wanted to ship a PDF manual along with the code.

I searched high and low for a standalone tool to convert markdown to PDF but surprisingly there are not a lot of options out there. The first tool I came across was GitPrint. It is conceptually innovative and straightforward to use with GitHub. Just add /your_user_name/repository_name at the end of http://gitprint.com and it prints the readme.md in the repository to PDF. The PDF generated is of good quality but there are no styling options. Also, it failed to include images in the PDF so I had to kept looking. One of the frequently recommended tool is PanDoc, which is a swiss-army knife to convert files from one markup format into another. However, in my experience it doesn’t do a good job of converting markdown to PDF. Another popular tool online is a markdown-pdf package for Node.js. Since, I have no prior experience with Node.js I haven’t tried it yet.

Earlier this year, I bought a MacBook Pro and installed a markdown editor called MacDown. It is really nice tool with side-by-side rendering of markup and HTML. It can export markdown as PDF and produces very good quality PDF’s. It also supports lots of styling options as well as a CSS to customize PDF generation. In the end, I used it to generate PDF for SLogLib.

Even though I had a PDF for SLogLib, I wanted to find/build a cross-platform tool to convert markdown to PDF.

The basic idea to convert markdown to PDF is simple. First convert markdown to HTML and then print HTML to PDF. I used hoedown to convert markdown to PDF because of several reasons:

  1. First and foremost it is cross-platform and compiles as a standalone binary for all three main platforms: Windows, Linux, and OSX.
  2. MacDown uses it too and I was quite happy with its rendering.
  3. It supports not only standard markdown but also several non-standard extensions.

To converted HTML to PDF one of the most popular tool I came across was wkhtmltopdf. It is also cross-platform and complies into standalone binaries for all popular platforms. In fact, it is possible to download the pre-built library right from its website. Wkhtmltopdf uses a modified version of webkit shipped with Qt. It uses webkit to render the html and print to PDF. However, while testing I found that on a Windows 7 machines there is a serious problem with font kerning. It has been reported by a lot of users but I haven’t found a solution to fix it. Wkhtmltopdf would have been ideal as I could simply write a command line and/or GUI tool wrapping the functionality of hoedown and Wkhtmltopdf.

Screenshot of markdown to PDF generated from MacDown

Screenshot of PDF generated from MacDown.

Screenshot of markdown to PDF generated wkhtmltopdf

Screenshot of PDF generated from wkhtmltopdf.

I could not find any other standalone cross-platform tool to convert HTML to PDF. So, for now I decided to use dompdf which is written in PHP. Once I started used PHP I thought why not make it a web based tool. This would allow me to learn about SEO which I have been promising myself to learn one day :). The tools is hosted at http://markdown2pdf.com. At the moment it doesn’t appear in first five pages in Google search for “markdown to pdf” or “markdown 2 pdf”. I am playing with various SEO tools and techniques and hope to get it within first five pages.

My quest for a standalone tool is not yet complete. I will try to find a solution for wkhtmltopdf kerning issue or find another standalone cross-platform tool for converting from HTML to PDF. I will update with my findings on this blog.




Onion Cells under Microscope

In this post, I will show how to make a wet mount slide for looking onion cells under a microscope.

Making the slide
  1. Take a clean slide and place a drop of water in the centre
  2. Take a small piece of onion and carefully peel the translucent membrane from the rough underside
    of the slide. To peel the membrane, you can either use a sharp blade or a pair of tweezers. It is important to do this step carefully so as to not break too many cells. So, ideally always hold the peeled membrane at the edges.
  3. Now carefully, place the membrane in the drop of water placed earlier on the slide.
  4. You may want to put a small drop of tincture iodine over the onion membrane. This is to help create contrast between cell nuclei and other parts of cells.
  5. Finally, gently lower a cover slip over the membrane.
Micrographs

Below are the micrographs of the onion cells. The nuclei are the small dark circles and the thick black lines are the cell walls.

Onion cells magnified 40 times
Onion cells magnified 40 times.
Onion cells magnified 100 times
Onion cells magnified 100 times.

Dog Cardiac Muscle l.s. Amscope 50PC Prepared Slides

Dog cardiac muscle longitudinal section (l.s.) is the 9th slide in the Amscope 50PC prepared slides. A cardiac muscle is found only in heart. These muscles are involuntary i.e. they contract and expand automatically to keep heart pumping. I am not 100% sure but most likely the cark blue dots in the micrographs are the nuclei.

Micrographs [19 July 2015]
Amscope 50PC Prepared Slides

This post lists all the micrographs I have done from the Amscope 50PC prepared slides.


Dense Connective Tissue section Amscope 50PC Prepared Slides

Dense connective tissue (section) is the 8th slide in the Amscope 50PC prepared slides. Dense connective tissue have densely packed fibers made up of mainly collagen (while lines in the micrograph below). The fibers in these tissues are regularly arranged and they are very strong but inelastic. Due to their in-elasticity, they can break if a strong force is applied across the fibers. Dense connective tissues forms the ligaments (connects muscles to bones) and tendons (connects bones to bones) in our body.

Micrographs [19 July 2015]
Amscope 50PC Prepared Slides

This post lists all the micrographs I have done from the Amscope 50PC prepared slides.