Error loading numpy arrays

If you received “Object arrays cannot be loaded when allow_pickle=False” while loading a numpy array using numpy.load() it is because numpy has changed the default loading behaviour since version 1.16.3. If you are using numpy version newer than this, at many places on internet it is advised to simply downgrade the numpy version. This is not the correct solution at all. From the numpy documentation:

allow_pickle : bool, optional
Allow loading pickled object arrays stored in npy files. Reasons for disallowing pickles include security, as loading pickled data can execute arbitrary code. If pickles are disallowed, loading object arrays will fail. Default: False
Changed in version 1.16.3: Made default False in response to CVE-2019-6446.

Thus, the correct solution is to pass allow_pickle=True to the numpy.load function. However, this should be used carefully and ideally only with the files you have previously saved yourself since picking in python can execute arbitrary code thereby compromising the system security.


String Selection Widget for Qt5

Some time back, I developed a data entry application in Qt5. One of the requirements was to let the user select a single string from a predefined list of string. I developed a custom widget called SStringSelector for this purpose. SStringSelector has two views: display and selection. The display view presents the currently selected string (blank if no string is selected), and a push button. To select a string, the user clicks on the button which presents the user with the selection dialog. The selection dialog consists of a list of string in an QListWidget and the user can select one of them by double-clicking a string. If the list of strings are long, the user can filter them using a filter QLineEdit present above the QListWidget.

SStringSelector is distributed as a part of QtUtils repository hosted on Github. The SStringSelector widget is really simple to use. Simple add the SStringSelector.h and SStringSelector.cpp files in your project and add an instance of SStringSelector in the layout of your app.

Below are some screenshots of the widget under Windows:

The SStringSelector Widget.
Selection Dialog of the SStringSelector Widget.
Filtering Strings in the Selection Dialog.

Color Picker Widget for Qt5

Qt5 support standard dialogs such as QFileDialog, QFontDialog, and QColorDialog, however, it does not provide a color picker to allow a user to pick a color. Recently, I need a color picker for one of my projects and I implemented a simple color picker widget.

SColorPicker is available from Github as a part of QtUtils repository. To use SColorPicker, add the header and cpp files directly in your project. Then, simply add an instance of SColorPicker in a layout. SColorPicker will appear as 16×16 pixels colored square in the layout. If you need a different size, change it in the SColorPicker's constructor. When a user double-clicks on the colored square, the system’s color dialog will appear allowing the user to choose a color. The selected color can be obtained from color() function or by connecting to colorPicked() signal.

Below are the screenshots of the SColorPicker_Demo and system color dialog present to the user on Windows 10 computer.


Fill Disk Partition

Recently, I had to give away a computer with couple of disks in it. I wanted to securely erase data on these disks as I stored personal sensitive information on them. Using a program such as DBAN was not an option as I was not allowed to remove the operating system from the computer. My goal was to simply overwrite free space from all the partitions. I couldn’t find anything I liked so I ended up writing a simple tool called FillPartition in python.

FillPartition is hosted on Github at https://github.com/saurabhg17/FillPartition. It is really easy to use with just one mandatory argument (the path of the partition) and one optional argument (–outputDir, -od) the directory in the partition where files should be written. FillPartition writes 1GB files filled with 0 bytes until the free space is less than 1GB and then write one final file of the size equal to the remaining free space.

Below is a screenshot of a run of FillPartition on Windows


Search Box using QLineEdit

This week, at work I had to implement a search box for a software I am working on. The search box is to filter some data dynamically as user types a query. I wanted to show a clear (cross) icon at the right side of the search box so that user can clear the results instead of selecting the current query and deleting it manually. Lastly, for clarity I wanted to show a search icon on the left side of search box. The search box looks like this:

Screenshot of the Search box implemented using QLineEdit

After the user enters a query a clear icon appears on the right. The clear icon is in fact a button and clicking it will clear the current search.
Screenshot of the Search box with keywords implemented using QLineEdit

It is really easy to make this search box using QLineEdit. We need only the following three lines of code:

Line 2 enables the clear button which adds the clear action and cross icon to the right. Line 3 adds another action with a search icon to the left of the QLineEdit. We don’t listen to this action as it is merely decorative. Line 4 adds a placeholder text which is shown in the QLineEdit but is cleared as soon as user starts typing.

We only connect textChanged(const QString&)  signal which is emitted both when a user clicks on the cross icon and when he enters a search query.




Markdown to PDF Converter

Few weeks ago, I published SLogLib (a cross-platform logging library) on GitHub. I wrote the user manual in a readme.md file as is the standard practice at GitHub. However, since most of users don’t have markdown viewers installed on their machines they would either need to access GitHub repository or would have to convert it to more popular format such as PDF or perhaps HTML. For many users going online is becoming standard practice to access documentation but I prefer offline manuals. Thus, I wanted to ship a PDF manual along with the code.

I searched high and low for a standalone tool to convert markdown to PDF but surprisingly there are not a lot of options out there. The first tool I came across was GitPrint. It is conceptually innovative and straightforward to use with GitHub. Just add /your_user_name/repository_name at the end of http://gitprint.com and it prints the readme.md in the repository to PDF. The PDF generated is of good quality but there are no styling options. Also, it failed to include images in the PDF so I had to kept looking. One of the frequently recommended tool is PanDoc, which is a swiss-army knife to convert files from one markup format into another. However, in my experience it doesn’t do a good job of converting markdown to PDF. Another popular tool online is a markdown-pdf package for Node.js. Since, I have no prior experience with Node.js I haven’t tried it yet.

Earlier this year, I bought a MacBook Pro and installed a markdown editor called MacDown. It is really nice tool with side-by-side rendering of markup and HTML. It can export markdown as PDF and produces very good quality PDF’s. It also supports lots of styling options as well as a CSS to customize PDF generation. In the end, I used it to generate PDF for SLogLib.

Even though I had a PDF for SLogLib, I wanted to find/build a cross-platform tool to convert markdown to PDF.

The basic idea to convert markdown to PDF is simple. First convert markdown to HTML and then print HTML to PDF. I used hoedown to convert markdown to PDF because of several reasons:

  1. First and foremost it is cross-platform and compiles as a standalone binary for all three main platforms: Windows, Linux, and OSX.
  2. MacDown uses it too and I was quite happy with its rendering.
  3. It supports not only standard markdown but also several non-standard extensions.

To converted HTML to PDF one of the most popular tool I came across was wkhtmltopdf. It is also cross-platform and complies into standalone binaries for all popular platforms. In fact, it is possible to download the pre-built library right from its website. Wkhtmltopdf uses a modified version of webkit shipped with Qt. It uses webkit to render the html and print to PDF. However, while testing I found that on a Windows 7 machines there is a serious problem with font kerning. It has been reported by a lot of users but I haven’t found a solution to fix it. Wkhtmltopdf would have been ideal as I could simply write a command line and/or GUI tool wrapping the functionality of hoedown and Wkhtmltopdf.

Screenshot of markdown to PDF generated from MacDown

Screenshot of PDF generated from MacDown.

Screenshot of markdown to PDF generated wkhtmltopdf

Screenshot of PDF generated from wkhtmltopdf.

I could not find any other standalone cross-platform tool to convert HTML to PDF. So, for now I decided to use dompdf which is written in PHP. Once I started used PHP I thought why not make it a web based tool. This would allow me to learn about SEO which I have been promising myself to learn one day :). The tools is hosted at http://markdown2pdf.com. At the moment it doesn’t appear in first five pages in Google search for “markdown to pdf” or “markdown 2 pdf”. I am playing with various SEO tools and techniques and hope to get it within first five pages.

My quest for a standalone tool is not yet complete. I will try to find a solution for wkhtmltopdf kerning issue or find another standalone cross-platform tool for converting from HTML to PDF. I will update with my findings on this blog.




Cross-platform high-resolution timer

Often there is a need to estimate the time it takes for a piece of code to run. This is useful not only for debugging but also for reporting the execution time of lengthy tasks to the user.

On Windows, QueryPerformanceFrequency() and QueryPerformanceCounter() can be used to determine the execution time of a code. QueryPerformanceFrequency() returns the frequency of the current performance counter in counts per second and QueryPerformanceCounter() returns a high resolution (<1µs) time stamp. Together they can be used to determine time it takes to run a piece of code is:

On Linux, clock_gettime can be used to get a time interval with a resolution of nano-seconds. clock_gettime() requires two arguments: clockid_t and timespec structure. To build a timer, CLOCK_MONOTONIC is a good choice for clockid_t as the time is guaranteed to be monotonically increasing. timespec structure have two field: tv_sec (time in seconds) and tv_nsec (time in nanoseconds). Code to determine the time it takes to run a piece of code is:

I have written a simple class which can be user on both windows and Linux. It has the following interface:

You can download the code from the following links:
Timer.h
Timer.cpp
Timer_Unix.cpp
Timer.zip


SLogLib: An easy to use, fully customizable and extensible, cross-platform logging library

A long time ago when I was doing PhD I was implementing a very complex geometric algorithm for computing intersection of two triangular meshes. There were bugs in code which would trigger only in certain edge cases. Since it was a GUI program using std::cout was not an option. Initially I tried writing messages to a file but soon realized it was too tedious as code was spanned across several files and I had to manually insert file names, function names, line numbers for every logging message.

A quick search on Internet revealed many logging libraries. I tried couple of them (unfortunately I can’t remember their names now) but none of them allowed customization of the output. The libraries I came across could output to variety of devices, supported multi-threading and many other fancy features but it was not possible to change the way messages was reported to the user. This was very important to me because I wanted to format my messages in a particular way so that I can easily check how my code was crashing on edge cases.

So, I wrote the first version of SLogLib sometime in 2005. It was build on a single principle that user should be in complete control of how messages are written to devices. In order to do that, SLogLib wraps all information required for logging into a structure called Message and passes it to a Formatter. The Formatter converts the Message structure to a std::string which will be outputted to the device. The Formatter must be written by the user. However, to make it easier to start using SLogLib and illustrate how to write a Formatter few Formatters are included with SLogLib.

Over past decade SLogLib has been very useful to me for a variety of projects and I hope that other can find it useful as well. SLogLib is hosted on Github under MIT license. You can clone of fork it from here: https://github.com/saurabhg17/SLogLib.




Computing area of all facets in CGAL::Polyhedron_3

In this post I will show how to compute area of a facet in CGAL::Polyhedron_3.

ComputeFacetArea() Functor

ComputeFacetArea() is a thread-safe functor for computing the area of a given facet in a CGAL::Polyhedron_3. The facet must be a planar polygon with arbitrary number of sides. We need facet’s normal vector to compute it’s area. The facet normals must be be initialized using the ComputeFacetArea()’s constructor. The code for computing the facet normals is presented in this post: Computing normal of all facets in CGAL::Polyhedron_3.

Using ComputeFacetArea() Functor

Area of a facet f can be computed as double area = ComputeFacetArea(h);.

For most purposes, it is better to compute area of all facets once and cache them for later use. It is best to store the results in an associative container which associates the facet handle with the area. In the following example, I use PropertyMap which is a wrapper for std::set.

Downloads

ImportOBJ.h
PropertyMap.h
ComputeFacetNormal.h
ComputeFacetArea.h
TestComputeFacetArea.cpp
Venus.obj
ComputeFacetArea.zip


Computing normal of all vertices in CGAL::Polyhedron_3

In this post I will show how to compute normal vector at a vector in CGAL::Polyhedron_3.

ComputeVertexNormal() Functor

ComputeVertexNormal() is a thread-safe functor for computing the normal vector at a given vertex in a CGAL::Polyhedron_3. The normal vector at a vertex is the average of the normal vectors of all facets incident on the vertex. The facet normals must be be initialized using the ComputeVertexNormal()’s constructor. The code for computing the facet normals is presented in this post: Computing normal of all facets in CGAL::Polyhedron_3.

Using ComputeVertexNormal() Functor

Normal vector at a vertex v can be computed as Vector3 normal = ComputeVertexNormal(f); .

For most purposes, it is better to compute area of all facets once and cache them for later use. It is best to store the results in an associative container which associates the facet handle with the area. In the following example, I use PropertyMap which is a wrapper for std::set.

Downloads

ImportOBJ.h
PropertyMap.h
ComputeFacetNormal.h
ComputeVertexNormal.h
TestComputeVertexNormal.cpp
Venus.obj
ComputeVertexNormal.zip