What I’ve been up to this semester:

The main project that I worked on this semester was a smoke detection matting problem for a statistical learning class taught by Dr. Raul Rojas (who made a guest professor appearance this semester at UNR). Throughout the course of the course,  we implemented different classifiers about every week.

After we got nearly up to date on statistical learning methods (excluding a lot of the techniques in deep learning, which deserve a class of their own), we split into teams to work on different sections of the problem of detecting smoke in images from the Tahoe & southern California regions.

My partner Banafsheh Rekabdar and I implemented two techniques which assume the linear separability in the images containing smoke, which was previously presented by Tian, Li et al. (http://ro.uow.edu.au/cgi/viewcontent.cgi?article=3301&context=eispapers)

Here is an example of the problem at hand:

The background image is used from the beginning of the sequences of images presented in the dataset’s from the seismology lab at UNR. We then take the red channel (as for why, we’ll get to that) then apply one of the methods presented in Tian, Li et al, we select the salient regions (in our work 16×16 pixels) that will then be passed along the the classifier at the end of the smoke detection pipeline.

Background Image:


Foreground Image:


Detection of important regions which will then be passed to the classifier (PCA is used for this example):


All of the methods used here assume the following linear combination of the background image and the smoke vector:

Smoke Detection

We also use correlation in order to check if a region is similar to what it was in the background:

Smoke Detection  (1)

There were two methods presented in Tian, Li et al which we implemented, the first was the assumption of local smoothness. (We also use attenuation and airlight assumptions from Narasimhan and Nayar (2002), for more information see the link for the paper at the bottom!) The local smoothness is based on the local structure of small groups of pixels in pictures of smoke and how similar and smooth they appear.  See more in the slide below: Smoke Detection  (2)

With the analytical solutions:

Smoke Detection  (3)

The problem as you’ll see in the results are there are lots of things in natural settings which also vary over time (and thus are not filtered by correlation) and are locally smooth:

Here are the full color images of a two fire case: image-0002

Onset of both fires:


During the middle of both fires:


Local Smoothness’ guess of smoke like regions in the early case:


Local Smoothness’ guesses from the later on in the fires case:


Green plots represent the more important points, blue points less important, red even less important. (Importance here is determined by the alpha value, again see paper/github link for more specifics and results!)

The next method that was implemented was the Principal Component Analysis approach. Originally this approach seemed intractable due to the need for pure smoke images in order to determine the subspace of information rich dimensions for smoke. After emailing Tian and Li we were lucky enough to obtain their original data set and thus created our own eigensmokes!

Smoke Detection  (6)

For more information on the methodology: Smoke Detection  (4)

As well as analytical solutions:

Smoke Detection  (5)

PCA did wonderfully on the southern California dataset (which were notably easier) as well as resulting in many less false negatives than were the output of the local smoothness model:

Early in the fires:


Later in the two fires (it does so well!):


Other team members in the pipeline plan on filtering out the sky and water via classical computer vision techniques, so a lot of the false negatives in both local smoothness as well as PCA shouldn’t be too much of an issue once the final system is assembled.

For more results, view our code, see more results, the data, and lots more I’ll direct you to the github page for the project:                           https://github.com/Duncanswilson/Smoke-Detection-Matting-Methods

If you’re looking for the paper specifically follow this link: https://github.com/Duncanswilson/Smoke-Detection-Matting-Methods/blob/master/Paper/Smoke_Detection_Paper.pdf

In any possible free time we’re hoping to implement the paper in an iPython notebook to make the code and methodology more extendable and implementing the third method in the Tian, Li paper, using independent component analysis.

If you see any errors in this or possible low hanging fruit, interested in more of the data set, or are just interested in talking to me about this work please email me! (duncanscottwilson@ymail.com)