Recent Thoughts on Neural Nets:

After attending Stanford’s Theoretical Neuroscience course yesterday I started wondering about more biologically derived methods of training neural nets, specifically incorporate  the neurogenesis , simplicity of early classifications, randomness/noise, mixtures of supervised and unsupervised learning and linear pre-processing with nonlinear processing and classification, & finally varying where the “raw” data is fed in order to obtain more general network architecture.

This has been inspired by a few sources. First was a paper Andrés Gomez said they were working through in his BCI class showing that there are roughly 20 different types of ganglion cells in the visual system some of which are doing tasks like photosensitivity, which initially due to my lack of knowledge of the visual system seemed unimportant, but this turned out to be quite a discovery. It means that simpler tasks *pre-processing* had evolutionary “reason” to be done later in the visual system. This lead me to the intuition that this may be a nice feature detection hack to use in a neural net. By taking some or possibly all of your data and feeding it into the hidden layers, there might be unexpected improvement. *highly speculative*

I had another idea of training a network with a regression problem of how salient the image is to the task versus noise filled images. It would start by estimating a real number in [0,1], with 1 being noiseless, (possibly a task for denoising autoencoders, not sure) then after it has learned what type of images it’s supposed to be learning add on an additional layer(s) and then do you classification task. Initially I thought this might be all you need to do in order for it to learn good from bad input, but that’s super silly and my non-machine learning friend immediately pointed out the flaw, the network once done with MNIST training, wouldn’t in any serious sense remember the first task of telling if a photo was salient or not. So my solution, again all of this post is just speculation, would be to add a bunch of the noisy images into the final training and augment the target vector so that the 11’th element purely corresponds to if the image is salient or not. (Would there be better performance if you train on non-digit salient images? like small pictures of random images? or better if you just left your salience sensor tuned for digits?, open questions)

The incorporating neurogenesis and mixtures of (un)supervised and (dis)continuous methods I’m much less speculative of and currently feel I have good reason to work on developing networks which use these. Sadly that means I’m currently going to keep the details to myself. If you’re reading this and are interested in collaborating, please email me and we’ll talk more.

Anyways, hopefully this summer will lead to these open questions about new neural net structure will be answered. If they have already and my searches were faulty in returning salient research that has been done, please link me up! (because barking up the wrong tree is useless only if someone else already tried barking there)


What I’ve been up to this semester:

The main project that I worked on this semester was a smoke detection matting problem for a statistical learning class taught by Dr. Raul Rojas (who made a guest professor appearance this semester at UNR). Throughout the course of the course,  we implemented different classifiers about every week.

After we got nearly up to date on statistical learning methods (excluding a lot of the techniques in deep learning, which deserve a class of their own), we split into teams to work on different sections of the problem of detecting smoke in images from the Tahoe & southern California regions.

My partner Banafsheh Rekabdar and I implemented two techniques which assume the linear separability in the images containing smoke, which was previously presented by Tian, Li et al. (

Here is an example of the problem at hand:

The background image is used from the beginning of the sequences of images presented in the dataset’s from the seismology lab at UNR. We then take the red channel (as for why, we’ll get to that) then apply one of the methods presented in Tian, Li et al, we select the salient regions (in our work 16×16 pixels) that will then be passed along the the classifier at the end of the smoke detection pipeline.

Background Image:


Foreground Image:


Detection of important regions which will then be passed to the classifier (PCA is used for this example):


All of the methods used here assume the following linear combination of the background image and the smoke vector:

Smoke Detection

We also use correlation in order to check if a region is similar to what it was in the background:

Smoke Detection  (1)

There were two methods presented in Tian, Li et al which we implemented, the first was the assumption of local smoothness. (We also use attenuation and airlight assumptions from Narasimhan and Nayar (2002), for more information see the link for the paper at the bottom!) The local smoothness is based on the local structure of small groups of pixels in pictures of smoke and how similar and smooth they appear.  See more in the slide below: Smoke Detection  (2)

With the analytical solutions:

Smoke Detection  (3)

The problem as you’ll see in the results are there are lots of things in natural settings which also vary over time (and thus are not filtered by correlation) and are locally smooth:

Here are the full color images of a two fire case: image-0002

Onset of both fires:


During the middle of both fires:


Local Smoothness’ guess of smoke like regions in the early case:


Local Smoothness’ guesses from the later on in the fires case:


Green plots represent the more important points, blue points less important, red even less important. (Importance here is determined by the alpha value, again see paper/github link for more specifics and results!)

The next method that was implemented was the Principal Component Analysis approach. Originally this approach seemed intractable due to the need for pure smoke images in order to determine the subspace of information rich dimensions for smoke. After emailing Tian and Li we were lucky enough to obtain their original data set and thus created our own eigensmokes!

Smoke Detection  (6)

For more information on the methodology: Smoke Detection  (4)

As well as analytical solutions:

Smoke Detection  (5)

PCA did wonderfully on the southern California dataset (which were notably easier) as well as resulting in many less false negatives than were the output of the local smoothness model:

Early in the fires:


Later in the two fires (it does so well!):


Other team members in the pipeline plan on filtering out the sky and water via classical computer vision techniques, so a lot of the false negatives in both local smoothness as well as PCA shouldn’t be too much of an issue once the final system is assembled.

For more results, view our code, see more results, the data, and lots more I’ll direct you to the github page for the project:                 

If you’re looking for the paper specifically follow this link:

In any possible free time we’re hoping to implement the paper in an iPython notebook to make the code and methodology more extendable and implementing the third method in the Tian, Li paper, using independent component analysis.

If you see any errors in this or possible low hanging fruit, interested in more of the data set, or are just interested in talking to me about this work please email me! (