Recent Thoughts on Neural Nets:

After attending Stanford’s Theoretical Neuroscience course yesterday I started wondering about more biologically derived methods of training neural nets, specifically incorporate  the neurogenesis , simplicity of early classifications, randomness/noise, mixtures of supervised and unsupervised learning and linear pre-processing with nonlinear processing and classification, & finally varying where the “raw” data is fed in order to obtain more general network architecture.

This has been inspired by a few sources. First was a paper Andrés Gomez said they were working through in his BCI class showing that there are roughly 20 different types of ganglion cells in the visual system some of which are doing tasks like photosensitivity, which initially due to my lack of knowledge of the visual system seemed unimportant, but this turned out to be quite a discovery. It means that simpler tasks *pre-processing* had evolutionary “reason” to be done later in the visual system. This lead me to the intuition that this may be a nice feature detection hack to use in a neural net. By taking some or possibly all of your data and feeding it into the hidden layers, there might be unexpected improvement. *highly speculative*

I had another idea of training a network with a regression problem of how salient the image is to the task versus noise filled images. It would start by estimating a real number in [0,1], with 1 being noiseless, (possibly a task for denoising autoencoders, not sure) then after it has learned what type of images it’s supposed to be learning add on an additional layer(s) and then do you classification task. Initially I thought this might be all you need to do in order for it to learn good from bad input, but that’s super silly and my non-machine learning friend immediately pointed out the flaw, the network once done with MNIST training, wouldn’t in any serious sense remember the first task of telling if a photo was salient or not. So my solution, again all of this post is just speculation, would be to add a bunch of the noisy images into the final training and augment the target vector so that the 11’th element purely corresponds to if the image is salient or not. (Would there be better performance if you train on non-digit salient images? like small pictures of random images? or better if you just left your salience sensor tuned for digits?, open questions)

The incorporating neurogenesis and mixtures of (un)supervised and (dis)continuous methods I’m much less speculative of and currently feel I have good reason to work on developing networks which use these. Sadly that means I’m currently going to keep the details to myself. If you’re reading this and are interested in collaborating, please email me and we’ll talk more.

Anyways, hopefully this summer will lead to these open questions about new neural net structure will be answered. If they have already and my searches were faulty in returning salient research that has been done, please link me up! (because barking up the wrong tree is useless only if someone else already tried barking there)

Summer Book Review Open Post

When having the time to be able to read more often I’m going to keep a general open post going of thoughts and highlights of things I’ve read this summer.

Speciesism: Why It Is Wrong, And The Implications of Rejecting It

This short work was posted on a few online groups for including a decent section on insect consideration and wild animal suffering in general. I found it to be a quick and concise read. If you’re new to the concept of sentience based moral consideration Magnus Vinding recommends you read his work Moral Truths: The Foundations of Ethics, which I haven’t read as of yet and can’t recommend. Although the general fundamental assumption I saw throughout the work was that bound experience is the only aspect or arrangement of physics which has intrinsic moral weight, whereas the rest of the universe is merely extrinsic in its spectrum of instrumental moral importance.

The book is first a guided tour through the reasons for rejecting the concept of speciesism mainly through comparisons to other mainly rejected forms of discrimination in the liberal/progressive western world: racism, sexism, or ranking moral importance of an individual by mental or physical potential. The questioning how well defined the notion of a species actually is, is also brought into question with a few different thought experiments.

Vinding also stresses veganism as our only means of being non-speciesist in our diets. Which even if you’re not making a case of intrinsic moral value in each life, veg*nism seems to me to be the most ethical and climate-considerate stance you could have with regard to what you eat. (Of course there are caveats which need to be considered)

Throughout the book the general method is to explain the speciesist aspect of some thought, ownership rights, use for food, etc.. and then replace the non-human animal in question with a human and see where moral intuition guides the reader. Probably nothing new to the animal rights activist but to someone just being introduced to the idea of speciesist thought it provides a concise and far reaching tour of the consequences of rejecting/accepting the moral consideration of all species.

After addressing the status-quo bias in most veg*n’s opposition to suffering in factory farming when applied to animals in the wild, it refreshingly touches on solutions to the problem. The two it presents are the only two fully fledged out theories to my knowledge (I’d love to learn about more, so please comment or email me any variants): reengineering or eradication.

The only point of personal contention that I had with the work was along the lines of vivisection, which Vinding claims is not morally defensible without being speciesist. But I take the more Singer-esque route and would bite the human version. Just as I feel abortion is morally permissible, I think that the sweeping magnitude of our scientific considerations should justify the inarguable exploitation of some species currently. I am hopeful of more thorough simulation software and ability to experiment non-invasively, but just as Vinding cites the eradication of rinderpest as a good intervention, he indirectly supports vivisection by nature of its result, in this case the ability to eradicate an unwanted illness.

This leads me to the only room for improvement I see in this work and that is an inclusion of stronger possible responses outside of the more strawman-like arguments posed while spelling out the validity of the argument. Especially since there is a nod to the effective altruist community in the latter pages but the EA opinion of animal issues is best summed up by David Pearce:

There is a gaping wound at the heart of the effective altruist (EA) movement. On the one hand, Peter Singer has perhaps done more than any person alive to promote the interests of non-human animals. Important strands of the EA movement give Sentience Politics in broadest sense a central role. (cf. http://sentience-politics.org/) On the other hand, Toby Ord, founder of the admirable Giving What We Can (cf. https://www.givingwhatwecan.org/) takes seriously the (to my mind transparently self-serving) “Logic of the Larder” argument.
http://www.animal-rights-library.com/texts-c/salt02.htm
Katja Grace argues at some length EA’s shouldn’t even be vegetarian. (cf. https://meteuphoric.wordpress.com/…/when-should-an…/)
MIRI’s Eliezer Yudkowsky doesn’t even believe nonhuman animals (or human babies) are conscious – which would make the whole question moot.

I very hope an EA consensus can be hammered out soon. Invoking the in vitro meat revolution offers one way forward. But how much longer until the death factories are shut and outlawed?

Overall Speciesism is a short and good read that I would highly recommend to anyone new to the ideas, with not too much to offer anyone already thoroughly acquainted with the material.

Upcoming Projects & Summer Research:

So i find it hard to balance the amount of work that I find interesting with the amount of time I actually can put into thorough and well done work. In this post I’ll be formalizing a few topics and areas of independent research I hope to make progress on this summer for two purposes:
1) to get myself to sit down and make a well-defined structure for research topics in areas of research I see as having high returns
2) to share my desires to do this work in a public format, with the hope that I’ll be stricter with my time management to follow through on the work and to  be able connect with people outside of my social circle doing similar work.

Summer Research:  
I got accepted into University of Maryland’s CS department’s Combinatorial Algorithms Applied Research for undergraduates program. I’m working on the project involving modeling pricing functions over social networks with Dr. Hajiaghayi and another student. Which hopefully will allow me to apply some of the principles I learned in game theory last semester. That starts at the beginning of June and I haven’t taken a pure algorithms class before, so in the few weeks after the semester ends I’ll be working through the majority of Dr. Roughgarden’s Coursera course, using the obvious textbook to accompany.

Independent Research: 
A friend also has been curious regarding the more specific details of Bayesian methodology to statistics and I’ve had this notebook sitting around unread for a year or two, so prior to me leaving to Las Vegas -> Maryland, we’re going to work through a few chapters and examples using PyMC. I plan on putting our work and implementations on github so the work will be extendable.

*I plan on fleshing out more of my projects here in the future. Stuff related to: AGI takeoff scenarios, the state space of qualia, generalized wada tests (total order of consciousness), negative utilitarianism, the dimensions of drug effect space, and on a less riveting topic I’ve also been thinking of making decision trees/random forests which instead of just using the Fischer Discriminant projection, they do a nonlinear projection, I want someone else to help with this work and I think it will be publishable*

Summer Book List:
– Speciesism: Why it is Wrong and the Implications of Rejecting it 
-The Effective Altruism Handbook
– Rationality: From AI to Zombies
– The Wave Function: Essays on the Metaphysics of Quantum Mechanics
– Quantum Computing Since Democritus
– I Am You: The Metaphysical Foundations for Global Ethics 

What I’ve been up to this semester:

The main project that I worked on this semester was a smoke detection matting problem for a statistical learning class taught by Dr. Raul Rojas (who made a guest professor appearance this semester at UNR). Throughout the course of the course,  we implemented different classifiers about every week.

After we got nearly up to date on statistical learning methods (excluding a lot of the techniques in deep learning, which deserve a class of their own), we split into teams to work on different sections of the problem of detecting smoke in images from the Tahoe & southern California regions.

My partner Banafsheh Rekabdar and I implemented two techniques which assume the linear separability in the images containing smoke, which was previously presented by Tian, Li et al. (http://ro.uow.edu.au/cgi/viewcontent.cgi?article=3301&context=eispapers)

Here is an example of the problem at hand:

The background image is used from the beginning of the sequences of images presented in the dataset’s from the seismology lab at UNR. We then take the red channel (as for why, we’ll get to that) then apply one of the methods presented in Tian, Li et al, we select the salient regions (in our work 16×16 pixels) that will then be passed along the the classifier at the end of the smoke detection pipeline.

Background Image:

Foreground Image:

Detection of important regions which will then be passed to the classifier (PCA is used for this example):

All of the methods used here assume the following linear combination of the background image and the smoke vector:

We also use correlation in order to check if a region is similar to what it was in the background:

There were two methods presented in Tian, Li et al which we implemented, the first was the assumption of local smoothness. (We also use attenuation and airlight assumptions from Narasimhan and Nayar (2002), for more information see the link for the paper at the bottom!) The local smoothness is based on the local structure of small groups of pixels in pictures of smoke and how similar and smooth they appear.  See more in the slide below: 

With the analytical solutions:

The problem as you’ll see in the results are there are lots of things in natural settings which also vary over time (and thus are not filtered by correlation) and are locally smooth:

Here are the full color images of a two fire case: 

Onset of both fires:

During the middle of both fires:

Local Smoothness’ guess of smoke like regions in the early case:

Local Smoothness’ guesses from the later on in the fires case:

Green plots represent the more important points, blue points less important, red even less important. (Importance here is determined by the alpha value, again see paper/github link for more specifics and results!)

The next method that was implemented was the Principal Component Analysis approach. Originally this approach seemed intractable due to the need for pure smoke images in order to determine the subspace of information rich dimensions for smoke. After emailing Tian and Li we were lucky enough to obtain their original data set and thus created our own eigensmokes!

For more information on the methodology: 

As well as analytical solutions:

PCA did wonderfully on the southern California dataset (which were notably easier) as well as resulting in many less false negatives than were the output of the local smoothness model:

Early in the fires:

Later in the two fires (it does so well!):

Other team members in the pipeline plan on filtering out the sky and water via classical computer vision techniques, so a lot of the false negatives in both local smoothness as well as PCA shouldn’t be too much of an issue once the final system is assembled.

For more results, view our code, see more results, the data, and lots more I’ll direct you to the github page for the project:                           https://github.com/Duncanswilson/Smoke-Detection-Matting-Methods

If you’re looking for the paper specifically follow this link: https://github.com/Duncanswilson/Smoke-Detection-Matting-Methods/blob/master/Paper/Smoke_Detection_Paper.pdf

In any possible free time we’re hoping to implement the paper in an iPython notebook to make the code and methodology more extendable and implementing the third method in the Tian, Li paper, using independent component analysis.

If you see any errors in this or possible low hanging fruit, interested in more of the data set, or are just interested in talking to me about this work please email me! (duncanscottwilson@ymail.com)

This is not another rationality/transhumanist/effective altruism/philosophy blog.

Alright I’m kidding, its all of them in one.

So I don’t plan on summarizing The Sequences or just spamming the same transhumanist memes that most people in these online communities are already familiar with. Rather I plan on synthesizing lots of topics which are often discussed disparately and hopefully contribute original insights as well.

My plan for content on here is to make synthesis posts, summarizing articles, monthly links, and hopefully creating a semi-active medium in which my thoughts won’t fade immediately in the case of an untimely passing.

Another value I foresee in writing publicly (as opposed to Facebook where I have found it harder to not fall into local maximum in the breadth of ideas considered) is to have an audience which will be able present flaws in my logic or give me material that I haven’t read on similar topics. My hope is to currently be wrong from the perspective of my future selves or at least have the same views but with more knowledge of why.

I’m also very new at the website/blogger etiquette & culture so I expect to learn quite a deal about all of that shortly.

Anyways hope you all enjoy and engage!

*if you have any advice regarding website setup/management/content please send it my way! *