Solving the Cyber-Security Bubble

There seems to be a big bubble in cyber security. Many awful products in the market, and many bad startups easily raising funds. I believe this problem needs to be addressed, either by governmental regulations or by independent companies. In this brief post I lay out the problem and what has been done so far to mitigate this effect.
Continue reading

Imaginary neuroscience

Having some trouble sleeping lately, I have tried to understand the limits of my visual imagination. I have found it very difficult to imagine clearly moving objects without also imagining my body moves, but after some practice I could do it better.

One very interesting experiment I did, was to try and sense the difference between the two hemispheres. As you may know, each eye transports information into each of the two hemispheres, such that the right side of the field of vision is processed in the left hemisphere, and vice versa. My experiment was simple: Fix the gaze at one spot, and imagine a vertical line in the right side, moving slowly to the left. The amazing thing that I discovered was a strange feeling when the line passes through the middle. The best description to this feeling is that it feels like changing the hand holding a stick from the right to the left. I have also replicated the study with one other subject, with the same result, so it must be true!1

The cool hypothesis is that the imagination action changes hemisphere, and what I feel is the result of the information traveling through the corpus callusom, which is the pathway between the two hemispheres. The less sexy option is that it is also very possible that the muscles in my eyes are still moving a tiny bit, which feels differently when it passes through the middle, and I interpret the feeling as the movement of the stick from one hand to the other because I have actively tried to feel some difference between the hemispheres…

1. The subject was aware of previous experiments, so the results are biased šŸ™‚ā†©

Effective Global Scientific Research

Scientific advancements are one of the greatest drivers for improving the quality of life of everyone on the globe. In this post I’ll present an overview of an idea, which is basically aimed at improving the allocation of resources on scientific research such that it will correspond better to what is important for society.

We’ll start by a bit of background, and then go to the actual idea. The idea is simply to construct a graph of the important objectives in science, and how each problem relates to other problems.

Continue reading

Security of Google’s Federated Learning

In this post I’ll collect some initial thoughts regarding the security of Google’sĀ Federated Learning, which is a method for learning a model on a server where the clients do not send their data, but instead they send an updated model trained on their device with their own data. Ā The main points are:

  1. Knowing the clients update can give information on his training data.
  2. Knowing the average of some updates is likely to give information on each user’s update.
  3. If an attacker can send many updates, he can get information on a specific client.

The first two points are acknowledged briefly in the article.

Continue reading

A possible improvement for black-box adversarial example attack

This paperĀ presents a cunning adversarial example attack on an unknown DNNĀ model, with a small amount of black box calls to the model available (which happen before the input-for-deformation is given). Ā The algorithm is basically to build a different model, an adversarial DNN, with some arbitrary choice of architecture and hyper parameters, and learn the parameters on a data set given by oracle calls to the model. The choice of inputs to the oracle is made iteratively by taking the inputs from the previous iteration and choosing points close by that are the closest to the decision boundary of theĀ last learnedĀ adversarialĀ DNN.

I think it may be possible to improve the choice of the new inputs. The best choices for a new input are inputs such that they should have a big impact on the decision boundary, weighted by the probability distribution of possible inputs.

Several thoughts regarding “big impact on the decision boundary”:

  1. The work is entirely done duringĀ preprocess, as the (adversarial) model is known.
  2. Points near (at) the decision boundary are very good.
  3. A point on theĀ decision boundary can be approximated in log-time.
  4. It may be possible to find good measures to the extent that a new input has changed the decision boundary.
    1. For example, maybe a form of regularization where we motivate changing as many parameters by as much as possible is good enough. (I guess not, but it is very simple to test)

Several thoughts regarding the probability distribution of possible inputs:

  1. It seems like a very important concept to understand deeply.
  2. It is probably heavily researched.
  3. If there is an available training set, it may be possible to approximate the manifold of the probable inputs.
    1. Maybe GANs can help with this problem.

10MA – Evolution strategies VS reinforcement learning

First, go and readĀ this OpenAI blog post. Read it? good!

In the next 10 minutes, I’ll write as much as I can on my thoughts regarding the claims posed in the above mentioned post.

I have a slight cognitive dissonance.. I got used to thinking that RL is very good, and that the results obtained on the Atari games, for example, are extremely high. However, it seems that Evolution Strategies (ES), as are any type of “local search” methods, are so generic and simple, such that they should be the lowest standard for any machine learning algorithm.

Is it correct to take away from this that overall RL is just not very good, but that it’s success is mostly a story of fast supercomputers?

OpenAI mentions that these kinds of local search methods are not good for supervised learning. This means that we do have some tools which are much better than local search, but that they are not easily transferable.

A different explanation could simply be that the Atari gamesĀ and OpenAI Gym-type games, are specific examples where RL algorithms are not working well. Maybe due to their small action space?