Daily Reading List

Daily Reading is a task I always want to challenge for years. Although I also summarized papers on a specific topic, I failed to track those miscellaneous papers I read to broaden my horizon. Writing summaries is an efficient way to document my thoughts. Hope that I can keep doing it and share more papers with you. 

Notice that most papers express their ideas in an extremely succinct way. Therefore when I am summarizing, many details might be left out. If you are interested in any of those, please read the original paper on your own. My summary serves more as a memory-enhancement and a discussion on possible future works. Talks are now also included in the Reading List. 

Daily Reading List

• [DR023] Deep Variational Information Bottleneck
  • Restrict the embedding by limiting the mutual information between the input and the encoding (don’t remember the details).
• [DR022] Machine Theory of Mind
  • An observer is asked to predict how another agent will behave in tasks. They design some supervised tasks to train that observer. 
• [DR021] SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient Daily Reading
  • Uses LSTM as the generator as well as the policy and CNN as the discriminator as well as the reward function. Train in a policy gradient way.
• [DR020] Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
  • ICCV 2017> A generalized improvement of CAM (Class Activation Mapping), which can localize the most/least contributing area to a specific class in the image. Interesting insights of various tasks and networks are discovered using Grad-CAM.
• [DR019] Efficient Estimation of Word Representations in Vector Space Daily Reading
  • arXiv 2013> Word2Vec: Use hierarchical softmax to encode the output and design a new dataset to test the embedding.
• [DR018] iLab-20M: A large-scale controlled object dataset to investigate deep learning
  • CVPR 2016> They introduced a dataset where every object is captured under different settings and studied several questions related to the invariability of CNN feature.
• [DR017] Understanding Neural Networks Through Deep Visualization
  • ICML workshop 2015> Visualization of activation and features. 
• [DR016] What is wrong with convolutional neural nets?
  • MIT Talk 2014> Several arguments against CNN/pooling are given and a new model with concept “capsule” is introduced, where agreement/clustering is an important measurement.
• [DR015] Pointer Networks
  • NIPS 2015> Apply the attention mechanism on the output of RNN so that it can generate outputs consisting of indices (positions) of the input sequence.
• [DR014] Dual Path Networks
  • arXiv 2017> Combine ResNet with DenseNet to balance between feature re-usage and exploration.
• [DR013] Distral: Robust Multitask Reinforcement Learning
  • arXiv 2017> Use Multi-Task Learning to improve RL by regularizing task policy over the policy space with a shared/average policy.
• [DR012] Attention is all you need
  • arXiv 2017> Use attention mechanism to replace RNN and CNN in machine translation.
• [DR011] Neural Turing Machine
  • arXiv 2014> Attach neural network with an external memory module.
• [DR010] Relative Entropy Policy Search
  • AAAI 2010> Limit the divergence between the updated policy and previous policy to prevent forgetting the learned experience in RL.
• [DR009] Learning to learn by gradient descent by gradient descent
  • NIPS 2016> Use LSTM to learn the learning rate along with the major network.
• [DR008] Self-Normalizing Neural Networks
  • arXiv 2017> Design an activation function so that the mean and variance of the activation will converge to a fixed point.
• [DR007] Unsupervised Pixel–Level Domain Adaptation with Generative Adversarial Networks
  • CVPR 2017> Adapt by translating images across domains.
• [DR006] Learning Step Size Controllers for Robust Neural Network Training
  • AAAI 2016> Hand-crafted some features based on the distribution of neuron activation to adjust the learning rate.
• [DR005] Depth Map Prediction from a Single Image using a Multi-Scale Deep Network 
  • NIPS 2014> Use a scale-invariant loss to reconstruct depth map.
• [DR004] Asynchronous Methods for Deep Reinforcement Learning
  • ICML 2016> Have multiple agents to interact with the environment independently.
• [DR003] Domain Separation Networks
  • NIPS 2016> Use two networks to learn shared and domain-specific feature respectively.
• [DR002] Scalable and Sustainable Deep Learning via Randomized Hashing
  • arXiv 2016> Accelerate the max-dropout with hashing method.
• [DR001] Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
  • NIPS 2016> Split RL into two components: the meta-controller is to choose intrinsic goals and the controller is to accomplish them.

Pending LIst:

• Emphatic Temporal-Difference Learning
• One model to learn all
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Explaining and Harnessing Adversarial Example