Daily Reading List

Posted on June 22, 2017

Daily Reading is a task I always want to challenge for years. Although I also summarized papers on a specific topic, I failed to track those miscellaneous papers I read to broaden my horizon. Writing summaries is an efficient way to document my thoughts. Hope that I can keep doing it and share more papers with you. 

Notice that most papers express their ideas in an extremely succinct way. Therefore when I am summarizing, many details might be left out. If you are interested in any of those, please read the original paper on your own. My summary serves more as a memory-enhancement and a discussion on possible future works. Talks are now also included in the Reading List. 



Daily Reading List

  • [DR023] Deep Variational Information Bottleneck
    • Restrict the embedding by limiting the mutual information between the input and the encoding (don’t remember the details).
  • [DR022] Machine Theory of Mind
    • An observer is asked to predict how another agent will behave in tasks. They design some supervised tasks to train that observer. 
  • [DR021] SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient Daily Reading
    • Uses LSTM as the generator as well as the policy and CNN as the discriminator as well as the reward function. Train in a policy gradient way.
  • [DR020] Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
    • ICCV 2017> A generalized improvement of CAM (Class Activation Mapping), which can localize the most/least contributing area to a specific class in the image. Interesting insights of various tasks and networks are discovered using Grad-CAM.
  • [DR019] Efficient Estimation of Word Representations in Vector Space Daily Reading
    • arXiv 2013> Word2Vec: Use hierarchical softmax to encode the output and design a new dataset to test the embedding.
  • [DR018] iLab-20M: A large-scale controlled object dataset to investigate deep learning
    • CVPR 2016> They introduced a dataset where every object is captured under different settings and studied several questions related to the invariability of CNN feature.
  • [DR017] Understanding Neural Networks Through Deep Visualization
    • ICML workshop 2015> Visualization of activation and features. 
  • [DR016] What is wrong with convolutional neural nets?
    • MIT Talk 2014> Several arguments against CNN/pooling are given and a new model with concept “capsule” is introduced, where agreement/clustering is an important measurement.
  • [DR015] Pointer Networks
    • NIPS 2015> Apply the attention mechanism on the output of RNN so that it can generate outputs consisting of indices (positions) of the input sequence.
  • [DR014] Dual Path Networks
    • arXiv 2017> Combine ResNet with DenseNet to balance between feature re-usage and exploration.
  • [DR013] Distral: Robust Multitask Reinforcement Learning
    • arXiv 2017> Use Multi-Task Learning to improve RL by regularizing task policy over the policy space with a shared/average policy.
  • [DR012] Attention is all you need
    • arXiv 2017> Use attention mechanism to replace RNN and CNN in machine translation.
  • [DR011] Neural Turing Machine
    • arXiv 2014> Attach neural network with an external memory module.
  • [DR010] Relative Entropy Policy Search
    • AAAI 2010> Limit the divergence between the updated policy and previous policy to prevent forgetting the learned experience in RL.
  • [DR009] Learning to learn by gradient descent by gradient descent
    • NIPS 2016> Use LSTM to learn the learning rate along with the major network.
  • [DR008] Self-Normalizing Neural Networks
    • arXiv 2017> Design an activation function so that the mean and variance of the activation will converge to a fixed point.
  • [DR007] Unsupervised Pixel–Level Domain Adaptation with Generative Adversarial Networks
    • CVPR 2017> Adapt by translating images across domains.
  • [DR006] Learning Step Size Controllers for Robust Neural Network Training
    • AAAI 2016> Hand-crafted some features based on the distribution of neuron activation to adjust the learning rate.
  • [DR005] Depth Map Prediction from a Single Image using a Multi-Scale Deep Network 
    • NIPS 2014> Use a scale-invariant loss to reconstruct depth map.
  • [DR004] Asynchronous Methods for Deep Reinforcement Learning
    • ICML 2016> Have multiple agents to interact with the environment independently.
  • [DR003] Domain Separation Networks
    • NIPS 2016> Use two networks to learn shared and domain-specific feature respectively.
  • [DR002] Scalable and Sustainable Deep Learning via Randomized Hashing
    • arXiv 2016> Accelerate the max-dropout with hashing method.
  • [DR001] Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
    • NIPS 2016> Split RL into two components: the meta-controller is to choose intrinsic goals and the controller is to accomplish them.

Pending LIst:

  • Emphatic Temporal-Difference Learning
  • One model to learn all
  • Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
  • Explaining and Harnessing Adversarial Example