Sign in Subscribe

Topic

Machine Learning

This section includes posts related to machine learning algorithm, structures, platforms, tools, and projects. The major posts of this section came from my personal study in this area of computer science.

Paper Reading Notes #03: meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting

The paper "meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting" from ICML 2017 by researchers at Peking University. This paper presents a technique to speed up machine learning model training by gradient sparsification. The key idea is sparsifying backpropagation gradients by retaining only the top-k

Paper Reading Notes #02: Towards Fully Sparse Training: Information Restoration with Spatial Similarity

Pruning is a popular technique for reducing the size of deep neural networks without sacrificing accuracy. However, traditional pruning methods can be computationally expensive and lack to hardware support. In this paper, Towards Fully Sparse Training: Information Restoration with Spatial Similarity, the authors propose a new approach to structured pruning

Paper Reading Notes #01: Attention Is All You Need

This is a new series of my notes on paper reading, covering various areas in computer architecture, algorithms, and machine learning. The first paper is Attention Is All You Need from Google that introduce self-attention mechanism and Transformer to NLP tasks. The motivation for the Google folks is that RNN

Tensorflow workflow Explanation

Data Dataset 1. Create dataset 2. Prepossess dataset 1. Mapping features and label to dataset 2. Shuffle the dataset 3. Create batched data 4. Create an iterator for loading data 5. Use the iterator for feeding data or in a input_fn for estimator Model You can build your model