cool hit counter Five of the most recommended deep learning papers of 2017_Intefrankly

Five of the most recommended deep learning papers of 2017


Source: Aurora Daily News

Short review: five of the best deep learning-related papers to read in depth in 2017, so if you haven't read them you can take action.

1. Coolest visuals: transforming between unpaired image sets using CycleGAN

essays:Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networksarxiv.org

Goal: Learn to convert between unpaired sets of images

The authors begin with two sets of images of different domains, such as horses and zebras, and learn two conversion nets: one converts horses to zebras, and the other does the opposite. Each conversion performs a style transformation, but not for an individual image style, but for an aggregated style of a set of images found in the network.

The conversion networks are trained as a pair of GenerativeAdversarialNetwork (GAN for short, a method of unsupervised learning that learns by having two neural networks play each other), each trying to trick the discriminator into believing that the 'converted' image is the real one. An additional 'cyclic consistency loss' is introduced to encourage the image to remain unchanged after two transformation networks (i.e., forward and backward).

The visuals in the paper are amazing, and it's highly recommended to check out GitHub for some other examples. I'm particularly interested in this one because, unlike many previous approaches, it learns to convert between unpaired image sets, opening the door to applications where matching image pairs may not exist. In addition, the code is very easy to use and experiment with, demonstrating the robustness of the approach and the quality of the implementation.

2. MOST ELEGANT: WASSERSTEIN DISTANCE, Better Neural Network Training

Thesis: Wasserstein GAN

arxiv.org

Goal: Use better objective functions to train GANs more consistently

This paper proposes the use of a slightly different objective function for training generative resilience networks, and the newly proposed objective function is much more stable than standard GAN training because it avoids vanishing gradients during training:.

Using the purpose of the submodification, the authors also avoid a problem known as pattern collapse, in which the standard GAN produces samples from only a subset of the possible outputs. For example, if a GAN is being trained to produce a handwritten sum of digits, the GAN may only produce, and not circumvent, this local minimum in training. This problem is avoided by eliminating the in-training target gradient, the so-called Wasserstein GAN.

This paper is very independent: the authors

Spark a simple idea

Show mathematically why the current level of technology should be improved

There was an impressive result demonstrating its effectiveness.

Furthermore, the modifications proposed by the authors are easily implemented in almost all popular deep learning frameworks, making it practical to adopt the proposed changes.

3.Most useful: unsupervised simulation training refinement data using GANS

essays:Learning from Simulated and Unsupervised Images through Adversarial Training

arxiv.org

Goal: Use real-world images to make simulation training data more useful in real-world applications

Collecting real-world data is more difficult and time-consuming. As a result, many researchers often use simulation tools that are capable of generating a virtually unlimited amount of labeled training data. However, most simulated data is not realistic enough for training deep learning systems that operate on real data.

To overcome this limitation, essays Using generative adversarial networks(GAN) to use unlabeled real-world images to improve labeled simulated images。 They train a「 fine network」 to spoof the discriminatory classifier, The classifier is trained to distinguish between delicate simulated images and real images。 Since the refinement network and classifier are trained integrally, The delicate simulated images are starting to look very realistic:

When this paper was published, I was immediately interested because it presented the first practical way to bridge the gap between simulated and real data. The key thing here is that the algorithm is unsupervised, which means that the user does not need a handwriter to label the real data. For deep learning applications, data is king, yet most academic labs like mine don't have the resources to generate the massive amounts of data needed to quickly process new areas of research: if public datasets don't exist for this problem you're trying to solve, then you're stalled on collecting and labeling relevant data. The take-away message of this paper is that as long as you have a simulator for the problem you are trying to solve, you should be able to generate the training data you need.

4.Most Impressive: Google's Go AI from the Ground Up

Goal: To learn Go without any human involvement

A best of 2017 list would not be complete without the otherwise-impressive achievements of Google's DeepMind over the years, especially his AlphaGo.

recent AlphaGo Zero essays Avoids integrating human knowledge or gameplay: It only works by「 self-play」 train, This is achieved through an improved reinforcement learning training procedure, The strategies in it will be updated as the game moves forward in the simulation, Neural networks for guided search improved during gameplay, Makes training faster。 Only in about 40 After an hour of playtime AlphaGo Zero Even more than AlphaGo Lee ( 2016 Defeated Lee Se-dol in 2007) presentable。

Although my interest in this paper is primarily focused on the engineering level, I am also encouraged by the hybrid classical and deep learning approach used by AlphaGo, in which the addition of Monte Carlo tree search allows the system to outperform monolithic neural networks. As someone who studies robotics, I am inspired by this combined approach: using classical algorithms as the backbone of decision making and using machine learning to improve performance or overcome computational limitations. Both this paper and the 2016 AlphaGo paper are great too, both well written and full of interesting technical details and insights. If for no other reason, these documents are worth reading in detail.

5.Most Thought-Provoking: Depth Images

Goal: To understand the experiments previously given to us by our neural network model.

Rather than training a deep neural network with a bunch of data, which is fairly standard these days, the authors of this paper want to explore how using neural networks themselves can help us solve some popular image processing tasks. They start with an untrained neural network, in the authors' words 'searching not in the image space for answers, but in the parameter space of the neural network', and avoid neural networks in large datasets.

I was immediately fascinated by the result: what does the structure of our neural network mean for our data? How can we better understand this? How can we use this approach to build better network models? Of course, as a group, we implicitly understand some of the limitations that our network structure imposes on our data: the CycleGAN approach is unlikely to work effectively if the 'zebra' images are all upside down. However, it raises some profound questions about our neural network models and provides some interesting directions for the coming year.

Original article: My Favorite Deep Learning Papers of 2017


Recommended>>
1、A picture to show you that blockchain is changing the world
2、Hyundais new hydrogen fuel cell car named Nexo teams up with Aurora to develop autonomous driving
3、C programming error tips for novice C beginners
4、The Stephen Hawking who appeared in essays and fillintheblank questions is really gone today
5、FUS Blockchain System Development

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送

    已发送

    朋友将在看一看看到

    确定
    分享你的想法...
    取消

    分享想法到看一看

    确定
    最多200字,当前共

    发送中

    网络异常,请稍后重试

    微信扫一扫
    关注该公众号