This article is a technical blog compiled by AI Research and originally titled.

The year in AI/ML advances: 2018roundup

Author Xavier Amatriain

interpret |leogle97

calibrate | avocado (Persea americana) collate (data, files) | pineapple girl

https://medium.com/@xamat/the-year-in-ai-ml-advances-2018-roundup-db52f7f96358

AI/machine learning 2018 annual progress roundup

For me, it has become a routine to summarize the progress of machine learning at this time of the year (e.g. my answer on Quora last year). As usual, this summary is bound to be biased by my own interests and concerns, but I've tried to make it as wide-ranging as possible. Please note that the following is my blog response on Quora.

If I needed to summarize in a few lines the main highlights of machine learning in 2018, these would be the ones I might mention.

The cooling off of the artificial intelligence hype and fear-mongering.

More attention focused on specific issues, such as equity, interpretability or causality.

Deep learning has played out and is applied in practice to more than just image classification (especially natural language processing).

the battle for artificial intelligence frameworks is heating up and if you want to be a celebrity, you better release some of your own.

Let's explore them in more detail.

If 2017 may have been the peak of fear-mongering and AI hype, 2018 seems to be the year we start to calm down a bit. Admittedly, some people have continued to preach their fears about AI, but they may have been too busy with other issues to make this an important part of their agenda. Meanwhile, publishers and other media outlets seem to have calmed down, arguing that while self-driving cars and similar technologies are coming our way, they won't be born right away. Nonetheless, there are still some who are defending the bad idea that we should regulate AI instead of focusing on regulating its outcomes.

But happily, this year the focus seems to have shifted to more specific issues that can be addressed. For example, there is a lot of discussion about fairness and there are many conferences on the topic (see FATML or ACM FAT) and even some online courses from Google.

(Photo) Google's online course on equity

Along these lines, other issues that have been discussed extensively this year include explicability, interpretation and causality. From the latter, causality seems to have returned to the spotlight, largely because of the publication of Judea Pearl's book, The Book of Why. Not only did the author decide to write his first "accessible" book, but he also took to Twitter to promote a discussion on causality. Indeed, even the popular press has described it as a "challenge" to existing AI approaches (see, for example, this article in The Atlantic). In fact, even the best paper award at the ACM Recsys conference went to a paper on how to include causality in embedding (see "A proposal for causal embedding"). That said, many other authors still consider causation to be somewhat of a theoretical distraction, and we should again focus on more specific issues, such as explicitness or interpretation. Speaking of interpretations, one of the highlights of the field may be the release of papers and code on Anchor, which are follow-ups to the famous LIME model by the same author.

(Photo) Judea Pearl's Current Classics

While there are still some questions about deep learning as the most general AI model paradigm (count me in on those doubts), and while we continue to peruse the nth iteration between Yann LeCun and Gary Marcus, it's clear that deep learning not only exists, but that it's still far from where it could be. More specifically, it was a year of unprecedented success for deep learning methods in areas different from vision, from language to healthcare.

In fact, it may well be in the area of natural language processing that we've seen the most interesting developments this year. If I had to choose the most impressive AI apps of the year, they'd all be natural language processing (and they'd all come from Google). The first is Google's super useful intelligent architecture, and the second is their duplex dialogue system.

These advances were accelerated by the idea of using language models, which was popularized this year by Fast.ai's UMLFit (see "Understanding UMLFit"). Then we see other (and improved) approaches such as Allen's ELMO, Open AI's Transformers, or more recently Google's BERT which beat many SOTA results. These models have been described as "Imagenet moments for natural language processing" because they provide pre-trained generic models that can be used, which can also be fine-tuned for specific tasks. In addition to the language model, there are many other interesting improvements, such as Facebook's multilingual embedding as an example. Interestingly, we also see how quickly these and other approaches integrate into more general natural language processing frameworks, such as AllenNLP or Zalando's FLAIR.

(Fig.) BERT is deeply bidirectional, OpenAI GPT is unidirectional, and ELMo is shallowly bidirectional

When it comes to frameworks, the "AI framework war" has intensified this year. Surprisingly, just as Pytorch 1.0 was released, Pytorch seemed to be catching up with TensorFlow. While use of Pytorch in production is still not optimal, Pytorch seems to be progressing faster in this area than Tensorflow in terms of usability, documentation, and education. Interestingly, it is likely that the choice of Pytorch as a framework played a major role in the implementation of the Fast.ai library. That said, Google is aware of all of this and is pushing in the right direction, such as incorporating Keras into the framework as the top tier or adding key developer-centric leaders like Paige Bailey. In the end, we can all benefit from these great resources, so keep up the good work!

(Graph) Search volume trend of pytorch vs tensorflow

Interestingly, another in the frame space that has many interesting developments in the frame space is reinforcement learning. While I don't think the research progress in reinforcement learning is as impressive as it was in previous years (I only think of DeepMind's recent work with Impala), it is surprising that in the space of a year we have seen all the major AI manufacturers release reinforcement learning frameworks. Google has released the Dopamine framework for research, and Deepmind (also within Google) has released the TRFL framework that somewhat competes with it. Facebook can't be far behind, as it releases Horizon, while Microsoft releases TextWorld, which is better at training text-based agents. There is hope that all these benefits of open source will help us see many reinforcement learning advances in 2019.

To conclude the framework-level discussion, I was pleased to see that Google recently released TFRank on Tensor Flow. Ranking is a very important machine learning application, and it may not be getting the love it deserves these days.

It seems that deep learning has finally eliminated the need for data intelligence, but this is far from the truth. There are still some very interesting developments in the field around the idea of improving the data. For example, while data augmentation has been around for a while and is critical for many deep learning applications, this year Google released Auto Augmentation, which is a deep reinforcement learning method that automatically augments training data. A more extreme idea is to train deep learning models with synthetic data. This has been tried in practice for some time and is seen by many as the key to the future of AI. NVidia has come up with interesting new ideas in its paper on deep learning training using synthetic data. In our "Learning from Experts" we also show how to use expert systems to generate synthetic data and use it to train deep learning systems, even when combined with real data. Finally, another interesting approach is to use "weak supervision" to reduce the need for large amounts of manually tagged data. Snorkel is a very interesting project that aims to facilitate this approach by providing a generic framework.

As for more fundamental breakthroughs in AI, that may be a concern for me and mine, but I don't see too many of them. I don't entirely agree with Hinton, who says that the lack of innovation is due to the fact that the field has "some senior people and countless young people", although there is a real tendency in science for groundbreaking research to be done at a later age. In my opinion, the main reason for the current lack of breakthroughs is that there are still so many interesting practical applications for existing methods and variations that it is hard to risk adopting methods that may be less practical. This is even more important when much of the research in the field is sponsored by large corporations. In any case, an interesting paper that challenges some assumptions is "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling". Although it is highly empirical and uses known methods, it opens the door to discovering new methods because it proves that what is usually considered the best is not actually the best. To be clear, I disagree with Bored Yann LeCun's view that convolutional networks are the ultimate "master algorithm", and believe that RNNs are not either. Even sequence modeling has a lot of room for research. Another highly exploratory paper is the recent NeurIPS Best Paper Award for "God's Frequent Differential Equations," which challenges some of the basics in deep learning, including the concept of layers themselves.

Interestingly, the motivation for the paper came from a project in which the authors were studying medical data (more specifically, electronic health records). I had to mention the research at the intersection of AI and healthcare in this summary because that's where my focus is at Curai. Unfortunately, there is so much going on in this space that I need to write another post. So, I will point out the papers presented at the MLHC conference and the ML4H NeurIPS workshop. Our team at Curai has managed to get papers accepted in both places, so you will find our papers among many interesting papers that will give you an idea of what is going on in our world.

Recommended>>
1、Maotai coin listed for trading market cap over Bitcoin
2、The mechanical whore mentioned in the movie Mecha doesnt see how scary she is at all
3、WeChat public management system RhaPHP is updated
4、Talking about Digital Innovation HPE China CIO Summit Successfully Concluded
5、Pharmachem Announces Executive Changes

已发送

发送中