In 2020, BERT / GPT-3 based on the Transformer technology of natural language processing (NLP) and Deep-Fake of image generation attracted a lot of attention, while ethical issues continued to smolder in various fields. How will the AI/machine learning world change in 2021? Make five predictions with reference to several sources.
It’s the end of the year, so I’m going to do AI / Machine Learning Prediction for 2021 next year, following the article from 2019 (I think more than half of them were hit). However, I do not have the information and confidence to predict the future, so I will summarize my thoughts with the same human loincloth style as last year, with great reference to the information sources from the following sites.
We reduce the number from 10 items last year to half because the number of information sources and the amount of information was small because of the corona disaster.
- The range of use of Transformer, a method that revolutionized natural language processing (NLP), will be further expanded.
- Widespread awareness and use of machine learning-related technologies other than deep learning
- AI/machine learning ethics will grow even bigger and data and privacy regulations will be tightened
- MLS will grow further and more companies will adopt it
Let’s start with the first one. Note that the numerical order is not the order of priority/possibility, but simply the order of writing.
The range of use of Transformer, which revolutionized natural language processing (NLP), will be further expanded.
In the last year 2019, we predicted that Natural Language Processing (NLP) will make further progress and the number of use cases will increase, but I think everyone realizes that 2020 was exactly the year of NLP. The transformer is the technology that is the basis of that breakthrough. You’ve often heard the words Transformer-based BERT and GPT-3 in the news.
The transformer is trying to revolutionize not only NLP but also image recognition (Reference: A revolution in image recognition. Vision Transformer, one of the hot topics in the AI world!). This movement seems to have just begun, so I predict that it will continue in 2021 and that great results will be obtained one after another.
On the other hand, the Transformer-based NLP model has the problem of steadily bloating. For example, the number of neural network parameters was 1.5 billion in 2019 GPT-2, but, it increased sharply to 175 billionin 2020 GPT-3. Some people predict that it will exceed 1 trillion in 2021, but what will happen in reality? If the number of parameters is already too large and it reaches this level, it is quite difficult for individuals and small companies to learn new things. Partly because of this, it seems that more and more researchers are wondering if similar performance can be achieved with smaller models. For that reason, the author hopes that 2021 will not be as devoted to Transformer as it is in 2020 and that some new technology will emerge. In any case, there is no doubt that NLP will continue to be a flower-shaped field in 2021.
Expand awareness and use of machine learning-related technologies other than deep learning
Until 2019, deep learning libraries such as PyTorch and TensorFlow received a lot of attention. In 2020, a library of Autograd and XLA (Linear Algebra Compiler) called JAX appeared, and the other day, it was announced on the blog that DeepMind’s internal project is expanding the use of JAX. In 2021, there is a possibility that the use of more general-purpose libraries that are not limited to such deep learning will expand.
Also, the probabilistic programming languages (PPL) Pyro and TensorFlow Probability were introduced in many occasions in 2020 where (I have the impression). This trend will continue in 2021, and information such as Bayesian modeling may increase online and in books. In any case, the atmosphere of machine learning, which was all about deep learning, is about to collapse, and we predict that this will be spurred in 2021.
AI/machine learning ethical issues will grow and data and privacy regulations will be tightened
Last year, I wrote that the ethical problem of AI/machine learning will be even bigger. But the ethical problem is deep fake (= face-changing technology). And the fairness of the data contained in the dataset (It’s been talked about in many ways. Such as the recent dismissal of AI ethics researchers by Google (for example, only white people). And it is said that last year’s forecast was completely correct. As AI research and utilization progresses, hidden problems are likely to be revealed. So in 2021 as in 2020, news of some ethical issues will continue to flow.
As a result, we anticipate that there will be movements to tighten regulations on data and privacy. Mr. Biden will be the new President of the United States in 2021. Competition with China will strengthen AI policies. But on the other hand (to thoroughly investigate the bad points of its predecessor). Racial issues are strongly regulate against data and privacy issues. I suspect that there is a possibility of calling. Recently, Federated Learning has devised a privacy-friendly data management method that will be developed in 2021.
MLS will grow further and more companies will adopt it
Last year, he wrote, MLOps will penetrate and companies will gain great momentum. I have the impression that this is still in the middle of 2020. However, within my observation range, MLOps information will be available during 2021. By the way, according to AWS, using Sage Maker (which can mainly automate training and deployment work). Which is a service related to MLOps, productivity has increased by an order of magnitude. And the number of companies that have actually introduced it is increasing.
I can’t find any element that will make MLOps go down in the future. In 2021, MLOps-related tools/services will continue to grow, and companies will continue to adopt MLOps. By the way, @ IT / Deep Insider will continue to disseminate information in the form of MLOPs event reports. So I hope you will watch the series and use the article in the practice of MLOps.