AI, Analytics, Machine Learning, Data Science, Deep Learning Research Main Developments in 2021 and Key Trends for 2022

Another year is coming to a close, and once again KDnuggets has reached out to experts for their take on what has transpired this year, and what may come to pass next.

This year, we have asked a selection of AI, Analytics, Machine Learning, Data Science, Deep Learning Research leaders the following:

What were the main developments in AI, Data Science, Machine Learning Research in 2021 and what key trends do you see for 2022?

While this article approaches the question from a research standpoint, in the next few days we will also be sharing articles which focus on the same question from both a technology and industry standpoint.

I would like to thank each of the participants in this round of opinions for taking time out of their busy schedules at such a hectic time of year to provide their insights and opinions: Anima Anandkumar, Louis Bouchard, Andriy Burkov, Charles Martin, Gaurav Menghani, Ines Montani, Dipanjan Sarkar, and Rosaria Silipo.

And now, without further delay, let's have a look at the AI, Analytics, Machine Learning, Data Science, Deep Learning Research Main Developments in 2021 and Key Trends for 2022.

 
Anima Anandkumar is Director of ML research at NVIDIA and Bren Professor at Caltech

AI4Science has matured significantly over the last year with the pandemic acting as a significant catalyst in bringing together scientists from multiple domains. We saw groundbreaking billion-atom molecular simulations to understand the Covid-19 virus and its interaction with aerosols, augmented by AI methods. We saw novel AI methods able to solve complex scientific simulations such as turbulent fluid flows for the first time. We saw many hospitals join hands and collaboratively train AI patient-care models using federated learning platforms that preserved privacy. Language models got even bigger, but widespread awareness of issues around bias has led to deeper inspection of these models and development of few-shot and fine-tuning methods to reduce harm.

 
Louis Bouchard is focused on making AI accessible on "What's AI" on YouTube and Medium

The first question is rather easy to answer for me. I am actually maintaining a GitHub repository with all the main developments in AI, so my answer would be quick: CLIP. It introduced so many exciting possibilities connecting text to images. Of course, this is the first to come to my mind but there were many more amazing discoveries and advancements this year, and I strongly invite you to check out the curated list I made, also shared here on KDnuggets, covering the most interesting AI research in 2021 with video demos, articles, and code if applicable.

I believe we will continue making a lot of exciting discoveries in image synthesis and text-to-image applications in 2022 with bigger steps than ever, and many more technological advancements will happen again. Of course, I will continue covering these exciting trends on my YouTube channel and blog if you would like to stay up-to-date with the trending research!

 
Andriy Burkov is Director of Data Science — Machine Learning Team Leader, and Author of The Hundred-Page Machine Learning Book and The Machine Learning Engineering Book

The main breakthrough in AI in 2021 were DALL·E and similar technologies that create images from text. Such technologies give an entirely new tool to creative people and democratize the creative process. In 2022 I think we will see more examples of creative AI: in video and music. Models will become bigger and we will see new multimodal models.

 
Charles Martin is an AI Specialist and Distinguished Engineer in NLP & Search

In 2021, with the pandemic still in full swing, we have seen a major uptick in online retail and general online presence, and more and more enterprises are trying to operational data science and machine learning to improve online sales and operations. This has led to an shift from pure Data Science as a siloed activity to the drive to getting more ML/AI models in production, causing more demand for ML Engineering, ML Ops ,and data-centric AI. And while traditional machine learning methods (i.e XGBoost) still dominate the enterprise, more modern AI is finding its place, with vector-space search, graph neural networks, and, of course, computer vision applications. Causal machine learning has also picked up interest as enterprises need to know why ML methods work.

In 2022, ML and AI will become more and more part of the standard software product development lifecycle, and better enterprise tools will emerge for managing their development, deployment, and monitoring.

Image by Gaurav Menghani

1. Sustainable AI: As the need for AI in various fields grows, so does its carbon footprint. Reducing this environmental impact will be critical for Sustainable AI.
2. Explainable AI: Why did this model make this specific prediction? Understanding the rationale behind model behaviour will help us understand the biases that AI brings in the world, from the fairness and ethics point of view.
3. Synthetic AI: Cadbury recently launched an Ad Campaign that empowers small business owners to create their own Ad with a popular film star promoting their grocery outlet.
4. No/Low Code AI: Companies like MindsDB are empowering users by making AI training and prediction available directly via SQL, allowing them to seamlessly harness the power of AI and predictive analysis.
5. On-Device AI: As silicon chips deliver more power per unit of energy consumed, on-device AI will begin to look a lot more attractive since it’s faster, more responsive, and more private.
6. Mission Critical AI: Current AI practices may not be suitable for mission critical applications (for safety/reliability in healthcare for example), where even the last 0.1% of accuracy matters a lot.
7. Regulation in AI: Since existing regulation won’t cover the expanding needs of AI, we will need additional regulation and governance to ensure that outfits using AI aren’t overlooking critical safeguards.

 
Ines Montani is CEO & Founder at Explosion

We've seen a lot of interesting developments in the field throughout 2021, but one thing has stood out to me the most: a steady decrease in hype-driven development. People have mostly come to accept that no, self-driving cars aren’t just around the corner, that AI won’t cure COVID, that this new model isn’t just one step short of general AI, that GPT-3 and bigger language models won’t magically solve every practical problem, and even that this one weird trick in the latest paper probably won’t help your production application.

There’s still plenty of excitement and enthusiasm, but it’s a lot more grounded, and it’s coming from the field having had much more time to mature. There are now a lot of people who’ve been working on AI and ML for several years, and the widespread acceptance of remote work throughout 2020 and 2021 has helped the right people find the right roles, to really get things done. In 2022, I think there will be much less writing that presents AI as this strange new alien thing. AI development is just software development, and it follows the same sort of trends. It’s mostly done in-house. Maintenance is a bigger expense than development. Tools are mostly open-source. And every project has its own challenges, so there are no silver bullets.

 
Dipanjan Sarkar is a Data Science Lead at Schaffhausen Institute of Technology Academy, Zurich, a Google Developer Expert in Machine Learning, a published author, and consultant

Based on my prediction last year, 2021 has definitely seen immense progress in areas of transfer learning and representation learning especially with transformers becoming the breakout tool to understand, represent and build effective solutions on unstructured data including text, images as well as audio and video. We also saw a lot of advancements being made in areas of automating machine learning training using Low-code and Auto-ML tools and the continued rise of Explainable AI and MLOps.

For 2022, I foresee the continued rise of encoder-decoder model architectures like transformers in solving tough multi-modal data problems and creating new benchmarks. We should also see more and more progress in areas of generative deep transfer learning and easy access to fine-tune these pre-trained models to solve diverse tasks using models even more powerful than GPT-3. Generative deep learning should also be something to keep a close eye on, in terms of being used in new and novel areas like data generation and content creation. Finally, automation in machine learning, data-centric machine learning and MLOps is something which will continue at a steady pace with more efficient tools being created to help us build, deploy, monitor and maintain machine learning models faster.

 
Rosaria Silipo is Head of Data Science Evangelism at KNIME

This past year was the year of AI productionization. New tools and new processes have sprout to comply, deploy, and monitor data-science-based solutions. Thanks also to this new branch of the data science life cycle, AI is now a mainstream discipline. It is not a research niche anymore, but more and more segments of the data analytics society are claiming an access to it.

Marketing analysts, nurses, physicians, CFOs, accountants, mechanical engineers, auditing professionals, and similarly specialized professional profiles, all with different backgrounds and different degrees of knowledge in coding and AI algorithms, they all need to quickly develop a data solution within an unfamiliar scope – coding, big data, or AI. In this scenario, the ease of use of low code tools can become the key to the creation of sophisticated AI solutions for non-data-science professionals.

More on a personal note, I hope that this 2022 will see a higher presence of women and other less represented categories in the data science space.

 
Related:

• AI, Analytics, Machine Learning, Data Science, Deep Learning Research Main Developments in 2020 and Key Trends for 2021
• Main 2020 Developments and Key 2021 Trends in AI, Data Science, Machine Learning Technology
• Industry 2021 Predictions for AI, Analytics, Data Science, Machine Learning