- How to Build a Knowledge Graph with Neo4J and Transformers - Nov 26, 2021.
Learn to use custom Named Entity Recognition and Relation Extraction models.
Knowledge Graph, Neo4j, Transformer
Where NLP is heading - Nov 18, 2021.
Natural language processing research and applications are moving forward rapidly. Several trends have emerged on this progress, and point to a future of more exciting possibilities and interesting opportunities in the field.
AI, Computer Vision, NLP, Transformer, Trends
- Deploying Serverless spaCy Transformer Model with AWS Lambda - Oct 22, 2021.
A step-by-step guide on how to deploy NER transformer model serverless.
AWS, Deployment, NLP, spaCy, Transformer
Surpassing Trillion Parameters and GPT-3 with Switch Transformers – a path to AGI? - Oct 1, 2021.
Ever larger models churning on increasingly faster machines suggest a potential path toward smarter AI, such as with the massive GPT-3 language model. However, new, more lean, approaches are being conceived and explored that may rival these super-models, which could lead to a future with more efficient implementations of advanced AI-driven systems.
AGI, Deep Learning, GPT-3, Transformer
- Five Key Facts About Wu Dao 2.0: The Largest Transformer Model Ever Built - Sep 6, 2021.
The record-setting model combines some clever research and engineering methods.
AI, NLP, Transformer
- DeepMind’s New Super Model: Perceiver IO is a Transformer that can Handle Any Dataset - Aug 11, 2021.
The new transformer-based architecture can process audio, video and images using a single model.
DeepMind, Modeling, Transformer
- The Best SOTA NLP Course is Free! - Jul 21, 2021.
Hugging Face has recently released a course on using its libraries and ecosystem for practical NLP, and it appears to be very comprehensive. Have a look for yourself.
Courses, Hugging Face, NLP, Transformer
- How to Train a Joint Entities and Relation Extraction Classifier using BERT Transformer with spaCy 3 - Jun 28, 2021.
A step-by-step guide on how to train a relation extraction classifier using Transformer and spaCy3.
BERT, NLP, Python, spaCy, Text Analytics, Transformer
- Fine-Tuning Transformer Model for Invoice Recognition - Jun 23, 2021.
The author presents a step-by-step guide from annotation to training.
Business Analytics, Image Classification, NLP, Python, Transformer
- Building a Knowledge Graph for Job Search Using BERT - Jun 14, 2021.
A guide on how to create knowledge graphs using NER and Relation Extraction.
BERT, Careers, Data Science Skills, Knowledge Graph, NLP, Python, Search, Transformer
- The Essential Guide to Transformers, the Key to Modern SOTA AI - Jun 10, 2021.
You likely know Transformers from their recent spate of success stories in natural language processing, computer vision, and other areas of artificial intelligence, but are familiar with all of the X-formers? More importantly, do you know the differences, and why you might use one over another?
AI, Computer Vision, Deep Learning, NLP, Transformer
- How to Fine-Tune BERT Transformer with spaCy 3 - Jun 7, 2021.
A step-by-step guide on how to create a knowledge graph using NER and Relation Extraction.
BERT, Knowledge Graph, NLP, Python, spaCy, Transformer
- How to Create and Deploy a Simple Sentiment Analysis App via API - Jun 1, 2021.
In this article we will create a simple sentiment analysis app using the HuggingFace Transformers library, and deploy it using FastAPI.
FastAPI, Hugging Face, NLP, Python, Sentiment Analysis, Transformer
- How To Generate Meaningful Sentences Using a T5 Transformer - May 3, 2021.
Read this article to see how to develop a text generation API using the T5 transformer.
API, Hugging Face, Natural Language Generation, NLP, Python, Transformer
- How to Apply Transformers to Any Length of Text - Apr 12, 2021.
Read on to find how to restore the power of NLP for long sequences.
BERT, NLP, Python, Text Analytics, Transformer
- Speech to Text with Wav2Vec 2.0 - Mar 2, 2021.
Facebook recently introduced and open-sourced their new framework for self-supervised learning of representations from raw audio data called Wav2Vec 2.0. Learn more about it and how to use it here.
Hugging Face, NLP, Python, PyTorch, Transformer
- GPT-2 vs GPT-3: The OpenAI Showdown - Feb 17, 2021.
Thanks to the diversity of the dataset used in the training process, we can obtain adequate text generation for text from a variety of domains. GPT-2 is 10x the parameters and 10x the data of its predecessor GPT.
GPT-2, GPT-3, Natural Language Generation, NLP, OpenAI, Transformer
- Hugging Face Transformers Package – What Is It and How To Use It - Feb 16, 2021.
The rapid development of Transformers have brought a new wave of powerful tools to natural language processing. These models are large and very expensive to train, so pre-trained versions are shared and leveraged by researchers and practitioners. Hugging Face offers a wide variety of pre-trained transformers as open-source libraries, and you can incorporate these with only one line of code.
Deep Learning, Hugging Face, Natural Language Generation, NLP, PyTorch, TensorFlow, Transformer, Zero-shot Learning
Getting Started with 5 Essential Natural Language Processing Libraries - Feb 3, 2021.
This article is an overview of how to get started with 5 popular Python NLP libraries, from those for linguistic data visualization, to data preprocessing, to multi-task functionality, to state of the art language modeling, and beyond.
Data Preparation, Data Preprocessing, Data Visualization, Hugging Face, NLP, Python, spaCy, Text Analytics, Transformer
- Vision Transformers: Natural Language Processing (NLP) Increases Efficiency and Model Generality - Feb 2, 2021.
Why do we hear so little about transformer models applied to computer vision tasks? What about attention in computer vision networks?
Attention, Efficiency, Modeling, NLP, Transformer
- Six Times Bigger than GPT-3: Inside Google’s TRILLION Parameter Switch Transformer Model - Jan 25, 2021.
Google’s Switch Transformer model could be the next breakthrough in this area of deep learning.
Google, GPT-3, NLP, Transformer
- OpenAI Releases Two Transformer Models that Magically Link Language and Computer Vision - Jan 11, 2021.
OpenAI has released two new transformer architectures that combine image and language tasks in an fun and almost magical way. Read more about them here.
Computer Vision, NLP, OpenAI, Transformer
- How to Incorporate Tabular Data with HuggingFace Transformers - Nov 25, 2020.
In real-world scenarios, we often encounter data that includes text and tabular features. Leveraging the latest advances for transformers, effectively handling situations with both data structures can increase performance in your models.
Data Preparation, Deep Learning, Machine Learning, NLP, Python, Transformer
- 5 Must-Read Data Science Papers (and How to Use Them) - Oct 20, 2020.
Keeping ahead of the latest developments in a field is key to advancing your skills and your career. Five foundational ideas from recent data science papers are highlighted here with tips on how to leverage these advancements in your work, and keep you on top of the machine learning game.
Data Science, Machine Learning, P-value, Research, Software, Technical Debt, Transformer
Roadmap to Natural Language Processing (NLP) - Oct 19, 2020.
Check out this introduction to some of the most common techniques and models used in Natural Language Processing (NLP).
Data Preprocessing, LDA, NLP, Python, Roadmap, Sentiment Analysis, Transformer, Word Embeddings
- Understanding Transformers, the Data Science Way - Oct 1, 2020.
Read this accessible and conversational article about understanding transformers, the data science way — by asking a lot of questions that is.
Data Science, Neural Networks, NLP, Transformer
Deep Learning’s Most Important Ideas - Sep 14, 2020.
In the field of deep learning, there continues to be a deluge of research and new papers published daily. Many well-adopted ideas that have stood the test of time provide the foundation for much of this new work. To better understand modern deep learning, these techniques cover the basic necessary knowledge, especially as a starting point if you are new to the field.
Attention, Deep Learning, GANs, History, ImageNet, Reinforcement Learning, Transformer
- A Deep Dive Into the Transformer Architecture – The Development of Transformer Models - Aug 24, 2020.
Even though transformers for NLP were introduced only a few years ago, they have delivered major impacts to a variety of fields from reinforcement learning to chemistry. Now is the time to better understand the inner workings of transformer architectures to give you the intuition you need to effectively work with these powerful tools.
Attention, Deep Learning, Hugging Face, NLP, Transformer
- Is depth useful for self-attention? - Jul 27, 2020.
Learn about recent research that is the first to explain a surprising phenomenon where in BERT/Transformer-like architectures, deepening the network does not seem to be better than widening (or, increasing the representation dimension). This empirical observation is in contrast to a fundamental premise in deep learning.
Attention, BERT, Deep Learning, Research, Scalability, Transformer
- The Unreasonable Progress of Deep Neural Networks in Natural Language Processing (NLP) - Jun 29, 2020.
Natural language processing has made incredible advances through advanced techniques in deep learning. Learn about these powerful models, and find how close (or far away) these approaches are to human-level understanding.
Deep Learning, LSTM, NLP, Transfer Learning, Transformer, Trends
20 AI, Data Science, Machine Learning Terms You Need to Know in 2020 (Part 2) - Mar 2, 2020.
We explain important AI, ML, Data Science terms you should know in 2020, including Double Descent, Ethics in AI, Explainability (Explainable AI), Full Stack Data Science, Geospatial, GPT-2, NLG (Natural Language Generation), PyTorch, Reinforcement Learning, and Transformer Architecture.
AI, Data Science, Explainability, Geospatial, GPT-2, Key Terms, Machine Learning, Natural Language Generation, Reinforcement Learning, Transformer
- Illustrating the Reformer - Feb 12, 2020.
In this post, we will try to dive into the Reformer model and try to understand it with some visual guides.
NLP, Reformer, Transformer
Top 10 AI, Machine Learning Research Articles to know - Jan 30, 2020.
We’ve seen many predictions for what new advances are expected in the field of AI and machine learning. Here, we review a “data set” based on what researchers were apparently studying at the turn of the decade to take a fresh glimpse into what might come to pass in 2020.
2020 Predictions, Adversarial, Anomaly Detection, Autoencoder, Convolutional Neural Networks, Graph Theory, NLP, Transformer, Trends
- The Future of Machine Learning - Jan 17, 2020.
This summary overviews the keynote at TensorFlow World by Jeff Dean, Head of AI at Google, that considered the advancements of computer vision and language models and predicted the direction machine learning model building should follow for the future.
2020 Predictions, Computer Vision, Machine Learning, NLP, Transformer
- A Comprehensive Guide to Natural Language Generation - Jan 7, 2020.
Follow this overview of Natural Language Generation covering its applications in theory and practice. The evolution of NLG architecture is also described from simple gap-filling to dynamic document creation along with a summary of the most popular NLG models.
LSTM, Markov Chains, Narrative Science, Natural Language Generation, Natural Language Processing, Recurrent Neural Networks, Sciforce, Transformer
- Research Guide for Transformers - Oct 30, 2019.
The problem with RNNs and CNNs is that they aren’t able to keep up with context and content when sentences are too long. This limitation has been solved by paying attention to the word that is currently being operated on. This guide will focus on how this problem can be addressed by Transformers with the help of deep learning.
BERT, NLP, Research, Transformer, ULMFiT
- BERT, RoBERTa, DistilBERT, XLNet: Which one to use? - Sep 17, 2019.
Lately, varying improvements over BERT have been shown — and here I will contrast the main similarities and differences so you can choose which one to use in your research or application.
BERT, NLP, Transformer
Deep Learning Next Step: Transformers and Attention Mechanism - Aug 29, 2019.
With the pervasive importance of NLP in so many of today's applications of deep learning, find out how advanced translation techniques can be further enhanced by transformers and attention mechanisms.
Attention, Deep Learning, NLP, Transformer
- Order Matters: Alibaba’s Transformer-based Recommender System - Aug 23, 2019.
Alibaba, the largest e-commerce platform in China, is a powerhouse not only when it comes to e-commerce, but also when it comes to recommender systems research. Their latest paper, Behaviour Sequence Transformer for E-commerce Recommendation in Alibaba, is yet another publication that pushes the state of the art in recommender systems.
Alibaba, Recommendation Engine, Recommender Systems, Transformer
- Adapters: A Compact and Extensible Transfer Learning Method for NLP - Jul 18, 2019.
Adapters obtain comparable results to BERT on several NLP tasks while achieving parameter efficiency.
BERT, NLP, Transfer Learning, Transformer
- Scaling a Massive State-of-the-art Deep Learning Model in Production - Jul 15, 2019.
A new NLP text writing app based on OpenAI's GPT-2 aims to write with you -- whenever you ask. Find out how the developers setup and deployed their model into production from an engineer working on the team.
Deep Learning, Deployment, NLP, OpenAI, Scalability, Transformer
- Pre-training, Transformers, and Bi-directionality - Jul 12, 2019.
Bidirectional Encoder Representations from Transformers BERT (Devlin et al., 2018) is a language representation model that combines the power of pre-training with the bi-directionality of the Transformer’s encoder (Vaswani et al., 2017). BERT improves the state-of-the-art performance on a wide array of downstream NLP tasks with minimal additional task-specific training.
AISC, BERT, NLP, Training, Transformer
- Examining the Transformer Architecture: The OpenAI GPT-2 Controversy - Jun 20, 2019.
GPT-2 is a generative model, created by OpenAI, trained on 40GB of Internet to predict the next word. And OpenAI found this model to be SO good that they did not release the fully trained model due to their concerns about malicious applications of the technology.
AI, Architecture, GPT-2, NLP, OpenAI, Transformer
- 10 Exciting Ideas of 2018 in NLP - Jan 16, 2019.
We outline a selection of exciting developments in NLP from the last year, and include useful recent papers and images to help further assist with your learning.
BERT, Bias, ICLR, Machine Translation, NLP, Transformer, Unsupervised Learning
- BERT: State of the Art NLP Model, Explained - Dec 26, 2018.
BERT’s key technical innovation is applying the bidirectional training of Transformer, a popular attention model, to language modelling. It has caused a stir in the Machine Learning community by presenting state-of-the-art results in a wide variety of NLP tasks.
Explained, Modeling, Neural Networks, NLP, Transformer
