Panoptic media blog
Industry leading news and advice
Google’s NLP: Fine-Tuned Language Net
This is a unique piece of software that explores a technique known as instruction fine-tuning but this is also known as instruction tuning. Before discussing what this new feature is and how it works, it is important to understand what NLP is to start with, as this is a core part of computer science.
NLP is known as natural language processing and comes under many different branches including computer science, linguistics, and even artificial intelligence. It essentially just refers to the interaction between computers and human language with an emphasis on how to program those computers to process large amounts of natural language data. From this, those concerned with marketing should have an inkling of how NLP applies to things such as voice searches, a rising method of searching on the web today.
Google has updated their previous NLP model to what is known as FLAN – Fine-tuned Language Net. As aforementioned, this model differs from the last because of its focus on instruction tuning. Normally, fine-tuning demands a significant number of training examples, but FLAN’s instruction tuning technique means models are fine-tuned not just for the purposes of solving a specific task, but in such a way that they can solve NLP tasks too.
Google’s FLAN is carefully designed on a large set of different instructions that consist of simple descriptions for tasks. For example, the description could be something like “translate this sentence to French” or “classify this show review as good or bad”. On the other side of this, creating a whole dataset of instructions from nothing to fine-tune the model would take up too much time as well as a considerable number of resources. Instead, FLAN utilises templates to change already existing datasets into instructional formats.
The introduction of this new model already provides a few benefits over existing NLP models. FLAN can show that teaching a model on a set of instructions allows it to become great at solving the instruction it has seen when it was taught it, and also illustrates that the model is great at following instructions in general.
However, when looking at the findings of Google AI when the performance of FLAN was compared against other models, it becomes obvious that the technology still has a long way to go. This is because at smaller scales, using FLAN resulted in a decline in performance, something that was only changed when larger scales were introduced as the model was able to generalise to unseen tasks due to instructions in the training data. One reason for this may be because models that are too small do not have the required number of parameters to perform many tasks.
While being nowhere near industry standard, Google AI will be hoping that FLAN will inspire more research into other NLP models that can execute unseen tasks and divine information from a small dataset.
Turn your visitors
Let's talk and discuss your requirements