Natural languages, though seemingly easy for humans to understand, contain various aspects that make it difficult for a machine to infer and extract information from. The field of Natural Language Processing (NLP) deals with representing natural languages in a way that facilitates machines to perform various downstream tasks. This is a crucial step in human-machine interaction and opens up various other possibilities. Some of the real-life applications of NLP are seen in products like Amazon’s Alexa, Google Translate and Grammarly.
To have a bird’s eye view of the field of Natural Language Processing by implementing fundamental Language and Neural Machine Translation models.
1. Sequence Modelling:
Sequential modelling deals with predicting the next entries of a sequence given its previous entries. In the context of NLP, it deals with predicting subsequent lingual entities given the previous context. This is called Language Modelling. We implemented 2 abstractions of language modelling namely character and word level language modelling. This was done using Recurrent Neural Networks and their variants. We also compared the difference in performances between these variants.
- Character Level Language Model: A network that predicts plausible dinosaur names given its initial letter. Besides using predefined PyTorch functions, we also implemented recurrent networks from scratch.
- Word Level Language Model: A network was trained on “Harry Potter and the Philosopher’s Stone” to generate a paragraph given its starting word.
2. Neural Machine Translation:
Neural Machine translation is a task that allows machines to translate a sentence from one language to another using Neural Networks. This can be modelled as a sequence-to-sequence task and is an active area of research in the NLP community. For exploring this side of NLP, we implemented 4 basic pillars in Neural machine translation and presented a comparative study of the results.
Sequence to Sequence Learning with Neural Networks
Convolutional Sequence to Sequence Learning
Neural Machine Translation by Jointly Learning to Align and Translate [PAPER LINK]
Attention is All You Need
- Rishika Bhagwatkar
- Khurshed P. Fitter
- Aneesh A. Shetye
- Diksha Bagade
- Kshitij Ambilduke
- Prayash Swain
- Thanmay Jayakumar
- Fauzan Farooqui