Last Updated on October 26, 2022 Having seen how to implement the scaled dot-product attention and integrate it within the multi-head attention of the Transformer model, let’s progress one step further toward implementing a complete Transformer model by applying its encoder. Our end goal remains to apply the complete model to Natural Language Processing (NLP). In...
Last Updated on October 26, 2022 There are many similarities between the Transformer encoder and decoder, such as their implementation of multi-head attention, layer normalization, and a fully connected feed-forward network as their final sub-layer. Having implemented the Transformer encoder, we will now go ahead and apply our knowledge in implementing the Transformer decoder as...
Last Updated on October 26, 2022 We have arrived at a point where we have implemented and tested the Transformer encoder and decoder separately, and we may now join the two together into a complete model. We will also see how to create padding and look-ahead masks by which we will suppress the input values...
Last Updated on October 26, 2022 We have put together the complete Transformer model, and now we are ready to train it for neural machine translation. We shall use a training dataset for this purpose, which contains short English and German sentence pairs. We will also revisit the role of masking in computing the accuracy...
Last Updated on October 29, 2022 We have seen how to train the Transformer model on a dataset of English and German sentence pairs and how to plot the training and validation loss curves to diagnose the model’s learning performance and decide at which epoch to run inference on the trained model. We are now...
Last Updated on January 22, 2022 There are different ways of debugging code in Python, one of which is to introduce breakpoints into the code at points where one would like to invoke a Python debugger. The statements that one would use to enter a debugging session at different call sites, depend on the version...
Last Updated on December 24, 2021 Classes are one of the fundamental building blocks of the Python language, which may be applied in the development of machine learning applications. As we shall be seeing, the Python syntax for developing classes is simple, and can be applied to implement callbacks in Keras. In this tutorial, you...
Last Updated on December 29, 2021 Running your Python scripts is an important step in the development process, because it is in this manner that you’ll get to find out if your code works as you intended it to. It is, also, often the case that we would need to pass information to the Python...