Machine Translation using Nvidia Nemo Megatron NLP on Rocky Linux 9

Photo by Philipp Katzenberger on Unsplash

Why Nvidia Nemo Toolkit

Nemo Megatron based Machine Translation models are trained on one of the widest corpus https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/machine_translation/machine_translation.html

The framework works on top on pytorch and has built in GPU acceleration and out of the box Distributed Data Parallel strategy (NLPDDPStrategy) for distributed training and inference.

Nemo vs Google Translate

Cherry picked ref: https://aclanthology.org/2022.clib-1.20.pdf

Google Translate (30 Nov 2023)

Nemo (es to en 1.5)

Why Rocky Linux

To deploy such models on enterprise systems, not all linux have a free to use license and qualify the safety compliance.

Rocky Linux is one such distribution. So, for deploying Nemo MT models in production on a very large scale, it can be a suitable linux dist.

In this article, I’m sharing the bare minimum code you need to run the inference on a Rocky Linux, including installing the Nemo NLP Framework.

Installation on Rocky Linux 9

There are some extra dependencies that you need to install before installing the packages mentioned in the nemo github repo

You can use this pre-requisite code:

dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
dnf install -y https://download1.rpmfusion.org/free/el/rpmfusion-free-release-8.noarch.rpm
dnf -y upgrade && dnf -y install libsndfile epel-release gcc ffmpeg
pip install wheel setuptools pip --upgrade

And then install the required nemo packages on top:

pip install Cython nemo_toolkit[nlp]

Download Model

To deploy a model to production, you would want to download it and bake it into the container and deploy on cluster, so we can download the image with this wget:

wget --content-disposition 'https://api.ngc.nvidia.com/v2/models/org/nvidia/team/nemo/nmt_es_en_transformer24x6/1.5/files?redirect=true&path=en_es_24x6.nemo' -O ./models/es_en_24x6.nemo

Inference

A minimal inference snippet:

from nemo.collections.nlp.models import MTEncDecModel

# Load model from the file downloaded in last step
model = MTEncDecModel.restore_from("./models/en_es_24x6.nemo")

# Translate a sentence or list of sentences
translations = model.translate(["Hello!"])

>>> translations = model.translate([“Hola”])
>>> translations
[‘Hello everyone!’]

Hope it was helpful :)