The dots that you see on a picture Is real time X, Y, Z coordinate points that tracks every joint. We used Computer Vision to isolate those landmarks, striping out the video background so the model focuses only on the geometry of the hand movement However the algorithm part was not the hardest one. Getting and training data was the one. When we started training our model we needed a massive dataset, however we couldn’t find any open-source data online for Kazakh Sign Language. No GitHub reps, no Kaggle datasets. Because we couldn’t download the dataset, we had to create our own from zero. We spent days and weeks recording hundreds of videos, extracting the hand landmarks and cleaning the data. From there, the training turned into daily routine. With the expansion of data the accuracy of sign recognition rose dramatically. However sometimes it messed up some signs.