While computers are very good at crunching numbers and executing logical commands, they still struggle with the nuances of the human language! Word embeddings aim to bridge that gap by constructing dense vector representations of words that capture meaning and context, to be used in downstream tasks such as question-answering and sentiment analysis. The study of word embedding techniques forms part of the field of computational linguistics. In this article we explore the challenges of modelling languages, as well as the evolution of word embeddings, from Word2Vec to BERT.

Computational Linguistics

Prior to the recent renewed interest in neural networks, computational linguistics…

This is the forth and final post in our series of blog posts focusing on the field of Natural Language Processing! In our first post, we saw that the application of neural networks for building language models was a major turning point in the NLP timeline, and in our second post we explored the significance of Word Embeddings in advancing the field. In our third post, we described the language model and how to build your own language-generating model in Keras! We are finally ready to tackle sequential processing, attention, and the Transformer!

In their highly-memorable paper titled “Attention Is…

It’s 2016. I just finished my B.Sc in Mathematical Science (Stream Computer Science) at Stellenbosch University in South Africa. Even though I had spent 3 grueling years juggling mathematics, actuarial statistics, graph theory, advanced algorithms, numerical methods, parallel programming, requirements specifications and web development, I felt no more prepared for the working world than I was when I was a first year. I had completed exactly one 3-week internship at a financial institution. And honestly, I did not know where to go next. Computer science taught me a set of valuable skills, rather than a vocation. I knew I did…

I was only about 5 years old when the DotCom Bubble took effect, and while the DotCom Bubble was recent enough to live in most people’s memories and not in the dusty history books, in the technology age 20 years is a millennium. Just look at that billboard, it is practically archaic! The DotCom Bubble highlighted the pitfalls of greed, over-promising, and ignorance. It also proves an interesting case study for the intricate relationship between innovation, and economic growth.

A DotCom company was called as such because many of them simply consisted of a website. They were online platforms that…

“Never give in, never give in, never, never, never, never give in. Nothing, great or small, large or petty, never give in except to convictions of honor and good sense. Never yield to force; never yield to the apparently overwhelming might of the enemy.” — Winston Churchill

We are often bombarded by the successes of others, resulting in this overwhelming feeling of “not keeping up” with our peers. The way our world is set up is that we only share our successes, and not the million wrong turns we took before we ended up there. …

This article is based on the paper:

Jeanne E Daniel, Willie Brink, Ryan Eloff, Charles Copley. 2019. “Towards Automating Healthcare Question Answering in a Noisy Multilingual Low-Resource Setting. ” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

Q: What are the signs of labor?

A: Signs of labor include a jelly-like discharge, your water breaking, and regular and painful labor contractions. Make sure you can get to a hospital.

TL;DR: We gained access to a fairly large, anonymized, noisy, multilingual question-answering dataset in the maternal healthcare domain (MomConnect). Our goal was to investigate ways in…

I was fortunate enough to recently attend the Forum for Artificial Intelligence Research (FAIR) 2018, hosted in Hermanus by CAIR. This was the first conference I have attended that placed a very serious emphasis on the Ethics of AI, with quite a few speakers coming from Philosophy backgrounds. Topics that were discussed included ethical issues surrounding Robot Rights and Citizenship, Robot Prisons, Artifical Moral Agency, Lethal Autonomous Weapon Systems (LAWS), and more.

Adversarial Examples

One of the speakers, Ettiene Barnard, a leading machine-learning researcher in South Africa, made the bold statement that “Deep Learning is both great … and pathetic”, by using…