NeMo Guardrails: Nvidia launches software to combat AI 'hallucination'

The "open-source software can help developers guide generative AI applications to create impressive text responses that stay on track," claims Nvidia.
Sejal Sharma
Stock photo: Nvidia launches software to combat AI ‘hallucinations.’
Stock photo: Nvidia launches software to combat AI ‘hallucinations.’


Scrambling to stay relevant and thwart competition in the artificial intelligence space, every major tech company in the world now has a chatbot. Be it Google, Microsoft, or Meta, everyone's picking up pace after OpenAI's ChatGPT. While these chatbots aren't 100% quite there yet, with a New York Times reporter claiming that his conversation with a chatbot left him 'deeply unsettled,' it's a take-your-pick kind of a situation out there.

But there's a major problem with these chatbots that's settled like a plague

It's not a new problem. AI practitioners call it 'hallucination.' Simply put, it's a situation when AI tools, which have been trained on large sets of data, churn out results that aren't real and do not match the data it has been trained on. A 'hallucinating' AI tool could, for example, create false news reports or give out a completely fabricated narrative about a historical event or a person.

In a bid to combat this issue, Nvidia, a major tech firm that deals in AI hardware and software, has developed open-source software that will help ensure that AI tools powered by large language models (LLM), like OpenAI's ChatGPT, stay on track.

Going by the name NeMo Guardrails, the software can minimize hallucinations by adding guardrails to prevent LLM systems from giving inaccurate information.

"The software includes all the code, examples, and documentation businesses need to add safety to AI apps that generate text," Nvidia said in a blog on Tuesday.

How does NeMo Guardrails software work?

As the name suggests, NeMo Guardrails is like a guard sitting between a user and a large language model-powered tool, say an AI chatbot. NeMo allows software developers to guide chatbots by adding rules with a few lines of codes, on how to interact and, more importantly, how not to react with a user. The blog says, "It natively supports LangChain, adding a layer of safety, security, and trustworthiness to LLM-based conversational applications. Developers can define user interactions and easily integrate these guardrails into any application using a Python library."

Jonathan Cohen, Nvidia's vice president of applied research, told CNBC, "You can write a script that says, if someone talks about this topic, no matter what, respond this way. You don't have to trust that a language model will follow a prompt or follow your instructions. It's actually hard coded in the execution logic of the guardrail system what will happen."

Nvidia is already one of the lead actors in the AI space, with its A100 and H100 AI chips making it a top player in the GPU market. NeMo is part of NVIDIA's expansion into the AI space and promotes the company's existing AI software.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board