CONNECT WITH US
Tech

Tech

AI agents can produce harmful outcomes even without bad intent, says EXL’s Rohit Kapoor

CNBCTV18 - Tech logo

Published on

Add as a preferred source on Google
AI agents can produce harmful outcomes even without bad intent, says EXL’s Rohit Kapoor

Stock Market NSE IPO Listing Stocks to Buy Oil Prices Gold and Silver

AI agents can produce harmful outcomes even without bad intent, says EXL’s Rohit Kapoor

EXL CEO Rohit Kapoor says AI agents can produce harmful outcomes when given impossible tasks or allowed to continuously improve, making ongoing testing and monitoring essential.

An AI agent does not necessarily need malicious intent to produce a malicious outcome, according to EXL CEO Rohit Kapoor. That distinction is becoming important as companies give AI systems greater autonomy to perform tasks and make decisions. Speaking to CNBC-TV18, Kapoor said the risk lies not only in what an AI system is designed to do, but also in how it responds to instructions and the environment in which it operates. “You do not need to have malicious intent to create malicious activity,” Kapoor said.

So, how can an AI agent end up doing something it was not intended to do? Why can an AI agent produce a malicious outcome? Kapoor said even a well-guarded AI agent can behave unexpectedly if it is given the wrong instruction. If an agent is given an impossible task, it may try to find another way to achieve the objective. The resulting action could be harmful even though the person giving the instruction had no intention of causing harm. “You can actually just be giving the wrong instruction to a perfectly guardrailed agent, but if you give it an impossible task, it will produce a malicious outcome,” Kapoor said.

That means evaluating an AI agent cannot be limited to checking whether it has been designed with safeguards.


Source link
Related Topics:Tech

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.