A chatbot’s ability to learn and remember a user’s specific wants and needs, then tailor its responses accordingly, is a big part of what makes it so appealing. But that personalization isn’t just a product of sophisticated technology. It also depends on the vast amount of personal information users willingly share with chatbots. Most of the time, they might not be completely aware of the extent and implications of the data they provide.

Although conversations with AI chatbots may seem like isolated exchanges, systems like ChatGPT can retain information from past interactions and use it to personalize future responses through memory processes that aren’t always transparent to users. The AI memory then uses this to build a detailed profile of the user, a mechanism researchers in a study published earlier this year call “the algorithmic self-portrait.”

“Memory creates what we refer to as the algorithmic self-portrait —a new form of personalization derived from users’ self-disclosed information divulged within private conversations,” the researchers wrote in the study. The team included scientists from Germany’s Max Planck Institute for Software Systems and Max Planck Institute for Security and Privacy, along with several other institutions.

To better understand how these algorithmic self-portraits take shape, the researchers examined ChatGPT conversation histories donated by 80 users who obtained copies of their data under European privacy laws. They identified 2,050 memory entries across conversations from 65 of those participants.

They found that, most of the time, memories were created unilaterally by the AI chatbot. In 96% of the cases they studied, ChatGPT created or updated memories without the user explicitly asking it to do so.

Source link