Ben-Gurion U. develops AI system to identify social norm violations

An artificial intelligence (AI) system that identifies violations of social norms has been developed by a researcher at Ben-Gurion University of the Negev (BGU). The project – one of the first to tackle the automatic identification of social norm violations – was financed by the US Defense Advanced Research Projects Agency (DARPA). 

While many social norms exist worldwide, social-norm violation boils down to only a few general categories. Prof. Yair Neuman and his engineer Yochai Cohen built the system using GPT-3, zero-shot text classification and automatic rule discovery.

The system used a binary of 10 social emotions as categories; binary describes a numbering scheme in which there are only two possible values for each digit – 0 or 1 – and is the basis for all such code used in computer systems to understand operational instructions and user input. 

For six decades, DARPA has held to a singular and enduring mission – to make pivotal investments in breakthrough technologies for national security.

The genesis of that mission and of DARPA itself dates to the launch of Sputnik in 1957 and a commitment by the US that, from that time forward, it would be the initiator and not the victim of strategic technological surprises. Working with innovators inside and outside of government, DARPA has repeatedly delivered on that mission, transforming revolutionary concepts and even seeming impossibilities into practical capabilities. 

Game-changing military capabilities

The ultimate results have included not only game-changing military capabilities such as precision weapons and stealth technology, but also such models of modern civilian society like the internet, automated voice recognition and language translation, and global positioning System receivers small enough to embed in myriad consumer devices.

DARPA’s Computational Cultural Understanding (CCU) program was aimed at creating cross-cultural language understanding technologies to improve a Department of Defense operator’s situational awareness and interactional effectiveness. Cross-cultural miscommunication not only derails negotiations but also can be a contributing factor leading to war, according to DARPA's explanation of the rationale for the program. 

Their findings were published recently in the prestigious journal Scientific Reports under the title “AI for identifying social norm violation.” 

Neuman and his engineer trained the system to identify ten social emotions: competence, politeness, trust, discipline, caring, agreeableness, success, conformity, decency, and loyalty. The system successfully characterized a written situation under one of these 10 classifiers and could perceive if it was positive or negative. The system was tested on two massive datasets of short texts and empirically proved the validity of the models. 

“This is a preliminary work, but it provides strong evidence that our approach is correct and can be scaled up to include more social norms," said Neuman, who heads the Functor Lab in the cognitive and brain sciences/ 

DARPA explicitly reaches for transformational change instead of incremental advances. But it does not perform its engineering alchemy in isolation, he said. It works within an innovation ecosystem that includes academic, corporate, and governmental partners, with a constant focus on the Nation’s military Services, which work with DARPA to create new strategic opportunities and novel tactical options. For decades, this vibrant, interlocking ecosystem of diverse collaborators has proven to be a nurturing environment for the intense creativity that DARPA is designed to cultivate.

DARPA has some 220 government employees in six technical offices, including nearly 100 program managers who together oversee about 250 research and development programs. They work hard to identify, recruit, and support excellent program managers – very-talented people who are at the top of their fields and are eager to push the limits of their disciplines.

These leaders, who are at the very heart of DARPA’s history of success, come from academia, industry, and government agencies for limited stints, generally three to five years. That deadline fuels the signature DARPA urgency to achieve success in less time than might be considered reasonable in a conventional setting.

Program managers address challenges broadly, spanning the spectrum from deep science to systems to capabilities, but, ultimately, they are driven by the desire to make a difference. They define their programs, set milestones, meet with their performers, and assiduously track progress – but they are also constantly probing for the “next big thing” in their fields, communicating with leaders in the scientific and engineering community to identify new challenges and potential solutions.

As the number of social norms may be enormous, a simple and natural way of learning norms is through a limited number of social emotions that are evolutionarily grounded and deeply associated with a universal valuation system of human beings. “For example, when people feel shame, embarrassment, or regret, it is supposed that they acknowledge the violation of a social norm,” the authors wrote. “Therefore, the violation of a norm is accompanied and signaled by social emotions that have an important function in the ‘recalibration of social evaluation in the minds of self and others.’ These emotions are universal, although their particular expression may be culturally grounded.”