AI testing company Mace AI is funding a doctoral student in AI safety at the University of Virginia’s School of Data Science, co-founder and chief scientist Greg Frank told James Heckman on T
AI testing company Mace AI is funding a doctoral student in AI safety at the University of Virginia’s School of Data Science, co-founder and chief scientist Greg Frank told James Heckman on TheStreet Roundtable. The Mace AI Fellowship in Trustworthy Computing provides an annual contribution toward a PhD student’s costs.
Frank said the research shouldn’t stay locked inside one company. “Safety should belong to everybody,” he said.
Companies are handing more work to AI agents, software that can act on its own by opening documents, sending emails or editing files. Mace AI tests whether those agents can be talked into misusing the access they’ve been given. Its product, Fisher, finds those weaknesses, recommends fixes and retests to confirm the fixes hold. Mace AI stands for Model Assurance on Cloud and Edge.
Chirag Agarwal, director of the Aikyam Lab and assistant professor of data science said, “The support from Mace AI helps further expand our work and creates opportunities for students and researchers to contribute to some of the most important challenges facing AI today.”
The university’s Aikyam Lab lists the fellowship on its website. The lab studies how to make AI systems accurate, explainable, safe and aligned with human goals.
Related: Mace Ai founders on what OpenAI's Hugging Face breach means for businesses
Frank said people have asked him why Mace AI would fund a university instead of keeping the work in-house. Part of his answer was the setting. A top university like UVA is “really an excellent environment for creative minds to come together,” he said.
His bigger reason was verification.
“But I think that more importantly, they are working on topics that we want other people to be able to inspect and independently verify,” Frank said.
He doesn’t want anyone left unsure about a method just because Mace AI owns it. “It should be in the open,” he said.
One open problem he pointed to is how to measure a “harm signal” across a swarm of agents, meaning several AI systems working together. Checking each agent separately isn’t enough. The risk has to be judged across the whole group.
“It’s a very abstract question,” Frank said. “And there needs to be some sort of academic consensus on how to do that.”
UVA’s assistant dean for research at the School of Data Science, Stephen Turner, believes that “companies and universities are giving AI agents privileged permissions faster than we can explain how those agents are making decisions. This interpretability gap is where Chirag and the Aikyam Lab come in.”
He went on to say, “a gift like this gives a doctoral student time to work on important open problems in interpretability, which is the kind of research that industry will rely on now and in the years ahead.”
The testing itself is called red teaming: attacking a system on purpose to find its weak points before real attackers do. For Mace AI, the target is an agent’s permissions.
“Can we convince that agent to step outside the scope that you gave it permission for?” Frank said.
Fisher doesn’t just replay a fixed list of known attacks. It tries many attack strategies, keeps the ones that work and combines the most successful into new attempts, a process loosely modeled on evolution. The goal is to find weaknesses nobody has written down yet.
Walton Comer, Mace AI’s CEO and co-founder, added that finding a weakness is only part of the job. The company recommends changes to a customer’s systems, he said, then tests again to see whether the fixes worked.
Comer called AI a “force multiplier for companies” that can make marketing, operations and research teams more effective. The more a company leans on agents, though, the more it needs to know what they’ll do with the access they have. Reducing those risks will take work from companies, governments and others, Comer said. If that work gets done, he expects everyone to end up “in a much better place,” with fewer vulnerabilities.
Heckman also asked what AI means for young people starting careers in finance. Frank, whose own child is applying to college, said they already “use it absolutely fluidly.”
Comer said that makes younger workers especially valuable on teams that rely on AI. Experienced professionals bring years of knowledge. Younger colleagues are simply more at home with the tools.
“I think they’re bringing in a unique set of skills that are just natural to them,” Comer said. “And so they’re actually one of the most valuable groups on a team.”
Frank sees the work itself moving up a level as AI takes on more of the detail. “So we handle higher-level tasks and hand the details down to tools.”
Related: Mace Ai founders warn of AI double agents amid banks detecting unauthorized AI agents
