Recent Comments on the AI Arms Race
April 25, 2024
In a recent interview with CNBC, Anthropic CEO Dario Amodei was discussing the Claude 3 model — their most recent language model — and its relationship to other models (e.g., Gemini, GPT-4, Llama). He also discussed the arms race between AI models. Amodei made an interesting and important point. As we look at the different models that are coming out, we have an instinctive desire to rank them. But each of the models does some things well and some other things badly.
There isn't a single dimension along which you can compare these things. There is language fluency, task ability, conversational skills, and how well they adhere to the truth. As people are building and evaluating these systems, each of these different dimensions must be considered. One system might be unbelievably good at providing accurate information about baseline facts (which are unchanging and tend to be the facts that these systems are good at) but not as good at conversation. Another system might be good at conversation but is doing less of a job in its ability to adhere to fact. When we look at these systems, we really need to think about what the capabilities are at a much more fine-grained level than how “smart” they are. We need to consider how these systems can be integrated into larger models, what aspects of the systems can be trusted, and how they can be used with other information systems.
It's important to understand that although the baseline technologies are very similar between these systems, the material that they were trained on and the tasks that they were exposed to end up being different. That determines different skills, capabilities, and places they can be trusted and defines the ways in which they can be integrated into larger systems. So, it's not just one thing when you want to know how smart it is. It’s dozens of different dimensions.
Kristian Hammond
Bill and Cathy Osborn Professor of Computer Science
Director of the Center for Advancing Safety of Machine Intelligence (CASMI)
Director of the Master of Science in Artificial Intelligence (MSAI) Program