As of last year, some California residents are able to take the written part of their driving test online—theoretically cutting down the time spent at an infamously disliked public institution.
There are some slightly invasive measures officials take to prevent cheating: You have to give the Department of Motor Vehicles access to your webcam to record test-taking and share your screen during the test so that you can’t pull up, say, a copy of the DMV manual midway through the exam. You also have to declare, under penalty of perjury, that you—and only you—are answering the questions on the test. (More on that later.)
Yet even with those safeguards, I personally witnessed an artificial intelligence agent in San Francisco last week that successfully passed the California driver’s test—with some help from a human operator.
It’s a testament to the possibilities of the emergent technology, while also providing ammunition for the argument that AI could be used for more nefarious purposes.
The AI agent in question is from a startup called MultiOn, and it was co-founded by Div Garg.
The Stanford doctoral program dropout initially started MultiOn as a way to automate routine tasks, like ordering toilet paper from Amazon or scheduling calendar appointments. It can easily order food on Uber Eats or make a Resy reservation, for instance, because it controls the user’s browser and does it for them, rather than plugging into the back-end technology.
The service, for now, is free while users wait to join the beta. Currently, the AI agent sits on the bottom-right of the browser—where you can type in its commands, not entirely unlike Microsoft’s Bing ChatGPT assistant.
Most of the capabilities offered by the “Personal AI Agent” are useful, if not exactly life-altering. During a demonstration of MultiOn at a San Francisco coffee shop last month, Garg used it to send mass “happy birthday” messages to Facebook acquaintances.
But he also hinted at the agent’s greater potential and used MultiOn to set up our next meeting to demonstrate passing an official DMV driving exam.
How, exactly, was it able to accomplish such a feat? The robot needed a paragraph-long prompt in order to select the correct answer and then click through to the next question without visibly notifying the user or alerting the anti-cheating measures.
To be fair, the program still needed human fingers at the ready in case something went wrong. The parameters that were set, oddly enough, meant that MultiOn halted to a standstill when it got a question wrong. The human test-taker, who declined to be identified, had to press the “Continue” button themselves.
It also couldn’t parse through images, meaning any question that required the AI bot to identify a traffic sign was answered incorrectly. One key current contradiction in AI agents: It can get tough questions about proper driving protocol right, but it can’t click a button to get to the next question without an exacting prompt to tell it to.
“This was riding on a lot of luck,” Garg explained, chuckling, adding that this was the first time MultiOn ever took the official state examination. Ultimately, MultiOn got only five or six of the 46 questions wrong, officially meeting the standard for passage.
Of course, using the technology in this way may constitute perjury under California law. Before taking the test, a warning pops up stating: “I declare under penalty of perjury that I will personally answer the following questions.”
The California DMV did not respond to a request for comment regarding the legality of this test-taking service. A department spokesperson wrote in a statement to The Standard, “As a fraud prevention measure, online test participants are required to verify their identity and agree to be monitored throughout the exam. The DMV continues to update the safeguards as technology evolves.” The spokesperson also pointed to a no-fail “eLearning” course for those looking to renew their licenses that all but makes this experiment a moot point.
The Future of AI Assistants
Even before outwitting the DMV, Garg said MultiOn has already caught the attention of OpenAI and its CEO, Sam Altman.
“We’re working very closely with them,” Garg said. “I got a chance to talk with Sam and a lot of the people, and they’re really supportive of what we’re working on. We have a direct line to what they’re building.”
But for now, he says, MultiOn is taking a cautious approach to the rollout, suggesting that this successful test is more a proof of concept than a key feature. “We want to make sure that we can moderate it so we can make sure there’s no malicious use cases that are happening,” Garg said. He’d rather people not cheat on exams of all kinds, so he plans on disabling any “gray zone” features like this one for the general user base.
Unlike OpenAI’s GPT assistants, which exist primarily in the sandbox that is ChatGPT, the MultiOn assistant operates autonomously as a Google Chrome browser extension. You must grant it permission to effectively take control of your computer.
“Our technology runs directly on someone’s computer,” Garg said. “It’s literally controlling that, doing things, and so many things can go wrong.”
However, people want in: Around 30,000 people, Garg said, have signed up to try out a beta version of the app. He anticipates more things to come for MultiOn, like a beefed-up mobile voice assistant and the possibility of a browser with MultiOn technology built in. (The likelihood that this can be used to cheat on tests like the SATs or ACTs, which have recently rolled out digital formats, is also low—both testing agencies require students to go to a test center.)
For now, passing a state-conducted test—and maybe violating state law—is all in a day’s work for this AI agent.