Harnessing the Power of Artificial Intelligence to Improve Online Proctoring
ProctorU has been a pioneer in the online proctoring industry since 2008 when we launched the most robust patented system for live online proctoring. As a company rooted in technology, we believe strongly in continuous innovation. So it is no surprise that we are now innovating with the power of artificial intelligence (AI).
Defining Artificial Intelligence
AI is loosely defined as a computerized system or machine made to simulate human intelligence. At ProctorU, we are working on a subset of AI called machine learning. One of the tenets characterizing machine learning is that an element of “learning” occurs within the system – either supervised, in which the training data contains a specific desired outcome, or unsupervised in which the training data does not contain a desired outcome.
Online Proctoring in a Nutshell
In broad terms, online proctoring can be divided into two methods: live and automated.
Live online proctoring involves a human proctor watching a test-taker through a webcam and screen-sharing program. The most important benefit to the live method is that a proctor can intervene at any sign of dishonest behavior. Learn more about live online proctoring.
Automated proctoring also utilizes a webcam and screen-sharing program, but the human proctor is replaced with a series of computer algorithms programmed to flag certain suspicious behaviors. This method requires a person to review exam session flags from a recording, and suspicious behavior can only be addressed after an exam has ended. Learn more about automated online proctoring. Learn more about automated online proctoring.
How We’re Applying AI to Proctoring
The exciting thing about innovating with machine learning technology is that our system is continuously learning, adapting and getting smarter with every exam. ProctorU’s goal in introducing AI into proctoring is not to replace humans but, rather, to strengthen the accuracy of proctoring by assisting humans in identifying details such as shadows, whispers or low sound levels, reflections, etc, that may otherwise go unnoticed.
We have focused on three areas where artificial intelligence can assist our service lines:
- 1) Identity fraud
- 2) Cheating behaviors
- 3) Content theft
ProctorU is using a supervised learning model in which human proctors are the teachers. This model can be boiled down to four steps:
This process is replicated thousands of times to build, train and fine tune every event defined in the system. For our purposes, an event is a single behavior or act indicative of identity fraud, cheating, or content theft. A data point would be evidence of a single defined behavior from an individual test-taker.
For example, someone looking off screen to the left one time would be considered a single data point and that portion of video would be segmented and labeled as such. Once 20,000+ data points of that same behavior (looking off screen to the left) are collected, the continuous event process of build, train and tune would be initiated.
Each of the thousands of events that run through the process above are further categorized as potential fraud, cheating or theft. The sum of events would then contribute to whether or not the session is marked with a suspected breach of integrity. When observing the entire process, demonstrated below, you can see it is exponentially larger than a four-step process.
How We’re Applying AI to Proctoring
ProctorU is working to build or integrate multiple machine learning technologies into the overall AI model. Here are just a few ways we’re using these technologies to strengthen our proctoring services.
Nearly every act of cheating is based on a pattern of specific behaviors. AI is incredibly adept at detecting patterns within data. Our model recognizes patterns humans can’t pick up on because of the natural limitations in the brain’s processing power.
This technology is useful in picking up the differences between sounds that should and should not be in a testing environment.
This is used in multiple capacities, from identity verification to detecting additional faces inside a testing environment.
The ability to transcribe verbal speech to written text gives us searchable data that can be indexed and referenced by the system.
Eye Movement Detection
A consistent pattern of even the most subtle eye movements can indicate misconduct. As such, this technology is crucial to the accuracy of our AI system.
This technology assists in itemizing objects present in a testing environment and detecting which are or are not permitted.
This helps determine the spatial definition of the physical environment being used by a test-taker. When paired with object recognition, our model has a full understanding of everything in a given space.
How AI is Improving Online Proctoring Services
In 2018, ProctorU became the first online proctoring company to layer artificial intelligence behind both its automated and live service lines. While it may be more clear why we would use AI in our automated service line, you may be asking why we would layer it behind our live service. There are five main benefits:
Enhanced Accuracy: Artificial intelligence acts like a second set of eyes for human proctors. The system creates an alert for anything it “sees” and, in some cases, it may catch things the human did not have time to recognize in real time.
Unmatched Security: If the system catches a negative behavior or unpermitted resource that a proctor happens to miss, we can still take immediate action against a test-taker. This adds an additional level of confidence in high-stakes exam situations in which the prevention or immediate remediation of misconduct is the preferred outcome.
Additional Scalability: The efficiencies gained through enhanced accuracy in the recognition of an aberrant behavior will allow us to proctor even more exams without sacrificing the additional security capability of human intervention.
More Data Creates a Smarter AI System: An AI model is only as good as its accuracy. For the system to be truly accurate, it requires hundreds of thousands of data points. Between our automated and live services, we proctor more than one million exams a year. All of that data is essential to the learning and training process of our AI.
Allow Humans to Do What We’re Best At: Our AI will constantly be watching and alerting humans to aberrant behaviors. Once the system becomes as accurate as (or more than) humans, there will be no need for humans to watch every single test-taker. Our role can change from that of proctor to strictly one of interventionist with a higher rate of accuracy and at a lower cost. Once this happens, there will be no need for our partners to choose between live and automated proctoring as those two methods will work in tandem, with the benefits of both, and every exam will have the same level of confidence in integrity.
The Future of AI at ProctorU
At this level of technology, the only constant is change. As new technologies become available, our AI model will evolve and adapt. We are laying a foundation that will allow us to be more agile and flexible in all of our solutions. As the methods of knowledge assessment change, we will be in a better position to adjust to the needs of our partners and test-takers alike.