After the 2023 SAT reform, all international SAT test centers replaced traditional paper-based tests with digital testing. Unlike other computer-based exams such as TOEFL and ACT, the SAT Digital Test requires test-takers to bring their own devices (supporting Mac, Windows, iPad, and Chromebook) and install the Bluebook app to take the exam. The test is divided into Reading and Math sections with a 10-minute break in between.
Taking the Exam
The exam I registered for was on October 7, 2023, held at Macao Polytechnic University. At the time, I had two devices: a 16-inch Intel MacBook Pro (2019) and an iPad Pro 2022. Since the exam lasts approximately 2-3 hours, and I had extreme distrust in the Intel MBP’s battery life (it often dies after just an hour and a half when running several IDEs in class), I decided to use my iPad for the exam.
After entering the test room, connecting to the test center WiFi, everything went smoothly until waiting for the exam to begin. However, when the proctor finished reading the instructions and I entered the Start Code, my iPad crashed directly back to the home screen.
Here’s some background: after entering the exam through Bluebook, it uses iPadOS’s Automatic Assessment Configuration framework to automatically lock the device, preventing screen switching, automatic spell check, shared clipboard, and other functions.
I certainly didn’t dare touch anything randomly and quickly raised my hand to get the proctor. At this point, my iPad was in a very strange state: judging from the top status bar, my iPad was still in AAC lock mode, but had returned to the home screen.
At this point, Bluebook was stuck in an infinite crash loop. After a few cycles, the iPad automatically restarted. I then reopened Bluebook, logged in, entered the Start Code, and the interface normally entered the first Reading Section. Everything seemed normal again, so I quickly started answering questions without thinking much about it.
By this time, the exam had already started 5 minutes late. Although each person takes the SAT digital test individually, the timeline for the break didn’t align - everyone went to break 5 minutes before me, which somewhat affected my mental state.
Just like that, I completed the Reading section without major incident.
Things Get Worse
After returning from the break, I took a deep breath and prepared to start the Math section. At this point, I was still relatively calm because the first module of SAT Math mostly consists of easy questions.
However, after I calculated a fill-in-the-blank question and clicked on the input box on the screen, preparing to enter my answer, Bluebook crashed again without warning. I quickly raised my hand to call the proctor, but had to repeat the same set of procedures as when the exam started.
Fortunately, after returning to Bluebook, my time and progress were saved. I continued answering questions nervously, and when I got to another fill-in-the-blank question, it crashed again.
At this point, my mental state had completely collapsed. Several proctors gathered around me to assess my situation. Here’s the key point: one proctor told me to uninstall and reinstall Bluebook. I did as instructed, but then for Bluebook, my device became a new device that couldn’t directly enter the exam and needed proctor approval.
However, since we had already entered the second section of the exam, even though the main proctor approved the device swap request, I was faced with only a cold popup:
I was completely locked out of the exam. Even though the test center provided backup Thinkpad computers, I had no way to continue the exam. I was still in a dazed state at the time, but had to accept the fact that this trip was wasted. The exam fee might still be refundable from CB, but my round-trip flights from Beijing to Macao and hotel costs were completely wasted.
I then went to negotiate with the test center staff, who coldly stated they could do nothing, only that they would write a ticket to SAT’s organizer College Board to describe the incident (but I later found they probably never wrote it). Since the exam wasn’t over yet, they made me sit and wait until 11 o’clock before I could leave.
(This really pales in comparison to domestic test centers - at least for various large and small exams in China, emergency response plans are well-established, and they would definitely arrange a retest as soon as possible)
Post-Incident Investigation
I was completely in no mood to wander around Macao. I returned to the hotel, grabbed my luggage, and headed straight to the airport. While waiting at the airport, I suddenly remembered that every crash occurred after I clicked on a text input area. I immediately opened my laptop and after some searching, a post on Apple Developer titled “iOS 17 bug - app crashes when input element is focused when using Automatic Assessment Configuration” from a few weeks prior caught my attention:
According to the post’s description, after entering AAC lock mode in WebKit-built apps on iOS 17 devices, the app crashes when focusing on HTML <input> input controls.
At this point, I was convinced I had found the answer. My iPad was using iPad OS 17, which was released on September 18 (about half a month before the exam date). And Bluebook was clearly built using WebKit.
The truth was now clear. Although there was a Device Test session a few days before the exam, it only checked basic items like display effects and whether test papers could load normally, and didn’t actually fully enter AAC lock mode to completely simulate the exam environment.
But what I found unacceptable was that Apple had released the first developer beta of iPadOS 17 on June 6 this year, yet CB (College Board) technical staff apparently never tested their Bluebook exam software on the new system, allowing this almost 100% reproducible time bomb to exist from the June Developer Beta through 8 beta versions until the official release after Apple’s September keynote, only to explode during the actual SAT exam half a month later.
If CB’s iOS-related testing personnel had executed even one test, or paid more attention to the Apple Developer Forum, such an incident would never have happened.
Taking a step back, even though my iPad kept crashing, I could still answer questions (it seemed to only trigger the crash after focusing on the second HTML Input). If I hadn’t sought help from the proctor at the time and listened to her terrible advice to uninstall and reinstall Bluebook, but instead wrote answers on draft paper first and slowly restarted the device while filling in answers, I still could have completed the exam.
I believe I wasn’t the only one who encountered this problem. Since the Macao Polytechnic University test center was relatively small, I seemed to be the only one with this issue. But anyone using an iPad upgraded to iPad OS 17 to take the exam would likely encounter this bug.
After the exam, I also found students on Xiaohongshu who had the same experience as me. I believe this exam was very important because it was the last SAT exam before the Early Decision (ED) deadline for US college applications. I hereby strongly condemn College Board’s inaction and the arrogance of the Macao Polytechnic test center.
Follow-up
After the exam, I submitted a support ticket on the official website and sent an email explaining the situation, including my investigation results and a link to the Apple Developer Forum. Unsurprisingly, this email disappeared without a trace.
Three days after the exam, on October 10, College Board finally became aware of the situation and published this incident on Bluebook Alert. On October 25, Apple released iPad OS 17.1, which fixed this problem.
Before the November 4 exam a month later, CB sent emails to all test-takers warning them not to use devices running iPad OS 17 for the exam.
Interestingly, when I later called CB to request a refund of the exam fee, the staff member seemed completely unaware of this issue, saying “maybe because you turned off your device too quickly, only the reading section answers were sent,” but I was too lazy to explain to her - as long as I could get the money back.
Summary
Obviously, this was an accident caused by force majeure factors. I hope College Board will promptly test devices in the future and improve the pre-exam device testing procedures - when they need to simulate entering anti-screen-switching mode, they should simulate it properly and not cut corners.
For us test-takers, there may not be much we can do, but perhaps:
Don’t update your system before the exam, even if it’s an official release (otherwise you’ll become a guinea pig testing potential bugs like I did)
Try to use mainstream devices for the exam (Mac/Windows)
Resolve problems as early as possible during the exam. If I had requested to use the test center’s backup device when the Start Code crashed at the beginning, I could have avoided the accident
Don’t delete Bluebook after the exam starts! In theory, each device should have a unique identifier, but I don’t know what Bluebook uses to identify devices - after reinstalling, it becomes a new device, and no one can help you get back into the exam
Looking back, this exam was quite unlucky. I had a fever the day before the exam and lay in bed all day, and due to the hotel’s fire drill during the day with alarms blaring for several hours, I didn’t get good rest either. Then I couldn’t even take the exam :(
评论
No comments yet