Clearing Compatibility Test Failures for the Orion Fold Certification

Good morning, everyone, and good evening to Kevin in California. We're twenty-five days from the Orion Fold launch and four days from the factory build. Today's topic is certification. Without Google's approval, we can't ship with Google's apps and services, so this is just as critical as any bug. Yerin, would you like to introduce your team?

Sure. I'm Yerin, certification lead for Orion Fold. My team runs Google's compatibility test suites and prepares the submission package. Sangwoo is the engineer who's been triaging the failures this week.

Hi, Sangwoo Lee. I have the full failure list and the logs ready, so feel free to ask about any specific test.

Hyemi, SQE lead for Orion Fold. I'm here because some of the test failures may overlap with issues we're already fixing.

Kevin Albright from Google partner engineering. Happy to help. My goal is to make sure there are no surprises when your submission arrives, so the more you share now, the smoother it will go.

Yerin, where are we?

We ran the full suites on the release candidate from Friday. For the main compatibility test suite, we pass ninety-nine point six percent, which sounds great, but with almost a million test cases, that still leaves forty-two failures. For the vendor test suite, we have twelve failures. The Google apps test suite passes completely. To submit, we need every failure either fixed or formally accepted as a test issue.

Fifty-four failures, four days before the factory build. That sounds alarming. How bad is it really?

Less bad than it sounds. I've sorted the forty-two compatibility failures into three groups. Twenty-six are a single problem: a camera test that assumes the camera orientation doesn't change while the app is running. On a foldable, it does change when you unfold, so the test fails in twenty-six variations. Ten are failures in display cutout tests on the cover screen. And six are flaky: they fail once and pass when we rerun them.

The camera orientation one sounds familiar. Did the tests run with the device folded or unfolded?

Unfolded, as the test instructions require for the primary display. But the test harness itself triggers a configuration change partway through, which our device interprets as a posture change. That's when the orientation flips.

I'd be careful about calling that a test issue. Google's compatibility rules for foldables say that a configuration change that isn't caused by the user physically folding the device must not change the camera orientation. If your device flips the orientation because of a software-triggered configuration change, that's probably a real bug on your side, not a problem with the test.

That's an important distinction, and honestly, it changes my plan. I was going to file it as a test exemption request. If it's a real bug, an exemption would be rejected and we'd lose a week.

It could also be related to a bug we're already tracking. Our posture detection sometimes treats any display configuration change as a fold event. We saw it in the memory leak investigation, where the taskbar listener was being re-registered on changes that weren't real folds. Taemin's team might already have a fix that covers this.

That would be convenient. Hyemi, can you check with Taemin today?

Yes. If the fix is related, Sangwoo can rerun the camera tests on a build with it tonight.

And for the six flaky tests, the right approach is to rerun them a few times and include all the results in the submission. If they pass consistently on rerun, that's normally accepted. If one keeps failing, even occasionally, I'd want to understand why before you submit.

What about the ten cutout failures on the cover screen?

Those are more straightforward. The cover screen has a camera hole, and the system reports its position to apps so they can avoid drawing under it. The test checks that the reported position matches the actual safe area. On our cover screen, the reported position is off by a few pixels, because the configuration was copied from an early prototype panel that had the camera slightly lower.

That's a real bug, and a simple one. It's a configuration value, not code. The display team can fix it in an hour, and it should go into the factory build.

Agreed. And honestly, I'm glad the test caught it. Apps drawing a few pixels into the camera hole would look sloppy, and reviewers notice that kind of thing on the cover screen.

And the twelve vendor test failures?

All twelve are in the audio and sensor hardware abstraction layers provided by Qorvix. Eight are in a sensor test that checks how the hinge angle sensor reports its values. The test expects a continuous stream of updates, but our hinge sensor only reports when the angle changes, to save power. The other four are audio tests where the latency measurement is slightly above the limit.

For the hinge sensor, the requirement allows on-change reporting, but the sensor has to declare that mode correctly in its description. If it declares itself as continuous but behaves as on-change, the test fails. That's usually a one-line fix in the sensor configuration, and I've seen it on two other foldables.

That's very helpful, Kevin. Sangwoo, can you check the sensor declaration and ask Qorvix to confirm today?

Yes. For the audio latency, Qorvix already told us there's a fix in their next audio drop, which arrives Wednesday. It reduces latency by about two milliseconds, which should be enough to pass.

Wednesday is tight for the factory build on Friday. An audio change needs at least a day of audio regression testing on our side. If it arrives Wednesday morning, we can test it Wednesday and Thursday. If it slips to Thursday, I'd rather not put it in the factory build.

Beyond the automated tests, there's one area I always check manually for foldables: app continuity. When a user unfolds the device in the middle of using an app, the app should continue on the large screen without losing its state. Our reviewers will test about twenty popular apps by hand. Have you looked at that yet?

We have. SQE runs the top two hundred apps through fold and unfold transitions every week. Most behave well. About fifteen apps restart instead of continuing, but in every case, it's because the app itself doesn't handle configuration changes, and the same apps behave the same way on other foldables. We've reported them to the developers through our developer relations team.

That's exactly the evidence the reviewers will want to see. If you include that list in the submission, with the comparison to other devices, it shows the behavior is caused by the apps and not by your platform.

Good idea. I'll add it as an appendix. Is there anything else in the foldable requirements that commonly causes delays?

The most common one is the letterboxing behavior for apps that don't support resizing. The requirement is that those apps are displayed in a compatible aspect ratio, centered, with a way for the user to restart them in a different mode. Partners sometimes customize this and accidentally break the restart option. If your cover screen and main screen both handle it correctly, you should be fine.

I'll add a manual check for that on both screens tonight, along with the reruns. It only takes a few minutes per app.

Looking beyond Orion Fold, what should Atlas do differently, so certification doesn't come down to the last four days again?

The biggest change would be running the full compatibility suites weekly from feature complete, not only on release candidates. On Orion Fold, we ran them monthly until July, so problems like the cutout configuration sat unnoticed for months. Weekly runs would have caught it the week the prototype panel was replaced.

From SQE's side, I'd like certification failures to be tracked in the same system as our bugs, with the same severity rules. Right now they live in a separate spreadsheet, so they never show up in the weekly status reviews until someone remembers to mention them.

And I'd be happy to do an informal preview review for Atlas at feature complete, not just before the official submission. It costs us a few hours and usually saves partners weeks.

I'll pass all of that to Hyejin, who runs Atlas. Thank you, Kevin; that offer alone is worth a lot.

Let's talk about the submission itself. Kevin, how long does approval typically take?

For a complete submission with all failures either fixed or explained, the review usually takes up to ten business days. If something is missing or unclear, the clock effectively restarts when you resubmit. So the quality of the first submission matters more than how early it arrives.

Ten business days from when? If we submit on Monday, August 31st, that brings us to September 14th, four days before launch. That's cutting it close.

That's why I'd like to submit on Saturday, August 29th, right after the factory build, rather than wait until Monday. And I'd ask Kevin whether it's possible to send the results of the main suite earlier, as a preview, so his team can flag anything unclear before the official submission.

That's possible, and I'd encourage it. Send me the preliminary results as soon as you have a build with the camera and cutout fixes, and I'll do an informal review. It doesn't count as the official submission, but it reduces the risk of a resubmission.

Let me ask the uncomfortable question. If approval doesn't come by September 17th, what happens to the launch? I need to know the worst case before I talk to the launch director.

Without approval, devices can't ship with Google's apps and services, which in practice means you can't launch in any market where those are expected. I've seen partners delay a launch by a week for this reason. It's rare when the submission is complete, but it happens, usually because of a late, unexpected failure.

Our mitigation is the preliminary review and the one-page summary. If Kevin's team sees everything before the official submission, the chance of a surprise during the formal review drops a lot. I'd estimate our risk of missing the seventeenth at under ten percent, as long as we submit on the twenty-ninth.

Under ten percent is something I can explain. I'll present the twenty-ninth as a hard internal deadline, with the preliminary review as the main risk reduction.

Then SQE will treat any new certification failure between now and the twenty-ninth as a launch blocker, with the same priority as a crash.

One more question. We may need a day-one update for Orion Fold, depending on the memory leak fix. Does a day-one update need a separate approval?

Yes, but it's a lighter process. If the update only contains bug fixes and security patches, with no changes to public interfaces or Google's apps, you submit a reduced set of test results, and the review usually takes about three business days. If it changes anything that the compatibility tests cover, it needs a fuller review.

Good news on that front: the memory leak fix for the always-on clock was found on Friday, and it's in the five-day soak now. If the soak stays clean through Wednesday, it goes into the factory build, and we won't need a day-one update for the leak at all.

That would be a big relief. But let's keep the day-one update path ready anyway, in case the audio fix or anything else misses the factory build.

Let me summarize. The camera orientation failures are treated as a real bug. Hyemi checks with Taemin today whether the posture fix covers it. The cutout configuration gets fixed today. Sangwoo checks the hinge sensor declaration with Qorvix today and reruns the flaky tests. The audio fix goes into the factory build only if it arrives Wednesday morning and passes a day of regression. Yerin sends Kevin a preliminary result set as soon as the fixes are in, and submits officially on Saturday the twenty-ninth.

Understood. I'll also prepare a one-page summary that explains every remaining failure and its resolution, so the review team doesn't have to piece it together from logs.

That summary will save a lot of time on our side. Thank you. If you can, please link each failure to the build number and the fix commit, so the reviewer can confirm that the fix is actually in the submitted build.

Thank you, everyone, and especially Kevin for staying late. Our next check-in is Thursday, August 27th, at 9 a.m. Korea time, the day before the factory build, to confirm what's in and what's out. Kevin, feel free to join if your schedule allows.

I'll try. Either way, I'll watch for the preliminary results and reply within a day, even if it's only to say everything looks fine.

I'll start the reruns right after this call. I'll run each flaky test ten times on three different devices, so we can tell whether a failure is related to a specific unit or to the software itself.

And I'll post the updated failure count in the launch channel every evening until we submit, so everyone can see the number going down.