Fixing Day-One Update Failures on Nearly Full Orion Fold Devices

Good morning, everyone. We're sixteen days from the Orion Fold launch, and tomorrow we're supposed to submit the day-one update to the carriers. Last night, SQE found that the update fails to install on some devices. I need to decide today whether we submit tomorrow as planned, or adjust.

Hyemi, SQE. My team found it during the final update validation.

Jinwoo Baek, OTA Platform lead. My team owns the update engine and the update packages. Seyoung is our expert on how updates use storage.

Seyoung Han, OTA Platform. I've been looking at the failure logs since early this morning.

Ryan Cha, Carrier Certification. I'm here because any change to the submission date affects nine carriers, and two of them are already tight on time.

Hyemi, what exactly happens?

The update downloads and installs normally, then the device reboots to apply it. On most devices, it finishes and boots into the new version. But on some devices, after the reboot, the device reports that the update couldn't be completed, rolls back to the factory software, and boots normally. Then, a few hours later, it downloads the update and tries again, with the same result.

How many devices?

Seven out of three hundred in our validation pool, so about two point three percent. All seven had more than eighty-five percent of their storage in use. On devices with more free space, we haven't seen a single failure.

Would a brand-new device on launch day have that much storage in use? Most people take the update during setup, before they've installed anything.

Not usually, but there's one important group: people who transfer data from an old phone during setup. The transfer can easily copy a hundred gigabytes of photos and apps before the update starts. Our validation pool deliberately includes devices restored from large backups for exactly that reason.

And people upgrading from a previous foldable are exactly our target customers. So this isn't an edge case for us.

Seyoung, what do the logs show?

Let me explain how the update works first. Orion Fold uses a virtual A/B update scheme. Instead of keeping two full copies of the system, the device keeps one, and during an update, it records the changes as a snapshot in free space on the user data partition. After the reboot, the device merges the snapshot into the system. If there isn't enough free space for the snapshot, the merge can't complete, and the device rolls back to stay safe.

The update engine is supposed to check the required space before it starts, and refuse to begin if there isn't enough. That check exists. The problem is that it underestimates the space needed.

Exactly. The estimate is calculated from the update package metadata, and it assumes the snapshot is about the same size as the changed data. But the day-one update is unusually large for a delta update: one point nine gigabytes, because it includes the fingerprint fix, the audio drop, the always-on clock fix, and several framework changes. For changes spread across many files, the snapshot ends up about forty percent larger than the estimate.

So the check says there's enough room, the update starts, and halfway through the merge after the reboot, it runs out of space.

Yes. And the rollback works correctly, so no user data is lost, and the device is fine. But the user sees a failed update, and the device retries every few hours, which uses battery and data each time.

At least the safety mechanism did its job. That's a relief, but it's still not a launch experience I want.

What are the options, and which can make tomorrow's submission?

There are three, and they're not mutually exclusive. One: enable compressed snapshots. The update engine in the factory build already supports compressing the snapshot, which reduces its size by roughly sixty percent. It's a setting in the update package, not a change to the device software. Two: fix the space estimate in the update engine so it accounts for scattered changes. Three: make the update package smaller by splitting it.

If compressed snapshots are already supported, why weren't they enabled in the first place?

Compression makes the merge after the reboot slower, because the device has to decompress while it merges. On Orion, we measured the extra time at about forty seconds, and the product team preferred a faster update. For a normal monthly update, that trade-off is reasonable. For an unusually large update on nearly full devices, it isn't.

Forty seconds once, during a day-one update, is completely acceptable. Can we enable compression for this update only?

Yes. It's a flag when we generate the package. I'd regenerate the package with compression today, and SQE can rerun the full update validation on the same three hundred devices, including the seven that failed.

With compression, the snapshot for this update would be about eight hundred megabytes instead of two point six gigabytes. Even with the estimate's error, all seven failing devices would have had enough space.

Compression helps this update, but the estimate is still wrong. What happens with the next large update?

That's why I want the estimate fixed properly, but in the update engine itself, which means it ships as part of an update. The earliest it can reach devices is in this day-one update, but changing the update engine in the same update that it's installing adds risk. I'd rather fix it in the first monthly update in October.

There's also a safety improvement we can make in the update client, which is a separate app that can be updated through the app store. It could check free space with a larger margin before even downloading, and if space is tight, ask the user to free some up, instead of starting an update that's likely to fail.

That's a much better experience than a silent retry loop. Can the client change be ready before launch?

Yes. Because it's delivered through the app store, it doesn't need carrier approval. I can have it ready and tested by September 10th, more than a week before launch.

Please also stop the automatic retries after two failures. Retrying every few hours forever is worse than failing once with a clear message.

Agreed. After two failures, the client will stop, show a notification explaining that more free space is needed, and offer a shortcut to the storage settings.

Ryan, if we regenerate the package today and SQE needs time to revalidate, can we still submit tomorrow?

Full validation on three hundred devices takes about thirty-six hours. If the new package is ready by noon, we'd finish on Friday morning, September 4th. Submitting tomorrow would mean submitting before validation is complete.

I wouldn't submit something that isn't validated. If a carrier finds the problem in their lab, they'll reject it, and we'd start their review clock again. One day late is much better than a rejection.

What does one day late mean for the carriers' approval dates?

Seven of the nine carriers review a day-one update in five to seven working days, so they'd still finish well before the eighteenth. The two slowest US carriers asked for ten working days. Submitting on Friday the fourth, ten working days ends on September 18th, which is launch day itself. That's too close.

Is there anything we can do for those two?

Two things. First, I can send them tomorrow's validation results from the current package as a preview, with a note that the final package differs only in snapshot compression, which doesn't change the installed software at all. They may be willing to start their review early. Second, if they can't finish by the seventeenth, devices on those networks would install the day-one update a few days after launch. With the client fix, nobody would be stuck in a retry loop.

I can live with that. Most users on those networks would still get the factory build, which already contains the critical fixes. The day-one update is an improvement, not a rescue.

I want to acknowledge one thing. We should have caught this earlier. The package grew from one point one to one point nine gigabytes over the last two weeks as fixes were added, and nobody re-evaluated whether the default settings were still appropriate.

And on the SQE side, our update tests ran on devices with plenty of free space until the final validation. We only include the nearly full, restored devices at the very end, because setting them up takes a full day. That's backwards; the hardest cases should run first.

We could add an automatic check to the package build: if the estimated snapshot for a near-full device exceeds a threshold, the build warns us and suggests enabling compression. That would have flagged this when the package crossed one and a half gigabytes.

Good. Let's add both: the build warning, and nearly full devices in the update tests from the first candidate, not the last.

To make that practical, I'll keep a permanent pool of twenty restored, nearly full devices for update testing, instead of preparing them from scratch each time. That removes the one-day setup cost, which was the real reason we left them until the end.

Is this only an Orion Fold problem, or could the same estimate error affect updates on other devices?

The estimate logic is the same on every device that uses virtual A/B updates, which includes Orion, Orion Lite, and Nova Tab. It only becomes a problem when the update is large and the device is nearly full. Most monthly updates are between three and six hundred megabytes, so the error is small in absolute terms, and we haven't seen failures in the field.

But the Orion and Orion Lite OS upgrades in the winter will be much larger than a monthly update, probably three or four gigabytes. That's exactly the scenario where this would hurt most, on devices that have been in use for over a year and are often nearly full.

That's a very good point, and it raises the priority of the estimate fix. If it ships in October, it will be on every Orion and Orion Lite device well before the OS upgrade starts. I'll also make sure the OS upgrade packages are generated with compression by default, regardless of the estimate.

Please share that with Joanne. She's running the OS upgrade program, and Minji already flagged data migration on nearly full devices as a top risk. This fits right into that conversation.

The update client fix will help there too. If a device doesn't have enough space for a large OS upgrade, it's much better to tell the user before downloading four gigabytes than after.

Let's also think about launch day. Even with all the fixes, some users with very full devices will see the client ask them to free up space before updating. What will they see, and what should support say?

The notification will say how much space is needed, in simple terms, like 'about three gigabytes', and offer a button that opens the storage cleanup screen. That screen already suggests large files, unused apps, and cached data that can be removed safely.

I'd make sure the message doesn't sound like an error. Users who just transferred a hundred gigabytes from their old phone shouldn't feel they did something wrong. Something like 'Your new phone needs a little more space to finish setting up' is friendlier.

I like that wording. I'll share the text with Customer Experience today, so support can prepare a short answer before launch.

I'll also include the wording in the carrier submission notes. Carrier store staff often help customers with data transfer during setup, so they're likely to be the first ones to see the message.

Let me summarize. The day-one update package is regenerated with compressed snapshots by noon today. SQE revalidates on all three hundred devices, finishing Friday morning. We submit to carriers on Friday, September 4th, one day later than planned. Ryan sends the two slow US carriers a preview tomorrow. Seyoung delivers the update client fix with the larger margin and the retry limit by September 10th. Jinwoo fixes the estimate in the update engine for October. Any objections?

No objection. I'll call both carriers this afternoon rather than just emailing them. They respond much better to a phone call that explains the situation than to a formal notice that just says the date has changed.

The new package will be ready by eleven. Before I hand it to SQE, I'll install it myself on two of the devices that failed last night, so we know within an hour whether compression solves the problem, instead of waiting for the full run.

Good idea. If those two pass, I'll start the full run immediately. If they don't, we'll know early enough to rethink before the end of the day.

I'll also measure the merge time on those two devices with compression enabled. If it's much longer than forty seconds on a nearly full device, Sarah should know before we commit to this approach.

Thank you, everyone. Let's meet again on Friday, September 4th, at 10 a.m. Korea time, to confirm the validation results before we submit.