Go or No-Go on the Orion Fold Factory Build after a Fingerprint Failure

Good morning, everyone. This is the go or no-go meeting for the Orion Fold factory build, which is due to manufacturing on Friday. Yesterday afternoon, a new issue appeared that I need to understand before I can recommend a go. I've invited Manufacturing Engineering, because the issue may be in the factory process as much as in the software.

Hyemi, SQE lead. My team found the issue on units from the new production batch.

Jun Seo, Biometrics Software lead. My team owns the fingerprint driver and the enrollment flow.

Ara Moon, also Biometrics. I've been analyzing the sensor logs since last night.

Dohyun Kwak, Manufacturing Engineering, joining from the factory. Bora is with me; she manages the final calibration stations on the production line.

Hi, Bora Lim. I can answer questions about what happens to each unit before it's packed.

Before the new issue, let me quickly confirm the other items. Hyemi, the memory leak soak?

Complete and clean. Six thousand folds on four robots, and both managed and native memory stayed flat the whole time. Both leak fixes are in the factory build.

And the audio latency fix and certification?

The Qorvix audio drop arrived Wednesday morning, as hoped. Audio regression is running today and looks clean so far. On certification, the camera orientation failures turned out to be the same posture bug Taemin had already fixed, so all twenty-six now pass. Yerin sent the preliminary results to Kevin last night.

Great. So everything was on track until yesterday. Let's talk about the fingerprint issue.

Last week, manufacturing switched to a new production batch that uses fingerprint sensor modules from a second assembly line at our sensor supplier. SQE received a hundred units from that batch on Monday. On four of them, fingerprint enrollment fails completely. The user places their finger, the progress indicator never moves, and after thirty seconds, it shows an error.

Four percent. And on the previous batch?

Zero failures in more than two thousand units tested over the last three months. So this is clearly tied to the new batch.

Just to clarify, on the four failing units, does the sensor respond at all? For example, does it detect that a finger is present, even if it can't capture an image?

It detects the finger, yes. The touch event reaches the driver. It's the image capture that fails every time.

Ara, what do the logs show?

On the failing units, every captured image is rejected by our quality check as too uniform, almost blank. The sensor is returning data, but there's no fingerprint pattern in it. That usually means the sensor's gain and offset settings are wrong for that particular module. Each sensor module is slightly different, so every unit gets a calibration on the production line, and the driver loads that calibration data at boot.

So either the calibration data on those four units is wrong, or it's missing and the driver falls back to defaults that don't suit these modules.

Exactly. And the log answers that question. On all four units, the driver reports that it found calibration data, but the data has a version number it doesn't recognize. When it sees an unknown version, it ignores the data and uses defaults. On the old batch, the version was always two. On these four units, it's three.

Where does version three come from?

I think I can explain that. The second assembly line at the supplier ships its modules with a newer firmware on the sensor itself. When our calibration station runs on a module with the newer firmware, it writes the calibration in a new format, version three, because the newer firmware supports extra parameters. Nobody told us the driver needed to understand that format.

And nobody told us the format was changing. That's a process gap on both sides. But why only four percent? If all the new modules have the newer firmware, shouldn't all of them fail?

That's the part I checked this morning. Our calibration station only writes version three when it detects the newer firmware. But the supplier's second line is still ramping up, so this week's batch is a mix: about four percent of modules come from the new line, and the rest from the original line. Over the next three weeks, the share from the new line will grow to about fifty percent.

So four percent today, but potentially half of the launch units by mid-September. That changes the picture completely.

Exactly. If we don't fix this, every unit with a module from the new line will ship with a fingerprint sensor that can't enroll. The customer would discover it during setup.

Can we tell which units are affected from the production records? If we know the serial numbers, we can at least identify them.

Yes. The calibration station logs the firmware version and the calibration format for every unit. I can give you a complete list of units with version three calibration by this afternoon.

Let's talk about fixes. Jun, what are the options?

There are two places we can fix it. Option one is in the driver: teach it to understand the version three format. The supplier's documentation describes the new fields, and Ara thinks the change is about a hundred lines, mostly parsing. Option two is in the factory: change the calibration station to always write version two, even for modules with the newer firmware. That needs no software change at all.

Option two is faster, but there's a catch. The extra parameters in version three exist because the newer sensor firmware needs them to perform well. With version two calibration, the new modules would work, but in my test this morning, recognition was noticeably worse: the false rejection rate went from about two percent to about five percent. Users would have to retry more often.

So option two trades a total failure for a degraded experience on half the units. Option one gives full performance, but it's a code change two days before the factory build.

There's a third combination. Do both: switch the factory to version two immediately, so no unit ships broken, and put the driver fix for version three into the day-one update. Then, after the update, we re-run calibration in the field? Is that even possible?

Not re-calibration, no. That needs factory equipment. But the driver could derive the extra parameters from the sensor itself at first boot, using a self-test routine the new firmware provides. It's less precise than factory calibration, but much better than version two. That would be a bigger change, though, and I wouldn't want to rush it.

Let's be concrete about the risk of option one. Ara, if you write the version three parser today, how confident are you that it would be ready and tested by Friday?

The code itself, I'm confident about. The format is well documented, and the parser is isolated from the rest of the driver. Testing is the question. I'd want to test it on at least twenty new-line units and twenty old-line units, to make sure version two still works. We only have four new-line units in the lab right now.

I can help with that. Using the calibration log, I can pull thirty new-line units from today's production and send them to the lab by courier this afternoon. They'd arrive by early evening.

Then the test plan is feasible: Ara's fix by tonight, overnight enrollment and recognition tests on thirty new-line and twenty old-line units, plus the standard biometrics regression on Thursday. If everything passes, it can make Friday's factory build.

I'm comfortable with that, as long as we also switch the factory to version two as a safety net, starting today. If the driver fix fails testing, units still ship working, just with slightly worse recognition. If it passes, we can switch the factory back to version three later.

What about units that were already built and calibrated with version three this week?

From the calibration log, there are about nine hundred of them. None have shipped to customers yet; they're in the warehouse. If the driver fix makes the factory build, they'll work perfectly once flashed with the final software, since all units are flashed with the factory build before packing.

And if the fix doesn't make it?

Then we'd have to recalibrate those nine hundred units with version two. That means unpacking them and running them back through the calibration station. It's about two days of line time, which we'd rather avoid, but it's possible before the launch shipments leave on September 9th.

Good. So in the worst case, we lose two days of line time, but no customer gets a broken sensor. That's an acceptable worst case.

Let's also think about customers, just in case. If some units with weaker version two calibration reach users, what will they experience, and what should support tell them?

They'd be able to enroll and unlock, but the sensor would reject their finger about one time in twenty instead of one in fifty. Most people would just try again. Re-enrolling the same finger a second time usually helps, because the driver stores more samples.

Then the support guidance is simple: re-enroll the finger, and install the latest update when it's available. I'd still prefer we never need it.

If the driver fix misses the factory build, it would go into the day-one update, and units with version two calibration would still behave well enough. After the October update with the first-boot self-test, even those units would reach full performance.

Good. I'll prepare a short note for the support team now, so it's ready if we need it, and I'll throw it away happily if we don't.

I want to raise the process issue before we decide. The supplier changed their sensor firmware, the calibration station quietly adapted, and the driver was never told. Three parties, and nobody had the full picture. This is the same pattern we've seen all month with vendor firmware.

Agreed. From manufacturing's side, any change to a component's firmware should trigger a notification to the software team that owns the driver, before the first unit is built. I'll add that to our change control process this week.

And on the driver side, we should never silently fall back to defaults when we see unknown calibration data. If the driver had logged a clear error, or even refused enrollment with a specific message, SQE would have found this on the first unit, not the fourth.

And SQE should test new production batches as soon as they start, not just at our regular sampling interval. I'll ask manufacturing to send us units from every new supplier line on day one.

Let me state the decision. The factory build handoff on Friday is a conditional go. Starting today, the factory switches to version two calibration as a safety net. Ara delivers the version three parser tonight. SQE tests it overnight on thirty new-line and twenty old-line units, and runs the biometrics regression on Thursday. If it passes, the fix goes into Friday's factory build. If it doesn't, the factory build ships without it, and the nine hundred units are recalibrated with version two.

Manufacturing agrees. We'll change the calibration station settings before the afternoon shift, and we'll verify the change on the first ten units from the new line before running the rest.

Biometrics agrees. And the first-boot self-test that Ara mentioned goes on the list for the October update, not the launch.

SQE agrees, with one condition: I want the driver to log an explicit error for unknown calibration versions in the same change, so we never have a silent fallback again.

That's two extra lines. I'll include it. I'll also make the error message name the calibration version it found and the versions it supports, so whoever reads the log next time knows immediately what happened, without having to dig through the source code.

Action items. Ara: version three parser and the explicit error, by 9 p.m. tonight. Bora: thirty new-line units to the lab by courier this afternoon, plus the full list of version three units. Dohyun: switch the calibration station to version two before the afternoon shift, and add firmware change notifications to change control. Hyemi: overnight tests and Thursday's regression, with a recommendation by Thursday at 6 p.m. Jun: coordinate with the supplier on documentation for any future format changes. And I'll brief the launch director with the conditional go.

The units will be on the 1 p.m. courier. I'll label each one with its calibration format and module firmware version, so the lab doesn't have to look them up before testing.

Thank you, that saves a lot of time. I'll post a message in the launch channel when the fix is built, so Hyemi's team can start the overnight run as soon as the units arrive.

We'll be ready. I've already reserved the biometrics test bench for tonight and tomorrow, so nothing else will compete for it.

Thank you, everyone. We'll meet again on Thursday, August 27th, at 6 p.m. Korea time, for the final decision on the fingerprint fix. Then the factory build goes out on Friday either way.