How to Measure Fluency: 14 Months of Cold-Start Data

116,434 dictation transactions  ·  one learner  ·  zero to C1 in Romanian  ·  2026

The only honest measure of language fluency is your success rate on audio you have never heard before. Not your overall practice score. Not your streak. Not how comfortable you feel. Your raw performance on genuinely new input — cold, no context, no prior exposure.

This page presents 14 months of that data, from the first dictation session in March 2021 through the point in April–May 2022 when C1 fluency in Romanian was reached. The data comes from 116,434 real transactions, logged automatically by the app as it was used to learn the language.

The thesis is simple: a cold-start success rate above 70% on native Romanian audio corresponds to functional C1 fluency. Here is the evidence.

The Idea, Written in Real Time

The metric was not invented after the fact to explain a result. It was the design principle from the beginning — articulated in a physical notebook while the learning was actively happening:

"Spaced-repetition: not to memorize, but to learn, because we value performance tests of which you have no experience. Can you correctly transcribe an audio clip you have never heard before? The spaced-repetition improves you in this activity."

— Notes.21, entry 62, written during the learning period, 2021–2022

And later, giving the metric its precise definition:

"Which stat is your power? Probably a longer term 'zero' success rate, not all time, but not recent either. And it must be increasing, for power is more power."

"A 'zero' success rate refers to success rate on positions for which you have no experience. It is monthly success rate on first impressions, basically."

— Notes.21, entry 455 and annotation, 2021–2022

The question was already framed as an experiment:

"An experiment is necessary to prove review (with SRS) improves zero-impression performance over six months more than totally random or totally new. Remember, we're not trying to remember the samples — to increase rate of success in the general case, or out of all special cases."

— Notes.21, entry 776, 2021–2022

The 14 months that followed ran that experiment.

The Numbers

Each row below shows one calendar month. Cold-start % is the success rate on sentences attempted for the very first time that month — no prior exposure, ever. Overall % includes all attempts, new and repeated. The gap between the two columns is the recognition effect: familiarity accumulated from prior sessions inflating the overall number.

Notice what happens in December 2021. The cold-start rate dips to 41.5% — a month where practice volume tripled (892 new sentences vs. 200 in prior months). The learner was deliberately expanding the corpus, accepting a temporary drop in cold-start rate to increase coverage. This is not failure. It is the fiat factor working as designed: expansion phases lower the cold-start rate, consolidation phases raise it.

"Don't stop playing when your rate of success goes down; build your STAMINA."

— Notes.21, entry 29, 2021–2022

January 2022 marks the inflection. The cold-start rate jumps to 57.7% — a 16-point gain in a single month after the December expansion consolidated. From there it climbs steadily: 66%, 66%, 74%, 77%. By April 2022, the cold-start rate had crossed 70% and held.

The 70% Claim

C1 on the CEFR scale is described as the ability to "understand a wide range of demanding, longer texts, and recognize implicit meaning." In practice, for Romanian, this translated to being able to follow native-speed audio without prior exposure and reproduce it accurately roughly three times out of four.

The 70% threshold is not derived from the CEFR documentation. It is an empirical observation: when the cold-start rate crossed 70% and held there across multiple months and thousands of new sentences, real-world Romanian — conversations, radio, unscripted video — became fully comprehensible without preparation. Below 60%, there was consistent failure on fast or colloquial speech. Between 60% and 70%, comprehension was functional but effortful. Above 70%, it became natural.

One data point is one data point. This is not a controlled study. It is the honest report of a single learner, with the full transaction log to support every number in the table above.

What the Overall Rate Hides

Notice how the overall success rate stays high throughout — 61% to 79% — while the cold-start rate starts at 39% and takes 14 months to reach 74%. In March 2021, the overall rate was 61% while cold-start was 39%. A 22-point gap, entirely explained by repeated exposures inflating the aggregate.

By April 2022, the gap had nearly closed: 75.9% overall vs. 74.2% cold-start. The reason is coverage — by that point, the majority of sentences in the active corpus were relatively new, so the overall rate could no longer be inflated by a deep library of mastered sentences.

This is why overall success rate is a lagging, flattering indicator. It measures where you have been. Cold-start measures where you are.

Why Spaced Repetition Produces This Result

"If you were coaching, and a student's rate of success on previously unseen exercises were increasing, why would you review? Thesis: don't review until the rate drops."

— Notes.21, entry 91, 2021–2022

The counterintuitive answer: review is what causes the cold-start rate to increase. This is the result the experiment confirmed. Spaced repetition does not just help you remember specific sentences — it trains the auditory pattern-matching that transfers to sentences you have never heard.

The mechanism is not mysterious. When you have heard a sentence three times at spaced intervals and transcribed it correctly each time, you have not memorized it — you have trained your ear on the phonemes, the rhythm, the elisions specific to that speaker and that corpus. That training transfers. The next new sentence from the same corpus is easier because the underlying listening skill has improved, not because you recognized it.

The cold-start rate is the proof. If spaced repetition were only producing memory effects, the cold-start rate on genuinely new sentences would not move. It moved 35 points over 14 months.