Plan the test so you can undo it

Generated from content/lms/race-car-mechanic-reference/05-prove-repairs-with-testing/01-plan-reversible-ab-tests.md; edit the source file, not this page.

Source path: content/lms/race-car-mechanic-reference/05-prove-repairs-with-testing/01-plan-reversible-ab-tests.md

Course: Service the race car that has to finish

Module: Prove repairs and changes with testing

Estimated duration: 45 minutes

This lesson is about one discipline: never let a repair or setup change become unanswerable. A reversible A/B test is not just trying a new part and asking whether the driver liked it. It is a planned comparison where you run the known car, make one recoverable change, run the comparison, and then return to the known car so the result has a fixed reference. The return step is what separates testing from paddock guessing. Without it, a faster lap can be the change, the driver learning, better rhythm, cleaner traffic, or simple scatter. With it, you have at least a fighting chance of separating the car from the human being in the seat.

The governing principle comes from race-car development practice: consistency and baselining matter more than the excitement of the new setting. Van Valkenburgh is blunt that one exceptional lap among scattered lap times is meaningless, and that every test needs a known basis of reference. He also gives the exact trap this lesson is built to avoid: a suspension-geometry change can look faster, but unless you can go back to the original setting you cannot know whether the improvement came from the change or from the driver improving during the session. For a mechanic, that means reversibility is not a convenience. It is part of the evidence.

The first skill is to write the test question in a reversible form before anyone touches the car. Do not write a question like, improve the car. Write the question as a cause-and-effect comparison you can actually reverse: does this suspension-geometry correction make the car repeatably better than the baseline, and does the old behavior return when the original setting returns? If you are proving a repair, make the question equally narrow: does the repaired condition behave like the known-good condition under the same kind of use, and can you restore the previous known setup if the repair introduces a new symptom? The wording matters because it tells you what must be measured, what must stay fixed, and what must be ready to undo.

The second skill is to protect the baseline. A baseline is not a memory of how the car felt last month. It is the current fixed reference you can return to during this test. It needs enough detail that the crew can restore it without argument: the setting you changed, the hardware position, and the driver instruction for the run. If the baseline is vague, the return step becomes theater. You may put the car back near where it was, but you cannot use the return run as evidence. A reversible test starts with the boring work of making the original condition well known.

The third skill is to demand driver consistency before you believe the car. This is uncomfortable because mechanics naturally want the car to be the object of the test. In racing, the driver is part of the test rig. Lopez warns that early in a racing career the driver can suddenly find a one or two percent lap-time improvement, and that the car may have a problem, but it may also be the driver. That is exactly why reversible testing exists. If the driver learns the corner between A and B, the B run can look like a mechanical win. If the driver loses confidence after a strange feel, the B run can look like a mechanical failure. The return-to-A run is the guardrail. If the car returns and the result follows the car rather than the timeline, you have stronger evidence.

The fourth skill is to make the change small enough to undo, but large enough to be visible. The corpus does not give numeric thresholds for how much toe, camber, brake adjustment, spring platform, or damping change to use, so this lesson will not invent them. The mechanic-level rule is simpler: if the planned change cannot be returned to the original condition during the same test window, do not call it a reversible A/B test. It may still be a valid development step, a repair, or a necessary replacement, but it is no longer the method being taught here. For this skill, the undo path is designed before the change is made.

The fifth skill is to run the return, especially when the change looks bad. Many teams are disciplined when the new condition is promising and sloppy when it is unpleasant. Van Valkenburgh says returning to the original condition can be even more important after a negative change. That is because a bad B run can have two different meanings. It can mean the change hurt the car, or it can mean the driver, surface, traffic, or adaptation was worse during that run. If you return to A and the car feels and times like the first baseline again, the negative result becomes more credible. If the return run is also bad, you do not have a clean A/B result. You have a new problem to understand.

The sixth skill is to keep the conclusion proportional to the evidence. A reversible A/B test rarely proves that a part is universally better. It can prove something narrower: in this test window, with this driver, this car, and this type of corner or load, the B condition was repeatably better, repeatably worse, or not separable from the baseline. That is still valuable. Race-car development is broad and complex, and the same Van Valkenburgh passage that teaches baselining also warns that testing and development are more important than simply placing the right pieces on the car. The right conclusion from a clean test saves parts, tires, crew time, and driver confidence because it tells you what to keep, what to revert, and what still needs another question.

Use an A/B/A structure as your default mental model. A is the known baseline. B is the reversible change. The second A is the proof that the baseline can still be recovered and that the first difference was not just the driver moving through the learning curve. If time and resources allow, a second B can strengthen the conclusion, but the first return to A is the non-negotiable lesson here. Without that return, you have only before and after. With it, you have a test.

This lesson deliberately does not duplicate the nearby lessons on cheap shakedown resources, brake safety checks, proving the car works before racing, or estimating before burning a test session. Those are related skills. Here, the narrower job is experimental design. You are deciding whether the next track miles will create evidence or just create more opinions.

There are five sub-skills to practice. The first is baseline capture: make the starting condition specific enough that another mechanic could restore it. The second is change isolation: make only the change needed to answer the question. The third is driver control: ask for repeatable driving rather than hero laps. The fourth is return discipline: plan the tools, access, time, and notes needed to go back to A. The fifth is conclusion hygiene: label the result positive, negative, or inconclusive instead of forcing every test into a win or loss.

A positive result has a recognizable shape. The baseline is repeatable, the B condition changes the car in the predicted direction, and the return-to-A run gives back the baseline behavior. The driver comments should also track the car rather than the clock. If the driver says B lets the car do the job more cleanly, then A brings back the old limitation, the evidence is coherent. You still avoid grand claims, but you can keep the change with a clear reason.

A negative result also has a useful shape. The baseline is repeatable, B makes the car worse or fails to remove the symptom, and returning to A restores the original behavior. That result is not a wasted session. It protects you from racing an unhelpful repair or setup change. In fact, this is where reversibility earns its keep: the crew can put the car back into the known condition instead of debating whether the bad feel was real.

An inconclusive result is not an embarrassment. It is what you call the test when the evidence does not deserve a stronger name. If the driver produces one brilliant lap and several scattered laps, Van Valkenburgh's warning applies. If the driver is improving rapidly during the test, Lopez's car-or-driver warning applies. If the return-to-A run does not resemble the first A run, the basis of reference has moved or the driver has changed enough that the comparison is dirty. The disciplined answer is not to pretend. The disciplined answer is to log the uncertainty and design the next run better.

The driver conversation is part of the method, but it must be structured. The useful question is not whether the car felt better in a general sense. The useful question is whether the car behaved differently at the same part of the task. Bryan Herta's page fragment points the driver back toward the distinction between doing something different with the car and doing something different with the approach to the corner. That is the exact distinction a mechanic needs. If the driver changed the approach, the test may be measuring the driver. If the car changed its response to the same approach, the test may be measuring the car.

If the driver is not consistent enough to test the part, change the test design rather than lowering the standard. Lopez suggests using a more experienced and accomplished driver in the same class on a test day as one way to settle whether the issue is the car or the driver. That does not insult the primary driver. It protects the answer. If the experienced driver can reproduce the symptom and the A/B/A pattern follows the car, the mechanic can act with more confidence. If the experienced driver cannot reproduce it, the repair plan may need to move from the car to the driver process.

The habit you are building is reversibility before confidence. A mechanic who can plan a clean return is less likely to fall in love with a clever change, less likely to blame the driver unfairly, and less likely to throw away a known-good condition. The car may still need deeper engineering, and some repairs cannot be made reversible in the strict sense. But when a test can be made reversible and you skip that step, you have chosen weaker evidence on purpose.

Worked example: a suspension-geometry change that seems faster

Start with the situation Van Valkenburgh names directly: a change in suspension geometry seems to make the car faster. The weak version of the test is familiar. The crew changes the setting, the driver runs, the watch shows a better lap, and everyone decides the setting worked. The reversible version is stricter. First, define A as the current geometry and make it restorable. Next, send the driver for a baseline run with the instruction to drive for repeatability, not one heroic lap. If the times are scattered, stop calling it a setup test and work on consistency.

Now make the geometry change as B. The question is not whether the driver can eventually go faster with B. The question is whether B improves the same task relative to A. After the B run, restore A. This return is the important move. If the car slows or the old limitation returns when A returns, the case for B is stronger. If the car stays fast after returning to A, the earlier improvement may have been driver learning. If A cannot be restored accurately, the test was not reversible enough to answer the question.

The useful conclusion is narrow. You can say B was better than A in this test only if the behavior follows the setting instead of following the passing of time. That is the mechanic's protection against being fooled by normal driver improvement.

Worked example: a brake-entry repair in a Formula Dodge style corner

The bonded corpus gives a high-load braking situation: a Formula Dodge type racecar approaching a 35 mph corner at 110 mph. This lesson does not turn that fragment into a brake-system safety procedure; that belongs to the sibling lesson on testing brakes before you need them. Here, use the situation as a test-design example. A heavy brake-entry zone is exactly where a driver can feel a repair or adjustment, but it is also exactly where driver timing and confidence can move the result.

The reversible test starts away from drama. Define A as the known repair-complete condition or the known setup you are comparing against. The driver repeats the same brake-entry task and reports whether the car accepts the same approach. Then make B, the single reversible change connected to the repair question. Do not combine it with a new line, a new braking marker, and a new driver target. After B, return to A and ask whether the original response returns.

If B lets the driver brake and enter with more repeatable control, then A brings back the earlier limitation, you have a coherent comparison. If the driver simply gets braver each run into the same 110-to-35 mph task, B may get credit for confidence it did not create. If the driver loses trust after B and the return run is also poor, the test has been contaminated by the human part of the system. The answer is not to invent certainty. The answer is to log it as inconclusive and design the next test with a cleaner baseline or a steadier driver.

Common mistakes

The first mistake is the flyer trap. One very fast lap feels persuasive, but Van Valkenburgh warns that one superfast lap among scattered times is meaningless. Good looks like a repeatable pattern, not a single exciting number.

The second mistake is the missing return. The team runs A, runs B, likes B, and never goes back. Good looks like A/B/A. The return run is where you learn whether B beat the baseline or merely arrived later in the driver's learning curve.

The third mistake is changing the car and the driver at the same time. If the driver also changes the approach to the corner, you may be testing instruction rather than hardware. Good looks like asking the driver to repeat the same task, then using comments to separate car response from driver adaptation.

The fourth mistake is treating a negative B as failure. A bad reversible change is useful if returning to A restores the known behavior. Good looks like removing the bad change quickly and leaving the test day with a clearer boundary.

The fifth mistake is a vague baseline. If the original condition cannot be restored precisely enough for the return run to mean anything, the test was compromised before B ever happened. Good looks like recording the starting condition and preparing the undo path before the change.

The sixth mistake is blaming the car too early. Lopez reminds the driver to look inward because the car may have a problem, but the driver may be the source of the change. Good looks like using repeatability, return-to-baseline behavior, and sometimes a more experienced driver to separate those causes.

Drill: the three-run return-to-baseline test

At your next test day, choose one reversible question and run a three-run A/B/A drill. The count is three runs: baseline, change, return. Keep the question small enough that the car can be restored during the same session window. A good first version is a setup or repair comparison where the original condition is easy to document and recover.

Run 1 is A. The driver's job is repeatability. The crew's job is to decide whether the baseline is clean enough to use. If the times or comments are scattered, do not proceed as though you have evidence. Treat the baseline as the first result.

Run 2 is B. Make only the planned reversible change. The driver repeats the same task and reports the same limited set of sensations: where the car changed, whether the old symptom improved, and whether the same approach still works.

Run 3 is A again. Restore the original condition and run the same task. The success criterion is not that B wins. The success criterion is that you can honestly classify the result. Positive means B changed the behavior in the predicted direction and returning to A brought the baseline behavior back. Negative means B hurt or failed to help, and returning to A restored the known condition. Inconclusive means the driver, lap scatter, or return condition moved enough that the comparison is not clean. If you can make that classification without arguing in circles, the drill worked.

Calibration cues

You are improving at this skill when your test notes become less dramatic and more useful. The first cue is that your baseline description is specific enough for another mechanic to restore. The second cue is that the driver comments are tied to the same task each run instead of general mood. The third cue is that you are comfortable writing inconclusive when the evidence is dirty. That is a sign of discipline, not weakness.

The lap-time signature you want is not a miracle lap. It is a pattern that follows the car state. A behaves one way, B behaves differently, and A returns. If the time improvement only moves forward with each run no matter what the car state is, suspect driver improvement. If the driver cannot reproduce the task, suspect inconsistency before you blame the part. If a more experienced driver can reproduce the pattern, the car-side conclusion becomes stronger.

An instructor or chief mechanic watching this process would probably care less about the chosen change than about the discipline of the comparison. Did you know the starting point? Did you control the human variable as well as the car allowed? Did you return to the baseline? Did the conclusion match the evidence? Those are the cues that the test is becoming a method rather than a habit of trying parts.

When this principle breaks down

Some work cannot be made reversible in the useful test-day sense. The current bonded corpus does not give examples of irreversible fabrication, rebuilds, or component replacement thresholds, so this lesson will not invent a catalog. The decision rule is enough: if you cannot return to the known condition during the comparison, stop calling the plan an A/B test. Treat it as a repair verification or development step and be honest about the weaker comparison.

The principle also breaks down when the driver is changing faster than the car. Lopez's warning about the driver finding speed is especially important for intermediate teams. If the driver is learning rapidly, the test may need a more experienced driver, a simpler task, or a separate driver-development session before the mechanic can draw a fair conclusion about the part.

Finally, the principle breaks down when the team already knows the car is unsafe or not raceworthy. Reversibility does not replace safety judgment. It is a method for proving a repair or setup comparison after the car is fit to run the test. If the car cannot safely produce a baseline, the correct move is not a clever A/B plan. The correct move is to fix the car enough to create a safe known condition first.

Author Review

No quiz questions are attached to this lesson.

Sources

#	Document	Chunk	Pages	Score	Collection
1	Race Car Engineering Mechanics Paul Van Valkenburgh	4a0085b1-a5b6-20ef-c288-ff092fa3e4d9	116	1	uio_books_raw_v1
2	Going Faster Mastering the Art of Race Driving - Carl Lopez	ef9ea5d6-92b2-e60a-d6d0-5adac150482c	234	1	uio_books_raw_v1
3	Going Faster Mastering the Art of Race Driving - Carl Lopez	f2410e4f-42d0-24db-af78-3d9940ff312d	75	1	uio_books_raw_v1
4	Going Faster Mastering the Art of Race Driving - Carl Lopez	2cc8fb73-bf8b-6575-5167-9dbef050bdfe	75	1	uio_books_raw_v1
5	Going Faster Mastering the Art of Race Driving - Carl Lopez	b2c44205-8e7a-2622-d998-a8b843b3229a	92	1	uio_books_raw_v1
6	Going Faster Mastering the Art of Race Driving - Carl Lopez	06a160fb-3b2a-e539-9ffc-8741bf0bd18d	91	1	uio_books_raw_v1
7	Going Faster Mastering the Art of Race Driving - Carl Lopez	f8e3be74-968a-a046-4ad6-3509a8108cfe	91	1	uio_books_raw_v1