Test your voice

Evaluate five separate questions. A single combined score would hide what was actually tested.

/vf-evaluate explainer-en

You can request a part, such as /vf-evaluate explainer-en fidelity. Evaluations run when you ask and are saved under evals/ in your private voice home.

Structure

Does the profile follow its format? Run validation from the app checkout:

python3 skills/voice-forge/scripts/vf.py validate --profile explainer-en

No errors means the structure passes. Warnings such as empty brief fields are listed. This says nothing about the quality or fidelity of generated prose.

Rule compliance

Do drafts and edits follow the rules that apply? Review each text against its profile and base rules. A useful test also checks the reviewer with planted violations and exception cases.

The reviewer reports PASS, VIOLATION, UNCERTAIN or NOT APPLICABLE for every rule. Compare texts written with Voice Forge to a baseline reviewed in the same way. Separate reviewer independence, caught violations, misses and false alarms.

The scoring command consumes an expected verdict file and a review report. The full method describes the inputs.

Voice fidelity

Does it sound like you? Your own judgment and texts are the evidence. Compliance and the absence of common AI phrases cannot answer this question.

1. Keep recent texts that you wrote alone outside all training, samples and analysis.
2. Add them to the holdout set. Do not ask the agent to open them for learning.
3. Run the overlap check. This checks repeated paragraphs; it is not a guarantee against every form of contamination.
4. Give a short brief of a holdout text without exposing the text to the writer.
5. Generate versions in fresh contexts: with the profile and with the same model without Voice Forge. Optionally add a generic-slop-only version.
6. Shuffle the versions under letters and choose the one most like you before the source is revealed.
7. Reveal and log the result. Repeat with different briefs.

/vf-holdout add unused-essay.md explainer-en
/vf-evaluate explainer-en fidelity

From the app checkout, the underlying helper sequence looks like this. Version files must already exist; replace the label returned by blind make where shown.

python3 skills/voice-forge/scripts/vf.py holdout check
python3 skills/voice-forge/scripts/vf.py blind make newsletter voice-forge.md baseline.md
python3 skills/voice-forge/scripts/vf.py blind score <returned-date-name> --pick A
python3 skills/voice-forge/scripts/vf.py blind summary

Read blind.md, never key.json, before choosing. Report counts and the number of comparisons. Fewer than about ten comparisons is thin evidence, not “fidelity validated.”

measure.py compare can compare sentence, paragraph and stance measures with a holdout text. The script may read it during an authorized test; the writer should not. Those surface measures are descriptive rather than proof of fidelity.

Writing quality

A judge that does not see the profile rates clarity, coherence, correctness, effectiveness and fit, with evidence. A text can sound like you and still need work for its audience or genre.

A fresh reviewer is preferred where the client supports one. If the same agent reviews its own work, record that limit. Plain chat packets do not provide an independent context by themselves.

Meaning and facts

Compare every claim to its sources: numbers, names, units, quotations, attributions, negation, uncertainty and logical links. An unauthorized change is a critical failure for that text, regardless of its other scores.

“May be associated with” cannot become “causes.” A clean lint report cannot excuse a deleted qualification.

Report what was tested

For each of the five parts, record passed, failed or not tested, evidence and limitations. Do not turn a valid profile into a claim about fidelity, or general writing quality into rule compliance.

Read the complete evaluation protocol, including the report template and helper commands.