Twelve questions, then a twin
Ordering for a friend
You know someone when you can predict what they would choose in a situation the two of you never discussed.
You are at a restaurant and your oldest friend is running late. The waiter is waiting. You order for her, and you get it right. Nobody told you what she wanted tonight. You know how she chooses.
A twin, in this work, is a system that can do that for one person. It listens to them once, for a short time, and afterwards answers new questions the way they would: the same decision, the same priorities, in their own voice.
That is a different thing from a recording. A recording can only repeat. The whole of this article is about how to tell the difference with a number.
Why a transcript is not a twin
What a person said in an interview is evidence about them. It is not yet a prediction of what they will say next.
A transcript holds what someone said. It does not hold how they decide. Ask your friend about her childhood and you learn a story. To order for her you need the pattern behind the story: what she protects, what she gives up first, how she talks herself into a choice.
A short interview makes this harder. Twelve questions is a few minutes of talking. Most of what the twin will be asked later was never touched in it. So the twin cannot look the answer up. It has to reason from the person to the answer.
This is also why the only honest test is one the interview could not have rehearsed. If a test question overlaps an interview question, the score is measuring memory, not understanding. In this work such a run is thrown out.
Three kinds of unseen question
Each person answers 42 questions the twin never saw them answer: 18 ratings, 12 forced choices and 12 open questions.
The test is a fixed set of 42 questions that are kept away from the interview. The person answers them. The twin, which has only heard the interview, answers the same ones. Then the two sets of answers are compared.
There are three kinds. Ratings: 18 statements answered on a scale of 1 to 5, such as how far a sentence describes you. Forced choices: 12 situations with a fixed set of ways to respond. Open questions: 12 situations answered out loud in the person's own words.
Back at the restaurant, these are three ways of checking your order. Would she rate this dish a 4? Would she pick the fish or the pasta? And what would she actually say to the waiter?
One number, and how it is built
Fidelity is the plain average of three agreements, one per kind of question, on a scale of 0 to 100.
Fidelity is the average of how well the twin agrees with the person on ratings, on choices and on open answers. Each part is a number from 0 to 1. The three are averaged and multiplied by 100.
For a rating, agreement is one minus the gap between the two answers, divided by four. Same answer scores 1. One step apart scores 0.75. Opposite ends score 0. For a forced choice, agreement is simply whether the twin picked what the person picked.
For an open answer, a separate judge reads both answers without knowing which is whose, and rates from 0 to 10 whether the twin took the same stance, reached the same decision and showed the same priorities. Wording does not count. Substance does.
The ceiling: people do not agree with themselves
Ask a person the same question twice and the answers differ. A twin cannot be expected to match someone better than they match themselves.
A fidelity of 100 is not the top of the scale. The top is how consistently the person answers the same question twice. Your friend would not order the same thing every Friday. If she agrees with herself eight times in ten, a twin that matches her eight times in ten has nothing left to learn.
So every person in the test answers every question twice. Their agreement with themselves is the ceiling. Dividing the twin's score by that ceiling gives normalized fidelity: the share of a person's own consistency that the twin reaches.
This is the measure used by the largest published study of this kind. Park and colleagues interviewed 1,052 people for two hours each and reported twins reaching 85 percent of the participants' own consistency. That figure is the reference point for the next part.
77.5, and what it is made of
On twelve people, from a twelve-question interview: fidelity 77.5, which is 86.4 percent of how consistently those people agree with themselves.
The twin scores a fidelity of 77.5, and a normalized fidelity of 86.4. That is across twelve people, each tested on the 42 unseen questions.
By part: ratings agree at 0.83, and the twin is within one step of the person on 89 percent of them. Forced choices are right 87 percent of the time. Open answers score 0.63 where the people themselves, asked twice, score 0.72.
The normalized figure sits beside the 85 percent reported for two-hour interviews of real people. The two are not the same test: this interview is twelve questions long, and these people are simulated and easier to predict. The next part is about exactly that.
Where it is right, where it is wrong
Strongest on moral lines and decisions, weakest on life story and personal stances, and it misreads statements that are worded in the negative.
The twin is not equally good at every part of a person. By kind of question it agrees best on moral lines (0.89) and on how the person decides (0.83). It agrees least on questions about their life story and their stances (0.67).
The hardest open questions are the ones about habits and priorities in situations the interview never reached: what you would do with a free year, what you would never compromise on. There the twin falls back on what a reasonable person would say. Your friend is not a reasonable person. She is herself.
Two errors are systematic. On ratings, about half of the error is a one-step miss toward the middle of the scale. And when a statement is worded in the negative, the twin tends to rate the trait and not the sentence, so it lands at the wrong end. Across people the score runs from 71.7 to 81.3.
Out loud, in about two seconds
In a spoken rehearsal the twin starts answering a median of 2.1 seconds after the caller stops talking.
The twin is meant to be talked to, so its delay is part of the result. In a real-time rehearsal over the full audio path, the median time from the end of the caller's speech to the start of the twin's reply is 2.1 seconds. The rehearsal ran on one laptop with a synthetic caller.
On the written test the twin's median time to answer a question is 1,065 milliseconds. A full scoring pass over all twelve people costs $0.87.
The twin speaks in a voice taken from the person's own interview answers. How close that voice is to the person's has not been measured here. It is stated as a limit in the next part.
What cannot be claimed
The people were simulated, no real person has been scored, the voice is unmeasured, and differences under a point are noise.
The twelve people in the test are written characters, played by a language model. They are more consistent and more articulate than humans: they agree with themselves 0.99 of the time on ratings and choices, where real people in a comparable study agree about 0.82 to 0.85. So the absolute numbers here are optimistic.
No real person has been scored. A second test set exists, built from a public dataset of 30 real participants and 40 questions. On it, answering every question with the neutral option scores 63.4 and people against themselves score 83.6. No twin has been scored on it. The system has also not been used by a real person on a real microphone.
The measurement has noise. The same configuration scored twice gave 77.7 and 77.5. Of 28 measured experiments, 3 were kept, 17 were discarded, 7 did not complete and 1 was voided. Most of the discarded ones were within about a point of the best, which is too close for this test to tell apart.
What is public, and what is on request
The scores, how they are defined and their limits are public. How the twin is built is not: ask.
Think Like Me is proprietary. What it scored, how the score is defined, the reference it is compared with and the limits above are public, here and in the paper. Which questions the interview asks, and how an interview becomes a twin, are not.
This article is the first of two steps. The paper sets the same results out formally. If you want to see more, or to try it, the request form is the way in.
Step 2: the paper