You ask that the result be "counterintuitive", but Feynman doesn't insist on that. He says that if you can phrase a true-or-false mathematical question in language that he can understand, he can immediately say what the right answer is, and that if he gets it wrong, it is because of something you did.

I think Feynman is being less than 100 percent serious. Not that he didn't win every time he put this challenge to people--- but he probably only issued this challenge when he wanted to make a rhetorical point (about either the impracticality of a lot of mathematical investigation, or about the inability of mathematicians to faithfully translate their problems into normal language).

The Banach-Tarski result is obviously a terrible example, because the key to any paradoxical decomposition of a sphere, nonmeasurability, is almost impossible to convey in non-technical terms, and has no physical meaning. And of course he would choose this example for his essay, if the only purpose of the challenge is to make the point illustrated marvelously by that particular response.

Here are some statements that might have given Feynman some pause.

• The regular $n$-gon is constructible with an unmarked ruler and compass. (Really a family of true-or-false statements, one for each $n \geq 3$.)

  It takes some work to properly spell out what "constructible" means here, but it can be done in plain English. It has been known since the 1800s (thanks to Gauss and Wantzel) that this statement is true if $n$ is the product of a nonnegative power of $2$ and any nonnegative number of distinct Fermat primes, and false otherwise.

  More concretely, the sequence of positive integers $n$ for which it is true is partially listed here. Could Feynman have generated that sequence with his series of answers to true-or-false questions given by taking $n=3,4,5,\dots$? I very much doubt it.

• The Kelvin conjecture (roughly, "a certain arrangement of polyhedra partitions space into chunks of equal volume in a way that minimizes the surface area of the chunks"--- but you can be more precise without leaving plain English). According to Wikipedia it was posed in 1887. It was neither proved nor disproved until 1993, when it was disproved.

  I find this example particularly compelling because Feynman presumably would not have caricatured Kelvin (something of a physicist himself) as a mathematician who only works on silly questions that nobody would ever ask.

• Other geometrical optimization problems come to mind, e.g. the Kepler conjecture, the double bubble conjecture, and the four color conjecture (all theorems now, but let's pretend they're conjectures and ask Feynman). My guess is that Feynman would have been right about the truth values of these statements. But the mathematician's response is, of course, "OK. Why are they true?"

This highlights a real difference between math and the physical sciences. It is much more common in the sciences to be in a situation where knowing what happens in a given situation is useful, even if you don't know why it happens. In math, this is comparatively rare: for example, the "yes or no" answers to the Clay Millennium problems are nowhere near as valuable as the arguments that would establish those answers. Feynman almost certainly knew this, but pretended not to in order to make the rhetorical points mentioned above.