To build a solid foundation, we are studying programming errors at the lowest level. We have designed several studies to gather data on the confusion generated by atoms of confusion, minimally small obfuscating transformations to source code.
We hypothesized a short list of extremely small transformations designed to confuse programmers. We tested these transformations against control code and measured the accuracy with which subjects were able to hand evaluate each type of code.
Explore the atom existence experiment
With the confusing code confirmed in the existence study, we built a larger experiment to study the effect of this confusion in a lengthier sample. We took slightly larger, complete programs, and introduced several confusing transformations.
Explore the atom impact experiment
What happens in the mind of a programmer when they read confusing code? Using commodity scalp EEG devices we can read the brainwaves of subjects as they reason through code snippets of various kinds.
Explore the EEG experiment
Outside of the lab, how prevalent are atoms of confusion? We mine several
popular open source C projects for known atoms, and look for correlations with
other relevant phenomena.
Explore atoms in the wild
What do developers think when they read atoms of confusion? We also analyze
common pitfalls in correctness-based program comprehension experiments.
Explore the Think-aloud Study