Quick Take: Death by Operational Definition

The 2023 Ig Nobel Awards have been announced.

Ig Nobel awards celebrate serious scientific work that sounds… silly (like, say, why geologists lick rocks). It’s fun to scan the list of winners and read the contest’s description of their work. Doing so also is instructive, because the truth is that a LOT of serious science sounds silly if you describe it concretely and out of context.

Not this kind of rock licking.

Using fake Latin, because Latin sounds very science-y, let’s call this problem mortem operationalis (MO), or death by operational definition. To illustrate MO, consider one of this year’s winners, a 1969 paper by Stanley Mlgram and colleagues that asked, according to one news report, “how many passers-by stop to look upward when they see strangers looking upward.”

That is exactly what the study investigated, but of course not the whole story. The researchers were interested in conformity in groups, in natural environments, so they needed a target behavior that was easy for a confederate to perform, easy for bystanders to notice, and easy for researchers to count. Unlike in certain real world circumstances, looking up per se was beside the point. It was a convenient means to an end. Of course, the Ig Nobels, being a cheesy, tongue-in cheek performative operation, tend to de-emphasize details like this.

At one time social psychology, out of which Milgram’s conformity work arose, suffered from a lot of MO (dig out some articles from the 1950s and 1960s to see). But researchers apparently learned from this. I have heard contemporary social psychology maligned for pandering to the crowd, that is, for describing its research in terms that too closely mirror everyday language. To illustrate, here are some phrases taken from titles of 2022 articles in Journal of Personality and Social Psychology:

  • left-wing authoritarianism
  • sexual harassment
  • pooling finances and relationship satisfaction
  • equality for (almost) all
  • revealing other people’s secrets

Some critics say that language like this makes Social Psychology essentially folk psychology (in other words, not real science). That claim could be debated but one thing that cannot is that research in social psychology gets attention. Example: According to the Altmetric Explorer tool, to date 2022 JPSP articles have accumulated about 1000 news media mentions and about 3500 social media mentions. By contrast, 2022 articles in Journal of Applied Behavior Analysis have tallied 2 news media mentions and about 250 social media mentions.

Part of the reason for the discrepancy, if you ask me, is MO. We behavior analysts are sticklers for precise operational definitions. If you read the titles (not to mention text) of our research articles, you can almost always tell the independent and dependent variables, in excruciatingly operational form. But why “face touching” and “schedules of response-independent stimulus presentation” matter may not be patently obvious to, well, anybody but us.

The MO problem is most acute, predictably, in basic behavioral science. Heck, in the ultimate embrace of operationalism, we even named our flagship journal (Journal of the Experimental Analysis of Behavior) — after methodology rather than topics in behavior to which the methodology is applied. That’s sort of like creating a biology periodical called the Journal of Microscopes. For what it’s worth, more-operational journal names weren’t uncommon in the 1950s when JEAB was founded. But some journals have wised up. For instance, what used to be the rather inscrutable Journal of Verbal Learning and Verbal Behavior became the much more transparent Memory & Language.

Details aside, operational fundamentalism is laced through the language of our entire discipline, and where it exists it drives a wedge between us and people who could benefit from knowing about our work. MO prevails when the words we use to describe valuable advances in the analysis of behavior strike people as unfamiliar or even unpleasant, or more generally when those words fail to signal that reinforcers are to be had by delving into our work. Say what you want about social psychologists; they know how to telegraph that their work overlaps with peoples’ pre-existing interests. After all, as Montrose Wolf taught us, to a large extent social validity is what people in society say it is.

To be clear about my sentiments, there is a tightrope to be walked here. Careful operational definitions are very useful, no, essential in science because a study makes sense only when you have a good handle on its variables. The same may be said about the conceptualization and planning of an applied protocol. But the same operational descriptions that facilitate the doing of behavior analysis hampers the communicating about it. How can we do better? Where science is concerned, maybe at times article titles can be crafted to speak simultaneously to two audiences, both scientists and nonscientists. Maybe articles, or at least applied articles, could be prefaced with a plain English statement about why they’re important. Certainly outside the pages of our journals we should avoid MO at all costs. After all, before anyone can decide whether we do anything well they have to understand what we do and how it connects to the problems they understand to be important.