In previous posts I’ve discussed how altmetric data (representing mentions of scholarly articles in non-scholarly sources like news stories, blogs, social media, public policy documents, and more) can be considered a measure of dissemination impact. These data show where people who are not scientists are paying attention to our science of behavior. The data aren’t by any means a complete picture of dissemination, the ultimate endpoint of which is for everyday people to demand and use what behavior analysis has to offer. But if that endpoint matters to us, we should by all means pay attention to who’s talking about us and what they are saying.
In this post I’ll summarize what altmetric data tell us about the dissemination impact of behavior analysis articles published so far in the 21st century. I used the Altmetric Explorer app on December 15, 2023, to determine the mentions of articles published in 16 behavior analysis journals (I checked for a few others but found no mentions in the database). For each article, those mentions combine to create an Altmetric Attention Score (see here for an explanation of how various types of mentions are weighted in creating the score).
The Attention Score ranges upward from 0 (no mentions), can currently a score of 20 equates to roughly the top 25% of the almost 25 million articles currently indexed in the database. I identified 330 behavior analysis articles, published since 2000, that have earned an attention score at least that high. The table at the end of this post lists the Top 30 articles, whose Attention Scores place them among the top 5% of all articles in the database.
Where The Action Is
The graph to the right shows the proportion of those 330 top-quartile articles that came from each of 16 journals. I’ve noted in previous posts that Journal of Contextual Behavioral Science seems to do an especially good job of recruiting societal attention, and indeed it ranks first in my survey despite being the youngest of all of the journals (it began publishing in 2012).
The other major players are Journal of Applied Behavior Analysis (JABA), Behavior Analysis in Practice (BAP, which has an explicit dissemination mission), and Behavior Modification (BM).
It’s unsurprising that applied journals dominate here. Among the Top 30 articles in the table at bottom, only two come from a basic-research journal (Journal of the Experimental Analysis of Behavior, JEAB), and both of those are translational articles. Probably the most surprising thing about the graph is that basic-focused journals (JEAB and The Psychological Record) are as well represented as they are.
Major Sources of Attention
Among all of the altmetric mentions of 21st Century behavior analysis articles, about 85% came from social media; about 10% came from news outlets; and a shade under 2% came from blogs. These numbers parallel what’s true overall in the Altmetric.com database: 80% social media, 7% news, and around 1% blogs.
Among the Top 30 articles in the table, 18 received most of their mentions from news sources, 6 were primarily mentioned in social media, and the rest got mentions from multiple sources.
It’s important to remember that not all mentions are complimentary. In practice, news outlets rarely cover science except to share interesting findings with the public. But the world of social media… well, this is a strange and volatile beast, as I’ll discuss in a moment.
Not Autism
For me, what most stands out among the most-noticed articles is how few of them represent the economic bread-and-butter of applied behavior analysis: autism. This is not to say that autism treatment is unimportant, only that, as other observers have suggested, it’s simply not a central concern for a large swath of the population. The table, which provides a glimpse of some of the other things that everyday people seem to care about, reminds us that attention is contingent.
This is hardly a novel observation. To hammer home the point that reader interests don’t always parallel the focus of research programs, see this recent list of most-read articles in APA journals. Five of the top 10 articles address behavior in or concerning popular media — a great example of behavior that might not be pathological or earth shaking, but people care about it nevertheless (here’s another example). A science of behavior, to be credible, must offer insights into this kind of behavior… indeed, to ALL kinds of behavior.
Other Trends
Among the top-quartile articles, there was a slight tendency for very recent ones to be overrepresented: 27% were from 2013 or earlier; 34% were from 2014-2018; and 39% were from 2019-2023. In part, this probably is because the kinds of electronic sources that the database tracks get more use each year and thus produce more mentions in general. Below you can see how three types of mentions have increased in recent years. Based on this pattern, you’d expect mentions to increase for just about any randomly selected journal.
And yet that’s not true of all behavior analysis journals. Below is the trend over time for the six journals with the most top-quartile articles. JCBS, BAP, and possibly JABA show the expected increasing trend. But JEAB and Behavior Modification have been trending toward less dissemination impact as measured by the Altmetric Attention Score. I can’t say why this might the case; if you have ideas let me know.
Not All Attention is Desirable
I’ve mentioned that some articles get a lot of altmetric attention in social media, and also that goofy things can happen in social media. In the latter case, remember that social media platforms pretty much allow anybody to say pretty much anything, and this allows for mischief. In a previous post I wrote about how neurodiversity advocates used social media to attack a fairly mundane 2022 applied behavior analysis (ABA) article. The particular grenade that neurodiversity advocates like to lob is that ABA constitutes torture and abuse, and I speculated in my earlier post that in social media they tend to lob this grenade at journal articles when they see key words like “applied behavior analysis” and “autism.” They may not even read the article….
And there’s possible evidence for exactly that in my survey. Among the 330 top-quartile articles, I checked out the comments for 21 that received at least 100 posts on Twitter/X. For three of these the comments were not in English. Among the remaining 18, “neurodiversity bombing” accounted for a substantial proportion of post for 12 of them. In other words, in 2/3 of the cases I could check, a hopeful sign of dissemination impact turned out to be something else entirely.
It’s interesting that, among the 12 articles mentioned above, in 9 cases the title included “autism” and/or “ABA” or the article appeared in the primary journal with ABA in its name (JABA). Lots of other ABA/autism articles did not attract the attention of the neurodiversity crowd, even though they focused on individuals and procedures much like in the articles that were targeted. This is consistent with my speculation that those critics rarely bother to read the articles. Also consistent: Most of the criticism I saw was boilerplate stuff that said little about the actual study (e.g., “Another example of ABA abuse.”).
I’ve described this problem in some detail to illustrate that counts of altmetric mentions are nice, but if you really want to be informed about dissemination impact you have to see what people out there are saying about a given article. If you were to subtract out neurodiversity bombing from the altmetric data set, at least a dozen articles would drop out of top quartile (these articles got most of their mentions in Twitter/X, and those turned out to be criticism).
next time: the 21st century articles with the most scholarly attention
Postscript: A Limitation of Altmetric Data
It should be noted that Attention Scores reflect mentions in electronic sources that the Altmetric.com database monitors. It doesn’t monitor paper-only sources (like print news) and it samples only selected digital sources. For instance, Twitter/X and Facebook are included but Instagram and Tik Tok are not. In a previous post I described one behavior analysis article that got extensive news coverage in about 100 different outlets; however, the database recorded only a minority of these.
For what it’s worth, this problem has a parallel in the various tools that monitor scholarly citations, none of which can be considered 100% inclusive. The best you can do is to compare different articles using the same tool to see their relative citation counts. There’s no illusion that you’re necessarily tracking every single citation that exists.
Similarly, altmetric data provide a way to register relative amounts of attention earned by different articles, but they aren’t the last word on the absolute amount of dissemination impact created by any given article.
Top 30 Behavior Analysis Articles (Published Since 2000) in Terms of Altmetric Attention Score
See at bottom for a key to journal name abbreviations. Publication year reflects online-first release where relevant.
Key to Journal Name Abbreviations
BAP = Behavior Analysis in Practice
BI = Behavioral Interventions
BM = Behavior Modification
BSI = Behavior and Social Issues
EJOBA = European Journal of Behavior Analysis
ETC = Education and Treatment of Children
JABA = journal of Applied Behavior Analysis
JCBS = Journal of Contextual Behavioral Science
JEAB = Journal of the Experimental Analysis of Behavior
JJBA = Japanese Journal of Behavior Analysis
JOBE = Journal of Behavioral Education
JOBM = Journal of Organizational Behavior Management
JPBI = Journal of Positive Behavioral Interventions
POBS = Perspectives on Behavior Science
TAVB = The Analysis of Verbal Behavior
TPR = The Psychological Record