I participated as a subject in a research study at Stanford involving race and stereotype threat in the early 2000s. The details are hazy, but the final readout was the distance I put my chair to a group of chairs that students of a particular racial group were supposed to sit. Evidently I put them in a position that was contrary to the effect the researcher was seeking. She intensely asked me a ton of questions about my background and eventually tossed my data point for having lived in a racially diverse area growing up. This wasn't a pre-inclusion criteria, but a possible act of scientific fraud. Huge bummer since there are honest people in every profession, and I imagine a lot of them didn't succeed the way that the fraudsters thrived.
I've had a similar experience in a long running survey. When I gave the "wrong" answers, the interviewer asked a bunch of questions and eventually told me to skip certain questions.
I had an experience like that where an education researcher was attempting to prove that a certain style of teaching was more effective. It involved a test measuring how much students remembered from a conventionally-taught course taken the year before; the researchers hoped to show that these scores were low, and therefore that conventional teaching methods were ineffective. I and another classmate aced the test so the researcher accused us of cheating, even though there was no incentive to cheat (the test wasn't used for any grade, so if anything, the incentive was to save time by leaving answers blank). We denied cheating, explained that the course instruction had been very memorable, and proved it by correctly answering followup questions on the spot. Ultimately our high-scoring test results were discarded as outliers and the hypothesis was successfully validated.
Usually psychology studies are conducted by field workers or assistants. I don't think the person you interacted with was the actual author of the study.
It sounds more like another day in vegas for the psychology field of that era. This was not an exception, and often researchers were not even aware they were doing something wrong. Even nowadays psychology researchers are clueless as to what really p-hacking and bad statistical practices mean. And because they consider themselves honest researchers, while these practices are obviously dishonest, they do not consider themselves actually doing anything like that - but somebody else may! it is always somebody else. Now it is not as bad as that period, and there is more awareness about the most blunt violations of statistical rigour, but the actual understanding is still low for the median researcher so many grey-to-black zones exist still.
Incentives determine outcomes. Most grad students are grad students because they are responding to incentives. It's competitive so cheating is ubiquituous. Reduce competition in academia and problems would lessen.
As someone who went through grad studies in the field, and who has met Dr. Inzlicht in person before, I have to say I deeply appreciate his perspective. He has consistently been humble when facing the issues in the field, in ways that call even his own previous research (following the typical practices of the day) into question. The field as a whole has been undergoing a reckoning, but Mickey has been one of the people who has encouraged his fellow researchers not just to wag their fingers at others, but also to look inward and reflect on their own research practices. He has done so by showing humility and acknowledging where his research has fallen short, and that indicates to me a great deal of integrity.
It is sad to see stereotype threat being one of those findings that seems less and less credible. I once worked as a research assistant on a project related to stereotype threat, and I recall the study going through several iterations because it all needed to be just so -- we were testing stereotypes related to women and math, but the effect was expected to be strongest for women who were actually good at math, so it had to be a test that would be difficult enough to challenge them, but not so challenging that we would end up with a floor effect where no one succeeds. In hindsight, it's so easy to see the rationale of "oh, well we didn't find an effect because the test wasn't hard enough, so let's throw it out and try again" being a tool for p-hacking, file drawer effects, etc. But at the time...it seemed completely normal. Because it was.
I'm no longer in the field, but it is genuinely heartening that the field is heading toward more rigour, more attempts to correct the statistical and methodological mistakes, rather than digging in one's heels and prioritizing theory over evidence. But it's a long road, especially when trying to go back and validate past findings in the literature.
>It is sad to see stereotype threat being one of those findings that seems less and less credible.
While I'm sure it is an honest statement, this sentiment is itself concerning. Science is ideally done at a remove - you cannot let yourself want any particular outcome. Desire for an outcome is the beginning of the path to academic dishonesty. The self-restraint required to accept an unwanted answer is perhaps THE most important selection criteria for minting new academics, apart from basic competency. (Acadmeia also has a special, and difficult, responsibility to resist broader cultural trends that seep into a field demanding certain outcomes.)
> While I'm sure it is an honest statement, this sentiment is itself concerning. Science is ideally done at a remove - you cannot let yourself want any particular outcome.
This basically never happens. I worked in academia for many years, and in psychology for some of that, and I have never met a disinterested scientist.
Like, you need to pick your topics, and the research designs within that etc, and people don't pick things that they don't care about.
This is why (particularly in social/medical/people sciences) blinding is incredibly important to produce better results.
> The self-restraint required to accept an unwanted answer is perhaps THE most important selection criteria for minting new academics,
I agree with this, but the trouble is that this is not what is currently selected for.
I once replicated (four times!) a finding seriously contrary to accepted wisdom and I basically couldn't get it published honestly. I was told to pretend that I had looked for this effect on purpose, and provide some theory around why it could be true. I think that was the point where I realised academia wasn't for me.
Now, the same thing happens in the private sector, but ironically enough, it's much less common.
You were told by whom? An editor? A reviewer? To how many journals did you submit this research? Did you work your way from top tier journals downward? In general, almost all empirical research will find a home. I find your story less than believable without some additional context, disgruntledphd2.
I was a PhD student, I was told this by both my supervisors and much of my department.
I attempted to submit it to about 3-4 journals and never even got it sent to review.
Ultimately I left academia both for reasons research and financial, so I stopped.
I'm a little confused as to why you are doubting an anonymous comment on the Internet deep in an old thread on a mostly meaningless message board, but I guess that's life.
I don’t think there’s anything wrong with wanting a particular outcome. It wasn’t wrong that people were excited about the prospect of a room temp superconductor a few months back because people knew that if it were true good things were possible. Insisting that you can’t be excited by one potential outcome from a study means that you’ll only study things that don’t have the potential to help
There is a useful distinction between "wanting" and "attachment", but one usually turns into the other. Your mention of room temp superconductors is ironic since they have all been precisely attachment-driven frauds that start with wanting.
These are the exceptions imo. Most of it is funding-driven (or sometimes wanting-status-driven) than pure wanting-driven. Most researchers do not even care that much about their actual field and would change the field to do what they really want to do, albeit funding keeps there where they are.
Unsurprising. Although this is the first time I can recall reading a psychologist accept culpability for the field’s bad “science” over the last twenty years. Have any of the “anthropology of science” researchers published an explanation of that yet?
Everyone says "unsurprising" after the replication fails. Idk, this was a really popular theory that lots of people believed. I doubt you were all so confident back in the heyday of this social science phenomenon.
If tomorrow, they say "growth mindset" is also a non-replicable phenomenon, will HN be full of smug people saying "I knew it all along, lol!"
Stereotype threat was a “popular theory” because it fit the social and political presuppositions that were fashionable at the time. It’s almost always wise to be skeptical of political fashions or new social science findings even on their own; new social science findings that match political fashions are doubly questionable.
It might have been surprising at the time, but it's certainly unsurprising in light of the replication crisis. We now know that most findings don't replicate. Anyone who continues to be surprised by the most likely outcome needs to update their priors.
>Biostatisticians Jager and Leek criticized the model as being based on justifiable but arbitrary assumptions rather than empirical data, and did an investigation of their own which calculated that the false positive rate in biomedical studies was estimated to be around 14%, not over 50% as Ioannidis asserted.
That’s just a heterogeneous population with selection bias. Everyone is always convinced because when they’re disproven they stay silent. These days prediction markets allow people to convert beliefs to money so it becomes not very convincing when everyone manages to be perfect at predicting things and is always complaining they aren’t making enough money while simultaneously giving up the chance at doubling their bet on a sure shot.
Combine that with people upgrading uncertainty to certainty post-hoc when debunking comes out and you have these entertaining things. Overall, I’m glad you called it out. Once I wished I had a profile for people’s past guesses to see how actually good they are and now I have Manifold, Kalshi, and Polymarket.
To me it seems like "growth mindset" is a buzzword that everyone defines differently.
The weak version is something like: "Even talented people need to work on improving themselves, if they want to be successful." Most people would agree with this.
The strong version is something like: "If people believe that there is such a thing as talent, they automatically stop working on improving themselves." This is used to justify things like: "never tell your kids that they are talented or intelligent, you must attribute all their success to their hard work". In my opinion, this is a false dilemma: there can be things like talent or intelligence, and hard work can be necessary to succeed among those who are comparably talented or intelligent.
FWIW, to me it is "unsurprising" as I can't remember the last time a major psych study actually did replicate, and it seems frankly like the entire field is in shambles. Does that mean that I knew before that this study in specific was false? No. But, that isn't what anyone means when they say "unsurprising"! If you get to the end of a movie and all of the major characters end up alive, and someone else seemed shocked by that fact, you might still explain to them that that's "unsurprising", even if you yourself got to feel the thrill of uncertainty -- or even were concerned for a bit -- while you were watching.
> If tomorrow, they say "growth mindset" is also a non-replicable phenomenon, will HN be full of smug people saying "I knew it all along, lol!"
I will be one of them! Many people I've met in the industry—disturbingly, especially among founders and VCs—are extremely susceptible to reductive theories about the human experience that have little relation to reality. Just in the last year we went through "founder mode" hell even though any person that this "founder mode" describes should probably be in prison for wildly abusive behavior.
Of course the flip-side of this is that there are concepts that are situationally useful even if they aren't broadly applicable. Both "stereotype threat" and "founder mode" may fall into this category. (I've never heard of "growth mindset" before and looking it up I'm not sure what the point of it is—I guess to contradict the idea that people and personalities are static?)
My understanding is that "growth mindset" represents a personal philosophy, not a testable hypothesis, so it would be irrelevant here.
It was entirely reasonable to be skeptical of stereotype thread when the concept was new, a priori, An "unsurprising" result is not necessarily one that someone confidently believed. If I flip a coin, I'm not "surprised" when the result is heads, nor when the result is tails.
I could be wrong but I believe the idea is that the growth mindset has measurable consequences. I.e., if you believe you can grow, you will learn/adapt more effectively to new situations and ultimately achieve more. It's an empirically verifiable claim that is either true or false, and comes with a pretty straightforward intervention (i.e., teaching people that they can grow) if true.
Of course people may also adopt it as a personal philosophy, but that's separate.
Even if you did suspect something was wrong you were careful because science said otherwise. Calling this wrong in public would feel like saying a big rock falls faster than a small rock - something else that seems obvious but science has disproved. And so the only one loudly saying this was bunk where the right wing crackpots (the reasonable people on the right side were much quieter about it because they don't like to argue with science even though it went against their bias) - the right wing crackpots were on this only because the left wing embraced it as a science that confirmed their bias. Unfortunately in this cast the conspiracy was true and so the crackpots won (even though they have no understanding of the real reasons it is false and it doesn't seem to have been a conspiracy).
Let this be a lesson: even if something seems like science and it confirms you bias - that doesn't mean it is true. You should look closer at things you embrace than things you reject lest you embrace a lie.
> Even if you did suspect something was wrong you were careful because science said otherwise.… And so the only one loudly saying this was bunk where the right wing crackpots (the reasonable people on the right side were much quieter about it because they don't like to argue with science even though it went against their bias) - the right wing crackpots were on this only because the left wing embraced it as a science that confirmed their bias.
If “crackpots” turn out to be right when “reasonable people” and “science” were wrong—and this is far from the only instance of this happening—maybe we should reevaluate some things.
Just because a peer reviewed paper published in a prestigious journal says something doesn’t mean it’s true. Even a survey of multiple peer reviewed papers published over time isn’t necessarily determinative if there are common methodological issues or publishing biases. Yes, a lot of times actual crackpots will make stupid criticisms, but not every criticism is stupid even if it comes from outside the ivory tower.
In particular, whether or not a big rock falls faster than a small rock [1] is a fairly basic question in physics, which has one of the most certain answers. Virtually nothing in psychology, let alone human social psychology, is at that level of certainty, and any psychologist worth their salt will agree with that. Basically any finding in psychology should have a level of certainty somewhere between “yeah, that’s probably mostly true” and “hmm, interesting hypothesis that’s not entirely crazy, I wonder if it holds up”.
[1] Also, the literal question of whether a big rock or a small rock falls faster is trickier than you might assume if you only know the middle school version of the question. If we’re doing this on the moon, so as to dispose with the tricky aerodynamic questions, and we are answering the question from the frame of reference of an astronaut standing on the surface of the moon, the bigger rock actually does fall faster. Both rocks accelerate towards the center of the moon at the same rate, yes, but gravity works both ways. Both rocks pull on the moon but the bigger rock pulls harder, resulting in an extremely small torque on the lunar surface, drawing the side of the moon closer to it very slightly upwards. Which, from the surface frame of reference, is equivalent to the bigger rock falling faster.
The way you've put it, the "crackpot" and "reasonable" right wingers seem to have the same beliefs, but differ in their willingness to speak their mind or the ease with which they're bullied into silence.
I was a psych major in undergrad, and did an experiment as a riff on stereotype threat and got a small effect. I had the participants solve brain teaser puzzles and the only difference was introducing them as coming from 11th grade or graduate level math. Undergrads did worse when they thought it was graduate level.
>this study (still a preprint) was preregistered (meaning all methods and analyses were specified before the data were collected)
This is the way forward -- preregistered studies. That, together with a promise from the publisher to publish the result regardless of whether the effect is found to be significant.
When you think about it, the incentives for publishing in science have been wrong all along. The future will be different: It will be full of null results, of ideas people had that didn't pan out. But we'll be able to trust those results.
Also interesting: p28 (labeled p470) of https://journals.sagepub.com/doi/pdf/10.1177/251524591881022... ("Many Labs 2: Investigating Variation in Replicability Across Samples and Settings") shows the results of attempting to replicate 28 published psych results across many different samples.
Unfortunately, "Stereotype Threat" is not one of the effects they attempted to replicate.
> Let’s be honest: that last sentence was far too generous. Many of us engaged in practices that, in hindsight, were borderline dishonest. We abused experimenter degrees of freedom, engaged in questionable research practices, p-hacked, massaged our data—you pick the euphemism. In contrast, this new replication study followed the most up-to-date best practices in psychological science, eliminating room for flexibility in analysis or results interpretation.
Exploiting researcher degrees of freedom remains unfortunately extremely common. There needs to be some sort of statistical vanguard in the ivory towers enforcing real preregistration and good analysis practices. Strict epistemic discipline is necessary to do real science.
The real problem is we have a lot of data. So it is easy to shove it all into a computer and then see what comes out. This is not a bad thing, but it isn't alone a result to publish it is instead justification to do a real study on whatever thing statistics turns up as a significant result to make sure it really is when you properly isolate for that data.
There’s also the issue that in psych, everything is measured through indirect proxies and most of the predictions will be directional. That gives experimenters a LOT of wiggle room.
In physical science you can often directly measure a phenomenon and have a theory that makes very specific predictions
early in my psych career, i was actively encouraged to do things that would be considered p-hacking or abusing degrees of freedom. it was seen as a way to not let any data go to waste and potentially 'show off' your cleverness/mathematical chops in the methods section (anybody can just run an ANOVA like they teach in the text book, but look what I can do).
I have seen stereotype threat mentioned in educational contexts - justifying the idea that, for example, it’s important not to make (deliberately or inadvertently) things like coding or engineering seem stereotypically masculine. Usually the recommendations are to ensure that if pictures of participants in classes or extracurricular programs are shown in advertising, diverse groups of people are pictured - or that if connections to popular culture are made in educational materials they’re diverse - for example, don’t make all your example coding projects about Star Wars or football.
Anecdotally, I’ve seen with my own eyes, for example, girls getting really into coding only after seeing it demonstrated by enthusiastic women that they can see as role models in ways they would not see men.
I guess this is a far broader thing than stereotype threat - but I’m sure this larger thing is real. I fear that people who themselves have stereotypes in mind about who ‘should’ be into certain topics will use the demise or deemphasis of stereotype threat to justify not making attempts to attract or be friendly to kids who really could flourish in non-stereotypical fields - to their and society’s detriment.
Yep, I agree that this kind of descriptive representation, role model thing is very real and can have significant effects. It's the best kind of evidence that someone of the same demographic likely can/can't achieve similar things.
one of my stances has been that psychology comes up with wacky unreplicable findings because there's no central organizing theories for how 'the mind' works and everything is just very black-box.
there are so many studies showing "X manipulation affects Y outcome", but there's not even a hint of an attempt to explain the mechanisms in a meaningful way (cognitive experiments are usually better, but often still guilty of this).
I actually disagree entirely. The amount of forcing theories that are well ahead of the facts that occurs in academic psychology is off the charts.
We could do with a shut up and experiment movement, to be honest.
In order to build useful theories, we need lots more data, and forcing theories onto data actively holds us back from running the wide variety of experiments necessary for a real theory to emerge.
I agree that there isn’t as much raw experimentation and just observing/documenting psychological phenomena that there should be. I’ve ranted about that here in the past.
I guess my peeve is that nobody really collaborates on theories. Every PI is off working on their own set of pet theories (because that’s how you establish a career), but there’s really no centralizing force to get people to collaborate and work on a shared theory. Physics has the standard model that everyone can use as a common reference point and it’s helpful.
This is partly the "psychology is the study of college undergrads" problem. The original study was on Stanford students, all of whom have already passed through a very selective filter. That's not a group to extrapolate to the general population. Too many psych studies are done on this convenient population.
It's great seeing the original author admit the problem.
I also have a high degree of skepticism about most psychology research. I find it frustrating that the author of this piece raises the issue of the imbalance of gender in STEM jobs as a reason to doubt this particular study. There is nothing about this failure-to-replicate that should allow us to conclude that innate differences in math ability underly that imbalance.
I’d love others to read the replication report and explain why I might be wrong?
The author doesn't claim that the imbalance is a reason to doubt the study. His claim is that the imbalance is a reason to doubt the claim that an underlying shift in cultural norms explains why the results might have been valid in 2005 but failed to replicate in 2024.
The author rightly observes that despite the undeniable shift, women are still significantly underrepresented in STEM and therefore that cannot explain the lack of replication. There are still many other reasons besides innate differences that might explain it.
I mean come on - if there is a pervasive cultural insinuation of gender differences in ability that leads to women experiencing more test anxiety (I’m not saying there is, but conditioning on that hypothesis) why would we expect a 20 minute PowerPoint to counteract 20 years of people’s life experience!?!?
Is there any evidence you can conceive of that would convince you that there is some innate group difference in some aptitude that’s valued by society?
I'm curious what the impact for most people's mindsets will be here? I'm imagining that it relates to what many people consider "talent" with kids? I know that has been an odd trend in some states trying to get rid of "gifted" programs and such. I have largely remained hopeful that that was not nearly as prevalent as online represents.
When I first heard about this stuff 20 years ago, it was being presented as "The differences between groups A and B go away when stereotype threat is removed" which is not what the original paper says. The original paper claims that whatever difference there is between groups A and B, it will be larger when measured under the condition of stereotype threat which I guess is plausible but much less interesting.
Part of the context of that time was that _The Bell Curve_ had been published fairly recently and there was great desire to disprove it and anyone doing that could count on lots of attention and speaking fees. So the grift was to present stereotype threat as this grand solution that could resolve all racial differences.
> When Black students at Stanford University were told that a test was diagnostic of intellectual ability, they performed worse than their white counterparts. However, when this stereotype threat was ostensibly removed—by simply framing the test as a measure of problem-solving rather than intelligence—the performance gap Black and white students nearly vanished.
Just reading this description motivates me to reject the study out of hand. It's not plausible that university-level students responded meaningfully differently to being told "this is a test of problem-solving skill" versus "this is a test of intelligence" because it is commonly understood that problem-solving skill is a major component of intelligence.
>it also became the darling of the political left who now had an answer to prevailing views of group differences held by the political right. This is partly because shortly before stereotype threat took its turn in the spotlight, Charles Murray and Richard Herrnstein published The Bell Curve... the octogenarian Murray is still considered a pariah, shouted down and deplatformed from talks he tries to deliver at respectable colleges to this day.
The characterization of Murray's views in the last several years has been grossly uncharitable and seems entirely disconnected from his actual arguments. It's strange that the book is 30 years old, but has seemed politically relevant for much less time than that.
Many years ago, long before "The Bell Curve", I spent way, way too much time reading Murray's book "Losing Ground" (1984) and going to the library to read the papers he referenced and reading the papers those papers referenced and then the papers those papers referenced. My conclusion was that his book is junk. My second conclusion was that citations are like a game of telephone and every time someone cites something, it changes slightly, so don't trust anything until you get to a primary source, and maybe don't even trust that. (This second conclusion applies in general, not just to Murray.) My third conclusion was that arguing online is a waste of time and approximately nobody cares about actual facts.
> My second conclusion was that citations are like a game of telephone and every time someone cites something, it changes slightly, so don't trust anything until you get to a primary source, and maybe don't even trust that. (This second conclusion applies in general, not just to Murray.)
This part is definitely true. It has also been my experience -- but more so within academic areas with sloppy "researchers" and political problems. Linguistics is pretty good, gender "research" is awful. Intelligence research is pretty damn good, social "science" and psychology (outside of psychometrics) is awful. Economics is awful, apart from the basic ideas of competition, capitalism, and low taxes. Keynesianism is largely a fraud.
Your interlocutor might not care about facts but the audience might. I’ve learned interesting new things from reading the back-and-forth of others going at it online.
Eh, I mean the stereotype being "activated" is that Black people are less intelligent / have lower IQs, not that they lack problem-solving skills.
Even the replication attempt had two scenarios:
1 where women were told the test was to establish performance levels on the test between men and women
1 where they were told the test was a test of problem-solving skill (or primed to disregard negative stereotypes before the test).
So even the replication has the incorrect framing you worried about. I tend to believe the problem wasn't this, but the way the field was lax about sampling, methodology, etc. After all, there were many stereotype threat studies, not just this one, boasting similar results. And they didn't all use that framing.
Not to necessarily defend the study here, but I don't think that's a good reason to dismiss the study out of hand. Words themselves, the actual sounds independent of their technical meaning, still have very strong, deep-rooted associations that are hard to fight even if you've been educated. The subtext and day-to-day use of "intelligence" and "problem solving skill" are different, and color the way you think of them even if you know they're technically synonymous (which, do note, they're not quite). That association operates at the same deep level at which the supposed effect works.
The characterization of Murray's views seem extremely accurate and the fact that his book is still brought up despite being disproved as racist trash decades ago is ridiculous
That book was not very good science or science interpretation and didn’t do a great job of proving that meaningful race-iq differences exist. I also don’t know where the disaster is, could you elucidate some of the cataclysmic results of DEI?
I’ve seen and read extensive reviews, but yeah I have not read it. I agree that’s a blind spot, so I’ll go ahead and read the book then get back to you after. Give me like a week.
It is wildly untrue that these bullets are all broadly accepted in the actual literature or among modern practitioners. This is more like a list of things you happen to believe. Which is fine, but be clearer about it.
Why do you think it’s “fine” that the parent commentator thinks “ yes, some population groups really are more intelligent than others”? What do you think he means by the “truly taboo comparisons” and why do you think he’s too chickenshit to say what he means? Why is that “fine” instead of “grounds for immediate ostracism”?
Are you ascribing that to innate genetic IQ inferiority or to social and environmental causes?
Eg: Resource starved German schoolchildren in WWII scored almost two std deviations lower than the same demographic from the same classes and schools five years earlier.
I mostly just want to preempt the inevitable citation to IQ & The Wealth Of Nations; it's a little squicky for me to predict it's coming (but it was definitely coming).
Sure, this John C Calhoun level racist will probably just change his mind about Blacks being untermenschen if you’re just polite enough to him.
I understand that the stated HN policy is that polite, fig-leafed White supremacy and 1920’s style race science is fine and name-calling is not, but get real. Your attitude is not just not helpful, it actively helps normalize scumbags like this piece of shit.
You're coming across as someone terrified that the literature is going to back this person up. It does not. Can you just go to Twitter and dunk on HN instead? All you're doing here is this person's work for them.
You're writing like the objective of the thread is to communicate with this particular person. It is not. The point of a thread is every other lurker reading it.
It’s extremely clear you don’t have any personal real-world experience with this, but let me assure you that no one who thinks Blacks are of sub-human intelligence gets there by sitting down and digging into research and weighing the various arguments for and against. They start with the prejudice and they latch on to whatever allows them to justify it. There is no point in making arguments at all, either to this particular piece of shit or to whatever pieces of shit will permit a nasty internet comment to justify prejudices they already hold. The only thing that works is to make it clear to these people that no one is fooled about what their views really are, that those views are repulsive to decent ordinary people and that holding them is grounds for ostracism and abuse.
Politely tolerating and debating them doesn’t sway any bystanders away into thinking “hey, maybe Blacks are fully equal to other humans after all”, it only gives social permission to fellow travelers who would otherwise feel pressured to conceal or temper their racism, and normalizes the idea that the essential humanity and equality of Blacks and other minorities is a legitimate topic of debate.
Once again: I do not care what the person you are threatening thinks. I do not think I am going to persuade him. What I care about is that he's wrong, and that looking super dumb on a thread about science is a lot more interesting and persuasive than a throwaway account pitifully threatening to beat him him.
You write like you don't know why he's wrong. Let me reassure you: he is. Look it up!
What you think you’re doing is showing off how smart and clever you are. What you’re actually doing is normalizing the idea that the essential humanity and equality of Black people is a legitimate topic for debate. Stop doing this.
> I think you're genuinely afraid you could lose an argument with that person, and that's sad.
No, you stupid motherfucker, I think it’s gross and debasing that anyone would think the argument is worth having in the first place, let alone validating, and if you can’t understand why, I think maybe you should consider stepping outside of your own smug interior life for a moment.
No, I don't think psychometrics and heritability research is like stereotype threat or "sky fathers", whatever that is. But I think we're clearer at this point where that list came from, and probably don't need to take this further.
No? You provided a list of things you personally believe to be supported by the balance of evidence (in several different fields of study), and many of them --- maybe more than half --- are things that are unsettled, or even unlikely to be true. Why would I retract anything?
Half? Seriously? I could understand believing a handful of them are "unsettled, or even unlikely to be true" given the current debate climate.
Honest question, no attempt at rhetorical jiu-jitsu or anything -- why do you believe "maybe more than half"? You must base that estimate on something.
Yes: I read papers and keep up with stuff. It's cute that you think we're going to dig into this rigorously after "Communism, sky fathers, and crystals", but we obviously will not. Again, no, I will not be "retracting" anything.
Differences in IQ currently exist, they're not based on racial genetics.
The Bell Curve isn't about organizing society fairly it's about justifying what is clearly an unfair and unjust society as natural and impossible to improve and therefore good
Executive Summary of Article: "new data now reveal what many of us suspected for at least ten years: stereotype threat does not replicate, and it does not undermine academic performance in the ways we thought."
The Stereotype Threat: "individuals who are part of a negatively stereotyped group can, in certain situations, experience anxiety about confirming those stereotypes, leading paradoxically to underperformance, thus confirming the disparaging stereotype." for example, if you remind a woman of the "Women are bad at math" stereotype, they will perform worse on a math test than if they are not reminded of that stereotype.
This has been an ongoing situation since around 2010. The field has been undergoing a huge shift in terms of re-evaluating research practices and past findings. It genuinely has nothing to do with national politics.
Also, the author is Canadian. So there's that, too.
>This has been an ongoing situation since around 2010.
Exactly. Yet the time to declare your change of heart is somehow now :)
>Also, the author is Canadian. So there's that, too.
Search for his grants brings grants from US, he studied and did PhD in US and still affiliated with NYU, and the major and most valuable part of his network of academia connections is US, his private foundation in USA, etc. If he isn't in-line with the prevalent winds in US, he will be reduced to nothing, just a professor in some Uni, not a renown researcher with his own Wikipedia page, grants and conferences in US, etc.
Yes, man, for 20+ years you are peddling some politically favored theory, supposedly doing experimental studies (by 2013 already getting at least $3.75M dollars in grants - https://www.apa.org/monitor/2013/09/apf-grant ), and you have no idea that your theory is total BS until suddenly somebody else failed to reproduce yours and/or similar studies, and that is exactly when political climate is changing 180. It isn't a science, it is an ideology department serving whatever political regime is in power at any given moment.
Inherited wealth was actually pretty rate in the history of this country, and is not as common now as many people think. Generational wealth until recently did not exist for most people.
I don't think that's what people have been taking away from the idea of stereotype threat at all. I think it's been much more likely for people to think, "Oh, instead of unchangeable genetic traits, the difference in outcomes might someday be reduced through deliberate social change.
Putting on my contrarian hat: the studies he mentions seem to be concerned with whether testing conditions affect stereotype threat effects. Logically, they can't prove or deny the phenomenon's existence, only whether testing conditions result in changes to test outcomes that could track with interventions to reduce stereotype threat. Much of what we know about how behavioral effects of identity comes with the understanding that the latter is something people carry with them, regardless of local or recent events. If there's a problem with stereotype threat as a concept, it's that it's positioned as a superficial effector that can be manipulated easily, rather than the surface level manifestation of complex interactions between self-identity, personal values, and cultural expectations. Based on the author's disclosure about his PhD thesis, he seems to be someone who capitalized on the former characterization, so of course he throws the baby out with the bathwater when it no longer works for those purposes. We might be looking at an ass-covering write-up.
>Let’s play “Find the Lebowski quotes game” again!
So, yeah, I find this a deeply unserious blog post.
He must throw the baby out with the bathwater - so far there is no evidence for anything here. The only alternative is to rerun the study, doing it correctly this time and then do a correct analysis. Until then you need to say "I don't know" when asked about this as that is all science allows. Maybe the rest of the baby they are throwing out will replicate and thus is correct - but nobody knows that and so we cannot say anything with confidence right now.
I'm assuming you are not aware of studies not mentioned here that replicate - I'm not in this field and so I would not know where to look. I'm guessing that you also are not in this field and are looking for some way to allow your bias to become true despite these issues - but of course I might be wrong.
Let me analogize. This is a blog post that claims that racism isn't real. As proof, the author explains that none of the studies that show a reduction in racism when you remind people that Michael Jordan and Beyonce exist can be replicated. Well, okay, sure. And the guy who earned a doctorate with the thesis, "Does Playing Jackson 5 Make People Less Racist?" might want to turn this into, "Racism does not exist," as a way to distance himself from his own terrible scholarship. (Also, did you find his Marvel Cinematic Universe Easter Eggs?)
So, thus far, there is actually not enough evidence to throw the, "Racism exists," baby out with the, "Do these interventions affect racist belief?" bathwater. And since we're second-guessing biases, it's super weird that you're always in comment sections of (politically-charged) articles concerning fields you're not in.
I think that contrarian hat may include a no-true-scotsman badge.
If stereotype threat is real, we should be able to have a replicatable study result that confirms it, right? We're not just limited to logical inference, I hope.
I participated as a subject in a research study at Stanford involving race and stereotype threat in the early 2000s. The details are hazy, but the final readout was the distance I put my chair to a group of chairs that students of a particular racial group were supposed to sit. Evidently I put them in a position that was contrary to the effect the researcher was seeking. She intensely asked me a ton of questions about my background and eventually tossed my data point for having lived in a racially diverse area growing up. This wasn't a pre-inclusion criteria, but a possible act of scientific fraud. Huge bummer since there are honest people in every profession, and I imagine a lot of them didn't succeed the way that the fraudsters thrived.
I've had a similar experience in a long running survey. When I gave the "wrong" answers, the interviewer asked a bunch of questions and eventually told me to skip certain questions.
I had an experience like that where an education researcher was attempting to prove that a certain style of teaching was more effective. It involved a test measuring how much students remembered from a conventionally-taught course taken the year before; the researchers hoped to show that these scores were low, and therefore that conventional teaching methods were ineffective. I and another classmate aced the test so the researcher accused us of cheating, even though there was no incentive to cheat (the test wasn't used for any grade, so if anything, the incentive was to save time by leaving answers blank). We denied cheating, explained that the course instruction had been very memorable, and proved it by correctly answering followup questions on the spot. Ultimately our high-scoring test results were discarded as outliers and the hypothesis was successfully validated.
Usually psychology studies are conducted by field workers or assistants. I don't think the person you interacted with was the actual author of the study.
Grad students and post-docs are often running studies.
> a possible act of scientific fraud
It sounds more like another day in vegas for the psychology field of that era. This was not an exception, and often researchers were not even aware they were doing something wrong. Even nowadays psychology researchers are clueless as to what really p-hacking and bad statistical practices mean. And because they consider themselves honest researchers, while these practices are obviously dishonest, they do not consider themselves actually doing anything like that - but somebody else may! it is always somebody else. Now it is not as bad as that period, and there is more awareness about the most blunt violations of statistical rigour, but the actual understanding is still low for the median researcher so many grey-to-black zones exist still.
> and often researchers were not even aware they were doing something wrong.
Unfortunately true. In other words: "often researchers were too stupid to ever have been let into university as freshmen".
Incentives determine outcomes. Most grad students are grad students because they are responding to incentives. It's competitive so cheating is ubiquituous. Reduce competition in academia and problems would lessen.
How would you know if they excluded your data?
As someone who went through grad studies in the field, and who has met Dr. Inzlicht in person before, I have to say I deeply appreciate his perspective. He has consistently been humble when facing the issues in the field, in ways that call even his own previous research (following the typical practices of the day) into question. The field as a whole has been undergoing a reckoning, but Mickey has been one of the people who has encouraged his fellow researchers not just to wag their fingers at others, but also to look inward and reflect on their own research practices. He has done so by showing humility and acknowledging where his research has fallen short, and that indicates to me a great deal of integrity.
It is sad to see stereotype threat being one of those findings that seems less and less credible. I once worked as a research assistant on a project related to stereotype threat, and I recall the study going through several iterations because it all needed to be just so -- we were testing stereotypes related to women and math, but the effect was expected to be strongest for women who were actually good at math, so it had to be a test that would be difficult enough to challenge them, but not so challenging that we would end up with a floor effect where no one succeeds. In hindsight, it's so easy to see the rationale of "oh, well we didn't find an effect because the test wasn't hard enough, so let's throw it out and try again" being a tool for p-hacking, file drawer effects, etc. But at the time...it seemed completely normal. Because it was.
I'm no longer in the field, but it is genuinely heartening that the field is heading toward more rigour, more attempts to correct the statistical and methodological mistakes, rather than digging in one's heels and prioritizing theory over evidence. But it's a long road, especially when trying to go back and validate past findings in the literature.
>It is sad to see stereotype threat being one of those findings that seems less and less credible.
While I'm sure it is an honest statement, this sentiment is itself concerning. Science is ideally done at a remove - you cannot let yourself want any particular outcome. Desire for an outcome is the beginning of the path to academic dishonesty. The self-restraint required to accept an unwanted answer is perhaps THE most important selection criteria for minting new academics, apart from basic competency. (Acadmeia also has a special, and difficult, responsibility to resist broader cultural trends that seep into a field demanding certain outcomes.)
> While I'm sure it is an honest statement, this sentiment is itself concerning. Science is ideally done at a remove - you cannot let yourself want any particular outcome.
This basically never happens. I worked in academia for many years, and in psychology for some of that, and I have never met a disinterested scientist.
Like, you need to pick your topics, and the research designs within that etc, and people don't pick things that they don't care about.
This is why (particularly in social/medical/people sciences) blinding is incredibly important to produce better results.
> The self-restraint required to accept an unwanted answer is perhaps THE most important selection criteria for minting new academics,
I agree with this, but the trouble is that this is not what is currently selected for.
I once replicated (four times!) a finding seriously contrary to accepted wisdom and I basically couldn't get it published honestly. I was told to pretend that I had looked for this effect on purpose, and provide some theory around why it could be true. I think that was the point where I realised academia wasn't for me.
Now, the same thing happens in the private sector, but ironically enough, it's much less common.
You were told by whom? An editor? A reviewer? To how many journals did you submit this research? Did you work your way from top tier journals downward? In general, almost all empirical research will find a home. I find your story less than believable without some additional context, disgruntledphd2.
I was a PhD student, I was told this by both my supervisors and much of my department.
I attempted to submit it to about 3-4 journals and never even got it sent to review.
Ultimately I left academia both for reasons research and financial, so I stopped.
I'm a little confused as to why you are doubting an anonymous comment on the Internet deep in an old thread on a mostly meaningless message board, but I guess that's life.
I don’t think there’s anything wrong with wanting a particular outcome. It wasn’t wrong that people were excited about the prospect of a room temp superconductor a few months back because people knew that if it were true good things were possible. Insisting that you can’t be excited by one potential outcome from a study means that you’ll only study things that don’t have the potential to help
There is a useful distinction between "wanting" and "attachment", but one usually turns into the other. Your mention of room temp superconductors is ironic since they have all been precisely attachment-driven frauds that start with wanting.
These are the exceptions imo. Most of it is funding-driven (or sometimes wanting-status-driven) than pure wanting-driven. Most researchers do not even care that much about their actual field and would change the field to do what they really want to do, albeit funding keeps there where they are.
Unsurprising. Although this is the first time I can recall reading a psychologist accept culpability for the field’s bad “science” over the last twenty years. Have any of the “anthropology of science” researchers published an explanation of that yet?
Everyone says "unsurprising" after the replication fails. Idk, this was a really popular theory that lots of people believed. I doubt you were all so confident back in the heyday of this social science phenomenon.
If tomorrow, they say "growth mindset" is also a non-replicable phenomenon, will HN be full of smug people saying "I knew it all along, lol!"
Stereotype threat was a “popular theory” because it fit the social and political presuppositions that were fashionable at the time. It’s almost always wise to be skeptical of political fashions or new social science findings even on their own; new social science findings that match political fashions are doubly questionable.
It might have been surprising at the time, but it's certainly unsurprising in light of the replication crisis. We now know that most findings don't replicate. Anyone who continues to be surprised by the most likely outcome needs to update their priors.
https://en.wikipedia.org/wiki/Why_Most_Published_Research_Fi...
From your link:
>Biostatisticians Jager and Leek criticized the model as being based on justifiable but arbitrary assumptions rather than empirical data, and did an investigation of their own which calculated that the false positive rate in biomedical studies was estimated to be around 14%, not over 50% as Ioannidis asserted.
That’s just a heterogeneous population with selection bias. Everyone is always convinced because when they’re disproven they stay silent. These days prediction markets allow people to convert beliefs to money so it becomes not very convincing when everyone manages to be perfect at predicting things and is always complaining they aren’t making enough money while simultaneously giving up the chance at doubling their bet on a sure shot.
Combine that with people upgrading uncertainty to certainty post-hoc when debunking comes out and you have these entertaining things. Overall, I’m glad you called it out. Once I wished I had a profile for people’s past guesses to see how actually good they are and now I have Manifold, Kalshi, and Polymarket.
I'm not a research psychologist, but my understanding is that growth mindset is already non-replicating.
To me it seems like "growth mindset" is a buzzword that everyone defines differently.
The weak version is something like: "Even talented people need to work on improving themselves, if they want to be successful." Most people would agree with this.
The strong version is something like: "If people believe that there is such a thing as talent, they automatically stop working on improving themselves." This is used to justify things like: "never tell your kids that they are talented or intelligent, you must attribute all their success to their hard work". In my opinion, this is a false dilemma: there can be things like talent or intelligence, and hard work can be necessary to succeed among those who are comparably talented or intelligent.
FWIW, to me it is "unsurprising" as I can't remember the last time a major psych study actually did replicate, and it seems frankly like the entire field is in shambles. Does that mean that I knew before that this study in specific was false? No. But, that isn't what anyone means when they say "unsurprising"! If you get to the end of a movie and all of the major characters end up alive, and someone else seemed shocked by that fact, you might still explain to them that that's "unsurprising", even if you yourself got to feel the thrill of uncertainty -- or even were concerned for a bit -- while you were watching.
> If tomorrow, they say "growth mindset" is also a non-replicable phenomenon, will HN be full of smug people saying "I knew it all along, lol!"
I will be one of them! Many people I've met in the industry—disturbingly, especially among founders and VCs—are extremely susceptible to reductive theories about the human experience that have little relation to reality. Just in the last year we went through "founder mode" hell even though any person that this "founder mode" describes should probably be in prison for wildly abusive behavior.
Of course the flip-side of this is that there are concepts that are situationally useful even if they aren't broadly applicable. Both "stereotype threat" and "founder mode" may fall into this category. (I've never heard of "growth mindset" before and looking it up I'm not sure what the point of it is—I guess to contradict the idea that people and personalities are static?)
My understanding is that "growth mindset" represents a personal philosophy, not a testable hypothesis, so it would be irrelevant here.
It was entirely reasonable to be skeptical of stereotype thread when the concept was new, a priori, An "unsurprising" result is not necessarily one that someone confidently believed. If I flip a coin, I'm not "surprised" when the result is heads, nor when the result is tails.
I could be wrong but I believe the idea is that the growth mindset has measurable consequences. I.e., if you believe you can grow, you will learn/adapt more effectively to new situations and ultimately achieve more. It's an empirically verifiable claim that is either true or false, and comes with a pretty straightforward intervention (i.e., teaching people that they can grow) if true.
Of course people may also adopt it as a personal philosophy, but that's separate.
Even if you did suspect something was wrong you were careful because science said otherwise. Calling this wrong in public would feel like saying a big rock falls faster than a small rock - something else that seems obvious but science has disproved. And so the only one loudly saying this was bunk where the right wing crackpots (the reasonable people on the right side were much quieter about it because they don't like to argue with science even though it went against their bias) - the right wing crackpots were on this only because the left wing embraced it as a science that confirmed their bias. Unfortunately in this cast the conspiracy was true and so the crackpots won (even though they have no understanding of the real reasons it is false and it doesn't seem to have been a conspiracy).
Let this be a lesson: even if something seems like science and it confirms you bias - that doesn't mean it is true. You should look closer at things you embrace than things you reject lest you embrace a lie.
I'm a rather reasonable "on the right" person and arguing about science is literally my hobby, there are few funner things in the world imho.
> Even if you did suspect something was wrong you were careful because science said otherwise.… And so the only one loudly saying this was bunk where the right wing crackpots (the reasonable people on the right side were much quieter about it because they don't like to argue with science even though it went against their bias) - the right wing crackpots were on this only because the left wing embraced it as a science that confirmed their bias.
If “crackpots” turn out to be right when “reasonable people” and “science” were wrong—and this is far from the only instance of this happening—maybe we should reevaluate some things.
Just because a peer reviewed paper published in a prestigious journal says something doesn’t mean it’s true. Even a survey of multiple peer reviewed papers published over time isn’t necessarily determinative if there are common methodological issues or publishing biases. Yes, a lot of times actual crackpots will make stupid criticisms, but not every criticism is stupid even if it comes from outside the ivory tower.
In particular, whether or not a big rock falls faster than a small rock [1] is a fairly basic question in physics, which has one of the most certain answers. Virtually nothing in psychology, let alone human social psychology, is at that level of certainty, and any psychologist worth their salt will agree with that. Basically any finding in psychology should have a level of certainty somewhere between “yeah, that’s probably mostly true” and “hmm, interesting hypothesis that’s not entirely crazy, I wonder if it holds up”.
[1] Also, the literal question of whether a big rock or a small rock falls faster is trickier than you might assume if you only know the middle school version of the question. If we’re doing this on the moon, so as to dispose with the tricky aerodynamic questions, and we are answering the question from the frame of reference of an astronaut standing on the surface of the moon, the bigger rock actually does fall faster. Both rocks accelerate towards the center of the moon at the same rate, yes, but gravity works both ways. Both rocks pull on the moon but the bigger rock pulls harder, resulting in an extremely small torque on the lunar surface, drawing the side of the moon closer to it very slightly upwards. Which, from the surface frame of reference, is equivalent to the bigger rock falling faster.
The way you've put it, the "crackpot" and "reasonable" right wingers seem to have the same beliefs, but differ in their willingness to speak their mind or the ease with which they're bullied into silence.
Be careful to not mistake "theory doesn't replicate" with "theory is not true [in whole or in part]".
Would you expect the theory of non replicative theories to be replicative or not?
Expectations may be part of the problem.
I was a psych major in undergrad, and did an experiment as a riff on stereotype threat and got a small effect. I had the participants solve brain teaser puzzles and the only difference was introducing them as coming from 11th grade or graduate level math. Undergrads did worse when they thought it was graduate level.
I think that result would ring true for anyone who's worked in education, but it doesn't sound very similar to stereotype threat
>this study (still a preprint) was preregistered (meaning all methods and analyses were specified before the data were collected)
This is the way forward -- preregistered studies. That, together with a promise from the publisher to publish the result regardless of whether the effect is found to be significant.
When you think about it, the incentives for publishing in science have been wrong all along. The future will be different: It will be full of null results, of ideas people had that didn't pan out. But we'll be able to trust those results.
Also interesting: p28 (labeled p470) of https://journals.sagepub.com/doi/pdf/10.1177/251524591881022... ("Many Labs 2: Investigating Variation in Replicability Across Samples and Settings") shows the results of attempting to replicate 28 published psych results across many different samples.
Unfortunately, "Stereotype Threat" is not one of the effects they attempted to replicate.
> Let’s be honest: that last sentence was far too generous. Many of us engaged in practices that, in hindsight, were borderline dishonest. We abused experimenter degrees of freedom, engaged in questionable research practices, p-hacked, massaged our data—you pick the euphemism. In contrast, this new replication study followed the most up-to-date best practices in psychological science, eliminating room for flexibility in analysis or results interpretation.
Exploiting researcher degrees of freedom remains unfortunately extremely common. There needs to be some sort of statistical vanguard in the ivory towers enforcing real preregistration and good analysis practices. Strict epistemic discipline is necessary to do real science.
The real problem is we have a lot of data. So it is easy to shove it all into a computer and then see what comes out. This is not a bad thing, but it isn't alone a result to publish it is instead justification to do a real study on whatever thing statistics turns up as a significant result to make sure it really is when you properly isolate for that data.
There’s also the issue that in psych, everything is measured through indirect proxies and most of the predictions will be directional. That gives experimenters a LOT of wiggle room.
In physical science you can often directly measure a phenomenon and have a theory that makes very specific predictions
early in my psych career, i was actively encouraged to do things that would be considered p-hacking or abusing degrees of freedom. it was seen as a way to not let any data go to waste and potentially 'show off' your cleverness/mathematical chops in the methods section (anybody can just run an ANOVA like they teach in the text book, but look what I can do).
I have seen stereotype threat mentioned in educational contexts - justifying the idea that, for example, it’s important not to make (deliberately or inadvertently) things like coding or engineering seem stereotypically masculine. Usually the recommendations are to ensure that if pictures of participants in classes or extracurricular programs are shown in advertising, diverse groups of people are pictured - or that if connections to popular culture are made in educational materials they’re diverse - for example, don’t make all your example coding projects about Star Wars or football.
Anecdotally, I’ve seen with my own eyes, for example, girls getting really into coding only after seeing it demonstrated by enthusiastic women that they can see as role models in ways they would not see men.
I guess this is a far broader thing than stereotype threat - but I’m sure this larger thing is real. I fear that people who themselves have stereotypes in mind about who ‘should’ be into certain topics will use the demise or deemphasis of stereotype threat to justify not making attempts to attract or be friendly to kids who really could flourish in non-stereotypical fields - to their and society’s detriment.
Yep, I agree that this kind of descriptive representation, role model thing is very real and can have significant effects. It's the best kind of evidence that someone of the same demographic likely can/can't achieve similar things.
one of my stances has been that psychology comes up with wacky unreplicable findings because there's no central organizing theories for how 'the mind' works and everything is just very black-box.
there are so many studies showing "X manipulation affects Y outcome", but there's not even a hint of an attempt to explain the mechanisms in a meaningful way (cognitive experiments are usually better, but often still guilty of this).
I actually disagree entirely. The amount of forcing theories that are well ahead of the facts that occurs in academic psychology is off the charts.
We could do with a shut up and experiment movement, to be honest.
In order to build useful theories, we need lots more data, and forcing theories onto data actively holds us back from running the wide variety of experiments necessary for a real theory to emerge.
I agree that there isn’t as much raw experimentation and just observing/documenting psychological phenomena that there should be. I’ve ranted about that here in the past.
I guess my peeve is that nobody really collaborates on theories. Every PI is off working on their own set of pet theories (because that’s how you establish a career), but there’s really no centralizing force to get people to collaborate and work on a shared theory. Physics has the standard model that everyone can use as a common reference point and it’s helpful.
This is partly the "psychology is the study of college undergrads" problem. The original study was on Stanford students, all of whom have already passed through a very selective filter. That's not a group to extrapolate to the general population. Too many psych studies are done on this convenient population.
It's great seeing the original author admit the problem.
No it's not. This is a completely different thing.
I also have a high degree of skepticism about most psychology research. I find it frustrating that the author of this piece raises the issue of the imbalance of gender in STEM jobs as a reason to doubt this particular study. There is nothing about this failure-to-replicate that should allow us to conclude that innate differences in math ability underly that imbalance.
I’d love others to read the replication report and explain why I might be wrong?
The author doesn't claim that the imbalance is a reason to doubt the study. His claim is that the imbalance is a reason to doubt the claim that an underlying shift in cultural norms explains why the results might have been valid in 2005 but failed to replicate in 2024.
The author rightly observes that despite the undeniable shift, women are still significantly underrepresented in STEM and therefore that cannot explain the lack of replication. There are still many other reasons besides innate differences that might explain it.
I mean come on - if there is a pervasive cultural insinuation of gender differences in ability that leads to women experiencing more test anxiety (I’m not saying there is, but conditioning on that hypothesis) why would we expect a 20 minute PowerPoint to counteract 20 years of people’s life experience!?!?
Is there any evidence you can conceive of that would convince you that there is some innate group difference in some aptitude that’s valued by society?
I'm curious what the impact for most people's mindsets will be here? I'm imagining that it relates to what many people consider "talent" with kids? I know that has been an odd trend in some states trying to get rid of "gifted" programs and such. I have largely remained hopeful that that was not nearly as prevalent as online represents.
When I first heard about this stuff 20 years ago, it was being presented as "The differences between groups A and B go away when stereotype threat is removed" which is not what the original paper says. The original paper claims that whatever difference there is between groups A and B, it will be larger when measured under the condition of stereotype threat which I guess is plausible but much less interesting.
Part of the context of that time was that _The Bell Curve_ had been published fairly recently and there was great desire to disprove it and anyone doing that could count on lots of attention and speaking fees. So the grift was to present stereotype threat as this grand solution that could resolve all racial differences.
Link for the claim that the results don't reproduce, without Facebook tracking: https://osf.io/preprints/psyarxiv/qctkp
> When Black students at Stanford University were told that a test was diagnostic of intellectual ability, they performed worse than their white counterparts. However, when this stereotype threat was ostensibly removed—by simply framing the test as a measure of problem-solving rather than intelligence—the performance gap Black and white students nearly vanished.
Just reading this description motivates me to reject the study out of hand. It's not plausible that university-level students responded meaningfully differently to being told "this is a test of problem-solving skill" versus "this is a test of intelligence" because it is commonly understood that problem-solving skill is a major component of intelligence.
>it also became the darling of the political left who now had an answer to prevailing views of group differences held by the political right. This is partly because shortly before stereotype threat took its turn in the spotlight, Charles Murray and Richard Herrnstein published The Bell Curve... the octogenarian Murray is still considered a pariah, shouted down and deplatformed from talks he tries to deliver at respectable colleges to this day.
The characterization of Murray's views in the last several years has been grossly uncharitable and seems entirely disconnected from his actual arguments. It's strange that the book is 30 years old, but has seemed politically relevant for much less time than that.
Many years ago, long before "The Bell Curve", I spent way, way too much time reading Murray's book "Losing Ground" (1984) and going to the library to read the papers he referenced and reading the papers those papers referenced and then the papers those papers referenced. My conclusion was that his book is junk. My second conclusion was that citations are like a game of telephone and every time someone cites something, it changes slightly, so don't trust anything until you get to a primary source, and maybe don't even trust that. (This second conclusion applies in general, not just to Murray.) My third conclusion was that arguing online is a waste of time and approximately nobody cares about actual facts.
> My second conclusion was that citations are like a game of telephone and every time someone cites something, it changes slightly, so don't trust anything until you get to a primary source, and maybe don't even trust that. (This second conclusion applies in general, not just to Murray.)
This part is definitely true. It has also been my experience -- but more so within academic areas with sloppy "researchers" and political problems. Linguistics is pretty good, gender "research" is awful. Intelligence research is pretty damn good, social "science" and psychology (outside of psychometrics) is awful. Economics is awful, apart from the basic ideas of competition, capitalism, and low taxes. Keynesianism is largely a fraud.
> My conclusion was that his book is junk.
This is unlikely to be true.
Your interlocutor might not care about facts but the audience might. I’ve learned interesting new things from reading the back-and-forth of others going at it online.
Eh, I mean the stereotype being "activated" is that Black people are less intelligent / have lower IQs, not that they lack problem-solving skills.
Even the replication attempt had two scenarios:
1 where women were told the test was to establish performance levels on the test between men and women
1 where they were told the test was a test of problem-solving skill (or primed to disregard negative stereotypes before the test).
So even the replication has the incorrect framing you worried about. I tend to believe the problem wasn't this, but the way the field was lax about sampling, methodology, etc. After all, there were many stereotype threat studies, not just this one, boasting similar results. And they didn't all use that framing.
Not to necessarily defend the study here, but I don't think that's a good reason to dismiss the study out of hand. Words themselves, the actual sounds independent of their technical meaning, still have very strong, deep-rooted associations that are hard to fight even if you've been educated. The subtext and day-to-day use of "intelligence" and "problem solving skill" are different, and color the way you think of them even if you know they're technically synonymous (which, do note, they're not quite). That association operates at the same deep level at which the supposed effect works.
The characterization of Murray's views seem extremely accurate and the fact that his book is still brought up despite being disproved as racist trash decades ago is ridiculous
[flagged]
>How does it help anyone to pretend that group differences don’t exist?
It helps people get along.
That book was not very good science or science interpretation and didn’t do a great job of proving that meaningful race-iq differences exist. I also don’t know where the disaster is, could you elucidate some of the cataclysmic results of DEI?
[flagged]
I’ve seen and read extensive reviews, but yeah I have not read it. I agree that’s a blind spot, so I’ll go ahead and read the book then get back to you after. Give me like a week.
[flagged]
It is wildly untrue that these bullets are all broadly accepted in the actual literature or among modern practitioners. This is more like a list of things you happen to believe. Which is fine, but be clearer about it.
Why do you think it’s “fine” that the parent commentator thinks “ yes, some population groups really are more intelligent than others”? What do you think he means by the “truly taboo comparisons” and why do you think he’s too chickenshit to say what he means? Why is that “fine” instead of “grounds for immediate ostracism”?
[flagged]
Are you ascribing that to innate genetic IQ inferiority or to social and environmental causes?
Eg: Resource starved German schoolchildren in WWII scored almost two std deviations lower than the same demographic from the same classes and schools five years earlier.
There's pretty decent odds this person is coming to this conversation with "IQ and the Wealth Of Nations"-level data. Lots of little tells.
But apparently being a barely-concealed white supremacist is “fine”. Reasonable people can agree to disagree, in your view.
little tells?
I figured the direct quotes from The Bell End were a bit of a give away.
I mostly just want to preempt the inevitable citation to IQ & The Wealth Of Nations; it's a little squicky for me to predict it's coming (but it was definitely coming).
[flagged]
You aren't helping.
Sure, this John C Calhoun level racist will probably just change his mind about Blacks being untermenschen if you’re just polite enough to him.
I understand that the stated HN policy is that polite, fig-leafed White supremacy and 1920’s style race science is fine and name-calling is not, but get real. Your attitude is not just not helpful, it actively helps normalize scumbags like this piece of shit.
You're coming across as someone terrified that the literature is going to back this person up. It does not. Can you just go to Twitter and dunk on HN instead? All you're doing here is this person's work for them.
You're writing like the objective of the thread is to communicate with this particular person. It is not. The point of a thread is every other lurker reading it.
It’s extremely clear you don’t have any personal real-world experience with this, but let me assure you that no one who thinks Blacks are of sub-human intelligence gets there by sitting down and digging into research and weighing the various arguments for and against. They start with the prejudice and they latch on to whatever allows them to justify it. There is no point in making arguments at all, either to this particular piece of shit or to whatever pieces of shit will permit a nasty internet comment to justify prejudices they already hold. The only thing that works is to make it clear to these people that no one is fooled about what their views really are, that those views are repulsive to decent ordinary people and that holding them is grounds for ostracism and abuse.
Politely tolerating and debating them doesn’t sway any bystanders away into thinking “hey, maybe Blacks are fully equal to other humans after all”, it only gives social permission to fellow travelers who would otherwise feel pressured to conceal or temper their racism, and normalizes the idea that the essential humanity and equality of Blacks and other minorities is a legitimate topic of debate.
Once again: I do not care what the person you are threatening thinks. I do not think I am going to persuade him. What I care about is that he's wrong, and that looking super dumb on a thread about science is a lot more interesting and persuasive than a throwaway account pitifully threatening to beat him him.
You write like you don't know why he's wrong. Let me reassure you: he is. Look it up!
What you think you’re doing is showing off how smart and clever you are. What you’re actually doing is normalizing the idea that the essential humanity and equality of Black people is a legitimate topic for debate. Stop doing this.
I think you're genuinely afraid you could lose an argument with that person, and that's sad.
> I think you're genuinely afraid you could lose an argument with that person, and that's sad.
No, you stupid motherfucker, I think it’s gross and debasing that anyone would think the argument is worth having in the first place, let alone validating, and if you can’t understand why, I think maybe you should consider stepping outside of your own smug interior life for a moment.
[flagged]
Right and many people believe in Phrenology/Genetic racial IQ differences/Human Biodiversity or whatever euphemism they're using these days.
That doesn't make them any more true or any less harmful then when they were measuring skulls
No, I don't think psychometrics and heritability research is like stereotype threat or "sky fathers", whatever that is. But I think we're clearer at this point where that list came from, and probably don't need to take this further.
Will you at least retract "wildly untrue"?
No? You provided a list of things you personally believe to be supported by the balance of evidence (in several different fields of study), and many of them --- maybe more than half --- are things that are unsettled, or even unlikely to be true. Why would I retract anything?
Half? Seriously? I could understand believing a handful of them are "unsettled, or even unlikely to be true" given the current debate climate.
Honest question, no attempt at rhetorical jiu-jitsu or anything -- why do you believe "maybe more than half"? You must base that estimate on something.
Yes: I read papers and keep up with stuff. It's cute that you think we're going to dig into this rigorously after "Communism, sky fathers, and crystals", but we obviously will not. Again, no, I will not be "retracting" anything.
>lots of places have high-IQ subpopulations who look a lot like their background populations
Can you rephrase this?
[flagged]
Differences in IQ currently exist, they're not based on racial genetics.
The Bell Curve isn't about organizing society fairly it's about justifying what is clearly an unfair and unjust society as natural and impossible to improve and therefore good
https://www.vox.com/policy-and-politics/2018/3/27/15695060/s...
https://www.vox.com/2018/4/10/17182692/bell-curve-charles-mu...
Executive Summary of Article: "new data now reveal what many of us suspected for at least ten years: stereotype threat does not replicate, and it does not undermine academic performance in the ways we thought."
The Stereotype Threat: "individuals who are part of a negatively stereotyped group can, in certain situations, experience anxiety about confirming those stereotypes, leading paradoxically to underperformance, thus confirming the disparaging stereotype." for example, if you remind a woman of the "Women are bad at math" stereotype, they will perform worse on a math test than if they are not reminded of that stereotype.
[dead]
[flagged]
This has been an ongoing situation since around 2010. The field has been undergoing a huge shift in terms of re-evaluating research practices and past findings. It genuinely has nothing to do with national politics.
Also, the author is Canadian. So there's that, too.
>This has been an ongoing situation since around 2010.
Exactly. Yet the time to declare your change of heart is somehow now :)
>Also, the author is Canadian. So there's that, too.
Search for his grants brings grants from US, he studied and did PhD in US and still affiliated with NYU, and the major and most valuable part of his network of academia connections is US, his private foundation in USA, etc. If he isn't in-line with the prevalent winds in US, he will be reduced to nothing, just a professor in some Uni, not a renown researcher with his own Wikipedia page, grants and conferences in US, etc.
FFS, This is in response to new data from a Registered Replication Report that is still in preprint.
Yes, man, for 20+ years you are peddling some politically favored theory, supposedly doing experimental studies (by 2013 already getting at least $3.75M dollars in grants - https://www.apa.org/monitor/2013/09/apf-grant ), and you have no idea that your theory is total BS until suddenly somebody else failed to reproduce yours and/or similar studies, and that is exactly when political climate is changing 180. It isn't a science, it is an ideology department serving whatever political regime is in power at any given moment.
Once again, that is incorrect. It's all spelled out if you read what he wrote.
[flagged]
Inherited wealth was actually pretty rate in the history of this country, and is not as common now as many people think. Generational wealth until recently did not exist for most people.
Large portions of existing wealth are inherited, most estimates putting it somewhere in the 30-60% range of all private wealth in the US
e: not sure why this gets a downvote https://www.washingtonpost.com/us-policy/2019/02/06/people-l...
I don't think that's what people have been taking away from the idea of stereotype threat at all. I think it's been much more likely for people to think, "Oh, instead of unchangeable genetic traits, the difference in outcomes might someday be reduced through deliberate social change.
Putting on my contrarian hat: the studies he mentions seem to be concerned with whether testing conditions affect stereotype threat effects. Logically, they can't prove or deny the phenomenon's existence, only whether testing conditions result in changes to test outcomes that could track with interventions to reduce stereotype threat. Much of what we know about how behavioral effects of identity comes with the understanding that the latter is something people carry with them, regardless of local or recent events. If there's a problem with stereotype threat as a concept, it's that it's positioned as a superficial effector that can be manipulated easily, rather than the surface level manifestation of complex interactions between self-identity, personal values, and cultural expectations. Based on the author's disclosure about his PhD thesis, he seems to be someone who capitalized on the former characterization, so of course he throws the baby out with the bathwater when it no longer works for those purposes. We might be looking at an ass-covering write-up.
>Let’s play “Find the Lebowski quotes game” again!
So, yeah, I find this a deeply unserious blog post.
He must throw the baby out with the bathwater - so far there is no evidence for anything here. The only alternative is to rerun the study, doing it correctly this time and then do a correct analysis. Until then you need to say "I don't know" when asked about this as that is all science allows. Maybe the rest of the baby they are throwing out will replicate and thus is correct - but nobody knows that and so we cannot say anything with confidence right now.
I'm assuming you are not aware of studies not mentioned here that replicate - I'm not in this field and so I would not know where to look. I'm guessing that you also are not in this field and are looking for some way to allow your bias to become true despite these issues - but of course I might be wrong.
Let me analogize. This is a blog post that claims that racism isn't real. As proof, the author explains that none of the studies that show a reduction in racism when you remind people that Michael Jordan and Beyonce exist can be replicated. Well, okay, sure. And the guy who earned a doctorate with the thesis, "Does Playing Jackson 5 Make People Less Racist?" might want to turn this into, "Racism does not exist," as a way to distance himself from his own terrible scholarship. (Also, did you find his Marvel Cinematic Universe Easter Eggs?)
So, thus far, there is actually not enough evidence to throw the, "Racism exists," baby out with the, "Do these interventions affect racist belief?" bathwater. And since we're second-guessing biases, it's super weird that you're always in comment sections of (politically-charged) articles concerning fields you're not in.
> This is a blog post that claims that racism isn't real.
No, it does not make that claim.
I think that contrarian hat may include a no-true-scotsman badge.
If stereotype threat is real, we should be able to have a replicatable study result that confirms it, right? We're not just limited to logical inference, I hope.
If the studies are concerned with determining that stereotype threat exists, sure.