Physics Nobel winner Peter Higgs (of the boson) said the same ten years ago: “Today I wouldn't get an academic job. It's as simple as that. I don't think I would be regarded as productive enough.”
https://www.theguardian.com/science/2013/dec/06/peter-higgs-...
Reminder that:
1) Ken Iverson who invented the APL programming language and went on to win the Turing Award in 1979, had already published 'that one little book' that was considered insufficient for tenure, but which formed the basis for the award.
2) tubes remained the main focus of MIT faculty for quite some years after the transistor was invented. It was Robert Noyce and the people he worked with at Grinnell College who knew more about transistors than MIT : https://web.stanford.edu/class/e145/2007_fall/materials/noyc...
Reading the comments here, it seems that even very prestigious universities are full of academic pettiness and dysfunction which deny all of us the output of brilliant people like Katalin KarikĂł.
It leaves me wondering: why do we not create any new universities? Why doesn't a Carnegie of our age create a new university? Brin University? Zuck University? This seems like a no brainer.
I think it might seem difficult to attract new talent to an "unestablished" university. But what if you make a simple promise: we will never, ever get in your way, the way that universities do today. We will never pressure you to publish subpar results. We will never nit-pick your purchase of a laptop. Have vision! Pursue things that are promising to you! We trust you, smart person, and we will give you autonomy to do what you think is promising. Based on what is discussed here, it seems like that would be extraordinarily compelling to the most optimistic, least cynical, and probably at least a handful of the most brilliant researchers out there. If the winning move is not to play the game, don't play.
I don't know. It just seems like there is a narrow-mindedness at play. A sense that "why try to fix this -- we'll never beat UPenn. Maybe not, but isn't it worth a try, based on how dysfunctional academia is? All it takes is the will.
Her book Breaking Through [1] also goes into more detail about this. Basically academia is now ruled by the same rotten economic lenses as the rest of the economy. Everything is about profits, labs are evaluated in "grant $/sqft." and people are evaluated on a "resume" or dumb metrics like papers published. It's really hopeless how this economic virus infects every little corner of our world and turns it to shit.
This isn't just one story, there are countless other researchers and even life-saving drugs that are not developed purely because of this mindset. For a brief moment in time during the COVID pandemic we saw that it is possible to have a better system but it's been forgotten just as quickly.
Some of this is universal, but much of her story is particular to how US med schools operate: their research faculty tend to be largely soft money in nature, so grant money is even more necessary than in other 'hard money' jobs in non-med school fields. Such a system is destined to fail when geniuses like Kariko pursue risky new territory for which large grants are hard to secure.
The really distasteful thing here is Penn as an institution. They have reaped the benefits of her work in terms of mRNA patent royalties (a very large number I believe), and of course reputationally. Yet, they treated her truly terribly and have never - and it seems like will never - acknowledge it. For example, Sean Grady, mentioned here as the one that essentially cleared out her lab in 2013 without telling her is the chair of neurosurgery at Penn Medicine. Will he apologize? I doubt it.
This article makes me wonder how many groundbreaking discoveries are buried under academia's bureaucracy and ego battles.
Being currently in the middle of belatedly reading The Black Swan, I can't help but see this as a classic case. Penn has a formula that's supposed to predict "success," and it's a linear formula: more papers & funding leads linearly to more success. y = mx + b which is totally how the world works, right? Not if you've read Nassim Taleb or even Paul Graham's essays about mining unfashionable/disreputable/heretical ground for ideas nobody else has thought of or is willing to consider. Just like startups, somebody is going to discover something huge in there. Even if you were willing to say a university isn't a place of ideas for their own sake, and is instead nothing more than a venture capital firm like their bureaucrats seem to be asserting - in short even with the profit/greed motive intact, it still seems like a dumb strategy to model the world as linear and boring.
Academia does not value quality, but quantity. It selects for scientists who are the best at marketing and networking, not necessarily doing quality science, though they can also be.
I have no idea how to fix this, but competition needs to be reduced, probably by more guaranteed funding for positions, not just projects, as grants are. This latest military aid package is 2x the entire NIH budget, so surely there is more money for science out there.
In many professions including the business of startups and academia you need to be at least as good at selling something as you are at developing/discovering it.
Just a reminder Einstien who was the most revolutionary scientist since Newton was unable to secure a teaching position prior to publishing 3 nobel prize worthy papers in 1 year. Why do we think academia has suddenly magically changed?
Well...yes. Treating the actual high-value workers like sh*t has been American Academia's SOP for how many years now?
> “I do hope that it causes Penn and a bunch of other institutions that fund science this way to reflect a little bit on what the chances are that some scientists who do not get funding, and wind up leaving, end up being like Katalin Karikó,” Scales said.
A brilliant woman scientist researching an uncool topic hits the trifecta of resistance to her work. It's wonderful to see her persistence vindicated but it sounds like time for a revolution in how university research is managed. The closing quote of the article is very disappointing.
> Unless something changes, this isn’t going to go well," Grady told Karikó, according to her memoir.
While unpleasant, this is a conversation that is sometimes necessary to have as someone in a position of power communicating to a subordinate.
> In 2013, KarikĂł said she returned to her lab after spending time away to find all of her belongings having been packed, moved, and misplaced at Grady's direction.
But this is just petty and cruel.
Tangentially related, but her daughter is Susan Francia, who is an Olympic gold medalist rower. It's wild to me when you see family members at the top of their completely different fields.
My favorite little bit about all this:
https://www.glamour.com/story/katalin-kariko-biontech-women-...
> In 2013—after enduring multiple professional setbacks, one denied grant after another, and a demotion at the institution to which she’d been devoted for decades—Katalin Karikó, Ph.D., walked out of her lab at the University of Pennsylvania’s School of Medicine for the last time.
> That morning at the lab, Karikó’s old boss had come to see her off. She did not tell him what a terrible mistake he was making in letting her leave. She didn’t gloat about her future at BioNTech, a pharmaceuticals firm that millions now associate with lifesaving vaccines but was then a relative upstart in the field. Instead the woman who had bounced from department to department, with no tenure prospects and never earning over $60,000 a year, said with total confidence: “In the future, this lab will be a museum. Don’t touch it.”
As someone who is not in academia, I'm curious how dysfunctional the incentive structures of these institutions really are? Is it more the case that aggrieved professors doing actually good research is just a rare situation bound to happen every once in a while?
I enjoyed this 2-Aug-2023 podcast interview covering some of this stuff: https://josephnoelwalker.com/147-katalin-kariko/
Everyone picks on Penn, but really the whole academic community made this mistake. Her research was public; any other university could have offered her a job as well. Nobody realized how important her research was at the time.
Penn sold the patent to an outside company? Maybe it should be standard in such cases (if it isn't already) to give the researchers the right of first refusal.
I wish we had a way to see whether someone was right or wrong in the past so we can judge their decision making abilities. This would help us pick good leaders.
If only there were a way to document disagreements publicly so they could be reviewed at a later date.
I had hopes that Internet discussion forums would be that, but the nukers destroyed that along with most training materials for LLMs.
In 2013, KarikĂł said she returned to her lab after spending time away to find all of her belongings having been packed, moved, and misplaced at Grady's direction.
As a woman who wonders how we solve this, I feel like the piece doesn't cast light on actionable means to make real progress. It implicitly seems to assume a thing the world calls sexism but offers no real solutions.
Presumably, Grady was a big factor here. Why was he such a problem? There's no real explanation for his behavior and we are unlikely to ever learn anything meaningful about what drove his decisions.
It airs dirty laundry, which is perhaps an essential first step in being able to analyze the problem space: First, we have to overcome the taboo against speaking of such things.
But then the initial attempts to speak of it tend to fall within some Overton Window of acceptable assumptions and this seems to do little or nothing to give women a viable path forward.
The research and discoveries that are most deserving of a Nobel Prize are precisely the sort that are unexpected and unpredicted in advance. All this "Monday morning quarterbacking" by everyone who now suggest that this discovery should have been obvious 20+ years ago or that the talent of those who made the discovery should have been obvious is rather silly.
Arguably the story of how this researcher was treated and what she still managed to accomplish can serve as inspiration and motivation to persevere to future generations of folks with unconventional ideas or ideas that are disparaged by the 'experts'. Yes, it can also serve as motivation to research institutions to take risks and go out on limbs every now and then as there can be some wheat hidden within the chaff.
My experience in university convinced me that: modern universities are not institutions of learning or discovery, they are businesses and are only concerned about the bottom line. As always, enshittification follows (and arguably happened a long time ago already). That includes amazing short-term decision making at the cost of long-term sustainability.
UPenn is in the news again. Something tells me the fish stinks from the head.
The thing is, any human group, company, academia, etc...are influenced by politics, and those who do it well get the attention and the resources. It doesn't matter if they are technically brilliant or not.
There is no place where you don't need good communication and selling skills. That's a fact of life and it seem impossible to remove this from any of these institutions.
Kariko seems to be that very hardworking intelligent person that really needs an eloquent and self-marketer sidekick to thrive. She is a Steve Wozniak in need for a Steve Jobs.
It'd be worth taking publicly available citation network data and developing a metric based on long term (say 25y) significance vs short term. Anyone who understands how to 'play the game' can make an impact over a 5-10y period, but people who actually change the game will stand out, as would their collaborators to a lesser extent.
I'm glad to see this article calling out the people who threw stumbling blocks in her way. They deserve any career damage they get.
So what are the repercussions for the admins who misjudged or mistreated her? If there aren't sufficient changes then maybe groups should move their funding out of Penn.
Some of this is universal, but much of her story is particular to how US med schools operate: their research faculty tend to be largely soft money in nature, so grant money is even more necessary than in other 'hard money' jobs in non-med school fields. Such a system is destined to fail when geniuses like Kariko pursue risky new territory for which large grants are hard to secure.
I knew there was trouble as soon as I saw "Nobel Prize [winner]" and "Adjunct professor" in the same sentence. What's it take to get tenure track these days? But she mentions the rate of publications. Kps noted that Peter Higgs said something similar as well. There's many others too! Turing prize winner Hinton had this to say about ML and I couldn't agree more
> One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas.
> Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad.
Or from Bengio
> In the rush preceding a conference deadline, many papers are produced, but there is not enough time to check things properly and the race to put out more papers (especially as first or equal-first author) is humanly crushing. On the other hand, I am convinced that some of the most important advances have come through a slower process, with the time to think deeply, to step back, and to verify things carefully. Pressure has a negative effect on the quality of the science we generate. I would like us to think about Slow Science (check their manifesto!).
> Students sometimes come to me two months before a deadline asking if I have ideas of something which could be achieved in two months.
I'm sure you can find one from LeCun too (drop it if you have it) and we have the 3 godfathers of ML. But as someone finishing my PhD, I'm utterly convinced that the whole process is psychotic and anti-scientific. I have written many rants on HN about this so what's another? Here's how I see it, and what I've been coining as Goodhart's Hell because the idea is more abstract that ML publishing or even academic publishing. There's just a huge fucking irony that this happens in ML.
It is Goodhart's Hell because everything in our world has become about easy to use metrics and bending over backwards to meet those metrics. There is not just a lack of concern about if the metric aligns with our intended goals, but an active readiness to brush off any concerns. We as a modern world just fucking embraced metric hacking as the actual goal. In ML we see this, as Hinton mentions, with benchmarkism with just trying to get top scores. But you need several (fwiw, I've held a top spot for over a year now on a popular generative dataset but the work remains unpublished because I don't have enough compute to tune other datasets. Reviewers just ask for more but not justify the ask by how another dataset says more). This is an insane world, especially as we've been degrading our statistical principles. The last 5+ years no one uses a validation set for classification but rather tunes their fucking hyperparameters on test set results. Generative models frequently measure metrics against the train set and don't have a test set! A true, honest to god, hold out set essentially doesn't exist (we might call it "zero shot", which is inaccurate, or "OOD"...). ML work has simply become a matter of compute. Like Higgs said, you need to publish fast, but these days top companies are asking for 5+ papers at a top conference for a newly minted PhD. I'm sorry, good work takes time. All this on top of several consistency experiments that demonstrate that reviewers are simply reject first ask questions later. Which why shouldn't they be? No one checks a reject and doing so increases the odds your work gets in since it's a zero sum game.
And in honesty, I don't see how conferences and journals are anything but fraud. Not in the sense that works in there are untrue (though a lot are and a lot more are junk. Regardless of field), but in the economic operation. The government and universities (double dipping on that gov money) pay for these to exist. Universities pay researchers to produce work. Researchers send to venues (journals/conferences). Researchers review other works submitted to the venue for no pay (so Uni pays). 80% of work gets rejected, and goes through the process again. And after all that, the only meaningful thing accomplished is that the university has a signal that the work that their researchers did is "good." Because the venue gets copyright ownership over the paper, which the university must now pay for to access (the "official" version, "preprints" are free). I'm sorry, but citation count is a bad metric but far more meaningful than venue publication and it's fucking free. Why don't we just fucking publish to OpenReview? The point of publishing is to communicate our work, nothing more nothing less. OR gives you hosting like arxiv but also comments and threads (and links to github). Do we need anything else? I mean no review can actually determine if a paper is valid or good work. But we forget that the world isn't binary, it's tertiary: True, False, Indeterminate (thanks Godel, Turing, and Young). In reviewing we do not have access to the "True" side, just as we don't have access to that in science in general. We do not know where the "True" direction points, but we know how to move away from the "False" and "Indeterminate" directions. That's why there's that famous substack named that way or Isaac Asimov's famous Relativity of Wrong paper[2]. We're not a religion here...
There is at least a few ways I know how to fight back. 1) Actually fucking review a work and do your god damn job. Your job isn't to be a filter, it is to earnestly read the work and to work with the authors to make it the best work it can be. Remember you're on the same side. 2) Simply don't review if you can't do #1. You're almost never required to and academic service isn't worth much, so why do it? 3) Flip the system on its head. Instead of concentrating on reasons to reject a paper (fucking easy shit right there), focus on reasons to accept a paper. Simply ask yourself "is there something __someone__ in the community would find useful here?" If yes, accept. Novelty doesn't exist in a world where we have 20k+ papers a year and produce works every few months. It's okay to move fast, but it's less novel and impactful, it's just closer to open science. Stop concentrating on benchmarks since if it's useful someone is going to tune the shit out of it anyways, benchmarks don't mean shit. These days benchmarks are better at showing overfitting than good results anyways (yes, your test loss can continue to decrease while you overfit).
[0] https://www.wired.com/story/googles-ai-guru-computers-think-...
[1] https://yoshuabengio.org/2020/02/26/time-to-rethink-the-publ...
[2] https://hermiene.net/essays-trans/relativity_of_wrong.html
UPenn made the right decision for itself. If Kariko was the real deal, UPenn would still get credit for Nobel. If Kariko was a dud, they let go of a liability early.
There is absolutely no incentive for UPenn to keep a professor who isn’t bringing in money.
The real problem here is that funding for sciences is way too low.
Grant agencies cannot always get it right. The more funding they have, the more they can give to researchers that they aren't 100% sure about.
The US academic system is focused on money, and operates like a for-profit business.
Does EU produce better science, I wonder?
I really love how bad Penn looks on this now. It's hilarious.
This is a good outcome. Ideally, star researchers are expunged from academia where good ideas go to die and they're moved to industry where success depends on their work working.
If there wasn’t COVID pandemic, and mRNA vaccines did not become widely used for another decade, ms Kariko would never get the deserved recognition.
Academia seems like a wonderful place into which we, as a society, should send unlimited borrowed money.
A pretty obvious consequence of the American war on funding scientific research.
What about the psychology angle?
I get the feeling Katalin KarikĂł got a lot of that flak because she made some narcissists look bad (directly or indirectly, by comparison).
- Your views on this?
Cinderella)
Reminder that Penn is the 2nd worst school for free speech: https://rankings.thefire.org/rank/school/university-of-penns...
Unpopular opinion, but this was a reasonable outcome from the university.
If a researcher makes a great discovery, but can’t get funding to do anything with it, You don’t keep them around not making progress.
They got pushed out, found funding, and finally furthered the technology.
It is unclear if more scientific progress would have been made if they were kept at penn without funding.
Well the fact is that she remained at Penn for 30+ years, so clearly it couldn't be that horrible. And there she made some groundbreaking discoveries that contributed to the COVID vaccine and she got a Nobel Prize, so she probably did get some people behind her.
University politics are terrible, but in this very case, whatever happened, it turned out pretty good for both her and the University.
I wish every disagreement was logged in a system so that, decades later, we could know who was right and who was wrong.
It would tell us who to listen to and who yo shun.
I had hopes that Internet forums would be that record, but the nukers destroyed that.
My wife was hired last year as a full time professor and leads her own lab. By far the largest pressure on new faculty is the ability to get money into her lab, and by extension the university since they take a very hefty cut (50-100%! btw this doubles the "cost" of the grant, it doesn't lessen the amount the professor gets). Getting approved for the money via the grant process means having published "interesting" research along avenues of inquiry that other folks find worth pursuing. Often times this means building on existing lines of research over pursuing new paths.
The hiring process is setup basically to filter for folks who they think are the most likely to publish lots of papers, collaborate to push existing lines of inquiry, write lots of hopefully approved grants, and grow a lab into what is effectively a "successful small business". Quality is an after thought taken care of by what passes for peer review.
The incentives for everyone involved is just a complete and total mess. I'm reading tea leaves here, but my guess as to why she was never hired is that she was deemed "unable to get grants". Had she been, then she would have found herself hired immediately somewhere because universities are incentivized to play a numbers game and get as many folks in writing grants as possible.