FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
64% Positive
Analyzed from 1769 words in the discussion.
Trending Topics
#things#llms#more#doesn#those#based#human#already#before#same
Discussion (40 Comments)Read Original on HackerNews
To me this effect doesn’t seem to reflect on AI very much, it seems to reflect on humans. Like maybe this is more evidence of the Babble Hypothesis and the incentives in research than AI, no?
https://en.wikipedia.org/wiki/Babble_hypothesis
It would have been useful to check whether less original work was already getting more citations before AI adoption. That could reflect broader trends and network effects: heavily cited research areas attract more authors optimizing for citations, so high-productivity researchers end up clustering on the same topics.
The aim of many scientists is discovery, publishing is a side chore to survive and to get funding. Automate paperwork and you get more time for discovering.
We tend to think that obvious potential is the same as realized potential, for new technology.
For any specific context, there are generally innumerable smaller adaptations and capability thresholds that have to be crossed. And the price for that journey is often temporary loss off overt productivity.
This seems like some variant of "why don't you short the market and become rich". It doesn't work like that.
many children have an unlimited capacity to ask "why?". many adults are the same
if the abilities of AI are finite, then we will continue to have burning curiosity, questions to ask, and discoveries to make
The first type happens when you are enthusiastically engaged in a topic, which LLMs will likely enhance.
The second type happens as a by-product of solving a, perhaps deeply uncomfortably, difficult problem. This is what people are talking about when they say LLMs will hamper human cognition. Instead of sitting there for an hour and struggling, people will instead reflexively give in and ask an LLM to solve it for them.
When the child is able to go to YouTube and find a tutorial rather than having to puzzle it out, yes, it absolute does. We've seen this for decades now.
By definition, creativity cannot be automated, and AI is a fantastic automation machine. It can explore thinking paths at a rate humans cannot match. But creativity is bringing the unthinkable into the thinkable, and that requires sensory experience [1]. Specifically, new definitions and symbols which never existed before. Imagine the word/token embedding vector space, and expanding that with new independent dimensions. Is that even possible ? When you look at history the answer is yes !. And each time there was an independent dimension added, it was an act of genius. It is an instructive exercise to name these moments in history where an independent dimension was added to human thought. Some examples in math would be the invention of a number, and in politics could be the idea of democracy. By contrast, LLMs are trapped in the vector space they are trained on.
[1] https://philsci-archive.pitt.edu/28024/1/Scientific_Inventio...
A lot of the time people state the kind of fundamental limitations of LLMs very confidently when it feels like it is too early for people to really know. Like we are already well past the point where where LLMs are just pre trains on the internet with some RLHF for chatbot… Most of the effort is spent on elaborate reinforcement learning.
Is it unconceivable that future generations of LLMs could be RL’d to use einsteins visual method for theories [1] with the right tooling and geometry representations? Or just something random like that.
[1]. https://www.visualscribing.com/blog/2019-11-11-einstein-on-v...
When we solve problems we usually follow a heuristically guided energy efficient path. We just prune a lot of possibilities based on our existing knowledge and experience.
Creativity happens when we consciously (or not) go off the beaten path and explore. Most of those explorations are dead ends. But some will yield unexpected connections, patterns etc that we call “creativity” .
An AI system could also go on those kinds of explorations. Today they aren’t it because we are not asking them to.
> The emergence of physically consistent World Models offers a pathway to a synthetic laboratory. By enabling agents to run counterfactual simulations—to experience the physical consequences of a thought experiment—we may finally mechanize the feedback loop between intuition and logic.
I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become bad. This is common in programming too—things that are correct in the beginning but become problematic as they grow larger.
From an individual researcher's perspective, it's rational. You get more citations, your career accelerates. Everyone knows this. Paper counts aren't everything. Citation counts aren't everything. Journal impact factors aren't everything. You shouldn't only play it safe. But everything is tied to those metrics anyway.
Most researchers who give me work are fully aware of these facts. But are they going to change anything? Funding is still distributed based on those metrics.
Max Planck said, 'Science advances one funeral at a time.' Science doesn't progress purely through reasoned argument. The authority of the older generation, research funding networks, journals, and school-specific evaluation criteria all move together.
And honestly, I think discoveries will keep happening—probably quite rapidly. Because AI doesn't have the factional conflicts or interpersonal issues that humans do. It's very good at connecting papers across schools of thought without bias. In other words, the current human system is flawed at consolidating research, but I think AI is actually strong in this area. I expect AI-driven discoveries will continue for some time. The people who ride this wave will clearly be the winners.
Everyone knows things are broken, but no one is trying to fix them. I always think human society is inefficient. I read this post, but I'm more curious about who will actually lead the improvement effort.
All the factional conflicts are in there, and there are also plenty of reports of people getting weird / toxic / passive aggressive responses from AI.
Because the model is trained with everything, you can in principle get anything out of it. You want to get an answer based on all the right things, while keeping all the wrong things suppressed. But it's easy to get something less than ideal, due to the specifics of training, harnesses, context, prompts etc.
Well, these AI are never going to die in any real sense, so expect them to make orthodoxy more sticky, not less.
I presume you are an expert in some field. Think carefully about the boundary of the field and all the subtlety and complexity of that boundary and all the oversimplification you do to communicate that stuff to lay people. AI is, in some large sense, directed at all lay people, not experts, and even if we wanted to direct it at experts, at the edges of knowledge, there really isn't a lot of training data for that. Mathematics is a sort of exception because it has very clear validation criteria which makes RF particularly easy for it.
https://news.ycombinator.com/item?id=48863490
LLMs don't just 'average' their data.
AIs do things no human has done before millions of times a day.
LLMs are aggressively trained to reproduce facts and consequently struggle to reject orthodoxy. There isn't any reason they can't, in principal, make big new discoveries just by getting lucky, which is sort of also how humans do it, but its ok to acknowledge that current AIs aren't so good at certain things.
https://www.youtube.com/watch?v=KtQ9nt2ZeGM