I’d blame it more on just what happens when interesting research gets reported on by those seeking clicks.
What i am referring to is in the original paper.
I think it’s what happens when all this work goes on in a silo outside of interdisciplinary interactions.
1 Like
I agree. This preprint feels a bit quackish in how it talks. The research results may or may not be interesting-- but the interpretation should error aggressively on the side of assuming that the model is generating text rather than reporting on experiences.
Most papers I see use the term “simulated reasoning”. They are more careful, although their language also slips because so many things are being “simulated” at the same time and it’s hard to tell what’s what.
I am also curious when the philosophers, linguists, neuroscientists, and other humanities people will finally chime in. To some degree I think they just don’t have any answers-- nobody agrees on Chinese Room-- but they must at least have ideas.
Also, who says we can’t predict next words? That would be an interesting experiment I think.
I think they have chimed in at different times. I don’t remember reading anything from anybody who wasn’t working for one of these AI companies who thought these models could really comprehend the text, but i haven’t read a lot.
I think if we could then it wouldn’t be so unintuitive what kinds of tasks these models are and are not good at.
Certainly we don’t usually predict words. The New York Times had an op ed some time ago on which noam chomsky was a co author which specifically pointed out that thinking was not simply the most probable word.
I suspect you’re right. We just don’t hear from them because they aren’t associated with multi-billion dollar companies or saying things that would be useful for those companies’ PR teams. Whereas “exciting” pro-AI articles get platformed.
1 Like
“She’s a sentence finisher!”
2 Likes
I think we do, for example my wife and I can predict what the other will say often times. Just need a big training set.
Perhaps we’re thinking of different things with the term “search”?
I read your post here as looking only at the next move to make a decision; which I agree isn’t exactly a search.
But most expert chess players I’ve known are looking at a minimum of 3 moves ahead . . . and I’ve met one who was really gifted at looking 5 moves ahead (when he’s played several games against the opponent). These sorts of assessment, IMO, is considered a search. Perhaps not the type of search that a search engine performs, but it’s looking at things.
I think that the reinforcement learning process of a neural net is probably a kind of search through possible games.
The result of that search is effectively cached in the nn’s weights. Then it is retrieved during a game.
Probably something somewhat similar occurs for chess masters. Years of studying games encodes important patterns in the right side of their brains. This works with a more conscious, introspective search of moves.
You should see some of the youtube videos of games involving the best players… end games where it makes a difference 30 moves out, etc.
How far ahead they calculate varies a lot depending on how many reasonable branches there are. Magnus Carlson had an interesting interview where he feels just being clued into knowing the position has one clearly correct path vs multiple reasonable paths at key moments in the game would make him significantly better, and he’s arguably the greatest of all time.
I think there’s also different kinds of recall… so in the beginning they’ve just memorized the most ideal line out to 10+ moves. Then the middle game is a balance of calculation with huge amounts of pattern recognition (i.e. tactics) and that gradually becomes more and more calculation as you get to endgame.
1 Like
I kind of feel like the reverse is true-- GMs can usually glance at an end game and tell if it’s a draw or a win, and what best play looks like without any calculation. The last big calculation that occurs is: “will this give me a winning end-game?”
Of course, it’s always funny when it seems like every reasonable move is a draw, and then suddenly one player blunders the game, but neither player notices.
A famous study(ies?) found chess masters are much better than average at remembering chess positions from real games. But no better when the chess positions are random.
2 Likes
Oh definitely a GM can tell if an endgame is won or lost very easily most of the time but I think when two closely rated GMs play eachother its the endgame where you get absurd numbers of moves ahead calculated. Partly because the games are 3 hours a side and partly because in the endgame there are fewer pieces so it’s far more feasible to calculate 10-20+ moves ahead.
2 Likes
Seems like a relevant book for the discussion…
1 Like
Nothing to worry about here, eh?
My dev guy is cranking at me with a version of ‘feature creep’ lol. yeah, I know.
My calls are recorded, then transcribed by the phone company. Dev is setting up the following sequence:
- download the audio and transcript, attach to client record
- parse the transcript and create a summary, attach that to the client record so I have a quick overview next time I speak to them.
- parse the transcript and create a list of tasks, inject the tasks into my calendar. Like ‘call them back next tuesday at 11am’.
So first of all, whatever crappy AI the phone company is using, doesn’t identify speakers. So now we’re going to have to generate our own transcript.
And then, and this is new, AI’s don’t understand dates very well. If today’s monday and the transcript says I’ll call you back friday at 10, the AI can’t connect that friday is say December 15th or something. This is nbd, my dev can write code to do that - figure out that today is monday the 11th and next friday is therefore the 16th or something. But…yeah, feature creep.
Our companies call transcription identifies the employee taking the call as the “agent”. Real confusing for an insurance company when our insurance agents call in and the transcript calls the company employee “agent” and the agent “customer”.