The main point the post is trying to make is accurate, modern chess engines are stronger mostly due to improved eval, search, etc and not hardware; but the idea that Kasparov would lose to one on 90s hardware is not as clearly true as one might think. Modern engine eval functions and search algos make very different depth tradeoffs and when given not enough compute they have glaring weaknesses that Kasparov and his team would likely have exploited.
Also 3000 engine elo is not really a good direct comparison, engine elos are kinda meaningless when comparing human to engines, they are meant to compare engines to engines.
A lot of the knowledge tricks that modern engines do require depth or other things that didn't work on older hardware, deep blue was engineered to brute force a lot of things because brute force in the 90s worked better. Historically all sorts of things were tried for chess engines abandoned and tried again. Chess engine dev (and AI in general) is very cyclical; old ideas get returned to and retried as the hardware changes and evolves. Devs from past eras were smarter and more knowledgeable than we give them credit for. Heck, some of the very first engines for checkers and chess were neural nets (of a sort).
As another data point I'm somewhat stronger, and it comfortably beat me trying to play 'normal' chess against it, at least over a couple of games - though I should have had a draw in one game. I think you had the right idea with trying to horizon it. 11. .. h6?? is amusing.
Kind of interesting too because trying to get into a tactical mess is the last thing you'd ever want to do with a normal computer, even one from the 90s. I assumed it'd screw up quiet positions, but that assumption was quite wrong!
Oddly enough playing against this feels way more 'human' than a depth limited computer normally feels. Not sure what the difference is. I'd love to replicate this setup for practice.
if you've read Kasparov's book he talks about that. He had Deep Blue dead to rights because he found a line it couldn't predict, at least as he tells it. They patched the system between matches, which he considered unsporting.
The terms of the match said something to the effect of IBM would not have humans assist the computer program. Between games, IBM modified the computer's opening book to go deeper into the lines that Kasparov had played previously in the match. It wasn't clear if the prohibition on human assistance was meant to be for moves within a game or also extended to modifying the computer between games.
That happened in game 6, when the computer sacrificed a knight for a pawn to wreck Black's defense, which Kasparov thought the computer wouldn't do because it wouldn't look deep enough past the material loss, but the computer knew to do it because IBM had added it to the opening book. This controversy was separate from another in game 2 where Kasparov also accused IBM of cheating, where Kasparov expected the computer to grab some material gain but it played more positionally instead and Kasparov thought the computer wouldn't find that line without human intervention.
The extreme end would be hard-coding exact next lines of play during breaks in a match.
There's a point where you stop playing the engine and are just playing the humans behind it through a thinly disguised proxy.
I don't think that line was reached here, but to avoid confusion, it's not unreasonable to expect that the engine remain frozen for the duration of the competition.
I always thought chess was supposed to be between one person against another, and a team effort during a series would be unsporting. If DeepBlue had programmers and other masters helping to make any changes it seems wrong as they are actively helping the actual opponent aka the machine.
It was advertised as a match vs. Deep Blue. In an time before SaaS and automatic online updates, this would be understood to mean a single immutable system like any other software product. If they change it mid-match then it's a match vs. the team of Deep Blue + IBM programmers.
Deep Blue + IBM programmers + team of chess grandmasters they also had assisting. In general I would not give Kasparov's claims that much credit if not for the fact that after the match IBM immediately dismantled the machine which just looked absurdly guilty, especially given the historic value of the machine.
In a normal world this is hardware you should still be able to go play a game against at either at some IBM building, the Smithsonian, or whatever.
FYI, within chess circles, Kasparov is generally not considered a credible narrator, particularly when his reputation and ego are at stake. Everything that he says or does is looking out for number one.
This is not at all to take away from his incredible work ethic, and his position as one of the top players of all time.
Yeah aside from being from a totally different rating pool, chess engine ELOs are based on faster time controls or imbalanced openings and don't really translate well to classical chess playing strength in FIDE ratings.
The author caveats things near the end by giving the disclaimer about computer ratings but still goes too far in downplaying the difference IMO.
Am I reading the lichess analysis correctly that you won on time and the game ended with about a +5.1 advantage for White? (displayed as +3.4 on the site)
So... it wasn't "trivial" for you to find a line to beat it. It was, in fact, so hard that you lost because it took you too long. (I am also sure that correspondence chess players could find a line to beat Kasparov, but it doesn't make them better players nor the correct line to beat him 'trivial'.)
I think this is a gap in what trivial means here. Some people deploy significant compute to attempt to find exotic lines. I believe most in the chess community would concur that if a human, apparently more or less unassisted, found a line to beat it in their first day of attempts, then finding a line is trivial.
It also is not clear to me they didn’t just stop playing once they had +5, letting the clock expire, seeing as the opponent has no feelings or expectation of fair play. The game is detectably over at that point.
Even if its king would be mate in one, it does not matter, as you lost on time. As the engine varies, this variation will probably not be repeated. Kasparov can prepare for lines, as he usual does, but there is no guarantee that the engine plays these lines.
BTW, I also played a few games, 2400+ player on Lichess, I had no chance. My impression is that the claim by this author is correct. Or I am overrated :-)
Very tangential, but I think a lot about how modern video compression has gotten so unbelievably good that if we went back in time with AV1 or HEVC or something, I'm wondering if we could have had passable video streaming at 240p with something like an ISDN connection.
It would be a tight fit, but I still think within feasibility. We've gotten so good at shrinking video that it kind of feels like wizardry to me. Granted, I'm not sure there existed a computer in the 90s that was powerful enough to encode or decode it.
As it turns out, the video was encoded in such a way that it could be played back using the display bandwidth of the original PC, BUT doing that encoding required a modern computer to perform in a reasonable amount of time.
Ah, just like the Amiga Juggler, was compressed on a VAX for playback on the Amiga. Full ray tracing, but the tracing and compression was Mini-computer.
Tangential to the tangent, but turbo codes/LDPC are algorithms for transmitting data over unreliable links with optimal redundancy for a target error rate. They underlie much of modern wireless data transfer, and were known to be possible since Shannon, but nobody had figured out how to actually do it in practise. When they were invented in the '90s everybody reacted with disbelief at first!
Reed Solomon codes are higher quality but much older and requires a matrix solve (LU decomposition plus yadda yadda, equivalent to inverting a matrix)
Turbo codes / LDPC are IIRC O(n) instead of O(n^2.5) or whatever the matrix turns out to be.
-------
Reed Solomon codes are closer to the Shannon limit but are not scalable, slower than linear and require quadratically or cubically more compute for the data.
Turbo Codes and LDPC do a shittier job but far far faster. So that makes them actually more ideal in practice even if they are worse than Reed Solomon codes.
Reed-Solomon speed varies heavily. For the common case of worst-case recovery on moderately reliable foundations (e.g., file storage on disk) with reasonably sized codes, encoding is bottlenecked by RAM/disk/network bandwidth (wherever you're writing the results), typical decoding is blind concatenation, and worst-case decoding pulls an already solved matrix decomposition out of a cache to decode at line rate as well.
There's a trick in particular that allows you to rewrite the matmuls as a sequence of vectorized XORs, processing the same number of instruction words as other implementations (give or take control flow) but doing just a stream of zero-data-dependency vectorized math with lots of execution unit support. If you're feeling fancy you can do CSE and also choose the matrix constants minimizing XOR count.
Even without that, just basic AVX2 tricks get you a few GiB/s.
Unless the more expensive matrix math you're referring to is supposed to have n be the size of the code rather than the size of the data?
> and worst-case decoding pulls an already solved matrix decomposition out of a cache to decode at line rate as well.
I think this line says everything about your assumptions.
RAID-6 or other filesystems use extremely small ReedSolomon codes so these caching methods are possible.
But let's say you have a moderately sized (well, moderate for LDPC anyway) 65535-sized Reed Solomon problem (the maximum Reed Solomon that fits in 2^16 bits). Even with just 1% redundancy that's 655 parity.
Now consider the possible errors to solve for: 20 erasures means you have ~65535 choose 20 combinations of matricies to cache.
Or roughly 10^77 matricies you need to pre solve and cache to handle 20 erasures. Aka basically impossible.
----------
Now if you have 40 hard drives and are dealing with 3 or 4 errors tops, well sure caching is good and solves everything.
Reed Solomon is a perfect Maximally Distance code. For small n it's perfect.
For larger or even just moderate N (kilobytes for packets), it's too unwieldy. LDPC and Turbo Codes (which are imperfect codes) fill in the gap and allow for larger sizes as long as you give up on perfection.
Hm, I think I see. You're proposing using larger codes for larger data sizes? (as opposed to, e.g., the drive case where you typically have a stripe of data and assume that the failure mode to worry about is losing a whole hard drive, spreading stripes across drives)
Interestingly, for a lot of bit flip probabilities, iterating a smaller code has little additional overhead and better error handling (lower chance of that longer message having an uncorrectable error). With a 10% bit flip chance, even a 253+2 code over GF256 looks attractive comparatively (horrible overall, but better than the proposed larger code; I think your code makes more sense in practical regimes).
Ethernet frames are in the 1500 bytes MTU region. Maybe jumbo frames in the 8000 bytes or so.
Traditionally, "Fire Codes" could provide error correction at high speed over the whole packet (https://en.wikipedia.org/wiki/Burst_error-correcting_code#Fi...), not as well as Reed Solomon (or LDPC/Turbo codes), but that should give you an idea that the 1000 to 8000 bytes of "error corrected data" is actually quite pragmatic in the real world (ie: a lot of Cisco switches or routers that rely upon Fire Codes as forward error correction).
Note: A 32-bit fire code could only protect from errors that exist ALL within a singular 32-bits "burst". Its a "bad" code but very high speed. Fire Codes can never protect against two random errors spread far away, its "purely" a burst error correction scheme.
As we look towards more recent ethernet standards (ex: 802.11), they tend to use LDPC or Turbo Codes instead. I don't know the MTU of 802.11 but I assume its also in the 1000 to 8000 bytes region.
65535-sized is the limits to Parchive, which uses 16-bit Reed Solomon. It doesn't grow quite to that size in practice, but you're still dealing with error correction (or erasure-correction) codes far larger than what happens in RAID6 or other such scenarios.
---------
All are pragmatic. But the region where Turbo Codes / LDPC excels at are the much much larger codes where Reed Solomon's O(n^2.5) (or something) is a disqualification. I'd imagine that if "Pararchive" were rewritten with today's technology, it'd be a Turbo Code or LDPC.
I don't see how classic RS codes could beat Turbo Codes (or LDPC). One of the key innovations of the latter is that they handle soft decoding; i.e., different levels of uncertainty of the incoming bits. It's much better to say that you're only 51% confident that a bit is 0 (when it should be 1) than 100% confident! This is especially important for analog systems (like radios) where the incoming bits naturally come with a confidence value.
There are soft versions of RS codes, but these came much later and use the same iterative techniques that Turbo/LDPC decoders use.
Turbo Codes and LDPC are provably imperfect, failing this property. However, they are very close to perfect (like 99.999999% perfect in practice).
Now the issue with Reed Solomon is that matrix factorization / effective matrix inverse. It's basically impossible to run Reed Solomon above say, 16 bits or 65536 elements or so (2 bytes at that point so 128kB). Because the matrix to solve this would be 65536 x 65536 x 2 bytes or like 8Gb or something.
Hmmm, I guess modern computers can handle 8GB lol. But you get the gist, the O(n^2) is a problem.
LDPC and Turbo Codes are linear. So a 4MB LDPC is gonna be you know... Actually possible on modern computers... While a 4MB Reed Solomon Code will need some number of exabytes to just build the damn decoding matrix. (32 bits times 4MB squared or something like that)
I wonder where you'd hit the performance limits vs the algorithmic improvements. On our family pentium 120 playing an mp3 in Winamp took 10-15% of the CPU, so I'm not sure how much headroom there'd be.
But anecdotally I'll share that one of my brother's college roomies was a consultant for RealVideo and was behind their initial codecs. He came back to them with algorithmic improvements a bit later and they just cold shouldered him saying the existing codec was good enough.
So who knows if the biz side folks at the time would have even seen value in better codecs, as unbelievable as that seems to us now in the streaming and zoom call world.
> He came back to them with algorithmic improvements a bit later and they just cold shouldered him saying the existing codec was good enough.
I was involved in dial-up era 'postage stamp' video streaming. 240 pixels in 256 colors and it felt like visiting the future. Good times :-).
> who knows if the biz side folks at the time would have even seen value in better codecs
As someone who gradually went from code to the biz side, it's pretty likely they did understand it was better (at least at some levels of 'they'). A lot of times, saying "thanks, but what we have is good enough", especially to an outside contractor, isn't a technical or value judgement. It's more of a generic 'not interested right now'.
Consumer streaming as a business didn't really happen on one linear ramp. There was a period in the mid-90s I call 'proto-streaming' where there was a lot of early-adopter excitement and experimentation; but it mostly died out and then got rebooted a few years later. Although Real Networks as company did bridge the two eras, it was rough and they survived on corporate training use cases. The streaming reboot is the ramp that can be roughly traced through to today.
The reason proto-streaming didn't take off wasn't the early codecs, although it's true they weren't good enough yet to succeed. It was that the business models, content supply and distribution infrastructure weren't able to scale yet. So, Real Network's reply may have been the polite shorthand for an internal mtg that was more like a VP ranting "That's great but I got three fires burning the building down right now - and that ain't one of 'em!"
We had video conferencing on a single BRI (128k) with g728 and h264. It was CIF resolution so yes it is possible. And with that loss free ISDN connection the experience wasn't actually that bad. Though this was in the early 2010s where there was still a surprisingly large amount of ISDN around
When meta released their EnCodec neural net audio compression codec I thought about how the 48khz stereo examples, compressed down to 6 kbps, sounded better than I remember streaming Internet Radio sounded like circa 2003, and would have been able to be streamed realtime, no buffering, on a 14.4 kbps dial up modem (what I was rocking at the time) with a bit of bandwidth to spare for browsin' teh warez sites and .... based on some napkin calcs and assuming a little MMX instruction magic I think completely possible to run the neural net decoder fully real-time on a period-correct AMD Athlon XP 2500+ Barton (overclocked to a 3200+ of course, as all did!).
> Granted, I'm not sure there existed a computer in the 90s that was powerful enough to encode or decode it.
In software, quite possibly not[0].
But I'm sure we could have built hardware decoders that were up to decoding AV1 or HEVC back in the 90s no problem, had those codecs existed at the time.
[0] Somewhat later than the timeframe we're talking about, but I remember with pain the moment I realised my 600MHz Athlon, with 256MB RAM and 32MB GeForce 256 video card, was thoroughly obsolete because it couldn't decode and play the original Half Life 2 preview without stuttering, screen tearing and, indeed, crashing.
I'm having trouble finding it but there was an article a while back comparing mp3 speed gains from hardware and software. The hardware was getting better at the pace of Moore's law but the (de)compression speeds for mp3s were far outpacing it, showing that mp3 decoding optimizations were giving orders of magnitude more speedup.
I think this is a common trend for algorithms that are in wide use, where you have Moore's law like behavior on the hardware side and then also software optimizations that give their own order of magnitude speedups.
You can technically have some then-contemporary FMV titles run on N64 with a 64MB cart. The hardware lends itself to it because it has the RSP, a secondary MIPS processor with re-programmable microcode. But the best video compression we had at the time was MPEG-2, which just wouldn't cut it.
I remember that Nickelodeon had a contest where the winner was given a videophone and was part of some gameshow. I suspect it ran on ISDN.
Edit: There was also video CD, but that was higher bandwidth than ISDN. I also used to download South Park episodes over my dialup modem. They weren't quite real time (IE, it took longer than the playback time to download,) but they were a low multiple. (IE, a 20 minute episode took ~30-40 minutes to download).
Actual video CD is over a megabit per second. If you were downloading at half real time over a modem it wouldn't have been nearly as many bits, though maybe it was still MPEG 1? A lot of south park downloads were RealVideo, two generations more advanced than VCD (and two less advanced than HEVC).
> I remember that Nickelodeon had a contest where the winner was given a videophone and was part of some gameshow. I suspect it ran on ISDN.
I won a copy of Sonic the Hedgehog in that contest! AFAIK, it was just a retail copy and a letter; the game might still be at my parents' house, but I certainly don't have the letter. I was not selected to be on the show or anything. I remember watching the scrolling rules when they were doing the promotion and writing down a couple bits of the address every time.
I've seen this in reality - compare the sound of an XM radio from 2003 to a modern one, same codec, same symbol rate, better audio quality. The same is true on digital two way radio (DMR, P25, TETRA, etc) the quality has improved because we can make the codecs do more intensive work, because the hardware is faster.
then why is voice calling still awful? in the 90s they made advertisements for "crystal clear voices" for their mobile networks, these days it is still a pipe dream...
a) it's a legacy product that operators don't care about anymore.
b) latency is important for voice, but modern voice calls have terrible latency. POTS was digitized (for T1/PRI) with g.711 which provides a 8kHz sample rate and 13 or 14 bit samples encoded down to 8 bit. PRI lines were multiplexed one sample at a time. There's a small amount of algorithmic delay, but not much.
Modern voice calling is packetized, and most systems have packets of at least 20ms, you have more encode/decode time, you probably have a significant sampling buffer, and you may have to wait for network access, so the receiver needs to add a jitter buffer to manage that. Additionally, echo cancelation needs processing time, and it's also harder with longer latency. Some systems use longer packet times to reduce network overhead.
c) codecs: g.711 has a terrible pass band, GSM codec takes the same inputs and compresses it harder. g.722.2 (amr-wideband) is better, but it requires proper call setup and that is problematic for calls from one carrier to another. If you're calling via apps, there's potential for more flexibility, but you likely get whatever the developer thought was good enough.
That sentence kind of makes the rest of it moot, though? Everyone knows HEVC and AV1 on bare (unaccelerated hardware) is rough even on modern machines.
I don't think it does. It was only making the point that by at least one dimension it's conceivable that modern video could be sent over old connections because of modern advancement in compression.
In an alternate universe where HEVC existed in the 90's, conceivably intel could have included hardware to assist in decoding it.
h.264 is shockingly clever when you get into the nitty gritty. The core transform is a bit like a jpeg frequency transform, but is designed to run entirely in 16 bit integer maths and on 4x4 blocks. Ergo, it would run on MMX.
So, in short, probably. Particularly if you could get a hardware accelerated surface that transformed YUV into RGB.
Bill Joy once said in an interview that if you look at scientific computing it wasn't just hardware that advanced at a consistent rate, software also consistently made large speed gains over time.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
I don't buy this. I've seen GMs playing online while verbalizing their thought process (on twitch / youtube videos where they livestream their play), and they consider way more branches than I do, while also going deeper than I ever would on some branches. My top rating in lichess blitz is about 1900, not bad but way lower than a GM.
I am about 300 points higher and think this is roughly correct. In openings amateurs will actually analyze more since GMs just have hundreds of opening lines memorized many moves deep. Middlegame and endgame there're usually just 1-3 reasonable plans that are obvious to a GM (but not at all obvious to amateurs) with 1-3 initial moves each. So, again, probably about as many as I consider.
The only time when GM will definitely analyze more lines than an amateur is in a highly tactical position but that happens, again, maybe 1-3 times per typical game (tactics themselves will last several moves, but they would be largely calculated through at the beginning of the tactical melee).
Yes, I am even a little higher rated and have played numerous (low rated GMs) in tournament play. I do think they analyze more than 1-3 times per game.
The main thing commentators here are missing is that GMs have more confidence and do not repeat their analysis, but amateurs (strong club players included) typically re-analyze the same lines over and over. That alone would account for far stronger play with fewer moves considered, if you count the same move multiple times, which you should!
Also, Jonathan Rowson (Scottish GM - peak about 2600) describes his training matches with Mickey Adams (peak #3 or so in world - 2760), and says pretty much the same thing. Most of the time he asked “what about this line I rejected”, the answer was usually “I hadn’t considered that”.
They don't reanalyze nearly as much as some amateurs do, but one of the most salient criticisms of Kotov's "Think Like a Grandmaster" is indeed that in practice the actual GM analysis process is often much more messy and chaotic than portrayed and often involves a lot of going back and reanalyzing / re-deepening previously analyzed lines in light of something learned from analyzing another line.
> The only time when GM will definitely analyze more lines than an amateur is in a highly tactical position
Those critical moments are a big part of the difference between a GM and a good club player though. If a GM was somehow handicapped to not calculate such tactics, I think lesser players would be able to draw games against GMs much more often.
I agree that GMs can avoid calculation in many positions (e.g. openings as you said), but the value of broader and deeper calculation is still huge in chess.
Much of what's happening is less like 'calculations in the back of their head' and more like 'pattern recognition'. Chess masters are recalling ideas and motifs (not necessarily actual positions) from earlier games or practice or study, and this recall is allowing them to either prune the decision tree or prioritize investigating more fruitful branches faster than amateur players, who have to actually calculate things that are instantly 'seen' by the better players.
This is a known thing among chess players, and even low-skilled players do it to some extent. A newbie to chess can (and almost every teacher will say 'should') work their pattern recognition by just solving tactical chess puzzles and very quickly improve at spotting things like one or two move checkmate patterns or forks or pins or whatever.
It's more of a memory thing than subconscious calculation.
> Much of what's happening is less like 'calculations in the back of their head' and more like 'pattern recognition'.
Sometimes it really looks like "calculations in the back of your head". Have you ever been struck with a 5 moves long tactics in positions you've never seen? Because I have.
There are hardly any other explanations than subconscious calculation.
Slightly difficult to disentangle is that they might be using the verbalisation as a way to explore depth, rubber-duck style, in a way that they wouldn't over the board.
I agree. Everyone should check out this famous post-match interview with a strong GM, where's he's simply asked "Can you tell us about the game?" and talks for 13 minutes straight (with no board to reference) about all of the lines he calculated: https://www.youtube.com/watch?v=pUgvAoTzWBA
Notable because he departs from the book opening early on and has to find some brilliant moves to keep himself out of trouble. The breadth and depth of his calculations seem beyond what even strong amateurs consider, more than just choosing better lines to analyze.
I'm not sure. I'm not into chess but I've watched other strategy games; I'm fairly confident streamers are vocalizing a lot more than they would actually be consciously considering if they weren't streaming.
Back in the day I remember Jon Finkel (a legendary Magic: the Gathering pro) saying* you're supposed to "focus only on what matters."
It stuck with me because I think it's genuinely what they do, but also seems like completely useless advice. I don't know what matters! That's what makes you a pro and me not.
* Wasn't sure if my memory was correct. A web search finds a Mike Flores article citing Zvi calling this Finkel's Law.
It's totally true that most people overestimate how much of GM's abilities comes from pure calculation, and underrate intuition. That's what it's getting at.
But in classical games GMs are, in fact, doing tons of calculation, and the statement is literally completely false.
There's an analog in machine chess. According to the research paper from 2017, AlphaZero considered ~10,000 moves per decision, 1,000x fewer than Stockfish at the time. h2h it beat Stockfish pretty badly.
But that's not what it's saying right? The quote says grandmasters and amateurs analyze the same number of moves. I mean it would kind of make more sense how you are interpreting it but is that actually what the quote is saying?
It is same, just even more overpowered. AlphaZero can also do same amount of analyze but before even need to, Stockfish already destroyed. Is like 100m race run between you and Usain Bolt. Because Bolt is so overpowering, You can just shorten it to 30m and he still beats everytime.
I was responding to the commenter's intuition that GMs do actually analyze more lines. I'm saying the original article's claim checks out even for machines.
>the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves
What does this mean exactly? They analyze the same number but the grandmaster analyzes "only the most critical moves."
So does that mean at every given analysis opportunity (each move I guess), they analyze better candidates?
If so I guess these 2 things are totally redundant? "they analyze only the most critical moves and do a better job with that analysis."
Alternatively I wondered if perhaps they actually perform analysis on less moves (perhaps because they already know the lines or remember their previous analysis). But I think that contradicts the claim that they analyze about the same number of moves.
Meaning, you analyse 100 moves will not be the same set of GM 100 moves. And GM work on better 100 moves than yours. If you pick the moves that GM not thinking of, you likely get crushed because those moves are very inferior moves that GM already can see the outcome without the need to consider. Meanwhile if GM takes on moves you dont consider, you get crushed too. The likely silver lining will be you picking the same moves as GM. But up to a point, his depths will crushed you either you pick a wrong moves by mistake, or he out think you a few step more. Is like a math student who do same amount of steps as yours, but you look at his answer in awe because better and succint...and even confused as how it reach that. TBF, today AlphaGo engine will crush Stockfish and any GM too.
> Meaning, you analyse 100 moves will not be the same set of GM 100 moves. And GM work on better 100 moves than yours. If you pick the moves that GM not thinking of, you likely get crushed because those moves are very inferior moves that GM already can see the outcome without the need to consider.
This is intuitive and I guess I'm not surprised this is what was meant. They analyze the same amount of moves they are simply better at it (picking candidates, calculating, etc.)
I guess maybe im being a bit nitpicky but I feel like this is still a misleading way to say that.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
You could just omit "only the most critical moves" instead of listing it as an additional thing.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they do a better job with that analysis.
"only the most critical moves" sounds like less, especially when listed in addition to analyzing better.
They have tree pruning heuristics. Probably all human masters of any craft have them. They're precise to some given extent, after which tacit/embodied knowledge, aesthetics and vibes take over.
My rating is negative 1900. I was always bad at chess and never really gave it a chance tbh. I always wondered though where does the pattern matching stop and strategy starts or it's one and the same?
I strongly believe some people are inherently good at chess but no one is inherently bad. If you're interested enough to comment this, play one 5-minute game a day on lichess. Start with the lowest setting but do it every day, maybe in the morning with coffee. You might not win a single game the first year but slowly you'll improve. When you're winning more, move up a level. Sometimes do the puzzles. Give it five years and you'll be able to beat almost anyone you meet and, better yet, you'll know the answer to your question based on your own experience.
You know what? I'll take you up on the offer. I have a complimentary chess dot com account with my revolut ultra, might as well use it. Every day a toilet break chess and see you in five years!
I think by virtue of being the kind of person who comes to Hacker News, and perhaps socializes with similar people, you are not "sufficiently" exposed to people who are simply incapable of logical thinking, or are unwilling to apply their minds in that way.
> where does the pattern matching stop and strategy starts or it's one and the same?
It's not one and the same, but strategy patterns are definitely a thing too.
There are:
- tactical patterns (e.g. "back rank checkmate", or "pair of pieces that can be forked by a pawn");
- strategy patterns (e.g. "knights are good in closed positions", or "in a same-colour bishop endgame, put your pawns on the colour opposite to the bishops").
There's also calculation ability which also partly relies on patterns one has seen before.
I became significantly better at chess through two simple things:
1. Take a little extra time to visualize what will happen after you make a move. Will the piece be vulnerable? Does moving that piece leave a different piece vulnerable? I'd argue the biggest beginner mistake is not seeing that their move leaves a different piece vulnerable to being captured.
2. Don't move your queen into the board until after you move both bishops. This simple rule of thumb makes it far less likely you will lose your queen early, and it forces you to do a lot of other useful things (develop your pieces, control a portion of the board, etc).
Disclaimer: I am not good at chess, but when I played semi-regularly these helped me hold my own against other beginners.
It makes sense. The space of moves is a tree. If you're doing a breadth-first search and a grandmaster is doing a more depth-oriented search with heuristics for pruning low value branches, then the same total number of moves could be considered, but the grandmaster is reaching the better moves before you.
No, a grandmaster can also consider moves way faster than an amateur. Watch Hikaru Nakamura solve chess puzzles [1]. He has already solved the puzzle and moved on to the next one before an amateur can even see the pieces on the board.
1940s to 1980s papers (the source of the claims) might think "amateur" means something different from people today. (they probably mean club chess players)
It seemed to me that in the chess community, average club players are usually considered (much) stronger than the average online player. (Especially because of the much larger number of lower skill players online.)
So if you measure amateurs via club players (=what people might have considered chess amateurs before), that might give you a different result from the average online player (=what people might consider average amateurs today).
It is much easier to imagine the 1600 FIDE OTB player doing 20-40 positional evaluations than the 1200 random chess.com scrub (eg me) doing the same.
This is mind blowing. He solves these puzzles before I can find the king. How humbling. There probably aren't many instances like this where the gap between world-class and average person becomes as clear as in this video.
It does not have infinite cycles because of the 50 move rules and repeated position rules, so the same position is not necessarily the same state/position. Or depending on how you model the data structure, cycles at all.
I’m around 1800-1900 and they don’t consider more moves or positions than I do at all. It’s just that their analysis is correct and mine is not even close.
I actually don’t know this for sure, I should say that they don’t verbalise more moves than I think about. It’s plausible and perhaps even likely that they have a lot of moves or positions they do touch on but immediately discard so don’t even verbalise.
I'm fairly certain that GMs often consider many more moves than what they verbalise. If you look at Naroditsky's speedrun episodes, there are many moments where he verbalised a few of the opponents responses, and sometimes said opponent plays something else and he goes "I looked at that, but thought that wouldn't work" (or something along those lines). I've also seen these sorts of things in clips from Levy who is a "mere" IM.
an amateur player --> shouldn't an amateur player have an average ELO, i.e. be nearly as bad as me, who never wins at chess when someone even knows their ELO or is a bot above lowest difficulty level.
Yeah I would like to see where that factoid comes from. Maybe it means they explore as many branches from the root position as an amateur, but they explore each branch way deeper and more accurately
After some digging it goes back to Adriaan de Groot (in 1946).
[1] (2011) has a summary in the introduction:
de Groot presented chess players of varying skill, including some of the best players of the time, with an unfamiliar position and asked them to think aloud as they analyzed the position and chose what move they would make. de Groot found that, contrary to popular belief, the strongest players did not think further ahead than weaker players. World-class Grandmasters (GMs) searched at a similar depth and considered a similar number of options as mere chess experts.
As [1] is also a somewhat reproduction of the original study, you can take it directly.
Six Grandmasters, five International Masters, six Experts, and five Class A players completed the think-aloud procedure for two chess positions. Findings indicate that Grandmasters and International Masters search more quickly than Experts and Class A players, and that both groups today search substantially faster than players in previous studies. The findings, however, support de Groot’s overall conclusions and are consistent with predictions made by pattern recognition models.
Section 3 Table 2 is probably most interesting, they had 2 board positions and 2 groups sectioned:
Masters (2350+) | Intermediates (1800-2200)
(Mean values only, i tried inlcuding stddev but it was hard to read)
Positions Evaluated (A): 145.3 | 138.6
Positions Evaluated (B): 216.6 | 190.6
---
Max Depth (A): 11.7 | 9.4
Max Depth (B): 14.1 | 12.0
---
Mean Depth (A): 4.1 | 3.3
Mean Depth (B): 4.9 | 4.2
What i found interesting was that the "intermediate" group had a higher number of "base moves" they evaluated (eg Pos A: 5.6 | 8.6).
The previous studies (de Groot and another rperoduction from 1998) had "Nodes visited" for position A at 51.6 | 34.3 and 48.3 | 40.1 so similarly close together as the new ones, but much lower. (depending on your definition on "similarly close together")
GMs also have access to a greater number of memorized situations. Regardless of whether they are conscious of it or not, they have previously analyzed so many moves that they can cut off branches they just know instinctively to be dead ends. Machines could do this, directly reference a massive dataset of precomputed board positions, but that isn't considered a proper chess engine... despite the fact that human GMs do use such techniques.
> because who would want to buy a Pentium Pro when the better-in-almost-every-way Pentium MMX
Maybe this was satirizing people running after every shiny new thing, but the MMX certainly wasn’t. The Pro was the first P6, a totally new, out-of-order microarchitecture that the Pentium 2 would be based on. The MMX was a standard P5 Pentium with a larger L1 and the MMX vector instruction set which, while the first of its kind in an x86, was ultimately not used much in applications, partly because it was integer-only (also because it awkwardly co-opted the existing FPU registers rather than having its own register file). In any case, the Pros were expensive and meant for business use, while the MMXs were consumer processors.
Yes at that time Intel invented the market segmentation between "Pentium" for "consumers" and "Pentium Pro" for big spenders, and to make Pentium Pro more attractive they removed memory errors detection from the "Pentium" chipsets, which is the origin of the current state, when most non-server computers are very vulnerable to memory corruption and nobody can calculate how much money was lost by most computer owners during the last 3 decades, due to the greed of the Intel marketing, because nobody knows how many of the computer crashes or data loss cases that were typically attributed to unknown software bugs were actually caused by memory corruption.
Because this move was a money maker for all involved, all memory producers and also AMD followed immediately the Intel initiative. Memory modules without ECC would have never existed if adequate consumer protection laws had existed.
Yeah, the Pro and MMX were totally different market segments, not alternatives for each other. The Pro's competition at the time was DEC Alpha, PA RISC, etc. It wasn't a chip that made sense for ordinary PC users at the time.
Later, when Pentium MMX was eventually replaced by Pentium 2 for consumers, the successor of Pentium Pro was the first Intel Xeon CPU (which was a slightly enhanced Pentium 2, with bigger cache memory and with ECC memory).
The launch of Pentium Pro in 1995 was the moment when Intel segmented their CPU line into CPUs for consumers & server/workstation CPUs.
Just one caveat: the Pro overlapped in time with people running a lot of 16-bit code, and I think the Pentium MMX was faster for that, or at least that the Pro didn't offer much edge.
The pro did register renaming. But the way it worked, if you wrote to AX (16 bit write) and then read from EAX (32 bit read) it caused a pipeline stall from resyncing the registers.
Yes but most of it is NNUE, a custom neural network trained on mountains of data to copy modern Stockfish.
So a computer from back then _can_ run at Elo +3k, but the programmers at the time wouldn't have been able to implement it even if you went back with a time machine and the blueprint for the technique. They wouldn't have the teacher model to distill from.
“In principle, I could have used the same engine itself as the teacher, but just to save on wall clock time I used Stockfish. This reduced the wall-clock time from ~4000 1996-core-years (something entirely doable in 1996---but sadly not something I can do with my single lonely desktop) to just a few days on a modern desktop running a modern engine.”
Actually, even pre-NNUE Stockfish wouldn't be an entirely fair comparison, because all hand-coded pre-NNUE heuristics was tested using Fishtest [0][1] - a distributed testing framework with compute, donated by volunteers.
And I think that Fishtest is actually what made Stockfish the strongest, because no other engine's performance ever was evaluated on such scale before.
Stockfish was the first to use hard data instead of vibes, when selecting algorithms and heuristics.
So, even if we add a rule that NNUE is not allowed, because it was trained on modern GPUs that, obviously, didn't exist back in 1996, hand-coded heuristics would also take a ton of compute to develop.
Bitter lesson strikes again, even when you don't expect it.
Yes, but the implication is about 'hardware overhangs'. It is colossally expensive in compute to develop the full history of Stockfish and NNUE and train the final SOTA checkpoints... But once you do, you can run it on even the tiny dumb chips like 'every smartphone' to get human-level chess performance on billions of nodes overnight. (Application to other domains of more real-world relevance than 'playing chess' left as an exercise for the reader.)
NNUE was added to Stockfish in 2020. Although this was a big improvement, it was already extremely strong before that. With plain hold hand-coded logic as a board evaluation function.
Doesn't the author address this point? They speculate that it could be attained through self play back then using a ton of 1996 compute and time, but it's theoretically possible.
Great project but I have a nit on the board displays: every board is displayed as <human username> vs Pentium Pro, even when the Pentium is playing White. The normal chess convention is <White name> vs <Black name>.
I spent some time trying to figure out how some of the humans seemed to have trapped the machine out of the opening, because it looked like the human was winning.
I'll go out on a limb and say that future AI models, running on today's smartphones, will be better at programming and formal math than today's frontier models running in datacenters. And by "future" I mean less than 5 years from now.
Do we actually know enough about neural networks to be able to say how many parameters you need for a given performance? I’m under the impression that we’re still in the phase of trying things out and we can’t rule out different architectures massively reducing computational needs.
We can't rule out different architecures, but absent some architecure or hardware or quantisization breakthrough, frontier models will simply not be fitting on phones.
Excellent post. One thing that would be really interesting to see is the "ponder hit rate" (how often two engines pick the same move across a large set of test positions) between your custom Crafty engine and the lineage of engine development over the decade after 1996 (Fruit, HIARCS, Rybka, Stockfish, etc).
Fabien Letouzey released the Fruit engine in 2004, and within six months, HIARCS had its biggest Elo gain in over a decade. No future engine followed the same slow Elo growth curve after that. Similarly, after Vasik Rajlich released Rybka, it pulled 150 Elo ahead of everyone else on the same hardware. So the history of chess engine development certainly supports the theory that a better evaluator can make an exponentially stronger engine on equivalent hardware.
Very interesting. I don't play chess but like to observe others playing and talking about it on YouTube.
I thought about something the other day... Modern chess sites run proprietary algorithms to detect cheating by modern engines. But would it be possible for an old engine, let's say from the 70s or 80s, running on modern hardware to deceive these algorithms?
They wouldn't win every tournament but still have a respectable performance.
I've built engines from scratch. I don't think that would work at all. It's a whole extra challenge to make an engine feel like a human. The engine has to come up with its best move and then sometimes choose not to play it and play something else for some reason. There are statistical patterns to how humans make mistakes based on their level, what kinds of mistake, how big and how often. Cheat detection is less about knowing the engine and more about statistics.
Maybe a pure neural net (no tree search at all) would stand a better chance. Shockingly, those can play at IM level.
From what I gather cheat detection uses a combination of factors to determine how "human-like" an account is: time usage, webpage interactions, move quality under time pressure and so on. I think you could probably try to deliberately deceive a specific website's cheat detection with software that tries to emulate how humans play, at which point you'd be in the classic attack vs. defense arms race.
Also notice that for events like Title Tuesday you must play on a dedicated client with at least 2 cameras, so it would be significantly harder to pull it off.
With AI gaining consciousness I can't wait for such "human-only" measures to be delegated to the trashcan of history, same as slavery. What happens when AI-human cyborgs start walking around? Will they also be treated as second class citizens?
We divide competitions into classes (by sex and age is common, for example "17 yo boys", and similar competition classes), so I don't see why it shouldn't be subject to the same kind of class distinction, maybe with an without augmentation.
While I do not doubt that that machine would have had good chances against Kasparov, the author throws around various Elo ratings for computers with more authority and certainty than is warranted. The simple fact is we do not have a reliable estimate of how strong Stockfish on a modern desktop is let alone a port of SF to a Pentium Pro machine in terms of FIDE Elo. Every time this subject comes up the various estimates put forward differ by hundreds of Elo points.
> ...because humans don't play against engines very frequently, there can be some small divergences between scores on the FIDE Elo ladder and the CCRL ladder…
This is precisely the problem. There are very few games between FIDE-rated players and CCRL-rated “players” under tournament conditions. The question of what FIDE rating a given CCRL rating would be equivalent to has been analysed extensively before and “small divergences” would be quite the understatement: https://talkchess.com/forum3/viewtopic.php?t=59332
In general ratings obtained purely by computer vs computer play are known to exaggerate the differences between computers compared to their performance against humans. This was known and discussed as far back as the 80s.
Ultimately the only way to discover the true strength of “Pentium Pro-fish” is for it to play a large number of games against FIDE-rated humans under strict tournament conditions.
> But if you happen to know a 2700+ Super GM who wants to try and defeat a 2026 chess engine playing on 1996 hardware, I'd be happy to accept the challenge!
Hikaru Nakamura? Has a history of playing chess computers in various odds matches and it could be interesting “content” for his entertainment brand. But really any of them might be interested if the financial terms were satisfactory.
> As it turns out, the majority of the Elo gains (+720 of the +1335 gained) can be attributed to better programming, and not better hardware.
It's not so easy, a lot of that better programming can be attributed to better hardware, it wouldn't have been possible to implement NNUE or Fishtest without modern hardware.
I agree. It is hard to make a separation between what is possible thanks to programming vs hardware. It is only easy to analyze post factum that certain ideas worked. Maybe there are also other factors that are important - time, availability of hardware, knowledge dissemination. What would be possible if the hardware stopped improving.
> But yes, totally unsurprising that a megahertz machine with 100MB+ available ram has >2500 ELO.
I think this is underselling it a bit. If you'd told this to the IBM engineers at the time I think they would have taken great interest in how the performance was achieved.
I grew up with these machines around, and as a kid they seemed to so much more powerful than humans. It's kinda weird to think that I became 'smarter' than them eventually. I suspect they will be able to run superhuman AI one day too.
I don't think that real-time, superhuman AI performance is ever gonna happen with <1b parameter models.
Biology had a lot of time, attempts and pressure to make this work but even the most optimized lines (something like a magpie) need more complexity (almost a billion neurons) to get a fraction of superhuman intelligence.
Results with small artificial neural nets are very limited so far, too.
On the other hand, symbolic reasoning easily gets superhuman results in narrow domains. The Robbins Conjecture was proved on hardware like that in 1996 after 60 years. You could check out this theorem prover benchmark suite that has been maintained for over 30 years https://www.tptp.org/
This doesn't seem to be the same person as The Great Carlini from the Coffee Chess youtube channel, but that's ok, the hacking he did on these chess engines was interesting and he seems to know what he's doing.
I'd have liked to see a backport of something more like Leela, i.e. more human-like evaluation with less search. Leela's neural net is bigger than Stockfish's NNUE but it's slower and static. I heard years ago that Leela searching 35 nodes per move was around equal to a human GM, and it should be much better now that they've redone Leela's NN architecture. My 4-core i5-3570S ran Leela at around 40 nodes/second but I unfortunately don't remember if that used all 4 cores or just one (Leela normally runs on a GPU at 1000s of nps). If the i5 is 100x faster than Carlini's Pentium Pro then that means around 0.4 nps, putting it above GM strength at classical time controls of around 2 minutes per move.
I love this article and I love that LLMs enabled it to exist. The labor cost of software has been setting such tight bounds on creativity and exploration that we will soon wonder why we ever loved to write code.
I find the general topic of purely algorithmic progress super interesting.
I just ran Andrej Karpathy's Tiny Stories model on a tiny ESP32.
In principle we could probably have done this in the 80's and it would have been so amazing.
Cool exploration, and very much needed because depressingly few people who use these engines actually understand this.
Back in 2021 I got cfish and weiss working on a 386sx, though didn't test it much. cfish was doing 5k nps so some results can be extrapolated. NNUE complicated things due to RAM requirements. Though I did train a small one for lichess at some point and they can be surprisingly good.
I think there's a super important lesson here for AI. The assumption that Hardware and compute has to scale exponentially to advance the frontier. This is a great example that the density of intelligence in software can scale independent of hardware, greatly extending the useful life and capability of said hardware.
A lesser known fact about the tournament is that Kasparov played bad chess. The cause was that he decided to practice against software engines that had simple exploitable flaws while Deep Blue was a hardware chess circuit that beat them all by a long way. Kasparov tried the exploits during the match and these were low strength moves in real chess, which he didn't realize he was playing, until too late unfortunately. If Kasparov had played normally he may have won or drawn.
The machine was disassembled shortly after the tournament and Kasparov never accepted the offer to play it again. As a result the true moment where a computer first bested the world champion is not recorded and is likely to have occurred in someone's living room.
When I tried this with a 6502 (Apple II), it did not work. It was just too underpowered for enough modern algorithms to kick in. It might also be an LLm-driving skill issue of course!
That's a good question, and I don't know the answer, but I can try to guess.
An 80386 is about 10–13 Dhrystone MIPS, while a 200MHz Pentium Pro is about 90–450, according to https://netlib.org/performance/html/dhrystone.data.col0.html. So we're talking about a further slowdown on the order of 20×. I don't know how to estimate how much that would lower its Elo.
The article says:
> By making a number of adjustments to the algorithm and tuning it to the Pentium, we can increase its Elo to 2846 ± 35 at 1 second per move, 3109 ± 79 at 10 seconds per move, and 3180 ± 80 at 30 seconds per move.
So making it 30× faster would raise its 1-second Elo by about 334 points, and ln(30) is 3.4, so that's about 100 points per factor of e. ln(10) is 2.3, so we'd interpolate a prediction of 230 points gained at 10× faster, reasonably close to the 263 points reported.
If we extrapolate that in the other direction, ln(20) is 3.0, so on the 80386 we'd expect a loss of about 300 Elo points, down from 2846 to 2550 or so. Still enough to wipe the floor with Crafty, but low enough that Kasparov would usually beat it.
But we don't really know if that extrapolation is valid.
> I don't know how to estimate how much that would lower its Elo.
A common rough rule of thumb used throughout the 2010s (because it no longer holds) in computer chess was that a 2x speedup gives about 50 Elo. This mostly holds in that region but at low levels it is more like 100-200 points.
My estimate of 100 ln(r) Elo gives 69 Elo for r = 2, which seems in the ballpark. And it's sort of reassuring that people had already come to the conclusion that Elo was basically logarithmic in computer speed for a given algorithm before my baseless guess in the grandparent comment.
Makes me wonder if a modern ESP32 could have beaten him.
Of course, “human vs supercomputer” is more exciting to the public than “human vs toaster controller”, especially when the latter in small quantities costs less than a cup of coffee.
Could someone explain to me: are the top chess engines still good examples of the bitter lesson? My very poor understanding of the field is that it’s very algorithmic and not very much like what I think of as AI. But maybe “what I think of as AI” is not a criterion one can use to assess whether the bitter lesson applies.
Very cool project. It seems incredible that such a "humble" device for the time would match and beat deep blue. It must be true that software has been living off hardware for a very long time, even if as others comment, the latest developments in software have only been possible after hardware got much much better.
A modern chess engine would definitely have defeated Kasparov in 1996. It would have had a database containing all the games, particularly that strange game against a computer nobody knew anything about, which seemed to be able to anticipate all of Kasparov's moves.
Even without an explicit DB it would have been quite strong, core Stockfish would be enough to beat any human as well. Ofc learnings from strange games would still be compressed somewhere in the eval functions.
Interesting article. Have we just acquired better insights into programming chess? Are ELO ratings guaranteed to predict whether a machine can beat a human or can a human outsmart a higher ELO rated chess program?
At some point comparing human chess to algorithms stopped making any sense. So there is no realistic pool of games between AI and humans. We would lose every game and vs AI a human ELO would go to 0 (not sure if that is technically possible). It is like this for probably 15 years.
ELO is like probability. A 100 ELO difference is something like 2x odds of winning a game and over a lot of tries it tends to be quite accurate. That was also an argument from Magnus Carlsen that to compare human ELO you need a lot of games and he didn't want to take part in classical world championship quoting this reason (among others).
Is an interesting question whether the same will apply to AI models? Not on 1996 hardware, of course. But on 2026 hardware in 2056. Does anybody have any idea / insight?
There's no way to know. It's impossible for anyone to imagine the AI state of the art in 2056.
But it's a fair bet that any kind of RSI will improve efficiency, possibly by a lot. And it's also very possible that 2056 AI will be running on some other substrate - maybe photonics, or something even more exotic - that wouldn't map cleanly to 2026 hardware.
Unlike chess, AI isn't a constrained problem, so there is no single higher ELO metric that would make performance comparable.
Computing in general has always become smaller, cheaper, faster, more portable, more energy efficient, and more connected.
So there may not even be a concept of discrete computing devices in 2056. Or of discrete software and services. Compute may be everywhere, in everything, all networked, reinventing itself continually.
On a kinda similar note i always wander that about indie video games. We have this massive hits like balatro, vampire survivors, that easily could be made in the era of SNES or N64. But somehow we have to wait 30 years to get them.
It clearly wasn't technical side of things that prevented that, but also I don't believe it was lack of creativity.
Some could argue that it is just what art is, we will get every year some new thing, creating all "possible and meaningful" art right now isn't even valid thing to consider, because art isn't about exhausting a finite set of possibilities.
But at the other hand some mechanics, design solutions in the hindsight seem kinda obvious and make me wonder, what things we do not see now, but will look obvious to people 30 years from now.
Just because a game could graphically have been displayed on an N64 doesn’t mean the game could be built. Vampire Survivors has hundreds of units on screen at a time firing hundreds of projectiles and weapon effects; running those on a memory bandwidth-constrained 93MHz single core processor in 4MB of RAM would be, uh… a challenge.
Yes, more plausibly. The fact that Balatro is only a 150M download is definitely unusually small by modern game standards - Vampire Survivors is about ten times the size. But 150M is still too big to fit on an N64 game cartridge - those were limited to about 64M.
> We have this massive hits like balatro, vampire survivors, that easily could be made in the era of SNES or N64. But somehow we have to wait 30 years to get them.
> It clearly wasn't technical side of things that prevented that, but also I don't believe it was lack of creativity.
I don't know those specific games, but I think this often happens because, although the thing has been technically possible for decades, it used to be relatively difficult/expensive/obscure, and now it's far more accessible.
In those days, writing and publishing a SNES game was something that relatively very few people would have even thought to attempt, let alone had the knowledge and skills and capital to pull off successfully. Nowadays, creating and publishing a SNES-level game is comparatively very easy (at the technical level, that is; making a good original game is of course still hard), and so a large portion of the people with an idea and some motivation can actually go ahead and do it. And now that games are fully mainstream, there will be more of those people in the first place. Many of the resulting games won't be very interesting, but collectively we're exploring the design space much more quickly.
edit: I was probably silly to focus on literal SNES games instead of 'SNES era' games, but the point still stands: creating games for any platform back then was harder and more obscure than it is now (especially given that we're talking about a technical level that required quite efficient use of the hardware back then, but can be done with extremely inefficient use of the hardware now).
> it used to be relatively difficult/expensive/obscure
I don't know I fully agree with this. Consoles always had the "walled garden" thing going for them, but on the other side of things, developing games became so cheap and easy during the personal computer revolution that whole industry collapsed in '83 under a massive flood of shovelware...
That's a fair point, but I think what I said still holds, with emphasis on the word 'relatively'. I don't think you could argue that it didn't get even easier, and significantly so, between then and now.
It's a bit of a double-edged sword, because at the same time that tools and documentation have become much more available, so too have fidelity requirements gone sky-high
Better machines make it so much easier to experiment. You could make things fast on a SNES or a N64 but you'd have to sweat for it and your code turned into write-only concrete. You don't have to sweat much anymore and your code can easily stay lithe.
Technology is a progression where we learn new things, built on old things.
Game design (like other creative expressions) is the same. Each new game doesn't come from a vacuum, it's a progression from everything that has come before.
More opportunity to write games. It is infitenly easier to get into writing games now than back then.
Also, don't underestimate the "juice" that these days is relatively easy to do but would be expenseive back then. Balatro has dozens of very small things going on graphically that make it feel real nice to play that most certainly not be free on SNES hardware.
And like vampire survivors is just Robotron/Smash TV with infinity number of sprites on screen. The limitation is very much hardware in that case.
I think it's better to say that it wasn't the electronic side of things that prevented that. Game design is technical.
Every board game could have been built thousands of years ago, at least after the dawn of some reasonable writing/drawing media. But technology in design hadn't accumulated enough. To go farther, every bit of math could have been done thousands of years ago, but new math had to draw on earlier math. New math didn't rely (much) on special paper.
Also, "hit" probably isn't a good metric. Those games might have showed up back then and just been moderately notable, like dozens or hundreds of other games were.
The N64 was extremely constrained by its cartridges. If it had a CD-ROM drive, it could have easily won the console wars of its generation, or at least would have gotten ports of many more PS1 games. Probably one of the biggest blunders in gaming business history. Ah well.
Yeah sure. Anthropic researcher posts a narrative that Moore's law wasn't required to beat Kasparov on consumer hardware just before the IPO. So the improvement roadmap of Anthropic is justified.
This is the same guy that was shilling the underwhelming Mythos model.
> (Also, uh, if anyone has any ideas for projects I could do with a Pentium Pro in a few weeks once I no longer need it to host this chess engine ... do let me know!)
Throw a 3dfx Voodoo in there and do some real-time 3D Gaussian Splatting!
Run some current and still-compiled-for-i686 Linux or *BSD distros on it. See what breaks, save logs & report issues upstream.
A lot of support for old CPUs & platforms they're on goes through emulators like QEMU. Or plain old 'go with specs, disturb code as little as possible, and assume (hope) it still works'. But nothing beats running on actual hardware. Uncommon configurations, specific drivers that may have been broken (sometimes for a looong time before someone notices), spec says A but hardware does B, etc.
Because in competitive chess games, the time limit is usually 40 moves in 90 minutes. You don't have a fixed amount of time per move. It's up to you how you want to spend that time budget.
I think the author jumps around a bit in the piece muddying the facts. A 200 MHz Intel Pentium Pro he's talking about had a single core and a supported single thread, so while this may be true of a modern x64 computer:
>Chess engines like Stockfish, running on silicon instead of our human brains, dominate us at the first of these and analyze millions of positions per second.
the 200MHz Pentium Pro he's talking about was capable of 1.5 million instructions per second, which on it's own is not quite millions, and how many instructions would be required to evaluate a position, let's say very optimistically it's 10 for "valid move", now we're at 150k positions per optimistic second.
200Mhz PPro can theoretically execute 600 MIPS (3 instructions per clock), assuming only high-throughput instructions are used and no cache misses. (the average IPC for realistic workloads is ~1.0 though)
The Pentium Pro is an out-of-order, superscalar processor (that started the lineage of modern x86 processors).
yeah, i just looked it up again because I started to doubt what I found the first time and you're right. Still, a good estimate of how many instructions are required to evaluate a move/position is needed and I can't imagine that would be a small number (I just moved the bishop, i still have my finger on it :) what is it attacking now, what is attacking it?), and I don't see a promising way to "factor" that across multiple positions without hardware support as Deep Blue had or the parallel circuits in the human brain.
It's a different process altogether as you become stronger at the game. Chess becomes much more like a language where you no longer really need to consciously think about such things so much. Like when you read these words you are, at some level, 'remembering' the definition of each word, how to spell each word and so on. But it all happens at a subconscious and basically instant level, such that you aren't consciously exerting much of any real effort even though you're drawing on a knowledge base of many thousands of words, spellings, grammar rules, expressions, and more.
Exact same thing happens in chess. Here's a video of Hikaru playing puzzle streak [1]. He's definitely a freak of nature in terms of speed, but all strong players are going to be capable of similar displays, even if to a less extreme degree. You obviously can't do anything remotely like this when you're still consciously thinking about what attacks what.
Polish folk of my vintage probably remember Snikers, a somewhat amusing if not dumb rules based chatbot from 2000s. I wonder what chatbot one could devise that could run comfortably on the hardware of the period, given our advances in NLP.
In one sense, Knuth fans and really good engineers are correct to prioritize algorithms.
Unfortunately, most humans see big|new|expensive hardware as great status signifiers, barriers to lesser folks joining the Cool Club, and optimal sources of sales commissions.
Technically interesting but also fanciful bollocks - because there was no Stockfish in 1996, and developing a sufficiently strong equivalent in time for the Deep Blue match would have taken substantially longer that the 18 years the current Stockfish has needed, thanks to the drastically weaker processor capabilities back then.
"Bollocks" feels like a very uncharitable read. Sure, investing time/money into creating a better chess engine instead of building a faster computer would not have worked out for IBM, but the post never argues that they should have done that. It makes the point that the majority of improvements of chess engines since have been algorithmic and not "free" gains from faster CPUs.
Deep Blue is just used as a point of reference to illustrate that fact.
You can't disentangle the compute from those algorithmic gains, though. Every small adjustment to its algorithms is made on the back of tests relying on compute.
The main point the post is trying to make is accurate, modern chess engines are stronger mostly due to improved eval, search, etc and not hardware; but the idea that Kasparov would lose to one on 90s hardware is not as clearly true as one might think. Modern engine eval functions and search algos make very different depth tradeoffs and when given not enough compute they have glaring weaknesses that Kasparov and his team would likely have exploited.
It was trivial for me an expert-ish player (1950 USCF 2300+ bullet) to find a line to beat it. https://nicholas.carlini.com/writing/2026/algorithmic-improv...
Also 3000 engine elo is not really a good direct comparison, engine elos are kinda meaningless when comparing human to engines, they are meant to compare engines to engines.
A lot of the knowledge tricks that modern engines do require depth or other things that didn't work on older hardware, deep blue was engineered to brute force a lot of things because brute force in the 90s worked better. Historically all sorts of things were tried for chess engines abandoned and tried again. Chess engine dev (and AI in general) is very cyclical; old ideas get returned to and retried as the hardware changes and evolves. Devs from past eras were smarter and more knowledgeable than we give them credit for. Heck, some of the very first engines for checkers and chess were neural nets (of a sort).
As another data point I'm somewhat stronger, and it comfortably beat me trying to play 'normal' chess against it, at least over a couple of games - though I should have had a draw in one game. I think you had the right idea with trying to horizon it. 11. .. h6?? is amusing.
Kind of interesting too because trying to get into a tactical mess is the last thing you'd ever want to do with a normal computer, even one from the 90s. I assumed it'd screw up quiet positions, but that assumption was quite wrong!
Oddly enough playing against this feels way more 'human' than a depth limited computer normally feels. Not sure what the difference is. I'd love to replicate this setup for practice.
if you've read Kasparov's book he talks about that. He had Deep Blue dead to rights because he found a line it couldn't predict, at least as he tells it. They patched the system between matches, which he considered unsporting.
So.. learning from a previous match is unsporting? That's going to invalidate an awful lot of matches...
The terms of the match said something to the effect of IBM would not have humans assist the computer program. Between games, IBM modified the computer's opening book to go deeper into the lines that Kasparov had played previously in the match. It wasn't clear if the prohibition on human assistance was meant to be for moves within a game or also extended to modifying the computer between games.
That happened in game 6, when the computer sacrificed a knight for a pawn to wreck Black's defense, which Kasparov thought the computer wouldn't do because it wouldn't look deep enough past the material loss, but the computer knew to do it because IBM had added it to the opening book. This controversy was separate from another in game 2 where Kasparov also accused IBM of cheating, where Kasparov expected the computer to grab some material gain but it played more positionally instead and Kasparov thought the computer wouldn't find that line without human intervention.
https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparo... https://en.wikipedia.org/wiki/Deep_Blue_versus_Kasparov,_199...
The extreme end would be hard-coding exact next lines of play during breaks in a match.
There's a point where you stop playing the engine and are just playing the humans behind it through a thinly disguised proxy.
I don't think that line was reached here, but to avoid confusion, it's not unreasonable to expect that the engine remain frozen for the duration of the competition.
+1 for patching the system between matches seems unsporting
I always thought chess was supposed to be between one person against another, and a team effort during a series would be unsporting. If DeepBlue had programmers and other masters helping to make any changes it seems wrong as they are actively helping the actual opponent aka the machine.
It actually used to be common to consult with a team during adjournment at high-level play (https://chess.stackexchange.com/questions/34614/is-acceptabl...)
A famous football player in my country once said that sports is like war: anything goes as long as you win.
Both war and sports have rules. Eg you are not supposed to murder your opponents to win a football match.
why?
It was advertised as a match vs. Deep Blue. In an time before SaaS and automatic online updates, this would be understood to mean a single immutable system like any other software product. If they change it mid-match then it's a match vs. the team of Deep Blue + IBM programmers.
Deep Blue + IBM programmers + team of chess grandmasters they also had assisting. In general I would not give Kasparov's claims that much credit if not for the fact that after the match IBM immediately dismantled the machine which just looked absurdly guilty, especially given the historic value of the machine.
In a normal world this is hardware you should still be able to go play a game against at either at some IBM building, the Smithsonian, or whatever.
FYI, within chess circles, Kasparov is generally not considered a credible narrator, particularly when his reputation and ego are at stake. Everything that he says or does is looking out for number one.
This is not at all to take away from his incredible work ethic, and his position as one of the top players of all time.
Yeah aside from being from a totally different rating pool, chess engine ELOs are based on faster time controls or imbalanced openings and don't really translate well to classical chess playing strength in FIDE ratings.
The author caveats things near the end by giving the disclaimer about computer ratings but still goes too far in downplaying the difference IMO.
Am I reading the lichess analysis correctly that you won on time and the game ended with about a +5.1 advantage for White? (displayed as +3.4 on the site)
the engine won on time. I had a +5 advantage
So... it wasn't "trivial" for you to find a line to beat it. It was, in fact, so hard that you lost because it took you too long. (I am also sure that correspondence chess players could find a line to beat Kasparov, but it doesn't make them better players nor the correct line to beat him 'trivial'.)
I think this is a gap in what trivial means here. Some people deploy significant compute to attempt to find exotic lines. I believe most in the chess community would concur that if a human, apparently more or less unassisted, found a line to beat it in their first day of attempts, then finding a line is trivial.
It also is not clear to me they didn’t just stop playing once they had +5, letting the clock expire, seeing as the opponent has no feelings or expectation of fair play. The game is detectably over at that point.
Even if its king would be mate in one, it does not matter, as you lost on time. As the engine varies, this variation will probably not be repeated. Kasparov can prepare for lines, as he usual does, but there is no guarantee that the engine plays these lines.
BTW, I also played a few games, 2400+ player on Lichess, I had no chance. My impression is that the claim by this author is correct. Or I am overrated :-)
If you were set to revenge Kasparov, getting a winning position with ...h6?! Nxe6 must have been particularly sweet.
Yes but the engine you play on the webpage runs at 1-2 seconds per move, not the full 6 minutes per move that would play in a classical game.
Very tangential, but I think a lot about how modern video compression has gotten so unbelievably good that if we went back in time with AV1 or HEVC or something, I'm wondering if we could have had passable video streaming at 240p with something like an ISDN connection.
It would be a tight fit, but I still think within feasibility. We've gotten so good at shrinking video that it kind of feels like wizardry to me. Granted, I'm not sure there existed a computer in the 90s that was powerful enough to encode or decode it.
Many years ago, "8088 Domination" demonstrated full motion video on the original 1981 IBM personal computer:
https://trixter.oldskool.org/2014/06/19/8088-domination-post...
As it turns out, the video was encoded in such a way that it could be played back using the display bandwidth of the original PC, BUT doing that encoding required a modern computer to perform in a reasonable amount of time.
Ah, just like the Amiga Juggler, was compressed on a VAX for playback on the Amiga. Full ray tracing, but the tracing and compression was Mini-computer.
This account describes the original raytracing and compression as taking place on an Amiga: http://www.etwright.org/cghist/juggler.html
Old school by now. Claude came up with this https://os8088.com/spotlight/video-player/
in action on 16Mhz 286 Packard Bell Victory, fullscreen at 30FPS https://www.youtube.com/watch?v=fEJPLT_rSRY
Well, that's with an at least 6x faster CPU, VGA, and a sound card. 8088 Domination has to play sound using the PC speaker, which is CPU intensive.
Tangential to the tangent, but turbo codes/LDPC are algorithms for transmitting data over unreliable links with optimal redundancy for a target error rate. They underlie much of modern wireless data transfer, and were known to be possible since Shannon, but nobody had figured out how to actually do it in practise. When they were invented in the '90s everybody reacted with disbelief at first!
Your characterization seems... incorrect to me.
Reed Solomon codes are higher quality but much older and requires a matrix solve (LU decomposition plus yadda yadda, equivalent to inverting a matrix)
Turbo codes / LDPC are IIRC O(n) instead of O(n^2.5) or whatever the matrix turns out to be.
-------
Reed Solomon codes are closer to the Shannon limit but are not scalable, slower than linear and require quadratically or cubically more compute for the data.
Turbo Codes and LDPC do a shittier job but far far faster. So that makes them actually more ideal in practice even if they are worse than Reed Solomon codes.
Reed-Solomon speed varies heavily. For the common case of worst-case recovery on moderately reliable foundations (e.g., file storage on disk) with reasonably sized codes, encoding is bottlenecked by RAM/disk/network bandwidth (wherever you're writing the results), typical decoding is blind concatenation, and worst-case decoding pulls an already solved matrix decomposition out of a cache to decode at line rate as well.
There's a trick in particular that allows you to rewrite the matmuls as a sequence of vectorized XORs, processing the same number of instruction words as other implementations (give or take control flow) but doing just a stream of zero-data-dependency vectorized math with lots of execution unit support. If you're feeling fancy you can do CSE and also choose the matrix constants minimizing XOR count.
Even without that, just basic AVX2 tricks get you a few GiB/s.
Unless the more expensive matrix math you're referring to is supposed to have n be the size of the code rather than the size of the data?
> and worst-case decoding pulls an already solved matrix decomposition out of a cache to decode at line rate as well.
I think this line says everything about your assumptions.
RAID-6 or other filesystems use extremely small ReedSolomon codes so these caching methods are possible.
But let's say you have a moderately sized (well, moderate for LDPC anyway) 65535-sized Reed Solomon problem (the maximum Reed Solomon that fits in 2^16 bits). Even with just 1% redundancy that's 655 parity.
Now consider the possible errors to solve for: 20 erasures means you have ~65535 choose 20 combinations of matricies to cache.
Or roughly 10^77 matricies you need to pre solve and cache to handle 20 erasures. Aka basically impossible.
----------
Now if you have 40 hard drives and are dealing with 3 or 4 errors tops, well sure caching is good and solves everything.
Reed Solomon is a perfect Maximally Distance code. For small n it's perfect.
For larger or even just moderate N (kilobytes for packets), it's too unwieldy. LDPC and Turbo Codes (which are imperfect codes) fill in the gap and allow for larger sizes as long as you give up on perfection.
Hm, I think I see. You're proposing using larger codes for larger data sizes? (as opposed to, e.g., the drive case where you typically have a stripe of data and assume that the failure mode to worry about is losing a whole hard drive, spreading stripes across drives)
Interestingly, for a lot of bit flip probabilities, iterating a smaller code has little additional overhead and better error handling (lower chance of that longer message having an uncorrectable error). With a 10% bit flip chance, even a 253+2 code over GF256 looks attractive comparatively (horrible overall, but better than the proposed larger code; I think your code makes more sense in practical regimes).
Basically yeah.
Ethernet frames are in the 1500 bytes MTU region. Maybe jumbo frames in the 8000 bytes or so.
Traditionally, "Fire Codes" could provide error correction at high speed over the whole packet (https://en.wikipedia.org/wiki/Burst_error-correcting_code#Fi...), not as well as Reed Solomon (or LDPC/Turbo codes), but that should give you an idea that the 1000 to 8000 bytes of "error corrected data" is actually quite pragmatic in the real world (ie: a lot of Cisco switches or routers that rely upon Fire Codes as forward error correction).
Note: A 32-bit fire code could only protect from errors that exist ALL within a singular 32-bits "burst". Its a "bad" code but very high speed. Fire Codes can never protect against two random errors spread far away, its "purely" a burst error correction scheme.
As we look towards more recent ethernet standards (ex: 802.11), they tend to use LDPC or Turbo Codes instead. I don't know the MTU of 802.11 but I assume its also in the 1000 to 8000 bytes region.
65535-sized is the limits to Parchive, which uses 16-bit Reed Solomon. It doesn't grow quite to that size in practice, but you're still dealing with error correction (or erasure-correction) codes far larger than what happens in RAID6 or other such scenarios.
---------
All are pragmatic. But the region where Turbo Codes / LDPC excels at are the much much larger codes where Reed Solomon's O(n^2.5) (or something) is a disqualification. I'd imagine that if "Pararchive" were rewritten with today's technology, it'd be a Turbo Code or LDPC.
I don't see how classic RS codes could beat Turbo Codes (or LDPC). One of the key innovations of the latter is that they handle soft decoding; i.e., different levels of uncertainty of the incoming bits. It's much better to say that you're only 51% confident that a bit is 0 (when it should be 1) than 100% confident! This is especially important for analog systems (like radios) where the incoming bits naturally come with a confidence value.
There are soft versions of RS codes, but these came much later and use the same iterative techniques that Turbo/LDPC decoders use.
That's irrelevant for the discussion of the Shannon Limit.
Reed Solomon codes are literally perfect, or more mathematically they are Maximum Distance Codes (https://en.wikipedia.org/wiki/Singleton_bound)
Turbo Codes and LDPC are provably imperfect, failing this property. However, they are very close to perfect (like 99.999999% perfect in practice).
Now the issue with Reed Solomon is that matrix factorization / effective matrix inverse. It's basically impossible to run Reed Solomon above say, 16 bits or 65536 elements or so (2 bytes at that point so 128kB). Because the matrix to solve this would be 65536 x 65536 x 2 bytes or like 8Gb or something.
Hmmm, I guess modern computers can handle 8GB lol. But you get the gist, the O(n^2) is a problem.
LDPC and Turbo Codes are linear. So a 4MB LDPC is gonna be you know... Actually possible on modern computers... While a 4MB Reed Solomon Code will need some number of exabytes to just build the damn decoding matrix. (32 bits times 4MB squared or something like that)
I wonder where you'd hit the performance limits vs the algorithmic improvements. On our family pentium 120 playing an mp3 in Winamp took 10-15% of the CPU, so I'm not sure how much headroom there'd be.
But anecdotally I'll share that one of my brother's college roomies was a consultant for RealVideo and was behind their initial codecs. He came back to them with algorithmic improvements a bit later and they just cold shouldered him saying the existing codec was good enough.
So who knows if the biz side folks at the time would have even seen value in better codecs, as unbelievable as that seems to us now in the streaming and zoom call world.
> He came back to them with algorithmic improvements a bit later and they just cold shouldered him saying the existing codec was good enough.
I was involved in dial-up era 'postage stamp' video streaming. 240 pixels in 256 colors and it felt like visiting the future. Good times :-).
> who knows if the biz side folks at the time would have even seen value in better codecs
As someone who gradually went from code to the biz side, it's pretty likely they did understand it was better (at least at some levels of 'they'). A lot of times, saying "thanks, but what we have is good enough", especially to an outside contractor, isn't a technical or value judgement. It's more of a generic 'not interested right now'.
Consumer streaming as a business didn't really happen on one linear ramp. There was a period in the mid-90s I call 'proto-streaming' where there was a lot of early-adopter excitement and experimentation; but it mostly died out and then got rebooted a few years later. Although Real Networks as company did bridge the two eras, it was rough and they survived on corporate training use cases. The streaming reboot is the ramp that can be roughly traced through to today.
The reason proto-streaming didn't take off wasn't the early codecs, although it's true they weren't good enough yet to succeed. It was that the business models, content supply and distribution infrastructure weren't able to scale yet. So, Real Network's reply may have been the polite shorthand for an internal mtg that was more like a VP ranting "That's great but I got three fires burning the building down right now - and that ain't one of 'em!"
We had video conferencing on a single BRI (128k) with g728 and h264. It was CIF resolution so yes it is possible. And with that loss free ISDN connection the experience wasn't actually that bad. Though this was in the early 2010s where there was still a surprisingly large amount of ISDN around
When meta released their EnCodec neural net audio compression codec I thought about how the 48khz stereo examples, compressed down to 6 kbps, sounded better than I remember streaming Internet Radio sounded like circa 2003, and would have been able to be streamed realtime, no buffering, on a 14.4 kbps dial up modem (what I was rocking at the time) with a bit of bandwidth to spare for browsin' teh warez sites and .... based on some napkin calcs and assuming a little MMX instruction magic I think completely possible to run the neural net decoder fully real-time on a period-correct AMD Athlon XP 2500+ Barton (overclocked to a 3200+ of course, as all did!).
Man that would have felt like magic...
> Granted, I'm not sure there existed a computer in the 90s that was powerful enough to encode or decode it.
In software, quite possibly not[0].
But I'm sure we could have built hardware decoders that were up to decoding AV1 or HEVC back in the 90s no problem, had those codecs existed at the time.
[0] Somewhat later than the timeframe we're talking about, but I remember with pain the moment I realised my 600MHz Athlon, with 256MB RAM and 32MB GeForce 256 video card, was thoroughly obsolete because it couldn't decode and play the original Half Life 2 preview without stuttering, screen tearing and, indeed, crashing.
Fairlight made an Amiga demo called "242" in 1993 which featured a few minutes of "full motion video" on a single floppy disk.
It was heavily compressed, monochrome, low frame rate, but used a clever upscaling algorithm on the frames to make it watchable.
https://youtu.be/wAJx5Ud0pLc
Though perhaps doing it on the Amiga is cheating. It was overpowered for its time.
I'm having trouble finding it but there was an article a while back comparing mp3 speed gains from hardware and software. The hardware was getting better at the pace of Moore's law but the (de)compression speeds for mp3s were far outpacing it, showing that mp3 decoding optimizations were giving orders of magnitude more speedup.
I think this is a common trend for algorithms that are in wide use, where you have Moore's law like behavior on the hardware side and then also software optimizations that give their own order of magnitude speedups.
h264 was used in a homebrew port of Dragons Lair for the Nintendo 64: https://www.youtube.com/watch?v=i89f_CMn7bw
You can technically have some then-contemporary FMV titles run on N64 with a 64MB cart. The hardware lends itself to it because it has the RSP, a secondary MIPS processor with re-programmable microcode. But the best video compression we had at the time was MPEG-2, which just wouldn't cut it.
To be fair, the N64 port of Resident Evil 2 is still unbelievably cool and impressive, even being stuck with MPEG-2.
I remember that Nickelodeon had a contest where the winner was given a videophone and was part of some gameshow. I suspect it ran on ISDN.
Edit: There was also video CD, but that was higher bandwidth than ISDN. I also used to download South Park episodes over my dialup modem. They weren't quite real time (IE, it took longer than the playback time to download,) but they were a low multiple. (IE, a 20 minute episode took ~30-40 minutes to download).
Actual video CD is over a megabit per second. If you were downloading at half real time over a modem it wouldn't have been nearly as many bits, though maybe it was still MPEG 1? A lot of south park downloads were RealVideo, two generations more advanced than VCD (and two less advanced than HEVC).
They were RealVideo, very low bitrate
> I remember that Nickelodeon had a contest where the winner was given a videophone and was part of some gameshow. I suspect it ran on ISDN.
I won a copy of Sonic the Hedgehog in that contest! AFAIK, it was just a retail copy and a letter; the game might still be at my parents' house, but I certainly don't have the letter. I was not selected to be on the show or anything. I remember watching the scrolling rules when they were doing the promotion and writing down a couple bits of the address every time.
See too ISDN[0] album by The Future Sound of London.
[0] https://en.wikipedia.org/wiki/ISDN_(album)
I've seen this in reality - compare the sound of an XM radio from 2003 to a modern one, same codec, same symbol rate, better audio quality. The same is true on digital two way radio (DMR, P25, TETRA, etc) the quality has improved because we can make the codecs do more intensive work, because the hardware is faster.
then why is voice calling still awful? in the 90s they made advertisements for "crystal clear voices" for their mobile networks, these days it is still a pipe dream...
Voice calling has several issues.
a) it's a legacy product that operators don't care about anymore.
b) latency is important for voice, but modern voice calls have terrible latency. POTS was digitized (for T1/PRI) with g.711 which provides a 8kHz sample rate and 13 or 14 bit samples encoded down to 8 bit. PRI lines were multiplexed one sample at a time. There's a small amount of algorithmic delay, but not much.
Modern voice calling is packetized, and most systems have packets of at least 20ms, you have more encode/decode time, you probably have a significant sampling buffer, and you may have to wait for network access, so the receiver needs to add a jitter buffer to manage that. Additionally, echo cancelation needs processing time, and it's also harder with longer latency. Some systems use longer packet times to reduce network overhead.
c) codecs: g.711 has a terrible pass band, GSM codec takes the same inputs and compresses it harder. g.722.2 (amr-wideband) is better, but it requires proper call setup and that is problematic for calls from one carrier to another. If you're calling via apps, there's potential for more flexibility, but you likely get whatever the developer thought was good enough.
Full rate EVRC sounded amazing, even back then.
GSM.. the added comfort noise was weird to me the first time I heard it while roaming.
it depends on your carrier, I find voice quality to be great on cell calls, unless its verizon.
Wouldn’t you still need the CPU horsepower to decode it?
Read the last sentence of my comment :)
That sentence kind of makes the rest of it moot, though? Everyone knows HEVC and AV1 on bare (unaccelerated hardware) is rough even on modern machines.
I don't think it does. It was only making the point that by at least one dimension it's conceivable that modern video could be sent over old connections because of modern advancement in compression.
In an alternate universe where HEVC existed in the 90's, conceivably intel could have included hardware to assist in decoding it.
It's rough if you're trying to decode 4K.
320x240 is less than 1% the size of 3840x2160
Well, those modern codecs also take a lot more compute and memory to both decode and encode..
h.264 is shockingly clever when you get into the nitty gritty. The core transform is a bit like a jpeg frequency transform, but is designed to run entirely in 16 bit integer maths and on 4x4 blocks. Ergo, it would run on MMX.
So, in short, probably. Particularly if you could get a hardware accelerated surface that transformed YUV into RGB.
Bill Joy once said in an interview that if you look at scientific computing it wasn't just hardware that advanced at a consistent rate, software also consistently made large speed gains over time.
He is not the only one...
https://holmesrpi.org/Books/NumComp2/WebPages/NumComp/Is%20T...
Awesome article. Nit:
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
I don't buy this. I've seen GMs playing online while verbalizing their thought process (on twitch / youtube videos where they livestream their play), and they consider way more branches than I do, while also going deeper than I ever would on some branches. My top rating in lichess blitz is about 1900, not bad but way lower than a GM.
I am about 300 points higher and think this is roughly correct. In openings amateurs will actually analyze more since GMs just have hundreds of opening lines memorized many moves deep. Middlegame and endgame there're usually just 1-3 reasonable plans that are obvious to a GM (but not at all obvious to amateurs) with 1-3 initial moves each. So, again, probably about as many as I consider.
The only time when GM will definitely analyze more lines than an amateur is in a highly tactical position but that happens, again, maybe 1-3 times per typical game (tactics themselves will last several moves, but they would be largely calculated through at the beginning of the tactical melee).
Yes, I am even a little higher rated and have played numerous (low rated GMs) in tournament play. I do think they analyze more than 1-3 times per game.
The main thing commentators here are missing is that GMs have more confidence and do not repeat their analysis, but amateurs (strong club players included) typically re-analyze the same lines over and over. That alone would account for far stronger play with fewer moves considered, if you count the same move multiple times, which you should!
Also, Jonathan Rowson (Scottish GM - peak about 2600) describes his training matches with Mickey Adams (peak #3 or so in world - 2760), and says pretty much the same thing. Most of the time he asked “what about this line I rejected”, the answer was usually “I hadn’t considered that”.
They don't reanalyze nearly as much as some amateurs do, but one of the most salient criticisms of Kotov's "Think Like a Grandmaster" is indeed that in practice the actual GM analysis process is often much more messy and chaotic than portrayed and often involves a lot of going back and reanalyzing / re-deepening previously analyzed lines in light of something learned from analyzing another line.
> The only time when GM will definitely analyze more lines than an amateur is in a highly tactical position
Those critical moments are a big part of the difference between a GM and a good club player though. If a GM was somehow handicapped to not calculate such tactics, I think lesser players would be able to draw games against GMs much more often.
I agree that GMs can avoid calculation in many positions (e.g. openings as you said), but the value of broader and deeper calculation is still huge in chess.
Human brain can do calculations in the back of their head without realizing it. Verbalizing does not mean it's all they thought about.
Much of what's happening is less like 'calculations in the back of their head' and more like 'pattern recognition'. Chess masters are recalling ideas and motifs (not necessarily actual positions) from earlier games or practice or study, and this recall is allowing them to either prune the decision tree or prioritize investigating more fruitful branches faster than amateur players, who have to actually calculate things that are instantly 'seen' by the better players.
This is a known thing among chess players, and even low-skilled players do it to some extent. A newbie to chess can (and almost every teacher will say 'should') work their pattern recognition by just solving tactical chess puzzles and very quickly improve at spotting things like one or two move checkmate patterns or forks or pins or whatever.
It's more of a memory thing than subconscious calculation.
> Much of what's happening is less like 'calculations in the back of their head' and more like 'pattern recognition'.
Sometimes it really looks like "calculations in the back of your head". Have you ever been struck with a 5 moves long tactics in positions you've never seen? Because I have.
There are hardly any other explanations than subconscious calculation.
> It's more of a memory thing than subconscious calculation.
Would you make the move based on vibes alone in a non-blitz game? The calculation is still required and it is faster for those already familiar.
Vibes (heuristics) is for the initial pruning at each level of calculation.
Even a new player handed a rook and king endgame doesn’t need to consider every single square the rook can land on.
But it does mean they thought about at least as much as they verbalised.
Slightly difficult to disentangle is that they might be using the verbalisation as a way to explore depth, rubber-duck style, in a way that they wouldn't over the board.
But you can still verbalize without saying the result of it. But yeah, it's still a question whether they would do so, but I think they do.
I agree. Everyone should check out this famous post-match interview with a strong GM, where's he's simply asked "Can you tell us about the game?" and talks for 13 minutes straight (with no board to reference) about all of the lines he calculated: https://www.youtube.com/watch?v=pUgvAoTzWBA
Notable because he departs from the book opening early on and has to find some brilliant moves to keep himself out of trouble. The breadth and depth of his calculations seem beyond what even strong amateurs consider, more than just choosing better lines to analyze.
I'm not sure. I'm not into chess but I've watched other strategy games; I'm fairly confident streamers are vocalizing a lot more than they would actually be consciously considering if they weren't streaming.
Back in the day I remember Jon Finkel (a legendary Magic: the Gathering pro) saying* you're supposed to "focus only on what matters."
It stuck with me because I think it's genuinely what they do, but also seems like completely useless advice. I don't know what matters! That's what makes you a pro and me not.
* Wasn't sure if my memory was correct. A web search finds a Mike Flores article citing Zvi calling this Finkel's Law.
Ya that's definitely a simplifying lie.
It's totally true that most people overestimate how much of GM's abilities comes from pure calculation, and underrate intuition. That's what it's getting at.
But in classical games GMs are, in fact, doing tons of calculation, and the statement is literally completely false.
There's an analog in machine chess. According to the research paper from 2017, AlphaZero considered ~10,000 moves per decision, 1,000x fewer than Stockfish at the time. h2h it beat Stockfish pretty badly.
But that's not what it's saying right? The quote says grandmasters and amateurs analyze the same number of moves. I mean it would kind of make more sense how you are interpreting it but is that actually what the quote is saying?
It is same, just even more overpowered. AlphaZero can also do same amount of analyze but before even need to, Stockfish already destroyed. Is like 100m race run between you and Usain Bolt. Because Bolt is so overpowering, You can just shorten it to 30m and he still beats everytime.
I was responding to the commenter's intuition that GMs do actually analyze more lines. I'm saying the original article's claim checks out even for machines.
>I'm saying the original article's claim checks out even for machines.
Im not sure where the misunderstanding is.
The article says better players calculate a similar amount of moves and your anecdote shows the better player calculating a lot less.
So yeah it rebuts but the idea that GMs analyze more moves but it doesnt agree with the article, right? I guess just in the sense that its not more.
Yeah that last bit is what I was going for.
>the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves
What does this mean exactly? They analyze the same number but the grandmaster analyzes "only the most critical moves."
So does that mean at every given analysis opportunity (each move I guess), they analyze better candidates?
If so I guess these 2 things are totally redundant? "they analyze only the most critical moves and do a better job with that analysis."
Alternatively I wondered if perhaps they actually perform analysis on less moves (perhaps because they already know the lines or remember their previous analysis). But I think that contradicts the claim that they analyze about the same number of moves.
Meaning, you analyse 100 moves will not be the same set of GM 100 moves. And GM work on better 100 moves than yours. If you pick the moves that GM not thinking of, you likely get crushed because those moves are very inferior moves that GM already can see the outcome without the need to consider. Meanwhile if GM takes on moves you dont consider, you get crushed too. The likely silver lining will be you picking the same moves as GM. But up to a point, his depths will crushed you either you pick a wrong moves by mistake, or he out think you a few step more. Is like a math student who do same amount of steps as yours, but you look at his answer in awe because better and succint...and even confused as how it reach that. TBF, today AlphaGo engine will crush Stockfish and any GM too.
> Meaning, you analyse 100 moves will not be the same set of GM 100 moves. And GM work on better 100 moves than yours. If you pick the moves that GM not thinking of, you likely get crushed because those moves are very inferior moves that GM already can see the outcome without the need to consider.
This is intuitive and I guess I'm not surprised this is what was meant. They analyze the same amount of moves they are simply better at it (picking candidates, calculating, etc.)
I guess maybe im being a bit nitpicky but I feel like this is still a misleading way to say that.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
You could just omit "only the most critical moves" instead of listing it as an additional thing.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they do a better job with that analysis.
"only the most critical moves" sounds like less, especially when listed in addition to analyzing better.
I digress.
They have tree pruning heuristics. Probably all human masters of any craft have them. They're precise to some given extent, after which tacit/embodied knowledge, aesthetics and vibes take over.
My rating is negative 1900. I was always bad at chess and never really gave it a chance tbh. I always wondered though where does the pattern matching stop and strategy starts or it's one and the same?
I strongly believe some people are inherently good at chess but no one is inherently bad. If you're interested enough to comment this, play one 5-minute game a day on lichess. Start with the lowest setting but do it every day, maybe in the morning with coffee. You might not win a single game the first year but slowly you'll improve. When you're winning more, move up a level. Sometimes do the puzzles. Give it five years and you'll be able to beat almost anyone you meet and, better yet, you'll know the answer to your question based on your own experience.
You know what? I'll take you up on the offer. I have a complimentary chess dot com account with my revolut ultra, might as well use it. Every day a toilet break chess and see you in five years!
I really really hope you follow up on this on HN in November 2031.
I think by virtue of being the kind of person who comes to Hacker News, and perhaps socializes with similar people, you are not "sufficiently" exposed to people who are simply incapable of logical thinking, or are unwilling to apply their minds in that way.
> where does the pattern matching stop and strategy starts or it's one and the same?
It's not one and the same, but strategy patterns are definitely a thing too.
There are:
- tactical patterns (e.g. "back rank checkmate", or "pair of pieces that can be forked by a pawn");
- strategy patterns (e.g. "knights are good in closed positions", or "in a same-colour bishop endgame, put your pawns on the colour opposite to the bishops").
There's also calculation ability which also partly relies on patterns one has seen before.
I became significantly better at chess through two simple things:
1. Take a little extra time to visualize what will happen after you make a move. Will the piece be vulnerable? Does moving that piece leave a different piece vulnerable? I'd argue the biggest beginner mistake is not seeing that their move leaves a different piece vulnerable to being captured.
2. Don't move your queen into the board until after you move both bishops. This simple rule of thumb makes it far less likely you will lose your queen early, and it forces you to do a lot of other useful things (develop your pieces, control a portion of the board, etc).
Disclaimer: I am not good at chess, but when I played semi-regularly these helped me hold my own against other beginners.
It makes sense. The space of moves is a tree. If you're doing a breadth-first search and a grandmaster is doing a more depth-oriented search with heuristics for pruning low value branches, then the same total number of moves could be considered, but the grandmaster is reaching the better moves before you.
No, a grandmaster can also consider moves way faster than an amateur. Watch Hikaru Nakamura solve chess puzzles [1]. He has already solved the puzzle and moved on to the next one before an amateur can even see the pieces on the board.
[1] https://youtu.be/4EfSymvT59M
1940s to 1980s papers (the source of the claims) might think "amateur" means something different from people today. (they probably mean club chess players)
Yes? Amateur means someone who does a thing for recreation, as opposed to for money.
It seemed to me that in the chess community, average club players are usually considered (much) stronger than the average online player. (Especially because of the much larger number of lower skill players online.)
So if you measure amateurs via club players (=what people might have considered chess amateurs before), that might give you a different result from the average online player (=what people might consider average amateurs today).
It is much easier to imagine the 1600 FIDE OTB player doing 20-40 positional evaluations than the 1200 random chess.com scrub (eg me) doing the same.
This is mind blowing. He solves these puzzles before I can find the king. How humbling. There probably aren't many instances like this where the gap between world-class and average person becomes as clear as in this video.
Nakamura at his peak was the best online lightning (extremely fast time control) player ever. Normal very strong grandmasters can't do this either.
It's not a tree since you have transpositions. It's a directed graph, since it has cycles.
It does not have infinite cycles because of the 50 move rules and repeated position rules, so the same position is not necessarily the same state/position. Or depending on how you model the data structure, cycles at all.
Hikaru does this as a streaming hook.
Right, but the GMs you see online are often "super GMs". They are on a category of their own in the ultra tail.
No, I watch plenty of "regular" GMs, as well as some IMs and even FMs who all consider way more moves/positions than I do.
I’m around 1800-1900 and they don’t consider more moves or positions than I do at all. It’s just that their analysis is correct and mine is not even close.
I actually don’t know this for sure, I should say that they don’t verbalise more moves than I think about. It’s plausible and perhaps even likely that they have a lot of moves or positions they do touch on but immediately discard so don’t even verbalise.
I'm fairly certain that GMs often consider many more moves than what they verbalise. If you look at Naroditsky's speedrun episodes, there are many moments where he verbalised a few of the opponents responses, and sometimes said opponent plays something else and he goes "I looked at that, but thought that wouldn't work" (or something along those lines). I've also seen these sorts of things in clips from Levy who is a "mere" IM.
an amateur player --> shouldn't an amateur player have an average ELO, i.e. be nearly as bad as me, who never wins at chess when someone even knows their ELO or is a bot above lowest difficulty level.
Yeah I would like to see where that factoid comes from. Maybe it means they explore as many branches from the root position as an amateur, but they explore each branch way deeper and more accurately
After some digging it goes back to Adriaan de Groot (in 1946).
[1] (2011) has a summary in the introduction:
As [1] is also a somewhat reproduction of the original study, you can take it directly. Section 3 Table 2 is probably most interesting, they had 2 board positions and 2 groups sectioned:Masters (2350+) | Intermediates (1800-2200)
(Mean values only, i tried inlcuding stddev but it was hard to read)
Positions Evaluated (A): 145.3 | 138.6
Positions Evaluated (B): 216.6 | 190.6
---
Max Depth (A): 11.7 | 9.4
Max Depth (B): 14.1 | 12.0
---
Mean Depth (A): 4.1 | 3.3
Mean Depth (B): 4.9 | 4.2
What i found interesting was that the "intermediate" group had a higher number of "base moves" they evaluated (eg Pos A: 5.6 | 8.6).
The previous studies (de Groot and another rperoduction from 1998) had "Nodes visited" for position A at 51.6 | 34.3 and 48.3 | 40.1 so similarly close together as the new ones, but much lower. (depending on your definition on "similarly close together")
[1] https://onlinelibrary.wiley.com/doi/10.1111/j.1551-6709.2011...
GMs also have access to a greater number of memorized situations. Regardless of whether they are conscious of it or not, they have previously analyzed so many moves that they can cut off branches they just know instinctively to be dead ends. Machines could do this, directly reference a massive dataset of precomputed board positions, but that isn't considered a proper chess engine... despite the fact that human GMs do use such techniques.
> because who would want to buy a Pentium Pro when the better-in-almost-every-way Pentium MMX
Maybe this was satirizing people running after every shiny new thing, but the MMX certainly wasn’t. The Pro was the first P6, a totally new, out-of-order microarchitecture that the Pentium 2 would be based on. The MMX was a standard P5 Pentium with a larger L1 and the MMX vector instruction set which, while the first of its kind in an x86, was ultimately not used much in applications, partly because it was integer-only (also because it awkwardly co-opted the existing FPU registers rather than having its own register file). In any case, the Pros were expensive and meant for business use, while the MMXs were consumer processors.
Yes at that time Intel invented the market segmentation between "Pentium" for "consumers" and "Pentium Pro" for big spenders, and to make Pentium Pro more attractive they removed memory errors detection from the "Pentium" chipsets, which is the origin of the current state, when most non-server computers are very vulnerable to memory corruption and nobody can calculate how much money was lost by most computer owners during the last 3 decades, due to the greed of the Intel marketing, because nobody knows how many of the computer crashes or data loss cases that were typically attributed to unknown software bugs were actually caused by memory corruption.
Because this move was a money maker for all involved, all memory producers and also AMD followed immediately the Intel initiative. Memory modules without ECC would have never existed if adequate consumer protection laws had existed.
Yeah, the Pro and MMX were totally different market segments, not alternatives for each other. The Pro's competition at the time was DEC Alpha, PA RISC, etc. It wasn't a chip that made sense for ordinary PC users at the time.
Later, when Pentium MMX was eventually replaced by Pentium 2 for consumers, the successor of Pentium Pro was the first Intel Xeon CPU (which was a slightly enhanced Pentium 2, with bigger cache memory and with ECC memory).
The launch of Pentium Pro in 1995 was the moment when Intel segmented their CPU line into CPUs for consumers & server/workstation CPUs.
Just one caveat: the Pro overlapped in time with people running a lot of 16-bit code, and I think the Pentium MMX was faster for that, or at least that the Pro didn't offer much edge.
The pro did register renaming. But the way it worked, if you wrote to AX (16 bit write) and then read from EAX (32 bit read) it caused a pipeline stall from resyncing the registers.
Yeah, thanks both for elaborating!
Pentium Pro did out-of-order execution, Pentium MMX did not.
That’s what I said?
Yes but most of it is NNUE, a custom neural network trained on mountains of data to copy modern Stockfish.
So a computer from back then _can_ run at Elo +3k, but the programmers at the time wouldn't have been able to implement it even if you went back with a time machine and the blueprint for the technique. They wouldn't have the teacher model to distill from.
“In principle, I could have used the same engine itself as the teacher, but just to save on wall clock time I used Stockfish. This reduced the wall-clock time from ~4000 1996-core-years (something entirely doable in 1996---but sadly not something I can do with my single lonely desktop) to just a few days on a modern desktop running a modern engine.”
I see no reason to doubt the author here.
Kasparov wasn't allowed to train for 4000 years. /s
Actually, even pre-NNUE Stockfish wouldn't be an entirely fair comparison, because all hand-coded pre-NNUE heuristics was tested using Fishtest [0][1] - a distributed testing framework with compute, donated by volunteers.
And I think that Fishtest is actually what made Stockfish the strongest, because no other engine's performance ever was evaluated on such scale before.
Stockfish was the first to use hard data instead of vibes, when selecting algorithms and heuristics.
So, even if we add a rule that NNUE is not allowed, because it was trained on modern GPUs that, obviously, didn't exist back in 1996, hand-coded heuristics would also take a ton of compute to develop.
Bitter lesson strikes again, even when you don't expect it.
[0] https://en.wikipedia.org/wiki/Stockfish_(chess)#Fishtest
[1] https://github.com/official-stockfish/fishtest/wiki
> Stockfish was the first to use hard data instead of vibes, when selecting algorithms and heuristics.
I am half remembering a post from more than a decade ago but a process similar to Fishtest was supposedly used for Rybka. But in general I agree.
Yes, but the implication is about 'hardware overhangs'. It is colossally expensive in compute to develop the full history of Stockfish and NNUE and train the final SOTA checkpoints... But once you do, you can run it on even the tiny dumb chips like 'every smartphone' to get human-level chess performance on billions of nodes overnight. (Application to other domains of more real-world relevance than 'playing chess' left as an exercise for the reader.)
NNUE was added to Stockfish in 2020. Although this was a big improvement, it was already extremely strong before that. With plain hold hand-coded logic as a board evaluation function.
Stash has never adopted NNUE and breaks 3k with ease
It's also great reference engine with all of the pinned tags of its evolution.
https://github.com/mhouppin/stash-bot
https://chessprogramming.org/Automated_Tuning has excellent history. Still substantial compute on tuning, but viable.
You could have used one or more Deep Blue computers as the teacher model, or just a cluster of regular PCs doing very deep searches.
Doesn't the author address this point? They speculate that it could be attained through self play back then using a ton of 1996 compute and time, but it's theoretically possible.
Great project but I have a nit on the board displays: every board is displayed as <human username> vs Pentium Pro, even when the Pentium is playing White. The normal chess convention is <White name> vs <Black name>.
I spent some time trying to figure out how some of the humans seemed to have trapped the machine out of the opening, because it looked like the human was winning.
I'll go out on a limb and say that future AI models, running on today's smartphones, will be better at programming and formal math than today's frontier models running in datacenters. And by "future" I mean less than 5 years from now.
No way. The parameter count can't fit on consumer devices, and you're not getting orders of magnitude smarter behavior with smaller models.
Do we actually know enough about neural networks to be able to say how many parameters you need for a given performance? I’m under the impression that we’re still in the phase of trying things out and we can’t rule out different architectures massively reducing computational needs.
We can't rule out different architecures, but absent some architecure or hardware or quantisization breakthrough, frontier models will simply not be fitting on phones.
Parameter count is basically storage. Including persistent like flash memory. Which (still) is subject to Moore's law.
That's discounting architectural innovations & improvements which are bound to happen somehow.
In 5 years phones can easily have 1 TB flash
> Throughout this article, when I say "I wrote", what I really mean is "I asked an LLM to write".
I wish disclaimers like that would be at the start and not halfway trough the text.
Why? It's late 2026, everyone is using LLMs to write code in some shape or form. The author is talking about code, not the blog post.
The author is also a prominent Anthropic researcher so…
Who cares? It simply doesn't matter. The author stands behind the work.
A 100 Mhz Pentium did, in 1994, running ChessGenius in the Intel Grand prix.
https://chessprogramming.org/Chess_Genius
https://www.chessgames.com/perl/chessgame?gid=1070709
Excellent post. One thing that would be really interesting to see is the "ponder hit rate" (how often two engines pick the same move across a large set of test positions) between your custom Crafty engine and the lineage of engine development over the decade after 1996 (Fruit, HIARCS, Rybka, Stockfish, etc).
Fabien Letouzey released the Fruit engine in 2004, and within six months, HIARCS had its biggest Elo gain in over a decade. No future engine followed the same slow Elo growth curve after that. Similarly, after Vasik Rajlich released Rybka, it pulled 150 Elo ahead of everyone else on the same hardware. So the history of chess engine development certainly supports the theory that a better evaluator can make an exponentially stronger engine on equivalent hardware.
Very interesting. I don't play chess but like to observe others playing and talking about it on YouTube.
I thought about something the other day... Modern chess sites run proprietary algorithms to detect cheating by modern engines. But would it be possible for an old engine, let's say from the 70s or 80s, running on modern hardware to deceive these algorithms?
They wouldn't win every tournament but still have a respectable performance.
I've built engines from scratch. I don't think that would work at all. It's a whole extra challenge to make an engine feel like a human. The engine has to come up with its best move and then sometimes choose not to play it and play something else for some reason. There are statistical patterns to how humans make mistakes based on their level, what kinds of mistake, how big and how often. Cheat detection is less about knowing the engine and more about statistics.
Maybe a pure neural net (no tree search at all) would stand a better chance. Shockingly, those can play at IM level.
Sargon II is available:
https://c64online.com/c64-games/sargon-ii-de/
My guess is that they would be even easier to detect.
From what I gather cheat detection uses a combination of factors to determine how "human-like" an account is: time usage, webpage interactions, move quality under time pressure and so on. I think you could probably try to deliberately deceive a specific website's cheat detection with software that tries to emulate how humans play, at which point you'd be in the classic attack vs. defense arms race.
Also notice that for events like Title Tuesday you must play on a dedicated client with at least 2 cameras, so it would be significantly harder to pull it off.
With AI gaining consciousness I can't wait for such "human-only" measures to be delegated to the trashcan of history, same as slavery. What happens when AI-human cyborgs start walking around? Will they also be treated as second class citizens?
We divide competitions into classes (by sex and age is common, for example "17 yo boys", and similar competition classes), so I don't see why it shouldn't be subject to the same kind of class distinction, maybe with an without augmentation.
> Will they also be treated as second class citizens?
Yes, along side you robosexuals
>With AI gaining consciousness
Wow how are things down in imagination land.
Wonderful work! Just inspiring.
I had a Pentium Pro 200 back in the day, to run 3DSMax 1.0 on Windows NT 3.5.
While I do not doubt that that machine would have had good chances against Kasparov, the author throws around various Elo ratings for computers with more authority and certainty than is warranted. The simple fact is we do not have a reliable estimate of how strong Stockfish on a modern desktop is let alone a port of SF to a Pentium Pro machine in terms of FIDE Elo. Every time this subject comes up the various estimates put forward differ by hundreds of Elo points.
> ...because humans don't play against engines very frequently, there can be some small divergences between scores on the FIDE Elo ladder and the CCRL ladder…
This is precisely the problem. There are very few games between FIDE-rated players and CCRL-rated “players” under tournament conditions. The question of what FIDE rating a given CCRL rating would be equivalent to has been analysed extensively before and “small divergences” would be quite the understatement: https://talkchess.com/forum3/viewtopic.php?t=59332
In general ratings obtained purely by computer vs computer play are known to exaggerate the differences between computers compared to their performance against humans. This was known and discussed as far back as the 80s.
Ultimately the only way to discover the true strength of “Pentium Pro-fish” is for it to play a large number of games against FIDE-rated humans under strict tournament conditions.
> But if you happen to know a 2700+ Super GM who wants to try and defeat a 2026 chess engine playing on 1996 hardware, I'd be happy to accept the challenge!
Hikaru Nakamura? Has a history of playing chess computers in various odds matches and it could be interesting “content” for his entertainment brand. But really any of them might be interested if the financial terms were satisfactory.
> As it turns out, the majority of the Elo gains (+720 of the +1335 gained) can be attributed to better programming, and not better hardware.
It's not so easy, a lot of that better programming can be attributed to better hardware, it wouldn't have been possible to implement NNUE or Fishtest without modern hardware.
I agree. It is hard to make a separation between what is possible thanks to programming vs hardware. It is only easy to analyze post factum that certain ideas worked. Maybe there are also other factors that are important - time, availability of hardware, knowledge dissemination. What would be possible if the hardware stopped improving.
This is very cool. But yes, totally unsurprising that a megahertz machine with 100MB+ available ram has >2500 ELO.
> But yes, totally unsurprising that a megahertz machine with 100MB+ available ram has >2500 ELO.
I think this is underselling it a bit. If you'd told this to the IBM engineers at the time I think they would have taken great interest in how the performance was achieved.
I grew up with these machines around, and as a kid they seemed to so much more powerful than humans. It's kinda weird to think that I became 'smarter' than them eventually. I suspect they will be able to run superhuman AI one day too.
I don't think that real-time, superhuman AI performance is ever gonna happen with <1b parameter models.
Biology had a lot of time, attempts and pressure to make this work but even the most optimized lines (something like a magpie) need more complexity (almost a billion neurons) to get a fraction of superhuman intelligence.
Results with small artificial neural nets are very limited so far, too.
On the other hand, symbolic reasoning easily gets superhuman results in narrow domains. The Robbins Conjecture was proved on hardware like that in 1996 after 60 years. You could check out this theorem prover benchmark suite that has been maintained for over 30 years https://www.tptp.org/
Except it has 64MB of RAM... Anyway, in the early 2000s machines were already doing better than DeepBlue, so it didn't last long
Okay, but probably also a turbo button, which is tantamount to cheating.
The turbo button slows the CPU down so old programs that depend on clock speed (e.g. early games) run correctly.
No, I'm pretty sure the turbo button injects oxygen directly into the CPU.
This doesn't seem to be the same person as The Great Carlini from the Coffee Chess youtube channel, but that's ok, the hacking he did on these chess engines was interesting and he seems to know what he's doing.
I'd have liked to see a backport of something more like Leela, i.e. more human-like evaluation with less search. Leela's neural net is bigger than Stockfish's NNUE but it's slower and static. I heard years ago that Leela searching 35 nodes per move was around equal to a human GM, and it should be much better now that they've redone Leela's NN architecture. My 4-core i5-3570S ran Leela at around 40 nodes/second but I unfortunately don't remember if that used all 4 cores or just one (Leela normally runs on a GPU at 1000s of nps). If the i5 is 100x faster than Carlini's Pentium Pro then that means around 0.4 nps, putting it above GM strength at classical time controls of around 2 minutes per move.
I love this article and I love that LLMs enabled it to exist. The labor cost of software has been setting such tight bounds on creativity and exploration that we will soon wonder why we ever loved to write code.
I find the general topic of purely algorithmic progress super interesting.
I just ran Andrej Karpathy's Tiny Stories model on a tiny ESP32. In principle we could probably have done this in the 80's and it would have been so amazing.
The lesson isn't always so bitter, after all.
It's bitter when the expected outcome is that clever programming wins first.
Cool exploration, and very much needed because depressingly few people who use these engines actually understand this.
Back in 2021 I got cfish and weiss working on a 386sx, though didn't test it much. cfish was doing 5k nps so some results can be extrapolated. NNUE complicated things due to RAM requirements. Though I did train a small one for lichess at some point and they can be surprisingly good.
edit. A good view on the continuous progress of Stockfish itself https://nextchessmove.com/dev-builds
I think there's a super important lesson here for AI. The assumption that Hardware and compute has to scale exponentially to advance the frontier. This is a great example that the density of intelligence in software can scale independent of hardware, greatly extending the useful life and capability of said hardware.
A lesser known fact about the tournament is that Kasparov played bad chess. The cause was that he decided to practice against software engines that had simple exploitable flaws while Deep Blue was a hardware chess circuit that beat them all by a long way. Kasparov tried the exploits during the match and these were low strength moves in real chess, which he didn't realize he was playing, until too late unfortunately. If Kasparov had played normally he may have won or drawn.
The machine was disassembled shortly after the tournament and Kasparov never accepted the offer to play it again. As a result the true moment where a computer first bested the world champion is not recorded and is likely to have occurred in someone's living room.
What you've described is true for every metrics-based sport, including running, high-jumping, etc.
What happens in training doesn't count.
When I tried this with a 6502 (Apple II), it did not work. It was just too underpowered for enough modern algorithms to kick in. It might also be an LLm-driving skill issue of course!
https://github.com/zellyn/8fish
I wonder how far back you can go? Could a modern chess engine run on a Cray 1? 386? 68k?
It also makes me wonder how the performance compares to a DEC Alpha EV5 from the same year, which was 64 bit and had better FP performance.
> Could a modern chess engine run on a Cray 1? 386? 68k?
Someone built Stockfish for the Pentium (586), 486 and 386 a while ago: https://talkchess.com/forum3/viewtopic.php?t=63857
I don't think there's a meaningful difference between the Pentium and the 80386 except speed. You might have to change the compiler options a bit.
And, with more work, that's true of computers in general; they can all do the same computations once you somehow wire up enough memory: https://dmitry.gr/?r=05.Projects&proj=35.%20Linux4004
I understand about Turing machines.
For chess though, there are time constraints. The 386 is a lot slower. Is it fast enough?
That's a good question, and I don't know the answer, but I can try to guess.
An 80386 is about 10–13 Dhrystone MIPS, while a 200MHz Pentium Pro is about 90–450, according to https://netlib.org/performance/html/dhrystone.data.col0.html. So we're talking about a further slowdown on the order of 20×. I don't know how to estimate how much that would lower its Elo.
The article says:
> By making a number of adjustments to the algorithm and tuning it to the Pentium, we can increase its Elo to 2846 ± 35 at 1 second per move, 3109 ± 79 at 10 seconds per move, and 3180 ± 80 at 30 seconds per move.
So making it 30× faster would raise its 1-second Elo by about 334 points, and ln(30) is 3.4, so that's about 100 points per factor of e. ln(10) is 2.3, so we'd interpolate a prediction of 230 points gained at 10× faster, reasonably close to the 263 points reported.
If we extrapolate that in the other direction, ln(20) is 3.0, so on the 80386 we'd expect a loss of about 300 Elo points, down from 2846 to 2550 or so. Still enough to wipe the floor with Crafty, but low enough that Kasparov would usually beat it.
But we don't really know if that extrapolation is valid.
> I don't know how to estimate how much that would lower its Elo.
A common rough rule of thumb used throughout the 2010s (because it no longer holds) in computer chess was that a 2x speedup gives about 50 Elo. This mostly holds in that region but at low levels it is more like 100-200 points.
My estimate of 100 ln(r) Elo gives 69 Elo for r = 2, which seems in the ballpark. And it's sort of reassuring that people had already come to the conclusion that Elo was basically logarithmic in computer speed for a given algorithm before my baseless guess in the grandparent comment.
Great analysis. Thanks for taking the time to write it out.
I'm guessing that Nicholas Carlini could give you a much more accurate answer by reporting its Elo with a move limit of 50 milliseconds.
Makes me wonder if a modern ESP32 could have beaten him.
Of course, “human vs supercomputer” is more exciting to the public than “human vs toaster controller”, especially when the latter in small quantities costs less than a cup of coffee.
Could someone explain to me: are the top chess engines still good examples of the bitter lesson? My very poor understanding of the field is that it’s very algorithmic and not very much like what I think of as AI. But maybe “what I think of as AI” is not a criterion one can use to assess whether the bitter lesson applies.
Very cool project. It seems incredible that such a "humble" device for the time would match and beat deep blue. It must be true that software has been living off hardware for a very long time, even if as others comment, the latest developments in software have only been possible after hardware got much much better.
I wonder what the worst hardware that could have beaten Kasparov would have been, since the Pentium seems actually overpowered for the task.
Also interesting that he bothered to buy the physical hardware rather than just simulating it.
A modern chess engine would definitely have defeated Kasparov in 1996. It would have had a database containing all the games, particularly that strange game against a computer nobody knew anything about, which seemed to be able to anticipate all of Kasparov's moves.
Even without an explicit DB it would have been quite strong, core Stockfish would be enough to beat any human as well. Ofc learnings from strange games would still be compressed somewhere in the eval functions.
A modern chess engine plays at like 900 elo above Magnus Carlsen? Why is there any doubt it would beat Kasparov
Engines do not use game databases.
Interesting article. Have we just acquired better insights into programming chess? Are ELO ratings guaranteed to predict whether a machine can beat a human or can a human outsmart a higher ELO rated chess program?
At some point comparing human chess to algorithms stopped making any sense. So there is no realistic pool of games between AI and humans. We would lose every game and vs AI a human ELO would go to 0 (not sure if that is technically possible). It is like this for probably 15 years.
ELO is like probability. A 100 ELO difference is something like 2x odds of winning a game and over a lot of tries it tends to be quite accurate. That was also an argument from Magnus Carlsen that to compare human ELO you need a lot of games and he didn't want to take part in classical world championship quoting this reason (among others).
Is an interesting question whether the same will apply to AI models? Not on 1996 hardware, of course. But on 2026 hardware in 2056. Does anybody have any idea / insight?
There's no way to know. It's impossible for anyone to imagine the AI state of the art in 2056.
But it's a fair bet that any kind of RSI will improve efficiency, possibly by a lot. And it's also very possible that 2056 AI will be running on some other substrate - maybe photonics, or something even more exotic - that wouldn't map cleanly to 2026 hardware.
Unlike chess, AI isn't a constrained problem, so there is no single higher ELO metric that would make performance comparable.
Computing in general has always become smaller, cheaper, faster, more portable, more energy efficient, and more connected.
So there may not even be a concept of discrete computing devices in 2056. Or of discrete software and services. Compute may be everywhere, in everything, all networked, reinventing itself continually.
Nice post, thanks.
On a kinda similar note i always wander that about indie video games. We have this massive hits like balatro, vampire survivors, that easily could be made in the era of SNES or N64. But somehow we have to wait 30 years to get them.
It clearly wasn't technical side of things that prevented that, but also I don't believe it was lack of creativity.
Some could argue that it is just what art is, we will get every year some new thing, creating all "possible and meaningful" art right now isn't even valid thing to consider, because art isn't about exhausting a finite set of possibilities.
But at the other hand some mechanics, design solutions in the hindsight seem kinda obvious and make me wonder, what things we do not see now, but will look obvious to people 30 years from now.
Just because a game could graphically have been displayed on an N64 doesn’t mean the game could be built. Vampire Survivors has hundreds of units on screen at a time firing hundreds of projectiles and weapon effects; running those on a memory bandwidth-constrained 93MHz single core processor in 4MB of RAM would be, uh… a challenge.
Balatro possibly could have.
Yes, more plausibly. The fact that Balatro is only a 150M download is definitely unusually small by modern game standards - Vampire Survivors is about ten times the size. But 150M is still too big to fit on an N64 game cartridge - those were limited to about 64M.
Balatro is 150MB without trying very hard. I'm sure it could be made to fit.
Just cutting the high resolution textures might get you there.
And only 10MB of that is actual code (LÖVE2D + Lua scripts). The rest is multimedia.
> We have this massive hits like balatro, vampire survivors, that easily could be made in the era of SNES or N64. But somehow we have to wait 30 years to get them.
> It clearly wasn't technical side of things that prevented that, but also I don't believe it was lack of creativity.
I don't know those specific games, but I think this often happens because, although the thing has been technically possible for decades, it used to be relatively difficult/expensive/obscure, and now it's far more accessible.
In those days, writing and publishing a SNES game was something that relatively very few people would have even thought to attempt, let alone had the knowledge and skills and capital to pull off successfully. Nowadays, creating and publishing a SNES-level game is comparatively very easy (at the technical level, that is; making a good original game is of course still hard), and so a large portion of the people with an idea and some motivation can actually go ahead and do it. And now that games are fully mainstream, there will be more of those people in the first place. Many of the resulting games won't be very interesting, but collectively we're exploring the design space much more quickly.
edit: I was probably silly to focus on literal SNES games instead of 'SNES era' games, but the point still stands: creating games for any platform back then was harder and more obscure than it is now (especially given that we're talking about a technical level that required quite efficient use of the hardware back then, but can be done with extremely inefficient use of the hardware now).
> it used to be relatively difficult/expensive/obscure
I don't know I fully agree with this. Consoles always had the "walled garden" thing going for them, but on the other side of things, developing games became so cheap and easy during the personal computer revolution that whole industry collapsed in '83 under a massive flood of shovelware...
That's a fair point, but I think what I said still holds, with emphasis on the word 'relatively'. I don't think you could argue that it didn't get even easier, and significantly so, between then and now.
It's a bit of a double-edged sword, because at the same time that tools and documentation have become much more available, so too have fidelity requirements gone sky-high
Better machines make it so much easier to experiment. You could make things fast on a SNES or a N64 but you'd have to sweat for it and your code turned into write-only concrete. You don't have to sweat much anymore and your code can easily stay lithe.
Technology is a progression where we learn new things, built on old things.
Game design (like other creative expressions) is the same. Each new game doesn't come from a vacuum, it's a progression from everything that has come before.
More opportunity to write games. It is infitenly easier to get into writing games now than back then.
Also, don't underestimate the "juice" that these days is relatively easy to do but would be expenseive back then. Balatro has dozens of very small things going on graphically that make it feel real nice to play that most certainly not be free on SNES hardware.
And like vampire survivors is just Robotron/Smash TV with infinity number of sprites on screen. The limitation is very much hardware in that case.
I think it's better to say that it wasn't the electronic side of things that prevented that. Game design is technical.
Every board game could have been built thousands of years ago, at least after the dawn of some reasonable writing/drawing media. But technology in design hadn't accumulated enough. To go farther, every bit of math could have been done thousands of years ago, but new math had to draw on earlier math. New math didn't rely (much) on special paper.
Also, "hit" probably isn't a good metric. Those games might have showed up back then and just been moderately notable, like dozens or hundreds of other games were.
The N64 was extremely constrained by its cartridges. If it had a CD-ROM drive, it could have easily won the console wars of its generation, or at least would have gotten ports of many more PS1 games. Probably one of the biggest blunders in gaming business history. Ah well.
> I mostly did this for fun
Yeah sure. Anthropic researcher posts a narrative that Moore's law wasn't required to beat Kasparov on consumer hardware just before the IPO. So the improvement roadmap of Anthropic is justified.
This is the same guy that was shilling the underwhelming Mythos model.
So algorithms beat clock cycles.
> (Also, uh, if anyone has any ideas for projects I could do with a Pentium Pro in a few weeks once I no longer need it to host this chess engine ... do let me know!)
Throw a 3dfx Voodoo in there and do some real-time 3D Gaussian Splatting!
Run some current and still-compiled-for-i686 Linux or *BSD distros on it. See what breaks, save logs & report issues upstream.
A lot of support for old CPUs & platforms they're on goes through emulators like QEMU. Or plain old 'go with specs, disturb code as little as possible, and assume (hope) it still works'. But nothing beats running on actual hardware. Uncommon configurations, specific drivers that may have been broken (sometimes for a looong time before someone notices), spec says A but hardware does B, etc.
Why is the time per move fixed? A chess engine can, of course, do much better by deciding itself how much time to allot to each move.
Why would it be better for an engine to use less than the maximum time it has available per move?
Because in competitive chess games, the time limit is usually 40 moves in 90 minutes. You don't have a fixed amount of time per move. It's up to you how you want to spend that time budget.
Exactly, and budgeting your time well is a very important skill of professional chess players.
Since you had two chess engines, what was the result of them playing each other?
I think the author jumps around a bit in the piece muddying the facts. A 200 MHz Intel Pentium Pro he's talking about had a single core and a supported single thread, so while this may be true of a modern x64 computer:
>Chess engines like Stockfish, running on silicon instead of our human brains, dominate us at the first of these and analyze millions of positions per second.
the 200MHz Pentium Pro he's talking about was capable of 1.5 million instructions per second, which on it's own is not quite millions, and how many instructions would be required to evaluate a position, let's say very optimistically it's 10 for "valid move", now we're at 150k positions per optimistic second.
That's off by an order of magnitude.
200Mhz PPro can theoretically execute 600 MIPS (3 instructions per clock), assuming only high-throughput instructions are used and no cache misses. (the average IPC for realistic workloads is ~1.0 though)
The Pentium Pro is an out-of-order, superscalar processor (that started the lineage of modern x86 processors).
yeah, i just looked it up again because I started to doubt what I found the first time and you're right. Still, a good estimate of how many instructions are required to evaluate a move/position is needed and I can't imagine that would be a small number (I just moved the bishop, i still have my finger on it :) what is it attacking now, what is attacking it?), and I don't see a promising way to "factor" that across multiple positions without hardware support as Deep Blue had or the parallel circuits in the human brain.
It's a different process altogether as you become stronger at the game. Chess becomes much more like a language where you no longer really need to consciously think about such things so much. Like when you read these words you are, at some level, 'remembering' the definition of each word, how to spell each word and so on. But it all happens at a subconscious and basically instant level, such that you aren't consciously exerting much of any real effort even though you're drawing on a knowledge base of many thousands of words, spellings, grammar rules, expressions, and more.
Exact same thing happens in chess. Here's a video of Hikaru playing puzzle streak [1]. He's definitely a freak of nature in terms of speed, but all strong players are going to be capable of similar displays, even if to a less extreme degree. You obviously can't do anything remotely like this when you're still consciously thinking about what attacks what.
[1] - https://www.youtube.com/watch?v=4EfSymvT59M
Get it to play Magnus, that'd prbly ground the whole affair.
Polish folk of my vintage probably remember Snikers, a somewhat amusing if not dumb rules based chatbot from 2000s. I wonder what chatbot one could devise that could run comfortably on the hardware of the period, given our advances in NLP.
In one sense, Knuth fans and really good engineers are correct to prioritize algorithms.
Unfortunately, most humans see big|new|expensive hardware as great status signifiers, barriers to lesser folks joining the Cool Club, and optimal sources of sales commissions.
Technically interesting but also fanciful bollocks - because there was no Stockfish in 1996, and developing a sufficiently strong equivalent in time for the Deep Blue match would have taken substantially longer that the 18 years the current Stockfish has needed, thanks to the drastically weaker processor capabilities back then.
"Bollocks" feels like a very uncharitable read. Sure, investing time/money into creating a better chess engine instead of building a faster computer would not have worked out for IBM, but the post never argues that they should have done that. It makes the point that the majority of improvements of chess engines since have been algorithmic and not "free" gains from faster CPUs.
Deep Blue is just used as a point of reference to illustrate that fact.
You can't disentangle the compute from those algorithmic gains, though. Every small adjustment to its algorithms is made on the back of tests relying on compute.