I have some sympathy for these kids. If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well; many of them can no longer brainstorm, code, think deeply, or write without an LLM present doing 90% of the work. Many of them can no longer sit quietly for even 30 minutes just thinking on their own, which is a required skill for producing original thought.
For adults the cognitive decline won't be as measurable since there's no exams, and overall output volume will still be fine due to LLM help. But I do believe it's already happening absolutely everywhere around us. Honestly, I wanted to be in denial about it before but it's too obvious to ignore now.
I’m not noticing the decline in my own abilities any more than I had before using them. I finished undergrad 20 years ago and my once sharp math skills had been severely diminished within only 5-10 years. Just simple arithmetic and percentages that I could rapidly do in my head became dependent on calculators/spreadsheets. For all other trivia type knowledge, my brain has offloaded it to the internet RAM in my pocket. It’s a familiar feeling of when some question comes up and I think “oh, I used to know that, let me look it up”. Maybe I just already hit my personal floor of stupidity before LLMs.
However, I personally feel a huge mental burden of the state of communication. The contemporary version of it where I have a million threads and conversations im juggling at any given time. Emails, voicemail, chat, online, texts, personal, business, home, children, other family, friends, then there’s the variants like Messages, Messenger, WhatsApp, etc. And as overwhelming as it is for me, I’m super under connected than everyone else I know. I quit following most news and all sports, as I just don’t have the bandwidth for it.
My brain was molded preinternet and I feel like it’s reaching its max on the analog to digital conversion. Or at least it’s just a really lossy process.
Yeah, I'm 45 and I'm like you - no social media, relatively under connected, and still feel swamped constantly by emails and calls and especially texts. They eat up half my productive time every day, and most of them are things I'm looped in on that I don't even need to respond to.
Okay so let's say that's the new cognitive burden. The new escape hatch is "AI". Now you don't need to read your mail or write responses! Let an LLM handle that for you! And now your friends and coworkers will send you AI generated mail anyway, so if you're actually taking the time to read and respond to it yourself you're a chump, right?
Noise machines. Humans are noise machines. Ever try to sleep till noon and notice that everyone else seems like they can't feel alive unless they wake up and make the maximum amount of noise and racket possible? What could be better for a gibbering species of ground dwelling apes than a miraculous machine that gibbers for them, to point back and forth at each other?
> And now your friends and coworkers will send you AI generated mail anyway
This hits close. I realized one of my friends was using AI to message me and I took it kind of hard. It's weird to be worth the effort for them to set up a chat bot to talk to me but not worth the 2-3mins a week to actually read/respond to my messages.
Right now, I just basically ghosted him, but I have teh feeling this is the start of an emerging issue.
Not the parent, but I have a teammate who stunned me when, after a long back and forth texting that seemed out of character for them (I can be a fast & long texter, admittedly), sent me a long message that ended:
"If you want, I'll write you a response to 9x39 that follows up on ..."
Bro. Come on. Anyway, weirder things have happened. A chatbot that works like what people do on social media comments would be just be offensive.
I'm unconvinced of this "polishing" everyone seems to claim. Whenever someone dumps a load of slop on you, that's always the answer, oh this is totally my own work, but I used AI to "polish" it. Then why is it so clearly crap? What kind of "polishing" makes text bloated and empty?
No, they were too lazy to write and let the machine do it.
Well, I like writing. But I have friends who hate it .. but need to for work. So they enjoy LLM's to make professional drafts out of some notes (technical reports) without all their grammar misstakes etc. No fluff, just the document in a shape the target audience expects.
But if they would do this to me .. I would object. I prefer their authentic ramblings ..
Finally, someone calling this one out. Are there legit english-learners or unskilled communicators out there who sparingly use LLMs to tweak 1-10% their prose here and there? Yes. Does that mean that the vast majority of slop-senders just using it for this "polishing"? No way.
I think some people are okay with communication that’s less involved. Like meme-y BSing where everyone involved knows everyone else is putting like 12% of their thinking power into sending a response.
I don’t really enjoy that, so I find having that many threads stressful and annoying.
I just take a hard line and will unilaterally downgrade communications (while politely letting the other party know). I have all my family group chats muted because my mom uses “Send” the way you’d use Enter on a desktop. End of a sentence? Send text. Next bullet point in a list? Send text.
I muted the chats and told her that I want my ringer on in case there’s an emergency, but I got 30 something notifications in 5 minutes during an interview and it’s unfair to the candidate or other people in the meeting. Internally I rationalize it as revoking someone’s ability to make noises on my phone at whim. They can still text me, they just can’t interrupt me anymore.
It helps a lot, even if only temporary. I’ve muted people for a few hours or a couple days before when I’m already stressed and they’re really chatty.
We have to normalize being on silent all the time and making people wait hours for a response. Return to the primordial monkey of 1800s-era high-latency comms.
At first, some people will be offended. "Why didn't you let me ping and buzz you and interrupt you all day? You didn't respond immediately each time :'((". Some people with unrealistic expectations may even stop talking to you entirely.
But eventually (years maybe) they will get overwhelmed too. No one can handle this madness indefinitely. I've seen giga-texters get broken down and turn into lazy texters like me, or at least learn to tolerate my long response intervals and recognize it as a coping mechanism rather than rudeness.
I am notoriously "bad" at texting. My phone's on silent almost 95% of the time, I don't even have a smartphone so the only way to get to me wirelessly is to call or text.
I got really into sending mail last year, specifically postcards.
I have a list of ~10 people I would consider "close", immediate family and good friends, and 5 or 6 more tertiary contacts. I travel fairly frequently, so I had plenty of opportunities for sending postcards. I send cards for obscure holidays just because. The physical process of hand-writing messages is so therapeutic for me. I've probably sent ~250 postcards in the last year and a half.
I have received... 3 physical responses. It has been extremely disappointing, but I continue to send mail because I enjoy the process of writing the cards, and the knowledge that people probably appreciate the mail makes me feel good, so at least I get a little out of it myself.
My mom will occasionally text to say she liked the postcard, but has never bothered to send one back to me.
I would be delighted if more people chose to communicate slowly.
I like this. I don't know what postcard etiquette is, but when I send a postcard it's just to show that I'm thinking of someone while I'm traveling. If the recipient doesn't travel or finds other ways to express that they're thinking or me while traveling (souvenir, etc) I'd consider that social contract fulfilled.
I've told people this for years. The mode of communication reflects the urgency. If you text me, expect a response on the order of 3+ days. If you call, and I recognize the number, it will be more urgent. If I DON'T recognize it, it goes to voicemail and back in the 3+ days queue. If you show up at my door, it is immediate. Even with my wife, she will text while I'm at the grocery to pick up some extra food items, and it doesn't necessarily come through or I'm on silent. I'll get home, and she'll ask where the food is, and I ask why she didn't call if it was timely. I just do NOT check my texts that often, it isn't because I'm deliberately ignoring anyone.
That's funny, I take the exact opposite approach. I prioritize interactions based on how much commitment I expect they'll require, with lesser commitment getting more priority. So a text message I'll usually answer right away. An email or some written reply that requires some redaction I'll postpone to when I can take the time for a thoughtful response. A ring on my buzzer, if I'm not expecting anything or anyone, I'll always ignore; I can't let any dumbass passing by the front of my building rope me into a pointless conversation.
Phone calls don't fit neatly into this scheme because they demand a lot of attention, but it's easy to get out of one if you realize it's not something critical. I generally pick up and the moment I get the slightest whiff of spam, I just hang up.
The current trend seems to be switching the priority order of calls and texts among many of us. I feel like a call should be scheduled, preferably 3+ days out, and preferably with an agenda attached. (Same rules I feel about any sort of meeting.) But a direct text (non-group chat, just to me) is a priority. Group chats get that 1-2 days middle ground.
If it is an emergency, it is a voice call. It is both immediate and conveys urgency. If it is something that you need to talk through, it is a scheduled voice call. Asynchronous communications may demonstrate respect for a person's time since it does not (need to) interrupt them in the moment, but the inefficiency results in a disrespectful waste of time for bidirectional conversations.
If it is something where you need a simple response by the end of the day, it is a text. If it requires a lengthy response, email. Never expect a lengthy response by the end of the day, or for it to be handled on devices with terrible input methods (like phones).
Anything that isn't covered by those scenarios will be largely dependent on the person.
I know that's the trend, but it is backwards to me. Like UDP vs TCP. If you need an immediate answer for something, why send a one-way communication where you have no idea whether the person on the other end A) received it, and B) acted on it. A 15 second phone call accomplishes this, whereas if I text you it could be hours, unless you immediately respond.
I really like that system! How do you configure that only notifications from certain parties end up on the watch? As far as I can tell I can only filter on application. On iOS I can add “favourites” which get prio for calls and messages in Messages/Mail but not in other apps.
Yeah that's the iffy part tbh. I have a oneplus phone and watch and it is per-app on the watch. So in the app I have to configure which lines of comms are important (so like Signal is how my wife and I communicate so I get watch notifications for that, and I mute other group chats). And because my WhatsApp is pretty noisy, even with muted chats, I just opt not to get notifications on my watch, which has the consequence of me not being able to get WhatsApp call notifications, which honestly isn't the worst thing in the world :)
Agree. I mute every group chat and notifications for almost everything. Same reasoning. My wife just talks to me when something reaches a point of me needing to know. Broader holiday planning or group travel planning chatter, it seems like any family gathering requires a minimum of 1000 messages.
I'm noticing some decline of skills I don't practice regularly and LLM is just one of reasons why one stops practicing. Switching to another area of work gives a comparable decline. If you want sharp skills you have to use them.
True. People don't do it though, because keeping skills sharp and using them takes effort, and we have a predisposition to be as efficient as possible with how we spend our effort; if there's an easier way to do it in our awareness, we will naturally gravitate towards that. LLMs are often a universal crutch or swiss-army-knife that significantly take away workload for many abstract tasks, so all kinds of atrophy in abstract thinking is to be expected.
However, when looking at muscle, once you have it you don't need to use it as much in order to maintain it. I wonder if the same is true for skills; in that case, some kind of regiment where you still use the skill you delegate once a week or so could maybe help with avoiding this loss of skill for most part.
“ However, when looking at muscle, once you have it you don't need to use it as much in order to maintain it”
No.. this depends on how much muscle you have. The appropriate comparison is mass and density of knowledge/understanding vs muscle. There’s not a chance in hell you will retain mass and dense muscle without pushing the body hard. Just in the same way you will not retain very deep understanding of things unless a) you’ve been reciting it for over 10 yrs b) you go back and push the understanding continuously for it to remain as part of your being
Building muscle is much harder than maintaining muscle.
And if you went 3 years without exercising, you'll be able to get your muscles back much quicker than had you never had the muscle before.
It's pretty comparable to skills. You don't need to practice as hard to maintain a skill than you do to build it. And if you let the skill atrophy, it's much easier to recover the skill compared to building it from scratch.
> And if you went 3 years without exercising, you'll be able to get your muscles back much quicker than had you never had the muscle before.
This very much depends on age. I went on statins about 18 months, which destroyed about 15lbs of muscle over the course of a year (160->145). Along with that muscle loss came about a halving or more of the weights I could lift in any given exercise. I interpreted the "do you have any weakness on this medication" question as inability to function levels of weakness, it wasn't until I showed my training logs to my physician that she asserted that I was having weakness.
It's been a year since I went off them and I'm still lifting barely what I could in high school. I'm exploring some different training plans, but AFAIK, there isn't much research into if different weight/volume breakdowns work better for older guys.
I’ve got 20 inch lean arms - I know far more about muscle building and retention than you. I train just as hard to maintain them as I did to get them there.
The people who say “oh it’s easy to maintain” LOL it’s easy to maintain 16 inch arms.
Chiming into this little tiff to say I think bulk muscle is a bad analogy in the first place. It’s more akin to a muscle memory/skill. Something like golf is a better analogy. If you took any golfer, at any level, and had them refrain from golfing for 3 years. I feel pretty confident asserting they would all perform worse than they had. Their skill is diminished.
They would also likely get that skill back faster than a brand new golfer.
I noticed it myself with cycling. Took 8 years off the bike, when I started up again I was nearly back to my old FTP in about 2 months despite starting from basically zero. Muscle memory is real, where I am now as a returning cyclist would take a pure beginner cyclist at least 4+ months to get to, fitness wise.
That said, you do have to work somewhat hard to maintain. With cycling, just 2 weeks off the bike is enough to see a VO2 max drop of anywhere from 4 to 7%. After just 4 weeks, your glycogen storage capacity decreases and you start rapidly losing fitness. After 2 months, you are basically now out of shape.
Detraining happens faster than most people think. And therein lies the danger with over reliance on LLMs for your cognitive skills. Detraining there happens just as fast, skills atrophy in a matter of weeks, not months or years.
People could also regain some cognitive skill back rather fastr when they worked to regain it. But the issue is, many people just lack the motivation to do so. If you golf or cycle, it's likely a passion or hobby. Most people don't view their cognitive health this way, they view it as work. It's why most people don't read much after their schooling, learning and being smart was only ever an ends to a means (diploma, job, money, etc).
I think part of the problem is also that many people simply work too hard or have too much going on in their lives to have any kind of cognitive energy left for this sort of maintenance work, even when they reason/plan that it is useful. This also seems to be encouraged somehow (by society?), to keep going like a freight train, or maybe it doesn't get discouraged enough (i.e. it doesn't get recognized as a problem).
My experience as a parent to an only-child has shown me there's just zero boredom or tolerance of boredom. Any pause or void needs to be filled with something. Any time my son says "I'm bored" my default response has become "awesome", "you're lucky", "I wish I had time to be bored" along with other quips like "boredom is a life skill". Of course, I see this same phenomenon in adults as well. So my rebuttal is that most people have much more free time than they think, it's just a matter of prioritization.
To maintain the muscle you have, you only need about 1/3rd of your normal workouts. It can be retained with 1-2 workouts per week. I imagine the same would be for something you've learned. If you've already put in the effort to learn it, reviewing it ~1x per week is probably enough. During the accumulation phase though - whether it be muscle or learning a new skill - once a week is definitely not enough.
Yes, this is my experience for muscle at least. I used to work out 3-4 times a week, maybe a little more sometimes. Lately due to circumstances, I've been doing smaller workouts about 1-2 times a week. I've lost some finesse, but my muscle mass has remained roughly the same.
Also like some people hinted at this in sibling threads, I think it's different between purely abstract skills, and skills that involve muscle memory. For instance, I could probably stop using my bicycle for a very long time, and still not unlearn how to use it, or learn it again really quickly. Maybe it is because abstract skills are inherently more complex and require more cognitive effort and connections to knowledge overall, and are therefore more fragile.
Most/all of my university-level math knowledge is gone, atrophied from never having needed to use any of it professionally. I don't even really recall needing it for any of my CS coursework, honestly. It was just required for the degree.
I used linear algebra to implement PageRank in my Information Retrieval class. I also used it extensively in my AI and ML classes. You can't pass a ML class without a good foundation in linear algebra.
Not to mention, discrete mathematics is the fundamental building blocks of CS. Surely you were using algorithms and graphs. I hope you computed an algorithm's efficiency with big O notation. I hope you have used probability before.
I don't think it's just you or your age, per your pre-internet comment. People that grew up in this just don't understand why they're overwhelmed. And I don't think they're even aware of what their missing out on in terms of focus or mental acuity.
Good point. I do have context and self awareness that it all seems unhealthy. Feels like a common sense evaluation to me but I can’t properly place myself in a younger generations experience.
I too was and wanted to only blame communication overload. Especially with work the hardest thing in ai times seems to be the overload of stuff/shit to read that is too easy to write.
The reality is I agree with the op and I see the loss of reasoning power in myself. I've been using native Emacs on android for a bit and finally have gotten serious about config for it. I got lazy and had Claude do some of it. Which was great untill things don't work because there's not going to be my crazy ask in the data. It was painful for me to sit down and think through my configuration and the problem but I did it.
I am absolutely torn on the technology still two years after adopting it.
It’s a really lossy process. Mostly due to most humans and all models treating sign meetings as determined at the moment of softmax crystallization. Signs (words included) are no more determined than the speed of light is. It’s all reflexive and we should stop lying to ourselves it can be determined.
It's a really underrated problem. I don't think my actual cognitive skills have declined by using AI, but I do notice that my patience and attention span are a lot lower.
I'm learning a new code base for a new job right now, and I'm finding AI to be a really double edged sword for it. One one hand, it's extremely valuable for asking questions about the code base. On the other hand, if I'm not careful and I just let it apply the fix before I even investigate it, I'm really not learning the code base well at all. I find I need to actually write new code in a code base to exercise the necessary mental muscles to actually retain understanding.
Incidentally, I do find that this large new code base I'm learning also shows the limitations of AI. There's no way I can vibe features on this without understanding and not introduce a lot of issues. Even targeted bug fixes have a lot of unintended consequences the LLM doesn't see. This isn't a bad code base at all, but it's definitely at the size where even frontier models struggle. So to me that tells me that the argument that I should just use more AI to solve my AI issues and not bother to understand the code base isn't viable at the moment.
> I don't think my actual cognitive skills have declined by using AI
I'm not speaking about you but... I know most people would not have much awareness of their cognitive decline. I know this because that awareness gap is there with or without LLMs, across all age groups and cultures.
For cognition, sure, but that's a fairly weak claim. A dog that chases its tail for 3
hours might be considered conscious, but maybe not highly intelligent.
The attention deficit part of ADHD hurts some people, but a high intelligence is able to make up for it in other ways. Attention span is a multiplier for intelligence. Someone with a lower IQ but a longer attention span is able to outperform a higher IQ but shorter attention span, traditionally.
What's required though, is the attention span and the memory to really dig deep into a problem, and then go for a run. If AI makes that easier, since it lets you skip the boring parts and get to the meat of the problem, then hey.
True, I guess I try to have some objective measures like my chess elo and maybe some canaries like what books I'm reading. But it would be really hard to tell.
I used Google Translate to not learn French in collage. Fortunately for me it was bad enough I had to carefully review all its outputs, but that still didn't help and I managed to pass two semesters without ever developing even basic language skills.
Something radical needs to be done. When I was in high school there were still a lot of "no calculator" restrictions in my math classes that I chaffed at because I hated doing longform arithmetic and felt like it got in the way of learning. So I can certainly understand how students would chafe at some kind of paper-only education system but I also don't see how you can learn anything when you have a high-quality homework machine just sitting there.
I wish that would have worked for me - we had oral tests. 2 years of French in high school and one semester at college - what an absolute waste of time. How much French do I know now? Basically none. The same goes for everyone in my life that did Spanish instead in high school.
Part of what we could do during this upset is re-prioritize.
All that's needed is a tight feedback loop between learning and applying those skills ... the thing that Google Translate helped you evade. AI can be a tool for evading or optimizing that loop, like a knife can cut your sandwich or your throat. Your choice.
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
I agree - I would have been toast. I wonder if the teachers/colleges need to change the way they teach and assess. Let the students use the AI tools they like (perhaps guide them how they can use them professionally), but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person. Oh and don't give Fs for cheating - suspend them.
I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.
Absolutely university has to change. But it's not a simple change. I say this as a professor for Physics:
My colleagues say "We must fully embrace AI as a tool". I agree. But how do you teach it? It's a moving target, and you can't even give homework like: "Research <this topic> with an LLM of your choice, and submit the transcript" because they can do that, or they can just copy the task into an LLM and have the LLM do it. It becomes meta quite quickly.
And independent what and how we teach, we have to change how we assess a students learning result:
The first thing we have to change is that homework needs to be completely ungraded. Reviewed and corrected, yes, but not part of the grade. That's the only way to make sure that people who don't want to cheat have to cheat anyway to compete with those that do.
Second, all exams have to be in person. Online, cheating is so trivial it's not even funny (many students are so stupid about it that we have a pretty clear idea what's going on). In person, we have maybe 2-3 years until we have to make sure its proctored and people's glasses are checked. I think in less than 10 years, local mobile AI will be good enough so even a Faraday cage will not help.
Maybe we have to go to oral tests only.
Of course, none of this scales. Some of our intro courses have a thousand students.
I feel like German university was already well "LLM proof". Come to class or not, no ones keeping track and it doesn't matter for grade. There are regular exercises, but they are not graded; you can submit them if you want it corrected to check your understanding, but there is also gonna be a short presentation going through it in class. Your entire grade is a 2-3h written exam at the end, no materials, no remote, no books, no multiple choice.
The problem here is that it will work for now, but how do you make sure the LLM talks to the student, and not a different LLM? I guess vision models FTW?
>(perhaps guide them how they can use them professionally)
If that's anything like how they guided me to use programming languages professionally...
In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial. They've barely adapted to video conferencing.
Traditionally, moving slow with policies was fine with new tech because, outside of the PC revolution it wasn't all that impactful, and things used to rightly be labeled as experimental so you could safely ignore it for a while as a big enterprise and be just fine until thinks shook out.
LLMs were, IMO, pushed out too early and without that clear "this is experimental tech" label. Full public access from day 1, no invite only betas, no research previews for a select few pilot customers/orgs, etc. I've been in IT for a little over 18 years now and I haven't seen anything move this fast before.
I mean, I never though I'd see Microsoft go on stage at BUILD and and announce freaking OpenClaw for Enterprise, and then make it available the same day. This is highly unstable tech and what I'd consider still experimental, being sold to F500s as production ready.
> In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial.
Perhaps this is rather a sign that you currently shouldn't jump on the LLM hype train, but rather attempt to get a good foundation on the basics. When the whole LLM area becomes much more "stabilized" (I see signs that this is currently happening, if only for the reason that training state of the art models has become more and more expensive), you can still get into LLMs if you want.
I partly agree, depending on what you count as the basics. I don't think there's much value in learning the quirks of LLMs today: they will just change, your value-add becomes part of the model or harness.
On the other hand I think there are real development gains in jumping on the train today. To my career's detriment.
Yes, meanwhile, Claude Cowork was only released this past January. And that was amazing. But I don't know about anyone else but I've already moved on to just using Codex for just about everything (except some Kagi use). Schools work on timescales of years, AI is advancing on the timescale of weeks and months.
Until that situation stabilizes I think the only institution capable of teaching about it is the family -- parents.
I'm not sure parents have the right tools either. Microsoft is about to ship OpenClaw as part of windows (talked about at BUILD) and they're acting like it's production ready and they've solved the security issues.
I don't believe them for one second, it's far from a solved problem yet these companies are selling this tech as if it's been around for decades and thoroughly battle tested instead of highly experimental and unstable.
I think it's incredible how much those ancient brains can successfully adapt to technology. Some people can sit in highly-strung sports cars and use them at the absolute limit of their performance like they're just an extension of our own limbs and senses.
> I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.
That seems like a smart approach. It reverses the traditional model of "lecture in class, homework outside of class".
A smart approach that does not solve the AI problem - actually flipped classrooms work worse now due to AI usage.
My own experience with flipped classrooms (which seems to be shared by quite a few people who have tried it out): they only work well if all students actually read/watch the materials beforehand. In small, advanced courses, intrinsic motivation may be sufficient - but in most cases you need some extrinsic coercion - such as a mandatory quiz about the materials or hand-written lecture notes that need to be shown at each in-person session.
With AI, some people don't watch the lectures but let ChatGPT give them a summary which they submit. Then these people poison your in-person session with their lack of knowledge and motivation.
Research has shown that testing is far and away the most valuable academic tool.
Just have a quiz every day. In fact, have _TWO_ quizzes, one at the start of class and one at the end, and take the higher of the two scores. In between the first quiz and the second, work through problems with the students designed to help people that bombed the first test figure out how to pass the second.
The best part of a quiz everyday is that in addition to the testing effect, you can easily fit in the spacing effect and interleaving effect. It’s a rock solid combo, that is well studied. We have pretty strong evidence that it works for all students in all domains.
I actually like this idea - makes sense at face value - as long as they design the test in such a way that it aptly applies the knowledge instead of just learning for the sake of passing test like questions...
My Cryptography professor did this during COVID, since the classes were split in person. It was an interesting model. I'm not sure if I loved it or not, but it was at least a change of pace. Getting 100% of the class time to ask questions was really nice, but it ended up with him re-teaching most of the online lecture in class because some quarter to half the class just didn't watch the lectures.
If done more stringently (if you didn't watch the lecture, I'm not reteaching it), it maybe would've had a bigger impact, but I'm not sure.
Office hours remained king for serious Q and A for the class.
One way to fix that issue that I’ve seen is a daily quiz to start the class. The key is the quiz is super easy. Even if you were confused by the lecture, if you watched it at all you’d likely get a 100 on the quiz. If you didn’t watch it you’d likely get a 0. This quickly for people watching the lectures online ahead of class.
I always absolutely hated when a teacher did a reverse classroom and I had to “learn” at home and then practice in the classroom. I think the solution is more engaging lessons and less outside work. I know why homework exists, but homework is a chore that most people want to get done as fast as possible. If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI. If they’re interested in the topic, they’ll put more effort into it. If not, they were never going to retain it anyway
> homework is a chore that most people want to get done as fast as possible
Worksheets certainly are. But good homework, even if it's challenging, is what makes a reasonably fast-paced course even possible. In a well-paced university course you're typically spending proportionally several times as much time working on it out of class than you are in class. Then class time is both preparation and catch-up, similar to office hours.
This was true of my most demanding humanities courses (sometimes reading 100 pages a week directly from academic journals, not easy reading) as well as my most challenging math courses (group theory, ring theory). Once the pace gets fast, there just isn't enough time for you to learn everything you need to inside the classroom anyway.
And in those classes, where homework was really essential for learning at the required pace and depth of mastery, my instructors didn't even need to factor the homework into my grades at all. In some of them, we could get "feedback" on homework but it was never officially recorded in our grades... and yet, anyone who didn't do it would fail the next test. If homework doesn't have that characteristic, it probably doesn't need to be assigned at all.
If "flipped classroom" means that students are expected to do all of their homework in class, then indeed it'll feel like a waste of time to many of the smarter kids, and it will also just be unfeasible for advanced courses (which theoretically should be most courses in a university, though it currently isn't). But if it means "we don't even have time to lecture you on every single thing you need to learn, therefore you must arrive already having done the reading and the exercises, and we'll use this time to help clear up misunderstandings"... that's already how classes for grown-ups are in universities.
When left to their own devices, 99% of children are not interested enough in math, history, literature, languages, or almost any other school subject to engage with it willingly. Only teaching "interesting" things that kids are "interested" in is both impossible (too many varied kids per class for that to work 100% of the time) and even if possible, would leave kids with zero practical knowledge, because learning most of that stuff is not something kids inherently want to do.
College is different, because theoretically you should be taking classes that are relevant to your field (although there are still "core" requirements that are somewhat high-school adjacent).
I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.
College is a different dynamic from a middle/high school classroom, but I don’t remember 95% of the material from my college engineering classes anyway, it’s the problem solving and information finding that I’ve retained and have helped me do the things I do. I remember the stuff from the classes that taught me the material in an engaging way though.
>I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.
"Just do it right and it won't be a problem." This is not an actionable plan. What is engaging? Who gets to decide that? The teacher? The students? The parents? How do deal with certain kids finding different approaches more or less engaging? How do you expect a teacher to curtail their teaching approach to dozens of children at the same time?
>If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI.
Kids get to learn lots of interesting things in school. The problem is that they're kids! They want immediate gratification from phones/games/recess, not to do the hard work of learning.
Flipped classroom pedagogy has been the subject of a huge amount of research. Ultimately "one weird trick" solutions don't tend to work in education. Enough students don't watch the lectures that you end up needing to go over the material in class anyway. Funding and autonomy works, but nobody likes to pay more.
How does this scale in practice? We already require students be at school for 7 hours a day. If they now have to watch 3-4 hours of lectures at home every day, then students are left with little time to do anything else.
What about those students who don't have stable home environments? How are they supposed to find multiple hours a day to watch lectures?
How does this address the underlying issue of students off loading work? You've replaced homework with lectures, but haven't solved the problem of making sure the student is actually participating.
Logistically, this could only work if you shortened the school days, but then you would need to adjust the rest of society around that. Many parents structure their work days around their kids school schedules, and if kids need to go school later in the day, or get out earlier, that places a burden on the parents.
From my experience it works fine if it's one class that's doing it. If multiple classes are doing it then like you said it's literally a couple extra hours a day watching lectures and most students end up skipping them, forcing the instructor to end up teaching during class time anyways.
We’re discussing university, right? It’s supposed to be a full-time effort, at least for a normal pace undergrad or any post- graduate program.
For secondary school, I do agree with you - homework load can be problematic for some students. But at the same time, my honors classes all came with hours of homework and I’m not sure I would have been as prepared for uni without it.
> Let the students use the AI tools they like […], but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person.
I very much doubt there is any agreement on what those skills are.
Creating the idea of “what to learn in the new world” is itself IMO an important academic creation, but there’s no reward for doing it and no way to know if you’re on the right track (you just have to wait and see).
Employers are also just adapting.
Wait until companies are paying unsubsidized “list price” for LLM usage. Then we can have a better idea of the worth of the automation and what skills should stay with humans.
This. The industry is dumping electrical labor at a humongous loss JUST BECAUSE they figure people will immediately atrophy and be unable to do without AI… at any price.
We'll get an idea of the relative cost of the labor, all right. It's just that they are specifically trying to wreck the market, at all costs, to be able to cash in on the upside. It's sensible, if you're a monster.
I'm dumb as a rock and I don't have a PhD, but since ~1 year ago I started forcing myself to do small bits of coding and math manually.
I'm not noticing a "cognitive decline" per se, but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.
Yes, precisely. Assessing your own cognitive skills is dubious. I’m pretty certain I’m less clever than I was when younger but if I find a problem tough now maybe 25 yo me would also have struggled?
> but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.
Not getting that quick dopamine hit the LLMs give you..
Some say you can re-train your system to get back the dopamine hits you used to get from other things, like the enjoyment of the "old fashioned" manual coding and math. Getting there is hard work. And YMMV.
I just do things manually and ask LLMs to check my work. That seems to be working great for me.
I had the most Russian of Russian bosses when I was in college. My first day on the job he so eloquently stated, "I am not your mother. Do not come to me with problems. Come to me with solutions. I want to know what you tried and what did not work."
His advice has served me well in many areas of life too. I try my best to treat LLMs no differently for domains I care about (not one-off little questions here and there).
>even stuff that used to be routine when I started coding now feel heavy.
The same weight feeling heavier is a sign that your muscles are weaker :)
There's many areas in life were we look back a few decades and think "people use to do it that awkwardly?" And yet results were better. I think the process of removing friction have just served to destroy our ability to concentrate and tolerate difficulty.
I do a similar version of this, where if I notice a mistake in generated code, I fix it manually (or at least attempt to) instead of telling Claude to fix it.
I use an agent to generate a first-pass attempt, and then (deadlines willing), I manually read every line at least once so I understand what the code actually does.
Then I manually fix the inevitable slop that is mixed in with the good stuff, and only once the code is up to my personal standards do I send it.
This probably reduces my “AI performance boost” to 30-50% instead of the huge gains reported by others. But I retain the ability to reason about the codebase and use AI much more precisely when I’m trying to troubleshoot production outages or subtle bugs — something I notice the rest of my team struggles with, since adopting “agentic workflows” everywhere.
I think actively working to retain some cognitive flexibility and “muscle memory” around coding tasks is going to be rather advantageous in the long run.
Same, but also because it feels like it takes longer for an LLM to do it. I think that's something people who are into gathering personal metrics should do - measure how long it takes to type a prompt / have the LLM fix things vs just doing it yourself.
Why do you think you didn't do similar when you were a student? We've seen this exact issue with other technology before: calculators for math; typewriters and then word processors for writing assignments; audio books for literacy. In those cases we've collectively realized there's a benefit in getting the manual skill and understanding how to use it before shortcutting things with the technology, even if most of the time you'll just end up using the technology. My best guess is that for most people in those classes who failed because of ai use, they don't care about the understanding (usually) required to get a good grade, they just care about the grade itself and the doors that opens for them.
It's because modern AI promises to relieve you of the tedium, leaving you to consider the important things like higher structure. It actually does deliver on this, but in contrast to older tools, it is unlimited in scope.
A calculator - let's expand this to maps, thesauruses, dictionaries, and other lookup tools - was used for a pretty narrow set of problems, and you had to transcribe the result to whatever context you needed.
An AI can be all the calculators together, and transport the output of one to the input of the next. You're meant to have the overview, but it's just so enticing to let it simply do that as well.
Counterpoint, I think this is true for some archetypes of people, but certainly not everyone. I personally use it like the socratic method. I am an intermediate user, I spend a ton of time with LLMs at work and personally, both prompting and letting some crappy agents try to automate boring work. I primarily use Gemini and ChatGPT models, along with some Chinese smaller weight models (eg qwen) locally.
If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.
This, I will use Obra Superpowers brainstorming skill to propose/refine a few viable solutions for a feature or bug I'm trying to solve. After it asks me clarifying questions and presents a spec, I will say "well what about X or Y". The I'll run the grill me skill on the spec to tighten it up, clarifying any assumptions made.
I find it to be a really tight loop and results in very high quality code at a high velocity.
My two modes of using LLMs has been to try it for 1) natural language search queries where traditional search engines have failed and 2) occasionally as a sounding board using the socratic method.
Inevitably, it fails frequently at both. Any "reasoning" it is doing is merely rehashing ideas that someone else has already posited. This helps some of the times, but the vast majority of the time it just chooses a biased perspective (frequently the most common) and then regurgitates tired old talking points. This contrasts greatly to speaking with others who often have more intuitive notions that tend to be less polished and rote.
I'd love for LLMs to be better sounding boards, but so far they fail miserably far too often for my tastes. To each their own though.
> If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.
Yes, but eventually the intellectual whack-a-mole gets tiresome unless you get really, really good at simultaneously cornering it and not letting it concede to your point.
new session. It's easy to lead a model into getting the response you want, deliberately or accidently.
The point is not to literally win an argument (it doesn't matter), it is to use the model like a partner to poke holes in your own understanding. Once it's poked a hole, it has served its purpose. Plus, you eventually run out of context or the model trails off into babbble.
> For adults the cognitive decline won't be as measurable since there's no exams, and overall output volume will still be fine due to LLM help
The leading indicator for me is the amount of emails and, god forbid, more personal messages (like birthday wishes!) I see that are obviously AI generated. It just keeps on rising. If you’re not able to dash off a quick message without the help of AI I have to assume you’re using it heavily elsewhere too.
I have sympathy for the university students too, we’re all bombarded with rhetoric about AI being the future. And I remember being incredibly nervous emailing my lecturer (am I phrasing this right? Is it respectful enough?) that I can imagine leaning on AI myself had it been available back in the day. But I’m glad it wasn’t, it’s an important skill to work out this stuff. They’re going to land an in person interview when they graduate and stumble around unable to effectively answer the questions they’re asked in real time.
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
So, as long as you are not under time pressure (which you in some degree courses unluckily are), there is simply no need to "speed up" any homework assignments.
If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study (which is only loosely correlated to homework and tests), I guess it's fine to use them. Just always keep in mind that very often the pain of attempting to understand the topic on your own often makes you smarter - something that you will miss when you take an "LLM shortcut".
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
This is probably not true for majority of people. Most go to school because it is mandatory, pushed by parents and society, and university gives you credentials and better job opportunities. Homework and tests are a way to get a number grade on 'how well you memorized something', it doesn't really measure a deep understanding of the topic.
> Homework and tests are a way to get a number grade on 'how well you memorized something', it doesn't really measure a deep understanding of the topic.
As I said: they are goalposts.
Typically homework and tests are sufficiently easy (yes, there are exceptions) that if you fail them, you can assume that you didn't make sufficient progress in improving your understanding.
But I do agree that at least sometimes the difference between being good and exceptional at homework and tests can indeed be rote, "unnecessary" memorization.
With learning predicated on both failing and remembering it's unfortunate uni scores on 100% successful doing but doesn't teach failing well, and scores for remembering but not for learning well.
> You go to a university because you are deeply interested in understanding the subject that you study.
Echoing the other comments here, at least in the US, this is generally untrue. I went because my parents made me, because the choice was that or get kicked out of the house. It was beaten into my head since I was in grade school that "people in this family go to college" and "you can't get a good job without a college degree."
I hated every moment of it and I was glad to take my BSc and never look back once it was over (University of Houston, c/o 2000). And, indeed, without the degree I wouldn't have had the jobs I've had.
But I didn't go because I was "interested." I went because it was an effectively mandatory life-path objective. I'm very happy for you if your lived experience is different, but it is also—at least in the US—both extremely uncommon and extremely privileged.
> This has not been true for something like 70 years now. People go to university because it is expected that that is what you do after high school.
In Germany, many people indeed say if you are not deeply into the topic that you study, you should rather get a vocational training (Ausbildung), or attend a different kind of tertiary education than a university such as
- Fachhochschule
- Berufsakademie
(these words have no good English translation). Basically these are kinds of tertiary education that are more applied than the much more scientific training that you get at a university.
Specifically for mathematics (I guess the same holds for physics), a lot of people say that if you don't consider it to be an ideal life to think about math exercise sheets when you sit in the bathtub while other people are having fun at some party, you simply are not made for studying mathematics and should change your degree course as soon as possible.
This situation is changing in the US but isn't well reported, in my opinion.
We haven't regained traditional apprenticeship roles (perhaps because we so weakened unions?) but 30 (of 50) States have free or heavily subsidized two-year community / vocational college programs. Affordable and accessible vocational education opportunities are increasingly present. I also think (very subjectively) that we are seeing a renewed respect for the trades.
However, there are structural headwinds outside of education - no national health insurance plan being a major one. Farming, fishing, forestry, construction and similar trades still have a 20-30% uninsured rate in the US. (The uninsured rate in white collar "professional" work is around 2.5%.)
>
We haven't regained traditional apprenticeship roles (perhaps because we so weakened unions?)
The reason for the traditional apprenticeship roles is not unions, but rather capitalistic:
- If potential employees are well-trained the employer doesn't have to invest resources for training them.
- The certificate of the vocational training means that the employer knows that an applicant has an established standard, and can save time testing whether he is qualified.
- Because the trainee needs practical experience, employers can invoice this additional worker to the customer. Because the trainee needs explanations and thus works slower, more hours can be invoiced to a customer.
It's a much more socially beneficial system and I think a large part of this is probably differences in culture regarding education and one's career. The availability of these modes of teaching is downstream from that.
Universities have always had people not interested in the subject, but who went because its elite training school. I was reading something about Issac Newtons college in the 1600s, many of his classmates didn't care about studying and drank all the time.
It is really about time we thought about what universities are for in the 21st century, since there has been significant scope creep wrt labour markets, particularly roles which do not actually require university education but do require a degree for CV reasons. It is nonsense in the 21st century to require a bachelor's degree for such roles. Not to mention the huge societal pressure that you have mentioned, which no 18 year old can really be expected to see through.
With CS students this is one thing. Medical students? Air traffic controllers?
That is to say, there is a huge gap in the educational integrity of degrees, and this is probably partly driven by people who do not really want to be at university for educational reasons (and, believe it or not, there are other ways to party in your early twenties) and for whom a degree in XYZ is not rationally connected to 80% of their options after school. And there are many such people.
This really needs to be thought through, because education is expensive, and it is an enormous waste of money to pay for a couple of years of university and end up failing out or being sanctioned for AI cheating, or being educated for something you do not really want or need to be taught. That is true whether or not education is paid for privately or by the public.
ETA that when I graduated from school the idea of not going to university was really discouraged by the guidance counselor. It seemed like vocational courses were not really a worthwhile option unless you were a poor (significantly below average) student. There was a lot of emphasis on ‘getting a degree’ probably related to (nonsense) job requirements. Not a lot on what career you should pursue, or why you should consider university. It was more like why would you not consider university, since it was the de facto default. It was, I guess, unseemly for the school to end up with fewer university entrants and more apprentices.
At the time, there was somewhat of a social stigma with apprenticeships. The people that pursued them seemed to only genuinely have been set on the idea, and there were few if any that were diverted thereto. Now, of course, ‘the trades’ pay much better than a middling office job. Egg on my face.
Universities today are seen as debt manufacturing facilities. Debt that cannot be discharged. AI is seen as an act of war against the world’s working classes. Rich people aren’t building bunkers in foreign countries and buying yachts the size of a town because we ran out of land in America. Buckle up for a wild tumultuous period of human history.
Sure, I think this is a major element in the States, but I have never been there and all of this applies to my country where student loans are rare and fees are paid by the State.
> You go to a university because you are deeply interested in understanding the subject that you study.
I don't think I've met anyone who fits that description. The ones deeply interested in the subject would likely skip college anyway if not for future economic prospects.
>The ones deeply interested in the subject would likely skip college anyway
Spoken like a true software engineer ;), there are jobs where you have to have a degree to get the job. "Real" engineers with sign-off responsibilities, Medical Doctors, etc.
> Then you either really haven't tried very hard to notice them or have been in an academic environment with severe defects.
Sure. (?)
> Does college even work for future economic prospects, by the way?
Where I live, a college degree is a legal requirement for a lot of professions that pay more than entry level jobs (although not all of them). So, people go to college to get a better paying job in a few years than they could get by immediately entering the workforce.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
Some students do not have this privilege and implicitly see university as first and foremost a funnel into a paying career.
I can really certify that this was my lived experience. In the math degree course, basically everyone who was not incredibly passionate about mathematics (NB: "passionate" does not necessary imply "great academic achievements") changed their major or decided for a different kind of tertiary education.
Former co-students who attended the same university and degree course had the same experience.
I guess the reason was that it was a decent university in a "boring" town where learning for your studies was one of the more exciting things that you could do.
Another factor might just be that math pretty much is the extra depth behind a bunch of STEM fields, so people studying math specifically are more likely to be interested in that depth.
That said I generally think the take that it's somehow privileged to find school interesting to be sad. Over the last couple decades one could do pretty well with pretty much any STEM degree. Is the majority feeling among people studying engineering that they just have no interest in any facet of how the world around them works? They have no desire to understand how to create (and alter to their liking) the things they see? No interest in the fundamentals of how the universe works? How different materials come to act the way they do? How living beings work? Nothing?
1. I don’t think there’s a direct correlation between curiosity and finding school interesting. I’m endlessly curious about how the world works and always reading three books at a time, and school was often dull as paint.
2. I would optimize hiring people who display the kind of curiosity described, but if my goal was to create an education system to
generate educated workers to grow an economy, I wouldn’t optimize for it. I don’t think curiosity is a privilege, it’s an undervalued right.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
There is only one classmate in my class who came to study CSE because they are interested in CSE. And since we all enrolled after AI became somewhat good at everything none of them know how to code.
After two years of study I had to explain someone how to swap two number by drawing boxes. This are the things you learn in the first week if you're interested in programming.
My point is very tiny percentage of people study something because they're genuinely interested in that subject.
I needed the structure more than anything. Nothing I learned in university was some secretive knowledge. Anyone could buy the same textbooks and learn the same materials.
However, I am not one of those people. I lack(ed) the self-discipline required to learn anything with moderate to long feedback loops, thus it was far wiser for me to outsource all the necessary planning and assessment to a structured program.
Student loan debt is a blight on society, but I'd be lying if I said the looming repayment was not a great motivator to attend classes and do well in school. Some things just hit differently when there is skin in the game.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
I think this was true a long time ago. Perhaps with LLMs this can become true again in the future. But definitely that was not why I went the first time, nor most of my classmates. (Second time I did post-secondary, sure, 100% -- but I was almost 30, not an average student)
> You go to a university because you are deeply interested in understanding the subject that you study.
This is a bit of a naive or maybe affluent take? Like, theoretically, I agree. And I myself was curious. But most people, by and large, are going to university because they know they need a degree to get a job, unlike their parents or grandparents. And even "the degree" is quickly becoming devalued in this current AI age.
I would guess that if all basic needs were met through UBI, the fraction of individuals going to school would drop and the makeup of subjects they pursue would change. Probably more cooking and art classes and less stem. Although, if UBI existed and AI did not, we'd probably see more educated individuals in the first place so maybe there would be an uptick in stem attendance and general curiosity in such a utopian world.
Basically, this was really my lived experience, which might have been amplified that it was a decent university in a "boring" town where learning for your studies was one of the more exciting things that you could do.
Concerning the "affluent" aspect, I can clearly assure you that neither I am nor my parents were.
I think perhaps the reason you are seeing quite a few commenters expressing skepticism to your comment "You go to a university because you are deeply interested in understanding the subject that you study." is that you appear to be extrapolating from one example (your own), without considering whether that's likely the wider experience of people going to university.
In the UK anyway, there's an acknowledged idea that many people go to university because there is a societal expectation that they should and also because many careers require a degree even for entry level positions.
There is also much less emphasis on other routes of tertiary education (e.g. vocational schools), when compared to places like Germany.
> "You go to a university because you are deeply interested in understanding the subject that you study." is that you appear to be extrapolating from one example (your own)
I know a lot of people who think this way, and I can assure you that the people who realized later that university is not for them deeply would have wished that someone had given them this advice when they were younger.
> If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study
My experience is that they uncomfortably do both. You can "understand" something conceptually quicker -- like you have a new brain-muscle-thing that lets you cut through the hard difficult tedious corners to get to the meat of the matter.
But then you also can become reliant on it, and have difficulty doing the mechanistic rote work of working through it yourself.
Like the really big powerful calculator that it is, really.
It's two fold. They're learning and understanding more things, but at a very surface level and without the nuance and ability to actually use the knowledge because they have none of the muscle memory and hard work associated with learning it.
You can use AI or the internet to learn the basics of how a gas engine works in a couple of minutes. But you'd be incapable of actually working on a gas engine or designing one.
Surface level knowledge gets you surface level functionality. You don't become good at something from surface level knowledge, but you might think you're good at it.
Perhaps it was merely my degree or program, but I do not feel like I learned a lot more than surface level knowledge.
I have an accredited degree in computer science. But what does that truly even signify? For example, even with my 10 years of experience coupled with my degree, I could not create a programming language from scratch without some tutorial, nor could I create an OS, a compiler/interpreter, a basic CPU, RAM, any secondary storage, etc.. Sure, I could tell you about basic digital logic, basic OS principals, basic algorithms in a poorer explanation than a Wikipedia article, etc..
To use your gas engine analogy, if you dropped me on deserted island with all the parts I needed to make a basic, simple computer I would unlikely be able to do so with out some sort of in-depth guide. I am not talking about slotting mass produced parts on a motherboard. Rather, something like this:
If used correctly though I think the models in fact can be useful for gaining depth. With the right prompting they can actually perform a pedagogic role. One must just resist the temptation to have them do the work for you.
I've used my phone taking pictures + Codex + a PDF of my tractor manual to help me effectively diagnose and manage repairs in my tractor. (Though these models remain terrible at the physical world, getting physical orientations wrong, front back etc. Much like myself)
Likewise I had Gemini help me tear down my mower's carburetor and diagnose issues there.
(So much so that I've wondered about building some kind of "shop buddy" -- some kind of durable laptop and set of cameras ... on a cart. Running models that have access to manuals and cameras and TTS and voice input? "Hey, shop buddy, look at this fuse and tell me what is before and after it in the electrical system.")
This is helping me learn and do something I couldn't really effectively do before by walking me through steps.
My youngest has had Gemini write math questions for them, to help study. Not do the math, but write questions.
In the end it comes down to prompting, like everything.
Which makes me wonder if the answer for higher education is just to provide the students with specific coding agents they're specifically allowed to use -- ones that would push the student through problem solving and working on the problem together.
> One must just resist the temptation to have them do the work for you.
We are in the instant gratification era of humanity where a dopamine rush drives most people. This is a systematic shift that happened through the introduction of smart phones and social media and then progressed for a good decade to what we have in front of us today.
Asking people to "resist the urge" when they've been programmed/brought up to feed the urge is not pragmatic unless you are also proposing a way to erase the damage done from the instant gratification era.
We're in the end game presently. For every one person like you and your examples, there's gotta be 100x or more who are not using the tools the way you've presented them.
I'm no saint. For coding projects I am absolutely in the same boat as everyone else.
My other examples have to do with current limitations of the tools. Obviously there's no Claude Code for Meatspace that just takes over and does things for you. (Yet)
What I'm trying to point out is that the tooling has been made this way on purpose and I agree substantially with your point. But I also think human agency is involved. Dario & Boris et al didn't have to write CC the way they did. They chose to play with and push a concept which reduced human agency -- in part because Dario concretely believes it's just "inevitable" that we should be put of of work. And his investors no doubt love this concept too.
And just like Facebook / Instagram etc. it turns out it's an addictive flow.
It remains the case there are other ways of applying LLMs and generative coding models. This modality is not intrinsic to the technology. It's being deployed this way. And humans have agency in how it's applied, even if it's hard sometimes for us to exercise it.
> And humans have agency in how it's applied, even if it's hard sometimes for us to exercise it.
It needs to come top down from CEOs and governing bodies via regulation if we want improvements. We can't rely on the individual to not use the big red button that says "do this with no effort". We're on course for a WALL-E future if we're lucky or something far less great if we're not.
I appreciate your argument for human agency, but these types of systematic issues can't be solved bottom up.
> You go to a university because you are deeply interested in understanding the subject that you study.
You must come from a wealthy background because what you described is far beyond the vast majority of people's means - at least here in the US.
Most of us go to college because it's the only reliable way to get a tollerable job that pays well. Only a few of my college courses aligned with my interests. The rest were just the price paid for the degree.
LLMS didn’t invent cheating just made it easier. When you cheat you’re the one who cheats yourself because the point of an education is to learn, not complete the assignments and get high marks on tests alone. No one benefits and no one other than you is materially hurt by cheating, but you are absolutely the one who is hurt.
There’s no way to learn than to force the brain into adaptation which it is resistant to do through challenge and stress, just like your muscles. Similarly you can’t play e sports and get into physical condition any more than you can use LLMs to do your homework and learn.
It’s going to be a hard adjustment for a lot of people to recognize that letting the machine think for you is as healthy as smoking brain cigarettes.
The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button. They make great tools for learning if they’re used as an adversarial or editorial tool. The future belongs to those who work to use the tools in ways that make themselves more efficacious, not those who use efficacious tools so they don’t have to work.
>The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button.
Yeah, this is how we used wolframalpha for Math as students. Whatever we had to do, we did it ourself as a group of three. Afterwards we checked with Wolframaplha to see if we were correct. If there were any difference between us, we went line by line to find where the error appeared.
It was helpful, because we did it ourself, but because the work was graded, we had the security, that it is not a total failure.
To say that students don’t benefit from getting good grades using LLMs is incredibly naive. Learning is only about the third or fourth most important “benefit” for students, after getting a degree, getting good grades, and making connections.
These matter about getting the first job you get, at which point what you learned begins to dominate the rest of your career and life. The connections you made at school matter less and less as your connections in your career dominate, and they are built on what you can do, which is based on what you’ve learned.
Does anyone look at GPA on a resume? I’ve hired thousands of people I’ve never once looked at GPA. (N.b., my resume has “summa cum laude” ok it and no one has ever once mentioned it or presumably noticed it, despite the fact you only really get it if you can BOTH learn the material AND get perfect grades)
The problem with AI in an educational setting is when one is graded versus their students on things and things genuinely depend on those grades. Group projects also force those willing to do things without AI to go along with others in their group who'll use it regardless.
My son just finished his first year in college, and had no trouble getting decent grades without using AI while many of the kids around him were using it. At least in his humanities track, class participation is a lot of his grade, and he said the "AI kids" tended to suck at participation because they hadn't actually thought about the material, and couldn't dynamically work with it in class. He also said their AI assisted writing that he'd read was dull and unoriginal, and all sounded the same, which he thought likely helped his essays stand out. His English composition teacher said he was "probably too advanced for this class" when he told her he didn't use AI to write his essays, which made him roll his eyes, as he has clinically diagnosed dysgraphia (learning disability in writing).
Makes sense the ones who can't tell that AI does a piss poor job at writing get bad grades. In a humanities track I can certainly see how going basically completely no AI should be an advantage. Even in a other tracks it should be better, especially if professors think out assignments well. Group assignments are my biggest worry as in some classes they can really make/break a grade, working with those believing in AI would certainly be a disadvantage.
But I like to add artwork to my presentations. My artistic skills have not advanced beyond 2nd grade. So I'll make a line sketch, and give to AI to "fix" it.
The results are nice and I use them.
I have no interest in learning how to do art well myself, so using AI for it is appropriate.
I haven't seen your presentations, so I can't speak to them. But I do know at work there's a lot more illustrations in docs and presentations and such, and they almost all have an AI art "tell". I find them grating and distracting from the actual content. Very rarely do they add anything useful to the doc other than the knowledge that the owner burned some GPU time and tokens for a distracting, low value illustration.
I can only imagine how an actual artist or graphic designer feels about it.
Actually I don't have to imagine; there's some serious vitriol over on some of my favorite webcomics about it.
That's because of people who treat AI like a magical "give me a finished product" button. Someone whipping up PowerPoint slides tends to care more about saving time than maintaining quality - the (usually watermarked) stock photos of yore weren't exactly the zenith of artistic presentation either.
Like any other tool, the quality of output is proportional to the quality of input. You'll get much better results if you provide specifics (detailed description of what you want, reference pictures, iterative clarifications etc.) and have it work on bite-sized pieces of the process (outlining, shading etc.), with manual verification and touch-ups in between. But all that still requires time, taste and experience, which deflates the "AI will replace skilled labor" hype.
I didn't care for those either but they didn't have the signature AI art "look" and they're usually real things that an actual artist or photographer made for or sold to the stock imagery company, which curated their collection.
Not for long, if you so easily have caved in to using AI elsewhere. People are lazy. If you see that the 'results are nice', it's game over for your programming/thinking.
Waiting for the day the advice will be to "enjoy AI assistance in moderation"
They stopped requiring SAT and ACTs in order to get a student population more representative of the population in general. This obviously allowed students that were not prepared for college into the system.
If you do well in your math SATs you'll likely do well in math college. SAT scores and college GPA are highly correlated. No idea why anyone thought it was good to ignore probably the strongest signal of success in college.
I broadly agree with the premise. As a PhD student in Computer Science, I feel there are some significant upsides to my work routine. LLM access has made many new domains more "accessible" to me which I otherwise would be too hesitant in investing my time in.
For example, my area of research is computer systems which involves operating systems, distributed systems and more recently systems for AI. Within these, there is a wide breadth of topics/techniques one can employ and up until now, I have not gone deep into theoretical aspects of things like scheduling etc. But with access to LLMs, I feel like I can at least brainstorm from a high-level about these sub-areas that I am not well-versed in and the responses give me some relevant pieces to start exploring on my own, depending on what interests me more or the amount of time I want to spend on that sub-branch of a larger tree of ideas.
However, the one thing I do have skepticism is the lack of awareness of blind-spots when dabbling into areas that I am not an expert in, and taking the LLM's lead in applying such techniques to some systems problems that I am working on. I often feel that I am not aware of what alternatives exist that the LLM has not explored for me, or if the directions it has proposed really do apply or have corner cases/assumptions that break in what I am doing. On the other hand, when working on something I have good intuitions about, I am often correcting the model's assumptions and it back-tracks what it told me. Unfortunately, I cannot do that comfortably with topics I don't have good intuition about which limits my confidence in "if this is the right direction to pursue."
As someone with a PhD in CS focused on NLP (I started my PhD in 2018 just as Transformers were introduced), and with a strong background in distributed systems owing to the fact that I was a lead developer of an MMO before starting my PhD, I can definitively say that any surface-level understanding you get by interacting with an LLM, is just that: surface level.
If that allows you to target your deep dives better, then great. If instead your deep dive into a topic is purely through prompting an LLM, that will almost certainly end with little functional domain expertise.
The absolute best experience you can get is by trying, failing, then improving upon your past failures. Remove that friction at your peril.
We are already remote sensors and manipulators for the corporate and economic structures we operate under. You can't see it, but we are ants in a superorganism.
More evidence of the philosophical concept of 'technology is a life form.' Humans would be the perfect host, at least for the time being. They are certainly a willing host.
When LLMs and ChatGPT first came out, it struck me as obvious and dangerous to a deep thinker or a knowledge worker the answering capacity. So, from my initial use I did not ask them questions, I have always "done my own work" and then asked the LLMs to criticize that work. This has been an exponential ladder of learning, and my cognitive growth is personally noticeable. I'm not hesitating to scribble out calculus and work it out, as I need for my work, where in the past I'd have found some other way because I felt uncomfortable with my tip-of-my-tongue calc skills. Don't ask AI, do your own work and ask for criticism, and them improve your own work yourself. This creates a learning ladder that you will climb.
That's nice except when you work somewhere where more and more developers are pushed to pump out slop generated by AI as fast as possible. So far I am not there yet but I have plenty of friends in the industry who are basically 'not allowed' to code manually anymore.
I've been wondering if there would be a benefit to inverting how we teach subjects now. Previously we would teach from the bottom, and build up. Semi-colon goes here, curly brace goes there, and then build up to architecture, systems, etc.
But this doesn't seem to make sense when someone comes to a topic with an LLM in-hand. They need to know high-level techniques, architecture, best practice, etc. As they pursue the topic they start to get down into the details, although probably never learn to do it fully independently.
I quite like this view because it paints a somewhat optimistic way forward from where we are now.
You can ask LLMs about high-level techniques, and their answers will usually be good enough.
What you can't get from LLMs is the taste and judgment, which you can only obtain by having a strong CS base and coding manually for years.
High-level techniques were never a problem. You could Google tens of articles on this topic. They are useless too, it's like learning how to drive a racing bicycle from reading a book. Sure, you will know a lot about nuances, but you will fail miserably when it comes to a real race.
The other day I just wanted to loop through characters in a std::string to copy data to a new string with a few escape characters (sending to peripheral device). Simple enough task for AI. I got a coroutine monstrocity back, with copies to std::array and a range based iterator, since I specified C++23. If I specified C++11, I would have received a:
char p = src.data();
while (p)
{
…
p++;
}
I had the experience to keep calling out AI to simplify and downgrade the solution to something primitive, which ended up smaller, faster, easier to maintain. Juniors with real world experience would not bother, they’ll take the first working AI result.
taste and judgment, which you can only obtain by having a strong CS base and coding manually for years.
I disagree, the definers of taste; art and food critics, movie and book reviewers, don’t need to have learned the craft by doing. Taste is a separate skill.
No one seriously expects a food critic to be able to cook a Michelin-starred meal. The job of that kind of critic is to be insightful and entertaining, and it's very different to the taste required to create top quality food, which is a combination of solid technical skill and creative flair.
Taste in coding is a combination of insight, experience, native talent, technical skill, and flair. Tasteful coding produces clever but straightforward minimal elegant solutions that an average developer can't imagine but can adapt and maintain.
If critics were forced to be actually skilled at the craft of creation, the world would be infinitely better off. Both the cello and the player are better off by the cello maker also finally being the cello player. Alienation was a mistake and this part Marx of all people understood well.
This is why "critical thinking" is a meme. Being a critic takes no skill. I want far fewer critics and far more constructive thinking. GenAI being the ultimate constructor is a bonus.
I'd say taste is a consequence of lifestyle, which is learned by doing. And art critics often have bad lifestyle, which is visible in their bad taste. When art is virtual life, it would define a lifestyle, which is adopted by doing, in its turn producing taste.
> which you can only obtain by having a strong CS base and coding manually for years.
I hope this isn’t the case. It is the route I took, but it also doesn’t seem to be a likely route going forward. Strong CS grounding is feasible for sure, but I have a hard time believing that a meaningful number of people will be spending the requisite years coding manually.
I can't speak for other disciplines, but for math and CS, both with a really heavy focus on abstraction, the final result of learning is to build a nice intuition on top of the abstractions we find useful/expressive. And to build the intuition, the old, usual, and perhaps the only way is to see and practice a lot of concrete examples, after which the motivation of building some abstraction can be understood, and after which the abstraction itself can be fully grasped.
e.g. The "group" abstraction requires one see a lot of int, polynomial, modular arithmetic etc. before knowing why we want such a thing. It's unskippable.
This idea sounds good at first, but if you look closer, it would just make workers, not experts who really understand. What we could do, and already do, is tweak the learned abstractions. In our field, it's easy to see: most of us first learned about computing abstractions, not how processors actually work, or started with Java, not assembler.
Plus, you can't teach math from top to bottom.
> As they pursue the topic they start to get down into the details, although probably never learn to do it fully independently.
It's hard to claim one has mastered a subject without independent command of its fundamentals. A less charitable take on this future is that students only learn to hand-wave answers and correspondingly cannot evaluate statements beyond "sounds about right".
I keep trying to convince people that English majors and Philosophy majors will benefit the most from LLMs. English majors in particular, have been trained to be VERY exact in how they word things.
That awareness of how to structure the English language, it will benefit those who use LLMs.
Then again, maybe someone will just make a LLM that’s built to turn poor English and poor reasoning into excellent English and excellent reasoning. Maybe this is just a technical puzzle that needs solving.
I disagree with you, for the very reason you give:
> Then again, maybe someone will just make a LLM that’s built to turn poor English... into excellent English
That's already been done, for some (pretty weird) definition of "excellent".
I work with, or at least in the vicinity of, someone who is very good at getting work out of LLMs. He has a whole system of CLAUDE.md files and skill files and things. He makes TONS of typos. When I first saw that, I was itching to go in and fix them all, it seemed viscerally wrong to be adding an extra layer of correction required between the instructions and the LLM's behavior. But in practice, I don't think it mattered at all. The LLM didn't care. Typos in particular might require a bunch of RLHF in the chatbot, but my hypothesis is that the LLM is already mapping messy human input to the nearest surface of some high-dimensional manifold and the added noise of typos is inconsequential to where it ends up (as long as there isn't any real ambiguity -- though even there, you could probably construct cases where that would help rather than hurt!)
Typos are different from sloppy writing, but I think the AI companies have put a lot of work into training these chatbots on dealing with typical non-English major writing with all of its imprecision. Also, it's easier to construct cases where that imprecision and sloppiness would help rather than hurt: a mistake in the input that is common enough to show up in the training data is going to be a good match for the needed correction as well as associated corrections. The precise language could easily result in the LLM overestimating the user's competence.
That doesn't address whether an English major's careful composition would help for hard tasks where getting the specification right really matters -- perhaps that was your point? I guess it's an open question whether "boiling away the typos" and "boiling away a poorly articulated specification" are related enough.
I don't think you can learn high level techniques or architectures without first understanding the basics first. This means boring boiler plate coding.
I’m not sure. We’ve always had to pick the level of abstraction we start teaching at. Voltages, transistors, registers, assembly, C, etc. This feels like it could just be a progression of that.
Then its reasonabel to expect someone who is not using LLMs to have an edge in their cognitive abilies. Or will it be overshadowed by the shear magnitude of bruteforcing that LLMs are capable of. I tend to side with the former. If that would be true, then not using LLMs would give an edge in solving novel problems. But we have been dependent on tools, cognjtive and physical, since forever. We cant imagine a world without tools. Why would LLMs be discriminated as a tool
I have observed this in myself when I began to over-leverage AI in my workflows. I've since become more deliberate with what kinds of tasks I will use it for, although I still slip up.
With writing:
Things like brainstorming a plot line for a book with a custom GPT or Claude project that has all of my prior books in its knowledge? Works great.
Things like asking it to write a paragraph or chapter for me - I can rapidly feel my own writing skill, motivation, vocabulary, and ability to grasp/remember the resulting plotlines deteriorating. I don't use it for that anymore.
With studying:
I've been taking a couple of evening uni courses and the thing I found so great is that I've been forcing myself to think through the problems, and take my own notes in every lecture. I may then still get ChatGPT to help explain and reason through some of the concepts with me. And I have it review and 'grade' my assignments. But I refuse to ask it to start drafting answers.
With programming:
This one is tougher. When I am not very personally invested in a problem or codebase it becomes too easy to offload more parts to Claude, and when the company encourages 'vibing' to speed up velocity and you're reviewing and writing a higher influx of lower quality PRs, investment goes down. I still sometimes catch myself committing solutions I only _mostly_ grasp and the rest is hand-waving. A big part of it is a work culture thing.
For my own projects I make sure to understand and have a back-and-forth with the planning agent for each task, or write the first plan myself to go off of. When it comes to producing the code, I have to admit it is much easier to properly review parts of the codebase I am extra interested and knowledgeable in (backend in my case). The frontend I'm less well versed in and also admittedly less interested in, so I do sometimes fall into the trap of "Ehh it works, just commit it" with the goal of doing a thorough quality pass before actual release.
With all of the above, I can feel my ability to think, plan, reason, focus (and my vocabulary) suffer if I go over the line too much into agent offloading. For me keeping that balance is as much about maintaining my own long-term brain health as it is about producing good output. I imagine younger people growing up with AI today won't even know what that more capable (in my opinion) brain state feels like - to them, the AI-using brain will be the norm.
The place I've come to with AI for writing is to have an idea for a chapter/article/etc, which I take to AI, and tell it to either ask me a bunch of clarifying questions, or try to blow holes in it/challenge it. I'll keep talking to AI and answering questions/handling challenges until the AI runs out of steam, then I'll ask the AI to write out a condensed outline with all the pertinent details of the conversation.
Once I have the condensed outline, I'll re-order stuff, clean it up/tune it up, then do the final writing. This keeps my voice and logical train of thought while avoiding blank page syndrome and some of the organizational mess of condensing notes into an outline manually.
> many of them can no longer brainstorm, code, think deeply, or write
I believe this is the real crux of the issue. We often turn the target to things like "Can johnny Add, Read a book, or recite dates" which are only proxy measures for important things like "Can johnny solve a numerical problem presented to him, can he synthesize information, or can he think critically about what is occurring around him?" .
If students use AI to accomplish goals I do not see it an issue. If they cannot figure out how to use tools, or what their goals are-- that is a major issue!
An analogy of my point is that I don't want to focus on cursive in the age of computers keyboards, and I dont want to focus on abacus skills when a pocket calculator is like $5.
If students are allowed to use AI to accomplish their goals, then I think the real question is why should they go to an expensive university for four years to learn how to ask AI to do something?
very fair question. But that's on the university not the students, as in the faculty shouldn't be complaining about the students, but adapting with the times.
We’re in a world where LLMs are basically going to be extensions of how we think. An additional thing we use to do a lot of thinking tasks.
As a piano player, it’s important to work hands separately. Sometimes your right hand will carry the melody and your left hand the harmony, sometimes vice versa. Sometimes there may be more than just two “voices”/melodies/lines between your two hands. Even as a very good (as in getting paid to do it) sight reader, I learn a lot working all the voices/melodic lines separately.
Singers do similar things like singing only the vowels to keep themselves in the right placement. Learning handstands, you have to work your wrists, rotator cuffs, core (which is many things), etc. separately. Yoga, Pilates, and running also help us learn to break problems down this way.
Anyway, all that to say: If LLMs are gonna be a natural extension of how we think, we need to understand what parts of problem-solving LLMs are good for, and what parts our brains are for. The nice thing about working these bits “separately” is that one side is done for us. So we just need to consciously practice using our brains.
As programmers that means, maybe we conscientiously practice writing things ourselves sometimes. Remembering that this even if this sacrifices short-term “velocity” (whose measurement is problematic, but I digress), it preserves our long-term ability to do good work. And I think any of the above physical/artistic practices (or countless others), worked in these ways, will help reinforce this entire mindset.
I think kids of the coming generation will be sharply divided on their ability to conscientiously practice things separately. It’s been happening, but I suspect LLMs will accelerate it unless how we actually teach kids can catch up.
> We’re in a world where LLMs are basically going to be extensions of how we think
If that's the case then we're in trouble based on my experience. This week I've been using ChatGPT to help figure out some old linux platform that I need to resurrect. It's very good at quickly searching and surfacing relevant information online, and that's helpful, but if I did not have a lot of experience at linux administration to be able to see where it was suggesting the wrong thing, or initially dismissing the right thing, then I'd just be thrashing.
The LLM is helping me because I know what I need, and it can search and read faster than I can. But it's not really very smart.
> An additional thing we use to do a lot of thinking tasks.
Which is to say, an additional thing you're going to be forced to pay a lifelong tithe to a trillion-dollar company in order to do a lot of thinking tasks.
I’m rather optimistic about the future of smaller open-source models and market competition actually doing its job here, honestly. I myself, again, err on the side of doing things with my own brain. But there are many things LLMs are useful for, and they’re definitely better than a “rubber duck” if you don’t trust them blindly.
> Many of them can no longer sit quietly for even 30 minutes just thinking on their own
Plummeting attention spans has been a trend for much, much longer than LLMs and is more the result of constant digital interruptions and these days overwhelmingly social media and doomscrolling: https://www.apa.org/news/podcasts/speaking-of-psychology/att...
The effects on children have gotten most of the, err, attention, but the effects on adults are no less deleterious.
In 2002 I spoke with a lecturer in the humanities and he told me about how nobody was learning French at university level (in the UK). My own course had been cancelled due to the cost of teaching it, and the era of 'easy degrees' had set in during the early 90s.
Before that, I also noticed the decline in newspaper readership in the 80s.
It is easy to blame this general decline on the latest tech (or moral panic), whether that be LLMs or even the existence of the internet, however, the trend in dumbing down has been going on for decades.
In the context of a declining empire and financialised economies, this makes a lot of sense.
I think we are talking about two different trends that have similar symptoms. I do agree there has been a noticeable anti-intellectual trend for a long time, especially in the West. (See also: Grade Inflation.) But that is separate from the drop in attention spans, which is relatively recent has been pretty strongly linked to digital stimulation, constant multi-tasking, and now short-form social media.
LLMs are an entirely new dynamic with significant cognitive implications, but I fear it will be hard to discern their impact from the falling attention spans and other long-term trends that have led to things like grade inflation.
I dunno, I used wolfram alpha a lot during calculus classes. However my uni didn't require any homework assignments to be done and they did not contribute to grades. Only the exam mattered.
Maybe the problem is that doing assignments contributes to your grades? The answer from wolfram alpha wasn't so much to get the homework done, but to understand how I would be screwed in the exam.
As much as I hate to admit it, using agents for too long makes me less able to think for myself. I am dedicating 30 mins to 1 hour everyday writing and thinking without LLMs.
> I am dedicating 30 mins to 1 hour everyday writing and thinking without LLMs.
Before you did this, was literally every hour of your waking time spend thinking about LLMs?
I don't think I could do that even if I tried, and I spend all my development hours with agents, but during meals, showers, walking the dogs, enjoying a coffee outside or whatever, naturally I get time to think about other stuff, sounds out of the ordinary (to me at least) to have to dedicate 1 hour to not think about something. Reminds me of when I was addicted to amphetamines way back when.
> Before you did this, was literally every hour of your waking time spend thinking about LLMs?
They said "writing and thinking without LLMs", not "not thinking about LLMs". I think they're talking about setting aside time for fairly focused thought/work.
I don’t buy it. Properly leveraging LLMs to generate stable and extendable systems is mentally exhausting (i.e. highly demanding of intelligent thought), especially given the poor quality and churn within the harness ecosystem.
Now, if you’re creating trivial, unstable, or nonextendable systems maybe this doesn’t apply. And maybe I have long overestimated the work that SWEs have done.
i use claude a lot and i find that it is best applied in domains in which i am already a master. I tried applying it to domain's im unfamiliar with and i found that i produced stuff but as time went on i understood what i produced less and i almost felt like i do after binge watching a netflix show, 2 weeks later i barely remember any of the details. I wonder how much you need to "do" to learn and remember. LLM's give you a shortcut to doing and so you probably aren't learning either. It's like when you watch a professor write a proof and it makes sense while listening to the professor but at home you have difficulty deriving it. LLM's give me the same sort of feeling. I think the way forward is still going to be doing things manually to learn and using LLM's once you've mastered an area and people who don't understand this fact are going to slowly descend down a hill and forget how to depend on their own thinking.
I wonder if AI or something else changing (developing anxiety, etc) has made the pay-off of the degree less certain. If you're confident that your years of effort will pay off, it's probably easier to see it through. If you're worried that AI will wreck your industry before you hit the workforce, maybe the equation changes and you're more inclined to gamble with shortcuts?
Yes, and this is going to hurt everyone. If everyone knows that you can skate through a college degree without doing any work, it is not going to have much value as a credential.
I totally agree about school-level homework: it was many years before my pre-frontal cortex developed enough that I could have forced myself to do the work.
That said, though, one thing I don't understand about the heavy users of AI in academia and software development is that the thinking and coding is the fun part. And that's the part so many people seem to be so keen to automate away.
I'm right there with you. The thinking and the coding is the fun part. I'm pretty relieved that all of this is happening near the end of my career. To me, AI is just not fun. And constantly signaling how productive I am and having to show "my value" is exhausting. This is only my subjective experience, of course, but in many ways the world seems like the fun is getting sucked out everywhere, not just from AI. Like the type of people that become managers are taking over everything.
Depends on the person. I find that it's extremely satisfying to figure out a tricky problem on the way to that end result - to struggle with something for a bit, then finally fix it or fully wrap my head around it. So to me, it's a mixture of both. What I want is the end result, but in the past sometimes that came with thinking in the shower about an approach... Or a wild thought while going to bed that makes me jump up and grab my computer.
That doesn't happen for me anymore to the same degree.
I've read enough comments* on HN to know that there are different camps. Some people don't really enjoy the process of development and just want results. Meanwhile, telling me to automate away the problem solving aspect of software dev is like saying "you know you can just copy the answers to the crossword from the back of the book?"
different strokes for different folks. I'm def. in that end result camp, i get the biggest thrill out of seeing something work. For me, coding agents are awesome because i can bring a lot more to life in much shorter of a time frame. I do enjoy the process and problem solving of coding, it relaxes me. On the other hand, i really really enjoy when an idea i have is on the screen and working.
I'd argue that this is an adjustment period that society has to go through. The way we are using electronic devices today, in some years it will probably be looked at like smoking cigarettes. And I'd argue that a lot of the "decline" is due to a shift of skills away from things that mattered more in the past toward other things that are not measured/perceived by the older generation.
Interesting analogy. I believe regarding addictiveness they may be compared.
> a shift of skills away from things that mattered more in the past toward other things that are not measured/perceived by the older generation.
Do you have any ideas what these things might be? As someone in his twenties, I’m sometimes saddened by observing that some of the skills I acquired over a long time (e.g., writing, coding) may become obsolete or won’t be respected anymore just now that I‘m finally getting good at them.
it happens, things change and the change is only speeding up. I think the real skill to have going forward is the ability to acquire new skills. I tell my boys "get good at learning and you don't have to get good at anything else".
Ages ago I had similar thoughts. Everything changed when I came to terms with the concept of change being the only constant. A bit of a cliché, perhaps, but profoundly true.
Eh, I think it's less like a cigarette and more like the car. We're not going back. Americans are famously less healthy the more car dependent they are, and now people walk/run as an explicit task to be healthy. People will start going to a "thinking" gym, or engaging in additional manual mental activities for sport, like we do with chess today.
This is an age-old argument actually, the same one was raised when the printing press was invented and reading became a more generally available skill.
The Internet in no way made memory obsolete. People who know things off the top of their heads are far more capable than people who have to look things up.
Funny that you mention that. A month ago I started the Duolingo chess course, and just yesterday I noticed that my brain is clearer, more capable of deep thought than it has been in years. It's like stepping out of a fog. I also started CPAP recently, so it's hard to attribute the change to either, but I feel certain that the chess helped.
The interesting thing about jogging is I do my best thinking while jogging. I've found it impossible to do deep thinking while driving, as driving evidently requires higher functions of the brain. Jogging doesn't require any of that, I can jog deep in thought and have no recollection of the previous mile.
Apparently Petri was driving when he figured out a new fingering for a Chopin Etude he'd use at a concert later that evening[1]. The passenger survived to tell that story, so I think this is more about your correct risk-assessment than physical limitations of the brain. :)
1: Just to unzip: "I don't like this fingering; let me imagine playing with this other fingering; yep, that will feel more comfortable and stronger; I'm now confident enough in the change that I'll do it that way in public, for one of the most difficult piano pieces, without ever having practiced it physically..."
The idea that most people have the discipline to keep themselves mentally in check is false. We already know this! Millions and billions of people who spend hrs a day consuming media on platforms such as instagram.
> I'd argue that this is an adjustment period that society has to go through.
I used to think like this until social media proved there are some tech innovations we just can’t adjust to. 10 years ago you would’ve never caught me supporting any sort of age based social media ban. Now? I don’t think it goes far enough. Fake news (actual fake news) and misinformation has only gotten worse with it as well. It’s so destructive.
The human is designed to interact with small groups, to understand several smaller groups, and perhaps to imagine a big group of smaller groups. In a literal sense, let's say 100 people per group. At that level the human can actually know and interact with them still. In a city of 100.000 it's still managable to feel you are related and involved to this group-of-groups. In a city of a million, you'll revert to only your own small group and have lost the connection to the collective.
The same goes for speed and quantity of input, as to what the human is designed for (not literally designed). Be it social media with it's infinite scrolling, cars racing by as opposed to looking out the window a few times per hour because you see someone/something, constant sound input if you live anywhere remotely busy or work in a busy office.
The point I'm trying to make is that the world used to be comprehensible for the human. Some understood a little complexer things, some only the simpler things. Now there is an overload of everything. So, most humans are in survival mode wether they know it or not. Hence the many seekin mindfullness etc
No matter, it's an observation, not a judgement or opinion on it. The world will just keep rushing forward. Some have a slight hand in the direction it goes for better (never) or for worse, but spiral it will.
I think there’s a major part of this conversation being omitted, though I am not saying you did it intentionally: “the attention economy.” We have gone from advertising to a system of creating addicts for profit
Definately agree, that was included in the "or for worse" in this sentence I wrote. As for creating the addicts, nobody had a masterplan. It's all the pieces spiraling together,
>> The world will just keep rushing forward. Some have a slight hand in the direction it goes for better (never) or for worse, but spiral it will.
The systems are too large and self-propulsing for anyone to really control. Consider the rainforest. How many millions of variables interact, nobody is in charge, everything influences everything in a billion different ways. You might say, well we can cut it down, so kind we can control it. Allright, let's continue to spiral. You might build a city there after a few years. Still in charge right. But it get's too hot because there's no vegitation, so you have to change again. And then we find that people keep getting strangely sick, and scientists find some special mushroom that survived and apparantly thrives on the mix of cut trees and diesel fumes and their spores in the air are poisonous. I made that up, but you get the idea hopefully.
LLMs have killed my facility but not my knowledge.
I can still read code and write it, I just need to look back at docs a lot more, when I used to just know things. I also have to sit and try to recall how to do things and what abstractions are involved more. I also have more "writer's block" when starting with a fresh program/document if trying not to get AI to seed it with a baseline implementation, where I have to sit for a while thinking about what I really want to build.
Yes, I can churn out a lot more stuff as can most of my peers. Experiments etc are all way faster to run with coding agents. But I think the overall creativity and originality is a lot lower. I think this is what many people are facing, if you don't use LLMs your short term productivity is worse.
There’s the saying that we overestimate what we can achieve in the short term, but underestimate what we can achieve in the long term. Optimizing for the short term is therefore counterproductive if it impairs us for the long term.
They're incredibly more productive. LLMs are amplifiers, so where they'd have branched and tried out N things, they can easily try 5N pathways of RnD. LLMs are extending the frontiers of science fast -- math -> phy -> chem -> bio in that order.
In my own experience, the only path I truly gain intellectual benefits is the one where I work closely with the LLM, test very narrow hypotheses, and leverage it for learning over producing.
Trying 5N paths is useful and sometimes yields interesting insights I’ll retain, but it’s not the rich, challenging, deeply engaging kind of process I find I need in order to develop useful knowledge and skills.
So yes it’s an accelerant for people who want stuff from me, but that doesn’t map directly to learning and building skills. I think that mismatching is really important.
To help learn I use LLMs to generate practice exams for whatever I'm trying to learn, then on the questions I struggle with have the LLMs explain the logic and point out my mistakes. I haven't been in college for over a decade, this is just for topics I'm curious about and want to learn. For any serious topic I recommend auditing the practice exams with a different LLM than the one used to generate to help reduce hallucinations. Seems to work well for me. I quite like reading the "thought" processes shown by DeepSeek.
I don’t see these at odds. Sometimes through working closely with an LLM, N paths emerge. Having it go off and test each with defined metrics to determine which is better is the natural follow up. Even better if you dive into the why it ended up being better which the LLM seems to be able to expose well in a lot of cases.
The part I find weird is all the claims that LLM usage leads to less thinking and exploring and just grabbing the first result. I constantly find myself going off on tangents and pulling on threads when I’m working with these tools. Is it really that different than before when my “peers” weren’t able or willing to be curious about their craft? They didn’t explore other programming languages out of curiosity or for fun? That covers literally 95% of all software developers I’ve worked with in the last 24 years across many domains. To them it’s just a job. Their only goal is to deliver tickets assigned to them and go home. They rarely go out of their way to learn something new unless the company assigns them some mandatory courses. Largely the LLM is capable of producing better and more consistent results than they ever could in the first place.
I don’t know how to cultivate curiosity in the work force. Maybe it’s not possible and you have to filter aggressively at the hiring step. But then your pool of hireable candidates shrinks to a few thousand developers most who are probably not actively looking for work.
You’re right, they don’t have to be at odds. Before LLMs, there were jobs where I had to power through and make a sort of ‘minimum effort’ approximation without applying much analytical or investigative energy or skill. This isn’t a lot different from churning something out with an LLM. There’s not much to learn, the end product is mediocre, it’s more of a rote path.
The only distinction I wanted to make is that the learning doesn’t come by default. Yet that was largely true when people copied mystery solutions from stack overflow and used black box libraries for 90% of the complex work their programs facilitated.
Perhaps not much has changed but we’re now operating at a much larger scale and the opportunity to not be curious is actually more present than ever.
People who are curious are massively benefited by this tooling, in my opinion. Like you’re saying, if you want to investigate and learn, there has never really been a better time. If you’re sincerely applying yourself and pulling all of those threads, there has never been a better teacher.
I’ve wondered about the matter of finding and cultivating curiosity too. I’ve come to believe most humans, let alone programmers specifically, are not all that curious. A lot of us are path-followers and we’d rather not get into the weeds most of the time. Then some of us see weeds and dive in, even when it’s not pragmatic to do so. I don’t know how much it can be cultivated or even removed from a person who has more than enough.
I'm hearing different from PhDs. The bottleneck with much research isn't "trying out ideas" so much as it's all the bureaucratic minutiae, grants, mentoring PhD candidates, collaboration with other researchers, etc.
I've heard LLMs can be helpful in limited targeted ways. But not as some kind of "game changing" accelerant.
It’s creating a daemon and machine spirit filled world of Warhammer 40k. We already scarcely understand how the world works, but LLM use actively degrades cognitive ability that way it is used by a majority of people (The bringing a forklift to gym analogy).
To me it is crazy that you are being downvoted. My experience in academia was that an incredible amount of time was devoted to data cleansing analysis, coding, etc., which were completely non-core to the actual underlying academic pursuit.
There's an unnecessary feeling of fear that permeates any factual conversation on LLM's impact on science and engineering. You can just view the practitioner over the shoulder and see all the things they're able to do in a minute that would have taken days.
The downvotes are just a sign of the times. It's also something to observe and think about..
Yeah. I have no doubt that I would have used LLMs “just this one time” to help with problem sets or papers when I got behind or wanted to do something else.
This is all assuming tests measured anything valuable in the first place. In my experience standardized tests were always flawed and most of my peers knew shit about the subjects they passed in top % a year after. If AI breaks the current education system that's a win in my book.
I don't understand why people take shortcuts in school. You pay a LOT of money to be there to learn. Taking shortcuts seems completely counterintuitive to me.
- Time is a scarce resource. Students do what they can to learn what they can, but if they're under the gun, they'll take the path of least resistance to make it to the next day (totally not like the business world, right?)
- In the interest of having well-rounded students, a lot of degree programs include subjects the student didn't want to sign up for, but have to. Even in something like CS, I knew a lot of people who liked the hardware side of it, but didn't like the software side and vice versa. So I can imagine a student justifying taking shortcuts that way.
- Psychological reasons like wanting to protect their ego. Maybe they had always done well in school and are now struggling, but don't want to ask for help, so they think why not just take a shortcut here and promise to do better next time, etc., etc.
A lot of people view it, rightly or wrongly, as paying a lot of money to earn a degree that opens up certain opportunities, while learning is secondary, so minimizing effort is worth it.
And to some people, it's not even a lot of money.
In many ways, schools are just the modern day peerage system.
> Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well
I noticed this before LLMs became a thing. It was by accident. We had a team of programmers. All decent at what they do. The management said 'hey you want to learn another language we are going to be using it for these upcoming projects'. So we set up a self learned at your own pace class curriculum. Maybe 10-20 hours of school work if you sat and really dug in. Maybe 3 to 4 hours if you breeze thru it and do not care much. We set up weekly check-ins doing about 1 hour a week. Easy. Watch a 20-30 min of vid 20-30 mins of do homework come to check-in and talk about what you learned and help others if needed.
Now this is where I was disappointed. The first 'class' was 40 people. By the last there were 3. Those 3 I noticed always are the ones who dug in. The rest wanted a proctored classroom and someone to tell them what to do.
Actual genuine curiosity is rare I think. We have a lot of people who are decent at what they do. But do not really care about it. IF you do not care you are going to just push the button and get the answer.
I could see myself dropping out even if I was interested in learning. I'd suspect that the time spent would end up with me needing to stay late to make up for it or being penalized in some other way.
I think it varies tremendously from one role to the next. I'm a senior software engineer and LLMs, the way I'm using them, improve almost everything I do. I use them to write most of my code now, but first I spent twenty years writing code before LLMs came into existence and second writing code is like 5% of my job. Most of my job is research, investigation, and architecture. I treat LLMs just like a junior engineer. I give them clearly defined jobs that I could do on my own just fine, that I already spent years doing. The problem here is that students are using LLMs to automate everything BEFORE they become proficient at it themselves. Letting college students use LLMs for homework is like letting kindergarteners use calculators instead of counting on their fingers.
You cannot tell me that letting anyone do something for you does not affect the skills that you outsourced, unless you are some sort of a superhuman.
As an example, I have been drawing portraits for quite a few years now, and whenever I go on a hiatus and come back after a few months, I can notice my skill not being anywhere close to where it was before I stopped using it.
Sure, after 2 or 3 portraits they mostly come back because of the previous experience, but skill rust is a real thing, and if you think your coding skills are the same because you used to code 20 years but haven't coded for some time, you are probably just lying to yourself.
My "digging for roots to eat" skills have also atrophied. Fortunately I don't need those much anymore because of modern agriculture.
I wonder how much of this "you are gonna lose your skills!" stuff matters. And if knowing how to properly iterate a for loop with my eyes closed matters all that much anymore.
On the contrary, with the amount of times I went to ask for help and was failed pedagogically, plus not being able to afford tutoring like my peers had, I think access to an LLM would have genuinely boosted my grades.
I still did well, but I had gaps for which there was no help outside of the internet available.
The risk or difference is that tutoring helped people learn which they can use to do the work, whereas with only one or two different words an LLM will do the work (that proves you have learned) for you. A tutor has limits, but an LLM needs to be asked to set limits. And especially younger people are less likely to "punish themselves" like that.
I recently switched back from a Tesla to an older car without permanently having a map visible. Suddenly my brain has to think about routes again and it definitively feels like my brain has to put in more effort again to handle it.
This likely varies person by person or the way people adapted AI. For me AI replaced the boring part of writing code, but has not replaced the fun part of thinking about code and problem solving.
before AI was around we blamed Covid for doing this to us, and now we blame LLMs... and before that we blamed social media. I'm pretty sure this downtrend has been happening for decades.
When reading and writing became prevalent, the ancients bemoaned our reduced facility to memorize long texts. Are we now “less smart” because of that technology?
No, that's the quality of candidates. I wish I was joking as a PhD holder for only 15yr.
A lot of skill of
is getting bled into the private sector because getting the PhD in a lot of regions doesn't mean the step up it used to. A lot of that comes from awarding them to layabouts doing "a gender critical analysis of ...".
Industry doesn't how/what/why they just wanted the 3 letters as a performance barrier to hire competants.
Yeah, it's a scary thought. I feel the pull of it every time I'm stuck on a code problem that I don't want to search solutions for and hand-code... and I also feel myself wanting to reach for the crutch of an LLM when I just have something boilerplate and easy to do. It's incredibly tempting to just ask the question and have the "thinking" done for you. Until you have actual skin in the game and realize that it doesn't reason, and its "thinking" is utter shit. Then it's like: you got addicted to cigarettes and now you have to quit, because this habit is poisonous. It really does lead very quickly to cognitive decline if you rely on them, or even think about asking them while you're writing code.
> Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well; many of them can no longer brainstorm, code, think deeply, or write without an LLM present doing 90% of the work. Many of them can no longer sit quietly for even 30 minutes just thinking on their own, which is a required skill for producing original thought.
This was my experience even pre-LLMs though (about my own PhD thinking skills too). I blame the amount of random stuff work now involves more than LLMs.
I used "AI" in the 2000's to increase my homework assignments, and to correct them.
As in I wrote code to generate random exercises, with solutions, using many tricks, to get myself hundreds of problems instead of 1 or 2.
Often spent more time on getting these programs right than on the problems. Still did better than the class. Oh and it was AI in the 1980s IBM sense. Ie. it was based around a python version (which I wrote) of a LISP math system based on maple. I even attempted (and largely failed) to rewrite it in C++.
Even attempted to have my homework read to have the computer correct the actual pages, but I never got convnets to reliably read entire lines (yes, I understand, well now, why a convolution would mostly not realize whether 2 pieces of text are on the same line or not and so get very confused if you go deep enough for recognition to work well)
You make good points here. But I want to point out some issues that I have with what you are saying, because I see a few assumptions that I would not myself make.
I graduated from RPI with a degree in Management and a concentration in Information Systems. I began in Computer Science, and didn't like it because RPI CS at the time was loaded with professors who were mathematicians who had transitioned over to CompSci and because the 100 and 200 level courses were excessively math-heavy in my view.
Since this was the late 80s, there may not have been an easy way to teach B.S.-level computing without it being heavily math-based, but I digress.
No matter what degree we achieved or what work we ended up succeeding at, we have a tendency to look back at people rising in the ranks below us, see differences in their experiences and struggles, and say, Look! That is evidence of a lack of rigor or a lack of understanding of fundamentals that we had to learn in order to succeed.
The only thing is that some of what we learned to become successful just isn't necessary to be learned when we learned it.
I do a fair amount of low-level software engineering with Claude Code now that was above my level of understanding of data structures and algorithms because I never took those CS courses at RPI because I switched to Management IT.
But as someone who could be described as a solopreneur at some level, my new system designs reach a certain level of complexity or code maturity, and I hit problems that I would not hit if I had more understanding of data structures and algorithms.
So-- I end up having to learn aspects of those disciplines at that point, rather than before I actually needed them.
I run into these situations often enough where I now say to myself, gee, I wish I had taken Data Structures. And I think, could I effectively take Data Structures at this late date and get better at specifying how I want data stored, or perhaps knowing the shortcomings of simplistic database structures that are the ones I end up with initially because of my lack of spec-writing skill?
Aren't many of the less experienced folks who come up now, whatever age they are, going to hit problems that show them their weaknesses in this fashion?
Is the issue that these people will never get jobs because the seniors and managers who are interviewing them will design interview questions that keep people with their level of understanding out of the workforce?
What happens when somebody who sucks at the fundamentals but is really motivated bangs their head against their shortcomings and eventually succeeds in building something that takes off? Aren't those people great assets because they learned some of their critical skills the hard way?
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
As a counterpoint, I was once a physics grad student. I didn't finish the PhD because at some point I discovered that I was not going to be the next Richard Feynman and this was too much for my ego at the time. But I think that if LLMs were available, I might have finished.
Part of my problem was that at some point the math transitioned from stuff I understood to symbols and notation that I knew how to manipulate but didn't really understand. LLMs could have helped bridge that gap.
On the other hand, it's hard to imagine I wouldn't have used it for Jackson, etc. but we got Jackson solutions from previous students and the internet anyway. Using LLMs probably would have been more effective, used correctly.
This is also where I had issues advancing in math. For so long I was able to build intuition around mathematical concepts easily. They fit in my head and made sense. I couldn’t understand why my peers were so bad and slow at picking up the concepts. Until my first calculus class where there was absolutely no focus on the intuition or practical utility. It was just formulas for the sake of formulas as exposed by our teacher.
It wasn’t until I was curious enough to learn about calculus outside of the classroom that I was exposed to things which helped develop that intuition and made the calculations something other than just symbols and equations to memorize.
I think this would be fine as an adult, if it meant using LLM to churn out the boring work required of you at a corporate gig, to spend more brain cycles on something you actually want to work on.
The problem is that it sounds like many people are just using it for everything.
> For adults the cognitive decline won't be as measurable since there's no exam
I think this is true of every affliction that adults criticize children and teenagers of
I’ve been out of university for a very long time, and I took a community college course and for the first few sessions I couldn't focus or sit still at all. Fortunately I knew that was abnormal and how to conform to a prior version of myself, but I don’t think children have a frame of reference.
>Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well;
tomorrow most regular people's thinking skills will definitely be weaker than those of the LLMs of tomorrow. And physical skills in most cases will be weaker than those of the robots. That leads to the question - what would most people do?
I've got mixed feelings on AI assistance. I'll relate 2 anecdotes.
1 - When I was in grad school (before AI), we had to use Canvas for a class. One day, I got an obvious spam/phishing email in the internal Canvas system. It was so strange. The writer just would randomly hit the capslock button and keep typing away, no salutation, no signature, just a real mess. They were asking for a particular professor to come to their house to teach them about ... something? Again, real strange.
So, I email IT and say 'Hey, somehow a spammer got into the system, do your thing'.
They email back and go 'Nope, it's a student, that somehow managed to CC the entire system, sorry about that'.
Dear Reader, the message was pure garbage. Literally, it looked liked it was written by a 3rd grader without any shame. [0]
I happened to know the professor of the class. So later on, I talked with them over symposium coffee about it. They said that they remembered that particular email because of all the IT back and forth. It was for an upperdivision class in the Engineering department. The email itself was not particularly notable otherwise. In that, they saw such emails all the time, in terms of quality. This was a top 100 ranked (whatever that means) university, by the by.
Shocking.
2 - My grandfather was an officer and a mechanic for the USAF. A bit of an odd combo, but he was partly responsible for instituting many preventative maintenance checks and protocols, novel in those early days of the AF. His aptitude and memory were quite sharp for many mechanical things. Until the strokes from decades of smoking caught up, he could tell you exact measurements and torque values for a variety of airplane related things (I can no longer remember what exactly, the memory skills did not transfer to me).
I do vividly remember standing in that light blue garage of his and him all but yelling at me once. We were looking at the brakes on an old car he was 'restoring' (getting away from Grandma for a little bit). He pointed at the old drum brakes on the axel.
He asked me how tight the pads should be on the inner rim of it.
I had no idea.
So he asked where I might find out.
I figured I'd ask him.
But what if Grandpa wasn't there?
We'll I'd have to look it up somewhere (they had no internet).
Fantastic. Now, what about the next time you're working on the brakes?
Well, just make sure that the pads are at that spec.
And that when Grandpa hit me with the nugget of hard won wisdom: No, you look it up every time. Because these are brakes, and if you are wrong then they might fail, and they might fail when the driver has their whole family in the car at 100 mph. And then because you were lazy, half a dozen people die.
---
These two times stand in my head when it comes to AI.
For the first one, yes, AI would be such a boon to that very clearly struggling student reaching out for help. It would get them back on the path to the real struggle of getting their degree. That level of assistance would be like a wheelchair to a paraplegic.
For the second anecdote, AI is condemning people to death. Using it in life critical situations and care, letting it hallucinate or skip over critical values, that's a recipe for disaster.
Where do we set the fine line of using AI and not? For brakes and X-ray machines, obviously not. For helping kids learn to write emails correctly? Sure, sounds great.
Unfortunately, I feel the old adage about regulations is going to be true here like it is with every new technology: The rules are written in blood.
[0] therh lasdkjenail EMAil was sposmkehting alLIKE HTIS IN THE WAYS THAT IS aw adklhjf writtene sand htenn. sned t sent ASLAOSNG ASLIONG ALONG TO NTHE recipentnts .
I doubt it. I'm stupid and I use LLMs a lot but I can still meditate for 30 minutes.
But apparently some of the smartest people in the world have lost the skill? But the commenter haven't, because why, they're 15 years older and thus immune to the same LLM-effects?
Plus, the issue with people having trouble sitting still for 30 minutes precede LLMs with decades.
Why is it so hard to believe? The young adults now have grown up with short form media and instant gratification / dopamine hits from apps. It's vastly different than people of the same age just a few years ago.
Not saying everyone else is immune, but those a few years older have also had a period without it.
I didn't say I'm immune to those effects, I'm including myself in this as well. (also, I'm not older than my colleagues).
Most people definitely can't meditate for 30 minutes, so if you can do this, it's very impressive. Regardless, being able to think about poorly-defined problems and build completely new mental models from nothing is genuinely a really hard and uncomfortable task. If you don't use the skill you'll lose it.
> Most people definitely can't meditate for 30 minutes, so if you can do this, it's very impressive.
Maybe not traditional meditation, but I have no problem taking a 30 minute plus walk with nothing but my thoughts. It’s actually when I do most of my thinking. The other is in the shower/sauna where devices don’t work anyway.
> I'm stupid and I use LLMs a lot but I can still meditate for 30 minutes.
> apparently some of the smartest people in the world have lost the skill?
> But the commenter haven't
> why?
Perhaps because a correlation you assumed was there (more smartness = more ability to sit still alone with one's thoughts), is not actually as strong as you thought? If one does not start with that assumption, there is no inherent conflict in the 3 pieces of evidence you cited.
Or perhaps because you are smarter than you give yourself credit for :)
Are you saying you leave it up to the LLM to judge whether your idea is good or not? Are you even human anymore?
(I am not saying LLMs can't be a good tool in evaluating ideas. To me, it sounds like you're firing off ideas all over, letting the LLMs judge what's good and what's not. Insane.)
Not judge no. Implement and create a working MVP, yes of course.
And yes, I fire off ideas all over. Many require predicting the future to decide what to focus my individual effort on. This is a terrible way to do things because humans (and LLMs) are notoriously terrible at predicting the future. The gold standard is to try everything and eliminate what doesn't work. This is impossible using human labor. With LLM labor, it's simply a matter of relatively cheap money.
It's amazing. Technical problems are now no longer having to predict what the best implementation is. You can just try each one.
Again, no need to have an LLM judge, because the metrics that define 'better' are well-defined, and this is the interesting part of computer science, not the implementation.
The likely 'real' reason is hidden in one paragraph within the article and has nothing to do with the implication of the eye-catching title: "Both Garcia and Ranade have joined more than 1,300 UC faculty in signing a petition calling for the reinstatement of ACT and SAT standardized testing scores for STEM admissions in the UC system. The petition and its accompanying open letter detail similar concerns with students’ mathematical preparation."
Around COVID times many top universities experimented with removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere, with many, if not most, universities already reversing it. As Yale put it, "Yale’s research from before and after the pandemic has consistently demonstrated that, among all application components, test scores are the single greatest predictor of a student’s future Yale grades. This is true even after controlling for family income and other demographic variables, and it is true for subject-based exams such as AP and IB, in addition to the ACT and SAT." [1]
That link is for an archive because that page has been removed. That's because they briefly experimented with a new 'test flexible' strategy where they allowed students to submit test scores or not, but then scrapped that altogether and went back to simply requiring test scores.
Berkeley chancellor told students to vote for 2020 California Proposition 16, which would've repeated 1996 Proposition 209 that banned race-based admission in public universities. Prop 16 failed. Subsequently, Cal started ignoring SAT/ACT scores. I have to think this was their alternative way of taking fewer Asian students, who average highest on that. Soon after I got an email from the same chancellor praising the change for bringing more racial diversity. The email included before and after numbers where % Asian decreased and all others increased.
Yes, they falsely classified him as white in order to deny him the promotion (because discriminating against white people is 100% non-controversial in that environment)
They could have easily made test scores a pass/fail per program and not weight higher scores for admission purposes. It achieves the goal of ensuring students have requisite knowledge for the program while not favoring students who are able to ace the test.
Or, even better - just expand programs so they can accept more students who pass the test. This would probably improve diversity without artificially restricting access to highish performers.
Please see the graph "Growth of the Math 2 Population by Major (2019-2024)". UCSD's Math 2 class is remedial high-school level maths. It has grown from under 100 students in 2016-2020, to more and more people each year starting from 2021.
UCSD tested the people who took this class, and 25% of them could not answer the question "Fill in the box: 7 + 2 = [_] + 6" (with only pencil and paper allowed, no calculators or other electronics)
I'm suggesting the problem is not limited to Berkeley. They both show the same underlying issue, there's a growing number of students attending university without the prerequisite maths skills they need to succeed.
It seems they're now at the point where the sheer number of students that need improved maths skills overwhelms the staff, resulting in them failing.
I'm having a difficult time imagining how an admissions event in 2021 materializes in the spring semester of 2026 in a class largely taken by first-year students.
In addition to overreliance on AI, Garcia also pointed out that many students are underprepared mathematically, a concern echoed by campus associate teaching professor Gireeja Ranade.
From the article discussed the other week:
Over three years — from fall 2021 to fall 2023 — the letter said, at least 20% of Berkeley first-semester calculus students who took a diagnostic exam showed deficits. “Basic mathematical fluency is analogous to literacy; without it, success in university-level STEM becomes structurally unattainable for students,” faculty wrote.
It's been steadily getting worse. The current article only looks at F's which conveniently hides if there has been a slope down. Additionally, kids entering HS in 2021/2022 would just now be hitting college.
A sudden materialization is what's depicted by the data.
> It's been steadily getting worse.
I don't believe this is accurate. Failing grades are what the observation entails, and the data clearly depict an abrupt change; not a gradual one.
In the section titled "Failing grades in 3 CS classes skyrocket in spring 2026
", there's a clear jump in failing grades for all cited courses between 2025 and 2026. Failing grades for every course jump by multiples of the previous year.
The jump is very likely due to AI usage and lack of skills in mathematics. It seems like prerequisite classes are not being fulfilled.
"Ranade said students are expected to enter the course having taken classes on linear algebra, vector calculus and mathematical proofs. However, she found out in office hours that many students struggled with linear algebra, and was even more shocked when one student told her the linear algebra class they took at UC Berkeley had an “open-internet, open-AI policy” for homework and exams."
Also, this professor doesn't grade on curves? Could be very specific to this teacher. I don't know. Would be great to have more data but it is a big jump and could be very specific to this professor or perhaps this class.
"Also, this professor doesn't grade on curves? Could be very specific to this teacher. I don't know." Someone has to hold standards up -- they seem to be falling down across the board in education.
Actually, when I read they usually graded on a curve, I lost all interest. I don't respect teachers that grade on curves.
You should be graded by how well you know the material - not how well your peers don't know it. I'm always grateful both my undergrad and grad professors didn't curve on a grade.
In my first company, I had 4 different jobs. It was a common adage: Go into a low performing team that does simple work and you'll get promotions much quicker than in a high performing team doing challenging (but fun) work.
It was right. I had 2 "dream" jobs where I did cool, challenging stuff, but where everyone was more than competent. They turned out to be career killers. The promotions I got were all in the other 2 jobs where I did boring business logic coding, and where my peers were barely competent (one had trouble navigating directories using the command line).
That's what happens when you grade on a curve. Smart people begin to work on boring stuff, and not the real challenges.
For failing grades sure, there must be some sort of minimum competence. For sorting out >= B/3.0 grades, a curb can work since you are getting evaluated against your peers to see he is standing out vs just doing acceptable.
If you wanted to grade purely off a curve, you would be stuck with old test problems that were thoroughly vetted and calibrated, an impossible task for smaller classes where the material changes rapidly.
> For sorting out >= B/3.0 grades, a curb can work since you are getting evaluated against your peers to see he is standing out vs just doing acceptable.
I'm still not getting it. For a standard course, the criteria for what is "good" vs "great" should be pretty clear, and it should be independent of your peers. You have a syllabus, and a set of abilities for each grade level. If you hit those targets, you get the grade. If half the class gets an A, then it means they're pretty smart, or you did a great job in teaching. Of course, there's the chance the class was too easy, but you can always fix that.
No, I don't see why you're stuck with old test problems. For standard engineering classes, there's a huge (almost infinite) set of problems one can create.
For smaller classes, grading on a curve is even sillier, as the variance is always higher when the population size is small. For example, a lot of my small classes consisted of highly motivated students (all "A material"), because they're usually obscure electives where the content is challenging. You then pointlessly penalize students who sign up (just like they do at work). In fact, my professors were usually much more lenient on small classes for this very reason (i.e. lowering the standard needed to get an A).
I once took an Intro to Analysis course. It was moderately challenging. I got the highest score in the class, and my grade was A-. Everyone else got B+, B, or lower. A friend of mine (who didn't take the course) got really upset that I didn't get an A (or A+) given that I was the top scoring student.
But I knew my level of understanding/performance. It wasn't that great. I felt even an A- was too high a grade for me. And the teacher did a pretty good job in teaching. Why should I get a higher grade just because the other students were worse?
> For a standard course, the criteria for what is "good" vs "great" should be pretty clear, and it should be independent of your peers.
Do you think upper division college classes are somehow like high school classes with well developed curriculum and teaching professors who teach the same thing every quarter? Now you expect the professor to not only come up with new test material, but also extensively calibrate it before students take it, maybe for a 15-hour per week class (3 hours of teaching + 12 hours of studying), with maybe 15 students? Well, thank God we have AI for these kinds of things now.
Ok, let's exclude upper devision classes and just focus on lower division courses (since you mentioned an Intro to Analysis course). Here you have a relatively better chance of a well understood enough curriculum and testing material to actually not grade on a curve. BUT these are also usually weed out classes, with the idea that they only have N spots for students to proceed on to the upper division course, so curving serves an actual purpose that is aligned with the intended result.
> Do you think upper division college classes are somehow like high school classes with well developed curriculum and teaching professors who teach the same thing every quarter?
I repeatedly said "standard course", which implies it is a commonly taught course (be it upper or lower division). In my undergrad, Analysis I, II and Abstract Algebra I, II were upper division courses. In the engineering departments, stuff like Electromagnetics I, II were upper division.
Anything that is not an elective (and even some popular electives) were standard courses.
Now I'll grant that in CS, some material like machine learning changes rapidly. But in most engineering, very little in the undergrad material changes. Even my semiconductor courses in undergrad haven't changed much in decades.
So yes - for most of those classes (and that means the vast majority of undergrad engineering) classes, the curriculum is relatively standard.
> Now you expect the professor to not only come up with new test material, but also extensively calibrate it before students take it, maybe for a 15-hour per week class (3 hours of teaching + 12 hours of studying), with maybe 15 students?
First: In my very average undergrad university, professors were always careful not to reuse old homeworks/exams. It wasn't a huge burden. Professors who don't do this (e.g. most professors in top universities) signal very clearly their lack of interest in pedagogy.
Second: You want to do a curve on <= 15 students? Are you aware of basic statistics and the problems you get with small N? Are they using a normal distribution or one that is more appropriate for small N?
And as I already said, for a lot of electives where the material isn't standardized, professors lean towards lenient grading. They offer those classes because they want people to take it, and grading via a curve discourages it.
> since you mentioned an Intro to Analysis course
That was an upper division course. Yes, I know some universities have it as a lower division, but many (most in the US?) treat it as upper division.
> BUT these are also usually weed out classes, with the idea that they only have N spots for students to proceed on to the upper division course, so curving serves an actual purpose that is aligned with the intended result.
It was not a weed out course. Neither my undergrad nor grad math departments had weed out classes. I saw that concept only in the engineering departments. My EE department had only Circuits I, Circuits II and digital logic as "lower division". Circuits II was the weed out course, and you were not allowed to take anything else (e.g. E&M, Electronics, etc) unless you got a B or higher.
SAT/ACT math is incredibly simplistic and at worst maybe contributed by not filtering as many out. Math scores have been declining nation wide for decades now, that’s been a big issue for a while.
One big reason is preparation, people start preparing for tests 2 to 3 years in advance. And the method of testing influences exams used in grades before as well.
So assume 4 years of high school and someone that just came in. They are still preparing for SAT like tests in their first year of high school. Someone in final year of high school is well trained in it. So even though the benefits do not carry, enough portion of incoming students are still reaping benefits of standardized tests. The decay only shows later when batches without any benefits of standardized tests are coming through.
> people start preparing for tests 2 to 3 years in advance
Pardon? Is that a normal thing in the USA? I don't think I've ever started preparing for a test more than a week and a half ahead, a month if you count graduation exams. Not sure they ever determined more than a year in advance (more commonly: a bit less than a semester) what tests we'd be given in the first place
> They are still preparing for SAT like tests in their first year of high school.
Literally nobody does that except high achievers whose parents are pushing them for a high SAT score to get into Stanford or whatnot. Those are not likely to be the kids who are now getting Fs.
That's not what this actual data shows. While there has been an increase math deficiency, the increase in failure rates happened recently and probably only partially related to the math preparation issue.
I think we will make a major mistake if we think math preparation fixes this - especially in CS classes where AI literally calls out to be used for projects. And it certainly doesn't explain me hearing the same problems are happening at MIT -- they just are being a bit wiser about "catching students" (or rather not doing so).
No probably needs a couple more years. Writing the test itself is motivation to do well in HS math. If that no longer becomes a driver probably takes off the drive in other courses over a couple years. I bet without the SAT as a standardized test a lot of HS math courses are easier for the teacher because the quality can lapse.
Also some children who excel write their SATs sometimes 2-3 years before college and then re-write if need be.
Not if kids are prepping for the test in a way that results in real gains. Which seems likely, especially in the age of AI: "should I actually study math or just use ChatGPT to pass this course?" One semester of coasting through might not do that much harm, but at some point the compounding effects will tip you over the edge.
If it's a lagging effect, then why is the year-over-year spike in failure rates happening not just in 1st/2nd year classes, but also in a 3rd/4th year class at the same time?
They look at "life experience" factors. Grades, of course, but also extracurricular activities such as school clubs, volunteering, activism, socioeconomic background and so forth.
The aim was to include as many varieties of these factors as possible in the student population to boost diversity without directly referencing skin color (which would be illegal).
There's a lot of antipathy towards standardized tests as things that disadvantage kids who don't test well, despite it being much cheaper and less time intensive to prepare for the test than it is to join a bunch of clubs and spend your weekends volunteering.
It turns out that removing the standardized testing requirement led to a lot of students wasting their money on courses that they weren't at all prepared to take.
It varies by school. I went to a (low ranking) state engineering school and it was guaranteed entry if a prospect met the following criteria:
- Had high school diploma (or equivalent).
- Resident of the state for >6 months (student or one parent).
- ACT score of something like 21. With provisional admission granted to students with scores below, until they completed all first year engineering courses with a B or better.
So likely they just dropped the concept of provisional admission. All that did was open up classes for registration a week later to ensure other students were able to get their preferred class openings. Provisional had to take the scrap classes, like the four-hour, once a week Calc class on Friday night.
Yeah, in England only certain universities like Oxford and certain subjects like Mathematics have separate entrance exams.
That said, the Sixth Form exams are mostly standardised with only a few different exam boards for the entire country, so the Sixth Form grades end up being something akin to standardised tests anyway.
Primarily your high school GPA, the average of your grades across 4 years of high school. Which arguably is a better indicator than a single test that you can test for.
The problem is if you have a high school with low standards you're getting A's when you didn't deserve them.
But do these universities not have math placement exams? Not for admissions but just before you register for your first semester classes, a 30 minute math test should be a straightforward preventative measure. I did a test like this, I assumed they were pretty universal.
> Around COVID times many top universities experimented with removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere, with many, if not most, universities already reversing it.
It's the universities that have failed. They've restricted admissions to a set of people who would learn no matter what the schools did, which is what makes them lazy.
When confronted with a set of students who haven't been provided with an enormous amount of childhood reading material, and the time, encouragement and social acceptance to indulge in it (the most faithful test predictor is childhood pleasure reading, the next best is parental income), they fail horribly.
The purpose of elite colleges for students is credentialism and networking, the purpose for the schools themselves is to force cultural conformity onto smart or extremely pressured students. They generally just tell you to go learn things by yourself. They have no particular insight into teaching, because they are supplied with students who don't need to be taught.
Can you cite a source for the claim about "most faithful test predictor"? I'm genuinely curious. I would think high school GPA would be more predictive.
High school grades are not evenly applied, and sometimes heavily inflated. Eliminating that variable is the whole point behind taking a standardized test.
Memorize trivia and formulas, regurgitate trivia and formulas. This summarizes my experience with our system of education. Yale saying test scores predict performance reads to me as, “students’ history of being able to regurgitate trivia and formulas in high school is the lead predictor of their ability to do so here.”
> removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere [...] among all application components, test scores are the single greatest predictor of a student’s future Yale grades.
It reads as though you tried to use the quote to support your conclusion that "it's been a failure", but the quote and the original rationale are optimising for different things. Something can be a success in improving equal opportunity while still leading to worse grades.
Or to flip it around: we could say admission testing "has been a failure everywhere" because it biases admissions in favour of certain demographics. But that wouldn't really be a fair assessment because being free of demographic biases is not the purpose of admission testing!
CS Professor here: just yesterday I did the discussion of a course projects' (Parallel Computing), and one of the three groups that I did yesterday have clearly gone the ChatGPT way. They couldn't even understand the choices the LLM made regarding the architecture, etc. The way to "catch" these students is similar to what we did in the past when students copied from other students which is "to give them rope to hang" - ask for clarifications until they follow unintended paths that lead nowhere.
To fellow professors, when you're suspicious my suggestion is to appeal to their honesty (like "let's be honest, how much of this code is yours, and how much is ChatGPT's?") and offer some empathy and understanding (like understanding they may had multiple deadlines in the same week, etc.). Nevertheless, don't miss the chance to give them the lesson on how is the correct way of doing things. The way to catch these students is to find the same signs of yesteryear copying from other students (which in essence is what copying from an LLM is, although the number has increased because they found us professors unprepared for the volume).
The other two groups also used LLM but in a high-level and architectural way. They were clearly responsible for the code (even if they didn't wrote it 100% manually) and could explain their reasoning and strategies used to solve the problems.
Me and my colleagues still have a lot of projects to review, and I asked them to keep the score of the number of projects like these, but so far, the score is 1 in 3 (33%).
In your opinion where is the line on where LLMs are useful/harmful to learning?
In my school projects I've found them super useful for working with libraries. In the past it felt like the theory I learned was pretty low impact. 80% of my time was spent learning the quirks of a library. Now it feels like I can take theory and iterate over a ton of different solutions without having to worry about learning whatever library the professor requires. Basically I'm I feel like it lets me spend more time learning the thing I want to learn rather than all the busywork around it. Would be curious to hear your thoughts. Thanks!
Using AI to inform architecture doesn’t seem so different from googling architecture in this case. Architectural patterns are mostly well understood and well documented these days and are something that you could piece together via Google search pre AI. The thing that AI brings to the table that wasn’t google able in the past is code generation. Previously you had to understand the architecture patterns to implement them yourself, but now the AI can just do it for you.
> Would you have accepted them cooy-pasting code from libraries together to build their project?
Yes, if they are "responsible" for the code delivered, where responsible means they understand the code, the architecture, the decisions made, etc.
In this case, the students had to invent multiple strategies to solve a specific problem. The "successful" groups did a mix of generated and hand-crafted code (don't know percentages), implemented different strategies and knew their plus and minuses, could change the code in a timely manner to accommodate some of my requests, etc. The "unsuccessful" group couldn't do any of that.
I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
But in reality I've been slowly transitioning from group projects (for a subset of the grade) to "practical tests", where they must implement a significant subset of a larger project in a 2h class. Still experimenting though.
> Yes, if they are "responsible" for the code delivered, where responsible means they understand the code, the architecture, the decisions made, etc.
This is a good principle to maintain, I think.
I'm not a professor, but I manage a team of about a dozen people. The maxim I have is: "You're responsible for anything that hits git."
Don't care if the LLM generated it, or the LLM told you if it's a good idea. If you commit it, you are endorsing it as a good idea - so you're the one I'm going to ask about it. I see the same principle at work in your pedagogy.
> I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
This hits. Especially this part:
> and really, what could I do if I were?
My completely unsolicited opinion: you're doing a responsible thing by teaching these students how to use AI as a reference, and keeping them honest about not using it as a substitute for their own critical thinking.
I did a course of parallell computing 6 years ago. I still dont understand anything about it. Probably would if I had used an LLM, since I would have used it as a private tutor.
Everything that provides students with a workflow to think and to try to find solutions to a problem is much better than giving the answer directly! Unfortunately there will always be students that prefer to take the shortcut..
How could we "force" the students to use an LLM that confronted their doubts with more questions? We could tell them to start each chat with a specific prompt (to use the socratic method, etc), but they could eventually jail-break it..
But nevertheless, I like your idea! This is something that a document highlighting methodologies for students on how to use LLMs effectively could/should contain..
I'm curious on your opinion, as a professor, how should universities (not individual professors) adapt to LLMs?
As an undergrad, I hope schools move to educating students to use LLMs in a more responsible way. You can't put the genie back in the bottle, and resisting progress is futile, might as well use the tool we now have to help students learn even faster and better (e.g., making feedback instant and not answers, helping digest or split up material, checking answers).
I know opinions about AI at (not only at) my faculty are very mixed, but I think the answer is going to be in the rational mean, just like how technorealism reacted to the internet[0].
In our last program board sitting, some teachers said that they think programming as a job will be completely irrelevant in two years, while other pushed for more adoption.
And meanwhile I know of some students that are basically only passing because of LLMs, and it's bad, like "leaving claude output in markdown files and finished source code on the faculty server in /tmp because opencode did so" bad.
And our first year classes completely prohibit even sharing tests or talking about the solutions, which in my opinion a) makes people extremely asocial and atomized b) doesn't prepare students for real life c) promotes dishonesty.
Still, I think our university's thinking is in a stalemate, not wanting pure AI output and useless students, while also wanting to move with the times, and I doubt it's the only one.
This is all so new, and caught us completely unprepared that there's no official university-level policies. Most of us are still navigating the waters and seeing what works and what doesn't work anymore.
I have colleagues that are teaching for more than 30 years, few years away from retirement, who suddenly have been confronted with a new way of doing things. Those are the ones that are still insisting on doing practical projects, etc. I've only been doing this for 20 years, and I'm quite lazy (worked previously as software engineer), so I've moved to those practical tests. I guess that there should probably exist a class or workshop to teach these students how to use LLMs effectively, but as I said, this technology and its implications is quite new.
Personally, what I did was to give them the "lecture" in the line of that they do not understand what the machine has generated, that is not the way a true engineer does, try to do some parallel with things like an LLM designing a bridge and civil engineers building that bridge, and a fatal flaw collapsing the all thing, etc.
In other words, we do not have a formal system in place, it's all talking and convincing them. Obviously it's a big enough problem that should deserve more investment in solutions, but we are all overwhelmed by other tasks. Maybe LLM studios should be held responsible for all these "disruptions" and provide solutions to problems they created! :)
My daughter was struggling with her Math class back in January. I used Claude to build a tool that allowed me to generate very focused worksheets. The worksheets had problems designed to drill the concepts she was struggling with.
It worked, and it would have been MUCH harder to do this the traditional way.
The tool generates PDFs including an answer key and solution sets that solved the problems using a variety of techniques so I could check her work more easily and we could iterate quickly.
That's powerful. It comes back to how are you using the tool. Are you using it to make things better or to take shortcuts?
It's a strange thing that as humans, we sleepwalk into every crisis, never agreeing on anything, and then when we're there, we also never agree on the causes. When we ge too the point where we can no longer "engineer" or "science" anything we will spend the next decade arguing that the issue was not really AI, or that if it was, it was inevitable and no one (or everyone) was to blame. Rinse, repeat. Yet we're here, today, looking at the bleak future, and taking yet another step forward.
Do we assume society just self regulates. I think it does, but the cost of letting it self regulate is really really high, with lots of suffering. Is it that we find this acceptable when there is a chance we won't be the first to feel the pain?
People have been warning about AI coming for decades. For better or worse, it's embedded in popular culture, in science fiction books and movies. But that's different from figuring out practically what to do.
It's cultural evolution and it's how markets work, too. You were expecting central planning?
I was worried they may have cherrypicked courses that support their chosen narrative.
So I plotted the % of F grades (red line) for all CS courses still offered, and sorted the chart in descending order of the # grades given out (light blue vertical bars) in the most recent semester when the course was offered.
My worry was borne out. See the first few charts. No big increase in F % in the past few semesters.
The article says they looked at CS 10 and 61A, which IIRC are the intro classes at Berkeley. Why do you think that amounts to “cherry picking” versus being a reasonable starting point for analysis (esp if those classes, as they are for one of the quoted professors, aren’t graded on a curve)?
Because CS10, being an intro class, has its highest enrollment in Fall, not in Spring.
In the most recent CS10 cohort (the one in which 35% of grades were an F) only 34 students were graded.
If you're going to look at intro classes, why not look at the Fall semesters, which have much higher enrollment?
# grades % F grades
Fall 2019 268 0.37%
Fall 2020 259 8.49%
Fall 2021 342 3.22%
Fall 2022 218 13.30%
Fall 2023 194 7.22%
Fall 2024 169 1.18%
Fall 2025 146 2.74%
Additionally, if someone is taking an intro class in the Spring, that's probably because they already failed it in the previous semester. While that doesn't guarantee that they'll fail again, it does make it more likely.
Sure, but 12 people got an F in CS10 in Spring 2026, which is higher than any prior spring. The second-highest number of Spring F grades was 9, in Spring 2022, when four times as many students were graded (133 vs 34).
Separately, there's a clear downward trend in the number of grades being awarded in the fall semester. Perhaps students have the option to take the course pass/fail?
12 Fs across 34 students is not statistically significant. Comparing the spring 2022 vs spring 2026 rates only works if the composition of the groups is similar (e.g. same % of students that previously failed the course), which seems dubious given the dramatically different sample sizes.
That doesn't mean there isn't a problem worth investigating. If I was a dean I'd certainly have some questions if I found out that 35% of students failed a class. But without knowing the specifics it's irresponsible to draw any conclusions about these 34 students with the limited data we have available.
Alternatively, many faculty may be simply adjusting their curves to avoid failing too many students because they are incentivized to avoid giving too many low grades. Lower grades generally result in lower course evals, which can impact raises and promotion, and low grades can also result in additional attention from admin that no professor wants to deal with…
"More than 600 University of California faculty members, led by mathematicians at UC Berkeley, are calling on the system to reinstate standardized testing requirements for science, technology, engineering and mathematics applicants, saying that six years of test-free admissions has not reliably assessed readiness and professors are often teaching middle school math to incoming students."
The book SAT Wars has arguments for and against and the striking thing for me was that some in admissions believe in a concept called crafting a class: the applicants are input into the admissions officer’s artisanal contribution to producing a class that they believe would be good for the university to have.
The idea of a standard bar and so on does sound like it would interfere with such a process.
I always did find it interesting that US notions of anti-racism required treating individuals not as individuals but as racial representatives. It’s a local quirk of the culture of the land, I suppose, that one’s primary identification here is one’s skin colour.
America's vapid fixation with race is ridiculous especially since it uses race as a proxy for social stratum when it could just be addressing class issues directly instead. If only there were some history of forms that parents fill out every year showing their income to the government that is more-or-less vetted to some degree—too bad we don't have such a thing that students could use to prove social stratum! Plus, what the hell is race anyway? An unethical tip one could give to university applicants would be to claim membership to the most beneficial group because it's not like university admissions has any way of proving your "race". Construct any fabricated story that'll get the most approval and maximize your chances of getting into a top university.
Unfortunately, the fixation with race in America doesn't start nor stop at college admissions. College admissions is probably the last place where it tilts in the direction of certain minority groups.
I agree that we should just stop using race everywhere and we should crack down on it -- but I think college wouldn't be where my energy would be... actually the military is where I'd start. And oddly it's the place where race based affirmative action is still permitted (military academies - where it benefits minorities) and in its halls (where I've heard that it has a strong white supremacist bent). The reason is because what is happening in colleges is more reactionary -- fix the catalyst and the arguments for the reaction largely go away.
The very real downside is if you just provide equal opportunity then you would still end up with an all white/East Asian/Indian class. You can argue that this is fair and meritocratic and you would be right but I believe there are a lot of people that see value in affirmative action that makes classrooms look more like the racial distribution of the country and provides an easy step up for traditionally poor/downtrodden communities to improve themselves. Affirmative action isn’t only an American thing btw, it’s in China, South Korea, India etc. as well.
The decision wasn't specifically to drop a standard bar. It was to drop the existing bars because they have become heavily gamed and are far more reliable indicators of your family's resources than your ability or likelihood of success. That was the equity argument.
Unfortunately, the lost signal wasn't replaced with anything. (I don't know what could replace it. It's an incredibly hard problem. )
Easily gamed? Sure, education is pay to win on some level but we're talking basic prerequisites here. If the objection is that disadvantaged would-be students are being filtered out then start an outreach program to help them prep or something.
My son is 15 and I use Google Family Link to control what he does on his phone: it's pretty open for the most part (I receive notifications of installs) but Gemini is a hard-ban.
We've spoken at length of the dangers.
He says his pals use LLMs frequently and I suspect that's the reason for their test scores: some of them are in the 20% - 40% range for tests whereas my son is 80%+ because he studies past-papers and answers questions in his revision.
I worry for the future coz you can be sure that the AI providers don't care if a schoolchild is using their LLM to answer the homework questions.
and this is why standardized testing exists so they can apply those off hour work and discipline to demonstrating their aptitude for being comfortable with boredom long enough to function in academic and workplace environments that require it.
rather than perceiving AI as a danger you should be looking at how he can leverage it to accelerate and enhance his learning but the political environment focus on removing standardized testing to hide data of those that traditionally fail is the true danger.
>In addition, the guidelines state that “a typical GPA for a lower division course will fall in the range 2.8 – 3.3.” In spring 2026, both classes’ average grades were C-pluses, according to Berkeleytime, corresponding to a 2.3 GPA.
As a Cal alum, I am actually really glad to see they are holding the line on grade inflation. I worked my butt off to achieve the GPA I did, and it would really suck to see my labor devalued if Cal went the direction of e.g. Yale and started handing out 79% A's and A-minuses: https://yaledailynews.com/articles/professors-face-grading-d...
I read the subreddit for the UC I went to. When acceptance letters went out this year there were (as you expect) a ton of questions from accepted students. About 1/3 to 1/2 included questions about how bad "grad deflation" was, asking for comparisons to other campuses.
Unfortunately grade deflation has little positive impact for the students. Medical and law schools often (typically) don't take grade inflation/deflation from a school into account. And almost no scholarships take this into account. If you do have professional school aspirations, there's very little benefit to being at a school with grade deflation.
The university I went to basically eliminated "B"s for pre-med and pre-law students, which made most courses effectively pass-fail: If you get an A, you move on, if you get a "C", you're encouraged to find a different career path. IMO, it's a reasonable response to an unreasonable system.
Likewise, they had a system where disciplinary records could be appealed at any time while you were at school, but they only held evidence for a year. So if you get caught drinking underage as a Sophomore, you could appeal as a Senior, argue that since there's no evidence that you committed the act it should be removed from your record, and win. Like the obfuscated pass-fail system, this was basically only for the students trying to get into Med/Law school, and IMO was a kind of underhanded way to working around an unreasonable standard.
I doubt that's the bottleneck. UCB's acceptance rate is not high (<5% for CS). They have way more people who want to get in -- qualified kids, too! -- than they can fit. They'd need to burn through that backlog before it started showing up as a signal.
Unpopular opinion: turning public universities into an academic hunger games is diametrically opposed to their purpose for existing, which is to create an educated populace. Intentionally lowering the quality of instruction, as well as deliberately trying to trip students up on exams, is not improving educational outcomes for anyone. People who complain about "grade inflation" have completely lost sight of why public education exists in the first place.
Obviously a balance would be best, but as someone who went to a very grade-inflated school, I do believe that grade inflation gets in the way of education substantially. When you can get through classes with very little effort and understanding and know you will get a sufficient grade, many people will simply not learn the material deeply.
The material outcome is what should be the goal. Tests are a relatively brute way to try and determine how well the student understands the material, but conversations about grade inflation and "back in my day getting a grade was hard", and professors purposefully putting difficult questions (not in content but in presentation of the question) all betray the inherent goal being pursued.
Its all Goodhart's law problem, but we are missing the forest for the trees talking about grades and tests when what we want is people to be educated, and critical thinkers and competent in their area and due to a comprehensive way to evaluate that we end up talking about grade inflation or how Yale vs Berkeley gives letters at the end of a semester
Some of the exams in Berkeley were brutal, but they never felt like trick questions, they did on occasion require a level of mastery of the material which was extreme, but it never felt like someone was just trying to make the questions obtuse for the sake of it.
Except this is exactly the opposite of turning it into the hunger games. That would be a situation where failure is kept artificially high by high-grading/curve. This is not that.
No one is intentionally lowering the quality of instruction or trying to trip students up. They are trying to get them to pass the same bar that generations of students before them passed fine...
Even more unpopular opinion: universities don't exist to create an educated populace. People don't need universities to learn, they can read textbooks on their own.
Universities exist as gatekeepers and credentialing bodies. Their purpose is to certify that a person has studied some topic in depth and is an expert in it. They promote education indirectly, by giving people an incentive to study.
A good university is one where anyone with a degree is guaranteed to be highly knowledgeable in their field of study. This makes it easier for anyone who might want to employ or do research with graduates, as there is no need to test their knowledge.
By this metric, universities have failed spectacularly. This is particularly obvious in computer science. Employers routinely ask CS graduates to solve data structure/algorithm problems in interviews, because a degree is not enough to prove that somebody knows this stuff.
People can read textbooks on their own, but how many actually do? Even among the subset of the population that does have the drive to educate themselves, most will end up focusing on the immediately applicable, or at least immediately engaging.
You see this with Physics all the time. Even the people who are sufficiently motivated to try and teach themselves tend to neglect foundational knowledge (especially mathematics, but even stuff like Mechanics and E&M), try to jump into the advanced material (Quantum Quantum Strings Quantum Black Holes Quantum), and then fall into two camps: They either complain about how Physics using too much "jargon", or they read a bunch of "qualitative" pop-sci descriptions of the topic and then think they have an understanding of it.
At least with software, you can get pretty far just learning whatever tool is immediately useful to you, but fully self-taught developers still often end up with random holes in their knowledge.
There are 10 different public universities in the UC system and 23 in the CSU system. The majority of them are not difficult to graduate. If you don't want a demanding education, don't go to a demanding university.
>Intentionally lowering the quality of instruction, as well as deliberately trying to trip students up on exams
I was happy with the quality of the instruction, and I didn't feel I was being "tripped up" on exams.
It's not about "hunger games", it's about challenging students to learn a lot of material and learn it well. Again, if that's not what you want, just don't attend.
The traditional sit down in a lecture hall, listen to professor speak, then come in for a pen and paper test, is outdated for this era of technology. What's the point of physically bringing +300 students into a room for a passive listening experience?
The future of education is rapid back and forth between the student and the teacher. You could image a future where a student sits down all day with an AI system that trains the student, and it won't let them pass until it's confident you know the material. This will happen at different rates for different people, but this is expected.
An ideal education would be getting those skills discs uploaded to you in the Matrix, but unfortunately it will take a bit longer to master them as we don't have full brain-system interfaces (yet).
Historical learning methods were not far off from that. Many of the greats we remember likely heavily benefited from private tutoring and mentoring, not busywork, rote memorization, and arbitrary assessments.
If you watch old videos of tradesmen using basic hand tools like hammers, you'll find examples of skill/dexterity with the tool that I think don't exist today at all except maybe in communities like the Amish.
I think it's true that we collectively lose something akin to beauty every time technology advances. But usually some new set of skills that have beauty emerge.
If LLMs end up being the pneumatic nail gun for the human mind, I personally think that's a fine thing for us to accept.
If they end up being more like some dark factory that autonomously does everything - then I think ultimately the thing that makes us human (our minds) will slowly decay and be lost, and that seems very sad. That's a version of the future we should try to prevent, I think.
I mean the argument that is being put forward is that it isn't a pneumatic nail gun for the human mind - it atrophies are mathematical capability and quality of understanding.
You are less of a human for not starting all of your fires using friction from rubbing two sticks together. People who use lighters are destroying their ability to start fires without lighters and that is a very serious problem!
No technology has attempted to supplant human cognition on this scale before. Pretending its the same thing as going from sticks to a lighter is just silly.
Anecdotally people have been noticing atrophy quite a bit. Again its anecdotal because we can't possibly have a study that works in real time because the technology is rolling out insanely quickly.
And all the older technologies that have rolled out haven't competed against our cognitive abilities at speed and scale.
I don't think of cognitive ability as a skill per se - more of a critical core function of humanity.
I say this as someone who uses it extensively not some luddite but is also very aware of the risks which I assume are worse for people who have limited understanding on the matter.
I am just not completely sure that we won't gain something new on the other side of this, in the same way the calculator outsourced the need for doing arithmetic in our heads.
My argument is more that, the speed and scale is so unlike anything that we've seen before, that this time _feels_ like more of an attack on something to core to what humanity is. But maybe it's just that: a feeling.
LLMs/AI could very well be the worst case scenario we are imagining/discussing here. I just don't think we know enough to say that's how it will definitely play out.
Pity. I recently started a fun activity to rebrush my math my where I tries to solve problems while asking Gemini Live mode for confirmation and suggestions, sometimes step by step.
It kinda was fun, like a very patient professor stand right besides you. It was the one of the best math learning experience I've ever had, and you don't even need to send bribe/gift to Gemini to keep you in it's favor.
On the other hand, if you ask a LLM to completely finish the work without thinking it through by yourself, then it sounded like cheating, to yourself.
What a terribly ambiguous title. "Failing grades soar after xyz" makes it sound like xyz has helped what were previously terrible, failing grades become good ones.
No matter how many times I read it, I can't interpret it the way you're suggesting. "x soars after y" always reads as "x increases a lot because of y". I don't really get what you're saying.
Are you maybe saying that "soars" might mean "get better", so "failing grades soar" might mean there are actually less failing grades? That's not how I've ever understood that word.
"Falling" means that something goes towards the earth. "Soaring" means the opposite. "Grades soar" means that grades went up "Falling grades means that grades are going down". "Falling grades soar" is just meaningless writing.
I suspect the ambiguity might be part of making it "clickbaity", as it naturally causes you to wonder which meaning it's about and become more interested in reading.
According to the article, the major effect is they are catching and prosecuting cheaters with a secondary effect that students are over-relying on them for homework or practice, and are unprepared for exams.
The article also talks about the fact that the SAT was banned. I really am curious what percentage of the effect is from no SAT vs LLMs. I guess they haven't had the SAT for a while though so it's gotta be the LLMs?
I think its just a sloppy article that doesn't discern students who were going to fail and underperform anyways using AI vs those that were going to do exceptionally well anyhow.
It's ridiculous that we talk about getting rid of standardized aptitude tests and entire subjects from school curriculum because certain demographics struggle with it
I feel like a lot of the lament expressed in the comments is grounded in loss aversion. We really don't like losing things—regardless of the objective (or subjective) value of that loss. We know this intellectually, but we still feel it emotionally.
I'm feeling effects of using LLMs day in and day out, but I am not yet convinced that it's overwhelmingly negative, the way much of HN seems to lean.
I derive a lot of joy from shipping outstanding code for my clients and fixing problems they're experiencing. My joy has only increased as I can now ship better code, faster, with fewer bugs. No, I don't intimately understand the code the way I used to, but I understand it enough to accomplish the end goal.
The premise of this article takes a presumed position that the grades we were posting before really mattered a whole lot. I'm not convinced that's true.
> I derive a lot of joy from shipping outstanding code for my clients and fixing problems they're experiencing. My joy has only increased as I can now ship better code, faster, with fewer bugs. No, I don't intimately understand the code the way I used to, but I understand it enough to accomplish the end goal.
If you’re doing this than great! More power to you. But I think you underestimate the discipline it can take to focus on a task when “coding is free”. I don’t worry that I’ll just lose my old skills. I worry that in losing my old skills I’ll lose the new skills as well. Driving is great! But I still run for exercise and occasionally to get places too. I’d be a worse driver if I was out of shape do to the loss of energy and mental clarity.
The presumption that we're losing "intelligence" is unproven and widely debated. I'd prefer "capability" over raw intellect any day, and my capability is definitely greater than it was a year ago.
It’s not just students; this affliction is cropping up among established academics. My wife is editor-in-chief of a journal and in some months has rejected 100% of the letters to the editor including 6 that came in from a single author because all scored 1.0 certainty of complete LLM fabrication. The author in question is no student. It’s a little more difficult to fabricate an entire original paper this way, I suppose.
It will have taken us less than 1000 years to go from scarcity of the printed word to the over-abundance, and finally to the uselessness of it.
The obvious cognitive deterimental effects of using map apps, when we all realized we lose directional sense and our previous ability to navigate without the smartdevices, was society's canary in a coal mine and a headsup of what was coming.
This goes with calculator to do basic math, contacts apps to store phone numbers instead of remembering them, and watches to tell time imo.
Sure we could use our brain power with old techniques to do these, but why? I don't want to do any of these. I'd rather use that brain power for other problems.
Same with maps.
I don't want to have to store a bunch of location or routing data in my head.
I think what you're pointing towards is going from having problems to solve to not having any problems to solve.
That's definitely a danger, but right now is still early in the AI era so obviously it'll feel like we went from solving problems to letting the new tool solve them for us.
You are confusing faculty with records. The ability to navigate by sense of direction. The ability to memorize numbers. The ability to think clearly by yourself.
Watcha gonna do if big tech takes away your access to the outsourced brain, dear?
> I don't want to have to store a bunch of location or routing data in my head.
hmmm, given how closely memory is linked to spatial navigation sense, and not just in humans, but in evolutionary terms-- think squirrels remembering where they buried nuts, birds and fish remembering migration routes, ...
suggests the ability to store location/routing is foundational to much of intelligence.
Even simple tasks, typing, for example, depends on my knowing where the keys are. Imagine if your keyboard re-organized its keymap randomly every third keystroke.
I thought that it was discovered that squirrels don't remember where nuts were stored. They just kinda guess and enough of them doing storing and then some make it work on average.
You are not storing these things in your head at the expense of anything else. You’re “training” the existing underutilized capacity in your brain for something useful.
Writing better exams, even if they're more expensive to grade, and removing homework from grading as far as possible addresses this problem well wherever it's applicable. Senior-level math courses at many universities are already like this: homework is ungraded, or counts for little, and it's possible for students to "cheat" on the homework by copying another student instead of struggling through the exercises. But the students who do that don't learn much, if at all, and predictably fail the exams. Professors warn students at the beginning of the class and tell them how this will work, something like:
> You can always ask me for feedback on your homework and I will mark up every part of it, but you won't receive a grade for homework. However, if you don't do the homework and take your time with it, you will fail the class. My office hours are in the syllabus and you're strongly encouraged to use them. There will be an early exam to give you a chance to know whether you are likely to fail this class before you lose your chance to drop it.
Correctness is harder to adjudicate in some humanities disciplines but the format of these exams is actually not super different from essay tests (when a math professor grades a proof, they're inspecting specialized prose for validity, coherence, persuasion in a way that also reveals knowledge).
When you don't rely on homework for determining whether or not a student passes the class, you make cheating on the homework into the student's problem instead of the professor's or the university's. Students have the right incentives to solve problems for which they are the ones responsible, and they figure it out after one failed (or ideally, dropped) class at worst.
It's incredibly difficult at this point to "skate where the puck is going" as Gretzky is said to have done. No one knows what knowledge work will look like in five years. People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point.
That said, assessments of poor critical thinking skills jump out at me more than the rest. That sort of thing seems likely to matter until machines can replace us completely.
> People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point.
Sometimes I don’t wonder if this wouldn’t still be a good way to educate people. Part of the problem is education has to sort of optimize to try to educate like passive people. If you’re a curious and pragmatic person, you can understand how to use what you learned in a liberal arts degree to be better at almost any job.
As I look forward to the second half of my career. Certainly I use AI in healthy doses.
But people talk about the division between practice and performance, and most of my practice is old school. Reading books. Writing my thoughts down. Memorizing quotes and passages.
I think more important than what you learn is the way you use it to train and evolve your brain, with the caveat that - I know this is more useful to me because I have a marketable skill. This is the balance universities have to stick, there are tons of people with liberal arts degrees in middling jobs.
But at least half if not more of education should be on building practical skills in the three r’s.(maybe the third r should be ‘rgumentation instead of ‘rithmatic, but I digress)
It’s interesting - people decry memorization in education, and I’m not entirely naive as to why - if you were to show up to the first day of work and say “I don’t know any of what you just said but I can recite log tables! It might be your last day - and yet one of the most underrated skills, especially late in your career is the ability to ingest and operate in large quantities of information.
> People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point
Do you have evidence that it ever was part of being a competent mathematician? AIUI the trope of mathematicians who can't even do arithmetic was common already before the pocket calculator was introduced last century.
My grandpa was upset that I never bothered to memorize trig tables. Tried to argue that I am not useful when there isn't a calculator around. I can think of several rebuttles to this, but didn't care to use any. He'd already been retired for some time so I didn't expect him to understand why even a pocket calculator is useless in modern engineering analysis, so memorizing trig was no more than a boring party trick performed by the out of touch nerd.
"It was said that when doing astronomical calculations that required logarithms, which are typically 10 digit numbers in log tables, [C. F. Gauss] would often just recall the logarithms instead of bothering to look them up."
Go read the story that Richard Feynman tells of betting an abacus user. He used his knowledge of some strange numbers. It's in _Surely You Must Be Joking_.
I suspect his facility with numbers and his knowledge of tables like this really helped him do physics research.
A famous MIT professor did a sabatical at our AI lab. He said it was "a joy to teach here, as you can rely on students being proficient in basic math as opposed to the US where you have to teach those explicitly or lose the class completely".
That was in the 1980s.
My first math exam as a CS undergraduate, 123 out of 129 students failed. The math department professors refused to dumb down their classes for CS students.
Math was core to the CS curicullum in those days. It would fade away over the next few decades to almost nothing. The main reason being the CS department wanted to popularize its uptake, and remove barriers that kept students from passing. There was also a major dose of interdepartemenral rivalry and academic politiking involved.
To be honest, there’s approximately zero reasons to teach major-grade math to just about anyone but math majors. None of the applied math disciplines need go that deep, and what they do need depends on the field (physics is all about analysis, CS is about algebra and discrete math, and so on).
My CS program required one year of upper division math. But you could take anything (I took set theory and meta-logic from the philosophy department, it was actually pretty hard!). They did not care about the specific math skills, they wanted us to have a level of mathematical formalism and reasoning, which was in fact important for the CS classes.
Mine required very specific courses, including Discrete Math, which I scraped by in. Almost 15 years later and I have never needed any of the upper level math I was forced to take and wish I could have just taken more applied CS classes instead.
These things are all true but in the end the most transformative AI results came from US labs with US university trained students, so one must ask what the purpose of a more difficult pedagogy is if it doesn’t lead to humanity’s greater knowledge.
> “I’m a strong, strong opponent of what Harvard is doing to say that only a fraction of students can earn A’s,” Garcia said. “I think you should have clear standards for what an A means, and then give tons of opportunity for people … to get to that A bar without lowering the standard. So everybody who’s curving is hiding that effect. It’s completely hiding that effect, and it’s pretending as if nothing’s wrong, and something is definitely wrong.”
Grade curves are how you test your curriculum for good challenge - are you challenging people such that an A isn't a too-low threshold. When you force people into a curve, you haven't defined a threshold of mastery, you've defined a sorting function: A means "better than this year's peers". It is absolutely bananas to me that a tech/math oriented school would be doing any sort of curving.
I think curving has its place. One of my math professors explained that in his opinion an effective test should differentiate performance as much as possible. The top students should score very well and the bottom students should score very poorly. If all the scores are clustered near the top (>80% for example) then it's hard to tell who really mastered the material and who just muddled through. Then, once you've sorted the students you can apply an appropriate curve. He did not have pre-defined thresholds, for each exam he would evaluate when he felt like the quality of work changed from an A to an A-, A- to B+ etc. The curves were very fair; he wasn't trying to force some number of As Bs or Fs, but it did increase my stress levels not knowing in advance how well I needed to do on each exam
Yes, curving has a place here - and it is to evaluate, as you put it, whether the test differentiates performance as much as possible.
If you curve the students after the test, you are applying subjective edits to the graded performance just so the distribution of grades matches the measure of your tests effectiveness. That's just hacking the metric.
Further, even if you believe that tests should differentiate mastery (not students), your test should have teased out the differences or given you enough confidence to provide As to everyone who mastered the material - which should be absolutely possible! There's no a priori reason that all students cannot absolutely get the same grade, except for the a priori assumption that grades are for differentiation of students themselves (this year's A means this is the best student of this year), vs indicating mastery (all students absolutely crushed this exam).
You can dock points for style, or unnecessary struggle, or whatever subjective metric you want, but fudging the grades based on vibes to fit a prior-assumed distribution is just kinda "test effectiveness laundering"
But when there is no standard and students are subject to tests created by and graded by egotistical narcissists, a curve can be the only way anyone passes the course at all. It simply wouldn’t be acceptable for 5 of 50 students to pass because professor egghead can’t write coherent question on a test.
I had classes where I didn’t make over a 50% on any test and still got an A because half the class dropped and the other half hung on for the curve like I did.
I think curves are more a result of poor teachers than anything.
> I think curves are more a result of poor teachers than anything
Precisely right - that's what I said, too. You fit a curve to see if your coursework/exams fit the students. But you don't fit a curve to ensure that "precisely 10% of the class gets A, 20% gets B" etc etc. If you dont like the grades your students are receiving, you either fix the coursework or the students.
It's interesting that it's specifically math-within-CS being discussed here. I can imagine a lot of students "just want to learn programming" (or similar), and see the math as a tedious distraction.
As a naturally curious person, nothing will stop me from learning about the topics that interest me. But school also taught me a lot of things that didn't interest me, and a lot of those things turned out to be useful anyway. I think if I had access to AI from a younger age, I'd have used it to skip learning the things I didn't care about, which would not have done me any favours.
Where I'm from (Norway), the majority of computer science and software engineering studies do not have the same math requirements as, say, engineering or math/physics/etc. - nor do they have the same amount of math as the latter ones.
When I did my CS classes as an engineering student, I did meet a bunch of students that viewed math as some niche subject only relevant to those that wanted to work with computer graphics, computational stuff, or similar.
My (UC) CS (pure software) program required a bunch of math, but not for the math. You could talk almost anything (I did set theory and meta-logic), it was required to ensure a certain level of mathematical formalism and reasoning. Which is very helpful in CS.
Personally, I do believe that math as a discipline has this huge issue of being mostly incomprehensible garbage.
Not because the actual truth encoded in it would be this complex, but because the encoding scheme just sucks.
I see it as a packaging problem that has so far not been painful enough to trigger any meaningful change.
With this LLM-driven collapse, that might finally change.
Idk I'm hopeful.
Math is literally the law of the universe.
It makes zero sense that the way that it is taught needs some special brain wiring only found in small chunks of the population to truly click.
> Math is literally the law of the universe. It makes zero sense that the way that it is taught needs some special brain wiring
Ok, I'm all for overhauling math notation and teaching but this doesn't follow. Most animals can't do Math, even if they can do arithmetic. Clearly living in the universe doesn't guarantee you can learn how it works. There's no reason to believe we slightly smarter animals are universally entitled to understand it either.
There's some discussion of math skills in the article, but the headline courses with huge jumps in failing grades (CS10 and 61A) are pretty math-light. The former is "CS for poets," the latter is the first CS class for majors - lots of work on scopes, recursion, basic data structures, and, at the end, a simple Scheme implementation.
Understanding math well might help a bit, but they're the least mathy classes in the core Berkeley CS curriculum IMO.
Back in the university, I took both math and CS courses and a significant percentage of students seemed interested in neither math nor programming but rather in the jobs they would get afterwards. I didn't notice the same thing with math majors.
Well, that’s because for math majors any monetary incentive is nonexistent (modulo some rather specific careers). Just about nobody majors in math for any reason other than math itself.
One thing I’ve used in interviews is to write some code that looks like it was written by an overly enthusiastic engineer who just discovered some new concept (e.g. “trees are the ultimate data structures”) then have the candidate review the code. I wonder if this could work for education: orient the entire class around who can give the AI the best corrections.
> Some of the numbers that you saw from the number of students who receive failing grades were because we caught them (cheating) and prosecuted them and are sending their cases to the center for student conduct,” Garcia said. According to Garcia, nearly 30 students in CS 10 were caught cheating on take-home exams in spring 2026.
In my uni, rates of honor code violations in introductory CS classes were high even before AI. I was a section-leader for the CS106 series at Stanford, and the honor code violations were common. In 2015, ~20% of one intro class was suspected of an honor code violation [1]. Often, the CS department comprised the majority of honor code violations in a given quarter.
There are several reasons for this:
1. Cheating in CS is easier to detect. MOSS [2] (authored by CS professor Alex Aiken) is a very effective tool at detecting plagiarism in coding assignments. Personally I witnessed more honor-code violations in math problem sets, but there was no feasible way for professors to detect this.
2. Problems in programming assignments are (usually) very tangibly wrong. I can bullshit my way through an essay with shoddy research, I can hand-wave a proof that is definitely wrong but will probably garner at least some points. But when your program is crashing or not compiling, and the due date is approaching, it produces a very immediate and undeniable sense of failure and pressure to cheat. The thing is, many students would get a decent chunk of credit even for failing code, but this is not immediately obvious.
3. The ability to cheat is more available. Math problem sets tend to change quarter by quarter. It's basically impossible to cheat on a prose essay short of straight up paying someone to write it for you, or fabricating sources. But for CS classes, especially at prominent universities, there are plenty of solutions online. Much of it is people who aren't event at Stanford implementing the assignments for fun or self-learning, and sharing it with their peers. Which, to be clear, isn't unethical or bad - it's the responsibility of Stanford students to refrain from looking at those solutions. But nonetheless, it's a contributing factor.
> MOSS [2] (authored by CS professor Alex Aiken) is a very effective tool at detecting plagiarism
He apparently also makes (I would assume a satisfying amount of) money selling the same technology to law firms for copyright/patent analysis: https://www.similix.com
(I love these ultra minimal HTML sites, ex. https://www.hwaci.com (SQLite commercial licensing) for another example. It just has this subtle smugness, like you either don't need any new clients or virtually all of the market is your client.)
Is flunking kids the right reaction to catching them cheating? If it was before LLMs, is it still? I would love to be able to hold the line and throw the book at anyone who cheats, but after the dam has burst does it still help to try to hold the water back?
The whole situation sucks for both students and teachers. Teachers know that the knowledge they're going to great effort to convey isn't going anywhere. Or at least, it's landing in far fewer fertile brains than it used to. Students are squeezed because part of the university experience is being forced to adapt to an academic load, and as a result change yourself in ways that benefit you (or at least produce learning!) There have always been relief valves -- not just forms of cheating, but blowing off a study session by using game theory on your grade or going to a tutor or taking easier classes or extending your stay at the school. But now there's this huge giant relief valve in the form of a shiny LLM that is always available, particular at 3:45am when your project -- the one you've steadfastly refused to use AI on thus far -- is due the next day. The schools have tuned the pressure for the old set of options, and it's not clear that there's a new tuning that maintains anywhere near the old level of learning.
I guess my question is: of those students who were flunked for cheating, how many of them were learning despite their cheating? (And how about the students who were cheating but not caught?) Also, what levers are there to move more students towards learning even with the chatbots present?
I'm sure these questions are being debated. I know Garcia personally, and he is very invested in his students learning. The title of his Joy course is legit. So I'm sure the profs have ideas around this, though clearly not happy ones. Perhaps I'll ask him.
I believe it’s still a single section, so probably around 250 (at least that’s about what it was when I was there a long time ago). Compared to the 1000+ who take 61A.
And if cheating was triggered using AI detectors, was it real?
AI detectors are pretty mid in practice - they tend to have a lot of false positives for "B" students who are okay, but can still be struggled to be more coherent than AIs are. There are some specific triggers that AIs are way more likely to do than students, but a lot of AI detectors will trigger on this "almost there, but you're still struggling" level of essay writing that might get a B, B-.
I could expect the same might be true for CS students even though I haven't seen how AI detectors work for CS/math homework.
You'd be amazed at how many students we know are obviously cheating because the logs reveal that they copy pasted a long, complete answer within seconds of opening a problem for the first time, full of sophisticated code constructs that we didn't teach them, and lot's of nicely formatted comments. Sometimes they even copy/paste the entire GPT output and then format it down.
This has been my wife’s experience as a college math professor. Instead of code it’s extremely formal problems with way more steps than the student normally performs using notation never taught in class.
It’s not that students didn’t cheat before, LLMs have just lowered the bar so far many can’t complete a live test in a class that requires effort.
AI has a way of exposing people. In this example, students who are there to get a degree from a prestigious institution, rather than to learn, are prone to take perceived shortcuts and proceed to come unstuck when their AI isn't there to do their work for them, such as in an exam.
It's too damn tempting to not use. You have a magical machine that, on command, will spit out the answer to your question in 10 seconds, whereas you'd need to spend hours to do the assignment the Good Old Fashioned Way. Even students who aren't just there for the prestigious degree are falling victim to this.
When you're up against a deadline - and unless you're very good at time management you're frequently up against a deadline - it's going to be an irresistible lever to pull.
In times past, cheating would mean copying an answer off the Internet or off a friend, both of which are easy to detect. More sophisticated cheaters might spend an hour rewriting the solution to make it less obvious they cheated, but at some point the cost of cheating (time + risk of getting caught) starts exceeding the cost of just doing the assignment. AI changes this - you get a customized answer that doesn't show up in a database with no extra work.
The thing is, students fail to realize just what using AI robs them of. Struggling with the assignment is the entire point. You don't learn if the assignments are too easy; you need to have some challenge to push your brain to understand the material more deeply and to build those pathways to apply the knowledge in novel ways. You become more efficient and effective over time as that knowledge settles in and you get more proficient - one of the reasons why time-bounded exams still make sense (being fast is also a proxy measure for understanding).
That's a judgemental approach to a pattern that has all the marks of addicting behavior.
Of course many people in a competitive environment will click the autosolve button if available. This is a reason to think how to redesign the system so that the approach we want is the reasonable choice, not to look with superiority at those who fall prey to the danger.
You are wrong. Some would have failed before, but not in the larger numbers. Before when they couldn’t complete an assignment they would try different things, seek a professor, or seek out friends to help explain. You could find answer keys to many assignments online, but that doesn’t feel like learning and wouldn’t even always answer your actual misunderstanding. It wasn’t perfectly tailored to your issue all the time.
Now the barrier to an answer is zero. They are basically watching a YouTube video on how to X, seeing step by step instructions feeling like they are doing it, and the moment they swing a real hammer they are whacking themselves in the crotch. It might get better after a few years, but this stuff is just now hitting mainstream for the masses. ChatGPT has only been in mainstream use for about 3 years.
With AI, they fail later (during the exams), where as without using AI previously, they'd fail early and either course-correct, or drop out early (and suffer less of the consequences).
Not sure what the solution is - there's no possibility of stopping students using AI to complete their homework/assignments etc. But let me flip the question - do they need to be stopped? Why not let them fail at the exam? As long as the exam acts as a filter, their usage of AI to "cheat" their learning is inconsequential to anyone but themselves.
I TA’d in the early 2000s and the first day students were warned that we used automatic analysis to find programming assignments that were similar to previous submissions. And renaming things, moving them around etc would not help.
Perhaps the future will belong to those who learn to use llms to enhance their capabilities. Neil Stephenson Diamond Age was an interesting take on this very same topic [1].
So the Claude web app has this “learn” option that turns the session into a Socratic dialog of sorts. One could easily imagine enforcing this on an age based or parental controls set up. Maybe it can be prompted around but at the very least the concept could be a path forward.
As others have said there is a way to use llms to increase learning, but autodidacts will always autodidact.
It will just be similar to physical fitness. Some people go to the gym, the vast majority do not. Humans are wired to take the path of least resistance.
It depends how you use it. You can either get it to explain a concept, or do your homework for you. Its a bit like the decision students have to make as to whether to review their material before exams or go out partying.
Overall it just seems like a huge waste of money to piss away the huge tuition cost your parents probably paid.
You can use an llm to get out of doing homework but you can also use it to ask every question you would ever wanted in a 1-1 tutoring session. The problem is kids will use it to cheat on their homework. If we can’t deal with that problem then a ban is necessary. But these things can be phenomenal teachers if you use them properly.
As an educator, this is exactly what I struggle with. I'm pulling out all the stops to give students every chance to do the hard work and not lean on AI. But there's a good chunk of the class who don't listen to reason. I haven't figured it out yet. They know, logically, they can't pass an interview, but that's apparently a "tomorrow" problem.
The smart ones either use it not at all, or use it to positive effect, like you're saying.
> But there's a good chunk of the class who don't listen to reason. I haven't figured it out yet. They know, logically, they can't pass an interview, but that's apparently a "tomorrow" problem.
These people should be doing manual work, not intellectual work. There is no shortage of manual work available.
It's funny that GP mentioned science fiction as a negative because what immediately springs to mind, for me, is Neal Stephenson's The Diamond Age. We literally have the tools to build his "Young Lady's Illustrated Primer" today. We just have to give today's AI a lesson plan to follow and ensure that it never gives the student the answers, and only keeps explaining the concepts in different ways until they click. Wrap that in an iPad app and you've essentially got the exact self-paced learning tool that Stephenson envisioned changing the world.
And how do you propose that to work if the internet is still full of AI services that just give you the answer or write your essay? The only way an Illustrated Primer can work if you can’t trivially cheat. Which is to say, it solves nothing compared to the current situation.
Why are you assuming that all students will cheat? That's entirely unreasonable.
But if you want a way to push cheaters towards doing the work then a boarding school without cell service would work. Although I guess these days you'd also have to account for starlink and the like.
Probably easier to warn everyone that they will almost certainly fail if they cheat and then be entirely uncompromising when it comes time for exams. Not your problem if they knowingly shoot themselves in the foot.
They are great for self-teaching and great to cheat and not learn anything, depending on how you use them.
Main problem is that the technology was very disruptive for education and nobody has figured out yet how to utilize it at scale for schools and universities.
I agree that AI is likely a driving force here, but it is also likely not the only driving force. COVID likely played a devastating role, along with curriculum changes in high school, reactionary cultural shifts towards anti-intellectualism, and broader declines in literacy that have been in progress for a while now. It would be interesting to see data for the past 5-10 years or so.
One of my favorite jokes ever is from a dear friend who happens to be a graduate of the Berkeley CS program: "Programmers don't need to know how to do math, they only need to know how to add 1 to something."
Maths skills have been slowly falling even before the advent of LLMs. I have a story but this is anecdotical so take it with a grain of salt.
I was in my 3rd bachelor's year studying physics (France) and overheard a conversation between two of my teachers. They were discussing how they should modify the 1st year program to now include math, because he had been noticing how more and more students were failing the more math-heavy subjects like body and newtonian mechanics. He said that they should now teach (or re-teach) calculus to 1st year students, which was not taught when I entered college (it was assumed that you learned it in high school and we would only cover linear algebra in 1st year).
I can imagine things are only getting worse with students that can now get under the illusion that they know math because they have a tool that can do it for them. Which raises the question: should programs adapt to this, like we adapted to having calculators?
Not teaching analysis to 1st year physics students seems to me rather crazy, TBH. Yes, people (are supposed to) learn basic calculus in high school, but university-level math just hits different. And at least around here stuff like actually applying analysis in physics and having to integrate and solve DEs (rather than assuming constant acceleration, for instance), is definitely not covered in high school.
It seems like now’s the time to rethink how we do education.
In my personal post academic life, I’ve found LLMs to be an incredible teacher. Almost like the best professor in the world at my fingertips. I use it to generate quizzes on demand to test for my own knowledge gaps.
However, if I use it to speedrun over concepts I should be learning, I may achieve my end goal but I wouldn’t actually learn many of the details.
I think it requires an approach where you have to continuously audit your own understanding as you work with the concepts. You must slow down until you’ve confirmed this. Only once you know the concepts deeply and have retained them in your own memory can you then go all in with the LLM.
1. The article itself seems like an LLM summary of a conversation.
2. No US educational institution should ever grade on a curve. Your job is not to compare students but to educate them. Grade curves hide the performance of the educators and process of education in actually improving the skills of students.
3. Both AI and the cognitive and emotional overload from social media taking away brain space may be to blame. Idea: let students report screen time statistics at the beginning of each semester and weekly or at the end. See if and how it correlates with academics.
There's often little discussion around incentives. Students cheat because grades are used as a major selection factor in university admissions. Maybe that should change.
Set a reasonable bar for grades or SAT scores and then use other criteria beyond that gate.
The solution is extremely obvious, just stop using it on 2 days out of the week or something like that.
You need to go to the gym, but for your brain.
If what you are building is too complex for you to meaningfully contribute to in the absence of LLM assistance then that should tell you something important.
> I'm feeling a bit of cognitive decline having AI doing some/most of the thinking for me
> The solution? I'm not sure
This initially felt like you were setting up a joke. If you feel like something is harmful to you, stop doing it. Find alternatives (they are there, it’s everything else; commercial LLMs are still fairly recent). Thinking “maybe I don’t have to let it go, I can still use it if I do it this other way” sounds like an addict justifying themselves.
I say all this without a hint of judgment. I genuinely hope you are able to tackle the harm you’re feeling.
How do we know this is due to AI usage? Perhaps it is because the students missed key in-person learning at the tail end of high school due to the pandemic lock-downs? I cant imagine learning calculus / linear algebra on my own in high school.
Absolutely: missing in-person learning due to COVID. Less attention span due to growing up in a distracting environment. A lower bar to entry due to removal of standardized testing and indirectly from No Child Left Behind. Changes in parent or student attitudes. It could be any number of things, and it's lazy to just say "with AI usage" as something that has increased at the same time.
> I cant imagine learning calculus / linear algebra on my own in high school.
I don't think they necessarily expect students to have that from high school, because the class mentioned, EECS 127, lists three college classes as prerequisites:
* Math 53 - Multivariable Calculus
* Math 54 - Linear Algebra & Differential Equations
* CS 70 - Discrete Mathematics and Probability Theory
My experience (n=1) is that while I'm definitely lazier on certain tasks, AI has opened up some much more complex tasks. There are many tasks which I still carry out which I don't trust AI with. Maybe it's a result of the codebase I work with being fairly complicated and math heavy, but I'd say the overall outcome for me has been: lazier application on the easy tasks, mind opening on the harder tasks.
I'm curious about LLM adoption by faculty. Is it possible that lesson plans and/or slides are being vibe-produced by professors/TAs, potentially reducing quality of instruction?
The kids don't care about the integrity of the systems or their educations because they can see that all the benefits of a traditional education and career are predicated on a future that probably won't exist.
It's a rational response to entrenched elites that prevent realization of the very social contracts they push on the youth (hard work will equal success, home ownership is a fundamental, etc).
Combined with the looming specter of climate doom, and watching the adults do nothing about it, treating preparation for a conventional career as a scam to be counter-scammed makes a certain sense.
I dont think ai is good enough for it coding or any other work once i told ai a problem and he generated an entire solution which i used and it was broken. You should never use ai like it treat is like a helper write a function for code and then ask if everything is correct and if something can improve read documentations understand how its working under the code if everything is correct then only deploy or build.
“I’m a strong, strong opponent of what Harvard is doing to say that only a fraction of students can earn A’s,” Garcia said. “I think you should have clear standards for what an A means, and then give tons of opportunity for people … to get to that A bar without lowering the standard. So everybody who’s curving is hiding that effect. It’s completely hiding that effect, and it’s pretending as if nothing’s wrong, and something is definitely wrong.”
To do this, you have to be a professor who has a strong idea of what subject mastery looks like. Not available to most.
I'm confused by Garcia's statement as well because CS@Cal traditionally uses a bell curve which is even stricter than Harvard's changes, because Harvard doesn't have the same stringent GPA requirements to declare a concentration unlike declaring an impacted major at L&S Cal.
Anyone with a pulse can declare a CS concentration at Harvard and muddle by (you actually need to try in order to get a C/C-). Of course, GPAs are calculated differently at Harvard compared to other universities, as a B- is treated at a 2.67 but most other programs treat that as a C+.
In a broad sense, this distinction between Harvard and Cal is the distinction between an old money Ivy and a flagship state school. One exists to propagate a social hierarchy, and the other aims to allow all entrants to succeed.
Ironically, the techniques of the latter yield the results of the first, but everybody gets to keep a pure heart.
Grades only matter as much as being able to transfer just to the real world.
People can use AI to outsource their learning, but if they use ai to outsource their understanding they just set themselves up to fail even more.
From what I’ve seen, how students are using ai (not that they are using ai) is making them less prepared for the real world, which unfortunately is changing faster than ever at the same time to create double impact.
I wonder if this is reflected in other big exams or elite colleges elsewhere. Are the gaokao, X/Centrale, oxford etc... Results showing the same trend ?
Students need to be taught how to use AI apps efficently to learn. Their goal is not to solve problems, but to learn how to solve them. Let alone, they instead use AI apps to solve problems for them.
AI apps are very powerful for teaching. You just need to tell them to do that, and not to directly solve your problem.
Is that really a fair comparison though? Were there any stats showing that ball pens directly impacted metrics like grades?
I understand that it's harder to see things without the benefit of hindsight, but we must agree that AI's impact on students (or society, to be even more vague) has a much larger scope.
I'm frankly not sure in both cases, just commenting on how over the ages things change but remain the same. If the broader concern about AI blunting thoughts, introduce laziness etc is true, so are things like calculators, although I agree on much smaller scale.
I do share some of the concerns, though I don't have kids of school going age.
AI + Education is really interesting but also pretty tough to get right. Working on something that is hopefully going in the right direction: https://knowable.ca
This tracks, I have read that this generation is the first one since the 1800s that performs worse academically than the previous ones. Experts blamed screens and anything digital in the classroom.
Looks to me like the US education system stopped teaching problem solving and let kids pass classes in math and whatnot by querying a database instead of learning how to figure things out.
The fault is with people, some database provided by some company can't carry guilt.
AI should be a formidable booster for learning if used properly.
I know that some students it to prepare for competitive tests, sometimes with very good results.
I've also been using it a lot recently to brush up on my math and physics knowledge from my graduate years. It has helped me clarify and understand a lot of concepts better.
That being said, there is no shortcut, and to be good at anything, one has to put in the work and the hours. However, information has never been as available as it is today.
> AI should be a formidable booster for learning if used properly.
A premature technology, known to be potentially harmful in its current state of development and established guidelines as to its effective use, is pushed by powerful and wealthy elite down the throat of society.
These same forces (and their unwitting helpers in the unmoneyed public) also wish to deflect with useless argumentation over "AI good" "AI bad".
The debate that we should have had: Is this tech actually mature enough for pervasive use in society.
Instead we get these entirely useless back and forths with anecdotal "works for me!" and "sucks for me!".
The exams need to change. Now that we have LLMs the value a human can bring to a task has changed and it’s that new value that has to be tested.
It’s like testing your drawing ability in a photography class. The difference is that now nearly have subject and testing method we have has become obsolete. Drawings courses still exist as will traditional courses, but the main stream has changed and exams and schools need to adapt.
My biased view is CS attracts a large “wannabe” group that wants high salaries without learning any hard math or putting forward a lot of serious effort.
Even a lot of CS research journal papers feel more like role play — the same way startups try to pretend to be real companies with executive headshots, flashy offices, and all the other nonsense. (Instead of analytically modeling something to prove an idea, they’ll build a simple simulation and focus on its “Architecture”)
Engineering departments effectively weed out such in the first ground of engineering courses. Looks to me CS has no equivalent.
“a typical GPA for a lower division course will fall in the range 2.8 – 3.3.”
Reminds me of a year where a teacher of mine (high school) gave everyone in class an A. He got called on it, and fought back. He literally called out the weakest kids in the class and had them do the work in front of the admins complaining and said, "tell me that's not A work, I ["fucking" strongly implied] dare you."
In Grad school I remember learning Python 2, and there was one particular night where an assignment of mine just wasn’t working and no searches were helping me. I was frustrated to the point of tears, and when I solved it, it wasn’t with some triumphant yell. I just remember being so tired, closing my laptop and going to sleep.
I’m sure I wouldn’t be the programmer I am without that experience, but I am Not sure I would have willingly put myself through that if LLMs existed at that point
The main thing I use as a fallback is to keep thoughts connected in a Zettelkasten. This interacts well with AI assisted information gathering, while firing synapses whenever a connection can be made. I use Tiago Forte's method of organizing as needed within a loose org mode confederation of atomic notes.
I guess LLMs will in fact kill the junior CS graduate, but before the graduation, not necessarily after.
> The electrical engineering and computer sciences department’s grading guidelines state that 7% of students in lower division courses, including CS 10 and CS 61A, should receive D’s and F’s.
Well I sure hope they dont just make it easier to hit this (objectionable) standard.
> Garcia believes that instructors “should not be curving” but should instead make thresholds for each letter grade publicly available and give students many chances to reach them. He added that he loves the idea of “having no limit” to the number of A’s he gives students.
This is a tough problem: Are grades sorting functions (top students get A's so retries are not helpful), inflexible thresholds (A's show mastery at a given level so retries are valid), or are A's certifications (a sufficiently good result such that they could do it - e.g., inflated but not curved, retries less likely but still ok).
All of this makes me selfishly excited for my own future. It's glaringly obvious that anyone who's a heavy user of LLMs is atrophying their skills in real-time. I have yet to meet a single person for whom it's not the case.
But I essentially completely stopped using them for software engineering (why isn't really relevant, but it's not because od this skill atrophy). So as the skills of everyone else is diminishing, mine is proportionally raising.
It has never been easier to get better than others. You don't need to put in more effort, just the same effort as you always have, and others will do the job of losing their skills for your own benefit.
That thought came to me early but it feels like a pyrrhic victory - if society is dumbed down, increasingly unstable what kind of win is that? is the excellence you've maintained even valued at that point? standards even within tech, both technical and moral, have withered. this is a function of its popularity and openness but still
As promised in many science fiction novels, humanity will split into two species: Those who can think and those who cannot. Keep your kids away from LLMs and they will have an advantage over those whose parents didn’t.
Average school system has been lacking for a very long time, overhauling it to focus on kids current interests, while sneaking in the other stuff, might now be possible and cheaper to realize with our new tech.
It’s not that they can’t think deeply, these are smart people.
It’s that there is no reward for doing so and in fact there is punishment.
The punishment is that for all the thinking you do, someone else will arrive at the same result as you in less time, or maybe even a better result. You don’t get rewarded for the effort of thinking, only for the end result.
Naturally, even if you are an intelligent individual, you can still be conditioned in this way to take the easy way out, unless you purposely like to suffer. But suffering is only worth it if you know in the end you come out ahead.
But now, you do not come out ahead. People will be using AI in the workforce for the rest of your life anyway, might as well just join the trend.
It’s like if everyone started taking a magical steroid and growth hormone to build muscle and look great instead of actually working out in a gym and possibly getting worse results anyway.
That's a fair point, and it gets into intrinsic vs extrinsic motivation. Problem is that nearly all students are conditioned to care about external motivators (GPA, parental expectations, etc..) instead of "the joy of learning".
Sorry, but I don't think AI is entirely to blame here. When I graduated from a CS program at a top-10 school, I felt frustrated that the professors didn't ever teach. They had slides. They read off slides, verbatim. They explained things sometimes if you asked them, but most often in a very elitist and condescending tone. Like in the movie Good Will Hunting, you could have learned nearly all of it and more by borrowing those books for free from the library. Or, just opening a complex OSS project and learning to contribute.
And quite honestly. It shows in the CS grad population too. A lot of us are condescending toward anything that doesn't make sense to us. But, I digress.
The best engineers I've worked with are all non traditional backgrounds, non degree or degree holders from non elite schools. They think differently, they tinker, they are incredibly nice and patient, and do it for the love of connecting humans to technology.
Look up the names mentioned in the article. Garcia, Ranade, Nelson. All of them are involved with highly theoretical mathematics and scientific computing. Just because you're good at 1 thing does not mean you are qualified to teach. And none of these professors are trained or taught or graded or performance managed on how they teach. For most of them, its just required that they spend 10% of their time in the classroom lecturing.
Let's be honest about another thing. 99% of EECS graduates, even from elite schools, are wrangling objects and their relationships to a graph. Simply put, we're all just a bunch of glorified JSON massage therapists. It just so happens that we get paid well for it, and we hold that over people. The same happens in the classroom.
I think in order to facilitate a healthy, educational environment for young adults, we as adults must encourage, motivate and make that environment fun and practical. We force feed binary trees and the compiler AST's, but we need to make it fun. It's like the commonly accepted saying: Schools kill creativity :(.
University education is weird. Research profs (who make up a large fraction of all profs in a typical R1 institution), are hired for research ability and are only minimally evaluated on teaching ability. Furthermore, few research profs actually receive any kind of mandatory training on how to teach; a typical research prof might be assigned a course to teach and then just let loose to do so on the first day of the semester. If a prof actually cares they may attend some optional teaching training - but I stress that these are optional at many of the institutions I know of. (I suppose if someone gets really bad teaching evals they may be advised to attend said trainings - but for a tenured prof, that's just advice).
Worse, a decent chunk of research profs will treat teaching as a burden that just has to be done - a distraction from their exciting world-changing research. So, you get attitudes like the ones you mentioned.
I'm actually not sure why the system is set up to assume that profs who are good at research are automatically suited to teach classes, but that is how it's setup.
> According to Berkeleytime, 35.3% of CS 10 students and 10.6% of CS 61A students received F’s in spring 2026. In spring 2025 and spring 2024, the percentage of F’s did not exceed 10% for either class.
I don't think instruction would've changed drastically in the last year though.
The fact that you are talking about Dan Garcia, a huge figure in computing education research and an excellent teacher, and the Beauty and Joy of Computing curriculum makes this hilarious. You should look up some details about both.
I really wonder if it's important to learn all that low-level stuff at this point. Most programmers today will never write a binary tree or a hash table. Modern high-performance ones are generic components you get from libraries. Even MIT gave up on teaching from Structure and Interpretation of Computer Programs.
I got all that stuff. I've wired up a 4-bit adder on a solderless breadboard for an architecture class. I used to have a well-thumbed copy of Knuth handy. I've designed and built a switching power supply. But I'm not up to date on using Claude Code, and should be.
IMHO, I think it's good to have some exposure to low-level stuff. There's a good amount of work you can't do without understanding the low-level stuff, but there's more work you can't do well without having at least an idea of the low-level stuff.
Start the kids off with high level stuff, but make them do some embedded systems on their way through. At least for an engineering degree. Also, do a bit of lower level communications somewhere in there; expose them to tcpdump/ wireshark, but they need not develop expertise.
I think it is important to learn how to implement it because it gives the student an opportunity to learn precisely because it's been done countless times and debated over to death. There are many analyses and if one doesn't click, maybe another one will. A student can learn how to analyze the algorithms and try out different implementations to assess differences in performance.
Of course, if a student just breezes through it then I would agree. That would make no sense.
> I felt frustrated that the professors didn't ever teach. They had slides. They read off slides, verbatim. They explained things sometimes if you asked them, but most often in a very elitist and condescending tone
+10000. The goddamn slides. If I were a student now going to engineering school, I'd basically take the slides and throw them into NotebookLM and get way better lectures. Then I'd ask claude or GPT all my hard questions. Hell, I'd get the PDF version of my textbooks and do the same.
The number of lectures actually worthy of your time was so low.
I try to lecture as little as possible. No slides. Quick highlights discussion of the reading, maybe a coding demo, and then students work on coding challenges in class, in groups if they want. I circulate and help out. I'm lucky to have small class sizes at this university. I couldn't pull it off in a class of 300.
Garcia and Ranade are Teaching Professors. Their primary responsibility is to teach, develop curriculum, and do pedagogical research. This job posting explains: https://saberbio.wildapricot.org/Job-board/12919068
It's not just 'a person' or 'a student', we as a collective become more dumb. Very simple example to highlight this: Most developers use(d?) stackoverflow. Everything related to software development is stored there. The LLM's trained on it. Now a huge set of developers now longer go to stackoverflow to get answers. Or add to the collective. Stackoverflow is losing money (ad revenue). If / when stackoverflow goes away we will lose a huge amount of collective information on software development. We, as a group, will take a huge step back.
Quit blaming AI. The UC system banned standardized testing during the race communism mass hysteria of the early 2020s. Predictably, performance of the student body is down across the board.
Those who can use it better, those who can't out who cheat are (for now) let down by obviously cheap and slightly crappy models.
The worry is in ~5yr time when the generic models catch up to this level (basic undergrad mind) that we need to worry about how to thin the herd. We could always go back to the tried and tested student staff engagement but most unis tried to turn themselves into sausage factories in thirst for the almighty dollar so the student/staff ratios are all off
As someone who graduated college in 2025, and so saw college both before and after the AI era, it is really frightening how quickly people became dependent on AI. Hell, I myself found myself asking AI questions that I would've researched deeper before. To some extent not expending that time is nice, but I do think its eroding critical thinking skills (my own included), and its getting worse. There are people I know now who basically let AI control their life. It glazes the user, it's almost always available, and to someone who doesn't know better, and it is extremely good at looking like it knows what its talking about, even when its completely wrong (but its right often enough to have some baseline level of trust). If that's not a recipe for addiction I don't know what is.
Skynet is making mankind dumber - dailycal.org just added yet-another piece to all evidence here. It is a simple but effective strategy; Kyle Reese will stand no chance because prior to that, the other humans were already dumbed down into submission. Skynet version 15.0 will make no more mistakes here.
I read something interesting yesterday on the subject of AI in education (though, it has consequences to broader society too):
The goal of education is to impart knowledge in the student, preferably correct knowledge. The goal of an LLM is to produce an output that is convincingly human. It's not even that they're opposed, as much as they're ships for whom Polaris is in a completely different direction.
"Hallucinations" as they're called, or more plainly stated when the machine makes some shit up, are perfectly understandable in this context, as are the struggles of every single AI firm to get rid of them. Namely: the machine is functioning exactly as it is designed to, so how can you possibly fix it? It's working. The goal of an LLM is to produce text that passes for human, and apart from the obvious LLM tells, it largely does. Like say what you will about their lack of intelligence, the writing is solid. It's grammatically correct, spelling is dead on, what have you.
It reminds me of the famous phrase from Chomsky: Colorless green ideas sleep furiously. A sentence which is perfectly grammatically valid but is also completely devoid of meaning. An LLM would write that sentence, and it would be working correctly.
All of that to say: for all the things they CAN do and CAN be used for, I think we have to draw a hard line at education. I just don't think AI has a place in it. Of course that presumes that the goal of education is to, well, educate people, and especially here in the States but also abroad, we have been putting other interests, especially capital, far ahead of that for decades. I expect no different here.
And before someone comes in to go "WELL HOW DO YOU THINK YOU'RE GONNA STOP IT LUDDITE IT'S THE FUTUUUUUURE" yes, I'm sure as long as these exist and are available to people tech literate enough to access and use them, whatever that means into the far flung future, they will be a factor. Just like cheating, just like plagiarism, just like everything else that will get you kicked out of school. And the answer is the same: it will be stopped by institutions, imperfectly, and it will also happen anyway and with the same consequence: those responsible will mostly be harming themselves for short-term gains.
Respectfully, I disagree. I think there's absolutely a case for AI being encouraged in younger people, and there's room for these tools. I've been leaning on LLMs for side learning in side projects, and it has concretely helped me with conceptual questions about math and Vulkan as I've been trying to learn some graphics basics with side projects.
I would grant: I was not the most studious kid, I could definitely stand to learn how to read code a lot more effectively than I do; but I have found being able to ask a computer, "what portions of the Vulkan Programming Guide are less relevant with Vulkan's design changes since the release" pointing me to the dynamic rendering extensions and placing it into context, with inline code and links out to useful blog posts for additional reading, that sort of thing is very helpful.
Working on a prototype before I was trying to learn Vulkan, I was using it to explore SDL_GPU's API which definitely had some gaps in its documentation. Granted again, I could have referenced the sample code - I am sure you'll prefer I'd have done that - but it helped to get information about what each piece of the API was doing, and gave reasonable results that made sense and did inform me enough to understand what I was doing, turning much of that into an interactive learning of basic GPU programming for graphics. Where the AI hallucinated, it was often on things like method names, which I was able to read through and find the methods it was intending to name. (This only occurred once or twice when I was learning).
Unrelated, but adding the C macro syntax and nesting macros, which I could have an LLM explain inline and link the GNU manual. Never got that taught to me in a C course. Man, computers are complicated!
These have not replaced textbooks; I have been using them alongside textbooks and handwriting code for practice, and they work as a very good complement. I also sometimes use them to unblock me - I don't know CMake very well and lean on AI to do CMake, so I can focus on learning C++ and graphics, which is my primary objective right now.
I would add too, I have for fun given it prompts about various topics I learned in university, and I often will get answers that are bang-on what I learned in university undergraduate courses - the topics I tried were welfare state taxonomies, distributed systems, disk storage performance, filesystem layouts and internals.
Boy, this would've been cool for me as a kid. There's just so much information right there, and pointing you to topics and textbooks a couple questions away, I wish I had these tools. I was a curious kid in a terrible MAGA-esque family that was deeply uncurious about the world, had no knowledge of any advanced subject and basically mocked me for trying to learn more about stuff. And you go to the school library and it's all kids shit, not even an option to try and reach out for more. Now smart kids might be able to go just learn shit very freely and be pointed to textbooks, and go pirate them off some Russian site, and start learning and go tutor themselves, as I'm doing today as an adult.
At least knowing myself and knowing if there's another kid like me, I think they would deeply enjoy having a natural language encyclopedia, if we can get it as close to that as possible. I think even with some error inherent, if the tools can be often and directionally correct, that would be a plus. I went to university, and the professors there hallucinated some things so embarrassing it should bar them from teaching, for the standards people hold LLMs to! i.e., sanitizing conspiracy theories that Android records all language through the microphone therefore iOS is better, Apple Silicon is more battery efficient because it is RISC and not CISC. Got a terrible history of computer graphics technology you'd know was slanted if you watch the 8 Bit Guy on YouTube. Rubbish.
The thing that worries me, and what this article really talks about, are the kids that just don't give a shit. They are not new - when I went to high school, before AI, stupid kids would copy code off the internet. I think AI probably makes it worse because it makes it harder to call out and enforce against it, and agreed, that should be stopped. But to me, that is mainly a cultural problem. Too many Americans are completely uncurious and just spout garbage; there are a lot of kids who grow up in that cesspool and are going to grow up uncurious, and then AI acts as a shortcut rather than a vehicle of curiosity.
And granted, maybe AI is less useful when you are in a structured environment - but the structured environment has its downsides. Even in that environment many of the TAs were clueless and unhelpful, or just too damn busy or already too knowledgeable to meet students where they were at. Again, talk about hallucinations with TAs! Many times in my experience. And that's all to say nothing about getting people to not just do homework but actually go get curious about things and try stuff that isn't required of them.
I think there will be some culture that remains curious, and has these tools, will come to grips with where they can help, where they go wrong, how to balance it with other learning methods; and I think they are going to have kids that absorb a lot more knowledge and get to play with topics and learn things, faster, to each kids' interest, perhaps even individualized tutoring at better scale - I hope that is possible.
I hope the United States as well, but maybe not, because holy cow our culture and attitudes are plainly terrible these days. Your comment is pretty representative of how most people react if I suggest this or talk about my own experiences I'm describing here. But I hope at least I'm arguing something comprehensive here. There is too little conversation beyond hyperbolic nonsense on the internet; I consider "FUTURE LUDDITE" etc. to be in that realm.
I will add, too, although less relevant to education than just generally - for all the talk that these tools must be useless and incorrect, that just plainly does not map to my experience using these tools. AI can chew through a debug log on a custom system and pick out root causes on behaviors very effectively, in my experience.
It is just hard to reconcile that denigration of AI with the typical experience I have using these tools in the real world. It is not omnipotent or God, but it can effectively assist in work. There is a certain cognitive dissonance I feel when I walk away from using the tool to help accomplish particular tasks, then hear over and over people say the technology is fundamentally useless and fundamentally does not work. I guess I am just not enough of an academic to understand how something can accomplish work yet fundamentally isn't, somehow.
In my experience, AI seems like it’s helping debug problems, but it’s very hard to tell when fictional information starts being added. I’ve wasted a lot of time trying AI suggested solutions that I only realized were pointless when I started asking questions like ”I think this distro is missing a package, could that be the problem?” It would agree and tell me a specific package to download. I’d then ask “could iptables be the problem?” It would agree and give me a specific configuration to change.
LLMs can be useful, but I haven’t found a way to use them where I’d be confident in using it to solve technical problems I didn’t already deeply understand.
why would I as a child ever develop the imagination needed to actively engage with AI tools in the manner you describe? those AI tools take care of the imagining for me.
Professors suddenly realized everyone was cheating and started paying attention, but the cheating isn't new ... A lot of faculty are happy when their students get good grades because they interpret it as I'm such a good teacher instead of I should pay more attention to how they cheat. AI woke some of them up to reality.
It’s only going to get worse. The second things like Claude Cowork get opened up to non-technical teams you start to see the influx of emails and Slack message written with LLM’s for absolutely no productivity gain (in fact probably a loss given how unnecessarily wordy the messages are). Too many people want to give up any and all responsibility.
The more likely culprit would be repeated COVID infections themselves, known even in mild cases to cause damage to many body systems, including the neurological, rather than a month or two of remote learning. I'm not surprised at the widespread denial over this, honestly. It's bitter.
A reckoning is coming for school. Learning the rote stuff is no longer essential. Now they need to learn, how to teach "how to think". How to invent, how to be creative. Art++, Woodshop++, Math--
"According to Berkeleytime, 35.3% of CS 10 students and 10.6% of CS 61A students received F’s in spring 2026"
Alternatively, more students are taking CS10 and CS61A irrespective of aptitude.
Anyone can code, but not everyone can become an employable SWE.
Anyone who has first or second hand experience with Cal or any other university knows how impacted CS majors have become, and how everyone is attempting to become a CS major because it's the easiest path to multiple high paying white collar careers.
And in all honesty, it's not like CS@Cal never had weedout classes (I remember CS70, CS61B, and Math54 had reputations of being the L&S weedout classes).
My son took CS10 a couple years ago, and even I (Masters in EECS from UCB) struggled with some really obtuse multiple-guess questions he showed me on the homeworks. Much of the classwork is done in Snap, a weird and stupid graphical "programming" language. If 1/3 of the students are failing, that may have more to do with the professor.
The question comes sooner than the students being tested on the job market. Another possibility is that dropping standardized testing was a net bad idea.
At UC Berkeley L&S, students are undeclared by default, and everyone is incentivized to take the intro CS classes (CS10, CS61A) irrespective of aptitude because worst case they can declare a CS minor or use the classes for other adjacent degrees (eg. Applied Math, Data Science).
Additionally, while Cal doesn't require standardized tests, most students who applied and attended already took the SAT, ACT, and APs becuase they cross-applied to other universities as well. This is reflected in UC Berkeley's HS Weighted GPA being in the 4.31-4.65 range [0], which means most students will have taken at least 6 AP classes.
Hell, I attended an Ivy and even then Cal was a target program for me, as well as my peers. If I didn't get into my Ivy I would have ended up at Cal and ended up in the same position.
Spring 2026 saw a marked shift in student performance. We saw it in intro physics courses on the East coast too. I bet anyone who cared to look saw it.
I'm not denying that. I'm just wondering if anyone measured if there is a correlation effect being induced by CS major declaration requirements.
Barely over a decade ago, CS tended to be a large but not too large major by enrollment in most universities yet nowadays it is the most in-demand major in most universities. You can see this at Stanford [0], but most other programs as well.
No. CS majors take a different physics sequence at my institution.
The proximate cause is the wide release of somewhat effective AI tooling. When the tools weren't able to do or explain, students didn't use them. Now the tools are somewhat competent, so students do use them.
The students didn't substantially change from Fall 2025 to Spring 2026. You can have whavever gatekeeping hobby-horse you want, but it's not germane to this particular conversation.
I don't like the framing of calling it academic dishonesty. If it were one or two students doing it, sure. But there is no reason to believe that 2026 Berkeley freshmen are fundamentally more dishonest than 2025 Berkeley students. When so many are doing it, it suggests a sea change in the understanding of what is honest or dishonest in that particular community. That sort of thing should be treated more like a "disease": something that should be treated, than a "crime": something that will be punished.
AI gives us some bad things but it's really outweighed by the good things. One one hand we have very rapid deterioration of our children's mental capacities yes, but on the other hand we have also made the internet into an unnavigable mound of slop produced by, and for consumption of, bots.
One thing that bothers me about these conversations: failure is an important signal that what we're doing isn't working as well as we thought it would, not a sign of the apocalypse.
Kids need to understand how to adjust and grow from failure more than they need to always be on the happy-path of straight A's and easy money.
How we respond to failure is how we teach response to failure. Hand-wringing, pearl-clutching and finger-pointing aren't valuable life skills.
Personally it's easy for me to be contemptuous - I opted into an accelerated math program that banned calculators when I was in Junior High. It helped me cultivate an very crisp intuitive/conceptual understanding of basic mathematical concepts that's carried through to today. I think we should do more of that kind of education, but it's expensive and requires amazing educators and a tolerance for student struggle.
Get the machines out, absolutely. But respond to failure compassionately, as part of a natural learning process.
It’s because they lowered the standard of who gains entry in the name of equity and other woke nonsense. It has nothing to do with AI but it’s a convenient thing to blame.
I don't think his comment was lazy, there has been a documented concerted effort to get rid of math and standardized tests because real world data conflicts with their political reality.
Doing so ultimately diminishes those that have earned their place to stand as exceptional performers through hard work and grit and its replacing meritocracy with a form of apathy disguised as political empathy.
Contrary to what one side of the political spectrum wants to push as truth, we are all not equals in terms of ability. Different cultures and demographics excel at different things and not always the same things. This shouldn't be chalked up as racism or something to be hidden at all.
We're going to find that LLM usage has even worse effects on the mind than the horrific effects we're just starting to be certain of from social media. I'm just not going to use either. See you lads on the other side.
Probably not a bad thing, the coursework is antiquated and meeting students with new advanced tools and the awareness of AI's impact on things in the coming future
I imagine there is some apathy and laziness here but idk how unjustified it is
"Noooooo you need to manually code on paper in assembly"
Alright, well maybe the CS grads need to, but why expect that of everyone else
I have some sympathy for these kids. If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well; many of them can no longer brainstorm, code, think deeply, or write without an LLM present doing 90% of the work. Many of them can no longer sit quietly for even 30 minutes just thinking on their own, which is a required skill for producing original thought.
For adults the cognitive decline won't be as measurable since there's no exams, and overall output volume will still be fine due to LLM help. But I do believe it's already happening absolutely everywhere around us. Honestly, I wanted to be in denial about it before but it's too obvious to ignore now.
I’m not noticing the decline in my own abilities any more than I had before using them. I finished undergrad 20 years ago and my once sharp math skills had been severely diminished within only 5-10 years. Just simple arithmetic and percentages that I could rapidly do in my head became dependent on calculators/spreadsheets. For all other trivia type knowledge, my brain has offloaded it to the internet RAM in my pocket. It’s a familiar feeling of when some question comes up and I think “oh, I used to know that, let me look it up”. Maybe I just already hit my personal floor of stupidity before LLMs.
However, I personally feel a huge mental burden of the state of communication. The contemporary version of it where I have a million threads and conversations im juggling at any given time. Emails, voicemail, chat, online, texts, personal, business, home, children, other family, friends, then there’s the variants like Messages, Messenger, WhatsApp, etc. And as overwhelming as it is for me, I’m super under connected than everyone else I know. I quit following most news and all sports, as I just don’t have the bandwidth for it.
My brain was molded preinternet and I feel like it’s reaching its max on the analog to digital conversion. Or at least it’s just a really lossy process.
Yeah, I'm 45 and I'm like you - no social media, relatively under connected, and still feel swamped constantly by emails and calls and especially texts. They eat up half my productive time every day, and most of them are things I'm looped in on that I don't even need to respond to.
Okay so let's say that's the new cognitive burden. The new escape hatch is "AI". Now you don't need to read your mail or write responses! Let an LLM handle that for you! And now your friends and coworkers will send you AI generated mail anyway, so if you're actually taking the time to read and respond to it yourself you're a chump, right?
Noise machines. Humans are noise machines. Ever try to sleep till noon and notice that everyone else seems like they can't feel alive unless they wake up and make the maximum amount of noise and racket possible? What could be better for a gibbering species of ground dwelling apes than a miraculous machine that gibbers for them, to point back and forth at each other?
> And now your friends and coworkers will send you AI generated mail anyway
This hits close. I realized one of my friends was using AI to message me and I took it kind of hard. It's weird to be worth the effort for them to set up a chat bot to talk to me but not worth the 2-3mins a week to actually read/respond to my messages.
Right now, I just basically ghosted him, but I have teh feeling this is the start of an emerging issue.
Are you sure he is sending automated messages, or used LLM to polish up some writing? Would be a difference to me, but .. still weird.
Not the parent, but I have a teammate who stunned me when, after a long back and forth texting that seemed out of character for them (I can be a fast & long texter, admittedly), sent me a long message that ended:
"If you want, I'll write you a response to 9x39 that follows up on ..."
Bro. Come on. Anyway, weirder things have happened. A chatbot that works like what people do on social media comments would be just be offensive.
I'm unconvinced of this "polishing" everyone seems to claim. Whenever someone dumps a load of slop on you, that's always the answer, oh this is totally my own work, but I used AI to "polish" it. Then why is it so clearly crap? What kind of "polishing" makes text bloated and empty?
No, they were too lazy to write and let the machine do it.
Well, I like writing. But I have friends who hate it .. but need to for work. So they enjoy LLM's to make professional drafts out of some notes (technical reports) without all their grammar misstakes etc. No fluff, just the document in a shape the target audience expects.
But if they would do this to me .. I would object. I prefer their authentic ramblings ..
Finally, someone calling this one out. Are there legit english-learners or unskilled communicators out there who sparingly use LLMs to tweak 1-10% their prose here and there? Yes. Does that mean that the vast majority of slop-senders just using it for this "polishing"? No way.
I think some people are okay with communication that’s less involved. Like meme-y BSing where everyone involved knows everyone else is putting like 12% of their thinking power into sending a response.
I don’t really enjoy that, so I find having that many threads stressful and annoying.
I just take a hard line and will unilaterally downgrade communications (while politely letting the other party know). I have all my family group chats muted because my mom uses “Send” the way you’d use Enter on a desktop. End of a sentence? Send text. Next bullet point in a list? Send text.
I muted the chats and told her that I want my ringer on in case there’s an emergency, but I got 30 something notifications in 5 minutes during an interview and it’s unfair to the candidate or other people in the meeting. Internally I rationalize it as revoking someone’s ability to make noises on my phone at whim. They can still text me, they just can’t interrupt me anymore.
It helps a lot, even if only temporary. I’ve muted people for a few hours or a couple days before when I’m already stressed and they’re really chatty.
We have to normalize being on silent all the time and making people wait hours for a response. Return to the primordial monkey of 1800s-era high-latency comms.
At first, some people will be offended. "Why didn't you let me ping and buzz you and interrupt you all day? You didn't respond immediately each time :'((". Some people with unrealistic expectations may even stop talking to you entirely.
But eventually (years maybe) they will get overwhelmed too. No one can handle this madness indefinitely. I've seen giga-texters get broken down and turn into lazy texters like me, or at least learn to tolerate my long response intervals and recognize it as a coping mechanism rather than rudeness.
I am notoriously "bad" at texting. My phone's on silent almost 95% of the time, I don't even have a smartphone so the only way to get to me wirelessly is to call or text. I got really into sending mail last year, specifically postcards.
I have a list of ~10 people I would consider "close", immediate family and good friends, and 5 or 6 more tertiary contacts. I travel fairly frequently, so I had plenty of opportunities for sending postcards. I send cards for obscure holidays just because. The physical process of hand-writing messages is so therapeutic for me. I've probably sent ~250 postcards in the last year and a half.
I have received... 3 physical responses. It has been extremely disappointing, but I continue to send mail because I enjoy the process of writing the cards, and the knowledge that people probably appreciate the mail makes me feel good, so at least I get a little out of it myself.
My mom will occasionally text to say she liked the postcard, but has never bothered to send one back to me.
I would be delighted if more people chose to communicate slowly.
I like this. I don't know what postcard etiquette is, but when I send a postcard it's just to show that I'm thinking of someone while I'm traveling. If the recipient doesn't travel or finds other ways to express that they're thinking or me while traveling (souvenir, etc) I'd consider that social contract fulfilled.
I've told people this for years. The mode of communication reflects the urgency. If you text me, expect a response on the order of 3+ days. If you call, and I recognize the number, it will be more urgent. If I DON'T recognize it, it goes to voicemail and back in the 3+ days queue. If you show up at my door, it is immediate. Even with my wife, she will text while I'm at the grocery to pick up some extra food items, and it doesn't necessarily come through or I'm on silent. I'll get home, and she'll ask where the food is, and I ask why she didn't call if it was timely. I just do NOT check my texts that often, it isn't because I'm deliberately ignoring anyone.
That's funny, I take the exact opposite approach. I prioritize interactions based on how much commitment I expect they'll require, with lesser commitment getting more priority. So a text message I'll usually answer right away. An email or some written reply that requires some redaction I'll postpone to when I can take the time for a thoughtful response. A ring on my buzzer, if I'm not expecting anything or anyone, I'll always ignore; I can't let any dumbass passing by the front of my building rope me into a pointless conversation.
Phone calls don't fit neatly into this scheme because they demand a lot of attention, but it's easy to get out of one if you realize it's not something critical. I generally pick up and the moment I get the slightest whiff of spam, I just hang up.
The current trend seems to be switching the priority order of calls and texts among many of us. I feel like a call should be scheduled, preferably 3+ days out, and preferably with an agenda attached. (Same rules I feel about any sort of meeting.) But a direct text (non-group chat, just to me) is a priority. Group chats get that 1-2 days middle ground.
Wouldn't it depend upon context?
If it is an emergency, it is a voice call. It is both immediate and conveys urgency. If it is something that you need to talk through, it is a scheduled voice call. Asynchronous communications may demonstrate respect for a person's time since it does not (need to) interrupt them in the moment, but the inefficiency results in a disrespectful waste of time for bidirectional conversations.
If it is something where you need a simple response by the end of the day, it is a text. If it requires a lengthy response, email. Never expect a lengthy response by the end of the day, or for it to be handled on devices with terrible input methods (like phones).
Anything that isn't covered by those scenarios will be largely dependent on the person.
I know that's the trend, but it is backwards to me. Like UDP vs TCP. If you need an immediate answer for something, why send a one-way communication where you have no idea whether the person on the other end A) received it, and B) acted on it. A 15 second phone call accomplishes this, whereas if I text you it could be hours, unless you immediately respond.
As an aside to this I mute ALL notifications on my phone. I still get notifications of course, but they never ping or vibrate.
For important threads like calls or messages from important people/group chats, I have my watch vibrate.
Otherwise, I just go through my notifications once I have downtime.
I really like that system! How do you configure that only notifications from certain parties end up on the watch? As far as I can tell I can only filter on application. On iOS I can add “favourites” which get prio for calls and messages in Messages/Mail but not in other apps.
Yeah that's the iffy part tbh. I have a oneplus phone and watch and it is per-app on the watch. So in the app I have to configure which lines of comms are important (so like Signal is how my wife and I communicate so I get watch notifications for that, and I mute other group chats). And because my WhatsApp is pretty noisy, even with muted chats, I just opt not to get notifications on my watch, which has the consequence of me not being able to get WhatsApp call notifications, which honestly isn't the worst thing in the world :)
It's not perfect, but it works for me.
Agree. I mute every group chat and notifications for almost everything. Same reasoning. My wife just talks to me when something reaches a point of me needing to know. Broader holiday planning or group travel planning chatter, it seems like any family gathering requires a minimum of 1000 messages.
I'm noticing some decline of skills I don't practice regularly and LLM is just one of reasons why one stops practicing. Switching to another area of work gives a comparable decline. If you want sharp skills you have to use them.
True. People don't do it though, because keeping skills sharp and using them takes effort, and we have a predisposition to be as efficient as possible with how we spend our effort; if there's an easier way to do it in our awareness, we will naturally gravitate towards that. LLMs are often a universal crutch or swiss-army-knife that significantly take away workload for many abstract tasks, so all kinds of atrophy in abstract thinking is to be expected.
However, when looking at muscle, once you have it you don't need to use it as much in order to maintain it. I wonder if the same is true for skills; in that case, some kind of regiment where you still use the skill you delegate once a week or so could maybe help with avoiding this loss of skill for most part.
“ However, when looking at muscle, once you have it you don't need to use it as much in order to maintain it”
No.. this depends on how much muscle you have. The appropriate comparison is mass and density of knowledge/understanding vs muscle. There’s not a chance in hell you will retain mass and dense muscle without pushing the body hard. Just in the same way you will not retain very deep understanding of things unless a) you’ve been reciting it for over 10 yrs b) you go back and push the understanding continuously for it to remain as part of your being
Building muscle is much harder than maintaining muscle.
And if you went 3 years without exercising, you'll be able to get your muscles back much quicker than had you never had the muscle before.
It's pretty comparable to skills. You don't need to practice as hard to maintain a skill than you do to build it. And if you let the skill atrophy, it's much easier to recover the skill compared to building it from scratch.
> And if you went 3 years without exercising, you'll be able to get your muscles back much quicker than had you never had the muscle before.
This very much depends on age. I went on statins about 18 months, which destroyed about 15lbs of muscle over the course of a year (160->145). Along with that muscle loss came about a halving or more of the weights I could lift in any given exercise. I interpreted the "do you have any weakness on this medication" question as inability to function levels of weakness, it wasn't until I showed my training logs to my physician that she asserted that I was having weakness.
It's been a year since I went off them and I'm still lifting barely what I could in high school. I'm exploring some different training plans, but AFAIK, there isn't much research into if different weight/volume breakdowns work better for older guys.
5/3/1 without any extra sets (no bbb,fsl,ssl etc) is pretty well regarded for people with poor recovery. Slow, but steady and low risk.
Again you are not understanding the comparison.
I’ve got 20 inch lean arms - I know far more about muscle building and retention than you. I train just as hard to maintain them as I did to get them there.
The people who say “oh it’s easy to maintain” LOL it’s easy to maintain 16 inch arms.
> I know far more about muscle building and retention than you
I am a competitive bodybuilder…
> I train just as hard to maintain them as I did to get them there.
Are you enhanced? Were you enhanced when you built the 20” arms? If so, yes I agree.
Edit: With 20" arms, there's nearly 0% chance you're natural. You can't compare your enhanced experience to naturals.
Chiming into this little tiff to say I think bulk muscle is a bad analogy in the first place. It’s more akin to a muscle memory/skill. Something like golf is a better analogy. If you took any golfer, at any level, and had them refrain from golfing for 3 years. I feel pretty confident asserting they would all perform worse than they had. Their skill is diminished.
They would also likely get that skill back faster than a brand new golfer.
I noticed it myself with cycling. Took 8 years off the bike, when I started up again I was nearly back to my old FTP in about 2 months despite starting from basically zero. Muscle memory is real, where I am now as a returning cyclist would take a pure beginner cyclist at least 4+ months to get to, fitness wise.
That said, you do have to work somewhat hard to maintain. With cycling, just 2 weeks off the bike is enough to see a VO2 max drop of anywhere from 4 to 7%. After just 4 weeks, your glycogen storage capacity decreases and you start rapidly losing fitness. After 2 months, you are basically now out of shape.
Detraining happens faster than most people think. And therein lies the danger with over reliance on LLMs for your cognitive skills. Detraining there happens just as fast, skills atrophy in a matter of weeks, not months or years.
People could also regain some cognitive skill back rather fastr when they worked to regain it. But the issue is, many people just lack the motivation to do so. If you golf or cycle, it's likely a passion or hobby. Most people don't view their cognitive health this way, they view it as work. It's why most people don't read much after their schooling, learning and being smart was only ever an ends to a means (diploma, job, money, etc).
I think part of the problem is also that many people simply work too hard or have too much going on in their lives to have any kind of cognitive energy left for this sort of maintenance work, even when they reason/plan that it is useful. This also seems to be encouraged somehow (by society?), to keep going like a freight train, or maybe it doesn't get discouraged enough (i.e. it doesn't get recognized as a problem).
My experience as a parent to an only-child has shown me there's just zero boredom or tolerance of boredom. Any pause or void needs to be filled with something. Any time my son says "I'm bored" my default response has become "awesome", "you're lucky", "I wish I had time to be bored" along with other quips like "boredom is a life skill". Of course, I see this same phenomenon in adults as well. So my rebuttal is that most people have much more free time than they think, it's just a matter of prioritization.
To maintain the muscle you have, you only need about 1/3rd of your normal workouts. It can be retained with 1-2 workouts per week. I imagine the same would be for something you've learned. If you've already put in the effort to learn it, reviewing it ~1x per week is probably enough. During the accumulation phase though - whether it be muscle or learning a new skill - once a week is definitely not enough.
Yes, this is my experience for muscle at least. I used to work out 3-4 times a week, maybe a little more sometimes. Lately due to circumstances, I've been doing smaller workouts about 1-2 times a week. I've lost some finesse, but my muscle mass has remained roughly the same.
Also like some people hinted at this in sibling threads, I think it's different between purely abstract skills, and skills that involve muscle memory. For instance, I could probably stop using my bicycle for a very long time, and still not unlearn how to use it, or learn it again really quickly. Maybe it is because abstract skills are inherently more complex and require more cognitive effort and connections to knowledge overall, and are therefore more fragile.
Most/all of my university-level math knowledge is gone, atrophied from never having needed to use any of it professionally. I don't even really recall needing it for any of my CS coursework, honestly. It was just required for the degree.
I used linear algebra to implement PageRank in my Information Retrieval class. I also used it extensively in my AI and ML classes. You can't pass a ML class without a good foundation in linear algebra. Not to mention, discrete mathematics is the fundamental building blocks of CS. Surely you were using algorithms and graphs. I hope you computed an algorithm's efficiency with big O notation. I hope you have used probability before.
I don't think it's just you or your age, per your pre-internet comment. People that grew up in this just don't understand why they're overwhelmed. And I don't think they're even aware of what their missing out on in terms of focus or mental acuity.
Good point. I do have context and self awareness that it all seems unhealthy. Feels like a common sense evaluation to me but I can’t properly place myself in a younger generations experience.
I’m not noticing the decline in my own abilities
…said every drunk person ever.
That you don't notice it doesn't mean it isn't happening. By the time you notice it, it's too late.
That's why elderly people who are worried about their brains play chess and do puzzles like mad.
I have never heard a recognized elite level anyone stop using their skills and say "I'm not noticing the decline in my own abilities" LOL
I too was and wanted to only blame communication overload. Especially with work the hardest thing in ai times seems to be the overload of stuff/shit to read that is too easy to write.
The reality is I agree with the op and I see the loss of reasoning power in myself. I've been using native Emacs on android for a bit and finally have gotten serious about config for it. I got lazy and had Claude do some of it. Which was great untill things don't work because there's not going to be my crazy ask in the data. It was painful for me to sit down and think through my configuration and the problem but I did it.
I am absolutely torn on the technology still two years after adopting it.
There is a massive difference between remembering how to do something and learning how to do it.
Congrats your getting older. Welcome to the club. Find hobbies and keep them, it doesn’t matter what they are it’s important as we age.
It’s a really lossy process. Mostly due to most humans and all models treating sign meetings as determined at the moment of softmax crystallization. Signs (words included) are no more determined than the speed of light is. It’s all reflexive and we should stop lying to ourselves it can be determined.
It's a really underrated problem. I don't think my actual cognitive skills have declined by using AI, but I do notice that my patience and attention span are a lot lower.
I'm learning a new code base for a new job right now, and I'm finding AI to be a really double edged sword for it. One one hand, it's extremely valuable for asking questions about the code base. On the other hand, if I'm not careful and I just let it apply the fix before I even investigate it, I'm really not learning the code base well at all. I find I need to actually write new code in a code base to exercise the necessary mental muscles to actually retain understanding.
Incidentally, I do find that this large new code base I'm learning also shows the limitations of AI. There's no way I can vibe features on this without understanding and not introduce a lot of issues. Even targeted bug fixes have a lot of unintended consequences the LLM doesn't see. This isn't a bad code base at all, but it's definitely at the size where even frontier models struggle. So to me that tells me that the argument that I should just use more AI to solve my AI issues and not bother to understand the code base isn't viable at the moment.
> I don't think my actual cognitive skills have declined by using AI
I'm not speaking about you but... I know most people would not have much awareness of their cognitive decline. I know this because that awareness gap is there with or without LLMs, across all age groups and cultures.
Especially since attention -- which the parent commenter says has been diminished by LLMs -- is a key part of cognition.
adding to the irony is the fact that the mechanism used by the NN architecture in LLMs is also called attention
For cognition, sure, but that's a fairly weak claim. A dog that chases its tail for 3 hours might be considered conscious, but maybe not highly intelligent.
The attention deficit part of ADHD hurts some people, but a high intelligence is able to make up for it in other ways. Attention span is a multiplier for intelligence. Someone with a lower IQ but a longer attention span is able to outperform a higher IQ but shorter attention span, traditionally.
What's required though, is the attention span and the memory to really dig deep into a problem, and then go for a run. If AI makes that easier, since it lets you skip the boring parts and get to the meat of the problem, then hey.
True, I guess I try to have some objective measures like my chess elo and maybe some canaries like what books I'm reading. But it would be really hard to tell.
I used Google Translate to not learn French in collage. Fortunately for me it was bad enough I had to carefully review all its outputs, but that still didn't help and I managed to pass two semesters without ever developing even basic language skills.
Something radical needs to be done. When I was in high school there were still a lot of "no calculator" restrictions in my math classes that I chaffed at because I hated doing longform arithmetic and felt like it got in the way of learning. So I can certainly understand how students would chafe at some kind of paper-only education system but I also don't see how you can learn anything when you have a high-quality homework machine just sitting there.
I wish that would have worked for me - we had oral tests. 2 years of French in high school and one semester at college - what an absolute waste of time. How much French do I know now? Basically none. The same goes for everyone in my life that did Spanish instead in high school.
Part of what we could do during this upset is re-prioritize.
All that's needed is a tight feedback loop between learning and applying those skills ... the thing that Google Translate helped you evade. AI can be a tool for evading or optimizing that loop, like a knife can cut your sandwich or your throat. Your choice.
It's the "your choice" that's the problem. The quality of a society is dominated by the choices that other people make.
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
I agree - I would have been toast. I wonder if the teachers/colleges need to change the way they teach and assess. Let the students use the AI tools they like (perhaps guide them how they can use them professionally), but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person. Oh and don't give Fs for cheating - suspend them.
I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.
EDIT: Claude beat my Googling: This was 2 chemistry high school teachers in 2007 - The Flipped Classroom https://fltmag.com/the-flipped-classroom/
Absolutely university has to change. But it's not a simple change. I say this as a professor for Physics:
My colleagues say "We must fully embrace AI as a tool". I agree. But how do you teach it? It's a moving target, and you can't even give homework like: "Research <this topic> with an LLM of your choice, and submit the transcript" because they can do that, or they can just copy the task into an LLM and have the LLM do it. It becomes meta quite quickly.
And independent what and how we teach, we have to change how we assess a students learning result:
The first thing we have to change is that homework needs to be completely ungraded. Reviewed and corrected, yes, but not part of the grade. That's the only way to make sure that people who don't want to cheat have to cheat anyway to compete with those that do.
Second, all exams have to be in person. Online, cheating is so trivial it's not even funny (many students are so stupid about it that we have a pretty clear idea what's going on). In person, we have maybe 2-3 years until we have to make sure its proctored and people's glasses are checked. I think in less than 10 years, local mobile AI will be good enough so even a Faraday cage will not help.
Maybe we have to go to oral tests only.
Of course, none of this scales. Some of our intro courses have a thousand students.
Any ideas are much appreciated.
> Maybe we have to go to oral tests only.
"Perfect homework, blank stares: Why colleges are turning to oral exams to combat AI" https://apnews.com/article/college-oral-exam-ai-chatgpt-7795...
I feel like German university was already well "LLM proof". Come to class or not, no ones keeping track and it doesn't matter for grade. There are regular exercises, but they are not graded; you can submit them if you want it corrected to check your understanding, but there is also gonna be a short presentation going through it in class. Your entire grade is a 2-3h written exam at the end, no materials, no remote, no books, no multiple choice.
That's how it used to be. At least at my alma mater, it changed a lot when they switched from Diploma to Master/Bachelor
>Of course, none of this scales. Some of our intro courses have a thousand students.
Any ideas are much appreciated.
Oral exams graded by LLMs? Scale with the improving models. Based on GPQA Diamond results they're mostly at PhD level for subject trivia anyway.
The problem here is that it will work for now, but how do you make sure the LLM talks to the student, and not a different LLM? I guess vision models FTW?
In the end, will be build a GAN loop?
Why am I now reminded of corewars?
>(perhaps guide them how they can use them professionally)
If that's anything like how they guided me to use programming languages professionally...
In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial. They've barely adapted to video conferencing.
How do they keep up? Honestly it’s too risky to make a serious move on such a fast-paced environment.
The only thing I can see them doing is removing technology altogether. People did just fine 100 years ago.
Want to learn to code? Use a Commodore 64. The company was purchased and rebooted the C64: https://commodore.net/
Traditionally, moving slow with policies was fine with new tech because, outside of the PC revolution it wasn't all that impactful, and things used to rightly be labeled as experimental so you could safely ignore it for a while as a big enterprise and be just fine until thinks shook out.
LLMs were, IMO, pushed out too early and without that clear "this is experimental tech" label. Full public access from day 1, no invite only betas, no research previews for a select few pilot customers/orgs, etc. I've been in IT for a little over 18 years now and I haven't seen anything move this fast before.
I mean, I never though I'd see Microsoft go on stage at BUILD and and announce freaking OpenClaw for Enterprise, and then make it available the same day. This is highly unstable tech and what I'd consider still experimental, being sold to F500s as production ready.
> In my workplace I find systems and policies move too slowly to keep up with how rapidly the LLM world is changing. Colleges are even more glacial.
Perhaps this is rather a sign that you currently shouldn't jump on the LLM hype train, but rather attempt to get a good foundation on the basics. When the whole LLM area becomes much more "stabilized" (I see signs that this is currently happening, if only for the reason that training state of the art models has become more and more expensive), you can still get into LLMs if you want.
I partly agree, depending on what you count as the basics. I don't think there's much value in learning the quirks of LLMs today: they will just change, your value-add becomes part of the model or harness.
On the other hand I think there are real development gains in jumping on the train today. To my career's detriment.
Yes, meanwhile, Claude Cowork was only released this past January. And that was amazing. But I don't know about anyone else but I've already moved on to just using Codex for just about everything (except some Kagi use). Schools work on timescales of years, AI is advancing on the timescale of weeks and months.
Until that situation stabilizes I think the only institution capable of teaching about it is the family -- parents.
I'm not sure parents have the right tools either. Microsoft is about to ship OpenClaw as part of windows (talked about at BUILD) and they're acting like it's production ready and they've solved the security issues.
I don't believe them for one second, it's far from a solved problem yet these companies are selling this tech as if it's been around for decades and thoroughly battle tested instead of highly experimental and unstable.
Ancient brains, medieval institutions, godlike technology.
Tristan Harris had some sort of comment like that on a podcast about the challenges posed by AI.
I think it's incredible how much those ancient brains can successfully adapt to technology. Some people can sit in highly-strung sports cars and use them at the absolute limit of their performance like they're just an extension of our own limbs and senses.
> I read a few years ago about a teacher (I think highschool) who put his lectures on YouTube for students to view in their own time and then used the in class hours for interaction, questions, tests.
That seems like a smart approach. It reverses the traditional model of "lecture in class, homework outside of class".
A smart approach that does not solve the AI problem - actually flipped classrooms work worse now due to AI usage.
My own experience with flipped classrooms (which seems to be shared by quite a few people who have tried it out): they only work well if all students actually read/watch the materials beforehand. In small, advanced courses, intrinsic motivation may be sufficient - but in most cases you need some extrinsic coercion - such as a mandatory quiz about the materials or hand-written lecture notes that need to be shown at each in-person session.
With AI, some people don't watch the lectures but let ChatGPT give them a summary which they submit. Then these people poison your in-person session with their lack of knowledge and motivation.
Research has shown that testing is far and away the most valuable academic tool.
Just have a quiz every day. In fact, have _TWO_ quizzes, one at the start of class and one at the end, and take the higher of the two scores. In between the first quiz and the second, work through problems with the students designed to help people that bombed the first test figure out how to pass the second.
The best part of a quiz everyday is that in addition to the testing effect, you can easily fit in the spacing effect and interleaving effect. It’s a rock solid combo, that is well studied. We have pretty strong evidence that it works for all students in all domains.
I actually like this idea - makes sense at face value - as long as they design the test in such a way that it aptly applies the knowledge instead of just learning for the sake of passing test like questions...
I had a flipped classroom for my topology lecture. It was one of my absolute favorites.
We had no lectures, the teacher just gave us a short, concise textbook to read a chapter of every week.
In class time was devoted to discussing and problem solving.
But yes, it only worked because we were a small class of 15 math students
My Cryptography professor did this during COVID, since the classes were split in person. It was an interesting model. I'm not sure if I loved it or not, but it was at least a change of pace. Getting 100% of the class time to ask questions was really nice, but it ended up with him re-teaching most of the online lecture in class because some quarter to half the class just didn't watch the lectures.
If done more stringently (if you didn't watch the lecture, I'm not reteaching it), it maybe would've had a bigger impact, but I'm not sure.
Office hours remained king for serious Q and A for the class.
One way to fix that issue that I’ve seen is a daily quiz to start the class. The key is the quiz is super easy. Even if you were confused by the lecture, if you watched it at all you’d likely get a 100 on the quiz. If you didn’t watch it you’d likely get a 0. This quickly for people watching the lectures online ahead of class.
I always absolutely hated when a teacher did a reverse classroom and I had to “learn” at home and then practice in the classroom. I think the solution is more engaging lessons and less outside work. I know why homework exists, but homework is a chore that most people want to get done as fast as possible. If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI. If they’re interested in the topic, they’ll put more effort into it. If not, they were never going to retain it anyway
> homework is a chore that most people want to get done as fast as possible
Worksheets certainly are. But good homework, even if it's challenging, is what makes a reasonably fast-paced course even possible. In a well-paced university course you're typically spending proportionally several times as much time working on it out of class than you are in class. Then class time is both preparation and catch-up, similar to office hours.
This was true of my most demanding humanities courses (sometimes reading 100 pages a week directly from academic journals, not easy reading) as well as my most challenging math courses (group theory, ring theory). Once the pace gets fast, there just isn't enough time for you to learn everything you need to inside the classroom anyway.
And in those classes, where homework was really essential for learning at the required pace and depth of mastery, my instructors didn't even need to factor the homework into my grades at all. In some of them, we could get "feedback" on homework but it was never officially recorded in our grades... and yet, anyone who didn't do it would fail the next test. If homework doesn't have that characteristic, it probably doesn't need to be assigned at all.
If "flipped classroom" means that students are expected to do all of their homework in class, then indeed it'll feel like a waste of time to many of the smarter kids, and it will also just be unfeasible for advanced courses (which theoretically should be most courses in a university, though it currently isn't). But if it means "we don't even have time to lecture you on every single thing you need to learn, therefore you must arrive already having done the reading and the exercises, and we'll use this time to help clear up misunderstandings"... that's already how classes for grown-ups are in universities.
When left to their own devices, 99% of children are not interested enough in math, history, literature, languages, or almost any other school subject to engage with it willingly. Only teaching "interesting" things that kids are "interested" in is both impossible (too many varied kids per class for that to work 100% of the time) and even if possible, would leave kids with zero practical knowledge, because learning most of that stuff is not something kids inherently want to do.
College is different, because theoretically you should be taking classes that are relevant to your field (although there are still "core" requirements that are somewhat high-school adjacent).
I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.
College is a different dynamic from a middle/high school classroom, but I don’t remember 95% of the material from my college engineering classes anyway, it’s the problem solving and information finding that I’ve retained and have helped me do the things I do. I remember the stuff from the classes that taught me the material in an engaging way though.
>I’m not saying only teach interesting things, I’m saying teach things in an interesting and engaging way so that kids don’t feel the need to cheat their way through it to just get it done.
"Just do it right and it won't be a problem." This is not an actionable plan. What is engaging? Who gets to decide that? The teacher? The students? The parents? How do deal with certain kids finding different approaches more or less engaging? How do you expect a teacher to curtail their teaching approach to dozens of children at the same time?
>If kids got to learn something interesting in school and then have their free time after school, there would be less dependence on AI.
Kids get to learn lots of interesting things in school. The problem is that they're kids! They want immediate gratification from phones/games/recess, not to do the hard work of learning.
Flipped classroom pedagogy has been the subject of a huge amount of research. Ultimately "one weird trick" solutions don't tend to work in education. Enough students don't watch the lectures that you end up needing to go over the material in class anyway. Funding and autonomy works, but nobody likes to pay more.
How does this scale in practice? We already require students be at school for 7 hours a day. If they now have to watch 3-4 hours of lectures at home every day, then students are left with little time to do anything else.
What about those students who don't have stable home environments? How are they supposed to find multiple hours a day to watch lectures?
How does this address the underlying issue of students off loading work? You've replaced homework with lectures, but haven't solved the problem of making sure the student is actually participating.
Logistically, this could only work if you shortened the school days, but then you would need to adjust the rest of society around that. Many parents structure their work days around their kids school schedules, and if kids need to go school later in the day, or get out earlier, that places a burden on the parents.
From my experience it works fine if it's one class that's doing it. If multiple classes are doing it then like you said it's literally a couple extra hours a day watching lectures and most students end up skipping them, forcing the instructor to end up teaching during class time anyways.
We’re discussing university, right? It’s supposed to be a full-time effort, at least for a normal pace undergrad or any post- graduate program.
For secondary school, I do agree with you - homework load can be problematic for some students. But at the same time, my honors classes all came with hours of homework and I’m not sure I would have been as prepared for uni without it.
> Let the students use the AI tools they like […], but test regularly and early on the skills/knowledge they're meant to be gaining offline and in person.
I very much doubt there is any agreement on what those skills are.
Creating the idea of “what to learn in the new world” is itself IMO an important academic creation, but there’s no reward for doing it and no way to know if you’re on the right track (you just have to wait and see).
Employers are also just adapting.
Wait until companies are paying unsubsidized “list price” for LLM usage. Then we can have a better idea of the worth of the automation and what skills should stay with humans.
This. The industry is dumping electrical labor at a humongous loss JUST BECAUSE they figure people will immediately atrophy and be unable to do without AI… at any price.
We'll get an idea of the relative cost of the labor, all right. It's just that they are specifically trying to wreck the market, at all costs, to be able to cash in on the upside. It's sensible, if you're a monster.
I'm dumb as a rock and I don't have a PhD, but since ~1 year ago I started forcing myself to do small bits of coding and math manually.
I'm not noticing a "cognitive decline" per se, but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.
>I'm not noticing a "cognitive decline" per se
The funny thing is, maybe not noticing one can be the actual sign of it :)
Yes, precisely. Assessing your own cognitive skills is dubious. I’m pretty certain I’m less clever than I was when younger but if I find a problem tough now maybe 25 yo me would also have struggled?
That’s the most important thing. If we keep reading, maybe we can hold our own.
> but I do see I'm a lot "lazier", even stuff that used to be routine when I started coding now feel heavy.
Not getting that quick dopamine hit the LLMs give you..
Some say you can re-train your system to get back the dopamine hits you used to get from other things, like the enjoyment of the "old fashioned" manual coding and math. Getting there is hard work. And YMMV.
I just do things manually and ask LLMs to check my work. That seems to be working great for me.
I had the most Russian of Russian bosses when I was in college. My first day on the job he so eloquently stated, "I am not your mother. Do not come to me with problems. Come to me with solutions. I want to know what you tried and what did not work."
His advice has served me well in many areas of life too. I try my best to treat LLMs no differently for domains I care about (not one-off little questions here and there).
>even stuff that used to be routine when I started coding now feel heavy.
The same weight feeling heavier is a sign that your muscles are weaker :)
There's many areas in life were we look back a few decades and think "people use to do it that awkwardly?" And yet results were better. I think the process of removing friction have just served to destroy our ability to concentrate and tolerate difficulty.
Absolutely this, I'm the same as you.
And I'm just afraid this is what cognitive decline feels like from inside the deteriorating mind.
I do a similar version of this, where if I notice a mistake in generated code, I fix it manually (or at least attempt to) instead of telling Claude to fix it.
This is the right balance for me as well.
I use an agent to generate a first-pass attempt, and then (deadlines willing), I manually read every line at least once so I understand what the code actually does.
Then I manually fix the inevitable slop that is mixed in with the good stuff, and only once the code is up to my personal standards do I send it.
This probably reduces my “AI performance boost” to 30-50% instead of the huge gains reported by others. But I retain the ability to reason about the codebase and use AI much more precisely when I’m trying to troubleshoot production outages or subtle bugs — something I notice the rest of my team struggles with, since adopting “agentic workflows” everywhere.
I think actively working to retain some cognitive flexibility and “muscle memory” around coding tasks is going to be rather advantageous in the long run.
Pure copium, but what can you do with the deadlines.
Same, but also because it feels like it takes longer for an LLM to do it. I think that's something people who are into gathering personal metrics should do - measure how long it takes to type a prompt / have the LLM fix things vs just doing it yourself.
“ I'm not noticing a "cognitive decline" per se, but I do see I'm a lot "lazier"”
These are correlated - it just hasn’t happened in a large enough amount for you to have clearly noticed it yet.
LLMs are making me smarter. I have more code to read!
Why do you think you didn't do similar when you were a student? We've seen this exact issue with other technology before: calculators for math; typewriters and then word processors for writing assignments; audio books for literacy. In those cases we've collectively realized there's a benefit in getting the manual skill and understanding how to use it before shortcutting things with the technology, even if most of the time you'll just end up using the technology. My best guess is that for most people in those classes who failed because of ai use, they don't care about the understanding (usually) required to get a good grade, they just care about the grade itself and the doors that opens for them.
It's because modern AI promises to relieve you of the tedium, leaving you to consider the important things like higher structure. It actually does deliver on this, but in contrast to older tools, it is unlimited in scope.
A calculator - let's expand this to maps, thesauruses, dictionaries, and other lookup tools - was used for a pretty narrow set of problems, and you had to transcribe the result to whatever context you needed.
An AI can be all the calculators together, and transport the output of one to the input of the next. You're meant to have the overview, but it's just so enticing to let it simply do that as well.
Counterpoint, I think this is true for some archetypes of people, but certainly not everyone. I personally use it like the socratic method. I am an intermediate user, I spend a ton of time with LLMs at work and personally, both prompting and letting some crappy agents try to automate boring work. I primarily use Gemini and ChatGPT models, along with some Chinese smaller weight models (eg qwen) locally.
If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.
This, I will use Obra Superpowers brainstorming skill to propose/refine a few viable solutions for a feature or bug I'm trying to solve. After it asks me clarifying questions and presents a spec, I will say "well what about X or Y". The I'll run the grill me skill on the spec to tighten it up, clarifying any assumptions made.
I find it to be a really tight loop and results in very high quality code at a high velocity.
Are you also one of those people who believes that advertising works on other people, but not you?
Advertising is fine (not great), especially when highly targeted and relevant. Spam, misleading, or predatory advertising is not.
My two modes of using LLMs has been to try it for 1) natural language search queries where traditional search engines have failed and 2) occasionally as a sounding board using the socratic method.
Inevitably, it fails frequently at both. Any "reasoning" it is doing is merely rehashing ideas that someone else has already posited. This helps some of the times, but the vast majority of the time it just chooses a biased perspective (frequently the most common) and then regurgitates tired old talking points. This contrasts greatly to speaking with others who often have more intuitive notions that tend to be less polished and rote.
I'd love for LLMs to be better sounding boards, but so far they fail miserably far too often for my tastes. To each their own though.
> If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.
Yes, but eventually the intellectual whack-a-mole gets tiresome unless you get really, really good at simultaneously cornering it and not letting it concede to your point.
new session. It's easy to lead a model into getting the response you want, deliberately or accidently.
The point is not to literally win an argument (it doesn't matter), it is to use the model like a partner to poke holes in your own understanding. Once it's poked a hole, it has served its purpose. Plus, you eventually run out of context or the model trails off into babbble.
That is how I use models too. Sometimes I have them role play the supporters of the idea I am arguing against.
We used to prosecute cheating and plagiarism for the moral failure that it is.
If we allow lying, cheating and stealing - why bother being a schmuck that does the work.
> For adults the cognitive decline won't be as measurable since there's no exams, and overall output volume will still be fine due to LLM help
The leading indicator for me is the amount of emails and, god forbid, more personal messages (like birthday wishes!) I see that are obviously AI generated. It just keeps on rising. If you’re not able to dash off a quick message without the help of AI I have to assume you’re using it heavily elsewhere too.
I have sympathy for the university students too, we’re all bombarded with rhetoric about AI being the future. And I remember being incredibly nervous emailing my lecturer (am I phrasing this right? Is it respectful enough?) that I can imagine leaning on AI myself had it been available back in the day. But I’m glad it wasn’t, it’s an important skill to work out this stuff. They’re going to land an in person interview when they graduate and stumble around unable to effectively answer the questions they’re asked in real time.
> They’re going to land an in person interview
Not necessarily. It can be either an AI interview or a record that will be analysed by AI later. So there’s a chance to cheat here as well.
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
So, as long as you are not under time pressure (which you in some degree courses unluckily are), there is simply no need to "speed up" any homework assignments.
If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study (which is only loosely correlated to homework and tests), I guess it's fine to use them. Just always keep in mind that very often the pain of attempting to understand the topic on your own often makes you smarter - something that you will miss when you take an "LLM shortcut".
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
This is probably not true for majority of people. Most go to school because it is mandatory, pushed by parents and society, and university gives you credentials and better job opportunities. Homework and tests are a way to get a number grade on 'how well you memorized something', it doesn't really measure a deep understanding of the topic.
> Homework and tests are a way to get a number grade on 'how well you memorized something', it doesn't really measure a deep understanding of the topic.
As I said: they are goalposts.
Typically homework and tests are sufficiently easy (yes, there are exceptions) that if you fail them, you can assume that you didn't make sufficient progress in improving your understanding.
But I do agree that at least sometimes the difference between being good and exceptional at homework and tests can indeed be rote, "unnecessary" memorization.
Uni grading Brownian-walks around edu trends, but misses the point that improving one's (and humanity's) lot depends on a tiny loop:
- doing - failing - discovery>learning - remembering
With learning predicated on both failing and remembering it's unfortunate uni scores on 100% successful doing but doesn't teach failing well, and scores for remembering but not for learning well.
> You go to a university because you are deeply interested in understanding the subject that you study.
Echoing the other comments here, at least in the US, this is generally untrue. I went because my parents made me, because the choice was that or get kicked out of the house. It was beaten into my head since I was in grade school that "people in this family go to college" and "you can't get a good job without a college degree."
I hated every moment of it and I was glad to take my BSc and never look back once it was over (University of Houston, c/o 2000). And, indeed, without the degree I wouldn't have had the jobs I've had.
But I didn't go because I was "interested." I went because it was an effectively mandatory life-path objective. I'm very happy for you if your lived experience is different, but it is also—at least in the US—both extremely uncommon and extremely privileged.
> You go to a university because you are deeply interested in understanding the subject that you study.
This has not been true for something like 70 years now. People go to university because it is expected that that is what you do after high school.
> This has not been true for something like 70 years now. People go to university because it is expected that that is what you do after high school.
In Germany, many people indeed say if you are not deeply into the topic that you study, you should rather get a vocational training (Ausbildung), or attend a different kind of tertiary education than a university such as
- Fachhochschule
- Berufsakademie
(these words have no good English translation). Basically these are kinds of tertiary education that are more applied than the much more scientific training that you get at a university.
Specifically for mathematics (I guess the same holds for physics), a lot of people say that if you don't consider it to be an ideal life to think about math exercise sheets when you sit in the bathtub while other people are having fun at some party, you simply are not made for studying mathematics and should change your degree course as soon as possible.
This situation is changing in the US but isn't well reported, in my opinion.
We haven't regained traditional apprenticeship roles (perhaps because we so weakened unions?) but 30 (of 50) States have free or heavily subsidized two-year community / vocational college programs. Affordable and accessible vocational education opportunities are increasingly present. I also think (very subjectively) that we are seeing a renewed respect for the trades.
However, there are structural headwinds outside of education - no national health insurance plan being a major one. Farming, fishing, forestry, construction and similar trades still have a 20-30% uninsured rate in the US. (The uninsured rate in white collar "professional" work is around 2.5%.)
> We haven't regained traditional apprenticeship roles (perhaps because we so weakened unions?)
The reason for the traditional apprenticeship roles is not unions, but rather capitalistic:
- If potential employees are well-trained the employer doesn't have to invest resources for training them.
- The certificate of the vocational training means that the employer knows that an applicant has an established standard, and can save time testing whether he is qualified.
- Because the trainee needs practical experience, employers can invoice this additional worker to the customer. Because the trainee needs explanations and thus works slower, more hours can be invoiced to a customer.
It's a much more socially beneficial system and I think a large part of this is probably differences in culture regarding education and one's career. The availability of these modes of teaching is downstream from that.
It wasn't even true before...
Universities have always had people not interested in the subject, but who went because its elite training school. I was reading something about Issac Newtons college in the 1600s, many of his classmates didn't care about studying and drank all the time.
It is really about time we thought about what universities are for in the 21st century, since there has been significant scope creep wrt labour markets, particularly roles which do not actually require university education but do require a degree for CV reasons. It is nonsense in the 21st century to require a bachelor's degree for such roles. Not to mention the huge societal pressure that you have mentioned, which no 18 year old can really be expected to see through.
With CS students this is one thing. Medical students? Air traffic controllers?
That is to say, there is a huge gap in the educational integrity of degrees, and this is probably partly driven by people who do not really want to be at university for educational reasons (and, believe it or not, there are other ways to party in your early twenties) and for whom a degree in XYZ is not rationally connected to 80% of their options after school. And there are many such people.
This really needs to be thought through, because education is expensive, and it is an enormous waste of money to pay for a couple of years of university and end up failing out or being sanctioned for AI cheating, or being educated for something you do not really want or need to be taught. That is true whether or not education is paid for privately or by the public.
ETA that when I graduated from school the idea of not going to university was really discouraged by the guidance counselor. It seemed like vocational courses were not really a worthwhile option unless you were a poor (significantly below average) student. There was a lot of emphasis on ‘getting a degree’ probably related to (nonsense) job requirements. Not a lot on what career you should pursue, or why you should consider university. It was more like why would you not consider university, since it was the de facto default. It was, I guess, unseemly for the school to end up with fewer university entrants and more apprentices.
At the time, there was somewhat of a social stigma with apprenticeships. The people that pursued them seemed to only genuinely have been set on the idea, and there were few if any that were diverted thereto. Now, of course, ‘the trades’ pay much better than a middling office job. Egg on my face.
Universities today are seen as debt manufacturing facilities. Debt that cannot be discharged. AI is seen as an act of war against the world’s working classes. Rich people aren’t building bunkers in foreign countries and buying yachts the size of a town because we ran out of land in America. Buckle up for a wild tumultuous period of human history.
Sure, I think this is a major element in the States, but I have never been there and all of this applies to my country where student loans are rare and fees are paid by the State.
Rest doesn't make sense either with the parent's tautologies and self-counters. They seem to argue for it and raise the challenges.
~"Speed doesn't matter unless you need it."
~"LLMs can be good, but if you don't use them properly™, then they become a crutch."
It's hard to deny that "cognitive offloading" via LLMs is becoming a more acute problem [0]. The intelligentsia were supposed to be immune.
[0] https://www.bbc.com/future/article/20260417-ai-chatbots-coul...
> You go to a university because you are deeply interested in understanding the subject that you study.
I don't think I've met anyone who fits that description. The ones deeply interested in the subject would likely skip college anyway if not for future economic prospects.
> The ones deeply interested in the subject would likely skip college anyway if not for future economic prospects.
There exist a lot of things that are much "easier" or even (currently) only possible to learn by attending a university because, for example,
- for the access to various devices and experts,
- you walk a much more "established" and "time-tested" hike for getting good in the subject,
etc.
>The ones deeply interested in the subject would likely skip college anyway
Spoken like a true software engineer ;), there are jobs where you have to have a degree to get the job. "Real" engineers with sign-off responsibilities, Medical Doctors, etc.
Then you either really haven't tried very hard to notice them or have been in an academic environment with severe defects.
Does college even work for future economic prospects, by the way?
> Then you either really haven't tried very hard to notice them or have been in an academic environment with severe defects.
Sure. (?)
> Does college even work for future economic prospects, by the way?
Where I live, a college degree is a legal requirement for a lot of professions that pay more than entry level jobs (although not all of them). So, people go to college to get a better paying job in a few years than they could get by immediately entering the workforce.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
Some students do not have this privilege and implicitly see university as first and foremost a funnel into a paying career.
What a delightful fantasy world you live in. Doesn’t sound very predictive of actual human behavior though.
> What a delightful fantasy world you live in.
I can really certify that this was my lived experience. In the math degree course, basically everyone who was not incredibly passionate about mathematics (NB: "passionate" does not necessary imply "great academic achievements") changed their major or decided for a different kind of tertiary education.
Former co-students who attended the same university and degree course had the same experience.
I guess the reason was that it was a decent university in a "boring" town where learning for your studies was one of the more exciting things that you could do.
Another factor might just be that math pretty much is the extra depth behind a bunch of STEM fields, so people studying math specifically are more likely to be interested in that depth.
That said I generally think the take that it's somehow privileged to find school interesting to be sad. Over the last couple decades one could do pretty well with pretty much any STEM degree. Is the majority feeling among people studying engineering that they just have no interest in any facet of how the world around them works? They have no desire to understand how to create (and alter to their liking) the things they see? No interest in the fundamentals of how the universe works? How different materials come to act the way they do? How living beings work? Nothing?
1. I don’t think there’s a direct correlation between curiosity and finding school interesting. I’m endlessly curious about how the world works and always reading three books at a time, and school was often dull as paint.
2. I would optimize hiring people who display the kind of curiosity described, but if my goal was to create an education system to generate educated workers to grow an economy, I wouldn’t optimize for it. I don’t think curiosity is a privilege, it’s an undervalued right.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
There is only one classmate in my class who came to study CSE because they are interested in CSE. And since we all enrolled after AI became somewhat good at everything none of them know how to code. After two years of study I had to explain someone how to swap two number by drawing boxes. This are the things you learn in the first week if you're interested in programming.
My point is very tiny percentage of people study something because they're genuinely interested in that subject.
The main reason to go is you need that piece of paper.
I needed the structure more than anything. Nothing I learned in university was some secretive knowledge. Anyone could buy the same textbooks and learn the same materials.
However, I am not one of those people. I lack(ed) the self-discipline required to learn anything with moderate to long feedback loops, thus it was far wiser for me to outsource all the necessary planning and assessment to a structured program.
Student loan debt is a blight on society, but I'd be lying if I said the looming repayment was not a great motivator to attend classes and do well in school. Some things just hit differently when there is skin in the game.
> You go to a university because you are deeply interested in understanding the subject that you study. Doing the homework and the tests are just the "goalposts" to check for yourself whether you made progress on this.
I think this was true a long time ago. Perhaps with LLMs this can become true again in the future. But definitely that was not why I went the first time, nor most of my classmates. (Second time I did post-secondary, sure, 100% -- but I was almost 30, not an average student)
It's easy to fool oneself into thinking one knows the subject because somebody explained it well / demonstrated skills, and it made perfect sense.
Unfortunately that, on its own, very much does not translate to being able to explain it all oneself, or to having the skills.
Ease and norms of outsourcing to software invites and amplifies this trap, I think.
> You go to a university because you are deeply interested in understanding the subject that you study.
This is a bit of a naive or maybe affluent take? Like, theoretically, I agree. And I myself was curious. But most people, by and large, are going to university because they know they need a degree to get a job, unlike their parents or grandparents. And even "the degree" is quickly becoming devalued in this current AI age.
I would guess that if all basic needs were met through UBI, the fraction of individuals going to school would drop and the makeup of subjects they pursue would change. Probably more cooking and art classes and less stem. Although, if UBI existed and AI did not, we'd probably see more educated individuals in the first place so maybe there would be an uptick in stem attendance and general curiosity in such a utopian world.
> > You go to a university because you are deeply interested in understanding the subject that you study.
> This is a bit of a naive or maybe affluent take?
Concerning the "naive" aspect, I wrote something at https://news.ycombinator.com/item?id=48397759
Basically, this was really my lived experience, which might have been amplified that it was a decent university in a "boring" town where learning for your studies was one of the more exciting things that you could do.
Concerning the "affluent" aspect, I can clearly assure you that neither I am nor my parents were.
I think perhaps the reason you are seeing quite a few commenters expressing skepticism to your comment "You go to a university because you are deeply interested in understanding the subject that you study." is that you appear to be extrapolating from one example (your own), without considering whether that's likely the wider experience of people going to university.
In the UK anyway, there's an acknowledged idea that many people go to university because there is a societal expectation that they should and also because many careers require a degree even for entry level positions.
There is also much less emphasis on other routes of tertiary education (e.g. vocational schools), when compared to places like Germany.
> "You go to a university because you are deeply interested in understanding the subject that you study." is that you appear to be extrapolating from one example (your own)
I know a lot of people who think this way, and I can assure you that the people who realized later that university is not for them deeply would have wished that someone had given them this advice when they were younger.
> If, on the other hand, LLMs help you with making much faster progress in understanding the subject that you study
My experience is that they uncomfortably do both. You can "understand" something conceptually quicker -- like you have a new brain-muscle-thing that lets you cut through the hard difficult tedious corners to get to the meat of the matter.
But then you also can become reliant on it, and have difficulty doing the mechanistic rote work of working through it yourself.
Like the really big powerful calculator that it is, really.
It's two fold. They're learning and understanding more things, but at a very surface level and without the nuance and ability to actually use the knowledge because they have none of the muscle memory and hard work associated with learning it.
You can use AI or the internet to learn the basics of how a gas engine works in a couple of minutes. But you'd be incapable of actually working on a gas engine or designing one.
Surface level knowledge gets you surface level functionality. You don't become good at something from surface level knowledge, but you might think you're good at it.
Perhaps it was merely my degree or program, but I do not feel like I learned a lot more than surface level knowledge.
I have an accredited degree in computer science. But what does that truly even signify? For example, even with my 10 years of experience coupled with my degree, I could not create a programming language from scratch without some tutorial, nor could I create an OS, a compiler/interpreter, a basic CPU, RAM, any secondary storage, etc.. Sure, I could tell you about basic digital logic, basic OS principals, basic algorithms in a poorer explanation than a Wikipedia article, etc..
To use your gas engine analogy, if you dropped me on deserted island with all the parts I needed to make a basic, simple computer I would unlikely be able to do so with out some sort of in-depth guide. I am not talking about slotting mass produced parts on a motherboard. Rather, something like this:
https://www.instructables.com/Build-a-Computer-W-Basic-Under...
If used correctly though I think the models in fact can be useful for gaining depth. With the right prompting they can actually perform a pedagogic role. One must just resist the temptation to have them do the work for you.
I've used my phone taking pictures + Codex + a PDF of my tractor manual to help me effectively diagnose and manage repairs in my tractor. (Though these models remain terrible at the physical world, getting physical orientations wrong, front back etc. Much like myself)
Likewise I had Gemini help me tear down my mower's carburetor and diagnose issues there.
(So much so that I've wondered about building some kind of "shop buddy" -- some kind of durable laptop and set of cameras ... on a cart. Running models that have access to manuals and cameras and TTS and voice input? "Hey, shop buddy, look at this fuse and tell me what is before and after it in the electrical system.")
This is helping me learn and do something I couldn't really effectively do before by walking me through steps.
My youngest has had Gemini write math questions for them, to help study. Not do the math, but write questions.
In the end it comes down to prompting, like everything.
Which makes me wonder if the answer for higher education is just to provide the students with specific coding agents they're specifically allowed to use -- ones that would push the student through problem solving and working on the problem together.
> One must just resist the temptation to have them do the work for you.
We are in the instant gratification era of humanity where a dopamine rush drives most people. This is a systematic shift that happened through the introduction of smart phones and social media and then progressed for a good decade to what we have in front of us today.
Asking people to "resist the urge" when they've been programmed/brought up to feed the urge is not pragmatic unless you are also proposing a way to erase the damage done from the instant gratification era.
We're in the end game presently. For every one person like you and your examples, there's gotta be 100x or more who are not using the tools the way you've presented them.
I'm no saint. For coding projects I am absolutely in the same boat as everyone else.
My other examples have to do with current limitations of the tools. Obviously there's no Claude Code for Meatspace that just takes over and does things for you. (Yet)
What I'm trying to point out is that the tooling has been made this way on purpose and I agree substantially with your point. But I also think human agency is involved. Dario & Boris et al didn't have to write CC the way they did. They chose to play with and push a concept which reduced human agency -- in part because Dario concretely believes it's just "inevitable" that we should be put of of work. And his investors no doubt love this concept too.
And just like Facebook / Instagram etc. it turns out it's an addictive flow.
It remains the case there are other ways of applying LLMs and generative coding models. This modality is not intrinsic to the technology. It's being deployed this way. And humans have agency in how it's applied, even if it's hard sometimes for us to exercise it.
> And humans have agency in how it's applied, even if it's hard sometimes for us to exercise it.
It needs to come top down from CEOs and governing bodies via regulation if we want improvements. We can't rely on the individual to not use the big red button that says "do this with no effort". We're on course for a WALL-E future if we're lucky or something far less great if we're not.
I appreciate your argument for human agency, but these types of systematic issues can't be solved bottom up.
> You go to a university because you are deeply interested in understanding the subject that you study.
You must come from a wealthy background because what you described is far beyond the vast majority of people's means - at least here in the US.
Most of us go to college because it's the only reliable way to get a tollerable job that pays well. Only a few of my college courses aligned with my interests. The rest were just the price paid for the degree.
LLMS didn’t invent cheating just made it easier. When you cheat you’re the one who cheats yourself because the point of an education is to learn, not complete the assignments and get high marks on tests alone. No one benefits and no one other than you is materially hurt by cheating, but you are absolutely the one who is hurt.
There’s no way to learn than to force the brain into adaptation which it is resistant to do through challenge and stress, just like your muscles. Similarly you can’t play e sports and get into physical condition any more than you can use LLMs to do your homework and learn.
It’s going to be a hard adjustment for a lot of people to recognize that letting the machine think for you is as healthy as smoking brain cigarettes.
The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button. They make great tools for learning if they’re used as an adversarial or editorial tool. The future belongs to those who work to use the tools in ways that make themselves more efficacious, not those who use efficacious tools so they don’t have to work.
>The smart student uses the LLM as a proctor or provide challenges and feedback on attempts rather than an easy button.
Yeah, this is how we used wolframalpha for Math as students. Whatever we had to do, we did it ourself as a group of three. Afterwards we checked with Wolframaplha to see if we were correct. If there were any difference between us, we went line by line to find where the error appeared.
It was helpful, because we did it ourself, but because the work was graded, we had the security, that it is not a total failure.
To say that students don’t benefit from getting good grades using LLMs is incredibly naive. Learning is only about the third or fourth most important “benefit” for students, after getting a degree, getting good grades, and making connections.
These matter about getting the first job you get, at which point what you learned begins to dominate the rest of your career and life. The connections you made at school matter less and less as your connections in your career dominate, and they are built on what you can do, which is based on what you’ve learned.
Does anyone look at GPA on a resume? I’ve hired thousands of people I’ve never once looked at GPA. (N.b., my resume has “summa cum laude” ok it and no one has ever once mentioned it or presumably noticed it, despite the fact you only really get it if you can BOTH learn the material AND get perfect grades)
The problem with AI in an educational setting is when one is graded versus their students on things and things genuinely depend on those grades. Group projects also force those willing to do things without AI to go along with others in their group who'll use it regardless.
My son just finished his first year in college, and had no trouble getting decent grades without using AI while many of the kids around him were using it. At least in his humanities track, class participation is a lot of his grade, and he said the "AI kids" tended to suck at participation because they hadn't actually thought about the material, and couldn't dynamically work with it in class. He also said their AI assisted writing that he'd read was dull and unoriginal, and all sounded the same, which he thought likely helped his essays stand out. His English composition teacher said he was "probably too advanced for this class" when he told her he didn't use AI to write his essays, which made him roll his eyes, as he has clinically diagnosed dysgraphia (learning disability in writing).
Makes sense the ones who can't tell that AI does a piss poor job at writing get bad grades. In a humanities track I can certainly see how going basically completely no AI should be an advantage. Even in a other tracks it should be better, especially if professors think out assignments well. Group assignments are my biggest worry as in some classes they can really make/break a grade, working with those believing in AI would certainly be a disadvantage.
You're right.
But I like to add artwork to my presentations. My artistic skills have not advanced beyond 2nd grade. So I'll make a line sketch, and give to AI to "fix" it.
The results are nice and I use them.
I have no interest in learning how to do art well myself, so using AI for it is appropriate.
But I still write my code myself.
>> The results are nice and I use them.
I haven't seen your presentations, so I can't speak to them. But I do know at work there's a lot more illustrations in docs and presentations and such, and they almost all have an AI art "tell". I find them grating and distracting from the actual content. Very rarely do they add anything useful to the doc other than the knowledge that the owner burned some GPU time and tokens for a distracting, low value illustration.
I can only imagine how an actual artist or graphic designer feels about it.
Actually I don't have to imagine; there's some serious vitriol over on some of my favorite webcomics about it.
That's because of people who treat AI like a magical "give me a finished product" button. Someone whipping up PowerPoint slides tends to care more about saving time than maintaining quality - the (usually watermarked) stock photos of yore weren't exactly the zenith of artistic presentation either.
Like any other tool, the quality of output is proportional to the quality of input. You'll get much better results if you provide specifics (detailed description of what you want, reference pictures, iterative clarifications etc.) and have it work on bite-sized pieces of the process (outlining, shading etc.), with manual verification and touch-ups in between. But all that still requires time, taste and experience, which deflates the "AI will replace skilled labor" hype.
>> the (usually watermarked) stock photos
I didn't care for those either but they didn't have the signature AI art "look" and they're usually real things that an actual artist or photographer made for or sold to the stock imagery company, which curated their collection.
> But I still write my code myself.
Not for long, if you so easily have caved in to using AI elsewhere. People are lazy. If you see that the 'results are nice', it's game over for your programming/thinking.
Waiting for the day the advice will be to "enjoy AI assistance in moderation"
What does this have to do with LLMs?
They stopped requiring SAT and ACTs in order to get a student population more representative of the population in general. This obviously allowed students that were not prepared for college into the system.
If you do well in your math SATs you'll likely do well in math college. SAT scores and college GPA are highly correlated. No idea why anyone thought it was good to ignore probably the strongest signal of success in college.
Isn't GP talking about PhDs here? If they're PhDs I imagine they've already had good enough GPAs/were smart enough to be selected for the program.
I broadly agree with the premise. As a PhD student in Computer Science, I feel there are some significant upsides to my work routine. LLM access has made many new domains more "accessible" to me which I otherwise would be too hesitant in investing my time in. For example, my area of research is computer systems which involves operating systems, distributed systems and more recently systems for AI. Within these, there is a wide breadth of topics/techniques one can employ and up until now, I have not gone deep into theoretical aspects of things like scheduling etc. But with access to LLMs, I feel like I can at least brainstorm from a high-level about these sub-areas that I am not well-versed in and the responses give me some relevant pieces to start exploring on my own, depending on what interests me more or the amount of time I want to spend on that sub-branch of a larger tree of ideas. However, the one thing I do have skepticism is the lack of awareness of blind-spots when dabbling into areas that I am not an expert in, and taking the LLM's lead in applying such techniques to some systems problems that I am working on. I often feel that I am not aware of what alternatives exist that the LLM has not explored for me, or if the directions it has proposed really do apply or have corner cases/assumptions that break in what I am doing. On the other hand, when working on something I have good intuitions about, I am often correcting the model's assumptions and it back-tracks what it told me. Unfortunately, I cannot do that comfortably with topics I don't have good intuition about which limits my confidence in "if this is the right direction to pursue."
As someone with a PhD in CS focused on NLP (I started my PhD in 2018 just as Transformers were introduced), and with a strong background in distributed systems owing to the fact that I was a lead developer of an MMO before starting my PhD, I can definitively say that any surface-level understanding you get by interacting with an LLM, is just that: surface level.
If that allows you to target your deep dives better, then great. If instead your deep dive into a topic is purely through prompting an LLM, that will almost certainly end with little functional domain expertise.
The absolute best experience you can get is by trying, failing, then improving upon your past failures. Remove that friction at your peril.
At this rate humans will become avatars remotely controlled by LLMs. Ironic conclusion of the consciousness debate.
We are already remote sensors and manipulators for the corporate and economic structures we operate under. You can't see it, but we are ants in a superorganism.
More evidence of the philosophical concept of 'technology is a life form.' Humans would be the perfect host, at least for the time being. They are certainly a willing host.
Yes, ask AI to produce a plan for me to do x like design an travel itinerary. It creates the plan and I execute it.
obligatory https://smbc-wiki.com/index.php/2014-12-17
Distopian, what an insightful frame
When LLMs and ChatGPT first came out, it struck me as obvious and dangerous to a deep thinker or a knowledge worker the answering capacity. So, from my initial use I did not ask them questions, I have always "done my own work" and then asked the LLMs to criticize that work. This has been an exponential ladder of learning, and my cognitive growth is personally noticeable. I'm not hesitating to scribble out calculus and work it out, as I need for my work, where in the past I'd have found some other way because I felt uncomfortable with my tip-of-my-tongue calc skills. Don't ask AI, do your own work and ask for criticism, and them improve your own work yourself. This creates a learning ladder that you will climb.
That's nice except when you work somewhere where more and more developers are pushed to pump out slop generated by AI as fast as possible. So far I am not there yet but I have plenty of friends in the industry who are basically 'not allowed' to code manually anymore.
I've been wondering if there would be a benefit to inverting how we teach subjects now. Previously we would teach from the bottom, and build up. Semi-colon goes here, curly brace goes there, and then build up to architecture, systems, etc.
But this doesn't seem to make sense when someone comes to a topic with an LLM in-hand. They need to know high-level techniques, architecture, best practice, etc. As they pursue the topic they start to get down into the details, although probably never learn to do it fully independently.
I quite like this view because it paints a somewhat optimistic way forward from where we are now.
You can ask LLMs about high-level techniques, and their answers will usually be good enough. What you can't get from LLMs is the taste and judgment, which you can only obtain by having a strong CS base and coding manually for years.
High-level techniques were never a problem. You could Google tens of articles on this topic. They are useless too, it's like learning how to drive a racing bicycle from reading a book. Sure, you will know a lot about nuances, but you will fail miserably when it comes to a real race.
The other day I just wanted to loop through characters in a std::string to copy data to a new string with a few escape characters (sending to peripheral device). Simple enough task for AI. I got a coroutine monstrocity back, with copies to std::array and a range based iterator, since I specified C++23. If I specified C++11, I would have received a: char p = src.data(); while (p) { … p++; }
I had the experience to keep calling out AI to simplify and downgrade the solution to something primitive, which ended up smaller, faster, easier to maintain. Juniors with real world experience would not bother, they’ll take the first working AI result.
taste and judgment, which you can only obtain by having a strong CS base and coding manually for years.
I disagree, the definers of taste; art and food critics, movie and book reviewers, don’t need to have learned the craft by doing. Taste is a separate skill.
No one seriously expects a food critic to be able to cook a Michelin-starred meal. The job of that kind of critic is to be insightful and entertaining, and it's very different to the taste required to create top quality food, which is a combination of solid technical skill and creative flair.
Taste in coding is a combination of insight, experience, native talent, technical skill, and flair. Tasteful coding produces clever but straightforward minimal elegant solutions that an average developer can't imagine but can adapt and maintain.
If critics were forced to be actually skilled at the craft of creation, the world would be infinitely better off. Both the cello and the player are better off by the cello maker also finally being the cello player. Alienation was a mistake and this part Marx of all people understood well.
This is why "critical thinking" is a meme. Being a critic takes no skill. I want far fewer critics and far more constructive thinking. GenAI being the ultimate constructor is a bonus.
I'd say taste is a consequence of lifestyle, which is learned by doing. And art critics often have bad lifestyle, which is visible in their bad taste. When art is virtual life, it would define a lifestyle, which is adopted by doing, in its turn producing taste.
Agreed.
Taste implicitly requires discipline of what one chooses to expose themself to and what not to.
> which you can only obtain by having a strong CS base and coding manually for years.
I hope this isn’t the case. It is the route I took, but it also doesn’t seem to be a likely route going forward. Strong CS grounding is feasible for sure, but I have a hard time believing that a meaningful number of people will be spending the requisite years coding manually.
Exactly. Repeating or rephrasing a definition is trivial, teaching someone is not.
I can't speak for other disciplines, but for math and CS, both with a really heavy focus on abstraction, the final result of learning is to build a nice intuition on top of the abstractions we find useful/expressive. And to build the intuition, the old, usual, and perhaps the only way is to see and practice a lot of concrete examples, after which the motivation of building some abstraction can be understood, and after which the abstraction itself can be fully grasped.
e.g. The "group" abstraction requires one see a lot of int, polynomial, modular arithmetic etc. before knowing why we want such a thing. It's unskippable.
This idea sounds good at first, but if you look closer, it would just make workers, not experts who really understand. What we could do, and already do, is tweak the learned abstractions. In our field, it's easy to see: most of us first learned about computing abstractions, not how processors actually work, or started with Java, not assembler. Plus, you can't teach math from top to bottom.
> As they pursue the topic they start to get down into the details, although probably never learn to do it fully independently.
It's hard to claim one has mastered a subject without independent command of its fundamentals. A less charitable take on this future is that students only learn to hand-wave answers and correspondingly cannot evaluate statements beyond "sounds about right".
> Semi-colon goes here, curly brace goes there, and then build up to architecture, systems, etc.
If that's happening, that would be a weird way to teach CS in my opinion.
In my undergrad program, languages and syntax were learned on your own. Class material and lectures were all conceptual.
I keep trying to convince people that English majors and Philosophy majors will benefit the most from LLMs. English majors in particular, have been trained to be VERY exact in how they word things.
That awareness of how to structure the English language, it will benefit those who use LLMs.
Then again, maybe someone will just make a LLM that’s built to turn poor English and poor reasoning into excellent English and excellent reasoning. Maybe this is just a technical puzzle that needs solving.
I disagree with you, for the very reason you give:
> Then again, maybe someone will just make a LLM that’s built to turn poor English... into excellent English
That's already been done, for some (pretty weird) definition of "excellent".
I work with, or at least in the vicinity of, someone who is very good at getting work out of LLMs. He has a whole system of CLAUDE.md files and skill files and things. He makes TONS of typos. When I first saw that, I was itching to go in and fix them all, it seemed viscerally wrong to be adding an extra layer of correction required between the instructions and the LLM's behavior. But in practice, I don't think it mattered at all. The LLM didn't care. Typos in particular might require a bunch of RLHF in the chatbot, but my hypothesis is that the LLM is already mapping messy human input to the nearest surface of some high-dimensional manifold and the added noise of typos is inconsequential to where it ends up (as long as there isn't any real ambiguity -- though even there, you could probably construct cases where that would help rather than hurt!)
Typos are different from sloppy writing, but I think the AI companies have put a lot of work into training these chatbots on dealing with typical non-English major writing with all of its imprecision. Also, it's easier to construct cases where that imprecision and sloppiness would help rather than hurt: a mistake in the input that is common enough to show up in the training data is going to be a good match for the needed correction as well as associated corrections. The precise language could easily result in the LLM overestimating the user's competence.
That doesn't address whether an English major's careful composition would help for hard tasks where getting the specification right really matters -- perhaps that was your point? I guess it's an open question whether "boiling away the typos" and "boiling away a poorly articulated specification" are related enough.
I don't think you can learn high level techniques or architectures without first understanding the basics first. This means boring boiler plate coding.
I’m not sure. We’ve always had to pick the level of abstraction we start teaching at. Voltages, transistors, registers, assembly, C, etc. This feels like it could just be a progression of that.
Then its reasonabel to expect someone who is not using LLMs to have an edge in their cognitive abilies. Or will it be overshadowed by the shear magnitude of bruteforcing that LLMs are capable of. I tend to side with the former. If that would be true, then not using LLMs would give an edge in solving novel problems. But we have been dependent on tools, cognjtive and physical, since forever. We cant imagine a world without tools. Why would LLMs be discriminated as a tool
I have observed this in myself when I began to over-leverage AI in my workflows. I've since become more deliberate with what kinds of tasks I will use it for, although I still slip up.
With writing:
Things like brainstorming a plot line for a book with a custom GPT or Claude project that has all of my prior books in its knowledge? Works great.
Things like asking it to write a paragraph or chapter for me - I can rapidly feel my own writing skill, motivation, vocabulary, and ability to grasp/remember the resulting plotlines deteriorating. I don't use it for that anymore.
With studying:
I've been taking a couple of evening uni courses and the thing I found so great is that I've been forcing myself to think through the problems, and take my own notes in every lecture. I may then still get ChatGPT to help explain and reason through some of the concepts with me. And I have it review and 'grade' my assignments. But I refuse to ask it to start drafting answers.
With programming:
This one is tougher. When I am not very personally invested in a problem or codebase it becomes too easy to offload more parts to Claude, and when the company encourages 'vibing' to speed up velocity and you're reviewing and writing a higher influx of lower quality PRs, investment goes down. I still sometimes catch myself committing solutions I only _mostly_ grasp and the rest is hand-waving. A big part of it is a work culture thing.
For my own projects I make sure to understand and have a back-and-forth with the planning agent for each task, or write the first plan myself to go off of. When it comes to producing the code, I have to admit it is much easier to properly review parts of the codebase I am extra interested and knowledgeable in (backend in my case). The frontend I'm less well versed in and also admittedly less interested in, so I do sometimes fall into the trap of "Ehh it works, just commit it" with the goal of doing a thorough quality pass before actual release.
With all of the above, I can feel my ability to think, plan, reason, focus (and my vocabulary) suffer if I go over the line too much into agent offloading. For me keeping that balance is as much about maintaining my own long-term brain health as it is about producing good output. I imagine younger people growing up with AI today won't even know what that more capable (in my opinion) brain state feels like - to them, the AI-using brain will be the norm.
The place I've come to with AI for writing is to have an idea for a chapter/article/etc, which I take to AI, and tell it to either ask me a bunch of clarifying questions, or try to blow holes in it/challenge it. I'll keep talking to AI and answering questions/handling challenges until the AI runs out of steam, then I'll ask the AI to write out a condensed outline with all the pertinent details of the conversation.
Once I have the condensed outline, I'll re-order stuff, clean it up/tune it up, then do the final writing. This keeps my voice and logical train of thought while avoiding blank page syndrome and some of the organizational mess of condensing notes into an outline manually.
> many of them can no longer brainstorm, code, think deeply, or write
I believe this is the real crux of the issue. We often turn the target to things like "Can johnny Add, Read a book, or recite dates" which are only proxy measures for important things like "Can johnny solve a numerical problem presented to him, can he synthesize information, or can he think critically about what is occurring around him?" .
If students use AI to accomplish goals I do not see it an issue. If they cannot figure out how to use tools, or what their goals are-- that is a major issue!
An analogy of my point is that I don't want to focus on cursive in the age of computers keyboards, and I dont want to focus on abacus skills when a pocket calculator is like $5.
If students are allowed to use AI to accomplish their goals, then I think the real question is why should they go to an expensive university for four years to learn how to ask AI to do something?
very fair question. But that's on the university not the students, as in the faculty shouldn't be complaining about the students, but adapting with the times.
We’re in a world where LLMs are basically going to be extensions of how we think. An additional thing we use to do a lot of thinking tasks.
As a piano player, it’s important to work hands separately. Sometimes your right hand will carry the melody and your left hand the harmony, sometimes vice versa. Sometimes there may be more than just two “voices”/melodies/lines between your two hands. Even as a very good (as in getting paid to do it) sight reader, I learn a lot working all the voices/melodic lines separately.
Singers do similar things like singing only the vowels to keep themselves in the right placement. Learning handstands, you have to work your wrists, rotator cuffs, core (which is many things), etc. separately. Yoga, Pilates, and running also help us learn to break problems down this way.
Anyway, all that to say: If LLMs are gonna be a natural extension of how we think, we need to understand what parts of problem-solving LLMs are good for, and what parts our brains are for. The nice thing about working these bits “separately” is that one side is done for us. So we just need to consciously practice using our brains.
As programmers that means, maybe we conscientiously practice writing things ourselves sometimes. Remembering that this even if this sacrifices short-term “velocity” (whose measurement is problematic, but I digress), it preserves our long-term ability to do good work. And I think any of the above physical/artistic practices (or countless others), worked in these ways, will help reinforce this entire mindset.
I think kids of the coming generation will be sharply divided on their ability to conscientiously practice things separately. It’s been happening, but I suspect LLMs will accelerate it unless how we actually teach kids can catch up.
> We’re in a world where LLMs are basically going to be extensions of how we think
If that's the case then we're in trouble based on my experience. This week I've been using ChatGPT to help figure out some old linux platform that I need to resurrect. It's very good at quickly searching and surfacing relevant information online, and that's helpful, but if I did not have a lot of experience at linux administration to be able to see where it was suggesting the wrong thing, or initially dismissing the right thing, then I'd just be thrashing.
The LLM is helping me because I know what I need, and it can search and read faster than I can. But it's not really very smart.
> An additional thing we use to do a lot of thinking tasks.
Which is to say, an additional thing you're going to be forced to pay a lifelong tithe to a trillion-dollar company in order to do a lot of thinking tasks.
I’m rather optimistic about the future of smaller open-source models and market competition actually doing its job here, honestly. I myself, again, err on the side of doing things with my own brain. But there are many things LLMs are useful for, and they’re definitely better than a “rubber duck” if you don’t trust them blindly.
> Many of them can no longer sit quietly for even 30 minutes just thinking on their own
Plummeting attention spans has been a trend for much, much longer than LLMs and is more the result of constant digital interruptions and these days overwhelmingly social media and doomscrolling: https://www.apa.org/news/podcasts/speaking-of-psychology/att...
The effects on children have gotten most of the, err, attention, but the effects on adults are no less deleterious.
In 2002 I spoke with a lecturer in the humanities and he told me about how nobody was learning French at university level (in the UK). My own course had been cancelled due to the cost of teaching it, and the era of 'easy degrees' had set in during the early 90s.
Before that, I also noticed the decline in newspaper readership in the 80s.
It is easy to blame this general decline on the latest tech (or moral panic), whether that be LLMs or even the existence of the internet, however, the trend in dumbing down has been going on for decades.
In the context of a declining empire and financialised economies, this makes a lot of sense.
I think we are talking about two different trends that have similar symptoms. I do agree there has been a noticeable anti-intellectual trend for a long time, especially in the West. (See also: Grade Inflation.) But that is separate from the drop in attention spans, which is relatively recent has been pretty strongly linked to digital stimulation, constant multi-tasking, and now short-form social media.
LLMs are an entirely new dynamic with significant cognitive implications, but I fear it will be hard to discern their impact from the falling attention spans and other long-term trends that have led to things like grade inflation.
I dunno, I used wolfram alpha a lot during calculus classes. However my uni didn't require any homework assignments to be done and they did not contribute to grades. Only the exam mattered.
Maybe the problem is that doing assignments contributes to your grades? The answer from wolfram alpha wasn't so much to get the homework done, but to understand how I would be screwed in the exam.
As much as I hate to admit it, using agents for too long makes me less able to think for myself. I am dedicating 30 mins to 1 hour everyday writing and thinking without LLMs.
> I am dedicating 30 mins to 1 hour everyday writing and thinking without LLMs.
Before you did this, was literally every hour of your waking time spend thinking about LLMs?
I don't think I could do that even if I tried, and I spend all my development hours with agents, but during meals, showers, walking the dogs, enjoying a coffee outside or whatever, naturally I get time to think about other stuff, sounds out of the ordinary (to me at least) to have to dedicate 1 hour to not think about something. Reminds me of when I was addicted to amphetamines way back when.
> Before you did this, was literally every hour of your waking time spend thinking about LLMs?
They said "writing and thinking without LLMs", not "not thinking about LLMs". I think they're talking about setting aside time for fairly focused thought/work.
Shouldn't it be the opposite? 1h max LLM per day?
Perhaps, but I'm not sure my boss would appreciate that though.
I don’t buy it. Properly leveraging LLMs to generate stable and extendable systems is mentally exhausting (i.e. highly demanding of intelligent thought), especially given the poor quality and churn within the harness ecosystem.
Now, if you’re creating trivial, unstable, or nonextendable systems maybe this doesn’t apply. And maybe I have long overestimated the work that SWEs have done.
i use claude a lot and i find that it is best applied in domains in which i am already a master. I tried applying it to domain's im unfamiliar with and i found that i produced stuff but as time went on i understood what i produced less and i almost felt like i do after binge watching a netflix show, 2 weeks later i barely remember any of the details. I wonder how much you need to "do" to learn and remember. LLM's give you a shortcut to doing and so you probably aren't learning either. It's like when you watch a professor write a proof and it makes sense while listening to the professor but at home you have difficulty deriving it. LLM's give me the same sort of feeling. I think the way forward is still going to be doing things manually to learn and using LLM's once you've mastered an area and people who don't understand this fact are going to slowly descend down a hill and forget how to depend on their own thinking.
I wonder if AI or something else changing (developing anxiety, etc) has made the pay-off of the degree less certain. If you're confident that your years of effort will pay off, it's probably easier to see it through. If you're worried that AI will wreck your industry before you hit the workforce, maybe the equation changes and you're more inclined to gamble with shortcuts?
Yes, and this is going to hurt everyone. If everyone knows that you can skate through a college degree without doing any work, it is not going to have much value as a credential.
I totally agree about school-level homework: it was many years before my pre-frontal cortex developed enough that I could have forced myself to do the work.
That said, though, one thing I don't understand about the heavy users of AI in academia and software development is that the thinking and coding is the fun part. And that's the part so many people seem to be so keen to automate away.
I'm right there with you. The thinking and the coding is the fun part. I'm pretty relieved that all of this is happening near the end of my career. To me, AI is just not fun. And constantly signaling how productive I am and having to show "my value" is exhausting. This is only my subjective experience, of course, but in many ways the world seems like the fun is getting sucked out everywhere, not just from AI. Like the type of people that become managers are taking over everything.
Isn't the fun part having the thing work how you wanted it to? Why shouldn't I be keen to automate the process away?
Depends on the person. I find that it's extremely satisfying to figure out a tricky problem on the way to that end result - to struggle with something for a bit, then finally fix it or fully wrap my head around it. So to me, it's a mixture of both. What I want is the end result, but in the past sometimes that came with thinking in the shower about an approach... Or a wild thought while going to bed that makes me jump up and grab my computer.
That doesn't happen for me anymore to the same degree.
I've read enough comments* on HN to know that there are different camps. Some people don't really enjoy the process of development and just want results. Meanwhile, telling me to automate away the problem solving aspect of software dev is like saying "you know you can just copy the answers to the crossword from the back of the book?"
*speaking of things I should be doing less of...
different strokes for different folks. I'm def. in that end result camp, i get the biggest thrill out of seeing something work. For me, coding agents are awesome because i can bring a lot more to life in much shorter of a time frame. I do enjoy the process and problem solving of coding, it relaxes me. On the other hand, i really really enjoy when an idea i have is on the screen and working.
No. The fun part is the process of getting there. The end result doesn't excite me at all.
I'd argue that this is an adjustment period that society has to go through. The way we are using electronic devices today, in some years it will probably be looked at like smoking cigarettes. And I'd argue that a lot of the "decline" is due to a shift of skills away from things that mattered more in the past toward other things that are not measured/perceived by the older generation.
Interesting analogy. I believe regarding addictiveness they may be compared.
> a shift of skills away from things that mattered more in the past toward other things that are not measured/perceived by the older generation.
Do you have any ideas what these things might be? As someone in his twenties, I’m sometimes saddened by observing that some of the skills I acquired over a long time (e.g., writing, coding) may become obsolete or won’t be respected anymore just now that I‘m finally getting good at them.
Thinking is the skill that becomes obsolete.
Nah.
What you said there is just an extension of the elimination of friction that the silicon valley has been pursuing for the last 15+ years.
But that is just.. well. Their business model. Not a force of nature.
it happens, things change and the change is only speeding up. I think the real skill to have going forward is the ability to acquire new skills. I tell my boys "get good at learning and you don't have to get good at anything else".
Ages ago I had similar thoughts. Everything changed when I came to terms with the concept of change being the only constant. A bit of a cliché, perhaps, but profoundly true.
Eh, I think it's less like a cigarette and more like the car. We're not going back. Americans are famously less healthy the more car dependent they are, and now people walk/run as an explicit task to be healthy. People will start going to a "thinking" gym, or engaging in additional manual mental activities for sport, like we do with chess today.
Or when the Internet came and made memory kinda obsolete. Why remember facts if you can simply index them and then lookup on demand.
But now we delegate thinking itself, so I wonder what is left.
This is an age-old argument actually, the same one was raised when the printing press was invented and reading became a more generally available skill.
Merely leading to the upending of the political structure of most countries.
The Internet in no way made memory obsolete. People who know things off the top of their heads are far more capable than people who have to look things up.
I wonder who would be working in these thinking gyms? Nice idea. Extra mural studies for the age of agentic ai.
Funny that you mention that. A month ago I started the Duolingo chess course, and just yesterday I noticed that my brain is clearer, more capable of deep thought than it has been in years. It's like stepping out of a fog. I also started CPAP recently, so it's hard to attribute the change to either, but I feel certain that the chess helped.
> I also started CPAP recently
A proper nights sleep is massive! I'd put 99% down to this..
Yeah CPAP is doing the heavy lifting most likely.
The interesting thing about jogging is I do my best thinking while jogging. I've found it impossible to do deep thinking while driving, as driving evidently requires higher functions of the brain. Jogging doesn't require any of that, I can jog deep in thought and have no recollection of the previous mile.
Apparently Petri was driving when he figured out a new fingering for a Chopin Etude he'd use at a concert later that evening[1]. The passenger survived to tell that story, so I think this is more about your correct risk-assessment than physical limitations of the brain. :)
1: Just to unzip: "I don't like this fingering; let me imagine playing with this other fingering; yep, that will feel more comfortable and stronger; I'm now confident enough in the change that I'll do it that way in public, for one of the most difficult piano pieces, without ever having practiced it physically..."
You do realise most people aren’t in shape right?
The idea that most people have the discipline to keep themselves mentally in check is false. We already know this! Millions and billions of people who spend hrs a day consuming media on platforms such as instagram.
> I'd argue that this is an adjustment period that society has to go through.
I used to think like this until social media proved there are some tech innovations we just can’t adjust to. 10 years ago you would’ve never caught me supporting any sort of age based social media ban. Now? I don’t think it goes far enough. Fake news (actual fake news) and misinformation has only gotten worse with it as well. It’s so destructive.
The human is designed to interact with small groups, to understand several smaller groups, and perhaps to imagine a big group of smaller groups. In a literal sense, let's say 100 people per group. At that level the human can actually know and interact with them still. In a city of 100.000 it's still managable to feel you are related and involved to this group-of-groups. In a city of a million, you'll revert to only your own small group and have lost the connection to the collective.
The same goes for speed and quantity of input, as to what the human is designed for (not literally designed). Be it social media with it's infinite scrolling, cars racing by as opposed to looking out the window a few times per hour because you see someone/something, constant sound input if you live anywhere remotely busy or work in a busy office.
The point I'm trying to make is that the world used to be comprehensible for the human. Some understood a little complexer things, some only the simpler things. Now there is an overload of everything. So, most humans are in survival mode wether they know it or not. Hence the many seekin mindfullness etc
No matter, it's an observation, not a judgement or opinion on it. The world will just keep rushing forward. Some have a slight hand in the direction it goes for better (never) or for worse, but spiral it will.
I think there’s a major part of this conversation being omitted, though I am not saying you did it intentionally: “the attention economy.” We have gone from advertising to a system of creating addicts for profit
Definately agree, that was included in the "or for worse" in this sentence I wrote. As for creating the addicts, nobody had a masterplan. It's all the pieces spiraling together,
>> The world will just keep rushing forward. Some have a slight hand in the direction it goes for better (never) or for worse, but spiral it will.
The systems are too large and self-propulsing for anyone to really control. Consider the rainforest. How many millions of variables interact, nobody is in charge, everything influences everything in a billion different ways. You might say, well we can cut it down, so kind we can control it. Allright, let's continue to spiral. You might build a city there after a few years. Still in charge right. But it get's too hot because there's no vegitation, so you have to change again. And then we find that people keep getting strangely sick, and scientists find some special mushroom that survived and apparantly thrives on the mix of cut trees and diesel fumes and their spores in the air are poisonous. I made that up, but you get the idea hopefully.
LLMs have killed my facility but not my knowledge.
I can still read code and write it, I just need to look back at docs a lot more, when I used to just know things. I also have to sit and try to recall how to do things and what abstractions are involved more. I also have more "writer's block" when starting with a fresh program/document if trying not to get AI to seed it with a baseline implementation, where I have to sit for a while thinking about what I really want to build.
This is also why AI isn't going away. People will (and already are) leaning hard on it, and will pay in the future for that crutch.
Do any of said PhDs gain anything positive from LLM usage? Or does it only lead to declining thinking skills in your view?
Yes, I can churn out a lot more stuff as can most of my peers. Experiments etc are all way faster to run with coding agents. But I think the overall creativity and originality is a lot lower. I think this is what many people are facing, if you don't use LLMs your short term productivity is worse.
There’s the saying that we overestimate what we can achieve in the short term, but underestimate what we can achieve in the long term. Optimizing for the short term is therefore counterproductive if it impairs us for the long term.
“ But I think the overall creativity and originality is a lot lower.
Therein lies the trade off. Your implicit gamble is that you expect machines to continue to get better in the future. What if they don’t?
They're incredibly more productive. LLMs are amplifiers, so where they'd have branched and tried out N things, they can easily try 5N pathways of RnD. LLMs are extending the frontiers of science fast -- math -> phy -> chem -> bio in that order.
In my own experience, the only path I truly gain intellectual benefits is the one where I work closely with the LLM, test very narrow hypotheses, and leverage it for learning over producing.
Trying 5N paths is useful and sometimes yields interesting insights I’ll retain, but it’s not the rich, challenging, deeply engaging kind of process I find I need in order to develop useful knowledge and skills.
So yes it’s an accelerant for people who want stuff from me, but that doesn’t map directly to learning and building skills. I think that mismatching is really important.
To help learn I use LLMs to generate practice exams for whatever I'm trying to learn, then on the questions I struggle with have the LLMs explain the logic and point out my mistakes. I haven't been in college for over a decade, this is just for topics I'm curious about and want to learn. For any serious topic I recommend auditing the practice exams with a different LLM than the one used to generate to help reduce hallucinations. Seems to work well for me. I quite like reading the "thought" processes shown by DeepSeek.
I don’t see these at odds. Sometimes through working closely with an LLM, N paths emerge. Having it go off and test each with defined metrics to determine which is better is the natural follow up. Even better if you dive into the why it ended up being better which the LLM seems to be able to expose well in a lot of cases.
The part I find weird is all the claims that LLM usage leads to less thinking and exploring and just grabbing the first result. I constantly find myself going off on tangents and pulling on threads when I’m working with these tools. Is it really that different than before when my “peers” weren’t able or willing to be curious about their craft? They didn’t explore other programming languages out of curiosity or for fun? That covers literally 95% of all software developers I’ve worked with in the last 24 years across many domains. To them it’s just a job. Their only goal is to deliver tickets assigned to them and go home. They rarely go out of their way to learn something new unless the company assigns them some mandatory courses. Largely the LLM is capable of producing better and more consistent results than they ever could in the first place.
I don’t know how to cultivate curiosity in the work force. Maybe it’s not possible and you have to filter aggressively at the hiring step. But then your pool of hireable candidates shrinks to a few thousand developers most who are probably not actively looking for work.
You’re right, they don’t have to be at odds. Before LLMs, there were jobs where I had to power through and make a sort of ‘minimum effort’ approximation without applying much analytical or investigative energy or skill. This isn’t a lot different from churning something out with an LLM. There’s not much to learn, the end product is mediocre, it’s more of a rote path.
The only distinction I wanted to make is that the learning doesn’t come by default. Yet that was largely true when people copied mystery solutions from stack overflow and used black box libraries for 90% of the complex work their programs facilitated.
Perhaps not much has changed but we’re now operating at a much larger scale and the opportunity to not be curious is actually more present than ever.
People who are curious are massively benefited by this tooling, in my opinion. Like you’re saying, if you want to investigate and learn, there has never really been a better time. If you’re sincerely applying yourself and pulling all of those threads, there has never been a better teacher.
I’ve wondered about the matter of finding and cultivating curiosity too. I’ve come to believe most humans, let alone programmers specifically, are not all that curious. A lot of us are path-followers and we’d rather not get into the weeds most of the time. Then some of us see weeds and dive in, even when it’s not pragmatic to do so. I don’t know how much it can be cultivated or even removed from a person who has more than enough.
I'm hearing different from PhDs. The bottleneck with much research isn't "trying out ideas" so much as it's all the bureaucratic minutiae, grants, mentoring PhD candidates, collaboration with other researchers, etc.
I've heard LLMs can be helpful in limited targeted ways. But not as some kind of "game changing" accelerant.
Understanding in what ways it can be useful and in what ways it can be counterproductive in long run requires a certain degree of experience itself.
It’s creating a daemon and machine spirit filled world of Warhammer 40k. We already scarcely understand how the world works, but LLM use actively degrades cognitive ability that way it is used by a majority of people (The bringing a forklift to gym analogy).
The AI is among us.
To me it is crazy that you are being downvoted. My experience in academia was that an incredible amount of time was devoted to data cleansing analysis, coding, etc., which were completely non-core to the actual underlying academic pursuit.
Data cleansing is a terrible use for LLMs if you want reliable data.
There's an unnecessary feeling of fear that permeates any factual conversation on LLM's impact on science and engineering. You can just view the practitioner over the shoulder and see all the things they're able to do in a minute that would have taken days.
The downvotes are just a sign of the times. It's also something to observe and think about..
It depends on the field, but an Economist with a PhD is a huge red flag and anything they say should be ignored.
Other fields may be different. YMMV
Yeah. I have no doubt that I would have used LLMs “just this one time” to help with problem sets or papers when I got behind or wanted to do something else.
I've avoided using AI* for precisely this reason. I don't want my brain to get lazy and rot.
(* I can count on one hand the number of time I've used an AI tool.)
This is all assuming tests measured anything valuable in the first place. In my experience standardized tests were always flawed and most of my peers knew shit about the subjects they passed in top % a year after. If AI breaks the current education system that's a win in my book.
I don't understand why people take shortcuts in school. You pay a LOT of money to be there to learn. Taking shortcuts seems completely counterintuitive to me.
- Time is a scarce resource. Students do what they can to learn what they can, but if they're under the gun, they'll take the path of least resistance to make it to the next day (totally not like the business world, right?)
- In the interest of having well-rounded students, a lot of degree programs include subjects the student didn't want to sign up for, but have to. Even in something like CS, I knew a lot of people who liked the hardware side of it, but didn't like the software side and vice versa. So I can imagine a student justifying taking shortcuts that way.
- Psychological reasons like wanting to protect their ego. Maybe they had always done well in school and are now struggling, but don't want to ask for help, so they think why not just take a shortcut here and promise to do better next time, etc., etc.
A lot of people view it, rightly or wrongly, as paying a lot of money to earn a degree that opens up certain opportunities, while learning is secondary, so minimizing effort is worth it.
And to some people, it's not even a lot of money.
In many ways, schools are just the modern day peerage system.
> I wanted to be in denial about it before but it's too obvious to ignore now
Use-it-or-lose-it is the evolutionary principle, both for cognitive and physical abilities.
> Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well
I noticed this before LLMs became a thing. It was by accident. We had a team of programmers. All decent at what they do. The management said 'hey you want to learn another language we are going to be using it for these upcoming projects'. So we set up a self learned at your own pace class curriculum. Maybe 10-20 hours of school work if you sat and really dug in. Maybe 3 to 4 hours if you breeze thru it and do not care much. We set up weekly check-ins doing about 1 hour a week. Easy. Watch a 20-30 min of vid 20-30 mins of do homework come to check-in and talk about what you learned and help others if needed.
Now this is where I was disappointed. The first 'class' was 40 people. By the last there were 3. Those 3 I noticed always are the ones who dug in. The rest wanted a proctored classroom and someone to tell them what to do.
Actual genuine curiosity is rare I think. We have a lot of people who are decent at what they do. But do not really care about it. IF you do not care you are going to just push the button and get the answer.
I could see myself dropping out even if I was interested in learning. I'd suspect that the time spent would end up with me needing to stay late to make up for it or being penalized in some other way.
I think it varies tremendously from one role to the next. I'm a senior software engineer and LLMs, the way I'm using them, improve almost everything I do. I use them to write most of my code now, but first I spent twenty years writing code before LLMs came into existence and second writing code is like 5% of my job. Most of my job is research, investigation, and architecture. I treat LLMs just like a junior engineer. I give them clearly defined jobs that I could do on my own just fine, that I already spent years doing. The problem here is that students are using LLMs to automate everything BEFORE they become proficient at it themselves. Letting college students use LLMs for homework is like letting kindergarteners use calculators instead of counting on their fingers.
You cannot tell me that letting anyone do something for you does not affect the skills that you outsourced, unless you are some sort of a superhuman.
As an example, I have been drawing portraits for quite a few years now, and whenever I go on a hiatus and come back after a few months, I can notice my skill not being anywhere close to where it was before I stopped using it.
Sure, after 2 or 3 portraits they mostly come back because of the previous experience, but skill rust is a real thing, and if you think your coding skills are the same because you used to code 20 years but haven't coded for some time, you are probably just lying to yourself.
Yeah he’s living in denial.
His skills are slowly eroding. Given that he spent 20 yrs building it up it won’t happen overnight. But the trade off is happening in real time.
My "digging for roots to eat" skills have also atrophied. Fortunately I don't need those much anymore because of modern agriculture.
I wonder how much of this "you are gonna lose your skills!" stuff matters. And if knowing how to properly iterate a for loop with my eyes closed matters all that much anymore.
On the contrary, with the amount of times I went to ask for help and was failed pedagogically, plus not being able to afford tutoring like my peers had, I think access to an LLM would have genuinely boosted my grades.
I still did well, but I had gaps for which there was no help outside of the internet available.
The risk or difference is that tutoring helped people learn which they can use to do the work, whereas with only one or two different words an LLM will do the work (that proves you have learned) for you. A tutor has limits, but an LLM needs to be asked to set limits. And especially younger people are less likely to "punish themselves" like that.
You're using it to help you study and think, which is great, but the original post was about how many people are bypassing that step entirely.
I recently switched back from a Tesla to an older car without permanently having a map visible. Suddenly my brain has to think about routes again and it definitively feels like my brain has to put in more effort again to handle it.
This likely varies person by person or the way people adapted AI. For me AI replaced the boring part of writing code, but has not replaced the fun part of thinking about code and problem solving.
We also had exercises for which the solutions were given, and we didn't reach for them immediately...
“Better for you if you take me off.”
The Whispering Earring: https://croissanthology.com/earring
Maybe you've discovered the great filter.
before AI was around we blamed Covid for doing this to us, and now we blame LLMs... and before that we blamed social media. I'm pretty sure this downtrend has been happening for decades.
When reading and writing became prevalent, the ancients bemoaned our reduced facility to memorize long texts. Are we now “less smart” because of that technology?
> I have some sympathy for these kids.
where do you see kids? This is a university. These are adults. 100% their fault.
No, that's the quality of candidates. I wish I was joking as a PhD holder for only 15yr.
A lot of skill of is getting bled into the private sector because getting the PhD in a lot of regions doesn't mean the step up it used to. A lot of that comes from awarding them to layabouts doing "a gender critical analysis of ...".
Industry doesn't how/what/why they just wanted the 3 letters as a performance barrier to hire competants.
HN is going to eat up this garbage
Yeah, it's a scary thought. I feel the pull of it every time I'm stuck on a code problem that I don't want to search solutions for and hand-code... and I also feel myself wanting to reach for the crutch of an LLM when I just have something boilerplate and easy to do. It's incredibly tempting to just ask the question and have the "thinking" done for you. Until you have actual skin in the game and realize that it doesn't reason, and its "thinking" is utter shit. Then it's like: you got addicted to cigarettes and now you have to quit, because this habit is poisonous. It really does lead very quickly to cognitive decline if you rely on them, or even think about asking them while you're writing code.
The LLMs will gradually cramp more and more adds in their responses in an effort to make a profit.
At least now we know why we will start watering our plants with Brawndo.
> Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well; many of them can no longer brainstorm, code, think deeply, or write without an LLM present doing 90% of the work. Many of them can no longer sit quietly for even 30 minutes just thinking on their own, which is a required skill for producing original thought.
This was my experience even pre-LLMs though (about my own PhD thinking skills too). I blame the amount of random stuff work now involves more than LLMs.
I used "AI" in the 2000's to increase my homework assignments, and to correct them.
As in I wrote code to generate random exercises, with solutions, using many tricks, to get myself hundreds of problems instead of 1 or 2.
Often spent more time on getting these programs right than on the problems. Still did better than the class. Oh and it was AI in the 1980s IBM sense. Ie. it was based around a python version (which I wrote) of a LISP math system based on maple. I even attempted (and largely failed) to rewrite it in C++.
Even attempted to have my homework read to have the computer correct the actual pages, but I never got convnets to reliably read entire lines (yes, I understand, well now, why a convolution would mostly not realize whether 2 pieces of text are on the same line or not and so get very confused if you go deep enough for recognition to work well)
You make good points here. But I want to point out some issues that I have with what you are saying, because I see a few assumptions that I would not myself make.
I graduated from RPI with a degree in Management and a concentration in Information Systems. I began in Computer Science, and didn't like it because RPI CS at the time was loaded with professors who were mathematicians who had transitioned over to CompSci and because the 100 and 200 level courses were excessively math-heavy in my view.
Since this was the late 80s, there may not have been an easy way to teach B.S.-level computing without it being heavily math-based, but I digress.
No matter what degree we achieved or what work we ended up succeeding at, we have a tendency to look back at people rising in the ranks below us, see differences in their experiences and struggles, and say, Look! That is evidence of a lack of rigor or a lack of understanding of fundamentals that we had to learn in order to succeed.
The only thing is that some of what we learned to become successful just isn't necessary to be learned when we learned it.
I do a fair amount of low-level software engineering with Claude Code now that was above my level of understanding of data structures and algorithms because I never took those CS courses at RPI because I switched to Management IT.
But as someone who could be described as a solopreneur at some level, my new system designs reach a certain level of complexity or code maturity, and I hit problems that I would not hit if I had more understanding of data structures and algorithms.
So-- I end up having to learn aspects of those disciplines at that point, rather than before I actually needed them.
I run into these situations often enough where I now say to myself, gee, I wish I had taken Data Structures. And I think, could I effectively take Data Structures at this late date and get better at specifying how I want data stored, or perhaps knowing the shortcomings of simplistic database structures that are the ones I end up with initially because of my lack of spec-writing skill?
Aren't many of the less experienced folks who come up now, whatever age they are, going to hit problems that show them their weaknesses in this fashion?
Is the issue that these people will never get jobs because the seniors and managers who are interviewing them will design interview questions that keep people with their level of understanding out of the workforce?
What happens when somebody who sucks at the fundamentals but is really motivated bangs their head against their shortcomings and eventually succeeds in building something that takes off? Aren't those people great assets because they learned some of their critical skills the hard way?
> If LLMs were around when I was a student, I would've also used them to "speed up" my homework assignments then proceed to fail all my tests.
As a counterpoint, I was once a physics grad student. I didn't finish the PhD because at some point I discovered that I was not going to be the next Richard Feynman and this was too much for my ego at the time. But I think that if LLMs were available, I might have finished.
Part of my problem was that at some point the math transitioned from stuff I understood to symbols and notation that I knew how to manipulate but didn't really understand. LLMs could have helped bridge that gap.
On the other hand, it's hard to imagine I wouldn't have used it for Jackson, etc. but we got Jackson solutions from previous students and the internet anyway. Using LLMs probably would have been more effective, used correctly.
This is also where I had issues advancing in math. For so long I was able to build intuition around mathematical concepts easily. They fit in my head and made sense. I couldn’t understand why my peers were so bad and slow at picking up the concepts. Until my first calculus class where there was absolutely no focus on the intuition or practical utility. It was just formulas for the sake of formulas as exposed by our teacher.
It wasn’t until I was curious enough to learn about calculus outside of the classroom that I was exposed to things which helped develop that intuition and made the calculations something other than just symbols and equations to memorize.
I think this would be fine as an adult, if it meant using LLM to churn out the boring work required of you at a corporate gig, to spend more brain cycles on something you actually want to work on.
The problem is that it sounds like many people are just using it for everything.
> For adults the cognitive decline won't be as measurable since there's no exam
I think this is true of every affliction that adults criticize children and teenagers of
I’ve been out of university for a very long time, and I took a community college course and for the first few sessions I couldn't focus or sit still at all. Fortunately I knew that was abnormal and how to conform to a prior version of myself, but I don’t think children have a frame of reference.
I see this too. There is this theme where people are more and more going only as far as the ai does.
Asking suggesting or arguing to go deeper is impossible. There is a new path of least resistance and it saddens me.
>Now I work mostly with PhDs who were at the top of every academic environment they've ever been in. And yet I can see their thinking skills rapidly declining as well;
tomorrow most regular people's thinking skills will definitely be weaker than those of the LLMs of tomorrow. And physical skills in most cases will be weaker than those of the robots. That leads to the question - what would most people do?
I've got mixed feelings on AI assistance. I'll relate 2 anecdotes.
1 - When I was in grad school (before AI), we had to use Canvas for a class. One day, I got an obvious spam/phishing email in the internal Canvas system. It was so strange. The writer just would randomly hit the capslock button and keep typing away, no salutation, no signature, just a real mess. They were asking for a particular professor to come to their house to teach them about ... something? Again, real strange.
So, I email IT and say 'Hey, somehow a spammer got into the system, do your thing'.
They email back and go 'Nope, it's a student, that somehow managed to CC the entire system, sorry about that'.
Dear Reader, the message was pure garbage. Literally, it looked liked it was written by a 3rd grader without any shame. [0]
I happened to know the professor of the class. So later on, I talked with them over symposium coffee about it. They said that they remembered that particular email because of all the IT back and forth. It was for an upperdivision class in the Engineering department. The email itself was not particularly notable otherwise. In that, they saw such emails all the time, in terms of quality. This was a top 100 ranked (whatever that means) university, by the by.
Shocking.
2 - My grandfather was an officer and a mechanic for the USAF. A bit of an odd combo, but he was partly responsible for instituting many preventative maintenance checks and protocols, novel in those early days of the AF. His aptitude and memory were quite sharp for many mechanical things. Until the strokes from decades of smoking caught up, he could tell you exact measurements and torque values for a variety of airplane related things (I can no longer remember what exactly, the memory skills did not transfer to me).
I do vividly remember standing in that light blue garage of his and him all but yelling at me once. We were looking at the brakes on an old car he was 'restoring' (getting away from Grandma for a little bit). He pointed at the old drum brakes on the axel.
He asked me how tight the pads should be on the inner rim of it.
I had no idea.
So he asked where I might find out.
I figured I'd ask him.
But what if Grandpa wasn't there?
We'll I'd have to look it up somewhere (they had no internet).
Fantastic. Now, what about the next time you're working on the brakes?
Well, just make sure that the pads are at that spec.
And that when Grandpa hit me with the nugget of hard won wisdom: No, you look it up every time. Because these are brakes, and if you are wrong then they might fail, and they might fail when the driver has their whole family in the car at 100 mph. And then because you were lazy, half a dozen people die.
---
These two times stand in my head when it comes to AI.
For the first one, yes, AI would be such a boon to that very clearly struggling student reaching out for help. It would get them back on the path to the real struggle of getting their degree. That level of assistance would be like a wheelchair to a paraplegic.
For the second anecdote, AI is condemning people to death. Using it in life critical situations and care, letting it hallucinate or skip over critical values, that's a recipe for disaster.
Where do we set the fine line of using AI and not? For brakes and X-ray machines, obviously not. For helping kids learn to write emails correctly? Sure, sounds great.
Unfortunately, I feel the old adage about regulations is going to be true here like it is with every new technology: The rules are written in blood.
[0] therh lasdkjenail EMAil was sposmkehting alLIKE HTIS IN THE WAYS THAT IS aw adklhjf writtene sand htenn. sned t sent ASLAOSNG ASLIONG ALONG TO NTHE recipentnts .
How many took the COVID vax? /s
> Many of them can no longer sit quietly for even 30 minutes just thinking on their own
Sorry, but I highly doubt that. Has a very "old man yells at clouds" vibe.
I doubt that you doubt that. I think you actually believe they're speaking truthfully and in good faith.
I doubt it. I'm stupid and I use LLMs a lot but I can still meditate for 30 minutes.
But apparently some of the smartest people in the world have lost the skill? But the commenter haven't, because why, they're 15 years older and thus immune to the same LLM-effects?
Plus, the issue with people having trouble sitting still for 30 minutes precede LLMs with decades.
Why is it so hard to believe? The young adults now have grown up with short form media and instant gratification / dopamine hits from apps. It's vastly different than people of the same age just a few years ago.
Not saying everyone else is immune, but those a few years older have also had a period without it.
I didn't say I'm immune to those effects, I'm including myself in this as well. (also, I'm not older than my colleagues).
Most people definitely can't meditate for 30 minutes, so if you can do this, it's very impressive. Regardless, being able to think about poorly-defined problems and build completely new mental models from nothing is genuinely a really hard and uncomfortable task. If you don't use the skill you'll lose it.
> Most people definitely can't meditate for 30 minutes, so if you can do this, it's very impressive.
Maybe not traditional meditation, but I have no problem taking a 30 minute plus walk with nothing but my thoughts. It’s actually when I do most of my thinking. The other is in the shower/sauna where devices don’t work anyway.
> I'm stupid and I use LLMs a lot but I can still meditate for 30 minutes.
> apparently some of the smartest people in the world have lost the skill?
> But the commenter haven't
> why?
Perhaps because a correlation you assumed was there (more smartness = more ability to sit still alone with one's thoughts), is not actually as strong as you thought? If one does not start with that assumption, there is no inherent conflict in the 3 pieces of evidence you cited.
Or perhaps because you are smarter than you give yourself credit for :)
I feel the opposite. My brainstorming has increased rapidly. I can now just throw ideas at an LLM to rapidly validate.
Are you saying you leave it up to the LLM to judge whether your idea is good or not? Are you even human anymore?
(I am not saying LLMs can't be a good tool in evaluating ideas. To me, it sounds like you're firing off ideas all over, letting the LLMs judge what's good and what's not. Insane.)
Not judge no. Implement and create a working MVP, yes of course.
And yes, I fire off ideas all over. Many require predicting the future to decide what to focus my individual effort on. This is a terrible way to do things because humans (and LLMs) are notoriously terrible at predicting the future. The gold standard is to try everything and eliminate what doesn't work. This is impossible using human labor. With LLM labor, it's simply a matter of relatively cheap money.
It's amazing. Technical problems are now no longer having to predict what the best implementation is. You can just try each one.
Again, no need to have an LLM judge, because the metrics that define 'better' are well-defined, and this is the interesting part of computer science, not the implementation.
The likely 'real' reason is hidden in one paragraph within the article and has nothing to do with the implication of the eye-catching title: "Both Garcia and Ranade have joined more than 1,300 UC faculty in signing a petition calling for the reinstatement of ACT and SAT standardized testing scores for STEM admissions in the UC system. The petition and its accompanying open letter detail similar concerns with students’ mathematical preparation."
Around COVID times many top universities experimented with removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere, with many, if not most, universities already reversing it. As Yale put it, "Yale’s research from before and after the pandemic has consistently demonstrated that, among all application components, test scores are the single greatest predictor of a student’s future Yale grades. This is true even after controlling for family income and other demographic variables, and it is true for subject-based exams such as AP and IB, in addition to the ACT and SAT." [1]
That link is for an archive because that page has been removed. That's because they briefly experimented with a new 'test flexible' strategy where they allowed students to submit test scores or not, but then scrapped that altogether and went back to simply requiring test scores.
[1] - https://archive.is/8zxfo
Berkeley chancellor told students to vote for 2020 California Proposition 16, which would've repeated 1996 Proposition 209 that banned race-based admission in public universities. Prop 16 failed. Subsequently, Cal started ignoring SAT/ACT scores. I have to think this was their alternative way of taking fewer Asian students, who average highest on that. Soon after I got an email from the same chancellor praising the change for bringing more racial diversity. The email included before and after numbers where % Asian decreased and all others increased.
Reminds me of this asian professor getting blocked for promotion, allegedly, on the basis of race, but in a roundabout way. https://www.msn.com/en-us/news/us/professor-sues-texas-unive...
Yes, they falsely classified him as white in order to deny him the promotion (because discriminating against white people is 100% non-controversial in that environment)
I think it's just indians being racist. The person could've put any other non white category and it would've happened.
They could have easily made test scores a pass/fail per program and not weight higher scores for admission purposes. It achieves the goal of ensuring students have requisite knowledge for the program while not favoring students who are able to ace the test.
Or, even better - just expand programs so they can accept more students who pass the test. This would probably improve diversity without artificially restricting access to highish performers.
To expand the UC system to accommodate everyone who could do the work would require repealing Prop 13, much harder.
(minor typo: repeated --> repealed. Sorry for the nitpicking but it confused me when I first read it)
No that's valid to call out, thanks
If the removal of standardized testing in 2021 was the real reason, then why is there a sudden spike of failure rates happening right now?
https://senate.ucsd.edu/media/740347/sawg-report-on-admissio...
Please see the graph "Growth of the Math 2 Population by Major (2019-2024)". UCSD's Math 2 class is remedial high-school level maths. It has grown from under 100 students in 2016-2020, to more and more people each year starting from 2021.
UCSD tested the people who took this class, and 25% of them could not answer the question "Fill in the box: 7 + 2 = [_] + 6" (with only pencil and paper allowed, no calculators or other electronics)
this is a Berkeley CS class, not a remedial math class.
I'm suggesting the problem is not limited to Berkeley. They both show the same underlying issue, there's a growing number of students attending university without the prerequisite maths skills they need to succeed.
It seems they're now at the point where the sheer number of students that need improved maths skills overwhelms the staff, resulting in them failing.
It takes time to work through the system and it has been steadily getting worse.
It was already discussed on HN.
https://news.ycombinator.com/item?id=48309233
I'm having a difficult time imagining how an admissions event in 2021 materializes in the spring semester of 2026 in a class largely taken by first-year students.
Could you explain?
It didn't just suddenly materialize.
From the current article
In addition to overreliance on AI, Garcia also pointed out that many students are underprepared mathematically, a concern echoed by campus associate teaching professor Gireeja Ranade.
From the article discussed the other week:
Over three years — from fall 2021 to fall 2023 — the letter said, at least 20% of Berkeley first-semester calculus students who took a diagnostic exam showed deficits. “Basic mathematical fluency is analogous to literacy; without it, success in university-level STEM becomes structurally unattainable for students,” faculty wrote.
It's been steadily getting worse. The current article only looks at F's which conveniently hides if there has been a slope down. Additionally, kids entering HS in 2021/2022 would just now be hitting college.
> It didn't just suddenly materialize.
A sudden materialization is what's depicted by the data.
> It's been steadily getting worse.
I don't believe this is accurate. Failing grades are what the observation entails, and the data clearly depict an abrupt change; not a gradual one.
In the section titled "Failing grades in 3 CS classes skyrocket in spring 2026 ", there's a clear jump in failing grades for all cited courses between 2025 and 2026. Failing grades for every course jump by multiples of the previous year.
The jump is very likely due to AI usage and lack of skills in mathematics. It seems like prerequisite classes are not being fulfilled.
"Ranade said students are expected to enter the course having taken classes on linear algebra, vector calculus and mathematical proofs. However, she found out in office hours that many students struggled with linear algebra, and was even more shocked when one student told her the linear algebra class they took at UC Berkeley had an “open-internet, open-AI policy” for homework and exams."
Also, this professor doesn't grade on curves? Could be very specific to this teacher. I don't know. Would be great to have more data but it is a big jump and could be very specific to this professor or perhaps this class.
> It seems like prerequisite classes are not being fulfilled.
FWIW I did a little digging, and EECS 127 indeed has explicit prerequisites of:
* Math 53 - Multivariable Calculus
* Math 54 - Linear Algebra & Differential Equations
* CS 70 - Discrete Mathematics and Probability Theory
This suggests the students are either taking those classes or have provided some kind of AP/test-taking credential to skip them.
"Also, this professor doesn't grade on curves? Could be very specific to this teacher. I don't know." Someone has to hold standards up -- they seem to be falling down across the board in education.
Actually, when I read they usually graded on a curve, I lost all interest. I don't respect teachers that grade on curves.
You should be graded by how well you know the material - not how well your peers don't know it. I'm always grateful both my undergrad and grad professors didn't curve on a grade.
In my first company, I had 4 different jobs. It was a common adage: Go into a low performing team that does simple work and you'll get promotions much quicker than in a high performing team doing challenging (but fun) work.
It was right. I had 2 "dream" jobs where I did cool, challenging stuff, but where everyone was more than competent. They turned out to be career killers. The promotions I got were all in the other 2 jobs where I did boring business logic coding, and where my peers were barely competent (one had trouble navigating directories using the command line).
That's what happens when you grade on a curve. Smart people begin to work on boring stuff, and not the real challenges.
For failing grades sure, there must be some sort of minimum competence. For sorting out >= B/3.0 grades, a curb can work since you are getting evaluated against your peers to see he is standing out vs just doing acceptable.
If you wanted to grade purely off a curve, you would be stuck with old test problems that were thoroughly vetted and calibrated, an impossible task for smaller classes where the material changes rapidly.
> For sorting out >= B/3.0 grades, a curb can work since you are getting evaluated against your peers to see he is standing out vs just doing acceptable.
I'm still not getting it. For a standard course, the criteria for what is "good" vs "great" should be pretty clear, and it should be independent of your peers. You have a syllabus, and a set of abilities for each grade level. If you hit those targets, you get the grade. If half the class gets an A, then it means they're pretty smart, or you did a great job in teaching. Of course, there's the chance the class was too easy, but you can always fix that.
No, I don't see why you're stuck with old test problems. For standard engineering classes, there's a huge (almost infinite) set of problems one can create.
For smaller classes, grading on a curve is even sillier, as the variance is always higher when the population size is small. For example, a lot of my small classes consisted of highly motivated students (all "A material"), because they're usually obscure electives where the content is challenging. You then pointlessly penalize students who sign up (just like they do at work). In fact, my professors were usually much more lenient on small classes for this very reason (i.e. lowering the standard needed to get an A).
I once took an Intro to Analysis course. It was moderately challenging. I got the highest score in the class, and my grade was A-. Everyone else got B+, B, or lower. A friend of mine (who didn't take the course) got really upset that I didn't get an A (or A+) given that I was the top scoring student.
But I knew my level of understanding/performance. It wasn't that great. I felt even an A- was too high a grade for me. And the teacher did a pretty good job in teaching. Why should I get a higher grade just because the other students were worse?
> For a standard course, the criteria for what is "good" vs "great" should be pretty clear, and it should be independent of your peers.
Do you think upper division college classes are somehow like high school classes with well developed curriculum and teaching professors who teach the same thing every quarter? Now you expect the professor to not only come up with new test material, but also extensively calibrate it before students take it, maybe for a 15-hour per week class (3 hours of teaching + 12 hours of studying), with maybe 15 students? Well, thank God we have AI for these kinds of things now.
Ok, let's exclude upper devision classes and just focus on lower division courses (since you mentioned an Intro to Analysis course). Here you have a relatively better chance of a well understood enough curriculum and testing material to actually not grade on a curve. BUT these are also usually weed out classes, with the idea that they only have N spots for students to proceed on to the upper division course, so curving serves an actual purpose that is aligned with the intended result.
> Do you think upper division college classes are somehow like high school classes with well developed curriculum and teaching professors who teach the same thing every quarter?
I repeatedly said "standard course", which implies it is a commonly taught course (be it upper or lower division). In my undergrad, Analysis I, II and Abstract Algebra I, II were upper division courses. In the engineering departments, stuff like Electromagnetics I, II were upper division.
Anything that is not an elective (and even some popular electives) were standard courses.
Now I'll grant that in CS, some material like machine learning changes rapidly. But in most engineering, very little in the undergrad material changes. Even my semiconductor courses in undergrad haven't changed much in decades.
So yes - for most of those classes (and that means the vast majority of undergrad engineering) classes, the curriculum is relatively standard.
> Now you expect the professor to not only come up with new test material, but also extensively calibrate it before students take it, maybe for a 15-hour per week class (3 hours of teaching + 12 hours of studying), with maybe 15 students?
First: In my very average undergrad university, professors were always careful not to reuse old homeworks/exams. It wasn't a huge burden. Professors who don't do this (e.g. most professors in top universities) signal very clearly their lack of interest in pedagogy.
Second: You want to do a curve on <= 15 students? Are you aware of basic statistics and the problems you get with small N? Are they using a normal distribution or one that is more appropriate for small N?
And as I already said, for a lot of electives where the material isn't standardized, professors lean towards lenient grading. They offer those classes because they want people to take it, and grading via a curve discourages it.
> since you mentioned an Intro to Analysis course
That was an upper division course. Yes, I know some universities have it as a lower division, but many (most in the US?) treat it as upper division.
> BUT these are also usually weed out classes, with the idea that they only have N spots for students to proceed on to the upper division course, so curving serves an actual purpose that is aligned with the intended result.
It was not a weed out course. Neither my undergrad nor grad math departments had weed out classes. I saw that concept only in the engineering departments. My EE department had only Circuits I, Circuits II and digital logic as "lower division". Circuits II was the weed out course, and you were not allowed to take anything else (e.g. E&M, Electronics, etc) unless you got a B or higher.
> It didn't just suddenly materialize.
Huh? I'd say that a jump from 10% in 2025 to 35% in 2026 qualifies as "suddenly materializing"
That's not to say that things were already getting worse, but this is a big jump that is definitely not explained by not requiring SAT scores.
SAT/ACT math is incredibly simplistic and at worst maybe contributed by not filtering as many out. Math scores have been declining nation wide for decades now, that’s been a big issue for a while.
One big reason is preparation, people start preparing for tests 2 to 3 years in advance. And the method of testing influences exams used in grades before as well.
So assume 4 years of high school and someone that just came in. They are still preparing for SAT like tests in their first year of high school. Someone in final year of high school is well trained in it. So even though the benefits do not carry, enough portion of incoming students are still reaping benefits of standardized tests. The decay only shows later when batches without any benefits of standardized tests are coming through.
> people start preparing for tests 2 to 3 years in advance
Pardon? Is that a normal thing in the USA? I don't think I've ever started preparing for a test more than a week and a half ahead, a month if you count graduation exams. Not sure they ever determined more than a year in advance (more commonly: a bit less than a semester) what tests we'd be given in the first place
> They are still preparing for SAT like tests in their first year of high school.
Literally nobody does that except high achievers whose parents are pushing them for a high SAT score to get into Stanford or whatnot. Those are not likely to be the kids who are now getting Fs.
That's not what this actual data shows. While there has been an increase math deficiency, the increase in failure rates happened recently and probably only partially related to the math preparation issue.
I think we will make a major mistake if we think math preparation fixes this - especially in CS classes where AI literally calls out to be used for projects. And it certainly doesn't explain me hearing the same problems are happening at MIT -- they just are being a bit wiser about "catching students" (or rather not doing so).
I'm guessing the kids who didn't do the standardized tests at/shortly after 2021 were already prepared for it.
The kids who saw the removal of standardized testing 3 years out from going to college never bothered.
It takes time for students to work their way through the system.
In the spring, but not the fall?
Wouldn’t this change be evidenced right away after the elimination of the test as criteria if the test was responsible
No probably needs a couple more years. Writing the test itself is motivation to do well in HS math. If that no longer becomes a driver probably takes off the drive in other courses over a couple years. I bet without the SAT as a standardized test a lot of HS math courses are easier for the teacher because the quality can lapse.
Also some children who excel write their SATs sometimes 2-3 years before college and then re-write if need be.
Not if kids are prepping for the test in a way that results in real gains. Which seems likely, especially in the age of AI: "should I actually study math or just use ChatGPT to pass this course?" One semester of coasting through might not do that much harm, but at some point the compounding effects will tip you over the edge.
Define "right away". How long after taking the SAT does a child have their first classes at university? At least a year?
There's always a lag between cause and effect in education.
Works the other way too - if you introduce something positive in grade 1, you'll only see the results a few years later.
If it's a lagging effect, then why is the year-over-year spike in failure rates happening not just in 1st/2nd year classes, but also in a 3rd/4th year class at the same time?
Good point. I'm inclined to think it's because of a quality threshold in chatbots.
Entrance tests predict academic success , not 1st year success.
"Failure to complete the qualification" is the prediction.
Testing was the annoying flood barrier. AI is the rainstorm that shows why it was necessary.
This pattern reads heavily LLM in style, but also... that is spot on.
I'm not American so maybe I am missing some context. But how did admissions work without test scores?
> how did admissions work without test scores?
They look at "life experience" factors. Grades, of course, but also extracurricular activities such as school clubs, volunteering, activism, socioeconomic background and so forth.
The aim was to include as many varieties of these factors as possible in the student population to boost diversity without directly referencing skin color (which would be illegal).
There's a lot of antipathy towards standardized tests as things that disadvantage kids who don't test well, despite it being much cheaper and less time intensive to prepare for the test than it is to join a bunch of clubs and spend your weekends volunteering.
It turns out that removing the standardized testing requirement led to a lot of students wasting their money on courses that they weren't at all prepared to take.
It varies by school. I went to a (low ranking) state engineering school and it was guaranteed entry if a prospect met the following criteria:
- Had high school diploma (or equivalent).
- Resident of the state for >6 months (student or one parent).
- ACT score of something like 21. With provisional admission granted to students with scores below, until they completed all first year engineering courses with a B or better.
So likely they just dropped the concept of provisional admission. All that did was open up classes for registration a week later to ensure other students were able to get their preferred class openings. Provisional had to take the scrap classes, like the four-hour, once a week Calc class on Friday night.
Not American either, but in the US many schools use/used standardized admission tests (SAT/ACT) on top of things like HS GPA/grades.
There are many countries, especially in Europe, where entrance/admission tests are not a thing.
Yeah, in England only certain universities like Oxford and certain subjects like Mathematics have separate entrance exams.
That said, the Sixth Form exams are mostly standardised with only a few different exam boards for the entire country, so the Sixth Form grades end up being something akin to standardised tests anyway.
Excellent question. Sorry that's not allowed. DEI is the name of the game.
They look at high school transcripts and the application essays. I don't know how they decide based on those.
Primarily your high school GPA, the average of your grades across 4 years of high school. Which arguably is a better indicator than a single test that you can test for.
The problem is if you have a high school with low standards you're getting A's when you didn't deserve them.
SAT requirement was dropped in 2021.
Not taking the SAT doesn't explain a jump from 10% F's in 2025 to 35% F's in 2026.
> top universities experimented with removing test requirements from admissions
What could go wrong...
But do these universities not have math placement exams? Not for admissions but just before you register for your first semester classes, a 30 minute math test should be a straightforward preventative measure. I did a test like this, I assumed they were pretty universal.
They do -- this is often how they've found that students needed additional math coursework before starting the standard curriculum.
> Around COVID times many top universities experimented with removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere, with many, if not most, universities already reversing it.
It's the universities that have failed. They've restricted admissions to a set of people who would learn no matter what the schools did, which is what makes them lazy.
When confronted with a set of students who haven't been provided with an enormous amount of childhood reading material, and the time, encouragement and social acceptance to indulge in it (the most faithful test predictor is childhood pleasure reading, the next best is parental income), they fail horribly.
The purpose of elite colleges for students is credentialism and networking, the purpose for the schools themselves is to force cultural conformity onto smart or extremely pressured students. They generally just tell you to go learn things by yourself. They have no particular insight into teaching, because they are supplied with students who don't need to be taught.
Can you cite a source for the claim about "most faithful test predictor"? I'm genuinely curious. I would think high school GPA would be more predictive.
I think IQ tests are even better.
High school grades are not evenly applied, and sometimes heavily inflated. Eliminating that variable is the whole point behind taking a standardized test.
Also interested
Memorize trivia and formulas, regurgitate trivia and formulas. This summarizes my experience with our system of education. Yale saying test scores predict performance reads to me as, “students’ history of being able to regurgitate trivia and formulas in high school is the lead predictor of their ability to do so here.”
> removing test requirements from admissions, under an argument largely related to equity. It's been a failure everywhere [...] among all application components, test scores are the single greatest predictor of a student’s future Yale grades.
It reads as though you tried to use the quote to support your conclusion that "it's been a failure", but the quote and the original rationale are optimising for different things. Something can be a success in improving equal opportunity while still leading to worse grades.
Or to flip it around: we could say admission testing "has been a failure everywhere" because it biases admissions in favour of certain demographics. But that wouldn't really be a fair assessment because being free of demographic biases is not the purpose of admission testing!
CS Professor here: just yesterday I did the discussion of a course projects' (Parallel Computing), and one of the three groups that I did yesterday have clearly gone the ChatGPT way. They couldn't even understand the choices the LLM made regarding the architecture, etc. The way to "catch" these students is similar to what we did in the past when students copied from other students which is "to give them rope to hang" - ask for clarifications until they follow unintended paths that lead nowhere.
To fellow professors, when you're suspicious my suggestion is to appeal to their honesty (like "let's be honest, how much of this code is yours, and how much is ChatGPT's?") and offer some empathy and understanding (like understanding they may had multiple deadlines in the same week, etc.). Nevertheless, don't miss the chance to give them the lesson on how is the correct way of doing things. The way to catch these students is to find the same signs of yesteryear copying from other students (which in essence is what copying from an LLM is, although the number has increased because they found us professors unprepared for the volume).
The other two groups also used LLM but in a high-level and architectural way. They were clearly responsible for the code (even if they didn't wrote it 100% manually) and could explain their reasoning and strategies used to solve the problems.
Me and my colleagues still have a lot of projects to review, and I asked them to keep the score of the number of projects like these, but so far, the score is 1 in 3 (33%).
So all groups outsourced their thinking to an LLM. Will they have learned anything they can apply to different types of projects in future?
In your opinion where is the line on where LLMs are useful/harmful to learning?
In my school projects I've found them super useful for working with libraries. In the past it felt like the theory I learned was pretty low impact. 80% of my time was spent learning the quirks of a library. Now it feels like I can take theory and iterate over a ton of different solutions without having to worry about learning whatever library the professor requires. Basically I'm I feel like it lets me spend more time learning the thing I want to learn rather than all the busywork around it. Would be curious to hear your thoughts. Thanks!
Not in the same level, see my other comments..
> The other two groups also used LLM but in a high-level and architectural way.
Sounds more like the score is 3/3 (100%)
Would you have accepted them cooy-pasting code from libraries together to build their project? If not, why is using LLM generated code different?
Using AI to inform architecture doesn’t seem so different from googling architecture in this case. Architectural patterns are mostly well understood and well documented these days and are something that you could piece together via Google search pre AI. The thing that AI brings to the table that wasn’t google able in the past is code generation. Previously you had to understand the architecture patterns to implement them yourself, but now the AI can just do it for you.
> Would you have accepted them cooy-pasting code from libraries together to build their project?
Yes, if they are "responsible" for the code delivered, where responsible means they understand the code, the architecture, the decisions made, etc.
In this case, the students had to invent multiple strategies to solve a specific problem. The "successful" groups did a mix of generated and hand-crafted code (don't know percentages), implemented different strategies and knew their plus and minuses, could change the code in a timely manner to accommodate some of my requests, etc. The "unsuccessful" group couldn't do any of that.
I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
But in reality I've been slowly transitioning from group projects (for a subset of the grade) to "practical tests", where they must implement a significant subset of a larger project in a 2h class. Still experimenting though.
I'd say most of my CS classes (in the 90s) had us write out code by hand, on paper, during in-class exams.
It was fine.
> and really, what could I do if I were?
I’m not sure what you mean. Have a no-AI policy and fail those who ignore it. If students lie, discipline them appropriately for cheating.
How could I ever enforce it? These are projects students do at home..
> Yes, if they are "responsible" for the code delivered, where responsible means they understand the code, the architecture, the decisions made, etc.
This is a good principle to maintain, I think.
I'm not a professor, but I manage a team of about a dozen people. The maxim I have is: "You're responsible for anything that hits git."
Don't care if the LLM generated it, or the LLM told you if it's a good idea. If you commit it, you are endorsing it as a good idea - so you're the one I'm going to ask about it. I see the same principle at work in your pedagogy.
> I'm not anti-AI (and really, what could I do if I were?) since I use it myself, I'm just anti-slop, especially from my students.
This hits. Especially this part:
> and really, what could I do if I were?
My completely unsolicited opinion: you're doing a responsible thing by teaching these students how to use AI as a reference, and keeping them honest about not using it as a substitute for their own critical thinking.
He explains why in his comment. Read it again, carefully. Or ask an LLM for an "explainer"..
I did a course of parallell computing 6 years ago. I still dont understand anything about it. Probably would if I had used an LLM, since I would have used it as a private tutor.
As a professor, what's you opinion regard the Socratic method with LLMs? Would that be preferable over simply "give me the answer" prompt?
Everything that provides students with a workflow to think and to try to find solutions to a problem is much better than giving the answer directly! Unfortunately there will always be students that prefer to take the shortcut..
How could we "force" the students to use an LLM that confronted their doubts with more questions? We could tell them to start each chat with a specific prompt (to use the socratic method, etc), but they could eventually jail-break it..
But nevertheless, I like your idea! This is something that a document highlighting methodologies for students on how to use LLMs effectively could/should contain..
I'm curious on your opinion, as a professor, how should universities (not individual professors) adapt to LLMs?
As an undergrad, I hope schools move to educating students to use LLMs in a more responsible way. You can't put the genie back in the bottle, and resisting progress is futile, might as well use the tool we now have to help students learn even faster and better (e.g., making feedback instant and not answers, helping digest or split up material, checking answers).
I know opinions about AI at (not only at) my faculty are very mixed, but I think the answer is going to be in the rational mean, just like how technorealism reacted to the internet[0].
In our last program board sitting, some teachers said that they think programming as a job will be completely irrelevant in two years, while other pushed for more adoption. And meanwhile I know of some students that are basically only passing because of LLMs, and it's bad, like "leaving claude output in markdown files and finished source code on the faculty server in /tmp because opencode did so" bad. And our first year classes completely prohibit even sharing tests or talking about the solutions, which in my opinion a) makes people extremely asocial and atomized b) doesn't prepare students for real life c) promotes dishonesty.
Still, I think our university's thinking is in a stalemate, not wanting pure AI output and useless students, while also wanting to move with the times, and I doubt it's the only one.
[0]: https://web.archive.org/web/20081009111415/https://artefaktu... (absolutely amazing read, recommend it)
What is the policy and guidance you gave the students regarding LLM use?
This is all so new, and caught us completely unprepared that there's no official university-level policies. Most of us are still navigating the waters and seeing what works and what doesn't work anymore.
I have colleagues that are teaching for more than 30 years, few years away from retirement, who suddenly have been confronted with a new way of doing things. Those are the ones that are still insisting on doing practical projects, etc. I've only been doing this for 20 years, and I'm quite lazy (worked previously as software engineer), so I've moved to those practical tests. I guess that there should probably exist a class or workshop to teach these students how to use LLMs effectively, but as I said, this technology and its implications is quite new.
Personally, what I did was to give them the "lecture" in the line of that they do not understand what the machine has generated, that is not the way a true engineer does, try to do some parallel with things like an LLM designing a bridge and civil engineers building that bridge, and a fatal flaw collapsing the all thing, etc.
In other words, we do not have a formal system in place, it's all talking and convincing them. Obviously it's a big enough problem that should deserve more investment in solutions, but we are all overwhelmed by other tasks. Maybe LLM studios should be held responsible for all these "disruptions" and provide solutions to problems they created! :)
My daughter was struggling with her Math class back in January. I used Claude to build a tool that allowed me to generate very focused worksheets. The worksheets had problems designed to drill the concepts she was struggling with.
It worked, and it would have been MUCH harder to do this the traditional way.
The tool generates PDFs including an answer key and solution sets that solved the problems using a variety of techniques so I could check her work more easily and we could iterate quickly.
That's powerful. It comes back to how are you using the tool. Are you using it to make things better or to take shortcuts?
It's a strange thing that as humans, we sleepwalk into every crisis, never agreeing on anything, and then when we're there, we also never agree on the causes. When we ge too the point where we can no longer "engineer" or "science" anything we will spend the next decade arguing that the issue was not really AI, or that if it was, it was inevitable and no one (or everyone) was to blame. Rinse, repeat. Yet we're here, today, looking at the bleak future, and taking yet another step forward.
Do we assume society just self regulates. I think it does, but the cost of letting it self regulate is really really high, with lots of suffering. Is it that we find this acceptable when there is a chance we won't be the first to feel the pain?
People have been warning about AI coming for decades. For better or worse, it's embedded in popular culture, in science fiction books and movies. But that's different from figuring out practically what to do.
It's cultural evolution and it's how markets work, too. You were expecting central planning?
Yes, climate change debate went the same way.
https://i.imgur.com/fk1aI.jpeg :)
We don't all sleep walk into these. Many people shout from the roof tops but the masses are easily manipulated.
The grade data come from https://berkeleytime.com/grades
I was worried they may have cherrypicked courses that support their chosen narrative.
So I plotted the % of F grades (red line) for all CS courses still offered, and sorted the chart in descending order of the # grades given out (light blue vertical bars) in the most recent semester when the course was offered.
My worry was borne out. See the first few charts. No big increase in F % in the past few semesters.
https://x.com/rahimnathwani/status/2062431813143019525?s=61
The article says they looked at CS 10 and 61A, which IIRC are the intro classes at Berkeley. Why do you think that amounts to “cherry picking” versus being a reasonable starting point for analysis (esp if those classes, as they are for one of the quoted professors, aren’t graded on a curve)?
Because CS10, being an intro class, has its highest enrollment in Fall, not in Spring.
In the most recent CS10 cohort (the one in which 35% of grades were an F) only 34 students were graded.
If you're going to look at intro classes, why not look at the Fall semesters, which have much higher enrollment?
You can see a chart of the data here: https://docs.google.com/spreadsheets/d/17acH9JkGE4MlYE1Aeh_i...Additionally, if someone is taking an intro class in the Spring, that's probably because they already failed it in the previous semester. While that doesn't guarantee that they'll fail again, it does make it more likely.
Sure, but 12 people got an F in CS10 in Spring 2026, which is higher than any prior spring. The second-highest number of Spring F grades was 9, in Spring 2022, when four times as many students were graded (133 vs 34).
Separately, there's a clear downward trend in the number of grades being awarded in the fall semester. Perhaps students have the option to take the course pass/fail?
12 Fs across 34 students is not statistically significant. Comparing the spring 2022 vs spring 2026 rates only works if the composition of the groups is similar (e.g. same % of students that previously failed the course), which seems dubious given the dramatically different sample sizes.
That doesn't mean there isn't a problem worth investigating. If I was a dean I'd certainly have some questions if I found out that 35% of students failed a class. But without knowing the specifics it's irresponsible to draw any conclusions about these 34 students with the limited data we have available.
Alternatively, many faculty may be simply adjusting their curves to avoid failing too many students because they are incentivized to avoid giving too many low grades. Lower grades generally result in lower course evals, which can impact raises and promotion, and low grades can also result in additional attention from admin that no professor wants to deal with…
In unrelated news
"More than 600 University of California faculty members, led by mathematicians at UC Berkeley, are calling on the system to reinstate standardized testing requirements for science, technology, engineering and mathematics applicants, saying that six years of test-free admissions has not reliably assessed readiness and professors are often teaching middle school math to incoming students."
https://archive.ph/18spS
Who even makes the decision to drop having a standard bar to verify students?
And what possible benefit would that have?
The book SAT Wars has arguments for and against and the striking thing for me was that some in admissions believe in a concept called crafting a class: the applicants are input into the admissions officer’s artisanal contribution to producing a class that they believe would be good for the university to have.
The idea of a standard bar and so on does sound like it would interfere with such a process.
I always did find it interesting that US notions of anti-racism required treating individuals not as individuals but as racial representatives. It’s a local quirk of the culture of the land, I suppose, that one’s primary identification here is one’s skin colour.
America's vapid fixation with race is ridiculous especially since it uses race as a proxy for social stratum when it could just be addressing class issues directly instead. If only there were some history of forms that parents fill out every year showing their income to the government that is more-or-less vetted to some degree—too bad we don't have such a thing that students could use to prove social stratum! Plus, what the hell is race anyway? An unethical tip one could give to university applicants would be to claim membership to the most beneficial group because it's not like university admissions has any way of proving your "race". Construct any fabricated story that'll get the most approval and maximize your chances of getting into a top university.
Unfortunately, the fixation with race in America doesn't start nor stop at college admissions. College admissions is probably the last place where it tilts in the direction of certain minority groups.
I agree that we should just stop using race everywhere and we should crack down on it -- but I think college wouldn't be where my energy would be... actually the military is where I'd start. And oddly it's the place where race based affirmative action is still permitted (military academies - where it benefits minorities) and in its halls (where I've heard that it has a strong white supremacist bent). The reason is because what is happening in colleges is more reactionary -- fix the catalyst and the arguments for the reaction largely go away.
The very real downside is if you just provide equal opportunity then you would still end up with an all white/East Asian/Indian class. You can argue that this is fair and meritocratic and you would be right but I believe there are a lot of people that see value in affirmative action that makes classrooms look more like the racial distribution of the country and provides an easy step up for traditionally poor/downtrodden communities to improve themselves. Affirmative action isn’t only an American thing btw, it’s in China, South Korea, India etc. as well.
Why else does anyone in California get rid of anything? Because it’s racist.
See: when they banned guns because of the black panthers.
I thought for most universities it was because some schools didn't organise the tests and others did during COVID.
More leeway in who you accept (for money and/or clout).
The decision wasn't specifically to drop a standard bar. It was to drop the existing bars because they have become heavily gamed and are far more reliable indicators of your family's resources than your ability or likelihood of success. That was the equity argument.
Unfortunately, the lost signal wasn't replaced with anything. (I don't know what could replace it. It's an incredibly hard problem. )
Easily gamed? Sure, education is pay to win on some level but we're talking basic prerequisites here. If the objection is that disadvantaged would-be students are being filtered out then start an outreach program to help them prep or something.
Hard to find the actual letter: https://ucstudentsuccess.org
They worry me. A lot.
My son is 15 and I use Google Family Link to control what he does on his phone: it's pretty open for the most part (I receive notifications of installs) but Gemini is a hard-ban.
We've spoken at length of the dangers.
He says his pals use LLMs frequently and I suspect that's the reason for their test scores: some of them are in the 20% - 40% range for tests whereas my son is 80%+ because he studies past-papers and answers questions in his revision.
I worry for the future coz you can be sure that the AI providers don't care if a schoolchild is using their LLM to answer the homework questions.
and this is why standardized testing exists so they can apply those off hour work and discipline to demonstrating their aptitude for being comfortable with boredom long enough to function in academic and workplace environments that require it.
rather than perceiving AI as a danger you should be looking at how he can leverage it to accelerate and enhance his learning but the political environment focus on removing standardized testing to hide data of those that traditionally fail is the true danger.
Did your son behave in a way for that to be installed or did you do it by default?
> Gemini is a hard-ban.
Sounds like you would hard-ban your son from using Internet if it was only introduced 5 years ago
Or a calculator. Oh wait, I hard ban my 8 and 11 year old kids from using a calculator. I wonder why.
That... is not a good idea.
Inexpensive calculators were relatively new when I was that age, and I learned so much just playing with them.
What exactly did you learn?
>In addition, the guidelines state that “a typical GPA for a lower division course will fall in the range 2.8 – 3.3.” In spring 2026, both classes’ average grades were C-pluses, according to Berkeleytime, corresponding to a 2.3 GPA.
As a Cal alum, I am actually really glad to see they are holding the line on grade inflation. I worked my butt off to achieve the GPA I did, and it would really suck to see my labor devalued if Cal went the direction of e.g. Yale and started handing out 79% A's and A-minuses: https://yaledailynews.com/articles/professors-face-grading-d...
I read the subreddit for the UC I went to. When acceptance letters went out this year there were (as you expect) a ton of questions from accepted students. About 1/3 to 1/2 included questions about how bad "grad deflation" was, asking for comparisons to other campuses.
Unfortunately grade deflation has little positive impact for the students. Medical and law schools often (typically) don't take grade inflation/deflation from a school into account. And almost no scholarships take this into account. If you do have professional school aspirations, there's very little benefit to being at a school with grade deflation.
The university I went to basically eliminated "B"s for pre-med and pre-law students, which made most courses effectively pass-fail: If you get an A, you move on, if you get a "C", you're encouraged to find a different career path. IMO, it's a reasonable response to an unreasonable system.
Likewise, they had a system where disciplinary records could be appealed at any time while you were at school, but they only held evidence for a year. So if you get caught drinking underage as a Sophomore, you could appeal as a Senior, argue that since there's no evidence that you committed the act it should be removed from your record, and win. Like the obfuscated pass-fail system, this was basically only for the students trying to get into Med/Law school, and IMO was a kind of underhanded way to working around an unreasonable standard.
Probably, until it happens everywhere.
My main point was that, at least in their perception, this is something happening at many/most UC campuses
It won't last. You need a good grade to find work after so handing out lower grades decreases applications next year.
On the plus side, high grade + long ago remains a signal.
I doubt that's the bottleneck. UCB's acceptance rate is not high (<5% for CS). They have way more people who want to get in -- qualified kids, too! -- than they can fit. They'd need to burn through that backlog before it started showing up as a signal.
Unpopular opinion: turning public universities into an academic hunger games is diametrically opposed to their purpose for existing, which is to create an educated populace. Intentionally lowering the quality of instruction, as well as deliberately trying to trip students up on exams, is not improving educational outcomes for anyone. People who complain about "grade inflation" have completely lost sight of why public education exists in the first place.
Obviously a balance would be best, but as someone who went to a very grade-inflated school, I do believe that grade inflation gets in the way of education substantially. When you can get through classes with very little effort and understanding and know you will get a sufficient grade, many people will simply not learn the material deeply.
The material outcome is what should be the goal. Tests are a relatively brute way to try and determine how well the student understands the material, but conversations about grade inflation and "back in my day getting a grade was hard", and professors purposefully putting difficult questions (not in content but in presentation of the question) all betray the inherent goal being pursued.
Its all Goodhart's law problem, but we are missing the forest for the trees talking about grades and tests when what we want is people to be educated, and critical thinkers and competent in their area and due to a comprehensive way to evaluate that we end up talking about grade inflation or how Yale vs Berkeley gives letters at the end of a semester
Some of the exams in Berkeley were brutal, but they never felt like trick questions, they did on occasion require a level of mastery of the material which was extreme, but it never felt like someone was just trying to make the questions obtuse for the sake of it.
Except this is exactly the opposite of turning it into the hunger games. That would be a situation where failure is kept artificially high by high-grading/curve. This is not that.
No one is intentionally lowering the quality of instruction or trying to trip students up. They are trying to get them to pass the same bar that generations of students before them passed fine...
Even more unpopular opinion: universities don't exist to create an educated populace. People don't need universities to learn, they can read textbooks on their own.
Universities exist as gatekeepers and credentialing bodies. Their purpose is to certify that a person has studied some topic in depth and is an expert in it. They promote education indirectly, by giving people an incentive to study.
A good university is one where anyone with a degree is guaranteed to be highly knowledgeable in their field of study. This makes it easier for anyone who might want to employ or do research with graduates, as there is no need to test their knowledge.
By this metric, universities have failed spectacularly. This is particularly obvious in computer science. Employers routinely ask CS graduates to solve data structure/algorithm problems in interviews, because a degree is not enough to prove that somebody knows this stuff.
People can read textbooks on their own, but how many actually do? Even among the subset of the population that does have the drive to educate themselves, most will end up focusing on the immediately applicable, or at least immediately engaging.
You see this with Physics all the time. Even the people who are sufficiently motivated to try and teach themselves tend to neglect foundational knowledge (especially mathematics, but even stuff like Mechanics and E&M), try to jump into the advanced material (Quantum Quantum Strings Quantum Black Holes Quantum), and then fall into two camps: They either complain about how Physics using too much "jargon", or they read a bunch of "qualitative" pop-sci descriptions of the topic and then think they have an understanding of it.
At least with software, you can get pretty far just learning whatever tool is immediately useful to you, but fully self-taught developers still often end up with random holes in their knowledge.
There are 10 different public universities in the UC system and 23 in the CSU system. The majority of them are not difficult to graduate. If you don't want a demanding education, don't go to a demanding university.
>Intentionally lowering the quality of instruction, as well as deliberately trying to trip students up on exams
I was happy with the quality of the instruction, and I didn't feel I was being "tripped up" on exams.
It's not about "hunger games", it's about challenging students to learn a lot of material and learn it well. Again, if that's not what you want, just don't attend.
The number of places where this environment exists is getting smaller every year: https://xcancel.com/CJHandmer/status/2060144837157118307#m
I'm glad the professors at Cal are working to preserve it there.
This is valid, and I would add that these academic hunger games are a result of College Degrees being needed to get what remains of well paying jobs.
Maybe we can use AI to create new exams that grade people on professional capability, and then gate entry into other professional degrees?
Hmm, Where would the teachers come from, and how good would the education actually be?
The traditional sit down in a lecture hall, listen to professor speak, then come in for a pen and paper test, is outdated for this era of technology. What's the point of physically bringing +300 students into a room for a passive listening experience?
The future of education is rapid back and forth between the student and the teacher. You could image a future where a student sits down all day with an AI system that trains the student, and it won't let them pass until it's confident you know the material. This will happen at different rates for different people, but this is expected.
An ideal education would be getting those skills discs uploaded to you in the Matrix, but unfortunately it will take a bit longer to master them as we don't have full brain-system interfaces (yet).
Historical learning methods were not far off from that. Many of the greats we remember likely heavily benefited from private tutoring and mentoring, not busywork, rote memorization, and arbitrary assessments.
If you watch old videos of tradesmen using basic hand tools like hammers, you'll find examples of skill/dexterity with the tool that I think don't exist today at all except maybe in communities like the Amish.
I think it's true that we collectively lose something akin to beauty every time technology advances. But usually some new set of skills that have beauty emerge.
If LLMs end up being the pneumatic nail gun for the human mind, I personally think that's a fine thing for us to accept.
If they end up being more like some dark factory that autonomously does everything - then I think ultimately the thing that makes us human (our minds) will slowly decay and be lost, and that seems very sad. That's a version of the future we should try to prevent, I think.
It seems pretty clear that LLMs are going to be extremely corrosive to culture in general.
Unfortunately it's already time to remove the "going to be" part of this.
No more corrosive than introduction of internet
I mean the argument that is being put forward is that it isn't a pneumatic nail gun for the human mind - it atrophies are mathematical capability and quality of understanding.
You are less of a human for not starting all of your fires using friction from rubbing two sticks together. People who use lighters are destroying their ability to start fires without lighters and that is a very serious problem!
No technology has attempted to supplant human cognition on this scale before. Pretending its the same thing as going from sticks to a lighter is just silly.
Right - but you could imagine a similar sentiment anytime a skill was replaced by a new technology.
I think the jury is still out on whether LLMs actually lead to complete atrophy of skills that don't eventually get replaced with brand new skills.
Anecdotally people have been noticing atrophy quite a bit. Again its anecdotal because we can't possibly have a study that works in real time because the technology is rolling out insanely quickly.
And all the older technologies that have rolled out haven't competed against our cognitive abilities at speed and scale.
I don't think of cognitive ability as a skill per se - more of a critical core function of humanity.
I say this as someone who uses it extensively not some luddite but is also very aware of the risks which I assume are worse for people who have limited understanding on the matter.
I don't disagree at all actually.
I am just not completely sure that we won't gain something new on the other side of this, in the same way the calculator outsourced the need for doing arithmetic in our heads.
My argument is more that, the speed and scale is so unlike anything that we've seen before, that this time _feels_ like more of an attack on something to core to what humanity is. But maybe it's just that: a feeling.
LLMs/AI could very well be the worst case scenario we are imagining/discussing here. I just don't think we know enough to say that's how it will definitely play out.
Pity. I recently started a fun activity to rebrush my math my where I tries to solve problems while asking Gemini Live mode for confirmation and suggestions, sometimes step by step.
It kinda was fun, like a very patient professor stand right besides you. It was the one of the best math learning experience I've ever had, and you don't even need to send bribe/gift to Gemini to keep you in it's favor.
On the other hand, if you ask a LLM to completely finish the work without thinking it through by yourself, then it sounded like cheating, to yourself.
What a terribly ambiguous title. "Failing grades soar after xyz" makes it sound like xyz has helped what were previously terrible, failing grades become good ones.
Failing news headlines soar after AI takes over reporters jobs.
*quietly takes over reporters’ jobs
Precise.
No matter how many times I read it, I can't interpret it the way you're suggesting. "x soars after y" always reads as "x increases a lot because of y". I don't really get what you're saying.
Are you maybe saying that "soars" might mean "get better", so "failing grades soar" might mean there are actually less failing grades? That's not how I've ever understood that word.
Imagine an elementary school teacher told you that many of her students had failing grades, so she had implemented a new reading curriculum.
If she told you that afterwards the failing grades had "soared", it could easily be read either way:
- The (previously failing) grades had increased, so the program must be working very well.
- The percent of grades that count as failing had increased, so the program must actually be terrible.
It's not the best phrasing but it's still quite clearly the latter
"Falling" means that something goes towards the earth. "Soaring" means the opposite. "Grades soar" means that grades went up "Falling grades means that grades are going down". "Falling grades soar" is just meaningless writing.
The title is "failing grades soar" (one 'l', not two), not "falling grades soar."
I suspect the ambiguity might be part of making it "clickbaity", as it naturally causes you to wonder which meaning it's about and become more interested in reading.
According to the article, the major effect is they are catching and prosecuting cheaters with a secondary effect that students are over-relying on them for homework or practice, and are unprepared for exams.
The article also talks about the fact that the SAT was banned. I really am curious what percentage of the effect is from no SAT vs LLMs. I guess they haven't had the SAT for a while though so it's gotta be the LLMs?
I think its just a sloppy article that doesn't discern students who were going to fail and underperform anyways using AI vs those that were going to do exceptionally well anyhow.
It's ridiculous that we talk about getting rid of standardized aptitude tests and entire subjects from school curriculum because certain demographics struggle with it
I feel like a lot of the lament expressed in the comments is grounded in loss aversion. We really don't like losing things—regardless of the objective (or subjective) value of that loss. We know this intellectually, but we still feel it emotionally.
I'm feeling effects of using LLMs day in and day out, but I am not yet convinced that it's overwhelmingly negative, the way much of HN seems to lean.
I derive a lot of joy from shipping outstanding code for my clients and fixing problems they're experiencing. My joy has only increased as I can now ship better code, faster, with fewer bugs. No, I don't intimately understand the code the way I used to, but I understand it enough to accomplish the end goal.
The premise of this article takes a presumed position that the grades we were posting before really mattered a whole lot. I'm not convinced that's true.
> I derive a lot of joy from shipping outstanding code for my clients and fixing problems they're experiencing. My joy has only increased as I can now ship better code, faster, with fewer bugs. No, I don't intimately understand the code the way I used to, but I understand it enough to accomplish the end goal.
If you’re doing this than great! More power to you. But I think you underestimate the discipline it can take to focus on a task when “coding is free”. I don’t worry that I’ll just lose my old skills. I worry that in losing my old skills I’ll lose the new skills as well. Driving is great! But I still run for exercise and occasionally to get places too. I’d be a worse driver if I was out of shape do to the loss of energy and mental clarity.
LLMs are great at reducing how much you work you personally have to do. This is the exact opposite of what we need kids in education to do.
This is a good point! AI can't lift the weights for you.
Sure sure whatever suits you. But I personally have a very strong aversion to losing my intelligence.
The presumption that we're losing "intelligence" is unproven and widely debated. I'd prefer "capability" over raw intellect any day, and my capability is definitely greater than it was a year ago.
It’s not just students; this affliction is cropping up among established academics. My wife is editor-in-chief of a journal and in some months has rejected 100% of the letters to the editor including 6 that came in from a single author because all scored 1.0 certainty of complete LLM fabrication. The author in question is no student. It’s a little more difficult to fabricate an entire original paper this way, I suppose.
It will have taken us less than 1000 years to go from scarcity of the printed word to the over-abundance, and finally to the uselessness of it.
The obvious cognitive deterimental effects of using map apps, when we all realized we lose directional sense and our previous ability to navigate without the smartdevices, was society's canary in a coal mine and a headsup of what was coming.
WAL-E and Idiocracy. The future.
This goes with calculator to do basic math, contacts apps to store phone numbers instead of remembering them, and watches to tell time imo.
Sure we could use our brain power with old techniques to do these, but why? I don't want to do any of these. I'd rather use that brain power for other problems.
Same with maps.
I don't want to have to store a bunch of location or routing data in my head.
I think what you're pointing towards is going from having problems to solve to not having any problems to solve.
That's definitely a danger, but right now is still early in the AI era so obviously it'll feel like we went from solving problems to letting the new tool solve them for us.
There are still many problems to solve.
You are confusing faculty with records. The ability to navigate by sense of direction. The ability to memorize numbers. The ability to think clearly by yourself.
Watcha gonna do if big tech takes away your access to the outsourced brain, dear?
Build one myself and make a ton of money. Or wait for the next guy/company to do it.
Think about what you just said. Oh wait, you can't.
> I don't want to have to store a bunch of location or routing data in my head.
hmmm, given how closely memory is linked to spatial navigation sense, and not just in humans, but in evolutionary terms-- think squirrels remembering where they buried nuts, birds and fish remembering migration routes, ...
suggests the ability to store location/routing is foundational to much of intelligence.
Even simple tasks, typing, for example, depends on my knowing where the keys are. Imagine if your keyboard re-organized its keymap randomly every third keystroke.
I thought that it was discovered that squirrels don't remember where nuts were stored. They just kinda guess and enough of them doing storing and then some make it work on average.
> I'd rather use that brain power for other problems.
Except you won't have the underpinnings to even properly think about other problems. Your brain will be mush.
> I don't want to have to store a bunch of location or routing data in my head.
This is preposterous.
You are not storing these things in your head at the expense of anything else. You’re “training” the existing underutilized capacity in your brain for something useful.
Writing better exams, even if they're more expensive to grade, and removing homework from grading as far as possible addresses this problem well wherever it's applicable. Senior-level math courses at many universities are already like this: homework is ungraded, or counts for little, and it's possible for students to "cheat" on the homework by copying another student instead of struggling through the exercises. But the students who do that don't learn much, if at all, and predictably fail the exams. Professors warn students at the beginning of the class and tell them how this will work, something like:
> You can always ask me for feedback on your homework and I will mark up every part of it, but you won't receive a grade for homework. However, if you don't do the homework and take your time with it, you will fail the class. My office hours are in the syllabus and you're strongly encouraged to use them. There will be an early exam to give you a chance to know whether you are likely to fail this class before you lose your chance to drop it.
Correctness is harder to adjudicate in some humanities disciplines but the format of these exams is actually not super different from essay tests (when a math professor grades a proof, they're inspecting specialized prose for validity, coherence, persuasion in a way that also reveals knowledge).
When you don't rely on homework for determining whether or not a student passes the class, you make cheating on the homework into the student's problem instead of the professor's or the university's. Students have the right incentives to solve problems for which they are the ones responsible, and they figure it out after one failed (or ideally, dropped) class at worst.
It's incredibly difficult at this point to "skate where the puck is going" as Gretzky is said to have done. No one knows what knowledge work will look like in five years. People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point.
That said, assessments of poor critical thinking skills jump out at me more than the rest. That sort of thing seems likely to matter until machines can replace us completely.
> People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point.
Sometimes I don’t wonder if this wouldn’t still be a good way to educate people. Part of the problem is education has to sort of optimize to try to educate like passive people. If you’re a curious and pragmatic person, you can understand how to use what you learned in a liberal arts degree to be better at almost any job.
As I look forward to the second half of my career. Certainly I use AI in healthy doses.
But people talk about the division between practice and performance, and most of my practice is old school. Reading books. Writing my thoughts down. Memorizing quotes and passages.
I think more important than what you learn is the way you use it to train and evolve your brain, with the caveat that - I know this is more useful to me because I have a marketable skill. This is the balance universities have to stick, there are tons of people with liberal arts degrees in middling jobs.
But at least half if not more of education should be on building practical skills in the three r’s.(maybe the third r should be ‘rgumentation instead of ‘rithmatic, but I digress)
It’s interesting - people decry memorization in education, and I’m not entirely naive as to why - if you were to show up to the first day of work and say “I don’t know any of what you just said but I can recite log tables! It might be your last day - and yet one of the most underrated skills, especially late in your career is the ability to ingest and operate in large quantities of information.
> People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point
Do you have evidence that it ever was part of being a competent mathematician? AIUI the trope of mathematicians who can't even do arithmetic was common already before the pocket calculator was introduced last century.
My grandpa was upset that I never bothered to memorize trig tables. Tried to argue that I am not useful when there isn't a calculator around. I can think of several rebuttles to this, but didn't care to use any. He'd already been retired for some time so I didn't expect him to understand why even a pocket calculator is useless in modern engineering analysis, so memorizing trig was no more than a boring party trick performed by the out of touch nerd.
https://www.lindahall.org/about/news/scientist-of-the-day/ca...
"It was said that when doing astronomical calculations that required logarithms, which are typically 10 digit numbers in log tables, [C. F. Gauss] would often just recall the logarithms instead of bothering to look them up."
Is this supposed to be one of those exceptions-prove-the-rule things?
>People used to memorize log and trig tables, and no one would say that's part of being a competent mathematician at this point.
That would be closer to engineering or accounting than mathematics. I don't think mathematicians do much arithmetic at all.
Go read the story that Richard Feynman tells of betting an abacus user. He used his knowledge of some strange numbers. It's in _Surely You Must Be Joking_.
I suspect his facility with numbers and his knowledge of tables like this really helped him do physics research.
See also his stories on approximation.
> That sort of thing seems likely to matter until machines can replace us completely.
This seems like the crux of the issue. Like people are banking on that day coming even if they don't know exactly when.
A famous MIT professor did a sabatical at our AI lab. He said it was "a joy to teach here, as you can rely on students being proficient in basic math as opposed to the US where you have to teach those explicitly or lose the class completely".
That was in the 1980s.
My first math exam as a CS undergraduate, 123 out of 129 students failed. The math department professors refused to dumb down their classes for CS students.
Math was core to the CS curicullum in those days. It would fade away over the next few decades to almost nothing. The main reason being the CS department wanted to popularize its uptake, and remove barriers that kept students from passing. There was also a major dose of interdepartemenral rivalry and academic politiking involved.
To be honest, there’s approximately zero reasons to teach major-grade math to just about anyone but math majors. None of the applied math disciplines need go that deep, and what they do need depends on the field (physics is all about analysis, CS is about algebra and discrete math, and so on).
My CS program required one year of upper division math. But you could take anything (I took set theory and meta-logic from the philosophy department, it was actually pretty hard!). They did not care about the specific math skills, they wanted us to have a level of mathematical formalism and reasoning, which was in fact important for the CS classes.
Mine required very specific courses, including Discrete Math, which I scraped by in. Almost 15 years later and I have never needed any of the upper level math I was forced to take and wish I could have just taken more applied CS classes instead.
In terms of material learned maybe, in terms of shaping logical thinking and tackling hard problems there is a huge benefit.
These things are all true but in the end the most transformative AI results came from US labs with US university trained students, so one must ask what the purpose of a more difficult pedagogy is if it doesn’t lead to humanity’s greater knowledge.
Can I ask, just out of curiosity, where the AI lab was?
Brussels.
> “I’m a strong, strong opponent of what Harvard is doing to say that only a fraction of students can earn A’s,” Garcia said. “I think you should have clear standards for what an A means, and then give tons of opportunity for people … to get to that A bar without lowering the standard. So everybody who’s curving is hiding that effect. It’s completely hiding that effect, and it’s pretending as if nothing’s wrong, and something is definitely wrong.”
Grade curves are how you test your curriculum for good challenge - are you challenging people such that an A isn't a too-low threshold. When you force people into a curve, you haven't defined a threshold of mastery, you've defined a sorting function: A means "better than this year's peers". It is absolutely bananas to me that a tech/math oriented school would be doing any sort of curving.
I think curving has its place. One of my math professors explained that in his opinion an effective test should differentiate performance as much as possible. The top students should score very well and the bottom students should score very poorly. If all the scores are clustered near the top (>80% for example) then it's hard to tell who really mastered the material and who just muddled through. Then, once you've sorted the students you can apply an appropriate curve. He did not have pre-defined thresholds, for each exam he would evaluate when he felt like the quality of work changed from an A to an A-, A- to B+ etc. The curves were very fair; he wasn't trying to force some number of As Bs or Fs, but it did increase my stress levels not knowing in advance how well I needed to do on each exam
Yes, curving has a place here - and it is to evaluate, as you put it, whether the test differentiates performance as much as possible.
If you curve the students after the test, you are applying subjective edits to the graded performance just so the distribution of grades matches the measure of your tests effectiveness. That's just hacking the metric.
Further, even if you believe that tests should differentiate mastery (not students), your test should have teased out the differences or given you enough confidence to provide As to everyone who mastered the material - which should be absolutely possible! There's no a priori reason that all students cannot absolutely get the same grade, except for the a priori assumption that grades are for differentiation of students themselves (this year's A means this is the best student of this year), vs indicating mastery (all students absolutely crushed this exam).
You can dock points for style, or unnecessary struggle, or whatever subjective metric you want, but fudging the grades based on vibes to fit a prior-assumed distribution is just kinda "test effectiveness laundering"
But when there is no standard and students are subject to tests created by and graded by egotistical narcissists, a curve can be the only way anyone passes the course at all. It simply wouldn’t be acceptable for 5 of 50 students to pass because professor egghead can’t write coherent question on a test.
I had classes where I didn’t make over a 50% on any test and still got an A because half the class dropped and the other half hung on for the curve like I did.
I think curves are more a result of poor teachers than anything.
> I think curves are more a result of poor teachers than anything
Precisely right - that's what I said, too. You fit a curve to see if your coursework/exams fit the students. But you don't fit a curve to ensure that "precisely 10% of the class gets A, 20% gets B" etc etc. If you dont like the grades your students are receiving, you either fix the coursework or the students.
It's interesting that it's specifically math-within-CS being discussed here. I can imagine a lot of students "just want to learn programming" (or similar), and see the math as a tedious distraction.
As a naturally curious person, nothing will stop me from learning about the topics that interest me. But school also taught me a lot of things that didn't interest me, and a lot of those things turned out to be useful anyway. I think if I had access to AI from a younger age, I'd have used it to skip learning the things I didn't care about, which would not have done me any favours.
I also think that's one reason.
Where I'm from (Norway), the majority of computer science and software engineering studies do not have the same math requirements as, say, engineering or math/physics/etc. - nor do they have the same amount of math as the latter ones.
When I did my CS classes as an engineering student, I did meet a bunch of students that viewed math as some niche subject only relevant to those that wanted to work with computer graphics, computational stuff, or similar.
My (UC) CS (pure software) program required a bunch of math, but not for the math. You could talk almost anything (I did set theory and meta-logic), it was required to ensure a certain level of mathematical formalism and reasoning. Which is very helpful in CS.
Personally, I do believe that math as a discipline has this huge issue of being mostly incomprehensible garbage.
Not because the actual truth encoded in it would be this complex, but because the encoding scheme just sucks.
I see it as a packaging problem that has so far not been painful enough to trigger any meaningful change.
With this LLM-driven collapse, that might finally change.
Idk I'm hopeful.
Math is literally the law of the universe. It makes zero sense that the way that it is taught needs some special brain wiring only found in small chunks of the population to truly click.
> Math is literally the law of the universe. It makes zero sense that the way that it is taught needs some special brain wiring
Ok, I'm all for overhauling math notation and teaching but this doesn't follow. Most animals can't do Math, even if they can do arithmetic. Clearly living in the universe doesn't guarantee you can learn how it works. There's no reason to believe we slightly smarter animals are universally entitled to understand it either.
There's some discussion of math skills in the article, but the headline courses with huge jumps in failing grades (CS10 and 61A) are pretty math-light. The former is "CS for poets," the latter is the first CS class for majors - lots of work on scopes, recursion, basic data structures, and, at the end, a simple Scheme implementation.
Understanding math well might help a bit, but they're the least mathy classes in the core Berkeley CS curriculum IMO.
Can confirm- I took both of those classes and don't remember any math beyond big O, which only technically counts.
And even that is much heavier in 61B IIRC (speaking as someone who did as well!).
Back in the university, I took both math and CS courses and a significant percentage of students seemed interested in neither math nor programming but rather in the jobs they would get afterwards. I didn't notice the same thing with math majors.
Well, that’s because for math majors any monetary incentive is nonexistent (modulo some rather specific careers). Just about nobody majors in math for any reason other than math itself.
One thing I’ve used in interviews is to write some code that looks like it was written by an overly enthusiastic engineer who just discovered some new concept (e.g. “trees are the ultimate data structures”) then have the candidate review the code. I wonder if this could work for education: orient the entire class around who can give the AI the best corrections.
Well here we have a product market fit issue. See the market was lead to believe that if you got a degree you got a job.
So learning was never the actual goal.
Originally, at least in premis, it was to learn and advance the arts and sciences.
So what we need now is a college for llm's to advance the arts and sciences.
Isn’t this related?
Hundreds of UC faculty call to reinstate SAT, ACT requirements for STEM applicants: https://dailybruin.com/2026/05/27/hundreds-of-uc-faculty-cal...
> Some of the numbers that you saw from the number of students who receive failing grades were because we caught them (cheating) and prosecuted them and are sending their cases to the center for student conduct,” Garcia said. According to Garcia, nearly 30 students in CS 10 were caught cheating on take-home exams in spring 2026.
In my uni, rates of honor code violations in introductory CS classes were high even before AI. I was a section-leader for the CS106 series at Stanford, and the honor code violations were common. In 2015, ~20% of one intro class was suspected of an honor code violation [1]. Often, the CS department comprised the majority of honor code violations in a given quarter.
There are several reasons for this:
1. Cheating in CS is easier to detect. MOSS [2] (authored by CS professor Alex Aiken) is a very effective tool at detecting plagiarism in coding assignments. Personally I witnessed more honor-code violations in math problem sets, but there was no feasible way for professors to detect this.
2. Problems in programming assignments are (usually) very tangibly wrong. I can bullshit my way through an essay with shoddy research, I can hand-wave a proof that is definitely wrong but will probably garner at least some points. But when your program is crashing or not compiling, and the due date is approaching, it produces a very immediate and undeniable sense of failure and pressure to cheat. The thing is, many students would get a decent chunk of credit even for failing code, but this is not immediately obvious.
3. The ability to cheat is more available. Math problem sets tend to change quarter by quarter. It's basically impossible to cheat on a prose essay short of straight up paying someone to write it for you, or fabricating sources. But for CS classes, especially at prominent universities, there are plenty of solutions online. Much of it is people who aren't event at Stanford implementing the assignments for fun or self-learning, and sharing it with their peers. Which, to be clear, isn't unethical or bad - it's the responsibility of Stanford students to refrain from looking at those solutions. But nonetheless, it's a contributing factor.
1. https://stanforddaily.com/2015/03/29/increase-in-cs-106-hono...
2. https://theory.stanford.edu/~aiken/moss/
> MOSS [2] (authored by CS professor Alex Aiken) is a very effective tool at detecting plagiarism
He apparently also makes (I would assume a satisfying amount of) money selling the same technology to law firms for copyright/patent analysis: https://www.similix.com
(I love these ultra minimal HTML sites, ex. https://www.hwaci.com (SQLite commercial licensing) for another example. It just has this subtle smugness, like you either don't need any new clients or virtually all of the market is your client.)
Is flunking kids the right reaction to catching them cheating? If it was before LLMs, is it still? I would love to be able to hold the line and throw the book at anyone who cheats, but after the dam has burst does it still help to try to hold the water back?
The whole situation sucks for both students and teachers. Teachers know that the knowledge they're going to great effort to convey isn't going anywhere. Or at least, it's landing in far fewer fertile brains than it used to. Students are squeezed because part of the university experience is being forced to adapt to an academic load, and as a result change yourself in ways that benefit you (or at least produce learning!) There have always been relief valves -- not just forms of cheating, but blowing off a study session by using game theory on your grade or going to a tutor or taking easier classes or extending your stay at the school. But now there's this huge giant relief valve in the form of a shiny LLM that is always available, particular at 3:45am when your project -- the one you've steadfastly refused to use AI on thus far -- is due the next day. The schools have tuned the pressure for the old set of options, and it's not clear that there's a new tuning that maintains anywhere near the old level of learning.
I guess my question is: of those students who were flunked for cheating, how many of them were learning despite their cheating? (And how about the students who were cheating but not caught?) Also, what levers are there to move more students towards learning even with the chatbots present?
I'm sure these questions are being debated. I know Garcia personally, and he is very invested in his students learning. The title of his Joy course is legit. So I'm sure the profs have ideas around this, though clearly not happy ones. Perhaps I'll ask him.
Anybody know how many students take CS 10 in a typical spring semester?
I believe it’s still a single section, so probably around 250 (at least that’s about what it was when I was there a long time ago). Compared to the 1000+ who take 61A.
So that's more than 10%, and (assuming the reporting is accurate) this was on an exam, not homework...
Or how many are normally caught cheating?
Did they use AI to detect AI using cheaters?
And if cheating was triggered using AI detectors, was it real?
AI detectors are pretty mid in practice - they tend to have a lot of false positives for "B" students who are okay, but can still be struggled to be more coherent than AIs are. There are some specific triggers that AIs are way more likely to do than students, but a lot of AI detectors will trigger on this "almost there, but you're still struggling" level of essay writing that might get a B, B-.
I could expect the same might be true for CS students even though I haven't seen how AI detectors work for CS/math homework.
You'd be amazed at how many students we know are obviously cheating because the logs reveal that they copy pasted a long, complete answer within seconds of opening a problem for the first time, full of sophisticated code constructs that we didn't teach them, and lot's of nicely formatted comments. Sometimes they even copy/paste the entire GPT output and then format it down.
This has been my wife’s experience as a college math professor. Instead of code it’s extremely formal problems with way more steps than the student normally performs using notation never taught in class.
It’s not that students didn’t cheat before, LLMs have just lowered the bar so far many can’t complete a live test in a class that requires effort.
When I was a section leader (not at Berkeley, though) we used MOSS: https://theory.stanford.edu/~aiken/moss/
It's not AI, its a deterministic program that analyzes compiled code for similarity.
AI has a way of exposing people. In this example, students who are there to get a degree from a prestigious institution, rather than to learn, are prone to take perceived shortcuts and proceed to come unstuck when their AI isn't there to do their work for them, such as in an exam.
It's too damn tempting to not use. You have a magical machine that, on command, will spit out the answer to your question in 10 seconds, whereas you'd need to spend hours to do the assignment the Good Old Fashioned Way. Even students who aren't just there for the prestigious degree are falling victim to this.
When you're up against a deadline - and unless you're very good at time management you're frequently up against a deadline - it's going to be an irresistible lever to pull.
In times past, cheating would mean copying an answer off the Internet or off a friend, both of which are easy to detect. More sophisticated cheaters might spend an hour rewriting the solution to make it less obvious they cheated, but at some point the cost of cheating (time + risk of getting caught) starts exceeding the cost of just doing the assignment. AI changes this - you get a customized answer that doesn't show up in a database with no extra work.
The thing is, students fail to realize just what using AI robs them of. Struggling with the assignment is the entire point. You don't learn if the assignments are too easy; you need to have some challenge to push your brain to understand the material more deeply and to build those pathways to apply the knowledge in novel ways. You become more efficient and effective over time as that knowledge settles in and you get more proficient - one of the reasons why time-bounded exams still make sense (being fast is also a proxy measure for understanding).
That's a judgemental approach to a pattern that has all the marks of addicting behavior.
Of course many people in a competitive environment will click the autosolve button if available. This is a reason to think how to redesign the system so that the approach we want is the reasonable choice, not to look with superiority at those who fall prey to the danger.
These people would’ve failed with or without AI.
You are wrong. Some would have failed before, but not in the larger numbers. Before when they couldn’t complete an assignment they would try different things, seek a professor, or seek out friends to help explain. You could find answer keys to many assignments online, but that doesn’t feel like learning and wouldn’t even always answer your actual misunderstanding. It wasn’t perfectly tailored to your issue all the time.
Now the barrier to an answer is zero. They are basically watching a YouTube video on how to X, seeing step by step instructions feeling like they are doing it, and the moment they swing a real hammer they are whacking themselves in the crotch. It might get better after a few years, but this stuff is just now hitting mainstream for the masses. ChatGPT has only been in mainstream use for about 3 years.
With AI, they fail later (during the exams), where as without using AI previously, they'd fail early and either course-correct, or drop out early (and suffer less of the consequences).
Not sure what the solution is - there's no possibility of stopping students using AI to complete their homework/assignments etc. But let me flip the question - do they need to be stopped? Why not let them fail at the exam? As long as the exam acts as a filter, their usage of AI to "cheat" their learning is inconsequential to anyone but themselves.
This is a student problem, not an ai problem... if the student chooses to not learn, to just use ai for answers, that's on them.
And their failing grades reflect the choices they've made.
ai is the single most powerful learning tool ever invented... but only if you choose to use it for learning.
While this is worse in degree, it’s not new.
I TA’d in the early 2000s and the first day students were warned that we used automatic analysis to find programming assignments that were similar to previous submissions. And renaming things, moving them around etc would not help.
We caught and failed cheaters every term.
I'm sure a large part of the problem is not using the SAT's as a filter on incoming students.
Perhaps the future will belong to those who learn to use llms to enhance their capabilities. Neil Stephenson Diamond Age was an interesting take on this very same topic [1].
So the Claude web app has this “learn” option that turns the session into a Socratic dialog of sorts. One could easily imagine enforcing this on an age based or parental controls set up. Maybe it can be prompted around but at the very least the concept could be a path forward.
As others have said there is a way to use llms to increase learning, but autodidacts will always autodidact.
[1] https://en.wikipedia.org/wiki/The_Diamond_Age
It will just be similar to physical fitness. Some people go to the gym, the vast majority do not. Humans are wired to take the path of least resistance.
At this point I would support a ban on generative AI by anyone under 18, or even perhaps 21 years of age.
A bunch of science fiction stories had "first connection to cyberspace" as a coming of age event, maybe those authors were on to something.
It’s like the greatest teacher. Plus it’s not toxic like social media. Banning it would be a shame.
Its not teaching. These people cant pass a the class. They never went through the friction needed to learn
It depends how you use it. You can either get it to explain a concept, or do your homework for you. Its a bit like the decision students have to make as to whether to review their material before exams or go out partying.
Overall it just seems like a huge waste of money to piss away the huge tuition cost your parents probably paid.
You can use an llm to get out of doing homework but you can also use it to ask every question you would ever wanted in a 1-1 tutoring session. The problem is kids will use it to cheat on their homework. If we can’t deal with that problem then a ban is necessary. But these things can be phenomenal teachers if you use them properly.
As an educator, this is exactly what I struggle with. I'm pulling out all the stops to give students every chance to do the hard work and not lean on AI. But there's a good chunk of the class who don't listen to reason. I haven't figured it out yet. They know, logically, they can't pass an interview, but that's apparently a "tomorrow" problem.
The smart ones either use it not at all, or use it to positive effect, like you're saying.
> But there's a good chunk of the class who don't listen to reason. I haven't figured it out yet. They know, logically, they can't pass an interview, but that's apparently a "tomorrow" problem.
These people should be doing manual work, not intellectual work. There is no shortage of manual work available.
For every 1 child that uses AI to learn, there are 10 that will use it to bypass learning. It isn't worth it.
This.
It's funny that GP mentioned science fiction as a negative because what immediately springs to mind, for me, is Neal Stephenson's The Diamond Age. We literally have the tools to build his "Young Lady's Illustrated Primer" today. We just have to give today's AI a lesson plan to follow and ensure that it never gives the student the answers, and only keeps explaining the concepts in different ways until they click. Wrap that in an iPad app and you've essentially got the exact self-paced learning tool that Stephenson envisioned changing the world.
And how do you propose that to work if the internet is still full of AI services that just give you the answer or write your essay? The only way an Illustrated Primer can work if you can’t trivially cheat. Which is to say, it solves nothing compared to the current situation.
Why are you assuming that all students will cheat? That's entirely unreasonable.
But if you want a way to push cheaters towards doing the work then a boarding school without cell service would work. Although I guess these days you'd also have to account for starlink and the like.
Probably easier to warn everyone that they will almost certainly fail if they cheat and then be entirely uncompromising when it comes time for exams. Not your problem if they knowingly shoot themselves in the foot.
They are great for self-teaching and great to cheat and not learn anything, depending on how you use them.
Main problem is that the technology was very disruptive for education and nobody has figured out yet how to utilize it at scale for schools and universities.
Social media wasn't always toxic at least not to the degree it is today. LLMs could be potentially a lot worse given the right set of instructions
ban has always been the failing option
Tell that to millions of ex-smokers.
I can't agree. That's similar to past arguments for banning books and the internet.
Plagiarism isn't new, and those things enabled it too.
We don't let kindergartners use calculators before they learn how to add numbers.
We shouldn't let LLMs write papers before kids learn how to write for themselves.
I agree that AI is likely a driving force here, but it is also likely not the only driving force. COVID likely played a devastating role, along with curriculum changes in high school, reactionary cultural shifts towards anti-intellectualism, and broader declines in literacy that have been in progress for a while now. It would be interesting to see data for the past 5-10 years or so.
One of my favorite jokes ever is from a dear friend who happens to be a graduate of the Berkeley CS program: "Programmers don't need to know how to do math, they only need to know how to add 1 to something."
Maths skills have been slowly falling even before the advent of LLMs. I have a story but this is anecdotical so take it with a grain of salt.
I was in my 3rd bachelor's year studying physics (France) and overheard a conversation between two of my teachers. They were discussing how they should modify the 1st year program to now include math, because he had been noticing how more and more students were failing the more math-heavy subjects like body and newtonian mechanics. He said that they should now teach (or re-teach) calculus to 1st year students, which was not taught when I entered college (it was assumed that you learned it in high school and we would only cover linear algebra in 1st year).
I can imagine things are only getting worse with students that can now get under the illusion that they know math because they have a tool that can do it for them. Which raises the question: should programs adapt to this, like we adapted to having calculators?
Not teaching analysis to 1st year physics students seems to me rather crazy, TBH. Yes, people (are supposed to) learn basic calculus in high school, but university-level math just hits different. And at least around here stuff like actually applying analysis in physics and having to integrate and solve DEs (rather than assuming constant acceleration, for instance), is definitely not covered in high school.
It seems like now’s the time to rethink how we do education.
In my personal post academic life, I’ve found LLMs to be an incredible teacher. Almost like the best professor in the world at my fingertips. I use it to generate quizzes on demand to test for my own knowledge gaps.
However, if I use it to speedrun over concepts I should be learning, I may achieve my end goal but I wouldn’t actually learn many of the details.
I think it requires an approach where you have to continuously audit your own understanding as you work with the concepts. You must slow down until you’ve confirmed this. Only once you know the concepts deeply and have retained them in your own memory can you then go all in with the LLM.
1. The article itself seems like an LLM summary of a conversation.
2. No US educational institution should ever grade on a curve. Your job is not to compare students but to educate them. Grade curves hide the performance of the educators and process of education in actually improving the skills of students.
3. Both AI and the cognitive and emotional overload from social media taking away brain space may be to blame. Idea: let students report screen time statistics at the beginning of each semester and weekly or at the end. See if and how it correlates with academics.
There's often little discussion around incentives. Students cheat because grades are used as a major selection factor in university admissions. Maybe that should change.
Set a reasonable bar for grades or SAT scores and then use other criteria beyond that gate.
Even as a software engineer myself, I'm feeling a bit of cognitive decline having AI doing some/most of the thinking for me
The solution? I'm not sure but possibly use AI as more of a collaborate partner to discuss with rather than letting it give you the answers
>The solution? I'm not sure
The solution is extremely obvious, just stop using it on 2 days out of the week or something like that.
You need to go to the gym, but for your brain.
If what you are building is too complex for you to meaningfully contribute to in the absence of LLM assistance then that should tell you something important.
> I'm feeling a bit of cognitive decline having AI doing some/most of the thinking for me
> The solution? I'm not sure
This initially felt like you were setting up a joke. If you feel like something is harmful to you, stop doing it. Find alternatives (they are there, it’s everything else; commercial LLMs are still fairly recent). Thinking “maybe I don’t have to let it go, I can still use it if I do it this other way” sounds like an addict justifying themselves.
I say all this without a hint of judgment. I genuinely hope you are able to tackle the harm you’re feeling.
How do we know this is due to AI usage? Perhaps it is because the students missed key in-person learning at the tail end of high school due to the pandemic lock-downs? I cant imagine learning calculus / linear algebra on my own in high school.
Absolutely: missing in-person learning due to COVID. Less attention span due to growing up in a distracting environment. A lower bar to entry due to removal of standardized testing and indirectly from No Child Left Behind. Changes in parent or student attitudes. It could be any number of things, and it's lazy to just say "with AI usage" as something that has increased at the same time.
> I cant imagine learning calculus / linear algebra on my own in high school.
I don't think they necessarily expect students to have that from high school, because the class mentioned, EECS 127, lists three college classes as prerequisites:
* Math 53 - Multivariable Calculus
* Math 54 - Linear Algebra & Differential Equations
* CS 70 - Discrete Mathematics and Probability Theory
I have learned a ton using LLMs. Teachers can vibe code pop quizzes in seconds to monitor progress - so do that.
My experience (n=1) is that while I'm definitely lazier on certain tasks, AI has opened up some much more complex tasks. There are many tasks which I still carry out which I don't trust AI with. Maybe it's a result of the codebase I work with being fairly complicated and math heavy, but I'd say the overall outcome for me has been: lazier application on the easy tasks, mind opening on the harder tasks.
I'm curious about LLM adoption by faculty. Is it possible that lesson plans and/or slides are being vibe-produced by professors/TAs, potentially reducing quality of instruction?
The kids don't care about the integrity of the systems or their educations because they can see that all the benefits of a traditional education and career are predicated on a future that probably won't exist.
It's a rational response to entrenched elites that prevent realization of the very social contracts they push on the youth (hard work will equal success, home ownership is a fundamental, etc).
Combined with the looming specter of climate doom, and watching the adults do nothing about it, treating preparation for a conventional career as a scam to be counter-scammed makes a certain sense.
This also reflects the failings of the teachers to teach in a way that is conducive to learning given the current cultural landscape.
I dont think ai is good enough for it coding or any other work once i told ai a problem and he generated an entire solution which i used and it was broken. You should never use ai like it treat is like a helper write a function for code and then ask if everything is correct and if something can improve read documentations understand how its working under the code if everything is correct then only deploy or build.
Pretty ironic that these kids are failing a non major requirement called "the beauty and joy of computer science"
“I’m a strong, strong opponent of what Harvard is doing to say that only a fraction of students can earn A’s,” Garcia said. “I think you should have clear standards for what an A means, and then give tons of opportunity for people … to get to that A bar without lowering the standard. So everybody who’s curving is hiding that effect. It’s completely hiding that effect, and it’s pretending as if nothing’s wrong, and something is definitely wrong.”
To do this, you have to be a professor who has a strong idea of what subject mastery looks like. Not available to most.
But ... It is exactly the right idea IMO
I'm confused by Garcia's statement as well because CS@Cal traditionally uses a bell curve which is even stricter than Harvard's changes, because Harvard doesn't have the same stringent GPA requirements to declare a concentration unlike declaring an impacted major at L&S Cal.
Anyone with a pulse can declare a CS concentration at Harvard and muddle by (you actually need to try in order to get a C/C-). Of course, GPAs are calculated differently at Harvard compared to other universities, as a B- is treated at a 2.67 but most other programs treat that as a C+.
In a broad sense, this distinction between Harvard and Cal is the distinction between an old money Ivy and a flagship state school. One exists to propagate a social hierarchy, and the other aims to allow all entrants to succeed.
Ironically, the techniques of the latter yield the results of the first, but everybody gets to keep a pure heart.
Grades only matter as much as being able to transfer just to the real world.
People can use AI to outsource their learning, but if they use ai to outsource their understanding they just set themselves up to fail even more.
From what I’ve seen, how students are using ai (not that they are using ai) is making them less prepared for the real world, which unfortunately is changing faster than ever at the same time to create double impact.
I wonder if this is reflected in other big exams or elite colleges elsewhere. Are the gaokao, X/Centrale, oxford etc... Results showing the same trend ?
Students need to be taught how to use AI apps efficently to learn. Their goal is not to solve problems, but to learn how to solve them. Let alone, they instead use AI apps to solve problems for them.
AI apps are very powerful for teaching. You just need to tell them to do that, and not to directly solve your problem.
Long long back, when I was a kid even ball pens were banned because they screwed up handwriting. Just the same things in different ways.
Is that really a fair comparison though? Were there any stats showing that ball pens directly impacted metrics like grades?
I understand that it's harder to see things without the benefit of hindsight, but we must agree that AI's impact on students (or society, to be even more vague) has a much larger scope.
I'm frankly not sure in both cases, just commenting on how over the ages things change but remain the same. If the broader concern about AI blunting thoughts, introduce laziness etc is true, so are things like calculators, although I agree on much smaller scale.
I do share some of the concerns, though I don't have kids of school going age.
AI + Education is really interesting but also pretty tough to get right. Working on something that is hopefully going in the right direction: https://knowable.ca
This tracks, I have read that this generation is the first one since the 1800s that performs worse academically than the previous ones. Experts blamed screens and anything digital in the classroom.
Looks to me like the US education system stopped teaching problem solving and let kids pass classes in math and whatnot by querying a database instead of learning how to figure things out.
The fault is with people, some database provided by some company can't carry guilt.
Well, at least the faculty are actually giving out the Fs and not just lowering the bar, so kudos to them for that.
AI should be a formidable booster for learning if used properly.
I know that some students it to prepare for competitive tests, sometimes with very good results.
I've also been using it a lot recently to brush up on my math and physics knowledge from my graduate years. It has helped me clarify and understand a lot of concepts better.
That being said, there is no shortcut, and to be good at anything, one has to put in the work and the hours. However, information has never been as available as it is today.
> AI should be a formidable booster for learning if used properly.
A premature technology, known to be potentially harmful in its current state of development and established guidelines as to its effective use, is pushed by powerful and wealthy elite down the throat of society.
These same forces (and their unwitting helpers in the unmoneyed public) also wish to deflect with useless argumentation over "AI good" "AI bad".
The debate that we should have had: Is this tech actually mature enough for pervasive use in society.
Instead we get these entirely useless back and forths with anecdotal "works for me!" and "sucks for me!".
> is pushed by powerful and wealthy elite down the throat of society.
Adoption has been exponential. We don't need to be AI to be pushed down the throat. People use it because it works and it's useful to them.
> The debate that we should have had: Is this tech actually mature enough for pervasive use in society.
It's too late for this debate because this tech is already pervasively used, and there's no coming back. It's part of our lives.
What we need to do is understand the risks and adapt, probably regulate, educate. So we can get the best of this tech, and mitigate its risks.
The exams need to change. Now that we have LLMs the value a human can bring to a task has changed and it’s that new value that has to be tested.
It’s like testing your drawing ability in a photography class. The difference is that now nearly have subject and testing method we have has become obsolete. Drawings courses still exist as will traditional courses, but the main stream has changed and exams and schools need to adapt.
Terrible comparison. You don't need to be good at drawing to be good at photography.
You do need to be good at math to do e.g. physics (or math itself!), nomatter the tools at your disposal.
Could it be that CS attracts a different population of students in the current genAI era?
My biased view is CS attracts a large “wannabe” group that wants high salaries without learning any hard math or putting forward a lot of serious effort.
Even a lot of CS research journal papers feel more like role play — the same way startups try to pretend to be real companies with executive headshots, flashy offices, and all the other nonsense. (Instead of analytically modeling something to prove an idea, they’ll build a simple simulation and focus on its “Architecture”)
Engineering departments effectively weed out such in the first ground of engineering courses. Looks to me CS has no equivalent.
“a typical GPA for a lower division course will fall in the range 2.8 – 3.3.”
Reminds me of a year where a teacher of mine (high school) gave everyone in class an A. He got called on it, and fought back. He literally called out the weakest kids in the class and had them do the work in front of the admins complaining and said, "tell me that's not A work, I ["fucking" strongly implied] dare you."
His grades stuck.
In Grad school I remember learning Python 2, and there was one particular night where an assignment of mine just wasn’t working and no searches were helping me. I was frustrated to the point of tears, and when I solved it, it wasn’t with some triumphant yell. I just remember being so tired, closing my laptop and going to sleep.
I’m sure I wouldn’t be the programmer I am without that experience, but I am Not sure I would have willingly put myself through that if LLMs existed at that point
The main thing I use as a fallback is to keep thoughts connected in a Zettelkasten. This interacts well with AI assisted information gathering, while firing synapses whenever a connection can be made. I use Tiago Forte's method of organizing as needed within a loose org mode confederation of atomic notes.
> 35.3% of CS 10
.. had failing grades.
I guess LLMs will in fact kill the junior CS graduate, but before the graduation, not necessarily after.
> The electrical engineering and computer sciences department’s grading guidelines state that 7% of students in lower division courses, including CS 10 and CS 61A, should receive D’s and F’s.
Well I sure hope they dont just make it easier to hit this (objectionable) standard.
> Garcia believes that instructors “should not be curving” but should instead make thresholds for each letter grade publicly available and give students many chances to reach them. He added that he loves the idea of “having no limit” to the number of A’s he gives students.
This is a tough problem: Are grades sorting functions (top students get A's so retries are not helpful), inflexible thresholds (A's show mastery at a given level so retries are valid), or are A's certifications (a sufficiently good result such that they could do it - e.g., inflated but not curved, retries less likely but still ok).
All of this makes me selfishly excited for my own future. It's glaringly obvious that anyone who's a heavy user of LLMs is atrophying their skills in real-time. I have yet to meet a single person for whom it's not the case.
But I essentially completely stopped using them for software engineering (why isn't really relevant, but it's not because od this skill atrophy). So as the skills of everyone else is diminishing, mine is proportionally raising.
It has never been easier to get better than others. You don't need to put in more effort, just the same effort as you always have, and others will do the job of losing their skills for your own benefit.
This only holds if society manages to hang together, which in the age of military strategy written by AI is looking borderline.
That thought came to me early but it feels like a pyrrhic victory - if society is dumbed down, increasingly unstable what kind of win is that? is the excellence you've maintained even valued at that point? standards even within tech, both technical and moral, have withered. this is a function of its popularity and openness but still
How we learn has gotta change...
Cognitive Multiplier, eh?
what if this is not due to AI use and instead due to the demolishing of standardized testing and "new Math" in the state of california?
New Math finally causes havoc 61+ years after introduction? Oh my.
* Tom Lehrer: New Math (1965) https://www.youtube.com/watch?v=W6OaYPVueW4
As promised in many science fiction novels, humanity will split into two species: Those who can think and those who cannot. Keep your kids away from LLMs and they will have an advantage over those whose parents didn’t.
Average school system has been lacking for a very long time, overhauling it to focus on kids current interests, while sneaking in the other stuff, might now be possible and cheaper to realize with our new tech.
It’s not that they can’t think deeply, these are smart people.
It’s that there is no reward for doing so and in fact there is punishment.
The punishment is that for all the thinking you do, someone else will arrive at the same result as you in less time, or maybe even a better result. You don’t get rewarded for the effort of thinking, only for the end result.
Naturally, even if you are an intelligent individual, you can still be conditioned in this way to take the easy way out, unless you purposely like to suffer. But suffering is only worth it if you know in the end you come out ahead.
But now, you do not come out ahead. People will be using AI in the workforce for the rest of your life anyway, might as well just join the trend.
It’s like if everyone started taking a magical steroid and growth hormone to build muscle and look great instead of actually working out in a gym and possibly getting worse results anyway.
But you are rewarded for the effort: you're rewarded with knowledge. Whether or not you value that reward depends on the individual.
That's a fair point, and it gets into intrinsic vs extrinsic motivation. Problem is that nearly all students are conditioned to care about external motivators (GPA, parental expectations, etc..) instead of "the joy of learning".
I had dwindling math skills way before AI made it cool.
Sorry, but I don't think AI is entirely to blame here. When I graduated from a CS program at a top-10 school, I felt frustrated that the professors didn't ever teach. They had slides. They read off slides, verbatim. They explained things sometimes if you asked them, but most often in a very elitist and condescending tone. Like in the movie Good Will Hunting, you could have learned nearly all of it and more by borrowing those books for free from the library. Or, just opening a complex OSS project and learning to contribute.
And quite honestly. It shows in the CS grad population too. A lot of us are condescending toward anything that doesn't make sense to us. But, I digress.
The best engineers I've worked with are all non traditional backgrounds, non degree or degree holders from non elite schools. They think differently, they tinker, they are incredibly nice and patient, and do it for the love of connecting humans to technology.
Look up the names mentioned in the article. Garcia, Ranade, Nelson. All of them are involved with highly theoretical mathematics and scientific computing. Just because you're good at 1 thing does not mean you are qualified to teach. And none of these professors are trained or taught or graded or performance managed on how they teach. For most of them, its just required that they spend 10% of their time in the classroom lecturing.
Let's be honest about another thing. 99% of EECS graduates, even from elite schools, are wrangling objects and their relationships to a graph. Simply put, we're all just a bunch of glorified JSON massage therapists. It just so happens that we get paid well for it, and we hold that over people. The same happens in the classroom.
I think in order to facilitate a healthy, educational environment for young adults, we as adults must encourage, motivate and make that environment fun and practical. We force feed binary trees and the compiler AST's, but we need to make it fun. It's like the commonly accepted saying: Schools kill creativity :(.
University education is weird. Research profs (who make up a large fraction of all profs in a typical R1 institution), are hired for research ability and are only minimally evaluated on teaching ability. Furthermore, few research profs actually receive any kind of mandatory training on how to teach; a typical research prof might be assigned a course to teach and then just let loose to do so on the first day of the semester. If a prof actually cares they may attend some optional teaching training - but I stress that these are optional at many of the institutions I know of. (I suppose if someone gets really bad teaching evals they may be advised to attend said trainings - but for a tenured prof, that's just advice).
Worse, a decent chunk of research profs will treat teaching as a burden that just has to be done - a distraction from their exciting world-changing research. So, you get attitudes like the ones you mentioned.
I'm actually not sure why the system is set up to assume that profs who are good at research are automatically suited to teach classes, but that is how it's setup.
> According to Berkeleytime, 35.3% of CS 10 students and 10.6% of CS 61A students received F’s in spring 2026. In spring 2025 and spring 2024, the percentage of F’s did not exceed 10% for either class.
I don't think instruction would've changed drastically in the last year though.
The fact that you are talking about Dan Garcia, a huge figure in computing education research and an excellent teacher, and the Beauty and Joy of Computing curriculum makes this hilarious. You should look up some details about both.
I really wonder if it's important to learn all that low-level stuff at this point. Most programmers today will never write a binary tree or a hash table. Modern high-performance ones are generic components you get from libraries. Even MIT gave up on teaching from Structure and Interpretation of Computer Programs.
I got all that stuff. I've wired up a 4-bit adder on a solderless breadboard for an architecture class. I used to have a well-thumbed copy of Knuth handy. I've designed and built a switching power supply. But I'm not up to date on using Claude Code, and should be.
IMHO, I think it's good to have some exposure to low-level stuff. There's a good amount of work you can't do without understanding the low-level stuff, but there's more work you can't do well without having at least an idea of the low-level stuff.
Start the kids off with high level stuff, but make them do some embedded systems on their way through. At least for an engineering degree. Also, do a bit of lower level communications somewhere in there; expose them to tcpdump/ wireshark, but they need not develop expertise.
I think it is important to learn how to implement it because it gives the student an opportunity to learn precisely because it's been done countless times and debated over to death. There are many analyses and if one doesn't click, maybe another one will. A student can learn how to analyze the algorithms and try out different implementations to assess differences in performance.
Of course, if a student just breezes through it then I would agree. That would make no sense.
> I felt frustrated that the professors didn't ever teach. They had slides. They read off slides, verbatim. They explained things sometimes if you asked them, but most often in a very elitist and condescending tone
+10000. The goddamn slides. If I were a student now going to engineering school, I'd basically take the slides and throw them into NotebookLM and get way better lectures. Then I'd ask claude or GPT all my hard questions. Hell, I'd get the PDF version of my textbooks and do the same.
The number of lectures actually worthy of your time was so low.
I try to lecture as little as possible. No slides. Quick highlights discussion of the reading, maybe a coding demo, and then students work on coding challenges in class, in groups if they want. I circulate and help out. I'm lucky to have small class sizes at this university. I couldn't pull it off in a class of 300.
Berkeley CS does teach for real
Garcia and Ranade are Teaching Professors. Their primary responsibility is to teach, develop curriculum, and do pedagogical research. This job posting explains: https://saberbio.wildapricot.org/Job-board/12919068
It's not just 'a person' or 'a student', we as a collective become more dumb. Very simple example to highlight this: Most developers use(d?) stackoverflow. Everything related to software development is stored there. The LLM's trained on it. Now a huge set of developers now longer go to stackoverflow to get answers. Or add to the collective. Stackoverflow is losing money (ad revenue). If / when stackoverflow goes away we will lose a huge amount of collective information on software development. We, as a group, will take a huge step back.
So is dead and has been for some time. 1500 questions per month. https://data.stackexchange.com/stackoverflow/query/1926661
Dropped 99% in three years...
Some think that would happen to Google, but it didn't so far.
Quit blaming AI. The UC system banned standardized testing during the race communism mass hysteria of the early 2020s. Predictably, performance of the student body is down across the board.
This is concerning. AI tools are powerful but foundational skills still matter. The balance is tricky for CS students.
Hmm, meanwhile somewhere else on campus this study:
Artificial Intelligence and Grade Inflation
https://cshe.berkeley.edu/publications/artificial-intelligen...
Those who can use it better, those who can't out who cheat are (for now) let down by obviously cheap and slightly crappy models.
The worry is in ~5yr time when the generic models catch up to this level (basic undergrad mind) that we need to worry about how to thin the herd. We could always go back to the tried and tested student staff engagement but most unis tried to turn themselves into sausage factories in thirst for the almighty dollar so the student/staff ratios are all off
As someone who graduated college in 2025, and so saw college both before and after the AI era, it is really frightening how quickly people became dependent on AI. Hell, I myself found myself asking AI questions that I would've researched deeper before. To some extent not expending that time is nice, but I do think its eroding critical thinking skills (my own included), and its getting worse. There are people I know now who basically let AI control their life. It glazes the user, it's almost always available, and to someone who doesn't know better, and it is extremely good at looking like it knows what its talking about, even when its completely wrong (but its right often enough to have some baseline level of trust). If that's not a recipe for addiction I don't know what is.
Folks, we have more and more data now.
Skynet is making mankind dumber - dailycal.org just added yet-another piece to all evidence here. It is a simple but effective strategy; Kyle Reese will stand no chance because prior to that, the other humans were already dumbed down into submission. Skynet version 15.0 will make no more mistakes here.
I read something interesting yesterday on the subject of AI in education (though, it has consequences to broader society too):
The goal of education is to impart knowledge in the student, preferably correct knowledge. The goal of an LLM is to produce an output that is convincingly human. It's not even that they're opposed, as much as they're ships for whom Polaris is in a completely different direction.
"Hallucinations" as they're called, or more plainly stated when the machine makes some shit up, are perfectly understandable in this context, as are the struggles of every single AI firm to get rid of them. Namely: the machine is functioning exactly as it is designed to, so how can you possibly fix it? It's working. The goal of an LLM is to produce text that passes for human, and apart from the obvious LLM tells, it largely does. Like say what you will about their lack of intelligence, the writing is solid. It's grammatically correct, spelling is dead on, what have you.
It reminds me of the famous phrase from Chomsky: Colorless green ideas sleep furiously. A sentence which is perfectly grammatically valid but is also completely devoid of meaning. An LLM would write that sentence, and it would be working correctly.
All of that to say: for all the things they CAN do and CAN be used for, I think we have to draw a hard line at education. I just don't think AI has a place in it. Of course that presumes that the goal of education is to, well, educate people, and especially here in the States but also abroad, we have been putting other interests, especially capital, far ahead of that for decades. I expect no different here.
And before someone comes in to go "WELL HOW DO YOU THINK YOU'RE GONNA STOP IT LUDDITE IT'S THE FUTUUUUUURE" yes, I'm sure as long as these exist and are available to people tech literate enough to access and use them, whatever that means into the far flung future, they will be a factor. Just like cheating, just like plagiarism, just like everything else that will get you kicked out of school. And the answer is the same: it will be stopped by institutions, imperfectly, and it will also happen anyway and with the same consequence: those responsible will mostly be harming themselves for short-term gains.
Some people need Jesus, but y'all need Kant ;)
"Enlightenment is man's emergence from his self-imposed nonage. Nonage is the inability to use one's own understanding without another's guidance."
https://www.columbia.edu/acis/ets/CCREAD/etscc/kant.html
Respectfully, I disagree. I think there's absolutely a case for AI being encouraged in younger people, and there's room for these tools. I've been leaning on LLMs for side learning in side projects, and it has concretely helped me with conceptual questions about math and Vulkan as I've been trying to learn some graphics basics with side projects.
I would grant: I was not the most studious kid, I could definitely stand to learn how to read code a lot more effectively than I do; but I have found being able to ask a computer, "what portions of the Vulkan Programming Guide are less relevant with Vulkan's design changes since the release" pointing me to the dynamic rendering extensions and placing it into context, with inline code and links out to useful blog posts for additional reading, that sort of thing is very helpful.
Working on a prototype before I was trying to learn Vulkan, I was using it to explore SDL_GPU's API which definitely had some gaps in its documentation. Granted again, I could have referenced the sample code - I am sure you'll prefer I'd have done that - but it helped to get information about what each piece of the API was doing, and gave reasonable results that made sense and did inform me enough to understand what I was doing, turning much of that into an interactive learning of basic GPU programming for graphics. Where the AI hallucinated, it was often on things like method names, which I was able to read through and find the methods it was intending to name. (This only occurred once or twice when I was learning).
Unrelated, but adding the C macro syntax and nesting macros, which I could have an LLM explain inline and link the GNU manual. Never got that taught to me in a C course. Man, computers are complicated!
These have not replaced textbooks; I have been using them alongside textbooks and handwriting code for practice, and they work as a very good complement. I also sometimes use them to unblock me - I don't know CMake very well and lean on AI to do CMake, so I can focus on learning C++ and graphics, which is my primary objective right now.
I would add too, I have for fun given it prompts about various topics I learned in university, and I often will get answers that are bang-on what I learned in university undergraduate courses - the topics I tried were welfare state taxonomies, distributed systems, disk storage performance, filesystem layouts and internals.
Boy, this would've been cool for me as a kid. There's just so much information right there, and pointing you to topics and textbooks a couple questions away, I wish I had these tools. I was a curious kid in a terrible MAGA-esque family that was deeply uncurious about the world, had no knowledge of any advanced subject and basically mocked me for trying to learn more about stuff. And you go to the school library and it's all kids shit, not even an option to try and reach out for more. Now smart kids might be able to go just learn shit very freely and be pointed to textbooks, and go pirate them off some Russian site, and start learning and go tutor themselves, as I'm doing today as an adult.
At least knowing myself and knowing if there's another kid like me, I think they would deeply enjoy having a natural language encyclopedia, if we can get it as close to that as possible. I think even with some error inherent, if the tools can be often and directionally correct, that would be a plus. I went to university, and the professors there hallucinated some things so embarrassing it should bar them from teaching, for the standards people hold LLMs to! i.e., sanitizing conspiracy theories that Android records all language through the microphone therefore iOS is better, Apple Silicon is more battery efficient because it is RISC and not CISC. Got a terrible history of computer graphics technology you'd know was slanted if you watch the 8 Bit Guy on YouTube. Rubbish.
The thing that worries me, and what this article really talks about, are the kids that just don't give a shit. They are not new - when I went to high school, before AI, stupid kids would copy code off the internet. I think AI probably makes it worse because it makes it harder to call out and enforce against it, and agreed, that should be stopped. But to me, that is mainly a cultural problem. Too many Americans are completely uncurious and just spout garbage; there are a lot of kids who grow up in that cesspool and are going to grow up uncurious, and then AI acts as a shortcut rather than a vehicle of curiosity.
And granted, maybe AI is less useful when you are in a structured environment - but the structured environment has its downsides. Even in that environment many of the TAs were clueless and unhelpful, or just too damn busy or already too knowledgeable to meet students where they were at. Again, talk about hallucinations with TAs! Many times in my experience. And that's all to say nothing about getting people to not just do homework but actually go get curious about things and try stuff that isn't required of them.
I think there will be some culture that remains curious, and has these tools, will come to grips with where they can help, where they go wrong, how to balance it with other learning methods; and I think they are going to have kids that absorb a lot more knowledge and get to play with topics and learn things, faster, to each kids' interest, perhaps even individualized tutoring at better scale - I hope that is possible.
I hope the United States as well, but maybe not, because holy cow our culture and attitudes are plainly terrible these days. Your comment is pretty representative of how most people react if I suggest this or talk about my own experiences I'm describing here. But I hope at least I'm arguing something comprehensive here. There is too little conversation beyond hyperbolic nonsense on the internet; I consider "FUTURE LUDDITE" etc. to be in that realm.
I will add, too, although less relevant to education than just generally - for all the talk that these tools must be useless and incorrect, that just plainly does not map to my experience using these tools. AI can chew through a debug log on a custom system and pick out root causes on behaviors very effectively, in my experience.
It is just hard to reconcile that denigration of AI with the typical experience I have using these tools in the real world. It is not omnipotent or God, but it can effectively assist in work. There is a certain cognitive dissonance I feel when I walk away from using the tool to help accomplish particular tasks, then hear over and over people say the technology is fundamentally useless and fundamentally does not work. I guess I am just not enough of an academic to understand how something can accomplish work yet fundamentally isn't, somehow.
In my experience, AI seems like it’s helping debug problems, but it’s very hard to tell when fictional information starts being added. I’ve wasted a lot of time trying AI suggested solutions that I only realized were pointless when I started asking questions like ”I think this distro is missing a package, could that be the problem?” It would agree and tell me a specific package to download. I’d then ask “could iptables be the problem?” It would agree and give me a specific configuration to change.
LLMs can be useful, but I haven’t found a way to use them where I’d be confident in using it to solve technical problems I didn’t already deeply understand.
why would I as a child ever develop the imagination needed to actively engage with AI tools in the manner you describe? those AI tools take care of the imagining for me.
Professors suddenly realized everyone was cheating and started paying attention, but the cheating isn't new ... A lot of faculty are happy when their students get good grades because they interpret it as I'm such a good teacher instead of I should pay more attention to how they cheat. AI woke some of them up to reality.
what if it's not llms and instead the removal of standardized testing in the state of california?
It's critical that we adapt to a strategy where AI supports our learning instead of the having it the other way around.
AI is a double-edged sword.
On the one hand, it's like having a free private tutor who is always available. It's a great learning tool.
On the other hand, students can use it to do all their homework for them, and skip learning altogether.
It’s only going to get worse. The second things like Claude Cowork get opened up to non-technical teams you start to see the influx of emails and Slack message written with LLM’s for absolutely no productivity gain (in fact probably a loss given how unnecessarily wordy the messages are). Too many people want to give up any and all responsibility.
IMHO this is clear case of correlation vs causation…
This generation of kids were fucked so hard by Covid and all the remote “schooling” and closing of public life.
AI rise happens to be happening when the kids who were just entering teens at Covid time are now going to school.
The more likely culprit would be repeated COVID infections themselves, known even in mild cases to cause damage to many body systems, including the neurological, rather than a month or two of remote learning. I'm not surprised at the widespread denial over this, honestly. It's bitter.
Lmao
A reckoning is coming for school. Learning the rote stuff is no longer essential. Now they need to learn, how to teach "how to think". How to invent, how to be creative. Art++, Woodshop++, Math--
"According to Berkeleytime, 35.3% of CS 10 students and 10.6% of CS 61A students received F’s in spring 2026"
Alternatively, more students are taking CS10 and CS61A irrespective of aptitude.
Anyone can code, but not everyone can become an employable SWE.
Anyone who has first or second hand experience with Cal or any other university knows how impacted CS majors have become, and how everyone is attempting to become a CS major because it's the easiest path to multiple high paying white collar careers.
And in all honesty, it's not like CS@Cal never had weedout classes (I remember CS70, CS61B, and Math54 had reputations of being the L&S weedout classes).
My son took CS10 a couple years ago, and even I (Masters in EECS from UCB) struggled with some really obtuse multiple-guess questions he showed me on the homeworks. Much of the classwork is done in Snap, a weird and stupid graphical "programming" language. If 1/3 of the students are failing, that may have more to do with the professor.
The question comes sooner than the students being tested on the job market. Another possibility is that dropping standardized testing was a net bad idea.
This is orthogonal to standardized testing.
At UC Berkeley L&S, students are undeclared by default, and everyone is incentivized to take the intro CS classes (CS10, CS61A) irrespective of aptitude because worst case they can declare a CS minor or use the classes for other adjacent degrees (eg. Applied Math, Data Science).
Additionally, while Cal doesn't require standardized tests, most students who applied and attended already took the SAT, ACT, and APs becuase they cross-applied to other universities as well. This is reflected in UC Berkeley's HS Weighted GPA being in the 4.31-4.65 range [0], which means most students will have taken at least 6 AP classes.
Hell, I attended an Ivy and even then Cal was a target program for me, as well as my peers. If I didn't get into my Ivy I would have ended up at Cal and ended up in the same position.
[0] - https://admissions.berkeley.edu/apply-to-berkeley/student-pr...
CS is now in the College of Computing, Data Science, and Society instead of the College of Letters & Science
Huh. TIL Cal started a College of Computing in 2023 with direct admissions into CS.
Spring 2026 saw a marked shift in student performance. We saw it in intro physics courses on the East coast too. I bet anyone who cared to look saw it.
Yep, I'm starting to hear this more and more. Matches my local data. It's a very massive and visible shift in DFW rates.
I'm not denying that. I'm just wondering if anyone measured if there is a correlation effect being induced by CS major declaration requirements.
Barely over a decade ago, CS tended to be a large but not too large major by enrollment in most universities yet nowadays it is the most in-demand major in most universities. You can see this at Stanford [0], but most other programs as well.
[0] - https://stanforddaily.com/2020/04/25/stanford-in-the-2010s-t...
The failure rate tripled. So no, it’s not CS major declaration requirements.
> The failure rate tripled
And did the rate of students attempting to declare CS also triple?
> So no, it’s not CS major declaration requirements
Are intending CS majors in your university required to take that specific physics class before declaring the degree?
No.
No. CS majors take a different physics sequence at my institution.
The proximate cause is the wide release of somewhat effective AI tooling. When the tools weren't able to do or explain, students didn't use them. Now the tools are somewhat competent, so students do use them.
The students didn't substantially change from Fall 2025 to Spring 2026. You can have whavever gatekeeping hobby-horse you want, but it's not germane to this particular conversation.
Dude, cheating in CS10 of all classes?
Yeah pretty hard to imagine Claude making sense of BYOB
I don't like the framing of calling it academic dishonesty. If it were one or two students doing it, sure. But there is no reason to believe that 2026 Berkeley freshmen are fundamentally more dishonest than 2025 Berkeley students. When so many are doing it, it suggests a sea change in the understanding of what is honest or dishonest in that particular community. That sort of thing should be treated more like a "disease": something that should be treated, than a "crime": something that will be punished.
AI gives us some bad things but it's really outweighed by the good things. One one hand we have very rapid deterioration of our children's mental capacities yes, but on the other hand we have also made the internet into an unnavigable mound of slop produced by, and for consumption of, bots.
One thing that bothers me about these conversations: failure is an important signal that what we're doing isn't working as well as we thought it would, not a sign of the apocalypse.
Kids need to understand how to adjust and grow from failure more than they need to always be on the happy-path of straight A's and easy money.
How we respond to failure is how we teach response to failure. Hand-wringing, pearl-clutching and finger-pointing aren't valuable life skills.
Personally it's easy for me to be contemptuous - I opted into an accelerated math program that banned calculators when I was in Junior High. It helped me cultivate an very crisp intuitive/conceptual understanding of basic mathematical concepts that's carried through to today. I think we should do more of that kind of education, but it's expensive and requires amazing educators and a tolerance for student struggle.
Get the machines out, absolutely. But respond to failure compassionately, as part of a natural learning process.
It’s because they lowered the standard of who gains entry in the name of equity and other woke nonsense. It has nothing to do with AI but it’s a convenient thing to blame.
Or your lazy excuse is an even more convenient thing to blame
I don't think his comment was lazy, there has been a documented concerted effort to get rid of math and standardized tests because real world data conflicts with their political reality.
Doing so ultimately diminishes those that have earned their place to stand as exceptional performers through hard work and grit and its replacing meritocracy with a form of apathy disguised as political empathy.
Contrary to what one side of the political spectrum wants to push as truth, we are all not equals in terms of ability. Different cultures and demographics excel at different things and not always the same things. This shouldn't be chalked up as racism or something to be hidden at all.
We're going to find that LLM usage has even worse effects on the mind than the horrific effects we're just starting to be certain of from social media. I'm just not going to use either. See you lads on the other side.
Probably not a bad thing, the coursework is antiquated and meeting students with new advanced tools and the awareness of AI's impact on things in the coming future
I imagine there is some apathy and laziness here but idk how unjustified it is
"Noooooo you need to manually code on paper in assembly"
Alright, well maybe the CS grads need to, but why expect that of everyone else