There's also the range based for loop in C++. An actual comparison of iterator-style loops might be interesting but this article is a pretty vacuous bit of basic Python.
It's also a confusing title, given that Scratch is the name of a programming language.
I don't really get the point of the article. Even if I knew little about python, would be it surpsing that a language with no real basic types is probably abstracting a lot?
Even a simple i=0, i=i+1 is "hiding" a lot in python then.
To be fair, when I started learning CS, the `for x in y` syntax was cryptic to me because I was unfamiliar with concepts such as iterators & generators. `for(int i=0; i<len(y); i++)` made way more sense since there is no hidden logic (besides additions as you highlighted in your comment, but which I think is easier to have a grasp of). So I really wish I had read this article when I started my CS journey a couple of years ago.
In the author's mind, it's unexpected/amazing that 'for' can iterate over many types. But it's NOT unexpected/amazing that 'iter' can iterate over many types. I have no idea why.
It's not like 'for' is limited to counting in other languages. The grand-daddy in c does something until some condition is false, and that thing can equally be incrementing/decrementing a number or invoking some function. That's what a loop does in any case, it compiles down to a conditional jump (JNE/JE..)
Maybe his reason for astonishment is obscured by over-use of an LLM to 'enhance' the text.
> It's not like 'for' is limited to counting in other languages. The grand-daddy in c does something until some condition is false
C 'for' is a while loop. It's strictly syntactic sugar for an already existing feature. And it's really, really transparent. `for(A; B; C) { do_stuff(); }` isn't just a while loop, it's this while loop:
A;
while(B) {
do_stuff();
C;
}
Other languages have treated for as a separate concept from while. C isn't really informative in that case.
It would be very surprising for somebody without a formal software development background and years of experience.
Looking back at 2015 when Python 2 was still supported, there was a lot of confusion for why Python 2 would create a tuple while Python 3 created a generator for the following statement:
foo = (x for x in [10, 20, 30])
The blog post is trying to help fill in a gap of knowledge for anyone trying to understand more of what goes on behind the curtains.
But it's not a Python thing. Rust is noted below, and there's also C++.
std::container<T> container;
for (std::container<T>::iterator i = container.begin(); i != container.end(); ++i) { ...}
auto iter = container.begin(); while (iter != container.end()) { ...; ++iter; }
for (auto const & t : container) { ... }
Barry Revzin has a talk explaining about different kinds of iterator. The C++ iterator is a maximalist, very powerful and flexible idea which - as a result - also has lots of opportunities to set everything on fire. Their model was the C pointer, we can iterate a pointer, compare it against some sentinel "end" value, dereference it, but also we could index into it, decrement the pointer and go backwards, comparing against a "start" value, and we can write to a reference rather than just read from it.
This is far more than most iterator designs allow, and as I said if you needed more this is very welcome, but if you didn't it's probably bad news.
C++ can sort iterators for example, so long as they're suitable for indexing back and forth as the sort progresses. Rust's sort functions are defined on its slice types, which makes sense in practical terms but if what you've got isn't quite a slice but could meet the requirements in C++ that's a problem for you.
Very few people who use Python realize the loop is not just looking into the values but asking the values to produce an iterator. It's only when they outgrow this early stage that they are ready to understand how to make a finite iterator themselves.
'for' loops in Rust do the same: they create an iterator and then iterate over that.
You can write the exact same loop with `let mut iter = v.iter(); while Some(x) = iter.next()`.
'for' loops in Rust are purely syntax sugar, and I somewhat wish they didn't exist. They provide you two ways of doing the same thing, but one of them hides the details from you. Having 'for' as a keyword is nice for folks coming from other languages, but then it hides the possibility of other interesting usages, like cloning an iterator inside a loop.
Which is funny, because whenever I encounter a language for which `for` *doesn't* work this way it feels antiquated. I do however wish another keyword was used in many cases, becuase `for` in C and Go is so much different then `for` in Rust or Python. I think the higher-level case (Rust and Python) should really use a word like `foreach` or maybe something compeletely different like `itr`, although I get that they want to "look" more like C
Iterators and the loops of the form "forall X in A do ..." have been invented simultaneously with the appearance of the first version of the language C (i.e. around 1974), in the languages Alphard and CLU.
From Alphard and CLU, loops of this form have passed into one of the versions of the language Algol 68 (1977), and from there into the UNIX Bourne shell (1979), which inherited a few Algol 68 syntax features (because its author had worked at writing Algol 68 compilers).
Iterators and "forall" loops have begun to spread into mainstream programming languages, e.g. into C++, only a couple of decades after their invention.
Is "hiding" in the sense that you just need to have read the docs or know how the language works at a pretty basic level really a problem, or even a negative? I would certainly say that the readability and clarity on the form of loop being used is a bigger win, either way.
It's true that they're just sugar but Rust does explain how the for-in loop de-sugars and you've over-simplified considerably. Your syntax also doesn't quite work.
The value is in idiom, turning everything into loop expressions (The "while" keyword is also just sugar, Rust's only fundamental loop is named loop) makes it harder to discern what's actually going on.
If you want to clone the iterator in some cases rather than consuming it, that should look different so that reviewers will see what you're up to.
It should be `v.into_iter()`, and that distinction matters because of ownership / move semantics.
It just so happens that for most collections, `IntoIter` is also defined for references to them, which typically gives you the same behavior that `.iter()` would give.
... shows that today ranges like 0..5 aren't Copy even if that would be possible, which means if they're consumed they're gone, whereas an array of integers is Copy and so consuming it doesn't mean it's gone, you can just consume it again.
The desire is that Rust 2027 edition will change the nice syntax for ranges to produce new ranges like core::ranges::Range which are Copy if possible and only IntoIterator, the original ranges are never Copy but are Iterator, we now regret this choice.
In addition to the points that sibling comments have made about how you're not quite right with the exact semantics of for loops (which also take ownership via `into_iter` and therefore are a bit different from `while let`), it's worth pointing out that if you peek a bit further in to MIR, all loops in Rust just desugar into the `loop` keyword with manual breaking, including `while`. It's not really clear to me why for loops in particular bother you.
TL;DR: if you want a general looping construct that works no matter the container type, you need an iterator protocol that types can opt into. This is a standard technique adopted by most programming languages that are not explicitly low-level, and has existed for the past 60[1] years. The OP (re)discovered it and thought it was worth blogging about.
Well, it is kind of interesting to see how the very basic programming building block (iteration) gets generalized without incurring syntactic costs. Whether it's worth a place on the HN front page is debatable, though.
[1] Too lazy to track the actual first implementation, but I'd be astonished if the concept wasn't well-known by the 70's.
> Compared to i++ from C/C++ or forEach from JavaScript, Python's version just works.
Comparing to forEach in JS is incorrect because forEach is an method of Array.
You should compare it to `for...of` in JS. Both operate on iterators.
Article is missing an important distinction between iterators and other "array like" types (including strings):
Iterators don't have to stop, e.g., they can take from a generator that never ceases.
Both Python and JS are happy to loop forever if the iterator never stops.
There's also the range based for loop in C++. An actual comparison of iterator-style loops might be interesting but this article is a pretty vacuous bit of basic Python.
It's also a confusing title, given that Scratch is the name of a programming language.
I don't really get the point of the article. Even if I knew little about python, would be it surpsing that a language with no real basic types is probably abstracting a lot?
Even a simple i=0, i=i+1 is "hiding" a lot in python then.
To be fair, when I started learning CS, the `for x in y` syntax was cryptic to me because I was unfamiliar with concepts such as iterators & generators. `for(int i=0; i<len(y); i++)` made way more sense since there is no hidden logic (besides additions as you highlighted in your comment, but which I think is easier to have a grasp of). So I really wish I had read this article when I started my CS journey a couple of years ago.
In the author's mind, it's unexpected/amazing that 'for' can iterate over many types. But it's NOT unexpected/amazing that 'iter' can iterate over many types. I have no idea why.
It's not like 'for' is limited to counting in other languages. The grand-daddy in c does something until some condition is false, and that thing can equally be incrementing/decrementing a number or invoking some function. That's what a loop does in any case, it compiles down to a conditional jump (JNE/JE..)
Maybe his reason for astonishment is obscured by over-use of an LLM to 'enhance' the text.
Why would it ever be surprising that I can y= [x,x,y,y] for x in y; x=y; return y;
and get [y,y,y,y]?
> It's not like 'for' is limited to counting in other languages. The grand-daddy in c does something until some condition is false
C 'for' is a while loop. It's strictly syntactic sugar for an already existing feature. And it's really, really transparent. `for(A; B; C) { do_stuff(); }` isn't just a while loop, it's this while loop:
Other languages have treated for as a separate concept from while. C isn't really informative in that case.It would be very surprising for somebody without a formal software development background and years of experience.
Looking back at 2015 when Python 2 was still supported, there was a lot of confusion for why Python 2 would create a tuple while Python 3 created a generator for the following statement:
The blog post is trying to help fill in a gap of knowledge for anyone trying to understand more of what goes on behind the curtains.But it's not a Python thing. Rust is noted below, and there's also C++.
Barry Revzin has a talk explaining about different kinds of iterator. The C++ iterator is a maximalist, very powerful and flexible idea which - as a result - also has lots of opportunities to set everything on fire. Their model was the C pointer, we can iterate a pointer, compare it against some sentinel "end" value, dereference it, but also we could index into it, decrement the pointer and go backwards, comparing against a "start" value, and we can write to a reference rather than just read from it.
This is far more than most iterator designs allow, and as I said if you needed more this is very welcome, but if you didn't it's probably bad news.
C++ can sort iterators for example, so long as they're suitable for indexing back and forth as the sort progresses. Rust's sort functions are defined on its slice types, which makes sense in practical terms but if what you've got isn't quite a slice but could meet the requirements in C++ that's a problem for you.
Very few people who use Python realize the loop is not just looking into the values but asking the values to produce an iterator. It's only when they outgrow this early stage that they are ready to understand how to make a finite iterator themselves.
'for' loops in Rust do the same: they create an iterator and then iterate over that.
You can write the exact same loop with `let mut iter = v.iter(); while Some(x) = iter.next()`.
'for' loops in Rust are purely syntax sugar, and I somewhat wish they didn't exist. They provide you two ways of doing the same thing, but one of them hides the details from you. Having 'for' as a keyword is nice for folks coming from other languages, but then it hides the possibility of other interesting usages, like cloning an iterator inside a loop.
Which is funny, because whenever I encounter a language for which `for` *doesn't* work this way it feels antiquated. I do however wish another keyword was used in many cases, becuase `for` in C and Go is so much different then `for` in Rust or Python. I think the higher-level case (Rust and Python) should really use a word like `foreach` or maybe something compeletely different like `itr`, although I get that they want to "look" more like C
Iterators and the loops of the form "forall X in A do ..." have been invented simultaneously with the appearance of the first version of the language C (i.e. around 1974), in the languages Alphard and CLU.
From Alphard and CLU, loops of this form have passed into one of the versions of the language Algol 68 (1977), and from there into the UNIX Bourne shell (1979), which inherited a few Algol 68 syntax features (because its author had worked at writing Algol 68 compilers).
Iterators and "forall" loops have begun to spread into mainstream programming languages, e.g. into C++, only a couple of decades after their invention.
This is what PowerShell does, it has both the archaic `for` style with a newer iterator-like `foreach`.
I would say Python really wants to look as much not-C as it can. In this case it's not the word "for" but the "for x in" construct.
Is "hiding" in the sense that you just need to have read the docs or know how the language works at a pretty basic level really a problem, or even a negative? I would certainly say that the readability and clarity on the form of loop being used is a bigger win, either way.
It's true that they're just sugar but Rust does explain how the for-in loop de-sugars and you've over-simplified considerably. Your syntax also doesn't quite work.
The value is in idiom, turning everything into loop expressions (The "while" keyword is also just sugar, Rust's only fundamental loop is named loop) makes it harder to discern what's actually going on.
If you want to clone the iterator in some cases rather than consuming it, that should look different so that reviewers will see what you're up to.
It should be `v.into_iter()`, and that distinction matters because of ownership / move semantics.
It just so happens that for most collections, `IntoIter` is also defined for references to them, which typically gives you the same behavior that `.iter()` would give.
More specifically for x in y consumes y in Rust.
https://doc.rust-lang.org/std/keyword.for.html explains how a loop is de-sugared
https://rust.godbolt.org/z/5jzhxYM51
... shows that today ranges like 0..5 aren't Copy even if that would be possible, which means if they're consumed they're gone, whereas an array of integers is Copy and so consuming it doesn't mean it's gone, you can just consume it again.
The desire is that Rust 2027 edition will change the nice syntax for ranges to produce new ranges like core::ranges::Range which are Copy if possible and only IntoIterator, the original ranges are never Copy but are Iterator, we now regret this choice.
In addition to the points that sibling comments have made about how you're not quite right with the exact semantics of for loops (which also take ownership via `into_iter` and therefore are a bit different from `while let`), it's worth pointing out that if you peek a bit further in to MIR, all loops in Rust just desugar into the `loop` keyword with manual breaking, including `while`. It's not really clear to me why for loops in particular bother you.
TL;DR: if you want a general looping construct that works no matter the container type, you need an iterator protocol that types can opt into. This is a standard technique adopted by most programming languages that are not explicitly low-level, and has existed for the past 60[1] years. The OP (re)discovered it and thought it was worth blogging about.
Well, it is kind of interesting to see how the very basic programming building block (iteration) gets generalized without incurring syntactic costs. Whether it's worth a place on the HN front page is debatable, though.
[1] Too lazy to track the actual first implementation, but I'd be astonished if the concept wasn't well-known by the 70's.
I got curious and checked: CLU had iterators in 1975.
> Another key feature of the CLU type system are iterators
It also had exception handling and tuple unpacking!
> For instance, writing x,y := y,x would exchange values of x and y
(I'm joking, that's "just" parallel assignment, it just looks like what you do in modern Python with unpacking. And yes, it's all from 1975.)
Details: https://en.wikipedia.org/wiki/CLU_(programming_language)
AI slop. Flagged.