When we run, or we write and run code, what we're typically doing is taking data, storing it, and then functionality, that we then apply to that data. Even as simple as I have a score for a player and I increase that score. We call our live data that's stored in our application our state. That just means the labels and the data, the values currently in our application at that moment. But that kinda masks the small part that's pretty significant a lot of time. I have my data, I have my functionality, but often my data is a collection of data. Maybe an array, a list of data or a set of data, or a map of data. Or an object containing a bunch of properties with data. It's a single element. So actually the process of accessing each of our elements from a collection of data is in itself a task. The data is just not there which we can apply functionality to, we have to go access that data. And in arrays, we do it using indexes to grab the individual element. You sort of have a static collection of data, and go and grab an element. And actually, that's not a small process in its own right. And we'll see, we'll play that out here with a for loop. And we'll discover that a beautifully designed process.
It's not clear that's the best unless it's, most of the time when I'm accessing my data, I know I just want my element, and I want the next element. Don't care about how I go about getting those elements. So wouldn't it be amazing if we could rethink collections of data as instead a sort of stream of elements coming towards me, that I could call a function, that's just going to instantly return me the next element from my stream. So I would run a function, it would return out the next element for my flow. Rethink my data, not as a static collection, I need to go and manually get each element to run functionality on it. But instead, flip it and say, my data's there, and I'm going to have access to it by running a function that kinda turns on the flow of the next element of my data. And that is a paradigm shift in how we think about applying functionality to our data. No longer is there the intermediate step of getting the elements, manually going into the data collection, and grabbing the next element. That's not a small thing. And it's an unnecessary step when most of the time all I want is my next element in order.
So why not instead rethink my processes being applying functionality to data as the data being given to me element by element from a sort of flow of that data? Before we see that beautiful new way of thinking about our collections of data and getting them, so we can apply functionality to them or getting them element by element one by one.
Old fashion way to grab data from a collection:
we're going to see the old version, where we have a list of data, and we're going to manually go and grab each element. We're going to discover, a beautiful new way of thinking about this, using each element one by one. But let's first see. And by the way, it's going to show us just how kind of imperative, procedural, bit by bit, kind of exactly how we're going to do this, this old fashion method is. Gives us a manual, fine-grain control, but at the expense of a clean, readable code. And also, I think a cleaner way for ourselves of thinking of our data as these flows of data, rather than static collections that we go and grab element by element. So let's first just at least see this traditional way. And I think it's going to show us we want a better way. But here's our traditional way. And I'm going to try and diagram this. But part of the point to say is that diagramming for loops is an odd thing to do, honestly. When we think about most of our code, we run execution context to run functionality.
const arr=[ 1, 2, 3 ]
for( let i = 0; i < arr.length; i++ ){
console.log( arr[ i ] )
}
But our for loops are these funny little things that happen kind of in isolation of our regular way of thinking about code. But there we go, all right.
Line One: We are declaring a constant arr and assigning it an array containing the values 1, 2, 3.
Line Two: Here we are running a for loop. So what's the check we're going to do, is i less than arr.length. Is i, which is initiated to 0, less than arr.length, which is 3? Yes, it is true then console.log( arr[ 0 ]) which is 1.
That's our for loop try to be written up in a slightly more logical sense than the kind of weird, check this, do the code, come back, do this, check this, a weird circular flow of an actual for loop.
The next thing to do is i++ with an incrementi to 1, is 1 less than 3, yes, it is. Now, we get to position 1 of numbers which is 2 and i becomes 2. Increment i to 2, 2 less than 3, yes, it is. So the number in position 2 is 3. So we console.log 3, i++ becomes 3, is 3 less than 3? No, it's not. So we come out of our for loop.
We're going to discover, if we rethink our collections of data, our flows of data, where we grab element by element, we can dynamically control those flows of data. We can set what our next element in our flow of data going to look like based on things that have happened in previous elements after they've been returned out. Rethinking our collections of data as instead flows of elements we want to grab one by one, It's going to give us control over what those next elements will be. We'll see that a little bit later on, really, really cool. But for now, programs store data and apply functionality to it. There are two parts to applying functionality to collections of data, 1. is the process of accessing each element, and then, 2. is what we want to do to each element.
Iterators:
Iterators, this new way of thinking about accessing data from collections of data. I mean, it's like lists, or arrays, or whatever, automate the accessing of the element. So we can focus on what we do to each element, and make it available in a super smooth way. They make it available so they say we've got a function that when called , returns out my next element. Run it again, gives me the next element. Run it again, gives me the next. In other words, the function attached and bundled on it somehow, in the background must be our underlying collection of data. Plus also, we must somehow hold on to the information of which element am I currently at, so that we don't give me out the same element each time, but give me out instead the next element. But we know that when a function runs it never remembers its previous running. Its local memory gets reset every time, right? It's empty, so how the hell am i going to have my function both be able to be run, give me the next element, but also, therefore, have underlying in it somehow bundled on it my underlying array of data that it's grabbing the next element from one by one? And bundled on it the sort of tracking variable that's tracking which element have I already given out so I know which one to give out next, how am I going to bundle that all up? Imagine, though, if we could create a function that stored the numbers, and each time we ran the function it would return out our next element from numbers.
Note you'd have to remember which element was next up somehow. That means between the function's invocations, runnings, it would somehow have to remember that what was the last element that was passed out. But if we could do it, this would let us think of our array, our list, 1, 2, 3, as a stream, a flow of data, with our function returning out the next element, and then the next element. This makes our code more readable, and we'll see in a minute, more functional. But it all starts with us returning a function from another function. Because all the most beautiful, elegant things in JavaScript begin with us returning a function for another function. Because that's going to give our function that's returned out superpowers.
function createNewFunction( ){
function add( ){
return num + 1
}
return add
}
const newFunction = createNewFunction( )
const result = newFunction( 2)
Let's go line by line.
Line One: We are defining a function createNewFunction.
Line Two: Defining a constant newFunction and assigning a return value comes after running createNewFunction. createNewFunction return a function definition which is labelled in the createNewFunction execution context as add.
Line Three: Defining a constant result and assigning the return value after execution of newFunction with argument 2 which is 3. But do not think that JavaScript, when it sees newFunction, is going, what's newFunction? I'd better go and check the line before. The line before will never be returned to. The line before, create an execution context, inside of which it is created add, was returned out, stored in newFunction, and at no point do we ever, ever, ever, do newFunction ever care about createNewFunction again. It only cared about createNewFunction in the sense of I don't yet know what to store in me, so I'm undefined while I go and run createNewFunction, get an actual value, this function, and store that function here. Never think that its newFunction is a command to go and run createNewFunction. It isn't, it's whatever at that moment you got out a createNewFunction and that's over. So it's the add functionality. So when we call a newFunction like below, it is just interested in the add functionality. It never goes back into createNewFunction.
We now have a situation here where we could create a little function inside another function, returned it out into a new global label, and then use the new global label for that functionality, for what possible reason? Why did I not to find add to globally?
It's going to turn out when we return a function from another function we get so much more than just a function. We're going to get a ton of what? A single profound bonus. Functions that when we call them give us our next element from our flow of data. And I might call them what a strange name. But instead, give us our next element from our flow of data. Let's see it here, so if you wanna create a function that holds, hold. You wanna create a function that has the ability to return our next element from a list of data, four, five, six for example. But then also, bundled on that function, it must have the underlying data to grab from, right? Otherwise, where's it getting the data from? And it must have the ability to track which element it was on before, so that when we run that function, again it doesn't give us the odd element, the previous elements. How the hell is going to do that? Because functions, when they call, do not remember their prior invitation.
They do not remember data created that was created in their prior running. The location you used on context, we run the function again, brand new function, add created. There's no memory of the previous running. So how can we have a function that when run, somehow remembers its previous running? That it's been run before and had a previous element given out? We shall see, but that's what we gotta try and do.
function createFunction( array ){
let i = 0
function inner( ){
const element = array[ i ]
i++
return element
}
return inner
}
const returnNextElement = createFunction( [ 2, 3, 4] )
const element1 = returnNextElement( )
const element2 = returnNextElement( )
Going to start with this calling createFunction that's going to return out an inner function into returnNextElement. And then we're hopefully get our first element, who knows how? We shall see, all right, we're going to walk through this very, very precisely because this is pretty much our main, really our only code on this whole section, but it's a very important code. And honestly, it can be considered quite hard to focus.
Line One: We're creating a function called createFunction in memory.
Line Two: We're creating a new constant called returnNextElement. So returnNextElement is going to be the output of calling createFunction, where we're passing the array 2,3,4, to it. We're going to create what call, a new Execution context. The first thing that is declared inside this function is an array with 2,3,4. Next to defining a variable i and set to 0. Next, we are defining a function with the label inner. Then we returning the function labeled with inner and store it into returnNextElement in global memory.
So how can we, now, in theory, what do we hope that calling the returnNextElement function's going to do, in theory? Return 2, that's what we hope. If we run it again it would hope it would return? Return 3, and again if we run it will give 4 and again undefined at some point. That's what we're hoping for desperately, because that allows us to rethink our collection of data as a flow. I run a function and get my x element, I run a function, get my x element, I run a function, get my x element. That's a beautiful way of thinking about data. No more, I have a collection, statically, of data in memory. I've got to go and look at it, get an element, use it, look at it, get an element, use it. Now I just run my function and I've given, I've given my next element. It's a really beautiful way of thinking about my collections of data as flows of the element after element after element. It's a beautiful way of thinking. So you're right, it is to run, call returnNextElement, so let's do just that.
Line Three: Defining const element1 and call returnNextElement that creates a new execution context. The first thing we're going to do inside returnNextElement is to define a constant element and assigning the eighth position of the array, what the hell are these? Well, let's start to figure it out. Where do we look first for our array and our i Local memory, yeah? Do we find them, no? I don't find it in this local one, where do I look next? In Global, exactly. Into Global I go looking desperately for my array and my i, do I find them? No, so I get an error, right?
Where is that array[ i ]? Because I am certain, do not at any point think I can go back into my createFunction execution context. This has long gone. I cannot suddenly, I'll just go up a group and createFunction. That is long gone. As soon as I define my inner function, inside of createFunction, while I was still back in createFunction, before it exited. As soon as I defined it, I got a bond to all the surrounding live memory, the surrounding data. You can call it to state, you can call it the variable environment, the live memory, the data around the function definition. I got a bond to it, a little link, a reference to all surrounding data. I gotta bond to surround data immediately. Meaning when I return that function out, I brought with it on the back of the function as the function got returned out, out on the back of it came all that surrounding live data. When I return that function out, return out the function that used to have the label inner, into returnNextElement. On the back of the function, I brought all the surrounding data from when it was born. And it got stored in this new label, we will give it a new label globally returnNextElement. But my surrounding data is attached to that very definition with the array literally in the memory as 2, 3, 4. Not it will be, but literally, store to memory. i was literally stored in memory as the number 0, and it's attached to the back of my function. There it is, on the definition itself. When I don't find array and i on my local memory, I do not go to Global immediately. Instead, I go look at my definition of my function, I see is there a backpack of data that was brought out with the function, and look, there it is. Now my array becomes [ 2,3,4 ] and ibecomes 0.
I'm not looking for my local memory, I'm looking at the function definition that's been called itself, and there attached to it, is my persistent. My backpack of data from when the function was returned out from where it was born. As soon as it was born, it got a link to all the surrounding data from the memory in which it was defined. When I return the function out, that bond didn't break. That bond pulled out on the back of the function all that surrounding data. And when I call that function by its new Global label here, and don't find some, it refers to labels in defined data of in the local memory, it doesn't panic. It looks first for the function's definition and finds attached to the function, our persistent live data from where the function was born.
I'm going to take the zeroth position of the array as 2 and next increment the i value. Then we return 2 to global constant element2. Look at that element one is four, exactly what we wanted From calling returnNextElement. Let's call it one more time to see what happens, and then we'll talk about what concept this backpack is. You may already know its posher name.
All right, returnNextElement pops off the call stack, all of its execution context cleared, right? So we better not be having this information in here, because it's all deleted. And we hit our next global line of code which is declaring the constant element2assigned with return value 3 which is the second index element of array.
The underlying state, the underlying data from which it's going to extract and return to us, on its call, one by one, the individual elements from that underlying collection of data. And it's keeping track of which one's up next using this little tracking value, also all bundled up on this return next element function. I think that's very very beautiful, we'll talk about it in a second, but I think it's very beautiful that you can have a function return its element that has everything you need. It has the ability when called, to return out an x element. It has bundled on it the underlying data, that it knows to extract from. And it has bundled on it the information about what next elements are returned out, all bundled up in a single function.
When the function inner is defined it gets a bond to surrounding local memory. You got that bit in which it's being defined. We return out the function known as inner. The surrounding live data is returned to attach the back of the function definition itself, which we then give a new global label, returnNextElement. When we call returnNextElement and don't find an array[i] and its local immediate execution context, we look into the function definition's backpack of live data. The backpack has some official names. Okay, let's first say, how is this bond to the backpack actually stored on the function? It's stored in a hidden property, we can actually see it in the Chrome dev tools. We can't access it meaningfully, [[scope]]. If you were to console.log return the next element, press a little down arrow, you'd see [[scopes]]. And that will have stored in it, so it's on this hidden property scope, and that would have stored in it all this data, okay? So with that in mind, by the way, in general, a language whose rules, or its rule about what data is available to you is about where the function is born, where it was defined, is known as a lexically-scoped language.
That means a language that says, where you defined me is what determines, the positioning of my definition inside another function, is what determines what data I have available to me when I'm eventually run, eventually called. Wherever you end up calling me. Because I attached the data from around me when I was born, to me. That's the first place I look, besides the function execution context itself. That is known as a lexically scoped language, as opposed to a dynamically. It's called a statically or lexically-scoped language. You could very easily imagine a language where the next place I'd look is global. So we can call this backpack of data, wanna be really fancy? We can call it a persistent lexical scope. This is our lexical scope bond, or reference, persistent lexical scope reference data.
You may also remember that the memory is called the variable environment. That means that the variables are available to you around. So you can call it also the variable environment has been enclosed, closed over. And returned out of the function. So, you might call it the Closed Over Variable Environment, the COVE. People intuitively call it the backpack. People also unfortunately I think unintuitively but colloquially typically call it, that's the best name. Typically call it the closure. You'll hear engineers say to put those values in the closure. We call the whole concept closure, the idea of functions persisting their lexical scope references, their surrounding data when they were born. We call the whole concept closure, and we call the backpack the closure. So I think that's just a bit too much under one label. And certainly, a label doesn't mean that much to me anyway. So I like the name backpack but people tend to call this backpack of data the closure. It is to say that our functions get to have memories. Not their local memory that gets deleted each time but a persistent cache of data attached to their very own definition. Meaning we can have a function that when called, doesn't find data inside itself and looks in its persistent cache attached to it. All bundled up on a single function. It's a pretty beautiful design.
All right, returnNextElement, that function has everything we need, by the way, the only way you can get a backpack in a meaningful way is to return the function from where it was born. Is to return the function and bring with it the data, that's how you get that persistent data. So returnNextElement has everything we need all bundled up in it. It has the underlying array in the backpack, it has the position we're currently in, in our flow of elements coming out to us in the backpack. And the ability when it's run when the function actually itself has run to return out that next element. This relies completely on the special property of functions in Java that also were born inside other functions and returned, they get a backpack of persistent data that they have access to when they're called. What is the posh name for returnNextElement? Who knows the posh name for returnNextElement? It's known as an iterator.
Any function that when called gives me out the next element from my flow of data. My collection of data gives me the next element. And when I run it I switch on the flow and get the returning next element, I run the function again, I switch on the flow, get the next element. That is known as an iterator function. It gives me one by one, every time it's called my next element from my underlying collection. I refer to the name returnNextElement, we call these iterators. Here, what our flow of data is, is preset. We can't go back and change it. It was a sort of preset form at moment we created the returnNextElement function, which was born inside create function. We can't go back and change it. I mean we could design returnNextElement in some way to be able to mutate it.
There's now a new way of dynamically controlling the values that come out of that flow, known as generator functions. But first, iterators turn our data, our collections of numbers, or list, or whatever, into streams, into flows of actual values that we can access one after another. No longer are we getting a static list of elements and we go grab element at zero, go grab element at one. Now we run a function and it gives us the next element, one by one by one. Now we have functions that hold our underlying array, the position we're currently in, and return out the next item in the streams of elements from our array when it's run. This lets us have for loops, known as for of loops, that show us the element itself in the body of a loop. No accessing some from some array the position, and you'll see these in the challenges, but rather instead actually give us out into the body of our for loop the actual element. Because behind the scenes it's running returnNextElement. And returning that individual element and giving it the label, I don't know, whatever you want, element, next element, whatever you want to call it. And more deeply it allows us to rethink our collections, our static lists of data, our arrays as flows of elements because most of the time we store a list of data, we don't really care about it so much as a static thing. We want to be able to get the elements one by one. That's actually what we want to do with it most of the time. We're not just sort of just leaving it there, we want to get the elements out one by one. And maybe conflate them, join them together, do something with them. It allows us to rethink arrays as flows of elements themselves. Which we can interact with by calling a function that switches that flow on to give us out our next element. We have truly separated, decoupled, the process of accessing each of our elements, from what we end up wanting to do to our individual elements.