What do you want to learn?
Skip to main content
by Kyle Simpson
Resume CourseBookmarkAdd to Channel
Table of contents
First of all, I said we need to talk about this notion of purity, but we have to understand the opposite first. And the clearest and easiest way to understand what an impure function is is to think about a function that produces a side effect. What is a side effect? A side effect by virtue of its name is something that occurs that's indirect as a relationship of an action that you took. So an action that you take like calling a function and asking for it to compute some value and return it back, that's the direct effect of that function. We get the value back. A side effect is basically when we change the state of the program through an indirect means. I call this function, I get a direct result back, I get some direct effect back, but I also change something else about the state of my programs. And side effects are the big evil in functional programming, like the big, big evil. That's why we start here. A functional programmer will tell you we've got to reduce and remove as many side effects as possible. If you've ever worked in a _____ language, I haven't done much, but I've read some things about Haskell, and Haskell and other functional programming languages. And they get virtually impossible to do side effects. Like the language is designed from the ground up to not even let you be able to do that, which to be honest with you sort of twists my brain. I'm not entirely sure I understand whether that's valuable or even how that works because I haven't done---personally, I haven't done a lot of Haskell programming. But it's this notion that this is so important that they baked that into the language. Now, unfortunately, there are some places where we kind of have to cheat. For example, input and output. Even something as basic as a console log statement to your console is from a theoretical perspective a side effect. You've changed the state of what is happening. So input and output, pretty basic importance. You're not going to get a ton of usefulness out of your programs if you have no notion of input and output. So even in those functional programming languages that draw very distinct line and say, You should not and cannot create side effects, they have this cheat, this side door cheat that allows them to do things like side effects on input and output. And they say, Well, that's just a necessary evil. We don't want to program that way, so we keep everything else pure, and we just leave that over to the side. And there's virtue and there's value in being very intentional and very explicit about what's pure and what's not pure, and virtually everything in Haskell programming is pure except for these few little cheats. So side effects, producing side effects can be things like the console log. It can be changing the state of some variable such that each time the function is called, you're going to get a different result. That's also a side effect. What's interesting is that there's a little bit of tension, and I'll just sort of project a little bit to a little bit later in our discussion about closure. Closure is about maintaining state inside of a function. And when you maintain state in a way where that function can be called over and over again and get a different result each time, that can start to---there's some tension there between that and this notion of side effects. So even the way that we put these Lego blocks together, we can have two very good Lego blocks that a functional programmer would say, Yup, those are both great. But there's a way to configure those that is not so great. So we have to be careful about these things. Side effects are the big evil that we want to avoid, so if we're looking for the big evil to avoid, this is just a quick illustration. You can see that I call foo. I pass in the value 5, and I have two side effects as a result of this function. I have the y and the z variables that are outside of the function. Now there's lots of fancy terminology for this, but I'm trying to keep us away from the terminology, but these are just variables in the global scope. And we're changing those variables. And by the way, they don't have to be in the global scope. There could be some wrapping function around this, and we could still be doing the exact same thing, which is changing variables outside. So the first time I call foo, I change the state of y and z forever; permanently I've changed that state. So foo is said to be an impure function. Now, one of the interesting things that I've observed, again, not coming at functional programming from being steeped in the academic tradition but trying to sort of come at it from the reverse, like I'm a programmer and I need to get my hands around some of these tools, one of the interesting things that I've observed is when I look at functional programming, even though they tell you the evil is side effects, you should not have impure functions, everything should be a pure function, and I'll show you pure in just a moment. But what's interesting is there're a whole bunch of impure functions in their programs. A functional programmer will write many times impure functions. But the magic is, this again is just my takeaway, it's kind of like an observational takeaway, the magic is they never leave something impure publicly exposed. So if you have some action that fundamentally needs to be impure, it needs to have some sort of side effect because that's all you have available to you, the way you make it functional again, the way you make it pure is to wrap another function around that to contain all of those side effects within that function, and from the outside world, that function itself, that outer function is pure. So it's kind of like turtles all the way down, you know that principle. It's not actually pure all the way down. It really only has to be pure at the highest most level, and under the covers, it can be as ugly as you need it to be. And we'll see some examples of that. So, if I had this impure function as I show this function on line 1, this impure function, what can I do to make it more pure? How could I wrap it in something that would be more pure? Well, I could---sorry, before I get to that. This is just another example of making those side effect changes. This is illustrating the fact that this change is happening over time. So when y and z have changed the first time, when I call foo on line 13, I'm making a change to the state that's not the same as the state was on line 8. So over time---and the reason why state change, by the way, the reason why state change is frowned upon, the reason why this side effect is frowned upon is that it makes code harder to reason about. It doesn't make it impossible to reason about. Most of you that are not steeped in functional programming traditions have been very successful at writing very, very stateful programs, non-immutable, non-pure programs. You can get the same job done. But, honestly, it does make the program a bit harder to reason about. So what we're trying to strive towards are ways to use some of these techniques in certain patterns that will help us make more reasonable programming, not things that we couldn't do before but things that we couldn't do as reasonably before. So here the fact that I have to know from the outside that foo changes some state, and the first time I call it, I'm going to get a different answer than the second time I call it, that's a complexity in my program that makes it harder to reason about. I have to trace the entire runtime stack to know how many times foo has been called before I can predict its output. If I could rearrange this program in such a way that every time I called the outer function, I got the same result, I don't have that state complexity to worry about. So, pure of course is no side effects. What is a pure function going to look like? Well, I'm going to wrap bar around foo. You notice foo is the same definition, but you notice what I'm doing with function bar declaration on line 1, I'm passing in the entire universe. The whole universe that is cared about, I'm passing in the x and the y and the z state. I'm passing those things in, and that means that every time that I call bar with the exact same universe, I'm going to get exactly the same answer back. And bar could be arbitrarily complex. Bar could be our entire program, and we could have the entire state of our program that we put in, and it churns a whole bunch of stuff and gives us an end result back. And every time we run the program, we're going to expect to get exactly the same result. So what I've done is taken something that's fundamentally impure, the actions that I'm doing with foo, and I've made it pure by wrapping a wrapper around it. Now the contract is not the same. You'll notice that I don't have independent free variables like y and z anymore. I get an array back with y and z in it. As I'm returning on line 3, I'm returning an array with y and z because I don't have a way with the return keyword to return multiple valuables other than to wrap it up in some sort of container. But it turns out that a functional programmer would tell you that's exactly what you want. You want some new value back that represents some new state because I can take that new state and run my program again with the new state if I care to, or I can discard that new state. I have those options. But I don't have to worry that this thing might have been called several times and might have changed state in weird ways that I can't understand. So creating a pure function, eliminating the external side effects, can be as simple as simply wrapping a function around a whole bunch of impurity. Now we could've gone back and refactored foo itself and made foo pure, but I wanted to illustrate to you, and we will see this later in some of our exercises, that the notion of wrapping one pure function around one or more impure functions is actually something you see not terribly rarely in functional programming.
So we do have an exercise, exercise 1 to help to try to drill that into you a little bit if you're not terribly comfortable with that notion of wrapping something around it. You'll notice that exercise 1 is going to say exactly what I just suggested, and it's going to be very similar to that. So let's pull up the README. I'll just orient you to exercise 1 very quickly. Ex1--we're going to pull up the README, and I'll go ahead and pull up the ex1.js as well. So the README is going to say Make a pure function bar that wraps around the implied impure function foo. Let me zoom out just a little bit so we can see. There I have a foo that's doing something very similar to what I just showed you in the slides. It is an impure function. It is making changes to a y and a z variable on the outside. And each time I call foo with a different value, I'm getting an actual different end result. And if I called foo with 20 each time, I would still get different results. I wouldn't get the same result every time with foo(20). So your task, very quickly, is to define a bar, a wrapper very much like the last slide that I just showed you, define a bar, a wrapper that turns this program into a pure program, encapsulates all of that state. I'll give you 3-5 minutes. We'll take a break for just a few minutes while you work on that, and then I'll walk you through the exercise. Just as a reminder as with all my exercises, you'll notice that if you get stuck, obviously you can ask questions, but if you get stuck, the fixed version of all these files is always available to you in that same folder. So if you get stuck or you just want to cheat, you can look at the fixed version of the ex1.
Exercise 1 Solution, Part 1
Exercise 1 Solution, Part 2
Composition and Immutability
Our next topic, Composition, and I know I said we'd try to stay away from fancy-sounding words, so let me try to help explain what composition really means. Composition in this sense is about taking the output of one function and putting it directly in as the input to another function. So in other words, instead of calling one function and then calling another function, we're going to call one function, and its output is going to become at least part of the input for another function. And then the output of that could become part of the input for another function. That's what we mean by composition from a functional programming perspective at a very, very simple basic level. So what would be an example of a program that might need some composition? Well, if I had a function called sum that takes two variables and adds them together, that's a super useful function. You can all add that to your utility libraries now. Underscore--I don't even know if they have that yet. I invented the sum function, and I invented the mult function, which multiplies--even more powerful. I know, silly functions. We got a---I don't have very much room on these slides to show you code, so I have to use simple concepts here. But if I wanted to perform an operation like 5 + (3 * 4), I'm going to have to compose these two functions in some way. I might compose them with state. I might save them into a variable like I'm doing with z. But now you can see that I'm doing sort of an impure thing. I have this side effect state. If, on the other hand, I were to take the result of mult and pass it directly into sum, never saving it into some side variable, then I haven't created an impure action. I've simply composed two pure functions together. So right now this is an impure program because there is some side effect state that's happening. Is it a side effect in the sense that our program is going to return different results? No, because we don't use that Z variable, but there's still a side effect that's happening. Z is still getting changed. That is still a complexity that could create issues for your debugging later. So how can I reorganize this program so that I wasn't assigning to some variable? Well, I can simply pass in the result of mult, calling in mult with 3 and 4, and pass in its result directly into sum. In a functional programming sense, this is kind of a manual composition. So I didn't assign to some side variable state, I just took the end result and pumped it directly into this next program. So the outer program here is pure even though technically at a programmatic sense, technically our programming language has to temporarily store that variable somewhere, but from the perspective of our program, we've written a pure operation. So is everybody following that? That's why these concepts build upon each other. So composition is a way of reducing the amount of side effects we have. It is absolutely true that this is not some magic silver bullet that works everyone because you're not always in control of the design of your function signatures. And if our function signature happened to not work in such a nice way where we could pass in a value like that, or if one of our functions needed to return two values but the thing we needed to pass it to only needed one of those, it gets a little more complicated. Granted, that is absolutely true. This is why we call this functional-light programming. We're looking for ways to develop those instincts. And to whatever extent that you can develop that instinct, it may guide your design decisions for the way you write your functions. You may write your functions assuming that somebody may want to compose that with a different function. That is not only the argument signature but also the return value signature. So if we wanted to create a pure wrapper around this to do this composition for us, we could do it manually. We could call it multAndSum and pass in all of the values, x, y, and z, and simply call mult and then pass its value into sum. That's a manual composition. We could create ourselves a little utility as a manual composition.
Questions on Immutability
Closure and Recursion
Let's now turn our attention to closure. We already sort of mentioned this several times. I wanted to give you a slightly more formal but still very, very informal definition for closure. This maps somewhat to another definition that I've given this in another one of my workshops. But, again, this is all about trying to boil this down to whatever kind of basic takeaways we can get. So closure is when a function "remembers" the variables around it even when they function is executed elsewhere. If a function accesses a variable outside of itself, and then you take that function and you send it somewhere else where it would normally have access to that variable, the fact that it still has access to that variable is closure. I'm not going to go to extreme depth, length to explain all of that. I wrote a whole book about scope and closure if you're interested in digging more into that topic. But we'll just illustrate in a very basic sense how closure's going to be something we're going to use or see being used in our functional programming. So here's an example. I have a function foo, and inside of it, I have a variable called count. And then inside of that, I have another inner function, in this case it happens to get returned. So I'm returning a function back. We've already seen the returning function pattern before, so we're just seeing that again. But you'll notice that I have that count variable inside of the function, and it's referencing a variable outside of itself. Strictly speaking, that's lexical scope. But what's important here is that that function gets transported outside of the function foo, and even when foo is finished, because foo finishes on line 9, and there's microseconds of time that happened between line 9 and lines 11, 12, and 13, and we would normally think about the whole state of foo going away, that foo would finish, and all of its internal state would just get garbage collected. But that doesn't happen in this case, and the reason it doesn't is because that inner function has a closure over that variable count. Just by simply referencing it, it gets a closure over it. And that means that that scope, that state is going to be kept. It's going to be preserved. So we return that function back. We give it the name x on line 9. And then we call it multiple times. And you'll notice that we are actually updating that variable every time. So we're getting out a new value each time. That's an exercise of closure. There are many, many other ways to think about closure. I'm sure you've seen them with your assigning click handlers to functions and referencing variables, setting up timers, ajax callbacks, hundreds of other examples. Here we see a closure because a function is remembering and able to access a variable that was around it, that is outside of it even though we take that function elsewhere, in this case, returning it out. Now something interesting about this closure that I want to point out is I that while this is closure, this is not strictly speaking what you're probably going to see as a very common pattern in functional programming because every time I call x, I get a different answer. So there's an internal state that's happening, which means I still have to track that state over time. I have to understand how many times x has been called. It's not an external state that's changing something where that state can be changed in an entirely different way. So it's not some free-range variable like count hanging out on the outside, it's hidden within a closure, but there're still side effects happening in the strictly pure sense. So the way we're using this function, we have two very, very functional concepts being married together here, but the way we've married them together is not as functional. It's a bit impure. That doesn't make it bad. I use this all the time, and many of you use this in your programming. But you should be aware if you're trying to use a functional concept, this usage of closure is perhaps not as functional, not as functional pure. Now, how could we use closure in a more effective way as a functional programmer? Same principle will apply, closure is the same principle regardless of how you look at it. But how we use it can be different. So in this case, I declare a function called sumX. It receives an x variable. And you'll notice that I'm closing over that x just like I was in the previous slide. I'm closing over that variable, and I'm returning that function. But what's the key difference here? The key difference is that x never changes. Does everybody see that? For the lifetime of my reference of that function, which I happen to call on line 7, I call it add10, for the lifetime of that function, every single time I call it, the x value is always going to be 10. Same principle, still closure, but here I'm not allowing it to make some side effecting change where I'm going to get a different answer every time. This is more functional. This is a technique for applying more functional programming idioms to our programs to make them more reasonable. So here I'm making a function, an incredibly useful function, and another one you should add to your utility library and get rich and famous, I'm making the add10 function, which takes anything that you add to it, like the value 3, and it adds it to the value 10 that it's already saved in its state, saved in its closure. It turns out that sort of technique of creating something sort of as a specialized thing that it doesn't have other stuff, this goes by fancier terms, so bear with me for just a moment. It goes by currying or partial application. Taking a function that would normally need multiple variables and presetting some of those using closure, that's called currying or sometimes depending on how you're using it, whether it's currying or partial application. They're kind of two sides of a Rubik's cube if you will. So there's some crazy complex terminology, but let's not worry about currying and partial application. Let's just worry about this idea that a function can remember the variables that it has around it. And we can use that to our advantage to save some state along with the function. The fact that that state doesn't change over time is what makes this easier to reason about.
Pull exercise 2 up with me. Again, pull up the README and the ex2.js. I'll go ahead and close exercise 1 since we're not using that anymore. Let's look at the README for exercise 2. It says, Define foo so that it produces a function (we've already seen that), and that function needs to remember only the first two arguments that were passed to foo, and then the function that we've returned back, it needs to always add those two together. Parse through that README. Make sure you fully understand what it's asking for. Here's the setup. I should be able to make a function called x that's already remembering the 3 and the 4, and every time I call x, I'm going to get 7 out. Not very many lines of code, but let's make sure you have your brains wrapped around how we can use closure.
Exercise 2 Solution
First off, the README says we're going to need to care about only the first two parameters passed in. The quick and easiest way to do that is to give those names. You could've messed around with the arguments array if you really wanted to, but here I'm going to just cheat and use just the two names, which means if you pass in any other arguments, I'm just going to ignore them. Now I need to return a function. I'm going to be lazy and not give it a name because it doesn't really matter here, but function expressions generally should have names. That function as we see the way it's being used, if we look at the test cases of all the ways it's being used doesn't take any arguments, so I don't need an argument here. And the README said that what we wanted to do with that is add those two values together and return it. So that's all we have to do. Can we see that it's closed over both x and y? The parameters act in this case like local variables that have a value. Those values don't change over time. So over time as I'm calling x over and over and over again, x and y are still always the 3 and the 4 values respectively, which is what gives us that value 7 each time. As silly and stupid as this is, being able to do that at a whim, being able to put a function and wrap it around some state, and maintain that state is one of the most important things you can get if you want to use functional programming techniques, functional-light, if you will, in your programs. Having a mastery over closure is really important. That's why I said at the beginning that it's so fundamental to programming. It's so much the most important thing. I literally can't count how many different places that closure shows up in programming techniques.
Exercise 3, let's pull up exercise 3 and do a tad bit of experimentation with our understanding of recursion. Exercise 3--Take mult, which is already defined, turn it into a recursive function that can work on as many arguments as necessary. So what is mult, how is mult currently defined? It only works with three parameters, x, y, and Z. Line 7 would create a problem for us because it's still going to return 60 instead of 360 the way we expect. You could do this iteratively with a for loop, and if that helps you, turn it first into an iterative solution if that helps. But the exercise here is to ask yourself, What is the multiplication of all these arguments? And, by the way, it's going to look an awful lot like the last slide we just looked at.
Exercise 3 Solution
Another thing we might want to do with a list of values is to compose those values together. Remember I said earlier, so I want to remind you, we're not talking---when I say list composition, I don't mean composing two lists together. We have a method for that, it's called concat. But that isn't as traditionally thought of from a functional perspective. So what we're talking about here is composition of items within a list. Either a full composition, which is what most people mean by it, or sometimes it's just a partial composition. That's also valid. And that goes by the name in functional programming of typically it's called a reduce. So what are some examples of a reduction that we might do, a composition that we might do across a list? Say I have a list of numbers. Remember I had a transform where I multiplied all those numbers by 2. What if now I wanted to say I want to take this list of numbers and add all of them together. I want the summation of all the items in my list. You can express the summation of items in a list as a reduction. It is a composition of values. Now what do I mean by that. I'm not trying to be too terminology heavy here. Composition is when I take value A and value B and I smoosh them together into value C, whatever that means. So for numbers, it might be adding or it might be multiplying or it might be dividing. For functions, it might be passing one function into another function. For promises, it might be chaining one promise off of another promise. There's a nearly infinite number of ways that you could think about taking any two arbitrary values and composing them into some third value. Again for illustration purposes, we're using numbers, but this is a more general concept, composition of values. And we turn to the reduce function to do that. Now, just as a quick heads-up, the reduce function is more complex than the previous two. That's why I did the first two. It's more complex than the first two, and it is possible to use reduce in a non-pure way. In fact, it's possible to do all of these, but it's kind of easy to do non-pure programming with the reduce function. So it's a tool that has more power to it but also a tool that you have to use responsibly if you want to functionally program. So assuming that you're trying to be functionally program-minded about it or, as I call it, functional-light programming, assuming that's your goal, you have to be a little bit more careful about how you use it. So list composition, reducing. What would that look like? Well, I could have a predicate function like mult. And importantly where because I'm doing a composition, that means I need two values. So I'm going to have an initial value, which I'm calling x here, and another value calling y. Now I've named them x and y because in the problem domain of multiplying, we usually think about x * y, but if your problem domain was something different, you might name those parameters differently. The most common name for those parameters to the predicate that's used in the compose option or the reduce function, the most common name is they call the first one the accumulator and the second one the value. So let me explain how that works by way of looking at the implementation of it that I have here called compose. You'll notice compose actually takes three parameters, not two. It takes an array, it takes a predicate, but it also takes an initial value. Why? Because this is still going to be called against every element in our list, but the first element in the list needs to have something to compose with, doesn't it? We haven't done any composition yet. If we're going to pass it into a composition, we need to pass something else in. Do you know what we're going to pass? An initial value. And then those will be composed together, and now we have a running total, which is why this is oftentimes referred to as an accumulator. We have a value now that was the result of the first composition, and then we have the second item in the list. So we take those two and pass them in. And those two get composed. And now we have this as our value and the third item in the list, and those two get composed. So that's how it works. That's why we need an initial value. What would the initial value, the most appropriate initial value be for summation? 0. What would be the most appropriate initial value for multiplication? 1. Everybody see why? Because those are both the identity values in those operations. Zero doesn't make a change to the summation. One doesn't make a change to the multiplication or the division. So great, excellent! What about a function composition? What would be a good initial function? A noop function, which returns everything back that it got. So the more complex your value types, the more you have to think carefully about what would my initial value be? If I were doing a composition of promises, like chaining them together, what would my initial value be? An initially resolved promise for example. So, when we're thinking about composition, you have to also think about the initial value. That's an extra complication. I've got my total here, which because I'm doing math, I can just do mathematics here. I would need to have a more complex implementation if I were doing something non-mathematic. Well, maybe not a different implementation, but I'd probably call the variables different, and I wouldn't call it total or whatever. But you notice that I call the fn function with whatever the current value of total is plus the new value, and then that's what gets assigned to total. So that's how I do that whole compose, compose, compose thing. And then I return total back out. So down I'm calling it on line 12, I just pass in the array 1,2,3,4,5, my predicate mult, and I start with 1 because that's the most appropriate initial value for this operation. It's important to note that this is what we would call a full reduction or a full composition meaning that we started out with a list, and what did we end up with? Something very much not list, in this case a number. It's interesting to note that the reduction here when it's a full reduction like that, it's not really chainable with the other functional programming things, is it? Here's where we start to see the nitpicks that functional programmers have because now we have a breakdown in the consistency of our compositional pattern because if you needed to do some other mathematic operation like a map on this, and the end result was a number, you can't just call map directly off of it. You're going to have to take that and put it in an array and then call map on the array or something. So there's some weird inconsistency. Usually your reduction, especially if it's a full one, usually that's the last step in your chain anyway, so it's usually not a big deal. But there have been places where I've had this issue. So the other way to think about it, and I don't have a code snippet for this, but it is possible that a reduce is not a full reduce. In other words, the end result of your reduction might just be another list. An example, we're going to come back to this later, but while you're thinking about it, an example--what if I wanted to perform a unique operation on a list? What if I wanted to filter out the list and say this list needs to only have unique values? I need to turn it from a list into a set basically. Well, we couldn't use filter. Why couldn't we use filter? What was the one characteristic I said about how filter gets used to solve problems? Remember filter can only make its decision based upon the value itself. That's the appropriate way of using it. There're ways to bastardize it, of course, but that's the appropriate way of doing it is making a decision based on the value itself. Well, we can't unique an array based on knowing the value itself, can we? We need something more context. So actually uniqueing can be implemented more effectively and more traditionally with the reduce because what we could do is our running accumulator could be an empty list, the initial value could be an empty list, and we could stick something in the list only if it's not already in the list. So we could run through the entire original list looking at values, and every time we're about to put one into the new list, we ask, Is it not in the list yet? Is it not in the set yet? That's an example of uniqueing. So that's what I would call a partial reduction because the end result is still a list, but it's not a one-to-one mapping, it wasn't a filtering based upon one individual value, it was something more complex. That's what I meant early when I said reduce is way more complex, more powerful, but there's also a bunch of ways that you can use it in impure ways. So you have to be careful about that.
List iteration, I have the exclamation marks here because I want to remind you that even though this is one of those helpers and it oftentimes get lumped in with the other functional programming things, most people say don't do this one, and you'll see very quickly why. If I made an iterate function like, for example, I wanted to take my array and log out all the value in the array because you notice I'm not actually performing an operation on the list to change something. I'm just performing an operation with the value of the list. Well, what's the only way that doing that would mean anything? It's if my predicate had a side effect, right? Some side effect, whether it's a console.log or changing it in the original array like an in-place change or something like that. The only reason that a list iteration would make any sense and do anything useful to us at all is if the predicate had a side effect. But you can probably spot why that's not the favorite tool in a functional programmer's bag. So most functional programmers will tell you never use the forEach because it's all about side effects. Now the problem with that is that sometimes I do want side effects, which is why I call this functional-light programming because I don't want to be restricted to only doing purely functional stuff. There are times when I want to iterate over a list and have some side effects, and that's why I use forEach. It is a tool in my bag. But I use it knowing that it's not the same thing as my other functional tricks. It's different, and I use it that way on purpose. There are people that will tell you, No, no, no, you should iterate with a map. Can you iterate with map? Of course you can. You can iterate with map where its predicate returns the same value. Whatever function you pass as that predicate, if it returns the same thing it got, that's the same thing as iterating, right? That's what a functional programmer tells you that that's what you ought to do. No, that's not what I would do. ForEach is the better tool if I want to produce a side effect for each thing. So that's the forEach function. Any questions about our list operations? Those are other tools that we have. So if we go back over it, we've gone over a whole bunch of these little basic things. We haven't really fully covered how we put all of them together. That's what exercise 4 is going to try to bring some of that to more concreteness, and we'll do that after our lunch break. But I just wanted to point out that we've covered quite a bit of ground here in just a couple of hours. We've talked---from the very beginning, we talked about pure versus impure and side effects. And then we talked about functional composition and how we can compose the function to take it from impure to pure. We can compose two functions so that we're not creating impure side effects. Then we talked about immutability, and we talked about the value of, not just the immutable value but the immutable way that use the value, that's the more important part to a functional programmer. Then we talked about closure, keeping state inside a function. We talked about recursion and why recursion can be useful, a graceful way of expressing our programs. And then we talked about these list operations--transform, exclude, transform's called map, exclude's called filter. What's compose called? Reduce. And finally iterate is called forEach.
Hopefully some of those ideas of functional-light programming have been stirring around in your head. Closures and recursions and immutable value usage and things like that. We're going to finish up our workshop with an exercise, exercise 4. This is basically the last thing we'll do with this workshop. And the goal here is to sort of walk through some steps that make you practice that and to borrow the term to compose those ideas together a bit into some---it's still silly problem solving. We would be here all day if I tried to give you like a real program and tried to in context do that. So they're still silly, simple academic-like examples, but hopefully they're a little more concrete, and we can start to figure out how these things fit together. So we're going to start with this empty ex4.js. And rather than this being an exercise that I just turn you loose on for the next 30 minutes and have you do yourself, we're going to kind of make this a little bit interactive. I'm going to say some stuff, and I'll stop for a minute and let you type. And then I'll say some more stuff, and that kind of thing. So this'll be an interactive thing, roughly probably 30-40 minutes-ish of us discussing, and we'll figure out when we've gotten all of the usefulness out of the exercise, and then we'll wrap up. So the first thing that we're going to start with, and if you follow along with the README, that's fine. If you want to jump ahead, you may miss a few of the points that we're making, so I would encourage you maybe not to do too much of the jumping ahead in the README unless you feel you're ready to go on to the next step. But you can follow along with the README because that's basically what I'm going to do. So what we're going to start out with---and one other thing to kind of give you a heads-up on. This is another one of those things that in functional programming I've observed from the outside. I'm coming at this all from the reverse angle. I didn't start out learning functional programming. Douglas Crawford has a great quote on this topic, I don't know if it's originally him, but he has a great quote on the topic that the problem with functional programming is that as soon as you understand it, you stop being able to teach it. And there's humor there, but there's also some truth there because as soon as you really understand it and you get steeped in all the academic tradition and the terminology, it's really hard to switch back. So rather than go that direction and be ineffective at teaching, my goal is to come at it from the back-end, from the reverse. So I don't whether---I'm not going to get certified as a functional programmer any time soon, but hopefully some of these things are starting to fit in a little more concretely for you. But one of the things that I have observed from that back-end perspective, coming at it from the reverse, is that there are sometimes when an abstraction gets created, and you look at the abstraction, and you think, What even remotely prompted somebody to come up with that? And the best metaphor I can give you for it is if you remember back maybe in high school or whenever when you took an algebra class or maybe it was a calculus class or something like that, but you'd be working along with some problem, and you're trying to solve for x on one side of the thing, and it's not clear how you're going to---you get to some point, and the teacher's working through the problem with you, and it's not clear looking at it that there's an easy way to isolate the x to solve for x or whatever. And all of the sudden, she's like, Well, if we multiplied by y-cubed on both sides, and you're like, What? But she does it, and all of the sudden, it just magically makes everything---it unsticks the log jam and then you can move on. And that used to drive me insane. How did you just pull this rabbit out of thin air that just magically worked? What was it that did that? And I don't have a terribly great answer for it, but it is to say this, I think functional programming, my perspective, one of the most important techniques that you can get or one of the most important skills that you can get at functional programming is developing instincts for things. And that's basically what my algebra teacher said was, I've done it enough that when I see a problem that kind of looks like this, I just remember that I can invoke this pattern, and it helps to unstick the solution. So really functional programming is kind of like that. Sometimes you invoke things out of thin air that don't logically flow directly from what you just did, but they unstick you. They create an abstraction that unsticks you and makes a solution to a problem work. So there're going to be a couple of these things where I'm going to pull something out of thin air, and you're going to be like, Why are you passing that function in that way? It's really more based upon those instincts. That's really what I'm trying to get across with this whole workshop is to help you understand what those basic principles are in functional programming and let you try to weave those together into instincts. When you look at a problem, I want you to have the instinct to say, Oh, I've seen that before. That's a transform. I know how to do that. I can pull out map. Or even if it doesn't look yet like a transform, but you're trying to figure out the cleanest and most straightforward way of expressing a solution to the problem, maybe something will go off in your head, and you'll say I have an instinct that this is probably something I could do with map. I can wrap this value in an array and do it with a map, and that will make more sense. It's those instincts. It's those pulling solutions out of thin air thing that seems to be how functional programmers are so good at what they do. That's just an observation that I'll make, and we will see a little bit of that as we go through.
Exercise 4 Solution, Part 1
So to start out with as the README says what we're going to have you do is write two functions, I don't care, just call them foo and bar for now. And each one of those functions should return a different number. So have one of them return 10 and one of them return 42. I don't care. Pick your favorite numbers. But literally just have it return a number. Your functions probably look something roughly like that. Now there's something interesting in functional programming, there is something interesting about the nature of having a value wrapped in a function. Kind of like I said earlier that there's an instinct to say, well, if given the option, I try to stuff values into a list because I know lists can---I can do certain sorts of operations on lists that are quite handy. In the same way, you also sometimes will say, If I can wrap a value in a function, and I know if calling that function will give me a value, that starts to produce patterns of more composability in our programs. It's easier to compose two functions together that return numbers than it is to simply compose the two numbers. That's what we're getting at. So this was one of those pull-out-of-thin-air things. I didn't tell you to just write the two values 42 and 10. I told you to wrap them in functions that produce those values. It won't be clear why yet, but as we go along, maybe you'll see a little bit more of a reason why. So item #2 in the README. What we're going to do, we're going to write a function, which we call add, and needs to take two numbers and return the result. We'll just leave our function bar and function foo along for a little bit. Add needs to take in two numbers and add the two together. So let me give you your 15 seconds to write that. I'm going to take away points from you if you pulled out underscore and you're like, Oh, there's some underscore function I can do for this. We're writing our own code here. We want to learn stuff. That was just a little dig. I apologize. I'm sure there probably is an add function in underscore. Add(x,y) is what I'm going to name them. How does add work? Pretty straightforward, x + y. Not rocket science here. This step progression that I'm going through is intentional. It's intentional to lead you two places where you start to get some of those instincts, so don't jump too quickly beyond what I'm doing and think that it's too remedial. This is a very carefully, intentionally designed exercise. The next thing we're going to do, I could observe that an add that takes two numbers, but an add that takes two functions would be even cooler. So I'm going to make an add2. And that function needs to take in two functions, and it needs to call both of those to get their values out and then return the sum. So your function might look something a little like this. I'm going to call it fn1 and fn2, and it's going to use the add function. I'm going to call fn1, I'm going to call fn2, pass those to the add function. Again, not totally clear to you why I decided to make add2 use add rather than just do its own addition, but in functional programming, you see this all the time. Every abstraction that you define is a combination, it's a composition of some lower level abstractions. We're always looking for that turtles all the way down sort of path to defining our utilities. So why repeat the plus operator when I already have a function that can compute that? Now I know this stuff seems totally remedial. It's about to get, here in a couple of steps, about to get a little more challenging, so just hang on with us. So add2 passes those values in just like we did. Now let's move on to item #4. Those two functions that we defined that had---before we do that, let's just verify that we know how we can use this. How do we use the add2 function? What would the call set look like? Add2, what am I going to pass into it? Foo and bar. And what am I going to get back out in this case? 52. So everybody make sure we understand that. Now, let's take the foo and bar functions and replace them with a new function. Now I don't care what you call this function. I'm going to still call mine foo, but if you have a different, better name that you like, the name doesn't matter here. But here's what this function needs to do. This function needs to take a value, a single value, and it needs to return back a function that when that function is called will give you the value of it. So take a moment to figure out how you would define function foo in that way. So your function foo might take a value like x, and what does it need to do? It needs to return a function that doesn't need to receive any parameters. And what does that function need to do? Return x. There's our closure again, right? We're producing a function that will remember that x value, and every time we call it, it will give us that x value back. Here's a little hint preview towards the async workshop--that's called a thunk, and we'll talk about thunks. But when we get to thunks, I want you to remember back to this idea of a function wrapped around a value. So now we don't have a foo and a bar anymore like that that we can just pass in. What do we need to do to call add2 now if we want to still produce our 52 result? What do we need to do? Foo(10) and foo(42). Everybody see that? I'm actually calling those functions and producing functions that get passed in.
Exercise 4 Solution, Part 2
Do we not call add inside that? We are calling add right here on line 12. Now it starts to get more interesting and little more challenging. We'll see how close you've been paying attention. I now want you---because we have this add2 function, which takes two functions, and if I wanted to be able to take three functions, I could make an add3 or an add4 or an add5, but I want you to make an addn. I want you to make a function that takes a list of an arbitrary number of functions and adds all of those function return results together, but that function has to use add2. There're a variety of ways that you could define your addn. Let's just leave the add2 here so that we see that in comments. There're a variety of ways to define the addn that we're talking about. But what I want you to do is instead of it receiving parameters, it's going to receive a single array, so I'll give you that hint. It's going to receive a single array. That array is going to have one or more, actually zero or more (but we don't care about those trivial cases) functions in it. And you need to go through and calculate all of the sums of all those things using only add2. Of course, you're going to have to call add2 quite a few times, right? So you might choose to set this up as a for loop, and that might be the first way that you want to write it. So, try it with a loop first. Try yourself setting up your addn function to calculate it with a loop. And then you'll notice the README says after you've done it with a loop, try it with recursion. And, finally, you may try it with map and reduce. So here's what I'm going to do. I'm going to stop talking for like 5 minutes. I'll let you try it with a loop first and maybe move on to your recursion attempt at it. But then we're going to talk through looping, recursion, and then how we might use map and reduce. So addn--I might have slightly misspoken before I gave you the work break. We want this to receive an array of values, not an array of functions, but either way, the solution's going to end up being mostly the same. So if you sent in an array of functions, then there's just one less step to do. If you're going to send in an array of values, there's one extra step to do. Either way. Let's assume an array of values is what we're going to do, so let's assume we're going to call something like addn, and we're going to pass in the values 10 and 42 and 56. And maybe we want another value like 73. So we expect that to give us 181 is the sum. How are we going to do this with a loop, with an iteration? Well, of course, we're going to need to keep track of it. We're going to need an accumulator. We'll call it sum and start it out at 0. We're going to say for (var i=0; i less than arr.length, i++). I'm going to take the entry in array, and I'm going to call it with foo so that's going to produce me a function wrapping around the value. That's the step you don't need to do if you're assuming you've already received functions, so either way. Actually, I did have to do the accumulator. So if you already have a function, then that step won't be necessary. But what we're going to have is the sum is going to need to call the function that it gets back, so what we're going to do is say---I'm sorry, not call the function, we're going to say add2, and one of the things that we're going to need to pass in is the sum, the current sum wrapped in a function, so we're going to need to say foo(sum). So that particular part is optional if you're already passing in functions. This part is not optional. And sum is equal to add2. You could've also stepped through the loop two at a time passing two elements in the loop in and getting their sum and just doing sum plus equals and add2 and passing those in. That's also an equally valid way of solving it with a loop. But at the end of the day, we can simply return sum. Well, what about if we wanted to do this with recursion? In recursion, we're looking for a base case. And what's our recursive definition for the addition of a list? The addition of a list is the addition of the first two items plus the addition of everything else. So it's the item plus everything else, and then it's the item plus everything else. That's how we add those two together. So our base case would be if the array that we've been given is length 2 because then we know what to do with that. So arr.length is 2. And you could for good measure if you wanted to say less than or equal to 2. But if we were given an array of at least in this base case of at least having 2 in it, then all we need to do is call add2, and if we've been given a set of values, of course, we're going to need to wrap those. So we're going to need to say foo(arr(0) and foo(arr(1))). You don't need those foo calls if you're passing in the functions already prewrapped. What's our recursion case? We're going to say return, and we're going to call arr(0) as a function, right? And then addn, so this is addn. We're going to need to have an array that has---sorry, we're not going to call it. We're going to need to add in an array, pass in an array. The first element in the array is going to be what was in array position 0, and the second one is going to be everything else in the list. So, that's going to be arr.slice(1). Now, addn returns us back a number, so we're going to get numbers and numbers and numbers coming back and back and back to us. If we're dealing with---I'm getting a little bit ahead of myself, so I'm going to try to not jump too far ahead. I don't mean to pass these separately, that's not what I meant to do. I meant to call apply. That's why I'm getting ahead of myself. So we want to pass in a single---we want to spread out a single array with that as the first value and then add2 in it. I'm sorry, getting a little bit off track. I think I've got my brain wrapped wrong, so let me just double-check here. There's that function. We'll start back here. I'm going to call addn with a single array where the first element in the array is a function that returns add2 with arr(0) and arr(1). So that's our first element. That's an array with a single element in it. And we want to concatenate that with everything else that's at the starting position 2. So I've taken the two items off the front of the list, passed them to add2, and replaced them with a single calculation for it, and then I'm going to keep doing that recursively. So I can concatenate that with the rest of the list, arr.slice(2). That should be what I show. So, single array with a function in it. That function is going to calculate the addition of the first two elements in the list. And then we are going to pass the rest of the arguments in. So if we do that the first time, we're going to go from a list of four elements to a list of three elements where the first item in the list is a function and the rest of the elements are either the values or the wrapped functions depending on which one you did. So we've gone from four down to three, and then we're going to do it again where we take that function and we add in another function, and so we go from three down to two, and then we're going to hit this base case, which is just saying simply pass in those functions, and we get the value back out. On 22, couldn't you call foo? The reason we don't want---if you call foo, you're going to be calculating the add2 now, so you could do it that way. What I'm trying to show is creating a function that will sort of defer the calculation until the final endpoint. So we can also observe as I have done here in the comments that---I mean, the other way of writing this is we only need to do the recursive call if the array length is already greater than 2.
Exercise 4 Solution, Part 3
Either iteratively or recursively, this problem can be solved. Finally, let's try to solve it with map and reduce. So with map and reduce, what we can observe is that the calculation--- the changing of a value to a function is a transformation, is it not? So I can take arr and simply call map with foo as my function. That's going to make sure to wrap all of my values into functions. And now I want to perform a composition of all those functions. Does everybody see that? How do I do a composition? Reduce. Right. What does reduce take? What's its first parameter or its first argument? The array. It takes a predicate function, remember, because it's operating on the array as a prototype method, so we don't have to pass the array in. It's going to get that context already. So the first argument to .reduce is going to be the predicate function, the function that's going to do the work. So we'll define that function. We know that function's going to get two arguments. I'm just going to temporarily call them x and y, but we're going to rename them. So let's do our function there. And then what else does it get? Initial value. It gets an initial value. What initial value should we use here? Well, there're a couple of different ways to think about this. We are doing a map across the entire list, right? So that's one way of doing it. But the other way of doing it is to slice off---let me just double-check to make sure I'm not---I'm trying to keep close to---my brain works differently when I solve this, and I'm trying to make sure to not get too off track of your solution file. So, we could do slice(1) because we want to only perform the reduce transformation across the second half of the list. We have the first item, and we want to deal with the first item, and then whatever happened, we're going to reduce across the second list. So if we did it that way, what's our good initial value? Possibly arr(0). Arr(0). You see how I did that. I took the first element of the list and made it my initial value and did a reduction against the rest of the list. That's just the shortest way of doing it. That may not always be the way you want to do it. You might have just started it out with a good initial value like a function that returns 0, so you could have said foo(0) here. But now we want to ask, What is our accumulation? And the typical names for these, as I said, the typical names in the reduce predicate are accumulator, acc, and current. That's how most people will call them. I don't love accumulator and current, so I'm going to actually use previous and current because I think previous is a little bit easier to understand than accumulator. But let's analyze what is going to happen with this first call? With this very first call, what's going to happen is that I'm going to---let's just for simplicity's sake go ahead and pass in these as functions so that I'm not having to do that map there. Of course we know we could do the map, but let's just for simplicity's sake pass those in. So, array of 0 is what? A function now. It's a function, right? And cur is going to be array in the 1 position, which is also a function, right? So I have two functions. How am I going to compose these two functions together? I'm going to create a new function. That's one of the most common ways that we compose functions is to wrap the two of them into one bigger function. I'm going to create a new function, and I'm returning it, which is then going to make it be the next previous. So what is this function going to do? It's going to actually call add2 with previous and current. So we're not calculating that sum now, we're just making a bigger and bigger wrapped function. The end result of our reduction here is not to have a sum, it's to have one giant function that when we call that function, it's going to calculate all those intermediate sums and put them all together. We're making a bigger and bigger function. The first time we make one function, and we replace the first two elements in our array with one function that's set when it's called, it will calculate the sum of those two functions. The second iteration of our reduction, we take that composed function plus a function in a list, we compose those two together. The third iteration, we take a bigger compose and the third item in the list, and we compose those until we get to the very end, we have one giant function. So if I just simply said return arr.slice, we would not have what we want here in the sense that we would not have calculated the sum yet. We simply would have produced a function. So what do we do? We then call that function. And that will call it and call it and call it and call it, however down it needs to go until we end up with our actual sum result. _____ Yes? There're a couple of questions here? Is it possible to leave off the initial value in reduce? If you leave off the initial value, it's a little more complicated. It will default to undefined, which might be an okay initial value depending upon what you're doing. That might be an okay initial value. But you should also be aware that if you leave off the initial value and your list has one or fewer items, that throws an error. So it's not really ever a good idea to leave off the initial value. You'll almost always want to give it some useful initial value. He's kind of just having a little---he's kind of wondering why you're--if you maybe could explain again, removing the first value and why are we doing the original slice I guess. The reason we do the slice(1) here is because what we want is to perform a reduction where we have two items, and we could have done a slice---not done a slice at all, and then we could have had an initial value here of foo(0). If I did foo(0), does everybody see that that would have been a suitable initial value? It would have been a function that when called just simply returns 0, which is going to have a noop. So I could have done my solution like that. Operate a reduction across the entire list where my initial value is simply basically a noop function. That's one way of doing it. Why is the other one preferable? One less function call wrapping layer. So all I was trying to do, I've already shown you starting out with an initial value, here I'm saying---actually, we already know that array(0) is one of our---is a suitable initial value, which is why we put array(0) there, and we do the reduction across the rest of the list. Either way is entirely valid. (Working) It's important to keep track---I know that you kind of get in this nesting thing, and it's hard to keep track of the reduce predicate. That's the one right here. It returns a function. So every time we pass in two functions, we get a third function out that's a composition of those two functions. And then when we have another function and another function, and this one is already a composition, we get a super-composition of a composed function with a regular function. And then over here on the fourth item, we get a super-super-composed function with a regular function, and we've just made functions wrapping together up the list. So if we were to unroll all of that, if you were to print out this thing, and we passed in a list of four items, we'd have four levels of function call. This function returns this function call returns this function call returns this function call returns the add2 of two values. And it would just simply call all those functions and unwrap them. So this isn't recursive. This is simply unrolling recursion into a call stack as many levels deep as we need. And just as a little side trivia note, has anyone ever heard of the Y combinator. The Y combinator is a general pattern for doing exactly that. I don't fully understand it, but I know that the Y combinator takes recursion and unrolls it to a progressively wrapped function loop, so we could have solved our problem exactly like we did here. We could have solved it with the Y combinator and got the exact same result. We would have had the exact same set of functions wrapped in other functions wrapped in other functions. That's how Y combinator gets around recursion or gets around the---the problem with recursion is allocating all that new stack depth. The Y combinator gets around it by doing fixed proper tail call function calls inside of function calls inside of function calls. So we make sure to call that function if we're going to end up... Let's just double-check this. This is my ex4.js. I'm going to---actually, let me just copy it in the console. You can run it in your browser using the HTML file if you'd like. I'm just going to run it in the browser and make sure I didn't miss anything. What did I do here? I forgot to point out that we're supposed to be passing in an array. Oh, by the way, ES6, how do I get it back to where I do want to pass them in as individual ones instead of as an array? I could've just done the dot dot dot. That's all I needed to do there. So if you want to have a call signature where you don't pass in an array, just declare your call signature as basically gathering together all arguments into an array for you. Let's try it again and see if I didn't mess something up. And there we get the 181.
Exercise 4 Solution, Part 4
I apologize if it got a little muddy because I started my brain down a different way of solving this, and I got you confused. So hopefully we backed up the stack enough to show this. But the point that I want to get across is that the expression with a reduce, and we could've taken care of this part with a map, but we just backed that part out. I was going too far down to the complication tree. But a single reduce here replaces the complexities of recursion or iteration for us, just simply knowing how to use that tool. It's like if you walk into a workshop, and I've got a hammer in one hand and a mallet in the other. They both fundamentally can be used to swing things and drive some nail-looking object in, but you're going to use a mallet differently than you're going to use a hammer. You're going to use them for different kinds of nails. So that's the distinction that we're making here. We're not talking about the difference between a hammer and a screwdriver. We're talking about the difference between a hammer and a mallet. Recursion was an equally reasonable solution, but the way we ended up expressing it might have twisted some brains. I hope that what I'm showing you here is being able to use functional programming oftentimes can give us also a graceful solution even when recursion doesn't give us a graceful solution. Yes? Someone still wants you to kind of explain---he doesn't understand why the first value had to be removed? Look, let's not do it. If that helps, we're going to do a reduction across the entire array, but we need a good suitable initial value. So tell me what my good suitable initial value is going to be for arithmetic sum. It's got to be 0. But I can't just pass in 0 because everything else here is functions. So I have to pass in a function that will produce 0, which is how I call foo(0). So if that works a little bit better with your brain, I feel weirder about that solution, which is why I didn't lead with it. It feels weird to me to create a function only for the purposes of returning a 0. What I think is that something got misaligned when you started applying map, like my brain started to go, okay, this is what we're doing, and now --- That's why I took the map back out because I realized I was going down the wrong rabbit trail. That's my fault for confusion you. --- it didn't quite reset enough right now. I apologize for the confusion. Trying to pop the stack back to look at, I could delete the file and start over, but trying to pop the stack back to thinking about passing this in as a list of functions rather than a list of values. I just got myself one step ahead because our next step is going to be to generalize this to where I can start out with a list of values. But I wasn't supposed to do the map so early. So my fault, apologies. Does this one perhaps make more sense? This expression of it where we're not doing a slice of the array, but we do need an initial value, so here's how I produced my initial value. This will work exactly the same way if I pop this out, drop this in. We're still going to end up with 181. Fundamentally, we ended up with one extra function call there because we made a function call for 0 when in reality we don't even actually need that. We know the properties of our problem are that the initial value really could just start out as the beginning of the list. There's more than one function call because it also does another iteration. Call foo, and then there's---anyway. Alright, I'll take your word for it rather than going down the rabbit trail. I've probably already confused everybody enough, so I apologize. Anymore questions about the reduce? On the live stream on the chat, did we answer those questions about difference in solution. I think we---they're saying they're good there. Some people are saying that their browser may not have the ES6 features turned on. So if your browser doesn't have the ability to do the dot dot dot, take that part out and just put a bracket around that list, and it'll work the same way. So now we get to where I was going to talk about map. I just did it too early. So now if I start out with a list of functions, that's great. But if I start out with a values, and I want to pump those things into an addn, addn is expecting a list of functions, not a list of values. So if I started out with a list of 10, 42, 56, and 73, how do I make that into a list of functions? You map it with foo. I map it with foo. Do you see why foo is a suitable predicate for us? Look at the foo function. What does it do? It receives a value, and it transforms that value by what? Wrapping it in a closed over function. That's a valid transformation for a value to wrap it in a function that produces that value. So that's why foo is a suitable transformation predicate for us. So we would now have at the end of that an array of functions. So that value could be passed in to addn. And that's where I was actually trying to go earlier. That's a suitable array. And because that's an array, we can either take that dot dot dot off or leave that one there if we want to and simply spread that array out. Either way. You're not always going to be in control of all the signatures of all the functions you work with, so part of the point that I'm trying to make here is that there are facilities available to us to make the adaptations between different signatures.
Exercise 4 Solution, Part 5
Kyle is a freelance developer based in Austin, TX. He runs several open-source projects (such as LabJS), writes books, and speaks at meetups and conferences.
Released22 Feb 2018