What do you want to learn? Leverged jhuang@tampa.cgsinc.com Skip to main content Pluralsight uses cookies.Learn more about your privacy Asynchronous Programming in JavaScript (with Rx.js Observables) by Jafar Husain This course will teach you to become a more effective JavaScript programmer by showing you how to master asynchronous programming. Start CourseBookmarkAdd to Channel Table of contents Description Transcript Exercise files Discussion Recommended Building Blocks Introduction Today the topic is asynchronous programming in JavaScript. First a little bit about myself. I'm the cross-team technical lead for the UIs at Netflix. I'm the architect of Netflix's UI data platform, Falcor.js, and I'm a member of the TC39. Netflix technically is a member and I'm a representative there on the TC39, which is the JavaScript standards committee. Who's heard of that, by the way? I'm just curious. A few folks? Okay. They're the standards body that designs the JavaScript language. And I've been about 16 years in the industry. I formerly worked for Microsoft and GE. Today I'm going to teach you how to solve big asynchronous problems. Hard problems, the kind that you run into when you build very, very large JavaScript codebases. But we're going to do it just by thinking differently about events. Probably everybody in this room has worked with events, DOM events. I'm going to show you a different way of thinking about events, really a fundamentally different way of thinking about events, that's going to make asynchronous programming problems that seem really, really hard suddenly seem a lot easier. So asynchronous programming seems hard. I'm sure a lot of us have heard of callback hell, or we've tried to structure large JavaScript programs with, you know, asynchronously, and it certainly seems to introduce a lot of complexity, doesn't it? To take a look at, you know, some examples here of the types of complexity you're going to run into when you start building asynchronous programs, you have race conditions, right? I start two actions, they're running concurrently, and I'm sort of hoping that one finishes before the other, or I want to at least enforce that one finishes before the other, right? Memory leaks. Anybody ever hooked up to a DOM event before and then forgotten to unhook their event, and if the app runs for long enough, you slowly start to see the memory go away? Complex state machines. This is one that I don't think as many JavaScript developers think explicitly about. This is actually the source of a tremendous amount of complexity when you start building asynchronous programs. And uncaught asynchronous errors. How do you deal with error handling? I mean, when we are in JavaScript, we've got try catch for synchronous errors, but as soon as we start doing asynchronous programming, try catch doesn't help us at all. And so JavaScript provides no facilities to deal with asynchronous error handling. And so if you find it hard, you're not crazy. It is hard, right? So let's take a look at a concrete example of some of the things I've just talked about here. So here at Netflix, we used to write code like this. We, this is sort of very similar to the code that we used to write in order to play a movie. So when you started playing a movie, you would, for example, this function here would accept a movie ID, it would accept a cancel button, and it would have a callback which would invoke when that movie was successfully playing. So I want to call your attention first and foremost to the state. So really what's going on is two concurrent things. The player in Netflix is asynchronous, and so when you ask it to play a movie, first we've got to start loading up the player, and that's an asynchronous process. Another asynchronous process that takes place is going to the server and authorizing you to play that movie. So we have to confirm that you're logged in and you're who we think you are and you're able to play that movie. So here we got two asynchronous actions, and we don't want to do them one after the other because that would be slow. So we want to kick them both off at the same time. So this is the race condition, concurrency issue we talked about before. In order to figure out, every single time one of these operation finishes, we need to check if the other one's finished, and if they both finish, we want to do something, in this case, play the movie. So what I've had to introduce here, I want you to notice, is variables to track, in this case, whether the player's initialized. So player.initialized. That's a boolean variable hanging off the property. Off the player, excuse me. We have to check that whenever the movie ticket is successfully retrieved from the server. And so I've got these two variables. One's tracking whether the movie ticket's been authorized, the other trackings whether the player's been initialized, and every single time one of the actions completes, I have to check the other one. So over time, if you try and do this, if you use this approach to tracking whether asynchronous programs are done by introducing state into your program, you'll find that over time, you've got all these variables and all these moving pieces, and it's really difficult to keep them set to the right values, right? And what happens if something goes wrong, God forbid? Then you have to reset them to the previous values. This is actually the real source of complexity, I think, in most large asynchronous programs. If you start using state to track your async programs, you're going to find it doesn't scale very well as your JavaScript codebase scales. The other issue here is error handling. Now, because an error can happen in either of those two actions, initializing the player or authorizing the movie, I have to make sure to clean up after myself and make sure that I, A, forward that error along, but also, if possible, cancel any asynchronous actions that are happening. I don't want to send a request, I don't want to keep a request sent off if, you know, there's no reason for it to be sent off because one of those two actions was errored. For example, I couldn't authorize your movie ticket or I couldn't, for whatever reason, initialize the player. Now, I've deliberately left a bug in this particular program. If you notice here, I'm hooking up to the cancel button an event listener as soon as you try and start playing, because what if you decide you just want to hit back or you want to cancel playing? Well, we'd like to stop this action of playing and then sort of cancel the whole thing. Notice right here this line. Can anybody spot the bug in this program? - I feel like you're missing your close paren, right? - Sorry? - Are you just missing a close paren? - Oh, that's an entirely different bug. But yeah, I am missing a close parameter, so this will not compile. However, the actual bug I'm trying to get at is I'm not, I'm forgetting anywhere to unhook from that event handler. So I've hooked up an event handler, and then you hit cancel, and then I just stop the function. And so the next time you try and play it, I'll hook up another event handler, and so on and so forth, until gradually the program starts to lose memory. And so that's another thing. Remember if I hit an error, I also need to remember to unhook any event handlers out there. It's a very complicated thing to do to clean up after yourself when you have an error. Well, when we're doing synchronous programming, try catch does all that for us, right? It unwinds the stack and any variables we've allocated are all cleaned up. In asynchronous programming, we're on our own. So I'm going to show you guys today a pretty radically different way of building asynchronous programs, and it's actually going to involve not much more than a few flexible functions that you guys are going to use again and again. In fact, these may some, may be actually some of the most flexible functions you will ever learn, the ones I'm teaching you today, and you're going to be able to use them for asynchronous programming as well as other things. JavaScript Tutorial Who is familiar, at least, with JavaScript 6, or what's sometimes called ES6? A few people here, okay. The language spec for JavaScript 6 is technically called ES2005 nowadays, has just sort of been finalized and you're going to start to see these features in browsers, being introduced in browsers. But the one that I want to call your attention to today and the one maybe if you've got a late, really early, a really new version of a browser, you might actually be able to use, is JavaScript now has a new shorthand syntax for functions so that you can take this and convert it into this. Much, much nicer. So today we are going to be writing a lot of functions, so I happen to know the latest version of Firefox supports this, so if while I'm talking here you want to go and download Firefox, because later we're going to be filling out interactive exercises, I think it would actually help you because you can use this arrow function there. Now, you can always use Chrome if you want to or whatever you're comfortable with. I believe the Canary of Chrome may also support this already. So shorthand for writing functions. I'm just going to go through that one more time and get rid of all the boilerplate and just write it like this. So we'll come back to this, but yes. So that is the end of the JavaScript 6 tutorial. That's all you need to know for this particular presentation. So first function of six that I'm going to be teaching you today, for each. Who here has used for each? So you'll notice the next few functions I'm going to be showing you are all on arrays. In fact, today, we are probably not going to get to covering any asynchronous programming whatsoever. We are going to be doing what I like to call wax on, wax off exercises. We are going to learn something which might not seem as if it's related to asynchronous programming at first, which is how to program without loops. Let's say today that if I was to say you can't use any loops when you go into work on Monday, could you still get your job done? No while loops, no for loops, no do while loops, no cheating. How many of you feel like, yeah, I could probably get my job done. Couple people out there. But generally I think most people are pretty comfortable using loops, right? We might be lost a little bit if our programming language didn't have loops. Well, when you start doing async programming, that's exactly the world you're in. You can't repeat an asynchronous function with a loop. And I'll explain a little more about that later. But that's effectively the world that we're in as soon as we start doing large asynchronous programming, right? Large asynchronous programs. Loops become useless to us. So we need to find ways of repeating actions, right, and processing streams of data as they come in without using loops. And these, this is what I'm going to be teaching you guys today. Programming without loops using just a few simple functions. So what for each does is it takes a function and it applies that function to every item in an array. In this case, I'm just printing each one of these items to the console using function, using console log. That's relatively straightforward. You can, if you want, you can have a JavaScript interpreter up right now and maybe in the console, you can play around with that very example yourself. If you don't, unfortunately, if your browser doesn't support the arrow function, you got to write it the long way, function x return, you know. Next I'm going to introduce you to the map function. The map function's similar to the for each function except the difference is what map does is it takes a function and applies it to every item in a collection, creating a new value, and then creates a new array with all the results of applying that function to each value in the array. So map does not change the array. And I want to call your attention to this notion, this thing very specifically. None of the functions I'm going to be showing you today change arrays. What they do are create new arrays. So what I'm going to do is I'm going to apply this function to every item in the collection and create a new collection containing the results. So if we run this, we get two, three, four. So it didn't change the original array. We actually created a whole new array. So if you ever want to transform every item in a collection and put it into a new collection, map's the tool for the job. Next we're going to talk about filter. Filter is very similar to map. It's going to apply a function to every item in the array. The difference is this function is a test function. Instead of transforming the value, it's going to apply a test to every single value, and only if that value passes the test will it make it into the new collection. So here if I run this, we actually get an entirely new array. We didn't change the original array. I'm going to keep hammering this point because it's really important. We haven't changed the original array. We just created a new one with only those items that pass the test. Does that make sense to everybody? Great. I, there's a quick question here. At the end of the current section, will ES for, I guess I can talk about this later, but for of lists. We'll talk about that pretty soon. ConcatAll. So concatAll is not in JavaScript. It's a function we're actually going to write later on and add to the array object. But what concatAll does is very simple. It takes a multi-dimensional array and flattens it by one dimension. So in this case, we've got a two-dimensional array, and if I run this, it's going to get flattened into a two-dimensional array. Now, I want to call your attention to something. It's going to be important later on. Notice what happens to an empty array in a two-dimensional array when it gets flattened. It just, whoof, disappears. You take an empty collection inside of a two-dimensional collection and you flatten it, that empty collection goes away. We'll see why that's important later on. So quick review. Map, filter, concatAll. One thing I want to mention is concatAll will not recursively flatten deep collections. If you have a three-dimensional collection, it'll turn it into a two-dimensional collection. If you have a five-dimensional collection, it'll turn it into a four-dimensional collection, and so on and so on. It doesn't recursively flatten n-dimension collections. So map for transforming every item into an array, every item in an array into a new collection, filter applies a function to every item in an array and only if it passes the test does it make it into a new array, and concatAll flattens multi-dimensional collections by one dimension. Creating Collections Who here is a Netflix user? Great. All right. That's what I love to see. The Netflix model's pretty simple. When you go in, you get merchandized a list of genres, and within those list of genres, we recommend you a list of titles. So let's try using map and filter and concatAll to get a list of your favorite Netflix titles. Here's how you would do it. So I'm going to create a function called getTopRatedFilms, and that's going to accept a user object. And every user object has a videoLists array, and that's just the array of genre lists that you saw earlier. Action movies, thrillers. We're going to map over all of those video lists, and before we return the video list, we're first going to filter it to make sure all of the videos, that we're only returning videos with a rating of five. Now, because for every video list, notice we're returning another collection, right? Notice the function being passed to map, return videoList.videos, which is another array? That means we have a two-dimensional array, and so we apply concatAll to it to flatten it down into a nice flat list of your favorite titles. Does that make sense? Should we go over that just a little bit? Or do we feel pretty good about it? Any questions? - Why was it 2D? - Why is it 2D? Well, so remember we're starting with pretty much, like, it's kind of like a two-dimensional collection, because we got genres, and within that we have videos, right? So first we're mapping over the outer array, the genre lists array, and then with that function that we're passing to map, its job is to take each one of those arrays and translate it. And what are we doing? Well, we're just returning the videos array, right? But first we're just filtering it. And so that means for every array, we're just returning a filtered version of that array. Does that make sense? Technically it's an object with an array inside of it. That's the videoList.videos. But you get the idea, right? For each one of these video lists which contains an array, we're returning that array, but first we're applying a filter to it, and we're just making sure that all the titles are only the titles you've rated highly. And so at the end, up until the concatAll line, we have a two-dimensional collection. And the end we apply concatAll, and that's what takes that two-dimensional collection and flattens it into a one-dimensional collection of your favorite titles, all the titles with a rating of five. Yeah? - So you mentioned that if we have just a normal for loop, it's not going to work asynchronously. - Yes. - So if we have this, and let's say it starts with three items and it starts running through, and then after it's done concatAll, we get a fourth item, does it run through again? Or can you explain how it works asynchronously? - Well, so I'm not going to talk about asynchrony at all yet. - Oh, okay. - This is just arrays. We are just working with arrays here. So a for loop would work just fine, to be honest. - Okay. - But we're trying here, what, the goal of what we're doing here and why we're talking about arrays at all is we're trying to take a data structure you guys are already familiar with and feel comfortable with... - Yeah. - and learn how to use these functions over this data structure before we tackle a very different, but not so different data structure where we introduce asynchrony. - Okay. - So it's just arrays. - Okay, okay, okay. - You can try this right now. - I'm with you. - So now I'm going to, well, I'm going to ask you a question, which is what if I told you you could write nearly the same code to build a drag and drop event? So I'm going to show you a little preview of what we're going to be covering and what you can do if you start to get good writing programs with these few simple functions. You can take them and not just use them for arrays, but also on asynchronous data sources like events. So I want you guys to look really carefully, because if you blink, you're going to miss it. Okay, so here we are. We're processing your favorite title. And here... ...is how you build a mouse drag. I want you to notice the structure of this code is almost the same. What is a mouse drag, right? Well, it's all of the mouse moves that happen between a mouse down and a mouse up, right? Imagine for a second that events weren't this weird thing that you call with, like, an add event listener and a remove event listener, this sort of quasi-object hanging off of a DOM. What if they were first class values that you could hold onto just like arrays, and they had methods on them just like map and concatAll and filter, for that matter? We can actually compose together events to build new, more complex events using these methods. So imagine we have a collection of all of the mouse downs on a DOM element from now until eternity, and we just map over it, and we map over all of the mouse downs that happen on a DOM element, like a button, for example, and every single time we get a mouse down, we then return with it, we map over it, and for each mouse down, we return all of the mouse moves that are detected on the document level. The reason why we listen on the document level is we're looking for bubbling, right? So when I start moving around a button, my mouse might go faster than the button, and so by the time I've begun a drag operation, we want to be listening at the document level for mouse moves just in case my mouse gets ahead of the button so that we can move that button around. Hopefully that's clear. So we're literally replacing, because what map is all about is about replacing. So for every item in a collection, we want to replace it with the result of this function. So in this case we're replacing the actual mouse down event, that little event object you would get if you did an add event listener, with all of the mouse moves from now until eternity. But instead of just returning that whole collection, which would go on forever, in the previous example, we saw that we used filter to take certain, a collection, and reduce the number of items inside of it. Here we're going to replace the filter method with a take until method. All that does is it takes some stream and another stream, and it listens to the source of some stream of information, an event, mouse moves, in this case, until another stream fires. So I might be listening to a mouse moves event and a mouse up event, and I'm just going to keep forwarding on all those mouse moves until the mouse up event happens, and then I'm going to unsubscribe from both the mouse moves and the mouse up. So what we've done is create a collection, we've taken two collections which'll go on for eternity, and we've actually combined them to create a collection that ends, because we listen to one collection until the other one fires. And that's what take until does. So just like filter reduces the number of items in a collection, take until reduces the number of items in a collection. What it does is it creates a collection of all the mouse moves until the next mouse up, and then the whole collection ends. So for every single mouse down, we're going to return, we're going to replace that mouse down object in the stream with another stream of all the mouse moves until the next mouse up. Now notice that I've taken every item in a collection and I've replaced it with a collection, which means I have how many dimensions of collection? So if I take one, two, three, and then I replace each one of those items with another array, so I have an array of five, six, comma, nine, an array of nine, seven, comma, and array of four, eight, how many dimensions is my collection? Two, right? Because every mouse down object that we got, we replace with a stream of all the mouse moves until the next mouse up. And so how do we flatten a two-dimensional collection? ConcatAll. Yeah? - There's a question, where did concatAll come from? - So we, concatAll doesn't exist on the JavaScript array. We're going to write it ourselves later, right? And it does exist on this data type, the magical data type I'm going to teach you about that lets you code this way. Because obviously this is, this won't work over arrays, right? You can't model an event as an array. Just not possible to do in JavaScript, because for one thing, events go on forever, so you'd run out of memory if you tried to do that, right? So I'm going to teach you a new data type later on with the exact same APIs that I'm going to teach you to use over arrays. And so we're going to introduce concatAll onto array today in our exercises, and then later on we'll show you how to write concatAll over the data type that I'm going to reveal to you very, very shortly, which is what makes this possible. But we can see that structurally, at the same level, if we just think about events as a collection, we should be able to do this. So the top half of this slide is all about taking the events that we have in our system, the primitive events that we have in our system, and composing them together to create more complex events. Now, this is something we do with functions all the time, right? You take Function A, Function B, call them to, you know, compose them together to create Function C. You create Function C, which calls A and B. But for some reason when JavaScript programmers do event-based programming, they don't do the same sort of compositional style. And that's wrong. We should approach events with the same sort of approach to composition that we do with functions. And that's what I'm going to teach you how to do today. It's a very powerful technique for controlling the complexity involved in asynchronous programs. I think you're going to be really shocked at what we're able to do with just a few simple functions. So the top half of this slide is just creating the stream of mouse drags. It's actually creating technically a function which, when given a DOM event, will return a stream of all of the future mouse drags that happen on that DOM element. In other words, all the mouse moves that happen between a mouse down and a mouse up. Now, that's the top half of this slide. All we've done is we've created a stream when a function gets called. The bottom half of this slide actually uses for each to consume the data inside of the stream and do something with it. Increasingly you're going to see as you write code your code will begin to bifurcate into two parts, building the collection that you want and then consuming the data in that collection and doing something with it. The bottom half of this slide is morally equivalent to add event listener. It turns out that adding an event listener isn't so different than foreaching over an array. What are you really doing? You're just consuming each item in a collection as it comes in and doing something with it, right? So here in this particular case, we're actually going to move the position of the image so that it actually drags around. You're actually going to see this code in action later on. It's pretty impressive. Iterators and Observers What's the difference between an array and an event? This was a question asked to me actually five years ago when I was at Microsoft and this technology was just being developed, and the answer absolutely blew my mind. It totally changed the way in which I approached asynchronous programming. And I hope that you guys feel the same way after this presentation and this class. So events and arrays are both collections. I want you to start thinking about them in exactly the same way. Even though they have very different interfaces today in JavaScript, it doesn't have to be that way, and I don't think it will be that way for much longer. So why don't we just program these things the same way if they're both collections? Why do we program slurping data out of arrays differently than we would events? We should approach them the same way. How did we get this place? How did we get to thinking about these things so, so differently when I just showed you that we can think about them in a very similar way? Well, to answer that question, you've got to go all the way 20 years back, more than 20 years back to 1994. It all comes from a little mistake that was made almost 20 years ago. Who's got this book or read this book? One guy back there. This was a popular book, you know, it was a book, the idea was 20 years ago, you had a lot of software engineers, and they sort of got together and they said, well, people are solving the same problems again and again all through the industry, but they're solving it slightly differently. What we need is to sort of take these common patterns in software and put them in a book, and that's how we got the Design Patterns book from a gang of four authors there, those four authors. And that was a tremendously influential book. It was targeted at C++ developers and Smalltalk developers, but nowadays this notion of design patterns is still popular in the Java community. It's probably less well-known in the JavaScript community. But it was very influential and it defined a lot of the patterns that we use today in computer science. Now, there was a sort of a picture in the back of this book. Don't worry, you don't need to, we're not teaching all this stuff today. I'm actually just going to focus on two of these things. They're actually showing how all these patterns were sort of interrelated, but today we're only going to focus on two of them, the iterator and observer pattern. So the iterator pattern is actually something that's coming in JavaScript 6, and we're going to talk a little bit of how that's going to work. Now, they did not think that these two patterns were related. They didn't even draw a line between them, if I go back there. They don't even think that these two patterns are related. You see the observer way out there. But I'm going to suggest to you they are related. They're actually deeply related. And understanding that relationship is key to becoming an expert in asynchronous programming. So here's how an iterator works. It's very simple. You have a producer and a consumer, and the consumer requests information one at a time from the producer until one of two things happens, the producer says, "I have no more information for you," or the producer says, "An error occurred." Does that make sense? So how this looks in JavaScript is in JavaScript 6, the next version of JavaScript, you're going to be able to walk up to any array and request what's called an iterator object. So think of this as a way of a function sort of returning multiple values. So an iterator object has a very simple interface. You can call next and you'll get out this little object which gives you the value that you requested and also tells, gives you a little boolean indicating whether or not there are more values for you to get. So here I've got one, but I see that we're not done, so I'm going to call next again. Now I have two. And I see we're still not done. So I'm going to call next again. One final time. Three, not done. Call next. Finally I get a little message that says, you know what, we're done. So go back to that consumer-producer relationship. I'm the consumer in this case, and I'm just pulling values out of this producer, which is the iterator. Pulling values out until the producer says, "I'm done. No more information for you." So the gang of four defined this pattern in the '80s. You know, I think the '80s, early '90s. And we're, now it's actually just making its way into JavaScript. So it's a nice constant, you know, it's a nice convention for having any data structure. I might have a set or a map or a vastly different data structure. It doesn't matter what it is. I can walk up to a data structure and using this simple protocol, I can pull all the elements out of it sequentially one by one. Does that make sense? That's why they defined this particular design pattern, so that you can walk up to any particular data structure, you don't know anything about its internal representation, but you can still consume the elements inside. And this is actually the underlying technology that enables the new for of loop. There's a new loop coming in JavaScript 6 specifically for consuming this so you don't have to write all this boilerplate code. You can write what just looks like a simple for loop. So map and filter and concatAll, in addition to being implemented over an array, can also be implemented over an iterator. If you, let's just think about that for a moment, right? I can write a map function which, given an iterator, returns another iterator. And every single time it pulls a value out, it applies that map function to the value and then returns the translated value. So you can apply a map and a filter and a concatAll. ConcatAll would be a two-dimensional iterator, you'd flatten it into a one-dimensional iterator. All the operations I've shown you thus far are actually at their root probably, you know, there, they can be expressed over iterators. Now I'm going to show you another pattern that the Design Pattern guys expressed, and this was all about UIs. They were concerned about how is it in the user interface if you have a change to a model, how do you communicate to the view that something is changed? Or vice versa, how does a view communicate to the model that a user wants to do something? And they defined this pattern called the observer pattern. Now, probably most people in this room are familiar with the observer pattern because it's something we in JavaScript use every day, because it's just DOM events. So they just defined a way that you could add a function to some data producer and instead of the consumer pulling the data out one at a time, instead the producer pushes data at you, right? That's how an event works. You hand me a callback and then when I want to send you information, I'm the producer, I push you information one item at a time. Does that make sense? So in one of these patterns, in the iterator pattern, the consumer's in control. The consumer is pulling values out. It decides when it gets the next value. But in the observer pattern, the producer is in control. The producer decides when you get the next value because it calls your callback and pushes it at you. Does that make sense? Right? So it's really, these two patterns are about doing the same thing fundamentally. They're about a producer giving a consumer items one at a time. The only difference is who's in control. But the design patterns guys made a little mistake. So this shouldn't blow anybody's mind. This is just what happens, right, when I move my mouse around and we get the results. They made a little mistake. They failed to see this connection, that at some deep level, these things were both about the same thing. They do, they're both progressive, about progressively sending information from, to a consumer. And they didn't see that the iterator and observer pattern were symmetrical. And, so this is how you understand the observer pattern if you already understand the iterator pattern, right? The observer pattern iterates you. That's kind of what's happening, right? The observer, the producer is iterating you, right? So they missed this symmetry, and as a result, 20 years later, we think about asynchronous programming very differently than we probably should. There is a subtle difference between the iterator and the observer pattern as they outlined it in Design Patterns. Remember what I said about the iterator pattern. You can pull values out until one of two things happens. What are those two things? Can anybody tell me? - Error or end of array. - Error and it's done, right? No more data. Or the producer throws. Like, that means if I call next, they could throw, right? And that's how I would know that an error occurred. But with the observer pattern, they failed to add any of those, any well-defined way of the producer expressing to the consumer that A, there was no more data, and B, that an error occurred. Look at, let's go back to our DOM event listener example, right? You pass one callback to this thing. The only thing that you can receive, the only message you can receive from the producer is here's some more data. And why did they do that? Well, they were very focused on the event use case. They were very focused on solving for UI events. And the thing about a UI events is there might always be another one, right? Because somebody could always click another mouse click, they can always move the mouse again. And so they weren't really thinking that, oh, you need this notion of completion, because they were very narrowly thinking about how to model events. But today, 20 years later in JavaScript, we deal with push streams that end all the time. Anybody ever work with a Node stream, for example? Which is an I/O stream that you use in Node? Obviously I/O streams can end, right? And so we need some well-defined way for a producer telling a consumer, "I've got no more data for you." Right? So because we miss this standard way of these three semantics, there's no well-defined way of saying completion or error, today on the web, we have proliferation of different stream APIs, and all of them have, are slightly different. They might, some of them might have a way of indication completion, indicating completion, but not an error. Some of them have no way of indicating completion or an error, or they have some ad hoc mechanism where I'll send you an object, but then if you check this property, et cetera, then you know it's done. We have all these different interfaces. Whereas if you look on iteration, where the consumer's in control, we have one interface. It's just standardized. Everybody uses it everywhere. This is a mistake. We should have one interface that we use for push streams in JavaScript, and that is something very much like the observable. So if you can have an iterable, so an example of an array is an iterable, which is something that you can ask for an iterator from, right? So if we go back, an array isn't an iterator, It's something that you can ask to give you an iterator. Well, the opposite of an iterable, something that you can ask to give you an iterator, is an observable. And the way I want you guys to think about an observable is that it's a collection that arrives over time. Does that make sense? A collection that arrives over time. - This is a question we got in. - Yeah. - Go. - You want me to read it? - Yeah, go. - How different are iterators from generators? So the question is how different are iterators from generators? Would it be naive to say that iterators you can get from arrays via iterator, by calling the iterator method, which we saw earlier. And that that iterator inherits from generators defined for an instance of array. So explaining that question will require me to sort of explain generators and what they mean in JavaScript, and frankly, that's a very rich and complicated topic that isn't tremendously relevant to this here. But I can answer that question, which is to say that it's fair to think of a generator as something which can both, you can both pull information out of as well as push information into, and the iterator that you get out of an array is just about the pull side. So for those of you in the room who don't necessarily understand that, that's okay. You don't need to know that for this particular talk. But hopefully I've answered that question. So it's just, for the iterator, when you get it out of an array, all you can do is pull. I mean, you could push information back in by passing a parameter to next, but the array's going to ignore that information. So generator is this object, this weird little object we'll talk about at some point in JavaScript which is, you can pull information out of but also send information back into. Not particularly relevant, so I don't want you to think about it too hard. Observables Observables Introduction Why are observables important? Why are they powerful. And you should learn them. Well, it's quite simple. Almost every UI when you do some interaction with it you see the same progression. You listen for an event. The user does something. Then, you might make an asynchronous request to the server, and then maybe follow up with an animation. Observable is capable of modeling all three of these things. So, if you model all three of these actions as an observable in your user interface you could then use the functions that I'm going to be teaching you guys about today to compose them together seamlessly. With very, very little code. And so that's why observable is powerful and important. It's capable of modeling all of the three most common asynchronous actions you see in user interfaces. So, in order to get the observable type today I'll be using the reactive extensions library. It's a library that you can grab out there. It's an open source, patch it to a library. And what it is is it introduces this observable type but then it also defines all of the array methods that I'm going to be teaching you today and many, many more over the observable type. So you can use those methods to compose together observables instead arrays. So this is a site where you can go to grab it but it'll be included in the exercise you do later so you don't need to worry about it too much. Now, the problem is we have all these old asynch API's. These weird shaped asynch API's out there. The DOM API's is one of them. And our goal, the first thing we need to do, is we need to be able to take all of these weird asynch API's and adapt them into observables. And the observable library comes with some helpful functions to help you do that. So, for example the fromEvent function will take a DOM event, excuse me, will take a DOM object and the name of the DOM event and adapt it into an observable. And so then now you can forEach over that observable which is exactly the same as doing an add event listener, remove event listener. So let's take a look at an example. So we want take all of these different API's and adapt them into one common interface capable of modeling all of them. So here is how you subscribe to an event. Hopefully most of us are familiar with this process, right? We take an event handler and we add it to some DOM node. In this case I'm using the document, right? But it could be a button. It could be a link tag. And so we give it the name. I think I made a typo there. It should be mousemove, not mousemoves. But, we give it the name of an event and then we associate with it a call back. And then every single time a mousemove or in this case happens that call back gets invoked and that event object describing the information about the mousemove is pushed at us by our call back. Now, here's how you would do the exact same thing with an observable. So, you can take any observable, that observable that we adapted from a DOM event for example, and you can just forEach over it and accomplish exactly the same thing. We're passing in a function which gets invoked for every item in the observable and what comes out, though is this subscription object. This is where the observables forEach method differs from an arrays forEach method. Remember that an arrays forEach method; when you do forEach over an array it's going to complete synchronously. Immediately. ForEach is just going to run through all the items in the array and finish. Not so with an observable. Because, remember, an observable is a collection that arrives over time. And so when you forEach over an observable, it's not like when you just block and wait for all the data to arrive and then just block the system. We don't want to block, in fact. So what a forEach does, is it actually, it doesn't block. It completes immediately. But then, as the observable items in the observable arrive over time that function gets invoked. So it's just like a DOM event where you hook up a handler and then immediately you're not blocking and then as mousemoves arrive it gets pushed to your handler. Now, if you forEach over data because this goes on for a very, very long time, you might decide that you don't care about that data anymore. You might decide just like you stopped listing for mousemoves for example you might decide you don't care about the data coming out of an observable. And so we need someone to unsubscribe from an observable. And that's why forEach method on an observable, not an array, returns this subscription object. So if you decide you don't want to receive messages anymore; that's one thing I forgot to call out earlier; which is with an observable the consumer should be able to say, look I don't care about those messages. That's different than a producer saying done. There's no more data coming. The consumer can say you know, I don't care if there's more data. I just don't want to listen anymore. That's the equivalent of calling remove that listener or here, calling the dispose method on the subscription. So the previous slide and this slide we're really doing exactly the same thing. Observable hasn't added any value. It's doing the same thing that you would do with a DOM event. It's this next slide where we're going to demonstrate the value that observable adds over and above a DOM API event lister interface. - Question. Is there any plans to add observable in ES7, or any of ES.Next? - Yes. As a matter of fact there's a proposal in place right now for ES7 or technically now called ES 2016 which would add observable to the language. So this is definitely something that you might see in JavaScript in the future. In just in the early processes. So, no guarantees. But, regardless of whether it makes it in the language or not there's nothing stopping you from using it in a library. - Just the water bottle is making a lot of noise. - Absolutely. - Sorry about that. - Yes. And so, the answer is yes. There is an attempt, by myself and a few others, to get observable in the JavaScript language. So now we see those two missing semantics that I talked about earlier. The two things that were missing from the observable pattern that the gang of four described 20 years ago which is a way for the producer to give the information to the consumer that an error occurred. Or, a way for the producer to give the information to the consumer that no more data is arriving. So, the first event handler here is for receiving data. That's just like the add event listener callback that you hand to the DOM event. But these next two callbacks are optional. This callback receives an error. Remember, error means function, guys. I know a little bit tough to remember the new syntax. But all this is is just a function which accepts an error and then does something, in this case we just put it out to the console. And then, beneath that there's function, this is a syntax for a function that accepts no arguments. So this is a function that accepts no arguments. This whole function just gets invoked as a way for the producer to tell the consumer no more data. That's why it accepts no arguments. We just call that function to tell the consumer no more data is arriving. Does that make sense? So here are the two missing semantics that are there with iterator, right? When you call next, and it says done true. Or when you call next and it throws that we've now added to push streams by this observable interface. So here's how you can get pushed the information that an error happened or get pushed the information that, well there's no more data in the stream. Yeah, question. - Is this forEach, could you have added any number of functions there? Or is this a really specific forEach that accepts up to three? - Specific forEach that accepts up to three. Because we're trying to create the same semantics for push streams where you're receiving information pushed at you that we have for iterators where you're pulling information out. And so when you're pulling information out of an iterator you can either get a value, you can attempt to pull information up by calling next and it throw so you can get an error. Or, you can pull a value out and that done can be true telling you that there's no more data. Here we're trying to create the same semantics the other way around. So, if I'm pushing you information I should be able to also tell you that an error happened or tell you that there's no more data coming. And that's what those last two functions do. So think of an observable provisionally, while you're trying to get your head around the concept, think of an observable as like a DOM event where you can pass three handlers instead of one. And those last two handlers tell you whether an error occurred or whether there's just no more data coming along. Does that make sense? So observable, right now the mental model I want you to use, is an observable is a little like an event that's a first class object that you can give three event handlers to instead of one. So, technically if we, it's also just short hand for this. You actually give it an object. This is what we call an observer object. So those three callbacks are actually just shorthand for providing an object with three callbacks; onNext, onError and onCompleted, right? And then the observable accepts an observer and invokes onNext, onNext, onNext, until it invokes either onError or onCompleted. Does that make sense? It doesn't call onError and then onCompleted, right? It will either just tell you error occurred and that means that the stream has ended. No more data is coming because an error occurred. Just like when you throw in a function. That function is not going to suddenly return a value. When you throw inside of a function, it's done. So that's how the observable works. You got an observable, you. If you want to consume the data inside of an observable. Create this little object with three handlers. You hand the observer object to an observable. The observable pushed data into the observer object and by extension your callbacks. And it'll call onNext, zero to end number of times followed by either an onError or an onCompleted. It might actually never call an onError or onCompleted because some streams go on forever. Mousemoves, you would never get onError or onCompleted called if your observable was just wrapping a mousemove event, right? Because, that's a streams that just goes on forever. It's perfectly okay to have an observable that goes on forever. Question. - There's a question about, can you show the line that declares mousemoves. Is that a new observable? - It's right back here. So, we actually created it by using a function provided with the Rx library to help you adapt an event into an observable. And we're going to see the actual definition of this function in just a moment. This fromEvent function which can take any DOM event and event name and combine them and create an observable type. Does that make sense? It's just the same thing. We're just adapting the interface. So, when you call forEach on mousemoves under the hood it's adding an event lister to the mousemove even on the DOM object. So, we're going to see how this works in just a moment. Do you have a question? - Another one came up on that, many can answer these. Do those events; do the functions have to be in that order, next, onError, onCompleted. - Not here. In this particular overload where you're passing in. It allows you to pass in either... It allows you to pass in either arguments that are functions. Or you can pass in this object. So if you want to be able to specify them in any order you can pass in the object because that's just name value pairs there, so you can rearrange that however you want but today, for most of our exercises we'll just be using this form where we just pass functions. And in this case, yes it is order. Because it's based on argument order. So you get the next function first, the error function second, and the completion function last. Does that make sense? Observable Metaphor -You had a question? - I like to understand things in very simple analogies. - Yeah. - I'm going to check my analogies with you to see if it's correct. So, an iterator function sounds to me a lot like a paper feed on a printer. Is there a piece of paper? Yes; print. Is there a piece of paper? Yes; print. It's empty. Error. - Yup. - And an observer would be a like somebody cutting pieces of birthday cake handing it to the person. Person just keeps handing pieces of cake down to other people until they get the piece of cake themselves. They don't need it anymore. Or, you don't get the piece of cake. And you get the error. - You know, sometimes it's tough to express it as metaphors because the truth is you can think of that as both a push or pull. Really it's all about who makes the decision when another value is delivered? So why don't we anthropomorphize the iterator and observer pattern and why don't I be the producer and you be the consumer. - Okay. - And let's say we're distributing cake and you decide you want some cake. What do you do? If iterator, what you say to me is, look I want to start asking for cake. And maybe I'm going to distribute it to folks. But, I'm going to be the person. I'm asking you for cake. And so the way it works is you say, can I have cake, please? So you start. Notice, she's starting. She's trying to consume the cake and she's starting, right? And so you say... - Can I have cake, please? - Sure. I cut her a piece of cake. I hand it to her. And then I wait for you to... - Can I have cake, please? - Right. Cut her another piece of cake. Hand it to her. We're running out of cake, right? I hand it to her. - Say it again. (laughter) - He's going to get to run out of cake now. - That's the next step, yeah. - Just keep asking. - Can I have cake. - Yes. You may. Oh no! I've given you some cake, but now I'm out of cake. So I'm going to also tell you, by the way, I'm out of cake. I got no more cake for you. Right? So that's the iterator pattern. What is the observer pattern? I cut some cake and I throw it at her. I don't even wait for her to be ready, right? She's like, okay, give me some cake, and I'm like, I'm just throwing cake at her as fast as I can possibly throw it whether she's ready or not. So I'm in control. I decided when she gets cake. And then, when I run out of cake I say, no more cake for you. At this point, you're probably happy, right? Nobody throwing cake at you. That, now that we've anthropomorphized the pattern, it's just about who's in control. Are we pulling? Is she pulling? Or am I pushing? Does that make sense? But, either way, we distributed cake, right? We got it done. We got cake from point A to point B. So, all we're doing here is we're adding the ability in those last two functions for me to tell her no more cake. Or, I guess the error would be I cut my finger or something. Something went wrong. So, that's how I would communicate to you that Ow, I cut my finger, I can't cut you any more cake. That's the error. Or, we're out of cake. Nothing there. And the first call back is just for sending the cake. Does that make sense? So now we kind of get what the observable type is, right? It's just another way of getting information from point A to point B except you're not in control. The producer's in control. And so here's the overload that you can use if you want to pass in an object. We'll be using this overload in our examples here, okay? But, from now on when I talk about the observer object, this is what I'm talking about. An observer observes an observable. It's the object with those three methods on it. So I might be mentioning that a little bit later on. Another question. - Are you limited to onNext, onError, onCompleted? Or can you broaden; it's actually on your next slide. - That's it. Well, it's the same semantics that we have with the iterator, right? That's what we're trying to do with this protocol. You can find a million ways for getting information from point A to point B. But what we're trying to do is come up with a very strictly defined way of getting information from point A to point B and it's got three semantics. I give you data. I tell you an error occurred. And I tell you something completed. You'll be amazed what you can actually do with that very simple protocol. Turns out it's enough to get the job done. So here is the function that we use to convert a DOM event into an observable. So it's actually a static function. We just hang off of the observable type. So you just call observable.fromEvent and you pass in a DOM object and an event name. An observable is nothing but an object with a forEach method. Every other function that I'm going to teach you about today map, filter, concatAll. All of them are implemented in terms of forEach. And so, if you want to create an observable, all you got to do is create an object for a forEach method. Because, of course, forEach is how you get the data out, right? So if you give an observer to the forEach method it's just going to push data at you. Yeah, another question back there? - What does the dispose method do when the producer indicates it as done sending events? - Under the hood what the dispose method means is that you're not going to get any more callbacks. I'm going to stop throwing. So, we left out the little piece of semantic which is I'm throwing cake at her and one thing she could understandably say is, whoa! No more cake, please. Right? And that's different than me saying I have no more cake left. She's just saying, I don't want any more cake. And that's effectively what the same thing is when you call dispose on the subscription method. It's like calling remove event lister. In fact, under the hood, as we'll see here. Notice down here at the bottom, I'm creating this subscription object. When you call forEach. And if you look at the definition of dispose, what does it do under the hood? It unhooks from the event handler. And that's how we assure that no more data is going to be sent. And so, when we return an observable and somebody calls forEach, and they pass in this observer object. Remember, the observer is just an object with the onNext, onError and onCompleted methods. What's going to happen is the observable, in order to deliver that information to you it's going to hook up an event handler to the DOM object. And whenever that handler function gets called it's going to call the onNext method of the observer. So that's how it pushes the information along to the observer. Now, we're going to every forEach method at the end has to return a subscription object. And so we create a subscription object which is nothing but an object with dispose method. And the contents of that dispose method are very simple. Whenever the consumer; whenever she calls dispose, whenever she says no more cake, please. I just have to make sure as the producer, that I never give her another piece of cake. And how are we going to do it here? I'm just going to call and remove listener. And that's how you adapt the DOM add events listener, remove that listener to an observable API. Observables in Action -What is defiantly not coming in JavaScript 6 is a special syntax for creating an observable. I am going to invent a special syntax for creating an observable. Just like we have a special syntax for creating an array with the square brackets. Let's imagine, we had a special syntax for creating observables which are collections that arrive over time. The only reason I'm going to invent that is so it's easier to understand how observables work and how the functions that we're going to be learning about map, filter, concatAll transform observables. So, definitely not coming in JavaScript6. I just invented it for this particular presentation. But this is the syntax that I'm going to use for a collection that arrives over time. We're going to imagine that we have a future version of the JavaScript language and we can invoke methods on this literal for describing an observable. Okay? So, let's say I had this observable. Imagine one of these dots as like 10 milliseconds, for example. Right? If you were to forEach over this observable you would immediately get called with one, and then a little bit later I'd push two to you, and then a little bit later I'd push three to you; and then I'd say onCompleted. Does that make sense? That's the callbacks that you would see. So let's see what happens when I call forEach on this. In real life, the observable you're going to be learning about has the forEach method which we saw defined earlier on mousemoves, right? Does this exact same thing as you would expect when you did it over array. The only difference is, it takes time. It arrives over time. Right? Remember when we did this with an example with an array it finished almost instantly. And now, whenever the data comes in from the underlying source then it gets translated and then immediately handed off. Question in the back. - There asking about, ever forEach call is returning a different dispose object? - Yes. - How do we hold all of them and use them? - Well when we call forEach you're going to get multiple values. Again, that shouldn't be too hard to think about because it's just the same as add event listener or remove event listener. When you call add event listener you have to remember to hold on to the handler because that handler is the same handler you have to pass back when you call a remove event listener. Here it's no different. Instead of holding on to a handler you have to hold on to a subscription object. So, when you call forEach you're going to get many, many values. And at some point you may decide you don't want those values anymore and you have to call subscription.dispose That's no harder than calling remove event listener and passing in the same handler you were holding onto before. So one shouldn't be more complex than the other. Same complexity. - I think they might think about it like every object comes back the disposed will just turn off the whole subscription, right? - Yeah. - Yeah. - You're not going to get any more data. So here's what happens. We call forEach over an observable. Remember map? Map applies a function to every item in the collection and creates a new collection containing all of the transformed results. That was true over an array and it's true over an observable. If we call map over an array what we get is an observable that pumps us out each value but first transforms it through the function. So we take one, two, three and apply a function that adds one to each item and we get two, three, four. But notice, it arrives over time. As soon as one comes along an observable doesn't hold on to data. As soon as the value comes along it's going to immediately transform it and then send it along. We collect up data in the memory. As soon as data arrives we translate it by throwing it through that function and then we hand it off like a hot potato. Does that make sense? So, we're mapping. Now let's try filtering. Imagine? Same thing as before. We're going to take the observable and we call filter on it, we create a new observable that contains only those items that pass the test function. And so we took an observable one, two, three and we got an observable that can two, three. Yeah. - That plus would be a greater than? - Yes, it should. Thank you. Live code reviews. Sorry. Yeah? Back in the back question in the back. - Does forEach take this arg? Should I even worry about it? - ForEach does take this arg. Probably not a super important point at the moment. But yes, it has the same interface as the array; forEach. Yeah? - So Map was returning an array, right? And now it's returning singular elements? - Sorry, this slide can be a little confusing. What's actually happening here is I'm shortening the slide from this to this. So, I'm creating a new array, but I'm also calling forEach so you can see the values that come out inside of it. So in this case I'm actually creating an observable. And when you call map, all you've done is create a new observable. And then the forEach in here, it's kind of implicit. I'm just putting in a forEach to show you what would come out in the console if you were to subscribe to that observable. But that's a good question. - And that's why, at the beginning that's why you're running the concatAll because we have all these mini arrays. - Yeah, we have a bunch of mini arrays. Which we don't want to print that out to the console, right? - You're clever. Okay, I'm with you. - So, I don't want to cheat. So I'm going to put that in here, so we can all see what's going on. Might go off the slide a little bit. Let's take another look at that, right? So map actually just returns an observable. And so when see stuff printed out of a console it's because we're forEaching over that observable. So we get two, three. And then eventually four. Right? That's how long it takes for the items to arrive. And as soon as the items arrive, their translated and then they're forwarded on. Hot potato. Same thing with filter. We're going to get two, or three. We skip over the first element. We never return it because it's one. And now I want to call your attention to concatAll. So this is over arrays. You remember how concatAll works over arrays, right? Now, how many different ways really is there to flatten a two-dimensional array. I suggest there's really only one sane way to do it which is to start with first array go through all those elements; move to the next array go through all those elements; move to the last array eventually go through all those elements and then put all that into a new array. And that's effectively what concatAll does. It starts top to bottom and goes left to right. It goes top to bottom, left to right. So here, this example we get two, three, four. One, two, three, four. That's the only way that you can flatten the two-dimensional array. It's the only way that makes sense. Here's where things get more complicated. An observable is in a collection over time. And it turns out that if you have collection, an observable of observables there's more than one way to flatten it. Because the items arrive over time. There's three strategies. In fact, you can solve almost every asynchronous concurrency problem where you got more than one thing going at once. Almost every concurrency problem that you're going to run into the UI with three basic flattening strategies for an observable of observables. So it's really key that you just get these three down. And you'll start to see that many, many problems that look brand new or very different can actually be classified into one of these three flattening strategies. So, we have concatAll. So, what does that mean when applied to an array? Or, excuse me. What does that mean when we apply to an observable of observables? So what this is is if you forEach over the outer observable a certain amount of time would go by and then eventually you'd get another observable. And presumably forEach over that, right? But in the meantime the outer observable could give you another observable while you're still not completed with that observable that you just got, right? Because the observable happens over time. And so you have this potential for concurrency now because in an observable of observables you got this new dimension of time which we didn't have with an array, right? All the data is there instantly at the same time. So with an array, we just exhaust things top to bottom, left to right. So, here's what concatAll does. The concatAll strategy would do when applied to an observable. So, let's say I forEach over the outer observable here. The observable of observables. First, I would get an observable. In this case let's just say it just contains one. And then I'm immediately going to forEach over that. And then, what we do, is we take the data and we put it into the flattened stream. So we see one up here immediately. So, now it's completed. And just as it completes the outer observable gives me another observable. This one contains two and three. And so I immediately forEach over that. That inner observable. Two pops out. I immediately pushed that out into the output stream. The nice flat output stream that comes out of concatAll. Now, while I'm waiting for that second observable to complete the outer observable throws me another observable. This one's empty. So I forEach... What concatAll does is one thing I could do, is I could begin forEaching over this observable immediately. But I don't. What distinguishes concatAll from the other strategies is it always makes sure that elements come out in the order that the collection that they were inside of arrived. So here I really want to see one, two, three, four. So we're not actually going to forEach over this observable yet. I'm just going to put it away. I'm going to hold on to it. But I'm not going to call forEach on it. Does that make sense? So now, I'm still just waiting for three to arrive. And while I'm waiting for three to arrive for that second observable to complete another observable comes to me. Once again, I'm not going to forEach over it. I'm just going to put it aside. The key thing to understand is in this particular example all of these observables do not start emitting values until you forEach over them. Does that make sense? All of these observables do not start emitting their values until you forEach over them. So, if I don't call forEach on these observables data is not going to come out. Now, not all observables behave that way. It's possible that you can have an observable that is emitting values regardless of whether you are calling forEach over it or not. Can anybody think of a data stream that will emit values and of you don't start listening to it soon enough you might some values? We showed you one earlier. Mousemoves. Right? Let's say I'm moving my mouse around and you haven't hooked up an event listener. You're not going to get those event objects, right? You've missed them, effectively. You only start getting those event objects when you add your event listener. We call that a hot observable. It's a hot data source. It's going whether you're listening or not. What I'm showing you in these slides is an example of a cold observable. A cold observable will not do anything until somebody calls forEach on it. It will not do a single thing. And so, the data inside of these observables is not going to come out until I call forEach on it. So all these slides use cold observables. So that's why, even though that times has passed where I could have emitted two. Excuse me, I could have emitted four here it hasn't been emitted yet because I haven't called forEach on it. So now, I'm still waiting on that second observable to complete and I actually got two observables in a buffer somewhere that I haven't called forEach on yet, does that make sense? That's that third and fourth observable. Now finally that second observable completes gives me three and completes. And says onCompleted. So onNext three, onCompleted. Now, I immediately subscribe to that third observable that I've been keeping over here in a buffer somewhere. What happens to this empty collection in the middle of a flattening operation? What happens to empty collections when they get flattened? - Disappears. - They disappear. Right? Because there's nothing to return. Yeah? So we know it stays empty because it's already fired it's onCompleted? - Yes, if you forEach over an empty collection, the only thing that'll happen is it fires onCompleted. That's how we know it's empty. Yeah, question in the back. - There's a question, if in the empty array, if five was in there that wouldn't happen until after the three? - That's correct. Yes. So, instead of an empty array, there was five you would end up, the outputted collection would be one, two, three, five, four. Because it returns elements in the order based on the collection that they're inside of arrives. So, it basically, what you need to know about observable concatAll is it behaves exactly the same way as array concatAll. It goes top to bottom, left to right. So, here in this case we're going to subscribe to this observable and basically it's like a no. Nothing happens. Because all empty collections, when they get flattened they just whew, disappear. Now, after we get onCompleted message from the empty collection we forEach over this other collection that we've been keeping on the side. And it calls onNext, gives us four and then immediately calls onCompleted. And so we return four. Notice what happens with concatAll. The output stream ends up getting elongated. It's longer than any of the input streams, right? Because we basically have to make sure to enforce that things come out in the right order. And that means we need to buffer things, right? Buffer observables, and make sure to only forEach over them when the previous observables have completed. Race Conditions and Nested Observables -Got a question in the back? - There's a question. So this is a way to stop the arrays condition? - Yeah. Exactly, that's a great way of putting it, right? If I subscribe to two different events, they're going to fire to me whatever order that they come out, right? They don't care that I subscribe to this event first and that event second. You're not going to wait until all the data from the first event is done before you start getting messages from the second. That's a very hard thing to do usually in concurrent programing and that's the way you would do it. This is what you use when you want to order events. You want to say, look I'm only interested in this event after these events have occurred, does that make sense? Another question. - And then he next one was, can you give another example of observable of observables like the cake cutting example? - Ah so, what would a... So, sometimes these things can be a bit of a distract and they defy metaphor, but we're going to try. So, let's give an example. Now, instead of dispensing slices of cakes, I'm going to dispense cakes. Right? I'm going to actually, so in this case, if the observable. So I'm just going to start throwing cakes at this poor woman. Now I'm throwing whole boxes of cakes at her, right? And now, you're going to try and before I throw you another cake you're going to try and cut that cake up and then hand the individual slices of the cake around. Does that make sense? A cake is a collection os slices and I'm pushing one at her. And she's going to then turn around and start cutting as fast as she can, cutting cake slices and then throwing them at the other people in room, right? But while this poor woman is cutting up cake I might take another cake and throw it at her, right? But what does she want to do? She wants to make sure that everybody in the right order in the room gets their cake in the right order. That nobody gets their cake before anybody else. So even though I'm throwing cake at her, what she's going to do; if she's in the middle of cutting a cake and I hand her a big cake, she's just going to put it aside. She going to put it aside and she's going to cut the slices and hand them out in sequential order to people next to her, does that make sense? So, I might be throwing cakes like crazy real fast, right? But, you're just piling up the cakes next to you, right? And as you can methodically get to them in order you're handing out slices. And that's the buffer of observables, right? I've handed you a cake right back here. I've handed you a cake, that second observable. This observable right here. I've just handed her a cake but she's not done cutting the slices from the second observable and handing them out yet. Does that make sense? So that's like an observable of observables, right? The individual cake is a collection of slices but we also have a collection of cakes. And that's the outer observable. The inner observable is the collection of slices. Outer observable is the collection of cakes. Does that make sense? Another question. - Would you ever have a case where one of the sub-observables never completes therefore restricting the forEach method? - Great question. Absolutely. If I give you; and I know this is where metaphors tend to break down; but I give you a magical cake, an Alice in Wonderland cake, where the slices never end, right? - Throw printers. Throw a printer with unlimited paper that keeps printing forever. - Sure. That works, too, I guess. So, now I've given you this magical cake, right? And then, you're never going to get to any of the other cakes I give you. You're just going to keep handing out slices from this magical cake that never, in the meantime you're buffer of cakes, I'm just going to keep throwing cakes at you, but you're just going to keep piling them up, forever. And so, yes. You would not want to use concatAll on an infinite stream. Probably not a good idea. Does that make sense? So these are hazards to be aware of. You don't want to try and concatenate an infinite stream to another stream, because you're never going to get to the other stream. So great question. - What makes a mousemove event not an infinite stream? Because, eventually the mouse stops moving? But, what if the mouse never stops moving? - Oh, the mouse is an infinite stream. Because we've gone from a DOM event which has no notion of completion in error. That was the missing piece that the design patterns guy didn't add and we've adapted it into an observable. But because the underlying source never says I'm done and never gives you an error, the observable that we've adapted it to will never say I'm done or never give you an error. It's perfectly reasonable and okay to have an observable that goes on forever. And that might seem odd at first. And it seemed odd to the design patterns guy, and it won't, actually sorry. That what seemed natural to the design patterns guys. To have a stream go on forever, because they were thinking about a UI event. But when you start thinking about observables what can you do with a collection that goes on forever? It seems impossible. It's hard for us to imagine the infinite, right? But it turns out, because of DOM events, we work with collections that go on forever all the time. Right? Those are all collections that go on forever. The difference is we have to manage when we as the consumers have to say, whoa! No more cake. Stop listening at the right time. And that usually takes a lot of work and coordination and it's that that I'm going to show you how you can avoid today. I'm going to show you how you can deal with observables that are infinite and combine them together with observables that are finite. All using simple commands without introducing a bunch of variables to keep track of what's going on. Another question in the back. - So, on this example if the observable ended up throwing an error between two and three, would four be prevented? - So what would happen, is if an observable throws an error between two and three, the outer observable, or rather the flattened observable; imagine a little X there between the two and the three. And then you'd never get three and four. The whole observable would stop because any error inside any of the inner observables is forwarded up to the outer observable. And as soon as you get an error the whole thing is completed. You're never going to get any more data. So, remember an observable works this way. You can get onNext, onNext, onNext infinitely. But if it does end, it's going to end with either an onError, or an onComplete. And so, if we got an error between the two and three it would be forwarded up and there would be, the stream would stop right after that error's arrived. Does that make sense? Yeah, question. - Will an inner observable still be iterated over even if you call dispose of its outer observable? - No. What happens is, every single time I call forEach on any of these inner observables we get a bunch of little subscriptions, right? When I call forEach over the outer observable, I get a subscription object which I can use to stop listening to the stream. But every single time I call forEach on any of these inner observables I also get another subscription object for that tiny little stream there. If you call dispose on the stream for the flattened observable, what it's going to do, first it's going to call dispose on the outer observables, right? It's going to say, I don't care about anymore cake so I don't want any slices. So I'm going to stop listening for cakes, and then, if you're currently cutting up a cake, right? Let's say you're the one doing the flattening here, right? You're just going to stop cutting up that cake. Does that make sense? So calling disposed on the subscription on the flattened observable will end up calling disposed on the outer observable subscription, and whatever inner observable you're currently listening to. If you are listening to one, right? Because you might be done with the inner observable and the next inner observable hasn't arrived yet. Does that make sense? In this case they've all overlapped but they could arrive over time. These observables might not overlap at all. - Got at least two more here. How does concatAll know what pattern to follow? - Well, this is the pattern of concatAll. So concatAll's job is to make sure that all the slices are handed out in order. And so it knows that if it receives an observable before it's finished with another observable it has to buffer them up and make sure that it only subscribes to the observables in the order in which they arrive. And if it does that, naturally what'll happen is the flatten will happen top to bottom, left to right. And you'll get all the items out in the order of the collections that they're inside of arrive. Does that make sense? Yeah. - And then; does dispose trigger the onComplete? - No. Great question. I really want to nail this. When you call dispose on a subscription, what you're saying to the producer is, what she's saying to me is, I never want you to talk to me ever again. Right? What she would understand, I'm throwing cakes at her, right? If I were her, I would say, dude, just go away. Don't ever say another thing to me. So that means I'm never call onComplete. If you think about it, it's kind of silly, right? You just said I never want to get a call back and then I say, onCompleted. I'm going to be, well yeah, I know. I don't want the data, right? So, yes. When we call dispose, what we're saying is, I never want to hear from you ever again. I don't care what method it is, right? Does that make sense? So it is different. Yes, another question. - There's some confusion now about dispose that you might have covered. If dispose is called within the on completion callback, nothing happens? - Yeah. The first thing to notice is you wouldn't even bother to call dispose when you get an onCompleted. Because the whole point of calling dispose is to make sure that the producer never sends you another message. And by the time I've said onCompleted you know that I'm never going to send you another message. So there's no reason to call dispose. TakeUntil -Another way of metaphor of thinking about what happens with flattening strategies, and I love using this metaphor because we all hate traffic is to imagine you're on the highway We've got three lanes open, right? And then there's some accident or some construction and they have to close two of the lanes. The concatAll strategy which isn't very good to use for traffic for obvious reasons because each one of those lanes is an infinite observable, would say, okay everybody on the left lane go through first and then everybody in the right lane and then everybody on the, you know, those two lanes opened. Right? You never use that in traffic because there's never going to stop being traffic on the left lane. They're both two infinite observables. So, you wouldn't use concatAll as a traffic managing strategy. However, this next function I'm going to show you guys is mergeAll. Wait, I lied. Sorry. We're going to come back to that. I want to show you all the functions you need to know in order to understand the drag and drop example I showed you earlier. And so I showed you concatAll in the drag and drop example. But I also showed you takeUntil function which I didn't explain very clearly. What a takeUntil function does is it takes an observable that's a source observable, and it takes an observable that's a stop observable. And every single time the source observable emits something it forwards that on but as soon as the stop observable emits a single value whether it's onNext or onCompleted it completed the overall observable that's taking these two observables and putting them together. So takeUnitl creates a new observable, just like map and filter. It doesn't change any observables. It takes a source observable and a stop observable. Creates a new observable with all of the data from the source observable and completes that observable as soon as the stop observable fires a value. And then, under the hood unsubscribes by calling dispose on both the source and stop. Does that make sense? So that's how we can take two infinite observables and combine them together to create an observable that ends. And this is the Hughes case that the gang of four weren't only thinking about. They were like, well you got these infinite observables. What can you do with that? You can really just listen to it until you're not interested anymore. Until you call dispose. This function right here is why I haven't unsubscribed form an event in five years. I have not typed the words remove event listener in five years. Because I don't need to. This, it's really important you understand this function because this is the first big mental step we're going to make to thinking about asynchronous programing very differently. With a DOM event listener all you can do is you can say whoa, unsubscribe. Right? That's what you have to do. You have to say, look, I don't want any more data. Unsubscribe. Another way of thinking about that, a different way of thinking about it is, well, I'm going to take the stream of data that I want and under some condition I'm going to create another observable that's going to fire some other asynchronous condition. Sorry, let me rewind. Here's how you usually do with events. So you'll add an event listener to an event and then you'll start consuming the data but then, at some point, when some event happens, because that's how everything happens in JavaScript. Some event happens, right? You'll decide that you don't want the data in that event anymore and then you'll call remove event listener. That's how we code today. But notice that we're going to call remove event listener when some event happens. Because that's how things happen in JavaScript. Somebody's mouse clicks. A timer fires. Some asynchronous entry happens and then under a certain condition we're going to decide we want to unhook from that event and stop listening to it anymore. Well, why don't we just take the source event the thing we're interested in and use takeUntil and combine it together with the stop collection and then we don't have to add a bunch of handlers in state to track whether we're listening to something anymore. We can just declaratively take these two streams, combine them together and cause the source to end when we want it to end. That's different than unsubscribing, right? When I call subscription.dispose, I'm saying, I don't want the data anymore. But if I take my source collection and I use takeUntil to combine it together with whatever event happens when I'm supposed to stop listening well, then I've created an observable that completes when I want it to. Don't unsubscribe from events. Creates streams of data that complete when you want them to. That's the big mental leap we're going to make today. We're not going to bother to write state machines, like oh, if this happened then unhook my event listener and then if that happened, hook my event listener up. We're not going to do that anymore. We are going to create new events from existing events and we're going to create those events such that they end when we want them to. Got a question in the back. - Yeah, we got a few here are piling up. Will on completion even trigger when observable is running in time asynchronously? - Yes, on completion can always fire, and that just indicates that no more data will arrive. - We will not know when the stream will complete as it's waiting in time? - Yeah, you have no idea when an observable is going to end. If you're the consumer, and you're getting information pushed to you. Right? If I'm pushing information to her, she has no inkling of when I'm going to stop throwing cakes at her, right? She has no idea. It's only when she receives an explicit message I'm done, no more cake, that she knows for certain. Any other questions? - The takeUntil emit onComplete as soon as the stop collection emits the first item, or will it wait until the stop collection emits onComplete? - The very first item. Good question. So, the question is, will the onComplete happen on the observable that comes out of takeUntil as soon as the stop collection calls onNext or will it wait 'til the stop collection calls onComplete? And the answer is onNext. As soon as it anything comes out of that stop collection, onNext or onComplete, it will complete the overall value. Now, if the stop collection threw and error, the overall observable would end with that error. We always forward errors along. We never swallow errors. So, if either the source collection or stop collection called onError, then the observable that we created will onError. Yeah. - Will the source collection keep feeding data that's now just kind of being ignored or will it know to stop feeding data or don't we care? - Well, what happens is, as soon as the takeUntil operator which has got these two observables that it's combining together for somebody else, right? Think of it that way. Each of these functions are like one step along the chain. Think of it as a guy who's got an observable and he's pulling items out and translating them and handing them off like a hot potato, right? So takeUntil has got two observables. And it's listening, every time a source item comes through it just hands that along. But then as soon as the stop item comes along, what does it do? Well, it calls onCompleted to the next guy, it says no more data is arriving. But then it takes the subscription objects it's got from the source collection or the stop collection and it calls dispose, well, in the case of where the source collection sends a value it'll call dispose on the stop collection. Because there's no need to call dispose on the source collection because it's just said onCompleted. Don't need to say, hey I don't want anymore data, if I just told you I'm not going to give you anymore data. So, stop collection because it hasn't said I'm done we would turn around and we would call dispose on that because we're not interested in the data anymore. Because we just told the guy down the line we're never going to give you anymore data. Does that make sense? - Yeah, that's perfect, that's great. - Another question. - I think you just covered this here. The stop collection value is irrelevant it'll just trigger a done for the source observable? - Yes. Totally irrelevant. Totally gets lost. Doesn't get forwarded along the stop collection value. So, again I want put an exclamation point on this. We're going to learn today how to declaratively create event streams that end when we want them to end, not have to build state machines wherein certain conditions we unhook using explicitly using remove event listener. And we're going to get code that's a lot simpler with very few moving pieces. Okay? So just like the mouse drag example we saw earlier. Yeah. Another question. - Yeah. Can you hook up an auto complete box that might listen for return from the server until a new key press event appears? - Absolutely, yes. What he's talking about, is an autocomplete, is he asking me to do it, or is he asking whether it can be done? - He asked me if it could be done. - So, let's say we're feeding an auto complete box with results from the server and maybe it's like arriving over a web socket. A web socket is a great example of something that we can adapt to an observable interface, right? As information is coming over to the web socket, we're going to call onNext, onNext, onNext, and we might fill up the list. But then as soon as somebody presses the key we could just stop and close the web socket. Does that make sense? Because, I'm the producer. As soon as she says to me explicitly I don't want anymore data that's an opportunity for me as the producer to potentially cancel anything that I'm doing to produce data for her. Let's say I've got an order out for more cakes, right? I'm relentless. I'm ordering more cakes for her. And she's like, whoa! Well, I'm just going to pick up the phone and cancel that order. And that turns out to be a very useful thing. Because a lot of the times the operations that we execute in order to produce asynchronous data can be very expensive particularly when you're talking about IO operations, right? Let's say I've got an open web socket. That's not cheap. Web sockets aren't free. They take resources, right? If she says she doesn't want anymore cake I'm going to close that web socket. Because I might be getting the cake from you some other entirely different web server. Mixing my metaphors, but you get the point. Yeah. - Can you use concatAll in the stop collection? - ConcatAll in the stop collection? - For in the stop collection. - It's a little bit of a confusing question because concatAll only works on an observable of observables. And in this case, we don't have an observable of observables. You just have two observables. That's not the same as an observable of observables, right? With an observable of observables, you have a stream of observables arriving over time. Here, I just got two observables in my hot little hands. And I'm listening to one, well, I'm listening to both, and as soon as the source collection sends data I move it along. But if he's asking if you can use concatAll on the result of a takeUntil, yes. Because you can use concatAll on every observable. It's safe to use concatAll on the result of a takeUntil because the whole point of a takeUntil is that that observable might at least end, right? So because hat observable might at least end we never know for certain if an observable is going to end, right? But in this case, we're using takeUntil. So presumably, we expect under certain circumstances it might end. Then, it's totally safe to use that in a concatAll. Does that make sense? But you would never want to use an infinite observable in a concatAll. Because you'll never get to the observables down the line. Implementing Mouse Move - With your experience with asynchronous programing how much gain do you see with proper tail calls? - In asynchronous programing? None. Because, just to explain briefly what he's talking about; proper tail calls means that if you have a function and you recursively call that function and it happens to be the last thing you do inside of that function which is recursively call it again, what is going to happen in JavaScript six is that instead of growing the stack which is what happens if you; usually if you call a function inside of itself with recursion. Instead of that stack growing, in JavaScript six it'll basically turn that into the equivalent of a loop, right? Because that's what you're doing. At the bottom of function you just call that same function again it's almost like you had a loop. Does that make sense, a little bit? I know it's hard without an example. But the reason why it totally makes no difference with asynchronous programing is that whenever you do something asynchronous in JavaScript what happens is that it gets posted onto the event loop. Who here know what the event loop is? Couple of people? The way JavaScript works is it actually has a list of a queue of functions to call. Who's used settimeout? Right? Whenever you do a settimeout what's actually happening under the hood is it takes the function; and what settimeout is, by the way; it's an API where you hand in a callback and a time and then the JavaScript environment is going to invoke that callback after a certain amount of time has elapsed. What happens under the hood is that JavaScript has got this big list of things to do. And that just takes that function and adds it to the big list of things to do. And then eventually, when JavaScript, a certain amount of time goes by, it'll execute that function. And then when that function completes it just moves to the next step in its list of things to do. Maybe that function that just completes inside of it, maybe that function that just ran, schedule another time out, and that would post it down here at the bottom of the cue in the list of things for JavaScript to do. My point is that if you're doing anything asynchronous it always just ends up getting posted on to the list of things to do and so it's not really, at that point, it's like effectively as if you're using tail recursion because JavaScript has got this big loop of just going through those things to do and so it's the equivalent of using tail recursion. My point here is that tail recursion really doesn't have very much to do with asynchronous programing because if you are scheduling the next function asynchronously, it's like tail recursion, because it's going to get put in this big list of things to do and JavaScript has this big loop where it's going through it. So it's like tail recursion as if you take a recursive function and optimize it into a loop just repeating the same operation again and again. So I think there may be a slight confusion there. Tail recursion not really a win for asynch programing. Yeah. - The next question was, can takeUntil take multiple observables as a stop collection or a collection of observables? - No. Actually, I'm not 100% sure. It might actually be able to take multiple collections as stop collections. However it doesn't matter because you can always go some source collection .takeUntil, some stop collection. .takeUntil another stop collection, .takeUntil another stop collection, .takeUntil another stop collection. So composition solves this problem for us. So any more questions? Great. So now you guys know everything you need to know to understand how this sample works at least at a high level. A conceptual level, right? We've got a function which accepts a DOM object and then all the mouseDowns, and mouseMoves, and mouseUps here were created using observable.fromEvent That's the piece of code I've left out of here. Remember that piece of code that adapts from a DOM event to an observable? So the mouseMove, mouseDowns and mouseUps on that DOM element were all created using observable.fromEvent So we adapted already those mouseMoves into observable objects. So I can go back and quickly remind you guys. Right. It was this function. So imagine that we took the DOM event and we actually called fromEvent for mouseDown, mouseMove and mouseUp. And then, we created the observable, we took the observable that came back and actually added it to the DOM event. Instead of just talking about it, I'll do it. So first things first. We create the mouseDown, mouseMove and mouseUp observable by adapting the from the DOM elements event name. We add those properties to the element itself. And then, that's where mouseDown, mouseMoves, mouseUps come from on the DOM element. Does that make sense? Is that code clear? Maybe not so much? I can make this even simpler. Just trying to get clever there. So the first thing we do is we adapt from all of the DOM events on this element. Close one. Still got me here? - Yeah. -So we adapt from all of the mouse events, the DOM events, to an observable, and when we just take the observable that's on there and we stick it on the element itself. And now we can use, now that we've adapted the DOM event which is this weird add event listener, remove event listener thing, into a single first class object, we can use these map filter functions to assemble a new event from three source events. So we're taking mouseDowns, mouseMoves and mouseUps and we're composing them together into mouseDrags. And then, at the bottom half of the slide all we're doing is; almost, a slight syntax error I want to fix here. Now that I've created a function with multiple lines; I need to add a return. The error syntax. Using the error syntax and you have to execute multiple lines of code, you need to add a brace and you need to add an explicit return method. But if you only have one line of code you can omit the return method and you can omit the braces. So now, this is a correct. And so what I've done is the very top of this slide I've created a function which when given a DOM element will return an observable of all the mouseDrags on that DOM element, and on the bottom half of this slide I'm consuming the mouseDrags. Now I'm forEaching over that observable that I've created. And I'm consuming the data and I'm doing something with it. In this case moving the image around. Does that make sense? MergeAll and SwitchLatest -We've already talked about concatAll. And the way it works top to bottom, left to right. Just the same way as it worked with array. So if you're ever confused about how observable works with concatAll just think about how an array gets flattened with concatAll. There's two more flattening strategies I want to call your attention to. And together these three flattening strategies you're going to be able to solve most asynch problems with. The next flattening strategy I want to call your attention to is what happens when you got three lanes of traffic and one gets closed down because of an accident? At least in a civil society. What happens? - They merge. - They merge first come, first serve, right? Whoever gets there first merges in. That would be the civil way to do things. Might not actually work that way in practice but in a perfect world, right? So let's run this and see what happens when we merge four lanes of observable, so to speak, into one. So once again we subscribe. We forEach over the outer observable and so we start getting inner observables arriving over time. Once we get an inner observable we forEach over that and one comes out immediately. And this time we immediately return. Just like in concatAll. Now, that one finishes, a second observable comes along, we forEach over that. We return two as soon as it comes out. And now we're waiting for three. Now in the previous example concatAll would not emit anymore values from subsequent observables until the observable it was currently listening to completed. Not so with merge. So while we're listening for three to come along another observable comes along. This time it's empty. We forEach over that. What happens to an empty observable when we flatten it? Disappears. There's no data in there. Nothing comes out. We're still listening for the second observable to complete, right? Another one comes along. Merge immediately forEaches over this one. It doesn't buffer it. It immediately forEaches over observables as soon as they arrive and we get out four. And then that completes. And then once that completes. We're still listening all this time, we've been listening to that second observable. Finally it gives us a value and then completes and we get this order right here. So in concatAll, data is emitted based on the order of the collection that it was inside of arrived. In mergeAll it's first come first serve. Whatever data comes out we immediately emit. We don't buffer up observables. We just forEach over them as soon as they arrive and as soon as their data comes through it just gets emitted. Is that relatively straight forward? So that's why we got items out of order here. Use mergeAll when you don't care about order. And you just want data throughput. You want data throughput as fast as you can. You don't care about order. You use concatAll when you care about order. That make sense? And we'll come up with more concrete examples later. The final pattern and this is perhaps the most common flattening pattern you will use in user interfaces. You will use it all the time, is switchLatest. So switchLatest starts out just like concatAll, so we subscribe to the outer observable. We get an inner observable. We forEach over that and then we immediately emit a value. Then we get a second observable. We forEach over that. Two pops out. We immediately emit that but while we're waiting for that second observable to complete another observable arrives. And switchLatest does is it switches to the latest observable. It says to the second observable I don't want any more data. By calling subscription.dispose So it unsubscribes from that previous observable. So we're never going to get the value three because we've said to the producer I don't want anymore data. And so once again, empty collection gets flattened there's nothing inside. Then another observable comes along we return this and finally when the outer observable completes. When the outer observable says I've got no more inner observable for you the whole thing completes. Yeah. - With the empty one, kill the second one? Or we only something with data killed the second one? - Really good question. Another way of doing switchLatest, and this is actually a legitimate flattening pattern you could use which is switchLatest on notification. Another way of handling it would be instead of switching from the second one when the third one arrive would be to wait for the third one to give us a piece of data. And that's what you're saying, right? What if we waited until the third one gave us a piece of data before switching? That's not how it works. We switch as soon as we get another observable. We don't wait for that observable to give us data. - So an empty observable will kill it. - Will kill it. Now that's actually a totally legitimate way of doing things as well. So there should be a pattern for that as well. I think there may be an Rx. There may not be. Frankly, this turns out to be the most common thing. Now why is this the most common thing? Because in user interfaces, users do this all the time. Let's say I got a simple button on a form and every single time you click it we go out and check Netlfix's stock price, for example, and put it on the screen. Now let's say I give that to my user, right? What's the first thing; better yet, let's say I give that to my QA person. I've got a quality assurance person at Netflix. Yup! The first thing that quality assurance person is just going to do is just going to bang on that button as fast as the can. They are such jerks. They always try to break stuff. What's going to happen is they jam on that button? Does anybody have an idea? - Just take the last one. - Well, with switchLatest, absolutely right. It'll cancel any outgoing request maybe before they were even made. Because remember, browser has queues of requests that are outgoing, right? And it will cancel a pending request. It might actually be able to cancel a request before it even heads out. Right? So it turns out to be very efficient to use switchLatest. Because you only got so many sockets that you can open to a server. There's only so many concurrent connections you can have. And that's why observable is so useful because the consumer can say to the producer I don't want anymore data. That gives the producer the opportunity to cancel or prevent any work that had scheduled to happen from happening. So in this case, the browser has a queue of outgoing requests. And if it turns around and calls abort; if the producer, if the observable turns around and says abort on the outgoing request it might actually prevent a request form going out before it even is issued. Sorry, you got a question back there? - Yeah. How does switch latest know when to dispose of some sort of debounce going on? - Well, it knows when to dispose because, remember switchLatest is subscribed to an outer observable. And every single time it gets an inner observable if it already is listening to another inner observable, that's when it knows to dispose. Because its job is to only listen to one inner observable at a time. And so as soon as it gets a new one, if it's listening to another one, it disposes of that one. So it disposes of the inner subscription to the inner observable it's currently listening to as soon as a new inner observable comes along. Does that make sense? Yeah. - So in your button example it's only going to listen to the first click and then all other clicks will be ignored? Or, will it be the last click? Or how does that work? - Good question. SwitchLatest. What we're really doing is we're taking clicks. And when we see this example later it'll make more sense. What I'm actually doing is I'm taking clicks and I'm mapping over every click and I'm replacing it with an observable that is actually the result of a network request. And so I'm taking a flat observable of just click object, click object, click object, click and then to object, right? And I'm mapping it into a two-dimensional observable. Because for every click object, I'm going to substitute in an observable that represents the result of the network request. And so now I have a two-dimensional observable. And what switchLatest is going to do is it's going to say, well whenever a new observable comes along because I've created this two-dimensional observable with map. When I apply switchLatest to it now I have a two-dimensional observable. And undo inner observables only going to come along when another click happens. And so that's, switchLatest can just; it doesn't know about the clicks or the map operation. All it knows about is this two-dimensional observable. And it knows that whenever a new observable comes along it disposes of the old one and starts listening to the new one. And just by calling disposed; what it's calling disposed on is on that observable that represents that asynchronous operation. And so when it calls disposed on the subscription object, returned by that observable that kicks off the asynchronous operation that producer can just say oh, abort the XHR. Abort the network request that goes out. I know that's a mouthful. We'll have examples and we'll go into it more clearly, right? But that's effectively what's going on. Every single click gets mapped into a network request which is also represented as an observable. And so we get a two-dimensional observable and we're only interested in one network request. One request for the stock price at a time. - Which one is the first click here? - Every single one of these observables represent a click because-- - Which was the first one that would lead to one. - One, yes. - Every single one of these durables represents a click because every single time I click a button I've mapped that into a network request. Think about each of these observables as a stock price. I know there's multiple values in here so you'd expect to see an observable of one. And you'd expect to see a bunch of dots in front of it. If you were to visualize what a network request would look like if that network request only returned one value it would look like dot, dot, dot, dot, dot, 52; and then ends. That's what a network request would look like as an observable. - I'm generally sad about losing data. Are we sad that we lose our clickstream. We don't necessarily... - We don't lose the clicks. Notice that this stream. Well, if the source of this stream is clicks this observable will go on forever. The flattened observable will go on forever because there could always be another click which always creates another network request. So if the only thing we're losing is the stock price that we tried to get for an individual click and that's a nuance. Sometimes it's hard for people. They think, oh we stop with the outer observable. No, we stop with the inner observables, and we've stopped listening to it, presumably because there's something newer that we're more interested in. And this happens in UI's all the time, right? I jump into forum and an animation starts but then I hit back. I want to cancel that animation or stop that animation, right? It happens all the time in user interfaces. Which is why this is going to be the most useful pattern that you're going to be using in user interfaces. Now, if you didn't have switchLatest you'd build state machines. You would have variables saying, oh I've got an animation going. I got to remember that I have an animation going just in case somebody hits back because then I need to check whether I've got an animation going and then I need to cancel that animation and then; It gets hard. It gets really hard. I'm going to show you how to do all this stuff declaratively. Instead of building a machine with a bunch of moving parts to compute an answer, we are going to write the answer. And that's all the difference in the world. We're going to do this declaratively. Got another question in the back. - Yeah, they were talking more of an example here for having a music player with previous, next song buttons. SwitchLatest. Would that accidentally go to the immediately next and previous song regardless of the amount of times you've clicked the control? - Well, it's hard to say without knowing how this two-dimensional observable was created. Every click could create an action object, right? Whether you click back, forward, or next, or whatever; you could map all the clicks on those buttons into actions which is observables that are asynchronous because doing player actions maybe asynchronous, right? When you play it might need to go to a server to load it. But some of those actions might synchronous. Stop might be able to be executed asynchronously. I don't know. But let's imagine for a second that each one of those player options was asynchronous. Maybe we're talking with a native player object so we have to send a message and we get a call back, right? Maybe it's not even a... Maybe it's a browser object and we need to talk through a native asynchronous interface. So let's say every single one of those clicks was asynchronous. We asynchronously got back a message telling us whether it was successful or not. Well then, let's take each one of those clicks map them into an observable that represents that specific player action. So when I call play on the player I get back an observable that completes when that play is actually successfully occurred. Well, then we have another observable of observables and then switchLatest would be perfectly fine. Because if I hit skip forward to the next chapter, and then I decided, oh no I want to pause. If the skipping forward to a certain amount of time well then we can cancel skipping forward and then we could just switch over to the pause operation. Does that make sense? So this will be more concrete when we start to see larger programs that use switchLatest. We understand how this all fits together in context. For now though, it's just important we learn focus in on this one thing and understand what switchLatest does to a two-dimensional observable. Yeah. - I have a question. Does two, does that kill the three, or does four kill the three? - Actually, the empty observable kills the three. - Oh. - So it's the third observable that comes along. And that's the distinction. I'm glad you asked that question because that's an important distinction. It's not when an observable gives you data that you stop listening to the current observable. It's not when a new observable comes along and then gives you data that you stop listening to the previous observable. It's when a new observable comes along. - Oh. - Data or no data. Then you stop listening to the current observable. - Okay, I understand. Netflix Search Box -So we already covered this point. SwitchLatest and takeUntil are why I don't unsubscribe to events, ever. I never call a remove event listener. And I haven't in five years. My code isn't littered with memory links because I do takeUntil and switchLatest. Turns out, that's pretty much all you need. So, let's take a look at another example. This is where things get particularly sexy. If we look at the mouse drags example, we're just composing the other events, right? But I already told you, UI's are made up of usually three asynchronous actions. Events, followed by an asynchronous request to a network, for example. And then finally ending with an animation. So let's do something just a little more interesting and let's use observable to combine together events and asynchronous requests. So, who's built an auto complete box. Oh sorry, you got a question back there. - Can you filter ahead of time and ignore the empty observable? - Well, ahead of time implies that you can look ahead; you can't. With an observable you can only get data when it arrives. You have no notion of what's coming down the pipe much as when you subscribe to a mouseMove. You don't know when the next mouseMove is going to arrive. There is no look ahead with an observable. There's only the right now. I just threw a piece of cake at her. She can't look ahead and see that I have no more left over or I have 15 more slices. There's no way of knowing that. So there is no look ahead. So basically, the switchLatest operator has to get an observable and then because it doesn't know that it's empty and it won't know at all until it forEaches it doesn't try and figure out that it was empty. What it does is that as soon as it get a new inner observable it stopped listening to the current inner observable by calling subscription.dispose You can't look ahead with an observable. There's no foresight. Can't see into the future. Because it's a collection over time. Much the same way you can't see into the future you do not know what an observable will give you. So who's written an auto complete box before? Anybody find it really easy? Was it hard? I mean write it from scratch, not just use a component. Somebody used a component, that's great. If you write it from scratch it's one of those problems that seems like it's going to be really easy and you're going to go home at five o'clock and you don't. It's surprisingly hard. And it's everywhere on the web. Why would it be so hard to write an auto complete box? Anybody got an idea? There's a reason why people make components so they don't have to keep solving this. - There race between the inputs. - There's a race condition. The race condition is, I type A. I'm on Netflix search and I type A. And then we go off and we send a network request for all the search results for A. And then I type B. And then we send a network request for all the search results for AB. Is there any guarantee at all that the request I sent for A will come back before the request I sent for AB? No. Networks. Shit happens. Right? So, it can come back in any particular order. So I might end up getting the request for AB back, before the request for A and then showing the old data on top of the new data. That make sense? So how do we prevent that from happening? - SwitchLatest. - SwitchLatest! This guy's on the ball. Absolutely, right? We're going to start to see that some common UI patterns just fits into one of those three strategies, right? So, how do we prevent it? What's another problem that we might run into with the auto complete box that we can only solve with a little asynchrony? What happens if I type A, B, C, D, E, F, G? How many network requests do I want to send off? A, B, C, D, E, F, G. - If you're google, six. Google instant, you get the updated auto complete coming for every language. - I bet that they probably have at least some debounce in there. - Yeah, there's some. - They will have some debounce. But the point here is that you probably don't, because most of us aren't Google, you probably don't want to send off a network request for every single key somebody types. You want to chill, right? Just wait a little sec, a few milliseconds after somebody types a key and then if they type no other key then you want to take whatever is in the text box and send it off as a search request. Does that make sense? So, something sometimes called debouncing. Here's what an auto complete box looks like at Netflix. So, as we begin solving these problems we're going to start describing them in four steps. What collections do I have? What collection do I want? How do I get from the collections I have to the collection that I want and then once I've got the collection that I want what am I going to do with the data that comes out it? So what collections do we have when we're building an auto complete box? When I say collections, I mean observables, right? Observables are collections. What are the collections of events that we have that are interesting to help us build an auto complete box? - KeyPress? - KeyPresses, right? That's one interesting collection. Another one is the one created when we make a network request. When we call this getJSON function that you see up here, what we're really doing is we're creating an observable that will eventually onNext us the one value of this is the result from the server. And so, if you were to visualize what that observable looked like it would look like dot, dot, dot, dot, dot, dot, results. It this case, it happens to be inside of an array. Don't get confused. The fact that an observable happens to give you an array. It's no different that it gives you one, or two, or three. Don't get confused about the fact that we're handing off an array and that happens to be a collection. It's an observable of a value, and that value happens to be an array. So the way this works is technically getJSON will return you a string. Well yeah, a string in this particular case. Or a JSON object. Whatever that JSON object is. So, getJSON returns an observable and it's the result of whatever we get back from the server parsed as JSON. Does that make sense? At that URL. So it's an observable with one value in it. Does that seem weird to anybody? To create an observable with just one value in it? Nobody? Collections have one value, right? You can create an array with one value. Nothing weird about that. You can have an observable with one value. And so again, if you were to use visualizing that syntax we made up, for what that getJSON observable, just the getJSON observable would look like. It would look like dot, dot, dot, dot, dot, dot, JSON object. And then onCompleted. So it will onNext you and then immediately onComplete. Because it's only got the one value. So what are we doing here? We're taking the keyPresses and then we're using this handy throttle method. And the throttle method is very simple. It takes an observable that looks like A, B, C, D, E, F, and turns it into C F. It takes any items that come very, very close together and it just drops them and get the last one. So the idea here is if you get an item in the observable, instead of immediately returning that observable. The throttle operator will wait a couple milliseconds and see if another item arrives. And if another item arrives it just drops the other item. And then it takes the new item waits a couple of milliseconds and if a couple milliseconds go by, in this case 250 of them and no other items have arrived then, and only then does it take that and forward it along to whoever is listening. So that's why it turns A, B, C, D, E, F, into C, F. Does that make sense? So now we've solved the problem of issuing seven network requests when somebody types A, B, C, D, E, F. Right? So that's how we solve that problem. Now we just have a less chatty stream. So, we're going to take each one of those key objects. We don't care about those key events. We're just going to map. And map is all about substitution. It's all about replacing something in the stream with something else. And we are going to map it and return the observable we get from calling getJSON. So we're going to take this flat stream of keyPresses and turn it into a two-dimensional observable because each one of those key events is going to get replaced with an observable that represents the network request to the server. Now remember, it's an observable of one. But it's still an observable. So now we have an observable of observables. Because for every key press we're creating an observable and substituting it in that represents the network request. For every key press we're substituting an observable that represents the network request. So now that we've got a two-dimensional observable we've got to flatten. And we've got to flatten it so that we solve the race condition problem, right? So here's how we can do that. I'm going to talk about the retry operator in just a moment. But notice what we're doing here with takeUntil. I've got a network request and it's out for A. I've created my network request for A. And the I create B, and I create another network request, and it's going out for AB. Does that make sense? I type B, now we've got a network request for AB. Notice that because I apply takeUntil with keyPresses to each network request as soon as I type B that outgoing network request for A completes. It fires onCompleted and just stops. And it becomes empty. Right? So I've triggered a network request for A and then I click B. And that network request for A is immediately going to complete because now I've pressed the key. So it's going to look like dot, dot, dot, dot. And just a few more milliseconds it would actually gotten us the result. But instead it completes. And then we apply concatAll to the stream. And what happens to that observable that we completed with noting inside of it. - It's gone. - Whew. Right? We cancel the network request. I didn't write abort, or remove event listener, or any of that stuff. I declaratively described the conditions under which that network request was to end. And it was to end regardless of whether we got the data or not as soon as the next key press happened. Very different way of thinking about asynchronous programing. Right? I'm not canceling things. I'm not changing things. I'm describing declaratively the conditions under which I want streams to end. And so that network request we turn for A has now completed and now we've got an open network request for AB. And eventually when that finishes, concatAll is going to return the network request for AB. Because the previous observable is completed, so it's now on to the second observable. And we get out the search results for AB. And that outer observable is never going to complete. Is the search results sets observable that we've created ever going to complete? Anybody tell me. Is it going to end? Well, first of all, what is search results sets? It's the stream of all of the search results for every search that you enter into the text box. That's really what we've done. We've taken the streams that we had which is keyPresses and a function which creates more streams which is the network request and we've combined them together into one stream, the stream that we want which is the stream of all of the search results that arrive from the server. And then, at the very bottom we forEached over them and for every search result set that we get back which is basically, could be an array of search results we're just going to put it on screen. That's what we're going to do with it. So let's go over the four steps. Yeah, question. - Question was, would we ever want to concatenate the throttle presses together to pass to get JSON instead of looking at input.value? - Yes. We could. So I'm making stuff easy on myself here. Now, because at any point I can just look up the value property of the text box, that's easy. But imagine you didn't have access to the value of the text box. Imagine all you got was key press signals. Well, then you have to figure out some way of concatenating them together and basically doing the job that the text box does which is oh, I'm going to aggregate up. What the current string is in that text box, I'm going to get a back press key I'm going to have to remove one. There is actually a way of doing that. It's called the scan operator which allows you to do exactly that to take an observable that's infinite and aggregate up values over time and then change them. We'll get to that later on. But, you can get the value of a text box. So let's not bother, that's much more complicated, so let's just do it this way. But that's a totally valid question. Yes. - What does retry three do? - Can I come back to that? I want to get the overall structure first before we come back to that. That is important but I want to come back to it. I really want to make sure we go through, think about things in these four stages. Because it's going to be really important for you when you do your exercises. What are the streams that I have or functions that give me streams? What do I have? What do I want to create? Well, I want to take the keyPresses and I want to take my ability to issue network requests and get them as observables. And I want to create just a stream of all the search results that come back from the server. Because then all I'm going to do with it is I'm just going to display them on the screen. I just wish; think about this as a process of inspiration. Wouldn't it be great if I had an event that just fired every single time I had search results from the server. That would be super convenient, right? Then I could just subscribe to that and whenever one came out I could slap it on screen. Well, we don't have that but we can create it. So first you start, it's almost like starting backwards. Imagine what event you want and then take a look at the events you have. And then the hard part is figuring out how to take the events you have and combine them together using these functions; map, produce, filter, merge, zip that I'm going to be teaching you into the event that you want. And then once you got the event that you want, the stream of data that you want all that's left to do is to forEach over it and do something with the data. So we're seeing all three stages here in the sense that we've got keyPresses, we've got getJSON. Those are the streams and functions that will give us streams that we have, that are very easy have them. Then we're using all the map and retry and concatAll functions and takeUntil to combine all these streams together into the stream that we want which is the stream of search result sets. And then at the end we're using forEach to consume the data and put it on the screen. That make sense? Optimizing the Search -Now you mentioned earlier that you kind of thought we would be using switchLatest for this, right? Notice I'm using concatAll instead of switchLatest? Well, that's actually more appropriate. I'm just aping what switchLatest does here. Because with takeUntil and concatAll I'm simulating what switchLatest does. Because every single time a new observable comes along I've applied takeUntil to the event that causes the new observable to come along. So whenever a new observable comes along the old observable is going to complete and that's exactly what switchLatest does, and that's exactly why we can simplify this to this. So, that's effectively what switchLatest does. Every single new observable is just takeUntil the next observable comes along. So let's simplify this to this. And that is not bad. In terms of code for an auto complete box. We've solved some pretty hard problems with very little code. Because we're thinking correctly about what asynchronous programs really are. They're thinking about them as streams of data that arrive over time. No different than an array. So, retry. Let's talk about retry. How many people here know what promises are? Some people, a little bit. One thing I will not be talking very much in this particular class about is promises. Does that surprise some people? Probably some people expected me to cover promises in an asynchronous JavaScript class. The reason why I won't be covering promises is that they are not very useful for the vast majority of things that you do in a user interface. That might seem odd. They are in JavaScript. They're actually being added to JavaScript six. That doesn't mean that they are the right asynchronous object to use for most of the problems that you faced in user interface design. In fact, I think they're not the right. And there's a very simple reason for that. Promises cannot be canceled. So, promises are obviously not the right; for those of you who don't know what a promise is. A promise is an object which eventually resolves asynchronously to a value. Now promise can resolve either with a value or it can reject with an error. It's a little bit like an observable that can only give you a single value. So you can call a Venn method which is like forEach and you can give it two callbacks instead of three callbacks. You only give it two call backs because a promise can only give you a single value so there's no need for the onCompleted call back. As soon as it gives you one value, you know you're done. The reason why we don't use promises for asynchronous network requests or for animations for that matter is because you might need to cancel them. And a promise cannot be canceled. So once you create a promise object it's either going to resolve or it's going to reject but there's no way of telling it hey stop what you're doing. And that just doesn't make it very appropriate for most of UI actions. Promises are useful for some things like asynch actions that can't be canceled effectively. But almost none of those things are what you do use when you use; almost none of the asynch API's you use when building UI's does that apply to, right? If you open a forum and then somebody goes back. You might want to cancel animations. You want to unhook event handlers and so on, and so forth, right? You want to clean up after yourself. With a promise there's just no way of doing that. So I don't think it's a very useful asynchronous primitive on the client. It's more useful on the server than it is in the client. But this is Front End Masters. And that's why I'm not going to be showing you promises. Observables are capable of doing everything a promise is capable of doing, right? Observable can send you one value. It can push you a thousand values. The only difference is it can do more. And that's why I'm teaching you observables. It's more appropriate because it's able with this one type to model, not just asynch request and animations but events as well. And so you can take the average unit of work that a user interface does and do it with a single type. And it turns out that it's actually much more elegant. Okay? So, one of the things, because you can't cancel a promise and a promise is always a hot data source. So, when you get a promise it's already going. The network request is already going. That's not true necessarily of an observable. Remember the getJSON function we called here? When we get that observable out, you know what? We haven't issued a single network request. Creating the observable doesn't issue the network request. What we're talking about here is that example of the cold observable I talked about before. Nothing happens until you forEach over it. So, when you get that observable out it's not going to do a thing. Until you forEach over it which the switchLatest operator will do, right? When it gets the inner observable it forEaches over it. So that's the distinction between that and promises. When you call a function that returns a promise the work is already happening. Does that make sense? So, a promise there's no way to retry a promise. A promise is a computation that's going right now. You can call the function again that gave you the promise and you can get it again. And that'll retry the work. But with a promise itself, by itself there's no way to retry an operation. That's because a promise the work is already being done. With an observable because it's forEach that triggers the action. You can call forEach on one observable three times. In this case the observable comes back and getJSON and trigger three network requests. So if you call forEach on this observable three times you'll get three network requests. That might not be what you want but that's what this particular function will do. So, you probably don't want to issue three network requests if you call forEach concurrently three times, right? You'd rather have it issue one and there's a way to do that. But by default this observable will actually issue three network requests. But now what happens if you get a network request and it fails. Well, you can just call forEach all over again and trigger another retry to get that information. So forEach in its lazy nature which is another way calling it laziness. An observable is lazy. It doesn't necessarily do any work, because it doesn't need to until you call forEach on it, right? Because it's like a tree falling in the forest with nobody around to hear it. Why should an observable do work to produce value is nobody is listening yet? Does that make sense? So when you call forEach that's when the observable starts doing work. And that's why we call it lazy. So, in this case if you call forEach on the getJSON observable it'll trigger a network request. And what retry does is you get retry an observable and it calls forEach and if it gets an onError back. If onError gets called then it'll increment a counter and call forEach again. Does that make sense? And it'll just keep calling forEach until the counter gets to a certain level. And then if it still gets an error it will forward the error along to whoever is listening. Does that make sense? And so, network requests are a great example of something we almost always want to retry. Because intermittent network failures happen all the time. It's probably not a good idea to issue a network request and if it immediately error just give up. Because, server is not perfect. Servers have problems. If you're scaled out you might as well retry and try to hit another server. So for those and many other reasons observable is a better way of modeling asynchronous requests than promises. Because A, you can cancel them B, you can retry them and C, frankly, some network requests might stream data. Right? They might stream multiple pieces of data. And then observables obviously a better fit than that than promises which can only send you a single value. So I won't be talking very much about promises in Front End Masters. If you have questions about it we can talk about it. But my recommendation is that you don't use them. And for UI development. For pervasive UI development. I don't think they're the right level of abstraction for most asynch problems. I think observable is the right level of abstraction for most of the asynch problems you use in a user interface. So, do we understand this slide? We're going to see the same process happen again. We're going to start with an event. We are going to map it into an asynchronous request thus creating a two-dimensional observable and then we're going to pick one of the three flattening strategies to flatten it out based on what we want. So those three steps. We'll take the observables we have compose them into the observable we want, those are three steps, right? Once we decide what the observable we want is and then we'll forEach over it and we'll consume the data and we'll do something with it. And that's that last step. I take the search results set and I put it on screen. I've done something with it. Any questions? - Not sure if this is directed to you but, is retry injected to the prototype of functions to work on any functions we create? - Retry is not on a function. So that's difficult to see here. The fellow who thinks that retry is on a function, it's not. It's on observable. getJSON returns an observable. So retry is on the observables prototype. So it's a method on observable. Because getJSON returns an observable. We're not calling retry on a function. We're calling it on the result of the function which is an observable. Right? So because observables are lazy you can write retry over an observable. Because you just keep calling forEach and restarting the action. But if observable wasn't lazy like a promise, you can't call retry on it. Yeah, question in the back. - If we use switchLatest why do we use the throttle? Is it for optimization? - No. The throttle is what turns A, B, C, D, E, F, into C, F. That to keep us from sending many, many network requests. Oh, but I see the question he's asking. That's actually a great question. If we remove this throttle right here we would actually get the exact same result. It's a very perceptive question, because he's looking at this and he's saying, well why are you doing this? Because, every single time we get a new key press. We're just going to cancel the outgoing network request anyway which is that all we've done with throttle is we've just eliminated a few keys before we issue a network request. So throttle is actually purely a performance optimization in this case because we dropped A and B, right? Let's say we didn't drop A and B, and D and E, and we just actually let A, B, C, D, E, F, all the way through. Well, we still would have just created a bunch of network requests and then canceled them. And so, by filtering out A and B, from A, B, C, D, E, F. And D and E from A, B, C, D, E, F, we just avoid creating network requests that we're going to turn around and cancel immediately anyways. So it's actually cheaper because we don't create the observable. So that's a great question. It's actually purely a performance supposition, yes. - So does it cancel the request or does it just not listen to the result? - Great question. Well, it's really all up to the observable, right? If she says, look I don't want anymore cake, right. I'm the producer. I might be able to pick up the phone and call the cake makers and say stop the order. Stop making cake. I don't want it anymore. Or, I could just, you know what? Leave the phone on the hook and when the cake comes just throw it in the garbage. It's totally up to me. Now, most observables understandably well written ones would want to do the former and not the latter and so in practice, yes that is what JSON does. It actually goes to the XHR and calls abort. - Oh, okay. So, that may or may not actually stop the network request from going out but at the very minimum it'll stop you from getting called with the results. - Okay. - Right? But very often in fact you will be able to stop the network requests from going out especially in cases like this where you've got a lot of inputs coming in. Because what is going to happen is the browser will build a queue of outgoing requests. It's not that you issued a request already. It's that actually you're waiting to issue a request. Yeah. - Can you use retry to poll a request. - To pull a request? - To poll. - You wouldn't use retry to poll, although retry is a lot like poll because you keep making a request again, and again, and again, and again. Right? But retry is specifically just for that one behavior of making a sequential request and only continuing as long as you're getting errors up until a certain point. We will learn how to poll, though. It is perfectly possible to do a poll by creating an observable that fires an item every 100 milliseconds or something. Like an interval. A set interval. So you could do a set interval which is just, you give the browser a call back and it just fires that callback again every 100 milliseconds or whatever interval you define. You can also create an observable that just onNext you every 100 milliseconds. And then you could take that and map that into a network request and now you got a poll. Right? Now you've got an input observable that's just going to fire every 100 milliseconds and if you map each one of those items it delivers an onNext it might be just null or true, or something into an observable that makes a network request now you've actually got a poll. Because every 100 milliseconds you make a network request. Yup. - Question. Are you going to get into the internals of getJSON later? - Yes, but not right now. For now, it's just you take it on faith that under the hood it's using XMLhttp request and it's going to onNext you the result and then onComplete immediately. Yeah. - Then, one more question. SwitchLatest is switching between multiple observables in this example. Is map returning multiple observables? - Well, no. Map always takes an observable and takes every item. Well, depends what he means multiple observables. Map, is just a very simple mechanical function that takes every item in a collection runs it through a function and it's effectively substituting that item for a new item into a new collection. Remember one, two, three. If I apply a plus one function to one, two and three. I get two, three, four into a new collection. Here in the case of map, the function is actually returning another collection. And so we're going from a one dimensional collection to a two-dimensional collection. Because the map function is returning yet another collection. So we go from one, two, three; to one, array of one comma, array of two, comma, array of three; effectively. So, that's what we're going to find again and again. We're going to see the same pattern. We're going to take events. We're going to map them into asynchronous requests creating a two-dimensional collection and then we're going to pick the right flattening pattern to flatten them out again. And once we've flattened our observable then, and only then will we call forEach. Because calling forEach over a two-dimensional observable is not a lot of fun. It's really weird, right? It's hard. Who want to have to deal with this inner and outer observable stuff. You let the flattening operators do that work for you and then you only have to worry about a nice flat observable of results. Does that make sense? - Where are the multiple observables coming from? - Ah. For every key press, right? Let's look what happens for every keyPress we run it through a map and getJSON returns an observable. And so that means for every keyPress object we create a new observable. So that's where the multiple observables are coming from. It's because now we have; before we just had a stream of key event objects and now after the map operation we have a stream of observables. We have an observable of observables. Because the map function is taking each one of those keyPresses and converting into another observable. Yes. - Is the inner collection returned by map, does that only have one element? - In this instance, yes. It's an observable of one because getJSON just makes a network request takes whatever comes back from the network and then onNext it and then immediately onComplete. So yes. It only contains one element. So we're creating an observable of observables of one. Three-dimensional Collections -So, one of the things that you guys are going to get more comfortable with as we do this is dealing with multi-dimensional collection. In fact, what you're going to be doing is you're going to be building many dimensioned collections and then applying the right number of flattens to flatten it back out into one and then calling forEach. And that is what the exercises you're going to be doing today are going to be drilling again, and again, and again. I'm going to keep mapping. Keep building nested collections until I've got all the data that I need in scope through a closure. So notice I've got; well this is a bad example of that, but that's what you'll be doing. And then you'll be applying the right flattening strategy, the right number of times to flatten it out into a nice flattened collection again and then forEach over the result. That is not a skill that comes naturally to most people. It needs to be drilled. It needs to be repeated. But once you've got it, you're only going to be learning five functions. And you're going to get the hang of this way of programing. And you're going to be able to apply it in a lot of places. Maybe unexpected places. Places even beyond asynchronous programing. So I'm really excited about that. So, I'm going to go with over one example. We're going to just bump it up to three dimensions now. So the Netflix player; c'mon, you can do this, right? Let's take a look at this hornet's nest of code that I used to have, right? We saw this earlier. This is the bad old way of doing things, right? We have an asynchronous programing with a bunch of callbacks and a bunch of state and all that stuff; and we're going to replace it with this. So what are the collections I have? What collection do I want? And how am I going to get from A to B? So with the collections I have are that somebody clicked play. Another collection is that somebody hits cancel. Another collection is that we successfully authorize a movie request. That's an observable of one that comes back from the server. Another collection is; I think that's it, actually. Oh, another collection is that the player is initialized, right? So, the player could initialize asynchronously. We could asynchronously authorize a movie request. Somebody could hit a cancel button asynchronously, right? I think those are the three ones. Yes. Those are the three collections. So if I express all of those as observables I can use these methods to compose them together into the collection I want which is movies to play. I as the UI developer just want to create a nice little event that says play this movie, play this movie, play this move. And then I'll forEach over it and I'll play the movie. So what I've created is a stream of authorized movie ID's. Movie ID's that were successfully authorized that some user tries to play. And once they're authorized I actually want to go ahead and play them. So we start with player.init Player.init just returns an observable that fires a single value onNext when it actually successfully initializes. Does that make sense? So it's just an observable of one. Just like making that network request. I call player.init and that gives me back an observable and that observable actually technically when I call init it's going to give me back an observable that does absolutely nothing. Nothing's happened. All I've done is create an observable. Observables are lazy. So it's going to wait until I forEach over that observable to actually try and initialize the player. And then it's going to complete after onNexting some sentinel value like true; like a finish. As soon as that player is initialized. Does that make sense? So I call player.init It gives me observable. Not a damn thing has happened. Then I call forEach on it. It actually then tries to initialize the player. And once that player successfully initialized it's going to call onNext, give me some value like true and then immediately call onCompleted. Does that make sense? I'm going to map over that observable. Notice it's an observable of one. I don't care, I'm still going to map over it. And when I get that value of true which I don't really care about so my function accepts no arguments I'm going to then listen to all of the attempts to play. All of the users attempts to play for ever. So this is actually going to happen at start up. I'm going to do this. I'm going to first listen for the player to start initializing and then all future attempts to play the movie I'm going to listen for this stream. That just means a click on the play button on the player. I'm just taking that click that DOM event and I've adapted it into an observable using observable from events. So that's what player attempts are. I'm going to map over all those. And that's just a stream of movie ID's that they try and click. So if they try and click Die Hard what might come out is 26295. They try and click House of Cards, 5291. Then I'm going to call player.authorize which is actually going to make a request to the server and just make sure that we can actually, you're allowed to play this movie because you're logged in and that type of thing, right? You're allowed to play the movie. And it's going to come back with a decryption key. So we're going to take that stream of play attempts of movie ID's and we're going to map it into a stream of actual decryption keys that we retrieve from the server. Does that make sense? That's simple substitution. For every movie ID we substitute it for a network request to get back a decryption key. So now, how many levels deep are we? I think we're three levels deep. Because for every player initialization even though there's only going to be one we're going to listen to all the play attempts. But for every play attempt, we're going to issue a network request that'll actually resolve with the decryption key. So we're three levels deep. And how do I know that. Well, every single map function is returning another observable here. We have two map functions each of which is returning another observable. So if the function passed to map returns another observable you know you're deepening the collection by one level. So we have a three dimensional observable. And that means we need at least two flattening operations to flatten it back out into a one dimensional observable. So always glance down there at the bottom and notice that there's to concatAlls. So we've built a three dimensional observable. We applied two flattens to it and we ended up with a one-dimensional observable. And then at the end we just listened to all of the authorizations and we just play it. Now I know I've got the; so the authorization is actually a stream of license that you see there is actually the movie decryption key. So I'm just slurping up the movie decryption keys as they come along and then I'm saying to the player. Go ahead now, play this title. Yup. - Two questions. One; is cancels a custom stream that you made? - Yes. That's literally the cancel button or the back button for example that I've adapted using observable.fromevent - And then, observable.empty, what does that do? - Oh, good question. So just like you can now, observables error and so just like in your language where you have things like try catch, you want to be able to handle those errors. Right? If an observable errors you don't always want to complete the stream immediately. Because just like in synchronous programing you can sometimes catch errors and do something about them. In this particular case I just want to listen for player authorization all the time. And if for whatever reason some attempt to access the server errored I don't want to make the whole stream of authorizations end. I basically just want to ignore it. In this case I'm going to return nothing. Meaning, the play just won't work. So, you'll hit play. That's probably not the best, we could probably do even better that that. Right? It can be actually just play them the message. But it's what I can fit on the slide. What catch does is it takes an error. Excuse me, it passes an error to a function and that function can optionally return an observable to resume from. So instead of that error just completing the observable instead we can run that error through a function, create another observable and then think of as literally concating it onto the current observable. And so we just start listening to that observable now, which keeps the error from getting bubbled up and stopping the overall stream. So in this case it's as if you did a catch and then there was nothing in the body of the catch. And so an observable empty is literally an observable that just immediately onCompletes when you subscribe to it. So it's just a constant that will give you an observable that always completes. Right? But now, of course... Actually, it's a bad example here. Down here I've got an error handler. But that error handler I don't believe will ever get thrown because play attempts. Oh no, it could get thrown if the player initialization fails, right? If an error happens on player initialization there nothing I can do, so that error will bubble all the way up and then we'll handle it and say sorry. Can't play stuff right now. Notice we're only catching errors on attempts to authorize a movie ticket. We're not catching errors on player initialization. Notice where the catch is. It's very important. It's only on that inner observable. Not on the outer observable. So if player fails to initialize we'll get an error. We won't catch that. So, in Netflix now we, in our UI's we use observable everywhere. It's pretty much for every single asynchronous action represented as observable. Yeah, question. - On that example, could we have switchLatest on play attempts? - Absolutely. I'm glad somebody pointed that out. Anytime you see; oh actually, no, no. Actually, I'm you pointed out, because we can't. Just because we got noticed in the previous example we had a takeUntil inside of the map function and then a concatAll on the outside. That worked because the incoming event that was creating observables was the same as the event that we would want to use to switch to new observables, right? For every key press we created a new observable and then as soon as they pressed another key we wanted to use that event to cancel the current observable. In this case that's not true. We're listening to an entirely different event. So, if you press play that event triggers us making a network request. But it's an entirely different event that causes us to stop making the network request which is cancel. And that's why I had to use play attempts, map, return the authorization observable and then I had to do takeUntil on cancels. If I did takeUntil on play attempts then this would be the same as switchLatest. So switchLatest only works when you want to stop listening to an inner observable based on the same condition that causes you to get a new inner observable. So, if every single time I press a key I'm going to create a new network request so I therefore want to cancel the current network request, switchLatest is great. Because I can just do takeUntil keyPress, and the key presses are the same things creating the new observable. In this case I got two different events. One is creating observables and the other one is completing them. So that's why I can't use switchLatest. I have to use map, concatAll with a takeUntil inside. So for every observable I create, I say oh yeah, only stop when this other event happens. But, whenever this incoming event happens create a new observable. So I'm glad that question was asked. So switchLatest only works when the incoming event that creates the observable is the same event we want to use to complete the current observable when a new one comes along. So, and Netflix, we use observable everywhere for every single asynchronous API. Every single asynchronous API. Doesn't matter if it sends us a single value. Doesn't matter if it sends us 15 values. Whether it's a callback, or it's a WebSocket, or whatever. We abstract it as an observable. The very first thing we do is abstract it as an observable and then treat everything like it's a collection and use these powerful data sources. I recommend you do the exact same thing. Any questions on the initial one? Yeah. - Clarification; why does init generate an empty observable? - Technically init it generates an observable of one. So it fires one. Because if it generated an empty observable the map function would never get called, right? Because mmap function get called for every item in the stream. So actually it just generates true and then completed. That might seem a little weird. But usually, often things will do that just so they can be used along with map, right? Because if an onCompleted then the whole stream would complete. Notice that map just ignores whatever comes inside of it. It doesn't really care. All it cares is that it got called. That's how it knows that the player was initialized. Creating Array Functions Exercises 1-5 Open your browser. If you have Firefox, I recommend you use Firefox for this. But if you have Chrome or IE, it'll work just fine. The only reason to use Firefox is to use the arrow function, which I showed you earlier. This would be a really good time to learn the arrow function, it's going to save you a lot of typing. There was a question last time. What happens when a user will click multiple times on the play button? Oh, good question. Implicitly here, the code is assuming that if you click play multiple times, it will trigger a cancel event. Does that make sense? You're never going to get play, play, play, cancel, cancel, cancel. You always get play, cancel, play. Play, cancel, play. That's not visible in the code here, but that's effectively how it works. It's like, if you press play again on a different title, it's an implicit cancel, and so that cancel stream doesn't just have back buttons but it also has further, newer play events. Let's load up the learning exercises here. Have we all got this up? The way this works is we've got exercises, and you go down here, you run them, and no news is good news. If it runs and nothing happens, you know that you can scroll down and go to the next exercise. The first few exercises are filled out for you. In this particular exercise, we're just solving stuff using loops. What we're going to be doing is, as we fill out these exercises, at first, we're going to have you solve a problem using the familiar loop approach, and then we're going to have you solve that same problem using one of the few functions, forEach, map, reduce, filter, and zip. We're going to teach you a new function, how to solve a problem with a loop, and then how do you solve that same problem with a new function? Here we're using what we shouldn't be using and what we can't use for async functions, which is for. We've got a for loop. We're just looping through a bunch of names, we're putting them out. Can everybody see that? If I run this, nothing happened, but if I scroll down, I'll see that the next exercise has been revealed. That first exercise is already filled out for us. We scroll down to find that the for loop has now been replaced with the arrays forEach method. It's the same exercise as before, except this time, we're using forEach instead of the for loop. If I run this, it works. I know that because the next exercise beneath has been revealed. I want to go through the first exercise, well, technically the third exercise with you guys. It's called Projecting Arrays. So, projection is just a fancy word for what map does. It takes an array of items and takes each item and creates a new item and then puts all the new items into a new collection. That's sometimes projecting, and the reason why is it's taking light and transforming it. That's where that idea of projection comes from. That's kind of what map does. As items comes through, it transforms them. You get a new collection with each item transformed because it ran through the function. That's why we call it projecting. Now have an array of Netflix new releases, for example. I have a bunch of these new releases. What I'd like to get out, and I don't care about all these properties. All I want to get out is the ID and title. What I can do is I can loop over them using forEach. Technically, the forEach method, not a loop. Here it says insert code here. Sorry? I think so, yeah. Is that good enough? It may work. We've got this videoAndTitlePairs, and the goal here is to accumulate up just the ID and title from all these videos and put them inside of videoAndTitlePairs. I'm going to go newReleases... I'm going to create a little JSON object which has just the video.id and just the title. All we're doing is we're going over each of these objects, pulling out just the ID and title, and putting that into a new array, and that is what we are returning. videoAndTitlePairs, empty array, forEach over the new releases function, add just the ID and title of each video to the collection, then that's what we return. Let's give this a try. Let's try and call return. Run, rather. And it worked. We know it worked because stuff was revealed beneath. Now that we've done this operation, you probably recognize that this is exactly what map does. Map goes through a collection, applies some transformation to each item in the collection, not actually changing it but creating a new item, and then accumulating up those new items into an array. Instead of just using map, we are actually going to implement map. We are going to write map ourselves, and every single function you guys use today, you're going to write yourself to make sure that you actually know how it works. Map already exists on array, but we're going to replace it. We're just going to override it. Notice that what we do, we're taking the common bit of this, the pattern where, first we create an array to accumulate up the results, and then we forEach over this. In this context, we're adding map to the Array.prototype, which means we're adding map to the array type. This, in this context, actually means the array that we're mapping over. So, since forEach exists on array, we can forEach over it, and now all we have to do is take the projection function, this is the function that we want to apply to every item in the array, and apply it to the item in the array. Now, we're not done because we have to take that result that we translated and we need to add it to the new collection. And that's how map works. That's all there is to it. An array, it's a method on array which accepts a function to apply to every single item in a collection. It takes the results of calling that function all those times and puts them into a new array, and then returns that. That's implementation of map, five lines of code. Let's see if this works, we'll run it, and it does. What I'm going to do from this point on, we're going to turn these exercises over to you. It's really important that you guys actually do these yourself, and not just watch me do them. I'm going to do them at the end of every section, for example, for the video purposes, but I really want you guys to try and do them yourself, because you're not going to learn this unless you really start to build those new neural pathways by putting yourself, you know, at a loss by trying something very, very differently. So, we're going to start from this point on. Is anybody having any problems getting to this point? Have you filled out the map definition? I'm going to teach you a coping skill. Coding inside of text boxes in web pages is not always convenient. One is, you can copy into your favorite text editor and you can finish it there and paste it in. That's one approach you can do. But if you're having some trouble and you're trying to figure out why something doesn't work, you can always take advantage of your browser's built in debugger tools. Let's go back to this particular example. Let's say, I don't know why something doesn't work. I can always use this approach. Who is familiar with this directive? Debugger, if you throw a debugger semicolon into any arbitrary piece of code, and then, in Chrome or in Firefox, open up your developer tools, and then run that code, you will see the code right here and it will break at that particular statement where you've written debugger semicolon. This is an invaluable tool to allow you to figure out what's going wrong with your program. I can tell you, the error messages are not that great. That's on me. But that's definitely something you want to do so you can step through and figure out, oh, why doesn't this thing look like what it's supposed to look like? Just open your developer tools, whether you're using Firefox or Chrome. I will also demonstrate why I told you guys to use Firefox, if you had Firefox, because the latest version of Firefox is great, because you can use the arrow functions that I showed off before. I'm not 100% sure if you can use the arrow functions with Chrome yet or not, I don't think so. Is that a no? - Canary. - If you have the Canary version of Chrome, you probably can. I'm going to fill out, ah, notice here? I'm giving away the solution to exercise five here, but notice I'm using that arrow instead of writing out function? That can be tremendous. Oh, let me make that bigger too. It's very, very nice to be able to use the arrow functions. So, if you can use them, I would highly recommend you use them to fill out the exercises. You can get Chrome Canary if you like Chrome, or you can use Firefox to do it, one way or the other. I think we left off on exercise three, which is, project an array of videos into an array of ID and title pairs. Remember, when we talk about projection, we're talking about the map function. We're talking about applying a function to every item in a collection, transforming it into a new item, and then putting all of those new items in a new collection. The goal here is to use forEach to forEach over new releases and then accumulate up just the ID and title of each video into this videoAndTitlePairs array, and that's what we've done here. So, I'll run it, it works, we'll move on. Now, implementing map is straightforward. All map does, we're adding a map method to an array, all it's going to do is forEach over this, which in this context is the array itself because we're adding the map to the array object, and then for every single one of these items, actually, let's look down here at the example. I find this is instructive. Let's take a look at this example, execution of map. Here, this is going to be this object, right? It's going to be the array on one, two, three. And projectionFunction is going to be this function right here. All we're going to do is we're going to forEach over ourself, so the first itemInArray is going to be one. First itemInArray is going to be one, then itemInArray is going to be two, and then iteminArray is going to be three. All we have to do is apply the projectionFunction to the itemInArray, and that will run this function here with x substituted as one the first time, x is two the second time, and x is three the third time. Take whatever comes out as the result and add it to the results array. And finally, return the result array, and that's how we get two, three, four. So, that's how map works. Let's solve the same problem we solved before. We want to get the ID and title from every single video in the newReleases, except this time, we're going to use map instead of forEach. Okay, it's there already, but that's effectively how we do it. We take the map function, we parse in a function which accepts the video, and then simply returns a JSON object containing only the ID and title. And we're done. As you can see already, there's a lot less boiler plate with map than using forEach and accumulating objects. I'm going to run that, great, works. Exercises 6-11 Let's try filtering. I think most of us would be very comfortable taking a collection, filtering the items in that collection and accumulating only those items in the collection that've parsed a certain test into a new collection using a loop. Most of us are pretty familiar with doing that using a loop. Let's try with forEach. I'm going to forEach over all the new releases, and if the video rating is five, I'm going to add it to a new collection and I'm going to return that. So, let's take a quick look at that. forEach is going to get called for every single one of the videos, one at a time. So, forEach is going to invoke the function with each one of these videos, and we are going to suck out just the ID and title and return that. No, excuse me, in this particular example, we're just grabbing the video. We're going to grab the whole thing if, and only if, the rating is five. If I run that, it works. Relatively straightforward. Notice that both map and filter, they're kind of doing the same thing. They're calling forEach to go over the array and then, based on some condition, they're accumulating up a new array of the objects in the previous array. So, I'm going to implement filter here. Oh, it's already implemented. Bear with me. Sorry guys, bear with me. I feel like the answers are already shown here, so I'm just going to hit restart the lab so we can run them together. So, we were on the filter example. So, we ran this, now we're going to implement filter. Hmm, still seeing these answers. Notice it's very similar to map. We're going to forEach over ourself. In this context, this is going to be this. This function is going to get called first with one, then with two, and then with three. So, x is going to be one, two, and then three. Our job is to apply, this time, instead of a projectionFunction which takes a value and then translates into another value, we're going to apply a predicateFunction. The whole job of predicateFunction, it's just a test function. It's just going to return true or false based on the value that comes in. The first time we parse this value in it's going to be one, and that's going to return false because it's not larger than two. So, we don't want to put it in the results array. So, we go if predicateFunction itemInArray, we so parse itemInArray to the predicateFunction, and if it parses, we add it to the results. So, that's how we run filter. Great. Now, we're going to solve the same problem we solved before. We're going to go through the newReleases function. We're only going to grab, actually, it's slightly different. We're going to grab the IDs of all the videos that have a rating of five. We're actually going to chain map and filter together now to solve a problem. I've got newReleases, all my video objects, and I'm just going to chain map and filter together. I'm going to run filter, and this is going to take a video, and if a video's rating is five, I want to keep it. I want you guys to pay attention to how I'm going to format my code as we do this. One of the things you guys are going to be noticing as you write these problems, you're going to start to write very, very long expressions. And if you're not careful, that can become not so readable pretty quick, so I want you to look at how I format these expressions. As soon as I have more than one method that I'm chaining off of a noun, like in this case, newReleases, I put each one of them onto their own function and I add a tab to indicate that the previous expression has not yet completed. Is that clear? If I'm only interested in getting the video, the ID and title of the video, so, after I filtered it for only those with a rating of five, I'm just going to map it, because what map does is it transforms objects. Filter just keeps objects from making a new collection, but map will take an object and substitute it for a new, transformed version of that object. I need to use parentheses here if I'm using the arrow function because JavaScript can't differentiate. If you've got a JSON object on the right hand side, it can't differentiate between whether you're trying to create or block that starts with a brace and then ends with another brace and you've got multiple lines, or if you're trying to create a JSON object. That's why I'm putting parentheses around this. That's ID title. Yeah? Oh, you're right. Yes, it's making the scrolling a little bit difficult, but I will try here. Yeah, because it's a window within a window, and if I scroll to the bottom of this, it won't let me scroll any further. - Oh, I see. Do the best you can. - Okay, so, first, we filter. That returns a new array, and so we chain the map operation right after that, and we can just grab the ID and title properties. Live coding here, see if this works. Uh oh, apparently I'm not supposed to grab the ID and title, I'm actually just supposed to grab the ID. Forgot about that. We'll shorten this to this, and we're good, it runs, okay? Now, the interesting thing about this is that we can query not just arrays, we can also transform trees. In other words, arrays of arrays. In this case, I have a movieLists array, each of which contains an object, so, this movieList... If you go to Netflix, you'll notice that there's all these movie lists. You have thrillers, horror movies, action movies, et cetera. And each one of them contains a list of titles, a list of videos. Here we have new releases, and that contains a videos array containing many video objects. But we also have a drama list, each of which contains videos. So, this is a tree, because we have an array of objects which contain arrays, right? We have an array of movieLists, action movies, thrillers. Each one of them has a videos array inside of it. What we need to do, we just want to collect up all the video IDs, all the IDs of the videos in all of the lists into one nice, flat collection. I'm actually going to do something I think most of us are pretty familiar with, which is nest a loop. I'm going to use my forEach loop and I'm going to do a nested forEach. Let's start with moveLists. For each movieLists, we are going to go through each of the titles, or is it videos? Yeah, videos. And then for each one of those videos, we are going to add the ID of that video to allVideoIdsInMovieLists. Formatter doesn't like it there. Don't forget the end parentheses after your forEach. So, it's just like a nested loop, right? How you would go over a two dimensional array. So, we run this. Uh oh, I probably mistyped that. Does that make sense? Just like a nested loop, what we'd do if we were traversing a two dimensional array, for example. Now, the thing is, if we have these two dimensional structures, what we would like is a method so that we can do the chaining that we did before to just flatten them for us so we don't have to write this nested forEach loop. If you remember in the slides, I showed you the concatAll method. The concatAll method is very simple. It takes a multidirectional array and decreases the dimensionality by one, it flattens it by one level. Here, we're going to add concatAll to the array. Here's an example of a two dimensional array. When concatAll gets called on this two dimensional array, this is going to be the outer array, this whole thing. Then within there, so, in other words, subArray is going to be each one of these. This function parsed to forEach is going to get called once for each of these sub arrays. So, all I have to do is, for each one of those sub arrays, call forEach again, and then I get an item inside each one. That'll be this one, this one, and this one for an individual sub array. Now all I have to do is add the items to the result and at the end, return the result array, and I will have created a function that we can use to flatten any two dimensional array. So, that's it. Now, we can factor out that nested forEach from our code and just put it into one helper function. Exercise 11, we're going to do our first transformation operation, pure transformation operation using map and concatAll on a tree, and this is a really important skill to get down. We have map and concatAll, and we want to flatten the movieLists into an array of video IDs. So, the same problem we saw before, but this time with map and concatAll. I'm going to start with movieLists just like before, and the first thing I'm going to do is I'm going to filter, or rather, I'm going to map over movieLists, and that's going to give me an individual movieList. In other words, first my function will be called with this, one of the movieLists, and next, the map function will be called with Dramas. For each one of these movieLists, I want to retrieve just the IDs of the titles inside, each of the IDs of the videos. So, I want to go through the videos but I only want to grab the IDs. Now, I'm going to nest. I'm going to put another map expression in here. Oops. We're going to map over all the videos in each movieList and I'm only going to return, for each one of these videos, I'm only going to return the ID. Let me ask you a question. How many dimensions is this collection? How many dimensions is what I have right now? Two, why is it two? Well, for every single one of the movieLists, I returned a filtered, excuse me, a transformed array of the videos inside. I took a look at each one of the video arrays in the movieLists and I mapped over it, and we know map returns another array. So, I've got an array of arrays. And how do I flatten an array of arrays? - concatAll? - concatAll. I'm going to finish that. So, that's less typing, right? - I was trying to do it the other way, doing concatAll first and then map after. - Interesting. Why not flatten first, right? Because what we've got is this nested object, why can't we flatten the nested object first? Sometimes we can do that. The reason we can't do that now is that concatAll only works on an array of arrays. It will not work on an array of objects which contain arrays, and that's what this is. It's an array of objects which contain arrays. But by applying map, I was able to transform each one of these objects into just the array inside of it, with just the ID that is. So, that's how we turned it into an array of arrays that we could apply concatAll to, and that's a skill you're going to use again and again and again. You're going to take a nested object, a nested tree. It might contain objects, it might contain arrays. By using map to convert it into an array of arrays, you'll be able to apply concatAll to it. But let's say movieLists look like this, which is totally possible. Maybe movieLists is just an array of arrays. The reason it's not here is that we want to have the name of the genre list, like drama and that type of thing, so we want an object with a name property. But if it was like that, then we actually could use concatAll right at the top and then filter it and then map it and grab only the video IDs after the fact. That's why we have to first map to turn it into an array of arrays, and then apply concatAll. Exercise 12 I want to talk to you a little bit about exercise 12, because exercise 12, you're going to make the first mental leap, first really important mental leap in order to get really, really good at asynchronous programming. First thing I want to call your attention to is that this is not allowed. When you fill out exercise 12, you're not allowed to pull a value out of an array synchronously like this. Why would I put that kind of restriction on you? Sorry? Because we're learning asynchronous. Because we're learning asynchronous programming! We are going to limit ourselves to only those operations which will work not just on an array, but also on an observable as well. So, good ahead, read that question very carefully and give it a shot. Let me know once you pass exercise 12. Yes, question? - There's a quick question, just the ordering of doing things. He did a map, then a concat, then a map again. - Which would totally work, because first he's using map to turn it into an array of arrays, because all he's returning is the videos. What he's saying is that he did this, which I want to point out, will work. At the same time as you're returning the videos, which turns into an array of arrays, you can also map that and just return the IDs. - He was wondering if it's less effective though. - It is less efficient probably. I'm not 100% sure about that, but less efficient. Actually, in this particular case, yes, it's almost certainly less efficient with arrays because every single map operation starts at the beginning and then traverses all the way down. If you're already going over the array, it'd be good to sort of, you know, do a transform. I'm not 100% sure in this case. If you're doing any filtering, certainly, you want to move filter operations up as early as possible because then all subsequent operations don't need to process those items that you were eventually going to filter out anyway. In this particular case, yes, it is probably less efficient because each time you call map, you're also allocating a whole new array. How many people got done? Not surprising, it's hard. I didn't expect any of you to get done. But I do want to talk you through this process because this is actually going to be a big mental leap. One of the things that developers are typically not comfortable with is dealing with nested collections. Like our friend over there, the first thing you tried to do was concatAll so that you could get that two dimensional collection down to a one dimensional collection, because that is so much easier to deal with, right? We're not comfortable with multidimensional collections for the most part, and that's what we have to become comfortable with in order to do asynchronous programming. That's actually key to really effective asynchronous programming, being comfortable. Basically, having nested map statements and effectively building trees, all secure in the knowledge that you can use concatAll to flatten it out again. You've got to be comfortable doing that. Here's what most people do. Let's go over the problem here. We want to retrieve the ID, title, and the particular box art that's 150 by 200, the URL for box art that's 150 by 200, for every single video in this array of movieLists. Once again, we have an array of movieLists, each of which contains, so, here's our movieList. We have instant queue, and then your new releases. That's the outer array of movieLists, each one of which contains an inner array of videos. What we need to get is the ID, title, and box art that's only 150 by 200 for each one of the videos in there. A nice flat array of ID, title, and the 150 by 200 box art. Let's try and solve it. Well, I'm going to take the movieLists, and just like last time, I'm going to map over them. That's going to give me an individual movieList. Then, I am going to go ahead and return the movieList titles. This time I'm going to map it. Now I've got videos. Is it video or titles? Videos. And I'm going to return the ID of the video. I'm going to return the title of the video. And now I need the box art, right? Now I just need to grab that smallest box art. Let's just go through the videos box art collections. Spell box art, and we want the URL, actually, of 150 by 200 box art. I'm going to go video. and if we look at an actual video object, notice that every video object has an array inside of it of potential box arts. What I have to do is I have to find the box art that's 150 by 200 right here, and I have to pull out its URL. That shouldn't be too hard. It seems sort of like the map and filter chain example that we did before, right? We're first filtering for box art that meets this condition and then we're mapping and pulling out only the URL. Let's write that right in line here. I'm going to format this a little bit better so it's clear what I'm doing. First off, filter. I'm looking for the box art with the width of 150, was it? Yeah, and a height of 200. Once again, I'm going to be careful about my formatting because I want to keep things readable. Even though we're building one long expression here, it doesn't mean it can't be readable. As soon as I have two verbs on a single noun, I put each on its own line, and I'm going to return just the URL. Now that I've narrowed this list of box arts to only the one that has 150 by 200, all I want to do is pull out this URL. I think we're pretty much there. Try the right number of parentheses. - They're saying the ID of the video, you need video.id in that title. - Ah, thank you. Perfect. Now, I think, if it's anything like the last example, all I really have to do is use concatAll to take this two dimensional collection and flatten it out. I have movieLists, which is a noun, and I'm applying multiple verbs, multiple functions to it. As soon as I have multiple, then I put each of them on their own line. And then within each video in the movieList, I'm mapping that into a JSON object, a little JavaScript object here, and I'm pulling out the ID, the title, and the box art. In order to get the box art, I'm filtering all the box arts in the video and I'm grabbing only the one with a width of 150 by 200, and then I'm using map to just pull out its URL. It feels like we're pretty much done here. All I need to do is run it. We'll, that's a problem. Property list. What's that, sorry? - You have to concat on that line. - Oh, do I? I think you're right, I do, but I'll get into that in just a moment. That's actually kind of what I wanted to highlight. But I think there's also a syntax error here, specifically here. - You don't need that semicolon after map. - Oh, thank you. That was the source of the problem. Now I've got a missing argument list. Bear with me. In the map function, into the other map function. Anybody spot my error here? I'll try and make this a little bit more... That's correct. - Now that you still have an array. - Well, you've hit on what I actually wanted to hit on here. I'm not sure what the syntax error is. We won't spend too much time on it. Long story short, this isn't the right answer. And it's not the right answer, my syntax error aside, because box art is supposed to be just a URL. Notice here, box art is supposed to be just a URL in these results, but it's not. How do I know it's not? Well, it's the result of a filter and a map function. What does filter return? An array. What does map return? - An array. - Right, so box art is actually going to be an array. If this program runs correctly and I can get it to run correctly, it's actually going to be an array of the box art. Does that make sense? What we need to do is somehow get the array-ness off of that box art. The logical thing to do, and I think a lot of us would love to just do bracket zero, that very thing that I stubbornly told you you could not do. So, what's the answer? What are we really doing when we do this? We're binding a variable to an item inside of a collection. What I mean when I say bind is now whatever's in the first item position in that array, we can now refer to using that variable name. There's another way that you can do that. If I want to bind a variable to every item in a collection, I can use, I'll just type it right here, even though it's crazy. What have I done when I do this? I've bound X to every item in a collection. This is the asynchronous equivalent of this. They're just two ways of doing the same thing. Now I can refer, inside of that function anyway, X is every item in that collection. And I can do something with it. What I'm missing, what I'm actually missing, the problem is I'm trying to create this object too soon. What you've got to do is you've got to keep using map until you've introduced all, so, notice here I've used map and I've brought this identifier, this variable into scope because I've bound video to each of the videos in the movieLists collection, but I'm not done yet, because I actually need one more binding. I need to bind box art to the one box art inside of that videos that has the width of 150 by 200 and the URL. What I'm going to do is, instead of creating the object so early, which is a rookie move, I'm going to go video.boxarts, I'm going to move all of this stuff up here. Now, I want you to notice right here inside of this function that now I have everything I need to create my result because I've brought in the box art, which is now bound to this boxart object right here, into scope so that I can refer to it. I'm going to write out the answer and hopefully it'll become clear what I did here. Now I'm ready, because I've got everything I need. I've got video up here in the closure, I've got boxart up here in the closure. Title. Through the magic of closure scope, I've brought in all the data that I need. Now, that can kind of get a little complicated. I mean, how many levels deep am I now? How many levels deep is this collection? For every movieList, we're returning a collection. - Four levels. - Four levels, I hear four. Anybody have anything else? I feel like an auctioneer up here. Anybody got two? It's three, I guess. - It's three, it's three, because for every movieList, I'm returning a collection, but for each one of those videos, I'm returning another collection, so I have three levels deep of collection here, which isn't crazy, because I started with a three dimensional collection. We nested our maps until we got another three dimensional collection. Now, all I need to do is, I need to go from this three dimensional collection down to a one dimensional collection. The point is, we shouldn't be afraid of getting ourselves into a deeply nested collection. It's okay, because we can always fall back on our flattening operators to flatten it back out into a one dimensional collection. So, assuming I've got my brackets right, if I do this, I've now taken the two dimensional collection and I've flattened it into a one dimensional collection. I'm actually going to change my formatting as I go. As soon as I add two operators to a single object, I each put them on their own line. Now I've flattened it to a two dimensional collection. And we're down to a one dimensional collection. I'm going to run this, hopefully it works. Still missing that after the argument list. I might have mistyped something somewhere so I'm going to cheat and show the answer, but I assure you it'll look very much like this except with long functions, unfortunately. So, I mapped over all the movieLists, then I mapped over all the videos within that movieList. For each one of those videos, I mapped over the boxarts. Well, first I filtered them for only the ones with width of 150. It should have a 200 check there also. And then I map it. Now that I've got all the information I need introduced via the closure scope, I'm ready to create my result. And that's what you had to do. You have to keep mapping until you have a variable bound to every value you need to create your result, and once you've got to that point, that's easy. You just have to figure out how deep you are and apply N minus one concatAlls. I've created a three dimension deep collection so I just need two concatAlls to flatten it out again. Keep mapping until everything you need is bound to a variable by a closure, create your result and figure out how deep you are, and flatten by N minus one. You're going to be doing this all over the place. Yeah? - Could you create kind of the pyramid structure with this if you kept going? - Definitely, you will create a pyramid structure. Does anybody think that's a bad thing? - What's the pyramid structure? - The pyramid structure is what he's pointing out here which is that you see the code kind of goes like this. It goes like that. Anybody ever heard the expression pyramid of doom? We've got one person there. If you've done any Node programming, you very often end up with this type of nested callback. There's a node in order to do one thing after another thing. You have to nest it inside of callbacks and you end up with what some Node programmers call the pyramid of doom. This nesting is not a problem. There's really actually nothing wrong or bad about this nesting. One of the problems with Node.js when you do this nesting is that because they don't have observables and they don't use promises and they just use callbacks, one of the real problems with the pyramid of doom is that you have to manually handle errors. You have to manually forward errors up yourself. That's why pyramid of doom is very damaging in Node.js, because if you can be seven levels of callback deep and something goes wrong and you can actually correctly clean up after yourself, you are a much smarter developer than I am. Observable will automatically forward errors for you. If an error occurs anywhere within here, we can count on throw automatically doing it for us. But in the world of observable, with asynchronous programming, observable will forward on any errors in any of these observables up to the top so that you can catch them in one place, just like tryCatch. You shouldn't be afraid of nesting. The key is not to avoid nesting. The key is to get really good with your code formatting so that you can make sure that your code stays readable even at multiple levels of nesting. I mean, you've probably nested three forLoops, right? It's not a terribly crazy idea. Very often, you don't see yourself more than three or four levels nested. So, if you can nest three forLoops, you've got exactly the same number of scopes here as if we nested three maps. That's the key to solving the next X number of exercises. It's always the same thing. Keep nesting map expressions until you have a variable bound to everything you need to create your result. When you've got a variable bound to everything you need to create your result, you know you can stop, return that result, and then figure out how deep you are, apply N minus one concatAlls, and you're good. Exercises 13-14 Now let's try and implement concatMap. I think it's already up here. The reason why we're implementing concatMap is that it's such a common pattern to go map, map, map, and then follow up with a concatAll, concatAll, etc. and flatten out again. Because you're often doing a map followed by a concatAll, well, why don't we just save some typing and fuse those into one function called concatMap? As you can see here, it's relatively straighforward. concatMap takes a function. Now, the one catch is that it expects that function to return array, because we're going to take some flat array, presumably, and then map over it, apply a function to each item creating a new array, with the resulting array being a two dimensional array. In other words, it has to be two dimensional so we can apply a concatAll to it. Absolutely, so concatMap takes a projectionFunction that's expected to return an array when given a value, and then it flattens it out. Here we have Spanish, French, and English words. Zero, one, and two. A bunch of words for them. We can take all of those and call concatMap and look up the return, in this case, any of these methods. Excuse me, one of any of these items from the Spanish, French, English words, and that'll create a two dimensional array, and we'll flatten it on out. It's just a map followed by a concatAll, nothing to it. Just a convenience function. Whenever you see a concatMap, you've got to realize, we're using a map, we're deepening one level of nesting, and then we're flattening it out again, so it's convenient to do those things at the same time. Let's use concatMap to retrieve the ID title and the 150 by 200 box art URL for every video. I think we solved this problem earlier with map. Let me just make sure it's not showing all the answers here. I'll restart the lab. Which exercise were we on? - Fourteen. - Fourteen, thanks very much. I think, here we go. Okay, so we're on 14, let's try this out. Our previous solution just used map and concatAll. We're going to map over each movieList, and each movieList has a collection of videos which we'll map over as well. Each video has a boxarts collection. Now, remember, instead of creating the object right now, we've got to wait and keep mapping until every object we need to create our result is bound to a variable. So, we're going to map now over boxarts. It's actually, technically, we're going to filter first. We're looking for the boxart with a width of 150 and a height of 200. I'm going to bring that to the next line because we're going to follow that up with a map, and I'm going to map over that, and I'm going to grab just the URL of that box art. Actually, technically, now that I've got box art bound to a variable, videos bound to a variable, I have everything I need to create my result. So, I'm just going to create the final result, which is ID and title. Pull out the video.id and video.title, and the boxart, which in this case, is actually the URL. This is what the result is supposed to look like. So, instead of just the whole boxart, I'm just going to pull out the URL property and I'm going to finish up the JSON object here and then finish up my map, and then finish up my other map. And now, I'm going to apply concatAll because I want to flatten out two levels. I'm going to apply two concatAlls. So, that was the solution to the previous example. Now, concatMap is really simple. Any place you see a map followed by a concatAll, you can replace that. It should line up here. Yes, you can replace that with a concatMap which is just a convenience function. Now, I'm going to delete one concatAll, I'm going to go here. How do I know that this concatAll follows this map? Because of good formatting practice. I can identify it with my eye. This is why it's so important that we learn how to format large expressions. It's not that large expressions are too complicated. Just like any regular program, if you've got rid of all the tabs in your program, it would be totally unreadable. It's not that it's too complicated to have loops in your program, it's just that you need tabs to visually guide your eye about what the structure of the program is. So, I'm going to take map here, replace that with concatMap, and now we have a shorter version of the answer. I still get that same missing argument list. That might have to do with the fact that I'm using arrow functions here, I'm not sure. But I'll show the answer and run and I'll prove to you it's the same answer with the long functions. It's just the same thing with functions written the long way. I'm going to slightly change the formatting here so that it lines up, but that's the way it's supposed to look. Exercises 15-17 Now, I want to introduce you to a new function. This is the first new function in a while you'll have been introduced to. We introduced you to map and filter, and we talked about concatAll in the presentation, and we learned about concatMap which is just, you know, a shorthand for writing map followed by a concatAll. Now, I'm going to show you a function called reduce. What reduce does is it take an array with many, many items and reduces it to an array with one item. We're effectively aggregating up all of the values in an array and producing an array with a single result. I'm going to call your attention to something, which is that I've actually mucked with the definition of array. JavaScript's array definition is that you process an array and it returns a single value. In my definition of array, the array that you reduce, the reduce that you guys will be writing actually returns a single value inside of an array. Does anybody know why I might do that? Here's an example of reduce, to show you why I would use the reduce, well, the canonical example for reduce is a sum. Here's an example of using the reduce function to create a sum of all the items in an array. This is a function which takes an accumulated value and the current value. This is typically the convention when you're building a reduce. The very first time, I can also parse in a value to use for acc the very first time around. Here, I've parsed in zero. If I was to omit zero, it would still work and it would still produce the exact same thing. If you don't produce an initial value for acc, it uses the first item in the array. Does that make sense? So, what's going to happen is, the very first time through this function, acc will be one, and current will be two. The result of this function will be the addition of these two things which will create, of course... My dad's trying to call me. Which will create three. Now, the way reduce works is it feeds the result of the function back into the function. So, the next time around, acc, the accumulated value, is three, and current is the only remaining item in the array we haven't processed yet, which is three. And then, the final value is six. Instead of reduce returning six, it actually returns an array of six. Reduce, even though it always returns a single value, I wrap it inside of an array in my reduce function. JavaScript doesn't do this. Can anybody think of a reason why I would do this? I say JavaScript's reduce is wrong. It's a pretty arrogant thing to say, so why would I say that? In fact, lots of languages have the same definition of reduce, and I think they're all wrong too. Compatibility how, compatibility of what? Interesting, you mean I can chain a map function off of this and chain a filter function. Is that what you're getting at? - Yeah. Because if it returns an array, I can keep on chaining. Yes, that is a very good reason why you should return it inside of an array. There's an even better one, which is... What's this return? Anybody want to tell me what they think that returns? - The array. - Well, in my definition, yes, but JavaScript's definition where they don't return array, they return a value, what would it return? Undefined. Maybe, I'm not sure. Let's find out, I'm curious. NaN. NaN? Possibly. Let's try it out. I don't remember, it's been a long time, but I can tell you, whatever it returns is probably not what you want. Does returning undefined make your life easier here? Probably not, so let's find out. This is what JavaScript will return. It'll happily give you, oh, wait, I've already redefined reduce in here. I'll do it. Yeah, let me know. So, that gives me six, but what does the same thing give me when I have an empty array? Ooh, error, that's even less fun. Right? Well, I mean, actually, it's kind of understandable, right? There's really no way to answer the question of what a reduce is over an empty array. But if we return an array of the final result, there's a very logical thing for it to return. Anybody want to take a guess what that is? - Empty array? - Empty array. I suggest this is much better than an error. That's effectively why we should do it. That's a good enough reason alone to do it for array, but now apply reduce to an observable, and if you apply reduce to an observable, that's an asynchronous collection that arrives over time. You can't return a simple value from a reduce over an observable because you would have to block. In order to give you this... This is the observable, the fake observable syntax, there's absolutely no way to return this as a single value because we'd have to block and wait for the observable to complete and then and only then can we return it as a simple value. Does that make sense? It's this stream that arrives over time. JavaScript can't just block and wait for this stream to arrive before it returns the next value, and even if it could, we shouldn't. We wouldn't want to keep people from entering stuff into the UI while we were waiting for an observable to add up. That's why we return this. That's what reduce returns when you apply it to an observable. It returns you an observable that eventually just completes with the one result. Does that make sense? Reduce is written in such a way that it cannot be implemented over an observable using that definition, and that's actually a smell. It's a smell because it looks like reduce is actually doing too much. People were just abbreviating. They were like, "Well, if this function always returns "one value, why keep it inside of the collection? "Let's just take it out of the collection." Because it's just one value, right? That's exactly the type of shortcutting that works when you're looking at the array definition, but when you get down to the mathematics of it and you look at the fact that observable and array can both be expressed as collections and streams, there's only one definition of reduce that's general enough to work for both of them, and that is for reduce to return an array of one, or an observable, in this case, to return an observable of one. So, that's why I'm going to have you implement reduce, what I think is the right way, and then we're going to use that reduce definition later on when we use observable. Now, you guys should have a pretty good idea how reduce works, right? What's interesting about reduce, when you're trying to figure out, what function should I apply to solve the problem? If you're doing substitution, if you've got an array and you've got stuff inside and you want to substitute each item for another item, what operator is that? Map. I have an array, I've got items, and I want to transform each item inside and sort of substitute it in that location of the new array, map. What if I have an array with a bunch of items and I want to apply some test function to each of them and only keep the ones that pass a test? Filter. What if I have a two dimensional array and I want to flatten it out? Reduce is for the case where you need to look at at least two values at the same time to do your job. Notice that map and filter have got tunnel vision. They only look at one item at a time. But in order to do a sum, for example, you would need to at least look at two values at a time to do that job. You couldn't do a sum with a filter operation, or a map operation even. You only have the one value at a time. Does that make sense? Those are the types of tasks you'll be using reduce for. Let me jump back to here. Okay, this is 15, so we're going to assume you guys finished this one. Our job here is to find the largest box art. So, aggregation operations, sums, maximums, minimums. These are the types of things that you would use the reduce function for. In this case, I'm going to use forEach for it. This is the way you would approach it if you had a loop. You'd create a bunch of variables to store, oh, what's the current size? What's the largest box art I'm currently keeping track of? Then, you would forEach over boxarts and then we'd say, okay, what's the current size? We compute the size. If the size is larger than the current maxSize then we reset the largestBoxart variable to the new boxart, and we reset the maximum size to currentSize. By the time the loop is done, we'll have the largest boxart. This is very similar to the way you guys have probably written this type of code in the past. Loop through an array, use a variable to keep track of your accumulated value. You can take this exact same pattern and abstract it into reduce. Anything where you're doing this form of aggregation, you can replace it with reduce. Question? Would the custom implementation of reduce always return an array of one element? Yes, because reduce always takes the entire collection and reduces it to one element. But by returning it in an array, we have a logical thing that we can do if we get an empty array as an input. We have a logical output that's not so bad as throwing an error, and we can continue chaining on operators after the reduce operation. This is a very common mistake that's made when stream processing, which is taking APIs like reduce and having them return scalers instead of vectors. What we're going to do is we're going to implement reduce. This is actually already implemented for us. We already explained how reduce worked. There's basically an optional initialValue you can parse to reduce, but let's go over it one last time inside a reduce here. If I write this reduce expression, what reduce does is the very first time around, if I haven't specified an initial value, it just uses the first value as the initial value, so that's one, and then uses the next value as the current value, so that's two. What comes out of this function is going to be three, and whatever comes out of the reduce function is going to be fed back in as the accumulator for next time around. So, next time around this is going to be three, and this is going to be the only item in the array we haven't processed yet, which is three. Then, the final one is going to be six, and then reduce is going to return an array of six. If we, however, were to use an initial value then it would work like this. It would start up with zero as the initial value, which is what we parse in first, this accumulated value. Then, we get one. Then, we would get, predictably, one from the function. The next time around, one would be the accumulated value and two would be the current value because now we've moved on here. And then we get three. Next time around, three is the accumulated value, and three is the current value, and we get six. So, we get the same result regardless of whether we parsed an initial value of one or not, or zero or not. That's how reduce works. I won't spend too long looking at this function because, you know, that's pretty straightforward thing. We're just using a while loop to do the expected thing. Notice that in the end, the key thing I want to call out is that we make sure to keep it inside of an array. I wonder if I've got a bug in here. I wonder what happens if we have no items in this collection. Right, so here, notice? If the length is zero, we just return the exact same array. A zero length array, if you attempt to reduce it, produces a zero length array. Let's run that, that works. Now we're going to use reduce to find the largest rating. We have an array of movie ratings, so we're going to call reduce and parse in a function which starts an accumulated value and a current value. We're just going to return whatever the largest value is, so we're going to say, if accumulated is larger than current, we'll just return the accumulated, otherwise we'll return the current. I believe that's going to get us the value. What reduce is saying is, look, if the accumulated value is largest, larger than whatever value we're looking at right now, just return whatever is currently the largest value, but if the current value happens to be larger than the accumulated value, use that as the accumulated value the next time around. So, if we run this, it should work. Yes, it works. Does anybody want me to slow down and take a look at that a little bit more? Or did we get past 17? Okay. Exercises 18-19 Let's retrieve the URL of the largest boxart, okay. Now, this is an interesting one, because notice that reduce... It's kind of a challenging one, because sometimes when you're using your accumulated value, it needs to be, I mean, sometimes the result you want is not the same as the accumulated value you'd use for reduce. Let me show you an example. I've got to compare boxarts to each other, and I want to come up with a URL of the largest one. So, what I'm going to do is I'm just going to call reduce without an accumulated. I don't need to use my arrow function here. We're going to function the accumulated and current, and if the accumulated width times accumulated height is larger than current width times current height, actually, I think we can shorten this into the same term or expression that we used last time. Return accumulated versus current. It's almost exactly the same as last time. We just compare the current boxart with the last boxart and if the current one is larger, we return the current one. If the one we've accumulated thus far is larger, we return that one. I'm going to close this reduce off, but I'm not done yet. I've picked the largest boxart, but remember, the goal was to actually return the URL of the largest boxart. In this case, I'm expecting, I think, 425 by 150 is the largest. I'm not sure, actually. I think, I'm expecting to just get this. Anybody want to tell me what operator I would also have to chain off of reduce to just wrap that one property? - You'd have to map it. - Yeah, I'd have to map it. Now I've got an array containing just one boxart, and what I want is an array containing just the URL of that boxart, so we're substituting the boxart object for just the URL inside. I think that's right. Oops. What's my semi-colon for? Yeah, that worked. So, that's reduced. Now let's try reducing with an initial value. One of the challenging things about reduce is that sometimes the types of the object in the collection is not the same type as the object we want to come out of reduce. The result of reduce will usually be the same type as whatever the accumulated value is. If you allow the accumulated value to just be the first item in the array, then you know the type that comes out of reduce is going to be the same type as what's inside of the array. However, sometimes you don't want that. Sometimes if you provide your own accumulated values as a starting point, that last optional argument, then you would expect the result to come out of reduce to always be the type of that accumulated argument. It's not always true in JavaScript because there's no types, but generally speaking, that's true. Here I have an array of videos, but what I'm trying to get is a... Ah, interesting. We're trying to create a map. So, have an array of IDs and titles and what we want to create is a map where the ID is the key, and the value is the title. So, let's show you what I mean here. There's an example down here I believe. Oh, there isn't, that's a shame. Oh, here we go. I would expect an array containing only one result where we have a map of all the video IDs and their titles, whereas now we have an array of IDs and titles, we want to get this. So, we're taking all these objects and we're aggregating them up into one big map. But notice that what comes out at the end isn't an object that looks like ID and title. It's a different type, it's this map with all that stuff. What I'm going to do is I'm going to give it an accumulated value that's an empty map, because what I want to do is I want to take all these titles, I want to suck their data out, and I want to jam it in to this accumulated map so that in the end, all I'm left with is that accumulated map with all the data inside. So, let's try this. I'm going to use my accumulated map, and notice here, down here, I'm parsing an empty map as my accumulated value. That's what I want to use as my accumulated value. Every single time a new video comes in, I'm just going to pull the ID and title and stick it in there. Now, here is one definition that would work, actually, but I'm not going to do it, which is accumulatedMap video.id equals video.title. If we did this, it would work. We of course have to return the accumulatedMap. Does everybody see why that's going to work? We parse in an empty map, and for every single item... Current is going to be this item right down here. The video is going to be this item right up here. That's going to be passed into the accumulatedMap, and now we're just going to add the data inside of this video, ID and title, to the accumulatedMap and return it. The next time around, the same map will be the accumulatedMap, and we'll just have a different video. Now, it'll be this one. So, we suck the ID and title out, add it to the accumulatedMap, return that accumulatedMap, and once again, we just keep going like that until finally we've exhausted everything and we return the accumulatedMap. Now, that will work in JavaScript, and I don't want us to do it. I don't want us to do it for one simple reason. Inside of all of our functions that we parse to these operators, map, filter, reduce, here's what I want you guys to do. Never, ever change a value. Inside of your map function, inside of your filter function, your projection function, your predicate functions, I never, ever want you to change a variable ever. What we're doing here is we've got this accumulatedMap and we're changing it, we're actually changing what it is by sticking some properties onto it. Does that make sense what I mean when I say change it? I can do this without ever changing the accumulatedMap. I can do it without changing it at all. Here's how I'm going to do it. Well, here's one way of doing it, which is, I could actually clone the map. Not so efficiently, but this would work. Does everybody know why this would create a clone of the map? I'm turning this map, I'm serializing it into a string that's a JSON string and then I'm parsing that string. I've actually created a complete copy of that map. If I did this, I would accomplish what I'm trying to accomplish. I'm trying to avoid changing values. In this case, I'm creating a brand new value, and I'll set a property on that. That's okay, as long as I'm not, the point is, I don't want to change a value that's passed in to me. Let's say I have a value, and I call a function, and I parse that value in, and somebody else mucks with it, they change it. That's actually kind of complicated, because I never know whether somebody's going to muck with my value or not. Ideally, they would create a copy, and so I would be able to hold on to my object and they would be able to do whatever computation they needed to do. This concept of mutation, of changing an object once you have it actually creates a lot of complexity in an application. Notice that all the functions I've been showing you up until this point, they don't change arrays. Map doesn't change an array. Filter doesn't change an array. Map creates a new array, right? Filter creates a new array. All of the expressions that we've been writing thus far never change anything. They just create new data from old data, and that turns out to be a really wonderful property in a program. Without explaining necessarily why, what I'd like you to do is avoid changing any variables that you don't own and you haven't introduced yourself inside of these functions, and the way that you can do that, JavaScript has actually got a neat trick which allows you to do this without this, which is much more expensive. It's actually a cheap way of cloning an object. Who here is familiar with prototypal inheritance in JavaScript? We have a few people. In JavaScript, we have this notion of prototypal inheritance and what that means is, if I have, actually, I'll show it in the console. Let's say I create a person. - Just for reference, our advanced JavaScript course really covers this in very specific detail. - Okay, so, I'll be very brief. I'll just show the concept. So, I've created a person, and now let's say I want to create another object that has all the same stuff inside of person, but I also want to add some stuff to it. So, I don't want to change the person object. I want to create a whole new object that has all the same stuff inside of person, but extra properties as well. So, I'm going to create-- Absolutely. This is a little bit odd, but what this does, it doesn't create a clone of... Good, good call. It doesn't create a clone of person, and notice what happens if I look up the name, it looks like it creates a clone. It looks like it's actually created a copy because it's got the same name property. But what's actually happening is that every object in JavaScript has what's called a prototype. It actually has a pointer to another object which it's using as its prototype. Another way of saying that is every object in JavaScript is like a linked list. Another person is actually just an empty object and I can prove that to you. It's empty, it's actually got no properties in it. If it's got no properties in it, how does this work? Oops. Well, the way it works is that another person actually has a pointer to person. If you attempt to look up a key on another person and that key is not found, it follows the link to the next object, and then it finds it on person and it stops there. Does that make sense? It created the linked list of objects. Every JavaScript object is in fact a linked list of objects. If you attempt to look up a property on them and it's not there, it'll keep following the chain up. So, in essence, it's actually a cheap way, in some ways, to think about it, as copying an object. Because instead of just going through all the properties and then copying them into another object, what you've actually just done is you've just created a little link, a little pointer, a new object with a little pointer to the old one. So, I can use that technique here to avoid ever changing the accumulatedMap, and instead, make a very cheap quote unquote copy of it. So, what I've done here is I've made a whole new object. In fact, I'll even name it differently so it's clearer. Since I created this object, I can muck with it all I want. The whole point is to avoid changing somebody else's object. If an object gets passed into a function, you shouldn't muck with it. Not in a map or a filter or a reduced function. You shouldn't muck with other people's objects. But here, I've created the object myself. So, I can set this one property here and return it, and it works. What we keep doing is we're actually creating new little maps that hold on to all the old maps which are all still there. We've never changed any of them. The end result is the same, that we have a single map. Not sure where that problem is but--- It's a pretty interesting little-- I got nothing. It's a cheap way of copying an object. Instead of actually copying the object, we create another empty object and create a little pointer to the old object, and then every time you try and look up something on the new object and you can't find it, it walks the line until it can find an object with that property. Yep? If you create object.create, and let's say you change the original copy-- Then it'll muck with it. Then it'll make the new copy look like it's changed. We can actually try that right here. Because remember, it hasn't copied the data. It's actually just holding onto the old data. I think that's actually a good example. - So, you can still have issues doing it this way too then. - Yes, you could. Somebody could muck with the original object and then you'd be in a bad place. Why do I still have that open? The goal here is, JavaScript can't make sure that nobody mucks with an object. Well, that's not entirely true. You can call object.seal which is a way of actually locking down an object to avoid anybody making any copies to it. So, every now and then, that's actually a nice thing to do. If you want to make sure that nobody mucks with your object, you can object.seal on it and then nobody else can muck with it. But some languages can actually statically verify that nobody can change your object. That's kind of nice. In JavaScript, we're relying on the honor system here in this type of programming. Who's familiar with the term functional programming? Quite a few people. The idea behind functional programming is very much this. You always take data in and create new data. You don't change anybody else's data. That turns out to actually be a very simple and powerful way of building a program. Up 'til now, it's pretty much all we've been doing, right? We've been taking arrays, creating new arrays. We've never changed anything except in our implementation of map and our implementation of filter where we've created a new array that we own and modified that. That's okay. Creating your own objects and modifying them is fine. But if you think about it, up 'til now, we've never, ever taken somebody else's object and changed it, and I want you to continue doing that as we go over time. Think that way. Inside of your map and your filter function, it's a smell if you're having to change an object that was passed into you. That make sense? Exercise 20 So, we want to retrieve the ID, title, and smallest boxart URL for every video. Now let's put it all together. Map, reduce, concatAll. Okay, I'm going to use concatMap. For every movieList, let's see if we can do this here. Return. We're going to go through each movieList, and then for each movieList, we're going to go through the videos and I'm going to keep mapping until I get all of the variables I need bound to something. Now, video has a boxart collection. In order to find the smallest one, I'm going to have to use reduce. How do I know that? Because whenever you say smallest or largest they're implying some sort of comparison. Whenever you're doing some sort of comparison, you need to look at two items at the same time, and reduce is the only function that takes two arguments. Map doesn't, filter doesn't. Reduce accepts the function that takes two arguments. I'm going to call reduce here on the boxart, so, video.boxarts.reduce, and we're going to use accumulated value and current value, and I'm going to look for, does anybody remember what the condition is? The URL with the width, smallest boxart. So, we just want to find the boxart that's the smallest and it'll be very similar to the code we wrote earlier. If the accumulated value is smaller, we return it. If the current value is smaller, we return that. An extra return statement here. Now, this sub expression right here is going to return an array of one, an array of one boxart. That's what reduce does, right? It gives us an array of one item. Now, if I recall correctly, it's my job to return... Smallest box art. I'm not sure if I'm supposed to return, oh, the URL, great. I'm supposed to return the video ID and title, and the URL. I'm actually going to bring reduce down to the next level. As soon as I have more than one function that I execute, I bring each one to its own line so that I can see which ones are lined up against which object. I'm going to take a boxart, and now, notice, I have a variable bound to the video, I have a variable bound to the boxart, and I'm ready to create my result. And we're going to pull out the URL of the boxart. I think Firefox has some sort of problem with this code, but we'll see if it works. I'm going to end my map, my concatMap. There's a problem here which is that I want to use concatMap here. Very often you'll see concatMap, concatMap, concatMap, map, which mirrors the fact that normally it's map, concatAll, map, concatAll, map. In the end, you don't need three concatAlls for a three dimensional collection. You need two concatAlls for a three dimensional collection. You flatten it out by one dimension each time. This should work, and if it doesn't, we'll just show the answer because it's the right answer. Great. You keep mapping until you've got a variable bound to everything you need to create your result, and then you figure out how deep you are and apply the right number of concatAlls, or translate the right number of maps into concatMaps. Same thing. Exercises 21-23 Now you guys are going to be learning about a new function called zip, so I'm going to turn it back over to you to go through the exercises, and we'll learn about --- this is the last function that you guys are going to be learning, and then the rest of this is all drills. It's not hard to learn these functions individually. The hard part is how do you take a big problem and break it down into combinations of these functions? And that's the real thing that we're going to be drilling today. So we're going to break, and we're going to come back to this in say 15 minutes maybe. What time is it now? So it's 3:45. Yeah, okay, 15 minutes then. So let's try and go to Exercise 24. Every now and then, information isn't just organized into trees, but it's correlated based on its position in an array. Rarely, but every now and then. So I might have an array of videos as I do in this example and an array of bookmarks, and the only way to link them up and put them together is positionally. So we see that the first bookmark is correlated to the first video, and the second bookmark is correlated to the second video, and on and on and on. Excuse me, second bookmark, second video, and so on and so forth. Sometimes we want to put those two things together. We want to collect a little information. It's almost as if we have an array on the left-hand side, an array on the right-hand side, and we want to kind of merge each item in the array like teeth in a zipper. So the first time I ever read the zip method, I was very confused, and I was very angry. I was like what the _____ does this method do? Why are these academics naming these methods so strangely? And then when I learned what zip does, it actually makes a lot of sense. So I've got an array on the left-hand side, array on the right-hand side, and the only way you can zip up a zipper, right, is if you've got a tooth on both ends. Anybody ever break a zipper on a jacket before? Really frustrating. You're like it won't go up because a tooth is missing or bent on one side. That's the way zip works. So it will go and slurp up items, one from each side of the array until one of the arrays stops, and then it won't go any further. So notice here I very deliberately have four videos and three bookmarks. So what we're going to do in this particular example is we're going to loop through, we're going to use a loop, and we are going to loop only to the length of the shortest array. So notice I'm using Math.min here to say we're only going to go as long as whichever videos or bookmarks length. And so I'm going to create an object, and I'm just going to grab both the ID of the video and the bookmark. So we can go through the videos on the left-hand side here, counter .id, and I'm going to go bookmarkId using the bookmarks in the same --- yeah, thank you. Don't forget your S. Thank you. Where? Here? So bookmark, we'll split this up over a few lines here, bookmark.id. And we've collected up a pair of the videoId and bookmarkId. And if we run this, we get bookmark is not defined, and that's because it isn't. I have to look it up in bookmarks at counter. Great, so that works. So we're just looping through the array, and we just look up that index on both sides. So on the video side and on the bookmark side we pull out a property. Now this is actually common enough, this pattern, that it makes sense to abstract it into a function. So we can take this process of a single counter and looking up what that item in the position on one array and the item in the same position on another array, and it's basically gluing them together to create one item. And we can abstract that into a zip function, which we're just going to add not to an instance of an array, but to the Array type itself. So you call it like a static function if anybody in the room is familiar with that. So you just go Array.zip. You don't go an instance of an array, 1, 2, 3.zip. And the reason for that is that we want to make it clear that these two arrays are kind of peers, right? It's not like where you go --- they're basically both being processed at the same time. We want to make it visually clear that that's happening. So I'm going to take the exact same code, except instead of this time doing something specific where I'm taking the ID out of the left-hand side and the ID out of the right-hand side, I'm going to do something generic. I'm going to use the combinerFunction passed in to the zip function, and I'm going to pass it the item on the right, or the item on the left, excuse me, so we take left at counter and the item on the right. And now people can pass in their own combiner function, which decides what to do with the item on the left-hand side and on the right-hand side, and we've generalized this operation now. So when people call zip, they just specialize what zip does by passing in that function that's responsible for combining the item on the left with the item on the right. And so if we run that, we should have zip. Great! So now let's solve the same problem we just solved with that loop and that counter using zip. So I'm going to zip the videos, and I'm going to zip, so that's the array on the left-hand side, and I'm going to zip bookmarks, the array on the right-hand side, and I'm going to pass in a collection that takes both a video and a bookmark, and I'm going to return the videoId and the bookmarkId. I think that's a good deal of less code, and I actually find it, once you start to learn to recognize these functions, you'll find it much more readable. The thing about a loop is that look, we're all very comfortable with loops, right? We know how to use those. We've been using loops for years. So at first it can seem very, you know, difficult to learn to use all these different functions, but once you learn these functions, and there aren't many, right, we've covered five or six today, what you're going to find is your code becomes much more self-descriptive. When I look at a loop, I have no idea what that loop is doing. It could be aggregating data, it could be transforming data, could be filtering data. I have no idea. When I look at a map function, I suddenly have a very good idea from a high level about what type of operation is being performed on that collection. And so if you learn to use these functions, which are specialized collection operations, your code suddenly becomes much more self-descriptive, much less opaque. You don't have to pay too close attention and read carefully to understand at a high level what's going on. So it's really important to master these functions and then master being able to combine them together because not only will it allow you to do very powerful things, which we'll get to with the async programming stuff, it will make your code much more self-descriptive and readable. And in the end, you end up writing less code, right? Wonderful thing about this is it takes those common operations of, you know, looping over a collection and adding stuff to an accumulator, and it makes it somebody else's problem, right? That's what you do whenever you create a function. You make that work somebody else's problem. And so now you can just specialize the little pieces of the operation you need to do. There are so many different ways, it turns out, to process a collection, and a lot of the things that we do that we think are whole new problems are actually just specializations of an overall pattern. Map, filter, reduce, merge, and zip, or concatAll, and zip. So let's see if this runs. Nice, right? Now we're writing some short code. Code is getting real nice and brief. Exercise 24 So now actually I think this function is going to --- this test, this problem is going to pull all of the functions we've learned today together, I believe. We're going to return the video ID, title, the middle interesting moment. That's kind of an interesting way. So every video has a list in Netflix, every video has a list of interesting moments where you might want to, we might want to display that screenshot when you're just browsing through the videos. And we might say oh, that's an interesting moment, you know, in Die Hard where here's falling off the building or something like that, right? We pick interesting moments. And so every video has an array of these interesting moments. So if we look at the type here, so here's one single video, we see that it's got id, title, and boxarts. It's also got url and rating. But it also has another array on it, interestingMoments. So we want to pick out the middle interesting moment, the one in the middle of the movie, right? We pick out one at the beginning, one in the middle, and one at the end, and we also want to pick the smallest box art. So how are we going to put all this together? How many people finished this question, by the way? It's alright. Nobody's judging you. It's a hard one. Not quite. Okay, let's take a look and see if we can solve it together. So we're going to start out with concatMap and movieList. I'm going to shorten this to use the new arrow function. So for every movieList, we're going to --- and I'm going to start off by mapping over the movieList here --- we're going to get out a videos, excuse me, we're going to get out a video. And now we've got a video. That's great. So we've got this thing that's got an array of interesting moments and an array of bookmarks. So I guess I could do this. I could start grabbing the bookmarks. Let's buy ourselves a little room here by making this a block function, and then we can put multiple lines in. So what if I collect up the bookmarks or the middleInterestingMoment? Let's start with that one. So as you're trying to choose, you've only got five functions, so there's not that many possibilities. Whenever you're trying to solve a problem or a subproblem like this, you've got to ask yourself what's the right function to use? Anybody want to take a guess? You can even take these big problems and break them up. They might seem big, but you can break them up into small problems. Anybody want to take a guess as to what function I can use to pull the middle interesting moment out of this little array on the video? What function should I use? Filter. Filter, right, because I've got a bunch of stuff, and I'm only interested in one, so I'm going to use the filter. So let's try that. Donezo. So that's one piece of this subproblem solved. What about let's tackle the box arts. So this tiny piece of a subproblem, we already know how to solve this, right? What's the operator we're going to use to get the smallest, operative word being smallest, box art from a collection of box arts? Reduce. Reduce. Because in order to see if something is smaller than something else, we need to look at two values at the same time, and reduce and zip are the only operators that accept a function which has two arguments. So we're going to have to use reduce. (Typing) So we'll have the smallest box art so far and the current box art, and we're going to use that familiar pattern we've been using, so… So if the current one is smaller --- if the smallestSoFar is smaller than the current, we'll return the smallest; otherwise, we will return the current. And that's all there is for reduce. So now we've got a middle interesting moment, and we've got the smallest box art, right? Now both of these --- I want to point out that both of these, what's the type of middleInterestingMoment at this moment? Can anybody tell me what that is? The type of middleInterestingMoment. It's an array. It's an array. An array takes how many items? One probably. What did I hear? I heard one or two. One. One, yeah. And smallestBoxArt is an array, right, because that's what reduce returns, an array. How many items does it contain? One. One. So we actually have two arrays that are both length of one, and what we want to do, and this is the challenging part, we know that we need to have middleInterestingMoment and smallestBoxArt in scope to return our value. And we need to have them in scope, right? The problem is we can't use the indexer 0 to pull them both out, right? It's almost like we need both of them in scope at the same time. So there's two ways we can do this. One is we can go alright, and now we've got them both in scope at the same time. That's one way to do it, so… And now we actually have everything in scope, video, moment, the smallestBoxArt, to create our result. But that's a little bit weird because I mean I could equally switch this around. I could do smallestBoxArt and then middleInterestingMoment. Usually you use map when you need to do some operation before some other operation. Now we need to map over all of the movie lists before we can map over their titles because we need to do that. The one comes before the other in the tree. We need to first, you know, if you were using a loop, you would first need to loop over all the movie lists and then within each movie list loop over all the titles. So there's an order dependency there. We have to do one before the other. But here we actually just have both of these in memory at the same time. We don't need to go inside of the middle interesting moments to get at the smallest box arts like we did with video list and videos. So what we need is a function that will bring both these values in scope at the exact same time that doesn't make one dependent on the other or anything like that, and map, zip, excuse me, is perfect for that. Now we're zipping up a zipper with only one tooth on each end. It's like a zipper of one tooth. And now in here I have all the information I need. So even though the map example would've worked, it's suboptimal. It means arbitrary when you pick one or the other. This is actually more efficient as well because we only do one loop, you know? (Typing) So now I've got a variable bound to the interesting moment, the smallest interesting, or the middleInterestingMoment, a variable bound to the boxart, a variable bound to the video, and I'm ready to create my result, which I believe is --- was it just the interesting moment? Was it the --- what do I have to grab here? Middle interesting moment. Time. I need to get the time and the URL of the box art. And I believe we have everything we need. Wonder if that'll run. Looks like I have a syntax error somewhere. But rather than spend a lot of time figuring out where that is --- oh, actually I want to be careful here because we're not quite done. So notice I've got concatMap, and I've got map, and then within this, A, I've forgotten to return, so within this I've got another array. So I've actually got at least a three-dimensional collection here because inside of this map, inside of reduce here, or excuse me, inside of this map I'm returning another Array, alright? And so we've got a concatMap, a map, and then inside of here returning yet another Array. So we've got a three-dimensional collection. So we need to flattens. So I'm going to change this map into a concatMap, and now we should flatten the whole thing out. But I'm probably still missing that. I'll just show the answer. So in this example, I actually inlined. Rather than us first assigning the middleInterestingMoment to a variable, this is getting the middleInterestingMoment, or excuse me, this is getting the smallestBoxArt, and then grabbing the middleInterestingMoment and assigning them both to variables, I just inlined them directly inside of the call to zip. There's a question about what if we wanted to keep more than two items in scope? What if we want to keep more than two items in scope? What if you did want to keep more than --- Like the asynchronous is three or four items that you need instead of just two. Well, so you can always use map to do that, right? But the point is if you --- let's say I have got two different async APIs, and I can call them at the same time, then zip is more appropriate because I don't have to call one in order to call the other, right? It's when you have to call one in order to get the other that you would want to call map because you map over the first, and somehow using the data that comes out of that, you acquire the ability to call the second. That's where you need map, where you have an order dependency. Just like I have to map over the video lists before I can map over each video inside. That's when map's appropriate. Oh, but I think actually what the question was getting at is what if you need 15 or 12? So the zip we've defined here works on only two just to keep the implementation simple. The zip that you're going to use in RX, and one you can run it right over array as well, just accepts N. So you can specify as many arrays as you want, and then the final function just accepts that many number of arguments to combine. So good question. Creating Trees Debugging Async If you go back up, let's say in our JSON we had one interesting moment that had two type nils, but that wasn't actually unique and we just thought it was. - Yup. - What would happen in our code here? - Good question, good question. Well in this case, nothing, because we're running it through zip. So if one array has two elements and the other array has one element, zip just stops zipping up. Just like when the zipper doesn't have a tooth on one side, right, you're like struggling with that zipper. It's because it stops as soon as there isn't an element on both sides. - Oh, okay. - Yeah, that may or may not be a bug or what you want but that's how it will behave. - This is a question about the testing. When building up these long chains of commands, what is the best strategy to debug them? - To debug them? So that, to me, is a really important question, right. It is not that you cannot debug these programs. It's that you can't debug them in the way that you're used to, right. The way ... When you're used to debugging imperative programs, which do something and then they do something else and then they do something else, sure enough, the debugging tools we have aren't built to accommodate that. Step in, step over, you know, so on and so forth. But let's debug this, let's actually see if we can debug this program. It is unquestioned ... But the first thing I want to level with you, it's harder to debug this. There's no two ways about it. It's easier if you just have 15 lines of code and you just want to step over them one and another, one and another. What we're doing here though, one of the reasons it's so much harder to debug this code, is because we're taking a large amount of code that we would've written in line and we're making it somebody else's problem and putting it into a library. That's a good problem to have, right. Now, all of a sudden the code you would've had to write, is now being written by somebody else. And so often, if you're trying to step through, your code's kind of jumping around, but understand, remember why that's happening. It's because you're taking an enormous amount of code and somebody else is now writing it for you. So yes, it is harder because all of a sudden, somebody else is calling your code. That's what asynchronous programming is all about. You're handing your code to somebody else and trusting them to call it. Well that makes it harder to follow the control flow, but you get amazing things from being able to do asynchronous programs. Right, we get more concurrency, more efficient things, more reactive programs that respond to user interfaces. So let's not forget why we get that. But how do we debug a program like this? Well instead of step over, step over, step over, it's set breakpoint play, set breakpoint play. So you figure out where the next point is in the code or your code is. Okay, right here, let's actually try this. You can see what I mean. I'll do it in Chrome because I'm a little more familiar with the Chrome debugger. I'm sure you can do that with Firefox, I'm just not Firefox proficient so. If you are Firefox proficient don't let that stop you from using Firefox, it's a wonderful browser. I'm going to zoom up here to maybe the last place we had some working code. - Yeah, Firefox is more finicky about when it catches the debugger. Like you have to ... Chrome always catches it, not matter what, but Firefox, like you kind of have to have the debugger open. - Opt into it, I see. - Like that or it won't ... - See if this works. There we go. So we got a spot in the debugger, right. And so you can always put a breakpoint or a debugger, a semicolon statement inside of your own function that you passed to map. And so this gives me the opportunity to examine variables, right. I'm like oh is movie lists what I expect it is? I don't know why it's not working right now, but it usually does. Sorry, in this resolution I'm not sure if the popup's happening somewhere else or? You can always explicitly add a watch option. So I can look at the video that's being passed into map. I don't know if that's big enough. Does that make sense? So if I wanted to step through multiple functions, right, I would go through, add another breakpoint again it's a bad example. And then just play to the next function that I own, right, so if I've got another function that's passed to filter or reduce or zip, I can put a breakpoint in there and then play. Instead of trying to step over and step into there. Does that make sense? So that just, you can still examine every single line of code that you write with the debugger and check the variable values. It's just that you need to use the breakpoint and play strategy, rather than the step over and step in strategy. And so that's how you debug this type of code. Yeah. - Sometimes I've noticed I can't set a breakpoint inside the asynchronous section until I've hid it above. Is that a case or? Do you know what I'm saying? - Debuggers like have bugs. I've run into that problem. I have run into that problem myself, sometimes I can't set a breakpoint and then something weird happens and I can. Sometimes it has to do with optimizations that the actual JavaScript interpreter is doing. So like, for example, I think the reason why it's not displaying, I'm not sure why it's not displaying. I mean to me, this is a bug. It should add it. But if we add it to watch, we should be able to see it. Movie list, no, interesting, not available. And you know what I think is going on here, even though you should be able to see movie list, you can't. And because Chrome is actually doing a lot of optimization. It's saying hey, you're not using movie list in this function and it's actually not making it available. So if you run into that problem, there's a very easy solution. Which is, you can just throw in a console log of movie list inside of the function, to force Chrome to make it available inside that function. So then you can add a watch to it. Yep. - What about unit testing? - What about unit testing? Well the wonderful thing about this, we haven't talked about asynchronous programming at all yet. We've just talked about loop-less programming. This entire day has been about how to program without loops. And so unit testing, honestly, it's no difference. When you make a unit test you write a function and then you want to confirm that that function outputs the right result. If inside of your function you happen to use this style of coding versus loops, that doesn't make it any more or less testable, right. You're still testing that that function returns the right result. So testing is totally orthogonal to this. Now asynchronous testing is a challenge. And we can talk about that later on when we get to the asynch stuff. But most frameworks support asynch testing, you know. Point is you basically has to asynchronously signal when the test is failing or succeeding. So if you use Mocha, I mean almost every JavaScript framework supports some form of asynchronous testing. SQL Comparison So I think we left off yesterday with some pretty intense code. We're getting to some pretty intense expressions here to calculate, you know, data, to eventually basically create the data that we want, the collections that we want, from some data structure in one or more collections that we have. And so I believe we left off on question 24 which was, the video ID titled, middle interesting moment and smallest box art of every video nested in all of these movie lists. So with the instant cue, new releases, each one contains a videos array and each one of the videos contains two collections. And we found out that we can make that happen by using zip, particularly, I want to recall, that zip is a great operator to use if you've got two collections basically at the same time. So you've got access to two collections, you want to have them both in scope. One item from each collection and scope, at the same time. That's when you use zip, right. Whereas map you use if you want to ... Well there's an order dependence so I need to map over all of the video lists before I can retrieve all of the videos inside because they're hierarchical. And so I would use map in terms of order. So map is a way of sort of enforcing the order of things in an interesting way. You're sort of saying hey I want to map over all the video lists and then map over all of the videos inside of it. Whereas, in this case since the video has both interesting moments and box arts, they're sort of parallel, they're sort of siblings in the tree. We have access to both of them at the same time. Zip is a great way of getting both of them in scope simultaneously because zip's function takes two arguments. Or actually, technically, the number of arguments that you passed arrays. So our implementation of zip, as I mentioned yesterday, just takes two arrays, but you would see an implementation of zip that takes n arrays and then a function that takes n arguments. So it's when you want to sort of process two collections, not quite in parallel in a sense of parallelism because it's JavaScript right. But effectively and that's the way to think about it, as if you're doing them both at the same time. - Bump the font. - Bump the font, okay. So how many people in the room, I'm curious, have experience with SQL? Quite of few, I think we've got a bit of an enterprisy crowd here. Which is good, because, has anybody noticed any similarity between SQL and what we've been doing here? - Joins. - Joins, interesting. Let's take a look at SQL and let's, even though it has a very different syntax, let's compare it to what we've been doing thus far. So let's say I had a bunch of titles or excuse me a bunch of lists and a bunch of videos. And they were much like in a SQL database, bound together by identifier, by ID. So let's say we had a list table. This is lists and let's say it has an ID field and a name field and then we have a videos table and it has a ID, it also has a list ID field and it has a name. And so here's an example. So we have a video inside of horror, it's actually a movie I just saw yesterday on Netflix, very good by the way. And this one is in thrillers. So we can see that in relational databases, information is actually ... It's kind of interesting, there's a difference between trees and relational databases. Can anybody tell me what the difference is between hierarchical structures and relational databases are? How do they organize information differently? There's one very key, obvious difference. In a hierarchical system, parents contain references to children, right. If you look at the previous examples, right, we had thrillers and they had a videos collection. And within that we had videos. Here it's the other way around. Videos actually have references to their parent. Notice the list ID there. So hierarchical and relational systems are swapped in that particular way. Now we use SQL to compose together relational systems. So let's say I wanted to get all of the names of all of the videos in all the lists. Or actually, for every video I wanted to get both the list name and the video name. So I wanted to know if it's Die Hard, I want to know what list it's inside of. I'm going to write a little SQL here, so I apologize for those of you who don't know SQL, but this is really instructive for those people out there who do know SQL. And it's going to demonstrate the similarity between SQL, for anybody out there who knows SQL, and what we've been doing today. Because the truth is, all the concepts we've been learning, map, filter reduce merge, zip, all the same as the concepts that are in SQL. So I want to take a little time to make that connection in your mind if you already know SQL. So I'm going to select the videos name and lists name from video. It's been a while since my SQL ... It's been a while huh. From videos and there's a where clause. I basically have to link up these things. Video ID is equal to the list ID. Did I do that right? Bump this font size down just a smidge. So what we're doing here is a join. Right, we're sort of saying hey I've got these two lists and they're connected by ID, right and we're sort of linking them up by ID. Now I'm going to write the same thing in JavaScript and we'll see what it looks like. So for every map, we're going to go through each list. And for every video that happens to be in that list, we are going to map and grab the name of the video and the name of the list. Can't forget that we need parentheses when we use arrow functions in JSON right-hand side. Bump this font size down just a bit. And how many dimensions is this collection? How deep are we? The rabbit hole here. - Two. - Two, right, because within each list, for each list we're returning in the list collection we're returning another array because that's what filter returns. So all we need to do is apply one concatall. And now we are going to have video names and list names. Is it clear why these two things are the same thing? Because SQL is just sugar, it gives you a lot of syntactic sugar for doing precisely this. I think my SQL may be off a little bit. It's been a while since I've done SQL here. Maybe you SQL experts out there can tell me if I'm wrong. I want to alias the item in the table there, the video and list. So I'm just trying to make the connection between map and select, filter and where and these two nested froms can be thought of as a map nested within a map. Does that make sense? For every item in one list, we're going over every item in the other list. So we go over the first list, we go over all the videos and we check to see if that particular video is in that list by checking it against the ID, right. So we're sort of just going all the possibilities and looking at the IDs. This is not actually the most efficient way to do a join in SQL, right, you'd want to actually use a join operator to do it more efficiently than this. But that's effectively what we're doing when we do this example. Exercise 25 So we see that we can use this exact same technique to join together relational data sources. So I'm going to take this, which is precisely the same example here. Notice that we have ... So in this case we want to build a tree from these two relationally connected lists. So we have a list of videos, a list of lists and they're connected by ID. And we want to actually use the operators that we used so far to actually build this structure, this hierarchical structure. So I want to show you that we can go from relational to hierarchical as well as hierarchical to relational. So everybody see this structure? We should be pretty familiar with it by now because this is kind of what we've been using thus far. So we've got videos nested inside of lists. But look at what we're starting with, we're starting with lists in a separate array than videos and the only thing to link them up is inside of videos, we have a list ID. So now we're just sort of, you know, taking our skills out for a walk and seeing if we can go between, kind of, all these very different data structures. So I'm going to try this example right now. We're going to go from lists, I'm going to map over a list. And for every list we are going to map over videos. Now notice that videos and lists are at the same level here, right. I don't need to go through the list in order to get access to the videos. But I'm nesting two maps to go through all the potential possibilities and combinations of lists and videos. That's what happens when you nest two maps. It's interesting that I can also, alternately, go the other way. I could go videos and then map through and look at the lists because I have access to both of these arrays at the same time. Now I just told you that if you have access to both arrays at the same time, zip is often a good option here right. But in this case, I'm not going through two arrays, where information is connected just by its position in the array, I'm actually just trying to get all the possibilities between two different lists. And this will actually turn out to be interesting and useful for asynch programming as well. So if I go through each list and then for each list I go through each video, I can go ahead and ... Well first I'll filter it, so I'm going to filter the videos, such that the list ID in the video is equal to the list ID. Right, so I'm taking a look at this collection right here. Notice that the video has a list ID and it matches up with ID property on the list. And so by doing this, we're going to go through every possibility of list and video and link them together only when the IDs match up. So now all I have to do is map that into the structure that we want, which is the video ID, or was it name? Oh I think it actually wants the whole thing here, okay. Take a moment to just read this. So we actually want to build the whole tree, so we're not just grabbing a few things here. So I'm going to map this into the list object that I want, the hierarchical list object that I want. So it's going to contain the list information, in this case, the name of the list. And the videos inside of that list. So I'm going to swap this around just a little bit and we're just going to take this down here and plug it in here. So I think we'll do the combinations later on but ... So once I format this correctly, we'll see that we built a tree from some lists. That's going to work. - Wouldn't you need, if you're doing a lists map, isn't that going to be lists? - Yes, great, you just caught it. I've got an ... It's looking for a ... it seems to have a list ID in the input there. Videos, ah, now it's complaining that videos has this list ID property. Whereas in here, we actually just have the ID and title. So I forgot to map out just the properties that we want. At this point, there's no need to have the list ID link anymore. Because I've actually nested each video in its list. So I'm going to go through these videos and instead of just filtering them, I'm actually going to map them and just pull out the properties that I want. I think it's just ID and name. So when you're filling out the exercises, wherever possible, try to keep the same property order here because it compares based on property order. Of course in practice in JavaScript, that doesn't matter. Let's see if that works. Got one of those annoying missing after argument list errors. And that looks reasonable to me, so I'm just going to hit the answer here. As you can see the same solution, albeit a little more verbose. So we're going through each list, we're creating the list object. We have the name. And then we're embedding just the videos. Now in order to get the videos for just that list, we'll filter it. Looking up just where the list ID matches up with the list ID that we're currently going through. So we compare there on line 41, the videos list ID with the list that we're inside of ID. And then finally we just map and select just those video properties we want. Which in this case is everything but the list ID. And now we've taken two lists and turned them into a tree. Exercise 26a As you're writing your expressions, it's a really good coping mechanism, right as you're finding that you're building big expressions, to be able to print them out and see where you are. Now remember the key to solving problems using functional programming, using this style, is to continue mapping until you have an identifier bound to everything you need to create your final result. And then you can figure out how deeply nested your collection is and then figure out the right number of flattens to apply and where to apply them, right. And so let's give a shot here on 26. Now it's been a while since I've seen this question so I'm going to be struggling through, just like you guys, I know this is a hard one. So we're going with even deeper levels of nesting, so what I'm going to do by the way is I'm going to just try to paste it into a text editor as these problems get bigger. Right, it'll be a little easier for us to work in a text editor. So I'm going to switch over to my text editor here. So here is the structure that we want to create. Or excuse me, here is the structure we have. So we have lists, we have videos. Now again, we're using a relational structure here so every video has a list ID and then we have box arts and every box art is within a video so it has a video ID. And every bookmark is also in a video, so it has a video ID. So let's take a look at what we have to create. So we're going from a relational list to a hierarchical structure. So the videos array. So this is what we want to produce. So let's take a close look at what's in here. So we've got our top level, we have an array of movie lists. Each movie list contains an array of videos, right, and each video contains ID, title, time and box art. Now I'm guessing that time is going to be pulled off of the bookmark and I think it's the, is it the middle interesting bookmark? Bookmark time in the smallest bookmark URL. This says bookmark time but I'm not sure which one because we've got several. Let's see. Has anybody finished that question, can tell me if it's the smallest bookmark time or the largest bookmark time? Expected it to be written here. Which one was it? - I think there's just one per video ID right. - Oh, I see, it's linked up for video by identifier, alright. Is it linked up by identifier? Or is it actually linked up by position? Let's see. Yeah, it's linked up by identifier so we're going to link up the bookmark to the video by video ID, the box art to the video by video ID and we're going to link up each video to the list by ID. So let's start at the very top because we want to create lists. Lists are what's on the outside, right, so first I'm going to go through the lists. I'm going to create one object for each list. And I know I'm going to be applying some concatalls here, so I'm going to move my map to the next line because that's the rule, as soon as you have multiple verbs, you want all of them on the next line, lined up. So now I'm going to go through each of the videos. So first thing's first though actually, I'm going to create my list object because we know, at the end of this, we want an array of list objects. Right, so for every list, we want to create an object containing properties so I'm going to create that right off the bat. And then that will contain a videos collection. Put that right in there and there may be some other properties we want on there, I believe just the name of the list. So I'm just going to stick that right in there, right now. So now we drill down, because we know that next comes video. So now we start mapping over video. So there's an order here, the same order that we see in the hierarchy, so we start with lists, we move on to videos and then within videos we're going to move on to bookmarks and middle interesting moment, or the time, excuse me. So we go through the ... I'm going to move this down one line here, video. So first thing I'm going to do is not map, but filter, because I'm only interested in the videos where the video's list ID is equal to the current list ID. And I'll move that onto the next line. Notice as soon as I have more than one chained operation, I move it on to the next line and I indent. So for each one of these videos though, I'm going to create a video object and I want to pull out which properties. I want to pull out the ID and title, so those are just the properties I'm pulling right off the video. And now comes the hard part, I want to look up the ... Well I don't think it's any harder than the previous examples, it's the same thing over and over and over again. Right, our structure top to bottom is just the same structure that we want to see in the output. Because we know lists are at the top. So we map through lists first. Videos are next, we map through videos first and so on and so forth. Now the interesting thing though, is recognize that when we go through box arts and we go through bookmarks, right, there isn't a hierarchy here. They're both on the same level, they're both on the video level. So there's no reason to map over box arts and then map over bookmarks any more than there is a reason to do them the opposite way, mapping over a bookmarks and then mapping over box arts. So effectively we have access to both of these lists at the same time, so there's no particular reason to map, because we don't have to go through one before we go through the other. So that may mean that we need to use a zip, I don't know. Let's play around with it and try it out. Or we can just process them both at the same time. So let's see I have title and now we want time and time is actually going to be the one time, the one bookmark time that matches up with this video ID. So I'm going to start doing something that I think is a mistake and it's the same mistake we made all the way back in question 12. I think I've made a mistake here, which is I've tried to create the result too early, we'll see if that's true. So I'm going to go through bookmarks. And once again, I just need to match that bookmark video ID up with the current video ID, alright. And I'm going to ... Before I keep going here, do I have a problem? Can anybody tell me what my problem is? Make sure I could move a few of these spaces. - You have an isolated time on the bookmarks. - Time, that's true, that's absolutely true. And I think I used a map where I should be using a filter here, right. I should be filtering for where the bookmark IDs are the same, not map, that's the wrong operation. So let's, you're absolutely right, let's map that in. So I've got bookmark and I just want to pull out the time off that bookmark. That's one problem. And let's just ... Here I want to show off a way in which, I think it's going to be easier for folks to cope with these big structures, if we can sort of work more iteratively . Because notice it's this big expression and we're kind of writing and writing and we don't really know where we are. So here's a way that we can kind of cope with this. And it's also easier to do this in a text editor because you can see where your brackets are matched up. So let me see, is that bracket matched up? No, I'm missing a map here. So let's take a look at ... Is that a question? - You need to map bookmarks with box arts before you can ... - You're absolutely right, I do, but I'm actually, what I'm going to demonstrate here is how we can sort of work and print out what we have, as we go. The key here is as long as we just do the map step first. As soon as we introduce concatall, it's a little bit harder because we have to make sure that the concatall only works on a two dimensional array. We have to make sure that at each level we're using concatall, we have to return a two-dimensional array. So if we just don't do any concating, if we worry about this in two separate steps, just keep mapping until we have all of the identifiers we need in scope, notice that's what's going on here. That's how I'm bringing up and into scope all the things I need to do my job. If we just keep mapping, we're just going to create trees. But the good thing about trees is we can print them out and then we can figure out how deep we are. So we don't have to be lost in a sea of just doing this all in our head. So I'm going to try do this right here. So what I've done here, these last two parameters to JSON stringify, actually I'm not entirely sure what the null is, I don't know off hand, but the two that describes the indent. So now if we do it like this, JSON stringify will actually print this out for us in a way that we can actually just see what's going on and see the indent. I'm just going to return the result to make the function happy. We know we won't get the right answer, so we'll get an err here, but that's okay. We had that problem with the Firefox debugger earlier, but let's see if we can, see if that still happens. Any Firefox experts here who can tell me how to get that debugger to kick in? Oh here we go, I think. Ah, alright, so we're now at a point where we can actually log. Instead of using the debugger, I'm just going to log what we have. So I've taken the expression so far, I've logged it out. Let's take a look at it. What am I doing wrong? We made this mistake in question 12. I made the same mistake that I think most people would make in question 12. Which I started to create my object too early. I created my result too early and then I found, oh no, I've still got this thing boxed up in an array. I want to get this value out of an array. Can anybody see the value that's boxed up in an array that shouldn't be? - Time. - Time, right, I hastily tried to create this object but I haven't managed to get that value in the time array out of the array. So let's go back to our example. And what if ... I want to map over time. Clearly, because that's the answer usually. In order to bind a variable, there's two ways to bind a variable to a collection. I can go item equals array at zero or I can go array map and this is the way we want to do it, this is going to work for synchronous and asynchronous collections and both accomplish the same thing. But before I do that, I'm just going to finish up, we're going to get mostly there by just finishing up the example for, I believe it's box arts. So I'm going to link up the box art with the video ID, but I think this one's a little more challenging, if I remember correctly. Because you have multiple box arts and we want to pick out, which one, the smallest box art is it? - There's just one. - Oh, there's just one box art? Because I see here multiple. Yeah, so here's two box arts with the same video ID. So it looks like we do need to get the smallest. So I'm going to stick, even though we know that this isn't the right answer yet, we're going to get a little bit closer and what I'll do is actually I'm going to take the time section and I'm going to pull it out and we're just going to assign it to a variable. So we used filter and map because we knew we could just pull out the time and the only condition to pick the right time was to use the video ID. So filter is the logical choice. And then we noticed that we just wanted to pull out the time property and whenever you want to just substitute an item in an array for some translation of that item, map is the right choice. But now we've got a little bit more of a challenge. We want to get the box shot, I believe it is, yeah, box art, excuse me, and there's a little more to it. It's not just about filtering for video ID, although that's definitely something we have to do. But now we have another challenge which is how do I pick the smallest box art? Because this is actually going to give me more than one result. It's going to give me multiple results. So can anybody tell me the right operator to use to use to pick the smallest box art? - Reduce. Reduce. Why do I have to use reduce? - Because you have to compare it. - Right, in order to pick the smallest of something, I need to look at least two things at the same time. So if I use accumulated and current, right, remember how reduce works. We have a value that we've got, that we've got in our hand and we're sort of holding onto and that's the value that we're accumulating over time and then we have the current, which is the value that's the current item in the array. So in order to pick the smallest one, I'm going to return the accumulated box art with times height, we're going to go with area, because I don't need the return statement there. And I'm going to move this to the next line so it stays visible. And we want the smallest one. So if the accumulated width times height is smaller than the current width times current height, we're going to return the accumulated, otherwise we're going to return the current. Okay. So now we have two arrays, each of which are one long. Which isn't super helpful, right. Exercise 26b How do we manage to get a variable bound to both of these at the same time? That's what we need. Yep? - Zip. - Zip, right, because zip will allow us to go through both of these arrays, positionally at the same time. And pass them both to a function, one from the left-hand side and one from the right-hand side. So if I use array.zip and I pass in time, and I'm actually going to ... To be very clear here, I always name array variables plural. Even if I only expect to have one in them, because we don't want any confusion about whether it's an array of one or just one value. So I'm going to call it times and box arts, because who knows, it could have more than one. And then we have time. Now we have a single time and a single box art. And here, notice, I have a variable bound to time. That is the time for the video, the one time. I have a variable bound to the smallest box art for the video. In fact I'm actually going to emphasize that by calling it smallest box art. I have a variable bound to video. I have a variable bound to list. I have everything I need in order to create my final result. So inside of here, I'm going to take what I was doing earlier and paste it in. - They're saying you're missing an opening param on 162. - 162, is that, am I? Ah. That's why that was saying that I ... Perfect, okay. Thanks guys. Time, smallest box art. Now I think actually, I had to grab not just the box art but the URL of the box art, if I remember correctly. Is that correct? - Yeah. - Ah okay so we've got more to it than just that. Now we could throw a map in there or right here we could just pull out the URL. That's probably what I would do, so I'll just do that. - Also on 158 to 159 you have both box arts. - Ah, thank you, yes, that is not good. So now we have an editor so it's much easier to line up our brackets. Going to lower this font size just for a second guys, I'll bring it right back up. So let's see if we've got ... Now, we've done the first step, which is just keep mapping, just keep mapping until ... Or zipping as the case may be, until we've got a variable bound to everything that we need to create our result. Now let's just see where we are. I mean maybe we don't have the right result, maybe we're off by a few brackets or something. So let's copy, paste and then see what it looks like when we run. Uh oh, I'm missing a semicolon. There's always a challenging one. So the first thing we do is make sure all of our brackets are matched up. That looks good to me. Anybody out there see my syntax error? Maybe right here. Hm, looks good. Ah, starting parentheses. - Paren. - Sorry? - Did you just catch that object, return needs paren is what they're saying? - Line number guys? - Line 170. - Line 170. This one? - No that's, a few down. - That was the missing piece, okay. Well I'm pretty sure this is the right answer so. Well I want to, I think I'm missing another parentheses here actually. To end zip. Yes. - Have you tried the browser console for debugging? - Oh yeah, that's actually what I'm doing. So if we switch over, we should see the printed results here. - That's the web console, there's something different called the browser console. - In Firefox? - Yeah, look under the developer, the web tools menu. On the browser page. - Actually, do you mind, I think we've actually got the result here, can we just take a quick look at this? And then I'll ... I think we're actually in a good place now, I'm just not sure if we've got the right one, so let me just clear. And I'll run that one more time. So it's complaining that we don't have the right expected output, but let's take a look at what we actually have. Now can anybody see if I've done something wrong here? Because to me it actually looks reasonably good. - You have to, don't you have to concat one more time? - Yep, it looks like we've got an extra level of array nesting here, because within each video, we have an array of one video. Right, that's probably not what we want. So we see right here under videos, we actually have a two-dimensional array. So this is the key to actually debugging your code, do a map, print it out and see if it actually matches up with what you expect. So we forgot to go through and do our level of nesting. To deduce one level of nesting here, so what I'm going to do right here in this map, this doesn't seem like a two-dimensional array to me. Because what we've got is we've got a list and for each one, we're returning a list object. So this isn't the problem. It's over here at videos that we seem to have an extra level of array. So how do we get it out? How do we get it out? - Concat. - Concat? I don't know. That might be the solution, but I don't think so. I think we have the exact same problem that we had before. I could be wrong here, but maybe not. Maybe we just need concat map. You're right. - It's just an array of arrays. - You're right, videos is an array of array, so right here we know that we can apply a concat map. I can just make that change directly in Firefox. - You have to implement it. - Oh yeah. I loaded this up this morning and I restarted it so we just have to make sure. If you ever run into a problem where you can't find concat map, just go up and rerun your concatall function. If you've closed your browser and then opened it again, concatall and concat map don't exist on the array. And so you've just got to go up and make sure to run your concatall example to add it back into the browser and then run your concat map example. Now I'm hoping that it'll work. Worked. So no news is good news. Actually, yeah, returning result. So as you can see, you can attack this iteratively, right. Just keep printing it out to the console. My Firefox, lack of proficiency notwithstanding, whatever your browser is that you feel comfortable using, write it out to the console, take a look at the expected output and you can just apply the right operator. Whether it's concats, figure out how many levels deep you are and then apply the concats where you need to apply them. So that's the coping mechanism for trying to deal with these large expressions and print them out. Exercise 27 Let's move on to stock ticker, how many folks actually got to this question or got past 26? Okay, we got one, two, three, excellent. So this question is actually an easy question. We're going to take it a little easy now that we've taken it real hard. What we're going to do is we're going to go through this historical list of stock prices. So these are the NASDAQ stock prices from who knows how long. And all we want to do is we want to just filter it for the ones that are only Microsoft and then I believe we want to just print it out. So it looks like we just want to print out all of the NASDAQ prices. So I think it's price record, we just want to look for Microsoft. So the name is MSFT. I'm going to run this. What's going on here? Does this surprise anybody? Why would these results trickle in? Every single result, up until now, has just popped right up onto the screen. Well what's actually going on here is instead of querying a huge historical database of NASDAQ prices, we are querying live data. Now in reality, I'm actually spoofing this data but the point is we're working over an asynchronous data source, so we just as easily could be hitting the NADAQ right now and querying for realtime stock prices. So now, we've actually made the switch from querying arrays, historical data that's just been sitting in a database somewhere or in memory, to querying data as it comes in. -Yay! - Asynchronous programming, I promised you'd get some. So now we're finally realizing, hopefully, that all of the lessons that we've been learning about how to process data up until now, every single function you've learned, if you can master that, you can query any asynchronous data source out there. Whether it's realtime stock information, whether it's an invent, whether it's a mouse move event, you can assemble it all together to build complex, asynchronous programs, completely declaratively. So let's move on and start exploring this new type, which I will call an observable. So you guys saw this in the presentation yesterday. Is that a question? - Is the reason this works, because we're using reactive Xs implementation of for each? - That's exactly right. So an observable has a for each implementation. The only difference between the arrays for each implementation, which blocks, until all the data in the area has be slurped up and processed, is the observables for each implementation finishes immediately. Because it doesn't block. But as the data arrives over time, because let's recall an observable is a collection that arrives over time. As that data arrives over time, this function that handles that data gets processed. Now let's remember for each is different than every other function that I've taught you guys yesterday and today. What's different about it, can anybody tell me? - It doesn't modify the data, it doesn't return an array. - It doesn't return an array. Every single other function I've taught you up until now has returned an array, with the exception of for each. And that's an important distinction. What we're going to be doing as we write code and as you've seen up until now, is we're going to be using map, filter, reduce, concatall and zip to be building the collection we want. And then finally once we've got a collection that's exactly what we want, whether it's an event or an array, and we're ready to process the data inside and do something with it, then and only then do we terminate with a for each. So for each is, so those are really thinking about things very separately. First were building the collection we want, but notice we don't change anything. When we build the collection we want, we don't change anything at all. When you take an array and you map it and you create a new array, you haven't changed the old array. When you filter it, you haven't changed the old array. Everything up to now has been taking data in and creating new data. But now as we begin to use for each, we move into a different phase. We say, you know what, now that I've created exactly the data I want, exactly the collection that I want, I'm going to take the data out and now I'm going to change something. So inside of for each you can expect to see us modifying things. So printing a record is a great example of modifying something. I've written something out to the console, that is actually a change. Somewhere there's a buffer where its value has been changed. So we're actually changing a global object. In this case the console. Whereas in map and filter and reduce and concatall and so on and so forth, I only said that you could change objects that you yourself created. If you hand an object into the function for map, you shouldn't be modifying with that object. So it's a very different way of thinking about. What you're kind of doing is you're carving your program up. You're taking the data which is really just computation, where you're taking data in and you're computing new data and you're separating it from data that changes data. Can anybody tell me why that would be a positive or negative thing? What's the point? What's the point of separating data that doesn't change anything from data that changes something? Well there's a whole class of bugs that just can't exist in code that doesn't change anything. A whole class of bugs. Has anybody here ever run off the end of an array for an example? Or had a bug because you changed a variable before you changed another variable? These are a class of bugs that sometimes are called state bugs. And remember when I told you, the enemy of a complex asynchronous program was? State. It's creating all those variables to track all those asynchronous things going on. That's what we're trying to eliminate. The interesting thing about what we're learning today is that asynchronous programming is another way of thinking about it, it's sometimes called reactive programming. Who's heard that term, reactive programming? Alright, a few people. Now asynchronous programming in JavaScript is necessarily reactive programming. What reactive programming means is, it's like the Hollywood principal, don't call me, I'll call you. So you guys hand me a call back and then I'm the callback callback at a totally unpredictable time. So you subscribe to an array, you hand it a callback and then you don't know when that array is going to call that callback. So it's a bit of an odd position because when you're writing just synchronous code, top to bottom, you control when your code is called. But as soon as you hand it to another function, that function might call it at any arbitrary time. Now if you structure your code, you use a lot of state and you change a lot of variables, that might be changed, excuse me, by a lot of other functions, well all of a sudden that's a real hard model, when you don't know when your code is going to be called. So that's why I'm teaching you functional programming. This idea of functions which take data in and generate new data at the same time as reactive programming. Which is that idea of you giving somebody else a callback and then pushing to you, deciding when your code gets called. Even though these are two separate concepts, the reason why we learn them together is that it's very, very hard to build a program where you're changing a bunch of state and variables and you can't predict when your code is going to get called. Because who knows, your code might get called in any particular order. And if you're changing variables, it's very hard to structure a program that way. So if we take these two notions and put them together, which is to say that look, I'm never going to change any data other than the data that I generally create and own and you combine that with reactive programming, which is like, I don't know when my code's going to get called, by not modifying any shared data, all of a sudden it's a wonderful thing. Because now it doesn't matter when your code gets called. If you're not changing any global objects or shared data, your code could get called in two seconds or it could get called in five seconds or it could get called in ten seconds and it will just work. That's why we take all of our code that changes the world, global objects, objects that we didn't create, like in this case the console, and we move it all into the for each. That's when we decide that we're ready to consume the data, we're ready to take it out and we're going to limit all of our change code, all of our do code, do things, change the world code, inside of the for each function. Whereas everything above, can run in any particular order. Maybe the callback gets called in two seconds, maybe it gets called in five seconds, it doesn't matter. And so that's why it's so important to learn functional programming, this ability of, I'm not going to change things, I'm going to take data and I'm going to create new data because it makes your code safe to run in any particular order. Does that make sense at a high level anyways? If you can't predict when your code's going to get called, well you better only change your own objects, not objects that somebody else can change. Because who knows what the state of the world will be in two minutes or 10 minutes. So that's why we're teaching you these two concepts together. But it's hard to absorb both of these concepts together and that's why I spent the first day on what is, by far, the hardest part, which is learning to change the way you think about code from I'm going to change this variable, I'm going to loop. Even a loop itself involves changing a variable. A for loop involves changing a counter, that's the kind of change that we want to avoid. Because you create a counter variable and every single time around the loop, you change it again. Even something as simple as that, all of a sudden, the order and dependence of your program is sort of dictated by when this counter changes. We're trying to get away from any kind of code changes. So the first day was learning all about functional programming and today we're going to be learning about reactive programming and asynchronous reactive programming, technically. Handling Events with Observables Exercises 28-30 Now that we've seen what this observable is capable of, let's see how it works. So first let's take a look at the way that we conventionally do this. Now, this is just reactive programming, asynchronous reactive programming. Let's take a look at the way that we conventionally do reactive programming. So I've got a button, and I've got a handler for that button, and as soon as the button is clicked once, I just want to unsubscribe. So in other words, I want to do something. I want to alert, print up an alert box as soon as you click a button. And then, as soon as you click that button every other time, I never want to print the alert box again. So let's see how that works. We'll add an event listener. All right, so I'm adding a function. And then I remove that event listener the very first time that handler is called. But I also alert a message that the button was clicked. So I'm going to run that. And I click this button. And now, every other time I click the button, nothing happens. So relatively simple stuff, right? This is the type of code that you write when you write event-driven code where maybe you, this is a good example, something you might do with window.onload, for example. You might subscribe a function to onload and then do something once and then, hopefully, you remember to unhook your event handler, because technically, that would be a memory leak if you didn't do that, right? That's a common mistake in programming. If you leave your event handler on onload, you might accidentally in your closure be holding on to a very big object that you might have only needed temporarily. That's one of the dangers if you're working with closures. If you happen to use an object inside of a closure, and if you don't free that handler, that closure will hold on to an object that might be big and expensive. So it's always good practice to unsubscribe from window.unload. So now, if we convert a click to an observable, we can see that we can do the exact same thing in a slightly different way. So, we don't think about things in terms of addEventListener or removeEventListener anymore. We now know that observables are collections, just like arrays, and we can use all the same operators that you've been using on arrays with observable. And so, one of those operators is forEach. Adding an event listener is no different than traversing a collection. That's really what you're doing at the heart. So, I will first adapt this button click into an observable. And then, I will just forEach over it. This code is doing the exact same thing as the previous code. Now notice that when I forEach over code, I get this subscription object back. And the subscription object is just a way of me saying as the consumer, as the consumer of the data, I don't want anymore data. So I can dispose of that subscription object. So I'm going to forEach over, and the very first time I get a click, this should say click, I'm just going to dispose of the subscription. That means I won't get no more messages. So this should run the exact same way as the last one. So I click it. Stopping traversal, so that means I've called subscription.dispose. And now, nothing. So now, hopefully we've seen that, you know, events in observable, you can accomplish the same thing by calling subscription.dispose. Just a slightly different interface. Instead of calling removeEventListener, you call subscription.dispose. Now, one of the very nice things about an observable that is not true of events is that observables can end. You remember yesterday we talked about those two missing semantics that the gang of four who wrote the design patterns book left out of the observer pattern. Now that we've added them to observable, we can take advantage of this notion of completion. So instead of after getting one message, we as the consumer saying, okay, now I want to unsubscribe from the event, what I want us to do is I want us, rather than unsubscribe from events, or by calling subscription.dispose explicitly, I want us to use operators to create events that end when we want them to end. So wouldn't it be great if we could just say, hey look, just take this event and just fire one. Just essentially, transform the observable so that it only fires once and then immediately completes. Here is the brilliance of an observable, which is that when it completes, when you have an explicit completion message, when that observable tells you it's done, is there any reason at all for it to hold on to your event handler? No. Let's say I'm the observable and she is the consumer. She hands me an event handler and I push her values. One, onNext one, onNext two, onNext three. And then I say, onCompleted. I've just told you that I'm never, ever going to send you another value, right? Is there any reason at all for me to hold on to the observer, to those three event handlers that you gave me? No, I can just free them. That's what's great about an observable. It cleans up after itself. That's why when an observable completes and it says, "I'm done," you don't need to go along and say, oh yeah, removeEventListener. It's already freed your event listener for you. That's the difference in the way that you think about DOM events and observables. That's why we don't worry about unsubscribing from events by calling subscription.dispose. That's just machinery. Under the hood, we're not going to use that API directly. We are going to take the observables and we're going to take operators that I'm going to teach you today that cause them to end when we want them to end, and then, we can just be 100% sure that they're not holding on to our callbacks anymore because they've told us that they're never going to call us again. So, this is how we avoid memory leaks. Memory leaks are a serious problem in large user interfaces where you're subscribing up to events all over the place, and you got to remember to unhook from those events. It leads to what we call state-machine-like code. You need to build the state machine. I need to remember if I'm hooked up to the event, or whether I haven't. Let's go back to that mouseDrag example. I want you guys to imagine for a moment how you would have written the mouseDrag example if you didn't have observable. What would you do? You'd probably listen for a mouseDown, right? And when you got the mouseDown, your event handler would fire, and then you'd hook up to the mouseMove. But now you have to remember that you've hooked up to the mouseMove, so that when the mouseUp comes along, you can unhook. And so you're writing code that's using variables to keep track of what's going on, right? So oh, okay. I'll hook up to the mouseMove and I'll hook up to the mouseUp, and if the mouseUp fires, I need to remember to unhook from the, I need to remember to re-hook up to the mouseDown. You write this code that's all kind of like, ugh, this, do this and do that, right? It's very easy to forget in a particular circumstance to forget to unhook your event handler. But when we write code like this, you'll find we do it declaratively so it's much easier. So, there is an operator that you could write over array, we never bother to write it over array but it's very trivial, called take. Take is something that you can apply to any observable, and simply say, I just want one of these. So what that will do is it will take an observable, which remember, this observable is based on click, and observables based on DOM events will never say onCompleted. Why is that, why is that observables based on, that we've adapted from a DOM event will never give us an onCompleted message? Because it will keep listening for the DOM event forever. Right, because DOM events don't have this notion of completion, right? DOM events just go on. That was the thing that was left out of the pattern 20 years ago. They just thought, ah, who needs to say we're done, right? So if you take a DOM event and you directly adapt it into an observable, the observable will never say onCompleted. But we can make the observable say onCompleted by applying take one to it, right? So now, instead of, back to the example of window onLoad, right? A good programmer remembers to remove their onLoad handler after it fires. We know it's only going to fire once, but we've got to go and remember to unhook that handler. Otherwise, we could leak memory. Because that closure could be holding on to some data. Well, that's kind of ugly code. You've got a removeEventListener inside of your event handler, and it's just ugly. Why don't we just convert window onLoad into an observable, and then do take one. What that's going to do is instead of just calling onNext, the one time onLoad would have called your event handler, now it's going to call onNext, onCompleted. Right? Because it's take one. What take does is it takes an observable and says, look, you just want to take this many number of elements, and then you want to complete. And as soon as that observable calls onCompleted, it's going to free our event handler for us. So we don't need to go back and add a removeEventListener. So if I take this buttonClicks and do take one, notice, I'm not even, I'm just ignoring the subscription object, I'm not even using it directly. I almost never call subscription.dispose when I write RX code, because I got takeUntil, and I've got take, and I've got switchLatest. So, I almost never actually explicitly unsubscribe from an observable. If we run this, we'll find that it does the exact same thing as the two previous examples. So, we got a buttonClick, and I get no more buttonClicks, because it's unhooked the event handler under the hood for be because I've told it to limit itself to only one element. So this is the power of giving observable the same semantics that are on the iterator pattern. Now that our observable can tell us it's done, well that would be the logical time for the observable to just free our subscription. Why hold on to a callback if you're never going to call it again? Exercise 31 So in addition to being able to complete an observable, with take, our very, very powerful function is takeUntil. You can complete an observable when another observable fires. Does that make sense? So I can say, listen to all the mouseMoves until the next mouseUp. So now, that's how I get away from writing all this event handler, addEventListener, removeEventListener code. I don't sort of say, addEventListener to mouseMove, but then addEventListener to mouseUp, and then, when the mouseUp happens, unhook the event listener from mouseUp, and unhook the event listener from mouseMove. That's all very confusing. TakeUntil does all that stuff declaratively for us. So, let's take a look at this. So let's go back to our NASDAQ example. One of the problems with the NASDAQ example that we ran earlier is that it will go on forever, and it will eventually actually use up memory potentially. Unless the browser's being really good about clearing that console buffer, it's just going to keep writing to the console, and well, maybe we don't want that. Maybe we'd like to just sort of see the stream for a while, and then decide, okay, I don't want to see that stream anymore. I just want to stop it, right? You could certainly see that happening if I open a form, for example, in a UI, and I want to see stock streams, but then what happens when I close that form? Obviously I want to stop listening to that stock stream, and maybe I've got a web socket under the hood that I also want to close, right? It's these types of things that I want to, you just have to think about when you're building UIs. You have to clean up after yourself when a user leaves a form or moves to another page and that type of thing, right? Now, I'm talking specifically about single page web applications, which a lot of us are building nowadays, right? In the old days at the web, it was really easy. You didn't have to really worry about cleaning up after yourself, because as soon as somebody went to a different page, boom, all your JavaScript was gone and it's reloaded. Today, we have a much more complicated model, right? If those of us that are building angular apps or react apps, the notion of a page is really actually just very often another div popped up on screen. So you have a single page, and then you just keep loading forms. Well that creates a lot of complexity, because now you really got to make sure to clean up after yourself. Let's say I open up a form, and I start streaming in stock prices, right? Well, as soon as somebody hits X, I better close that web socket so that I'm not continuing to access the NASDAQ long after that person is actually looking at that form. So this is the type of thing, this is why it's become so complicated and asynchronous programming has become such an important thing for developers to worry about. As we build single page web apps where the page lives for a very long time, we want to make sure not to leak memory, because after a while the application can slow down and then eventually even crash the browser. So takeUntil is a really, really key tool to solve this problem, right? Imagine saying, well, listen for formOpen. Right, that's a stream too. When somebody clicks a button to open a form, I'm going to listen for a formOpen. And then, I'm going to begin listening to stock prices until the form is closed. That's a very common form that you're going to see again and again. It's not so different than listen for mouseDown, than I'm going to be listening for mouseMoves until the mouseUp. You're going to see a lot of that. Listen for the start condition, map it into some, listening to something else, right? Map it into some other observable until the stop condition. That make sense? So we're going to start doing that. And that's going to be incredibly common in user interfaces. So here I'm going to use the exact same approach, except instead of listening to these stock prices forever, I'm going to listen until Ah, so we are getting passed in a stopButton. So I want to actually be able to listen for the click for this stopButton. It's right down here. So there's a stopButton along with a runButton. So when I click the stopButton, I want to stop printing out stock prices. So, first thing I have to do is I need to convert the stopButton's event, excuse me, the stopButton into an observable. So that's always the first step. Whenever you run across an asynchronous API that doesn't emit an observable, the first thing you do, convert it to emit an observable. Adapt it to the observable interface. Because now we can use all those cool functions like map and filter and so on and so forth. All right, so now I just have to takeUntil stopButtonClicks. So now if I run this, oh, we see our stocks. And you can think of this as the user going along and maybe closing a form, and they hit stop. That could just as easily be an X button on a form. And now, we're not getting stock tips anymore. And under the hood, remember, when you tell an observable that you're not interested in listening to the result anymore, that gives the observable to do the opportunity to clean up after itself. So in this case, if I were really listening to the NASDAQ under the hood here, I could close the web socket and clean up after myself. And as soon as that observable says onCompleted, I'm not sending you any data anymore. Remember, the consumer here didn't say, oh, I'm not interested in the result anymore. What we did is declaratively forced the observable into simply saying, I'm done when this button clicks. And that way, we can be 100% sure that it's not holding on to our subscription anymore, right? Why would it hold on to a callback if it's never going to call it again? So this is how we make sure that when we build user interfaces with all of these events that we're hooking up to, we never run out of memory, because we're always making sure to clean up after ourselves. Code like this is much easier to write than adding an event listener and removing an event listener, because one thing I want to call your attention to, remember I talked about changing things, right? Adding an event listener and removing an event listener is a classic example of changing an object you don't own. There's a list somewhere that you didn't create, of listeners, and now you're going to push something into it, and then later, you're going to pull it out. That is happening under the hood, right? That's what's going on under the hood. But it's not something your code is doing directly. From the perspective of this, all your code is doing is it's taking a bunch of observables, combining them, and creating a new observable. That's all that's happening on this top level of code here. And I want to call your attention to something that we talked about yesterday but it's been a while since we talked about. If I comment this out, and then I run, we won't see any stock prices printed on screen. An observable doesn't do anything until you begin listening to it. All we've done is create a bunch of objects which hold on to a bunch of objects which hold on to a bunch of objects. Hasn't done a thing. Think of an observable. It's a little bit like an event, because it pushes you information over time, but it's also a little bit like a function in the sense that nothing happens until you call it. Really, observable only has one function, which is forEach. All the other functions are actually implemented in terms of forEach. Map, filter, reduce, and so on and so forth. They all use forEach under the hood. So that's really all an observable is. It's one object with a function just waiting to be called. So that's the way to think about it. It's a little bit hard to get your head around an observable, because it's a little bit like an event and it's a little bit like a function. Got a question. Is that an observable? Yes, so if, sorry if that wasn't, if I didn't really hammer that point home. PriceNASDAQ, I was sort of tricking you guys into thinking it was an array, just like every other example thus far, but I pass in an observable and the point of doing that was to hammer home the equivalence of the API between the array and the observable. Everything you learned yesterday, painstakingly learned, can now be applied to asynchronous data sources, not just arrays in memory. Observables and Events So all the map and filter and zip, and reduce stuff all works over observables. Oh, this one's fun. So we're going to create that mouseDrag event. But first we're going to start with an example that you guys already know, that you've been, very similar to the example you guys have been working with the last few days. Alright, let's take this example and let's slowly morph it into a mouseDrag event. So what did we do? We've been going through this, this stuff ad nauseam all yesterday. We went through movieLists, and then we went through videos, but we filtered the videos collection before we returned it, and then we ended up with a two-dimensional array, and then we used concatMap to flatten that by one level. That's what we've been doing again and again yesterday. So let's see why we've been doing those wax on, wax off exercises. Now, I'm going to slowly change this. a console log in that filter function, that it would do nothing until forEach is called? That is absolutely correct. Not a bit of code in here. Well, excuse me. Not a bit of code inside of the handlers, because handlers, like the filter code, this code is only going to execute when data makes its way through. And we're only going to start getting data, the way to think about this is actually, here's a good way of thinking about it. The event handler has not, remember when we adapted the buttonsClicks to an observable? That did not hook up an event handler. The event handler is not going to be hooked up until you actually call forEach. Because what happens, and what we're going to do later on, it's going to be fun. We're going to actually implement observable. So you can see all the innards inside, and how simple, amazingly simple observable really is. And we're going to step through the code and we're going to watch what happens when you call forEach. We're going to follow it all the way up to subscribing to the event. But first I want to make sure you're proficient in using it, and then later on, if we have time, we'll actually just implement a mini observable, and you know, all those, map and filter and from event, and that type of thing. So, you'll actually be able to follow up the progression. But that's what happens when you call forEach on an observable. Let's say, I take an observable and I've mapped it. Well, when I call forEach on that observable, it calls forEach on its data source, and that'll call forEach on its data source, and on and on and on up until finally we reach that observable that adapted from the buttonClicks, and then it'll call addEventListener. Does that make sense? So, an observable's a little like an event, but it's also a little like a function. It's like an event in that it'll push you data, but it's more like a function in that it won't do a thing until you call it. Does that make sense? Until you call that one forEach function. So, we're going to try and leverage all the metaphors for how you're already used to thinking here. I'm going to combine them, meld them together to get a good idea of how observable works. So now we've got, I was just in the middle of changing this. So filter takes a collection and potentially makes it smaller, right? Well, takeUntil does the exact same thing. It takes a collection, and then makes it smaller. Just slightly, we use it by instead of applying a function to each item, to see whether it passes a test or not, instead it listens to another observable and it completes the source observable when that other observable fires. And so, mouseDrag. All that stuff about processing data. Remember, we talked about why we use map instead of zip. Well I have to process the videoList before I process the videos. Well it's order, right? You can use map to enforce order of events. That's what we're doing here. We're saying, well, I'm going to listen to the mouseDowns first. And then when I get a mouseDown, I'm going to go through the mouseMoves, right? Before we had to do that with an array because, well, you can only get to the videos by going through the video list. But here, you could also take two. We notice that when we did our example where it's relational, where we had IDs, we had two collections at the same time. In order to create the structure we wanted, we just chose in which order to map through. What we said, well, I want to create lists that contain videos, so I'm going to map through the lists array first, and then within there, I'm going to map through the videos array. So that was the relational example where we linked stuff up by IDs. So, we were creating the structure we wanted by imposing an order. Sometimes an order is imposed on us, like with the videoList and it contains the videos, and then we have to map and nest through. But sometimes we just have access to two collections at the same time, and the order is really up to us. If we map through the former before the latter, we get one structure if we're building a tree. If we swap that around, we can get the other structure. That's really more akin to what we deal with when we deal with observables. Usually I have access to the mouseMove. In this case I have access to the mouseMove event, I have access to the mouseUp event, mouseDown event, all at the same time. In order to enforce an order, I just choose in what order I map through them. If I map through mouseMoves and then mouseDowns, well then I wouldn't start listening for a mouseDown until a mouseMove came along. Does that make sense? Or is that worthy of a little more hammering home? Because here, I'm going to write this the long way first, because remember, because that's all that's going on. And always switch a concatMap to a map followed by a concatAll. This function right here is only going to get called when we get a mouseDown, when a mouseDown arrives in that collection over time. And so therefore, we're not even going to begin listening to mouseMoves until a mouseDown comes along, because how would we? This function's not even going to get called, right? It can't get called until a mouseDown comes along. So when a mouseDown comes along, we execute this function and we return that new collection. And concatAll, remember, concatAll is the thing responsible for flattening. So it sees a two-dimensional observable. And every single time an inner observable comes along, it's going to forEach over that, right? And this right here is the inner observable. So first, the concatAll subscribes to the outer observable, which is mouseDowns, and then every inner observable that comes along is just going to be all the mouseMoves until the mouseUp. So then it subscribes to the inner observable. And that's how we enforce order. I want to listen to this event happening before I listen to that event happening. Does that make sense? So if you just got a bunch of events and you want to order them, map. If you want to listen to them both at the same time, what do you think's the operator we would use? If we were listening to two events at exactly, at the same time, and any time you get an item from each side, you want to do something, let's say you've got two network requests and they're both observables of one, and you want to wait for both of them to finish before you do something else, what's the operator I would use? Zip. Zip. Now we know why we learned zip, right? Zip, the truth is when you're doing programming, most of the time in arrays, items don't happen to be organized by index. It's actually very rare. Usually they're organized relationally but they're linked together by some ID, or they're organized hierarchically. And so, I'm the first to tell you that when it comes to arrays, zip isn't that useful. You're probably not going to use it very much. When it comes to asynchronous programming, that's probably the most common usage, when you're dealing with concurrent network requests, right? It's very often that you have, like, maybe even two or three network requests, and they're all going to give you one result. Zip is perfect for that. Because you can zip, you can just wait for them all to be done, and then your function gets called with the results of all of the different observables of one. All at the same time. So you can make a concurrent request to three different services, and then take the results and combine them together into some object that's a composite of the data from all three. So that' where zip's going to apply, right? So we've seen how we can order events using map, filter is self-explanatory, same thing with arrays and observables. Sometimes you're only interested in a certain type of event or an event that matches a certain condition. I'm only interested in listening to this mouseMove when it's within the bounding box of this button, right? That's what filter's for. And concatAll, it's the same thing really. You have nested observables and you want to flatten them. Because in the end, what are we going to do? At the very end, once we've built the observable we want, what's missing from this code? In this code, we're not even going to hook up an event handler, not even an event handler's going to be hooked up, nothing's going to happen. What do I have to do to actually make this code work? To actually listen for mouseDrags. ForEach, yes. All I did here was create an observable. And that's just like a function waiting to be called. Haven't done a thing, just created a function that I can call later if I want to. And how do I call it? I'm cheating a little bit here. Let's not cheat. I think it's client X and client Y. Now I'm actually going to be listening. It's like I've created this event but I'm not listening to it. But now that I'm listening to it, I can do something. Notice that I'm changing an object that I didn't create here. That's what's happening in do, I'm changing the world. Now I've created a stream of interesting events. I'm going to traverse that collection and do something with the information, I'm going to change things. Anatomy of an Observable Question? Will observable fire when forEach is called, no matter if they are in a function that hasn't been called? What's that? Will observable fire when forEach is called, even if they are in a function that hasn't been called yet? You mean if forEach is itself nested inside of a function, and that hasn't been called yet? I believe that's what they're getting at. No. The point is, the forEach function needs to run, and so if you haven't called this function yet, then you're not going to get that forEach function running. Now if I call it, then in this case the forEach function runs. So it's really just about whether the forEach function runs or not. And this is going to, you know, how we can create all these objects and have nothing happen is going to be pretty clear when we take a look at the implementation of observable. But I want you to get at least a 5,000 foot idea of what's really going on here. Let's say I create a function called print. And it's going to console log "hi." I've created this function. If I run this now, am I going to see hi on screen? No, right? Now let's say I create a function to help me glue other functions together. All it's going to do is it's going to take two functions and then run them one after the other. And so I'm now going to call then, I'm going to pass in print, and I'm going to pass in another function which is going to log I want you to take a look at this. Is any code going to run? Am I going to see hi on the screen? Am I going to see hi and after on the screen now? What's really going on here? Oh, you know what? I am actually. Sorry. There's a lot of functions going on here, right? But what I'm actually doing is I'm building a function, and then I'm using another function to glue it together with a function that I want to run afterwards. But I'm still creating another function. And in all this stuff, I haven't actually run any code. What I mean is, I haven't run the print code, nor have I run this code. Because if you look inside of what then's doing, it's just returning another function that calls the first function and the next function. I know this is confusing code, but the point I'm trying to call out is that we're not really running the code inside of the functions. We're just creating bigger and bigger functions by gluing functions together. Does that make sense? Kind of? That's what's really going on with observable. Observable's just an object with a forEach function waiting to be called And when we call map, we glue it together with another function and create another object with a forEach function waiting to be called. That's what we call laziness. Instead of doing something, we create a function which can be called, later on, to do something. It's a very lazy way of coding, right? And it turns out that in programming, laziness can be a good thing, can be a very very good thing, because it buys us flexibility. A, one of the positives of this, is I can actually run this function now as many times as I want, right? Instead of just running the function, I can create a function that I can call as many times as I want. That's what an observable is Remember when I said an observable that represents a network request will issue a network request again every single time you call forEach? That's really at a high level what we've been doing all along. Observable isn't so much of an object, it's really just window dressing around a single function, which is forEach. And we've been combining those forEach functions together every single time we call a map and a filter, and creating another function waiting to be called. So everything on the top, half of this, is just doing, morally, the equivalent of this. It's just creating functions and gluing more functions together to create new functions. That's why nothing happens at the top of this code, because we haven't really run any code inside of any of the functions. We've just been gluing a bigger and bigger function together, waiting to be run. This turns out to be a really powerful programming technique. At a high level, what we're really doing is we are passing functions to functions, which create functions. And that can be confusing, but that, it's probably self-evident that that's actually a very high level and powerful technique to build a program. You're at a very high level of coding now, when you're taking functions, gluing them together with other functions to create more functions. And so that's one of the reasons why we're able to write so little code to do so much. So that's the level of abstraction we're working at here. It's very expert level, right? But it doesn't have to be, we don't have to think of it as all the complexity, because we can just look at these simple five functions, understand what they do to collections, and use them again and again. But there's a lot going on under the hood, yes? observable use under the hood to run reactively? Well, not much, in the sense that it's just thin window dressing over the simple asynchronous APIs that are already in the browser. So if I make an observable based on setTimeOut, well when I call forEach, under the hood it's just going to call setTimeout. And then when I call subscription.dispose, it's going to call clearTimeout using that handle. Let's take a look at what an observable looks like. I wanted to get to this a little later, but just so we don't feel like it's voodoo going on under the hood here, let's take a look at how I would take setTimeout, which is a browser API, which fires after a few seconds, and adapt it into an observable. Because you can take any push API, any async API that calls you instead of you calling it, and you can make it into an observable. So in the browser, you can call setTimeout, and you can pass it a function, and then log something, and then you give it a time. And after that time elapses, it will run your function. How many people have used this API before? Few people, yeah? So it's a useful little API, right? It also gives you a little handle. It gives you a handle, which is a way that you can use, let's say after one second you decide, well the guy closed the form, he doesn't want that to fire anymore, I don't want to show this alert box or do what I was doing, because now he's closed that form, he's hit the X button and he's gone away. Well, in the tradition of having many different async APIs, well if you were working with a DOM event you would call removeEventListener, but setTimeout is different. It's got actually another, clearTimeout, I think it's clearTimeout, and then you have to pass in this handle. Does that make sense? Where, I mean with a DOM event, it's got a different interface, you call it create a handler. And then a DOM, addEventListener. And then if you want to remove, it looks like this. Notice, both these things are kind of doing the same thing. I mean you're just, you're registering a callback to be called at a certain point, right? You don't necessarily know when it's going to get called. And then if you want to stop receiving any information, if you want to cancel that, you call this API. Here, with a DOM event, it's kind of the same thing. You're giving somebody else a callback, somebody's calling you instead of you calling them. And then later on if you decide you want to cancel, it's a totally different API, but they're both doing the exact same thing. It's thins kind of complexity that really doesn't need to exist. We can have just one API for dealing with all push data sources, where somebody's pushing information to you, and that's what an observable is. So let's say I want to take this setTimeout API, which is just an arbitrary design of how a push API should work, and I'm going to convert it into an observable. So it looks the same as all the other observable APIs. So I'm going to use, now remember I told you, an observable is nothing but an object with a forEach method, and I didn't lie. That's all it is. It's an object with a forEach method. And it accepts an observer object. Now I want to take a little moment to remind you what an observer object is. An observer object is an object with three functions. That's all an observer is, right? Remember, when you subscribe to an observable, it's a lot like subscribing to an event, but giving three callbacks instead of one. Does anybody remember why we have those two extra callbacks? What do they represent? What are they for? Yeah, they're for completing observable contract with those two extra semantics that were left out of iterator all those years ago. We want the producer to be able to push a message to the consumer, not just to be able to give them data, which is what onNext is for, but we also want a well-defined way for the producer to say to the consumer, no more data is arriving, or an error occurred. So that's what those two callbacks are for. So when you forEach over an observable, like I did down here, this is actually just a shorthand. This is actually just a shorthand for this. So it's just a shorthand for only providing an onNext handler. That's all you're doing, you're creating, it's a shorthand that creates the observer object for you. Does that make sense? So whenever you implement an observable, you expect that forEach will accept an observer object, because RX, the underlying library, takes any possible overload of forEach and it basically converts all of those into observer objects. You only have to worry about, you don't have to worry about all the different possible overloads because I can also We covered this is the presentation yesterday, but it's been a while so let's cover it again. So you can pass in onNext and then onAir, you can pass in three callbacks. Function with no arguments, which is onCompleted. And that will just get converted into an observer object. Does that make sense? So the first argument is onNext, the second argument is onAir and the last argument is onCompleted. Or you can pass it an object with three named functions, onNext, onAir and onCompleted. It's totally up to you, either one of these will work. However, when we're writing observable ourselves, we can count on the RX library to always convert into this form. So we'll always get an observer object with those three properties. So under the hood RX takes this and converts it into this, before passing the observer to an observable. Does that make sense? It's just got a little sugar where it just converts from one form to another. So when I write an observable, I can be sure that this observer object is in this form. Creating a setTimeout Observable Let's do setTimeout as an observable. I'm just going to create my handle. Actually, I'm just going to take this code right here, paste it in. However, instead of just calling some arbitrary handler, inside of here, we're just going to call observer onNext. Now I'm not really going to get, I'm just basically going to give it undefined. We don't care about the value, right? SetTimeout doesn't really emit a value. So I'm just going to call onNext to send the message, essentially, that the timeout has elapsed to the observable. And now, because setTimeout's never going to fire again after this, right? SetTimeout fires only once after a certain elapsed time, so I also want to say, you know what? There's not going to be any more callbacks. And so I tell the observer onCompleted. So that's what an observable is. Now I'm not quite done, but it's pretty close. So now if I take this timeout, I'm actually going to create a timeout function, which, when given a time, will return an observable, which fires after that time. Now I can just call timeout, five seconds, and now what I've done is I've created an observable that will fire after five seconds once you forEach over it. But notice, just this code, all I've done is create an object with a function. Hasn't done a darn thing, yup? Do we have observer in scope? Like where are we getting the value for our observer variable? We're not, sorry, I'm going to expand this out a little bit. We're not, because observer, forEach is just a function waiting to be called. It's when we call forEach that we will actually pass in that observer object. Does that makes sense? So timeout is just going to create us this observable right here. Notice what we haven't written? We haven't run a thing, right? Just created an object with a function on it. Haven't done anything. But as soon as I do this, now I'm traversing the observable, I've called forEach. And now we can see why nothing happens until we call forEach. It's only when the forEach function gets called that we're actually going to call setTimeout on the browser. Does that makes sense? Now, we're missing a piece. ForEach always returns a subscription object. Remember that? That subscription object? Because we need a way for the consumer of the data to say, I don't want that anymore. I don't want the result of timeout. We need a way for them to unsubscribe, so to speak. And a subscription object, much like an observable, is just an object with a single function on it. Called dispose. Anybody want to tell me how to complete this function? So this is how you unsubscribe from a setTimeout, right? So when somebody calls dispose on the subscription object, I want to make sure to never call, ever call the onNext, onAir, onCompleted functions ever again. So how do I make sure that never happens? OnNext empty? So I could effectively set the onNext method to an empty function, and that would work, but now I'm actually changing someone else's object. Somebody passed this object to me. I don't want to muck with somebody else's object. Now, I told you earlier, there's an API that will cause a timeout to stop basically. You can cancel a timeout using this clearTimeout API right here. So all I have to do, is when somebody calls subscription.dispose, call this. That's how an observable cleans up after itself. It's the observable's responsibility, such that when you say, look, I don't want the data anymore, for it to clean up after itself. So now, by doing clearTimeout(handle), what we've actually done is the browser has now freed this handler, because that's what happens when you call clearTimeout. Because you've basically said to the browser, don't call me. So then the browser, naturally, frees your callback, because why would it hold on to your callback if it's never going to call you? So we've cleaned up after ourselves. This is what's different about creating a promise from creating an observable. There's this extra piece here at the end, which is, you have to describe how, if somebody decides they don't want to get that data anymore, how to clean up after yourself. It's just a slight change to this to make this work for DOM events. So let's actually write fromEvent. So once again we see the same pattern. First, as soon as forEach is called, we want to subscribe up to the event. We want to cause whatever side effect is going to happen that is going to lead to us eventually getting pushed that value. So in this case I'm going to go DOM, addEventListener, eventName, and I'm going to pass this handler directly to it. Just a slightly different shape of an API, but basically doing the same thing. This time, however, setTimeout doesn't really give you an interesting value, it just sort of calls you and there's nothing interesting passed to the function. However, here, events obviously do give you an interesting value. So I want to thread along the event object that you get in addEventListener into the stream. So the observable just becomes a stream of all of the event objects that you would have been, that your handler would have been called with had you used addEventListener. And now, because an event doesn't end after the first item, there could be many many more of such items. I'm not going to call onCompleted because the stream isn't completed. The event could call you 100 more times. There could be 100 more mouseMoves. So I'm not going to say it's completed, because it isn't. So now we're actually adapting a DOM event into an observable, but what's the missing piece? RemoveEventListener. Right, we have to clean up after ourselves. So cleaning up after ourselves with a setTimeout means calling clearTimeout, but cleaning up after ourselves with an event means calling removeEventListener. Oops, now I have to give my handler a name. This is why dealing with events is so annoying as far as I'm concerned. It's a classic example of why it's so annoying. You have to hold on to the handler. This is the sort of stuff that we don't have to do, thanks to takeUntil. Instead of having to hold on to some handler and then later on calling removeEventListener, I'm actually just going to use takeUntil to declaratively describe when I want an observable stream to end. Turns out to be much less code. So now that I've given this thing a name, now I'm going to pass that same handler, and that's how we adapt a DOM event into an observable. So that's what's going on under the hood. Hopefully you guys have got a good enough sense now of how an observable works. It's just an object with a function waiting to be called. And then, at that point, it'll hook up to an event and return a subscription which, when disposed of, will do whatever it needs to to make sure that you're never called again, and clean up after itself. So let's go back to our exercises. throw this into a gist for them? That's a good question actually. And then your setTimeout example, someone was saying, wouldn't setInterval be more appropriate? Well, appropriate, yes, in the sense that setInterval is just like a setTimeout, except it actually calls your callback every 200 milliseconds. But both are equally appropriate. So the only difference between the setTimeout example we saw and the setInterval example is that we would call onCompleted, because a setInterval calls you again and again and again and again, every two seconds. A setTimeout just calls you after two seconds. But the answer is that they're both totally appropriate to convert into an observable. The message here is take every single asynchronous API in your system, and adapt it into an observable, because then you can combine together these asynchronous data sources seamlessly. So yeah, adapt the setTimeout into an observable. Have it call you once and then call you onCompleted. And yes, adapt setInterval into an observable, and have it just keep calling you onNext forever. So if I take that same example, let me see if I can just throw in a gist here. So now if you take that and simply run it, it won't work, and the reason is that the observable is actually a constructor with a prototype. So it has map and filter and all those other chains on it. I'm just showing the actual, what the object really looks like structurally. Now if you wanted to make this really work with RX, it just has a helper function for creating observables called observable.create. And instead of creating an object with a forEach method, you just directly pass in the only method on an observable, which is forEach. So it's kind of a handy little helper function. And this will actually work in RX. And the other thing that the observable.create does, so instead of forcing you to create an object with only one method, that's just a helper function where you can pass in just the definition of forEach, the same thing happens. Instead of forcing you to create a subscription object with only one method that's a function, it allows you to just directly return the dispose function. And then it wraps it inside a subscription object. So if those of you out there are trying it, this will actually work. Yeah, question? that onCompleted would never be called on a DOM event, but when would onAir get called? Is it always up to the specific API? Well, for a DOM event, for your typical DOM event, never, because a mouseMove is never going to tell you air, right? But there are some events that look like DOM events, like on an XML http request, for example, where, although XML http request doesn't have a specific API for giving you an air, well it does, my point here is that different push data sources may have different arbitrary ways of communicating airs, and if you want to adapt that data source, that push data source into an observable, you need to just sort of check for whatever their particular way of communicating an air is, and then if you get that, call on air. So that's your job, to take whatever the specific semantics are of whatever that push data source is, and convert it into those three well-defined semantics on an observable. So if I was in an XML http request, for example, I believe I would check to see if the state, if the return value was a 500. And so, if I got a 500 on the response, I would call onAir and then pass in that response. That might be one potential way of mapping an XML http request into an observable data source. Does that make sense? So the problem we have today is we have all these different APIs, and they all have different, just like you saw with setTimeout and addEventListener, for no particularly good reason, they all have different APIs for doing basically the same thing. And what observable can do is it can give you a single common API for all these data sources. So it's well worth your time to take whatever weird API you happen to be looking at, adapt it into an observable, so that you can begin to use the operators, those powerful operators, map, filter, takeUntil, to glue them all together. So that's the first step. Take whatever API you have, adapt it into an observable. Observables vs. Promises Did I see a question back there? Yeah, just, so to reiterate the differences between a promise and an observable. A promise is something that only happens once, right? And then, the promise also cannot be canceled. And then an observable is basically a stream that can go on continuously. Yes. And so those are the main Yeah, and one more, which is that an observable is lazy, and let's go back to this example, right? If I create an observable, all I've done is I've created an object with a forEach method. That's all I did, I haven't done a thing. So if you call a function which returns you an observable, you have every reason to believe that nothing has actually happened yet. Not a thing. Whereas if you call a function that returns you a promise, something is already happening. A promise is eager and an observable is lazy. And so, by the time you have a promise, it's issued the XML http request already. Or it's hooked up to the event already. It's already done that. And why is that not necessarily a good thing in certain circumstances? Well let's go back to the retry example, right? You can't retry a promise, there's no way to retry it. It's already happening and it will never happen again. And so the only way to retry that promise is to go find the original API that created the promise and then call it again. But the problem is, when you make programs and you distribute types like a promise, you might pass that promise to 15 other functions. And then down here at function 15, it doesn't know about the API that was used to create the promise. All it knows is, I have a promise. Oh, but I want to retry this thing because it aired. Well it's too late. That's the advantage of passing around an observable. It's a richer type, it can do more. I can adapt an observable into a promise, lickety-split, no problem. Just do a takeOne, right? And there's APIs to adapt them into promises, it's very easy. Because an observable can do more than a promise. So it's very easy to turn an observable into a promise. You can't go the other way. And so it behooves you to structure your program around richer types that have the semantics that even trivial UIs need, like cancellation and retrying. And then, if you happen to come across some API somewhere that wants to take a promise instead of an observable, well it's easy to convert from an observable to a promise. You see what I'm saying? It's safer to start with a richer type, and then, if need be, if you're working with a library that happens to use promises, convert it into a promise. Because you can't go the other way, you've lost something. You can't go from a promise to an observable. You can't go from something that's already happening to something that's lazy. It's already happening, it's too late. Does that make sense? So those are the key differences between promises. You can't cancel them, they're one and done, they only give you one value and they finish. So today, if you're using a promise to do a network request, and then tomorrow it becomes a websocket, well guess what? You've got, and in other words it sends you multiple values? You got to go through your program, wherever you had promises, and change them to accept observables. Why don't you just use observable in the first place? An observable that finished once is fine. There's nothing wrong with an observable one. We're going to be using them all over the place. Usually for network requests. Is that a good clarification of promises? So now we know what an observable is. Just an object with a forEach function waiting to be called, and when that forEach function is called, it's asking to do whatever work it has to do to schedule you getting pushed data and return a subscription such that, when that's called, it's going to do the work to make sure to cancel whatever operation under the hood would have triggered you getting data in the future. Yeah, question. Yes. Observable is the type, is one type to rule them all for asynchronous programming. You can use observable for nearly every asynchronous API. There are some exceptions, but for the most part the types of APIs which you use in user interfaces, observables are appropriate for every single one of those asynchronous APIs you're likely to come across. So the first step is, take whatever weird callback API you have, take the time to adapt it into an observable, and then, when your API just looks like observables, all of a sudden you have this power to be able to just squeeze them together with the functions that you've learned yesterday. But as soon as you got an observable and 15 callback APIs, you got to resort to using weird callback and state machines, and then everything gets much more complicated. The first thing we did at Netflix, even before people fully understood observables, is we went through and we started to take the APIs that were callback-based, and we adapted them into observables. And you know what? The thing about an observable is, as we've seen earlier, you can use it just like an event, right? I mean it's not that different if you go forEach and then subscription.dispose instead of addEventListener or removeEventListener. So that's actually my recommendation to people, which is that they start emitting observables and just teach people how to consume them the old way, which is just like an event. Okay, well you can call forEach and you can call subscription.dispose. That's not a lot more complicated. They're not getting the full benefit of observable, because the full benefit of observable is using map, filter, takeUntil to glue it all together declaratively. But they can just start coding the same way they would've coded against events. But over time, as you give your developers time to learn how to code more powerfully and code more compositionally, then when they're ready, there's all these APIs just waiting to be glued together with a few lines of code. And so that's a very good way to ease your system into something that can be used in this compositional reactive style. Yes, question. of any of those exceptions? The best exception I can come up with at the moment is not on the UI and it's a node stream. So who's familiar with node streams? Now the interesting thing about what's different about an observable and a node stream is that, we've talked about push and pull data sources, right? Where I just push information, or the, so in the observation, the producer is in control, and decides when information is pushed to the consumer. Iteration, the consumer's in control, and the consumer requests, it decides when it gets the next value, because it requests it from the producer. A stream is a little like both. So, although the producer is in control and is pushing values to the consumer, the consumer can say something which is not "don't give any more values ever again." The consumer has this notion of saying "pause" right? That's different, that's like a VCR hitting stop versus pause. So subscription.dispose is stop. VCR shuts down, who knows where you're, you're not even sure where your play position is. But pause just says, okay, hold it. I'm cutting up cake, I'm distributing the cake, but just cool it man, stop throwing cakes at me. But then later on you can press play and then the producer just starts again, right? That's not how an observable is. An observable doesn't let you hit pause. It will just keep throwing stuff at you, doesn't care if you're ready. The only thing you can say to it is never call me ever again. A node stream is this notion of push and pull, where you can send values, but then the consumer can say, whoa, cool it for just a little while, and then later on can say, start where you left off. In general that's very useful for asynchronous IO, but frankly, doesn't really come up on the front end. On the front end, almost all of the data sources that you're going to be working with, setTimeout, setInterval, events, XHRs, web sockets, they're all just push only, you can't pause them. So observable, that's why I don't bother to teach streams in a front end class, because there are no node streams on the front end. Not yet, maybe there will be at some point, you know, for things like file system APIs in the browser, but that's not currently the case. So my advice is, don't worry about them so much. Observable is definitely the best bet if you're doing front end APIs. Question in the back. Is there an equivalent to ES7's wait async that promises have for observables? Not yet. So current, what he's referring to is that there's an API in ES7 where you can declaratively write code that looks like it blocks but actually waits. And so, what that's going to look like in the next version of Javascript, very possibly, is, so if you're working with promises, this is going to be like result equals await somePromise. Actually I'm going to make this a little more concrete. So let's say you want to write a function that returns the price of a stock and accepts the name, the English language name of that stock. And then the first thing it does is it gets the symbol for that stock, so Microsoft turns into MSFT, and then you can go to another API which takes the symbol and gets the price. Using this magic await keyword on promises in Javascript 7 allows you to write this code. Even though it's non-blocking and asynchronous, it actually looks like it's synchronous. That's pretty cool, right? What a great way to write async programming. Well the first thing I want to call your attention to is this is only good for a very narrow form of async programming where there is really no concurrency going on. In other words, I want to do something asynchronous, and when that one thing that's asynchronous completes, I want to do something else that's asynchronous. And when that completes, I want to do something else that's asynchronous. And that is great, when it happens. And it'll happen, I think, for often on the server than it will on the client. Because, on the client, very often you're not dealing with this, do this and then do that and then do that. You're dealing with concurrency, meaning you've got multiple things going at the same time all the time when you're on the client. Let's go back to that stock price example, right? I click a button and then I issue a network request, but then if somebody clicks another button, well I'm going to issue another network request. And they're all going to happen concurrently unless I do something about it. That's really more the state of being of what you deal with on UIs, because almost everything is triggered with an event and then may lead to another asynchronous action. And so you're dealing with concurrency, and I can't do that with this. I cannot model that example of, I click a button, I issue a request to a stock, but then if somebody clicks another button, I cancel that request and then start another one. That requires us thinking about two items happening at the same time, and this, as nice and as pretty as it seems, and it's a very nice syntax, does not help you solve the hardest problems that you have on user interfaces, which is concurrency. It is a very nice syntax for that narrow case of you got to do one thing and then another thing and then another thing. Frankly, I'd think it would be much more useful on the server than it will on the client, because in the client, you're doing so much with events, which means concurrency, right? You're listening for an event, and then every time that event happens, you might do something else, but then, while that thing is going on, another event might happen and then you do that as well. So that, whenever you're doing two things at the same time, this syntax isn't generally very useful. Now, the question was, will this syntax work for observable, and that is a very good question, and that's currently actually a proposal that we have for Javascript 7. We're actually working to make sure that it is possible, in those narrow cases, to be able to use the await and async syntax for observables as well. So let's say you had several observables of one. You could use, hopefully, if the proposal passes, you could use the same async await syntax for observables of one. But currently, you'd just do forEach and then would do it like, the next when the client's inside, right? Yes, although technically, you could also use the Javascript 6. So this is actually just syntactic sugar for something like this in Javascript 6, where you could write almost exactly the same code. It looks slightly different, but it's basically the same idea. This is what you can actually do in Javascript 6. Using a helper function, using this little async, so now it's not a keyword, it's a function. Using a little helper function, you could actually take what, I don't have time to explain all these, all of what's going on here. I can just point out to you that it's pretty much the same amount of code. This right here is an example below, which will work in Javascript 6, and you can make work on both observables and promises. Up top there's a syntax that will work in Javascript 7, and it's an open question about whether it'll work with promises or observables. I highly recommend you don't use observables just to keep the syntax above there. Because that is a very narrow case of concurrency when you're dealing on the client, so it's not going to solve most of your problems. And furthermore, if you take on promises, you have just lost a lot of expressiveness, which will be very very useful to you in modern single-page web applications. So I think async await is not as big a deal as people may think it is, because A, there's already a feature in ES6 which allows you to write code almost exactly the same way, but the example in the top I think is a little clearer, because you have the await keyword which I think is a little clearer to people. But there's actually not that much more there in terms of syntactic expressiveness. Any other questions? So let's get back to observable. Now that we've kind of got an idea of how an observable works, it's an object with a single forEach method, right? And we're using functions to just build a bigger and bigger object with a single forEach method. And then finally we run it by calling forEach. Let's take a look at some examples of stuff in action here. Exercise 32 So here we're going to complete our mouseDrag example. And what I'd like us to do is I'd like us to take a little break and have you guys tackle this yourself. This is question 32. There's a mental leap that goes on here where we draw a line from the example that we've been learning, from the way we've been thinking about arrays, and make that link now to make the leap to thinking about events the same way. So I've showed you the answer up on board, but I really want you to work through it and try and solve this particular question. But when we're done, we should be able to drag this sprite across the screen. And if you run this sample and it doesn't work, make sure to refresh your browser, because under the hood it's actually hooking up an event, and if you keep running it'll just keep hooking up that event. So make sure to refresh your browser if you run it and it doesn't work, and try again. It'll save your state and everywhere you are. So somebody out there was having trouble getting the exercise to work, so it might be a little bug in the exercise here. That's okay, we'll switch over and make it work somehow. Did anybody actually get the sprite to move? Not yet? Okay, that's not a good sign. I got a moving sprite. Oh you did? Okay. Which browser are you on? Curious. Firefox. Firefox, alright. Putting that concatAll in might fix it for you. This one? Yeah I think. Let's see. Alright, huh? So let's take a look at this code. What did I do here? Well, for every mouseDown, I mapped over all the sprite container mouseMoves. Now this is the, this is container that you see right here, this gray container, this is a sprite container. So why do I listen to the mouseMoves on the sprite container instead of on the sprite itself? Does anybody know? Well because of this case. If I move real quick, my mouse can actually get ahead of the sprite. And so then I won't, the mouseMove won't register on the sprite. It'll only register on the container above. So that's why we added an event handler to the sprite container, and we listened for mouseUps for the same reason on the sprite container. So we map over every mouseDown. And so now we got a collection of mouse, that we start with a collection of mouseDowns. And then for every one of those mouseDown event objects, those are the same event objects you would have gotten in an addEventListener, we substitute it using map for the stream of all the mouseMoves until the next mouseUp. How many collections level deep is that? How many levels deep? How many levels deep is this? Why is it, so somebody said two, why is it two? This is the most important thing for you guys to be able to, it's hard, it's the most important thing for you guys to be able to do, to be able to visualize how deep the collection you're creating is. And here's something you can always go back to if you have problems. Always go back to first principles. So I got this, right? That's an array, I understand an array. If I map over this and return this, what pops out? Array of arrays. Right, because if I map over this, and I put in this, I get this. The whole point is the map function is a applied to every item in the collection, and then the result is substituted into the new array. So if I apply this function here to one, what comes out? An array of one. If I apply this function here to two, what pops out? An array of two. So if your map function returns another collection, you're going to end up with a two-dimensional collection. Because the income, if the incoming collection is flat, which it is here, we can see the incoming collection's flat, if, for every single one of these items, we substitute another collection, we have a two-dimensional collection. And so the same thing applies here. Sprite container mouseMoves, if we take out the takeUntil it's very obvious, sprite container mouseMoves is an observable. And so if, for every event object in the stream of mouseDowns, we substitute in another observable, we have an observable of observables. And so, first thing I want to do, just like I used filter previously on videos to limit the videos to only the ones I want, now I'm going to use takeUntil to limit the mouseMoves to only the ones that happen before the next mouseUp. But that's still a collection for every mouseDown click. I've still created an observable for every mouseDown click there is. So I have a two-dimensional collection. So I need to follow it up with concatAll to flatten it. So just like I take this here and apply concatAll, and I get one, two, three. Does that make sense? So, can I shorten this? Yes. Because whenever we see a map followed by concatAll, we can always just shorten it, helper function. But now we understand, hopefully, how powerful this type of compositional programming can be. That's a very little amount of code for something pretty impressive. Exercise 33 Let's move on. This example's great, but anybody think of, I mean usually when you see a mouseDrag, it actually, it's kind of got something that's not so great about it. Can anybody tell me what we can improve about this mouseDrag? Smoother transition? Smoother glide? Well the glide's not that bad, but if I click here, and then I move, notice what happens? It kind of jumps to the mouse position. What we want to do with a mouseDrag is usually, wherever we click, the click point on that button, we want it to maintain the same offset from the top left of the sprite as we move around. So let's try that. How would I do that? So here, we start off with the exact same program that we started off with before. But notice that, for every mouseDown, we get an event object that, within it, contains the contact point, the offset on the object where we clicked. So remember, whenever we convert an event to an observable, every single one of the objects in the stream is actually that event object that you would've gotten in your handler from addEventListener. And so, this event object, for a mouseDown, has a lot of information on it. It has, for example, the absolute position, page X, page Y, and I believe it also has offset X and offset Y. So this is what an object looks like. At least some of the properties on an event object we get. So offset X and offset Y describes how far we were from the top left of the sprite when we clicked on it. Does that make sense? So what we'd like to do is we'd like to turn the objects that come out of the mouseDrag stream, instead of the X being the mouseMove position, because that's what it currently is, for every mouseDown we're returning all the mouseMove event objects until the mouseUp, what we'd like is we'd like to translate that mouseMove position so that instead of it being the exact mouse position, we'd like to offset the X and Y positions by the amount of offset that there was when we first clicked it when we got the mouseDown. So for every mouseDown, I'm going to remove some of these comments here so we can see some of the code onscreen. So for every mouseDown, we get the mouseDown object, which is what we can use to get the contact point, how far we were offset from the object that we clicked on by looking up the offset X and offset Y property. And then what we want to do is instead of returning mouseMoves, just whatever those mouseMove objects are, it's almost like we want to substitute each mouseMove object for an object that has the same X and Y positions offset by the amount of distance we were when we clicked the original top left position on that button. So what we really want to use is we want to use the offset X and offset Y on the contact point, and then we want to translate the mouseMove objects before we send them back. So we want to convert all the X and Y positions on the mouseMove events that we get before they get returned. What operator am I going to use for this? So I want to take every mouseMove event and I want to substitute it for some other object that has different properties that are offset by a certain amount. Anybody know? Map. Yeah, we heard map, right? Just like if I want to take one, two, three and map over them and turn them into two, three, four by adding one to every single one, I can map over all the mouseMove event objects and change their X and Y positions. It's the same thing. We're just applying a math operation to each one of those event objects. So I'm actually just going to show answer on this because it'll be faster, but So page X and page Y are what we read in forEach when we position things, because we actually position it absolutely in the page. So we'll position this drag point absolutely on the page. But notice, we're subtracting the X and Y position of the contact point that we got from the mouseDown event. Because we've captured the event object we got from the mouseDown in the closure scope, so we have access to it, and we can use it to offset the page X and page Y position of the mouseDown event that we're returning. And so now we should see Oops. Should see a bug. Sorry? Reset? Might be. Live coding, people. While public speaking. Yeah, live public speaking too. Keep going, it's the next one. Next one? Thank you. Darn it. If you refresh, didn't it lose your concatAlls and such? Yeah, but those are on the array, so all methods are already defined and observable. So I'm not sure what's going on here, but we can probably find out with not too much effort. So I'm going to remove my map statement, which should mean that it works exactly the same way as the last example. A little more down. Thank you. switch to Chrome to get this one to work. Oh okay, this might be a Firefox thing. If so, I apologize. This was originally written in Chrome, so there may just be a, you know, what I expect is, different DOM iterations may have different properties, so I may have used a proprietary property like offset X or something like that. So, if so, I apologize. It's always good to test on multiple browsers. Let's check it out working on Firefox. Or excuse me, on Chrome. There we go. So I'm probably just using a proprietary property like offset X or page X on Chrome, or it doesn't work the same way in both browsers, which I think all of us in here have probably run into at some point. But that's effectively all I'm doing. I'm mapping over each one of these mouseMove objects, and I'm translating the points. So we're just using the same functions we used on array for the same purpose, to translate each one of those objects, okay? Handling HTTP Requests with Observables Exercise 34 We've tackled events. Let's see if we can tackle HTTP requests. This isn't showing up here. I believe in this particular example, we're showing off the jQuery API for getting a JSON object. What we're going to do now is we're actually going to break out of this little, we're looking at this much code, and we're actually going to try and widen our scope and see what it's like to actually use Rx for real in an app that you have. So we're going to try and start from the beginning and go to the end of the problem. I'm going to be using for this exercise, I'm going to be using JS Bin. Who here is familiar with JS Bin? It's a really cool online coding tool that you can try out. If you go to jsbin.com, what is really nice about JS Bin is that you can easily import the Rx library and play around with it. First of all, just go to jsbin.com in your favorite browser. I'm going to go to add library. What JS Bin does is it actually tabs. It's really easy to print out of the web application because you're going to have tabs displaying the console, the JavaScript window, CSS, which I won't activate, and HTML. For this example, we're going to try and create a dropdown list, exactly similar to Wikipedia. We're going to start by including Rx. If you see where I am right here, I'm on the, I go to the add library tab on the top left of JS Bin, and if you scroll all the way to the bottom, you'll see rx.all. And that's the library that has all of Rx's operators in it so if you just select rx.all, It'll add it to the JavaScript page. The other thing we're going to load in, although we're not going to use it for very much, I promise, is jQuery. We're only going to use jQuery for its JSONP ability and then we're going to wrap that as an observable, to demonstrate how we can take existing APIs, async APIs, and wrap them as observables which is a really key thing to starting to use observable in your existing system. Then you'll be able to take those callback APIs and wrap them as observables. So I'm going to import jQuery, which we're going to use to make network request but we're going to wrap the jQuery API so they're in a bits and observable instead of uses a callback. One thing we're also going to do is we are going to add a textbox to the body. I'm going to add it. I'm going to give it an ID so we can pull it out later. Now to demonstrate how JS Bin works, here in my JavaScript window, one thing that we notice we can do is we can actually select from a dropdown list of many different possible languages. In this case if you chose 6to5, that's actually the old name of the Babel transpiler which I may have mentioned to you yesterday. So if you just choose 6to5, no matter what browser you're in, even if it's Chrome, you can start using the arrow function. So I can do something like this. 6to5, that should be updated, technically the new name of Babel is, the new name is Babel. Is that clear to everybody? Now we're up and running. We got to just got a running example, and we're using 6to5 so we can use arrow functions. We're going to try and put together at least a simple dropdown list based on Wikipedia. You know how in Wikipedia your start typing and you see a variety of different results. The very first thing we're going to do is we are going to start listening for the keypress event on the textbox. Because that's where we have to start, right? We've got two different async APIs we're going to be working with. One is, the event on the textbox, and the other is the asynchronous request that we get from jQuery. We're going to start by adapting them both to observables. Let's just see, let's use the old way of adding an event listener to a textbox. I'm going to make sure my HTML tab is open here. It is, although I'm not seeing it. Oh, output tab, there we go. If I select my HTML, 6to5, console, and output tabs, are we going to have some problems with the font size on this? - Yeah. I mean, in JS Bin you can select the font size and all that in the settings. You can bump it but, then you'd have to. - Tell you what, what I'll do is I'll, now that we've added the HTML, I'll hide the HTML pane, because we don't need to see that anymore. We've added our textbox, we've added jQuery. Is this any better? Do we need to go higher? - Ideally, but, that's-- - I can make it work, I can make it work. - That's better. - Okay, not going to leave us a lot of room, but we'll make it work. Now hopefully what should happen is if I get a keypress, it's going to start logging stuff. Doesn't like that's. Oh, what's going on? It's actually showing us our console log which is when I type a key, it's logging it out here to the console. If we take a look at that. We'll see this actually is showing us that big event object, so if I clear this console and I click, ah, we see that big event object. Let's not print out that whole big event object. It's a lot of information. I think the keypress object has a keyCode. I think it's keyCode. So now we see the keyCodes of event object are printed. Instead of building on top of add event listener, remove event listener, which is all DOM element, DOM event-based, we're going to build on top of observables. So we need to adapt the event to the observable interface. Remember how we do that. I'm just going to pull in observable and I'm going to, the observable tape is actually hanging off the Rx namespace which is broaden when you bring in the Rx library right over here. I'm using rx.all and so we're binding a variable to observable so we don't have to type Rx.Observable every single time. I'm going to buy us some room here. The first thing to do is what collections do we have? The very first thing we want to start with is keypresses. And we want that to emit an observable instead of being a DOM event. Let's grab a handle to our textbox, first thing's first, we need this document.getElementById. Okay. Oh, do we already grab that? Okay. Now we're going to create the keypresses observable. So it's the collection of all the keypresses that will ever be by using fromEvent passing in a textbox and the keypress is the name of the event. Let's try this out to see if this worked. I'm going to turn off auto-run so it doesn't keep filling up our console here. How do I subscribe to an observable to get the data out? I'll just forEach over it. Okay? So if everything worked, we should run this, and if I type into the textbox that you see on the right over here we should start to see results in the console. But first I have to just run. So run, and now if I type, um, didn't quite work. I get that right. I was prepared for this eventuality. Hmm. Trying to hide my forEach in here. Let's hope that's not a big problem. So far we're actually getting into the handler there when I get to keypress, but we're not getting the x object. And that might be because I'm preventing the default action here. - Someone says they think that you need to add an ID to the HTML? - Ah, thank you very much guys out there. No, I've got an ID on the HTML. - You can close your input tag. Does that matter? - Sorry? - You need to close that input tag. - No, shouldn't need to close on HTML. So alert test. It's probably a small problem. Oh I know what it is. I need to return. So I want to clarify. I actually I don't need to get to this just yet. I'll delete this. Test, okay. And now, let's write out x. This should be the event object. Great. And now let's console log it instead of alert it, which will be slightly less intrusive. Great. So we're back where we were before when we were using add event listener. Now we can just forEach over it and we're printing each event object out to the console. Searching Wikipedia Let's make it a little bit shorter so we don't see that whole event there and go keyCode. Yeah, once again. Now we've got a collection of all the keypresses from now until eternity. It's never going to end until we dispose of the subscription. What we want to do now in order to be able to go get search results from Wikipedia, we need to be able to make a network request. That's going to be a little bit of a challenge because in JS Bin, from within JS Bin anyway, you can't make an XHR request because it would be cross domain. But who here has heard of JSONP? Anybody heard of JSONP? That's the neat little trick we can use to get around the cross domain restriction and thankfully Wikipedia supports JSONP APIs. Let's try and pull in a jQuery API to do that. So let's see how we would do this with jQuery. I'm going to pull in a Wikipedia search. Let's take a close look at this function. There's not much going on here. You take a term, right, and then we embed that term into a URL and notice I've got that little on the very end, I've got that URL parameter, just callback equals question mark. What jQuery does is it's going to use that, it's going to enter something in there and then create a callback under the hood and that's how the JSONP machinery works. When you download a JSONP request, it's actually a script that invokes a callback and so what's going to happen is jQuery's going to create a local function, give it a name, and then take that same name and give it a callback URL. And then when Wikipedia gives us a request, it's going to invoke the function that we just defined. If you don't understand all the JSONP stuff, don't worry too much about it. It's not really core to understanding this. What is core to understanding this? It's that when we execute this function, all you need to know if we've gone to jQuery and we're printing the result that we get back from jQuery. So I'm going to run searchWikipedia and let's look up Terminator. Now I'm going to clear the console. And run. Look what we got. We got this object back from Wikipedia. I'm going to close the other tab just for a moment so we can take a close look at it. What is it look like? It looks like it got this object. Looks like perhaps, possible an array, with zero that seems like the search we made, and here seems like an array of possible searches. That array of possible searches is what we're going to use for our auto-complete box. Does that make sense? So as we type things in, we're going to keep hitting Wikipedia and bring that array of possible searches back. But, we don't want to work with this weird callback API. We want to convert this API into returning an observable. Because that's going to give us the powerful combination operators to squeeze them all together. Now that we know what this thing looks like, hopefully I can get JS Bin to show me the. JS Bin gets a little bit finicky every now and then. There's a little toolbar up here. Oh man. Bear with me. Alright guys, we're back. Let's get started with Plunker. Plunker to the rescue. I've got jQuery and I've got Rx. Now that I've got my textbox, remember, we're going to create the keypresses collection and that's just a matter of going, well first I'm actually going to bring in the observables so I don't have to keep typing Rx.Observable. This is implicitly done in your exercises by the way, when you've been using observable. The Rx were implicitly running that line of code so you can use observable without going Rx.Observable. Let's go Observable fromEvent textbox, keypress. Now, if I go keypresses.forEach, by the way anybody know how we can use ES6 and Plunker? If you do just shout it out on the chat room and that would be better. Now if we've done this correctly, and we can find the run button. Anybody know what happen to that run button? Oh stop, okay, here we go. Run. Great. So did everybody see that? We've converted our keypresses on the textbox to an observable stream. We've got one of our collections. We've got keypresses. And now we need the other, we need a function that will give us a collection of one that will make a network request for us from Wikipedia to get those search results we saw earlier. I'm going to comment this out. Let's go, so I've got jQuery. I'm going to paste in a piece of code that I had earlier on JSDoc which somewhere, here we go, this piece of code which will use jQuery to go out and get a network request from Wikipedia. I'm going to go, see there seems to be some sort of collaboration going on here. I notice some people selecting stuff. Is that expected? I think everybody out there can actually edit this code. If you'll tread lightly that would be ideal. - You clicked presenter, didn't you? - Yeah I did. - I thought that's supposed to. - Might be like collaboration mode. It's alright. - I think maybe just the highlighter or stuff. - It seems to be actually moving so, there we go, more. I don't know if somebody out there is selecting code. - I think maybe people can highlight but I don't think they can edit. - Okay, interesting. Alright. Okay so you see, we explained JSONP already. I'm going to call searchWikipedia and then I'm passing Terminator. Hey. I'm getting keyboard event, that's fine, but, we haven't gotten our request from the server yet. I hope that pans out. - It's weird. - URL visited. Let's try and run with debugger. Will that work? Come on, Plunker. Ah, I may just have, no. I see. We're good. There we go. Debugger should make that network request and we get not so obvious, but we get an object back from the server. And that object looks like, I believe, if you remember we had an array where the first item was actually the search we made and the second item I believe was an array of the results. There we go. There's our array of search results for Terminator. JSONP as an Observable How can we take this and turn it into an observable so that instead of when we call searchWikipedia, it immediately making a network request? We want it to return an observable to us which will onNext us this array and then complete and then tell us done. So an observable of one in other words. We're going to show you how to create an observable. It's going to keep running if I don't comment it out. Whenever you want to take an existing API and adapt it into an observable, we're going to call this getWikipediaSearchResults, remember that an observable is an object with only one method, the forEach method. And there's a helper function which will return an observable, actually I'm going to write it the long way and then we're going to convert it into the way that we saw before. I'm going to have a forEach method in here. And remember that that forEach method accepts an observer which is three methods, onNext, onError, and onCompleted. That's the observer. That's the thing that we're sending information to. In here, I'm actually going to make this call to get JSON, the jQuery API, and inside of this code, instead of alerting, all I'm going to do is I'm going to send this information. In this case, I'm actually just going to pull out data at one which is the array of search results to the observer. Now at this point, we know we're never going to call the observer ever again, because the way jQuery API works, the JSON API, it just goes to server, gets one thing and then it finishes. We want to tell the consumer, hey look, we're never going to call you again. So we just tell them onCompleted. That's how we create an observable with only one element in it. Now remember that we also have to return something else. Can anybody tell me what it is when I forEach over an observable? What has to be returned? - A dispose? - A subscription object with a dispose method on it, right? Because if I forEach over this observable, a second later maybe the network request is taking a really long time, and I decide I don't want that network request results, I need some way of canceling that. We're going to return an object with a dispose method. And in here, we have to sort of somehow cancel the request if possible. The interesting thing about this is that it's not really possible to cancel this request. And you're going to come across this all the time. Most asynchronous APIs don't have one of these four semantics that observable has. The ability to deliver a value. The ability to deliver an error. The ability to say I'm completed. And the ability for the consumer to say I don't want anymore data. Those are the four semantics of an observable. But most asynchronous APIs only have maybe two or three or one and so jQuery API or the JSONP API is an example of where, well, they don't really have the notion of cancel. If you think about what's happening under the hood, it's actually adding a script tag with a source that points to that URL and then that code is going to get run and it's going to get call our callback whether we want the data or not. And so, sometimes, when the underlying asynchronous API doesn't support the semantics of the observable, you have to fake it. In this case, we just want to make sure that if somebody calls dispose on our subscription, we never call them, and so that means we need to remember that somebody's called dispose. I'm going to create a canceled Boolean. I'm going to set it to false. And then, inside of dispose, I'm going to set canceled to true. Inside of my callback, I'm going to set an if condition that says if not canceled, so while I can't cancel the underlying request, and you'll run across this all the time, where async APIs don't provide you this flexibility. At the very least, you can avoid, you can satisfy the observable contract by making sure that if somebody dispose of the subscription, we don't call them. If we were using an XHR instead of a JSONP we could actually call XHR abort, and we should probably do that instead so that we cancel that network request. Does that make sense? Now we've got an API that'll give us an observable if we ask for it, an observable that triggers a network request, and we've got an observable stream of all the keypresses. Let's see if we can build something like an auto-complete box. But now that I've written this out the long way, I want to show you that the real correct way of doing this is that observable has a helper function called create. Because we know that an observable is only one forEach method, we can just shorten it. So Observable.create just takes the definition of forEach. I don't need to create that object. And the same thing here. Instead of the same ceremony of always creating an object with a dispose method, we'll just return the function that we use to dispose. And that's just will allow you to write less code. Because that part's always going to be the same. Every single time you create an observable. Now we've got hopefully something we can call, getWikipediaSearchResults. And let's pass in Terminator. If I run this, what will we expect to happen right now? - Nothing. - Nothing? - It's just returning an observable and you're not even storing in a variable. - Right. Just creating an observable does nothing. Because observables are lazy. We've got to forEach over this observable in order to call that forEach method right up here and cause it to do something. Let's call forEach. Let's pass in a function. Let's alert them. If we run this, great. Now we've done the same thing except now we have an observable API. Now we get to do the fun stuff which is compose these things together using map, filter, concatAll et cetera. Composing a Stream Let's listen first. Let's go through the process, the problem solving process. The first is, we take, comment that out, so it doesn't keep. We take the streams that we have, like keypresses and in this case the search results that come out of getWikipediaSearchResults and we compose them together to create the stream that we want. What's the stream that we want? We have to start there. We have to decide what is the stream of data that we actually want? This is an auto-complete box. What I want is the stream of all the search results that come back from the server. Sometimes when you're trying to figure out the stream that you want, it helps to work backwards. Which is to say what would I do with the data that comes out of the stream? Would I, what are we going to we do with the data that comes out the search results that come back from the server? Anybody tell me, an auto-complete box? What's the action where we do something or where we change something? - A live search from auto-complete. - Yeah, so when a search result comes back from the server, we want to put them on screen. That's the thing where we're changing something, changing something inside of the browser. I'm going to create a spot inside of here. We're not going to be too fancy with the formatting here but we're just going to create a spot to store our search results. Or now I might even just make it a textarea and we'll make it something a little fancy later. I might just fork right now. That looks scary. Save that. Now we've created a little spot on screen to store our search results. If I head to script js, what I want to do is I want to start with keypresses. Obviously, everything starts with the keypress, right? And I'm going to create my searchResultSets observable. The searchResultSets observable is going to look like this. If you were to visualize it, it would be an observable, remember my special syntax for observable? This is the syntax I'm inventing, which is this collection of items over time. If we were to picture what searchResultSets is going to look like, it's going to look like this. Let's say we type A and then we type B. We're going to get an array, an observable of array of results. The fact that it's an array doesn't matter. It could be a number. But it still a flat observable of results. Just happens to contain arrays. If I type A first and then I type A-B, I'll only be left with abacus, and notice this searchResultSets observable is going to go on forever. It's never going to stop. Because in any moment, somebody could hit a keypress. That's the observable that we want to create. What observables do we have? We have keypresses, and what is it look like? What would a keypress look like? Something like, it's actually an event object, but something like this. If we were to just pull out the key that was pressed, that's what keypresses would look like. How do we go from this and this? It's a long function name. What will getWikipediaSearchResults look like? If we were to visualize it as an observable. Well, it would be an observable that goes on for a long time and then eventually gives us these results, and then completes. Notice that it completes. As you're building observables, it's really helpful, and I find, I recommend this is your coding to comment on top of these observables. It's actually comment and use the stream to convey what the stream's going to look like. So you can actually visualize it using this fake syntax here. We've got keyPresses, we start off with this infinite observable of A, B, C, whatever they happen to click. And now we've also got getWikipediaSearchResults which when given a term will give us an observable that will eventually give us an array of search results and end. How do we put them together to build searchResultSets? Which is an infinite observable of search results. Because this is the problem. How do you take the event that you want and then take the events that you have and squeeze them together to create the one that you want? I want some brainstorming ideas here. How can I ease this? It's always just a little bit at a time. How can I ease this closer to this right here? What would be the first operation I might use? - A throttle? - Sorry? - A throttle. - Well the throttle, yeah we talk about yesterday that we want to turn something like this and that would be turning something like this into this keyPresses throttle will look like this. Anybody tell me why it'd look like that? - Because B overrides A. - Yes, so every single, the way throttle works is we get a key and then after a certain duration, if we get another key, we'd drop the previous key and we keep that key. And then we wait a certain duration, and if nothing happens, if another key doesn't come in, then we return the item. So now we visualize it, we can start to see how these operators transform the stream. So let's say, fine we do a throttle. Now how do we go from this to this? - Throttle use a switch next or whatever that one is. - Doesn't use switchLatest. SwitchLatest would only work on a two dimensional observable, and in this case, under the hood it just creates a time or using something like setTimeout to keep track of it. So how do we get closer here, guys? We've got a stream of keys, and we want to turn, somehow turn it into a stream of search results. It's almost like we want to substitute each key for search results. Map! I hear map. I'm going to start out coding here. Keypresses, and I'm going to code a map. We've got a key in here. Let's throw in a throttle as well because now we want to do that to make the stream a little less chatty. Let's just give a 20 milliseconds. I'm going to keep, as we go, I'm going to paste above, and I think this is a nice thing to do, paste above what it looks like at every stage. As you're going through your code, this is a nice way to comment your code. If you want folks to understand what's going on, this is actually a nice technique to use. As you write code to keep showing what an example stream would look like after the transformation. For every key, what do we want to substitute in this place? What do we want to replace the B key with? I think ideally, the search results for B, right? - Yeah, an array. - An array? Yeah, you're right, absolutely an array. Good, the array of search results for that. How are we going to get there though? Well we know that getWikipediaSearchResults will give us an observable of array. That's a little bit closer than the keypress. - We have to call the forEach of the WikipediaSearchResults, right? - Good thinking, good thinking. But here is the thing. We are only going to call forEach once in this program, because remember, we're going to separate our code into two parts, one we're build the stream that we want, and you don't use forEach at all to do that. Once we've built the stream that we want, then we're going to call forEach and we're going to do something with it. Now, the only choices on the table are map, filter, concatAll, switchLatest, mergeAll, and reduce. -I was going to say maybe concatMap, because we don't necessarily need it to be that layer deeper. - Interesting. So you're saying if we map, why don't we start with map and come back to the concatAll? Because remember the technique, guys, keep mapping until you've got everything in scope that you need to build your search result and then at the end figure out how deep you are and flatten it out again. How do we get closer? We know we've got this function, which presumably, wouldn't it be better if we substituted this for an observable of the search results, for B? - Uh-uh. - Right? That would be closer. Let's just, using map, because map's all about substitution. We can replace B with the getWikipediaSearch, did I have that name right? Yeah, okay, getWikipediaSearchResults. For, now am I going to use the key? Do I want to, look, search for B? - No, you're going to use the value of the textbox. - Absolutely. I'm going to pull up the value of the textbox. Now what does our stream look like? Let's say I typed A first, and then I type B. What if, while the request is going out for A, somebody types B and now we're searching for A-B? We actually start another network request while we're waiting for the previous one. Concurrency. Right, and it's going, it's going, and now we're only left with abacus. How many dimensions deep are we? How many dimensions of observable do we have? - Two. - A two dimensional observable, right? Displaying Autocomplete Data Now here we come to the central question that we have to deal with when we move from arrays to observable. Almost everything else is the same. You still keep mapping. You figure out how deep you are and you flatten. But here's where the extra level of complexity comes in when you have the dimension of time. Instead of figuring out how many flattens I need, you need to ask yourself one extra question, which is how many funds do I need and which flatten should I use? Remember, there's only one way to flatten an array of arrays. What is it? - ConcatAll? - ConcatAll. There's no time, so all the data is there at the same time. The only logical strategy is to go top to bottom left to right and just go the first arrays, slurp up all the data, go to the second array, slurp up all the data. But now we actually have a collection of collections over time. So there's more than one way to skin a cat here. What are the three strategies? - MergeAll, concatAll, switchLatest. - SwitchLatest. What will happen if I use mergeAll here? What would I get? A merged all observable. - You get all of the answer, all of the search results, even the norms that are, should be defunct by later keypresses. - Right, I'll write this out just so we're all clear on it. I would get, in exactly the order in which they arrive, like lanes merging on a highway. Does that make sense? That's what I would have get if we just merge all, it's just like lanes merging on a highway. Let's see what we would get with concat. Anybody tell me? - It would be the same, but with the advantage that the former could not possibly override the latter even in a case of the latter happening faster. - It'll definitely be in the same order, but something will change. Something will change, and remember what concatAll does. It gets, it has an observable of observables. And concatAll forEaches over the outer observable and then it starts receiving inner observables. That's what these are, inner observables. Once it receives an inner observable, it forEaches over that but it will, if it receives another inner observable while it's listening to the first, to an existing inner observable, all it does is it puts it away. Caches it. Remember when she was piling up the cakes? It caches it. It doesn't forEach over it. Which means this time right here doesn't start going yet. Until this observable is complete, it's not even going to call forEach on this observable which means this time isn't going to start moving. What do I mean when I say this time isn't going to start moving? Why wouldn't the network request start as soon as the inner observable arrived? What happens if you create an observable and you don't call forEach? - Nothing. ConcatAll is going to get to the end of this observable and then it's already got this observable. It's just holding it in memory somewhere, piling it up and then it's going to call forEach on it. Which means this time will only start counting once this finishes. Instead of like mergeAll where these things are so close together, you'll actually take all this time and you will concatenated on the end of this and we will get the same results over a much longer piece of time. Is it starting to make sense now that we're visualizing it? We're making it more concrete? That's probably not what we want. Probably don't want like, imagine queuing up every single time they typed a key, queuing up the network request and they've got to wait for all the search results to come back before they even make next network request. Probably not the flatten we want, right? Let's take a look at the last flattening strategy. Help me out here, what is it? - SwitchLatest. - SwitchLatest. What's going to happen with switchLatest? What are we going to get? What do you guys think? - You're only going to get the last request. - When is it going to arrive? When is the data going to arrive? Dot, dot, dot, dot, dot. When's it going to arrive? - As soon as it's sent. - Yep. Why is that? - When the second request was sent, we switched to favoring that request, and it threw away the first one. - Right, so as soon as, the switchLatest forEached over this inner observable but while it was waiting for the onNext and the onCompleted, another one came along and then immediately subscription disposed and it stopped listening, excuse me, dispose, and it stopped listening for these results. It never got this result. Instead, it started listening to this one. SwitchLatest only listens to one inner observable at a time. With switchLatest we get what we want. Now why is it so important to use switchLatest here because if you at it, I mean, merge gives us actually more results, doesn't it? Isn't merge better? Merge actually gives us both results. Maybe it's better to show something on screen earlier even if they've already typed the key. In the case of an auto-complete box, I generally don't think it is better. Why? Because let's say you show somebody result. Anybody ever have this happen to them with an auto-complete box where you show results and the user move their mouse to click something and the results disappear? And some more results come in. Probably not good usability. Very frustrating right? That's why we probably want switchLatest, but there's an even better reason why we want switchLatest. And that's because this can easily happen. Right? Just because I issue a network request first, and then another network request, doesn't mean the results from the first one are going to come back before the results of the second one. - Especially if you're doing search like this where there's going to be a lot of words that have A in them and much fewer that have A-B in them. - Well, maybe the search, maybe it'll take a longer time to execute the query on the server for example, right? So A-B will be a much shorter result to execute the query and the result comes back earlier. That's possible. But now let's see what happens if we use merge. Nothing good. Nothing good happens if we use merge now. That we definitely don't want. And we certainly don't want the concatAll case where the data is, it'll actually just elongate time even further. We don't want that. Hopefully now it's clear why we want switchLatest. And why we almost always want switchLatest in user interfaces. We just want to cancel the old thing and focus on the new thing they ask us to do. Makes sense? So now we know that if I apply switchLatest, I'm going to get this. It's always nice to help out your fellow programmers by adding comments like this. Which help them understand what the stream looks like after these intermediary steps. Not sure if that bracket's needed or not. Okay. Shall we see if that works? I'm kind of excited. Let's try it out. Let's try searchResultSets. So now that we've created the stream of data that we want, notice that this is actually what we originally wanted. We wanted something like this. Now we've actually built that stream. Now it's time to call forEach. Now it's time to consume that data and do something with it. And each one of these will be an array. That's why I call it a resultSet, oops. Then for now I'm just going to, well alert will be annoying, so I'm going to try and put it into an inner text of the search results which I believe we created, oh so it's textarea.value. Okay, yep, question? - Do you have to change textbox to result? Do we define textbox? - I think we defined textbox up here, and then we got the keypresses collection by converting the event, keypress event on the textbox to that keypresses stream of observable. Let's see if this works. Ladies and gents. I don't see my HTML bar. There we go. Rarely do these things work on the first time, so this will be an opportunity for us to do some debugging. Oh, we've still got our alert there. Oh, hey! We're not 100% sure everything work correctly yet so it'll take a few more keypresses. But let's run it again. Now that we got our environment situation all set up, I have a feeling things are going to be much smoother. I'm going to run this. T-E, rminator. (terminator) Alright! Not too shabby for an auto-complete box, right? Not a lot of code. distinctUntilChanged() Now auto-complete box will probably be a little bit better. It's probably not ready to ship just yet. I'm probably not going to turn around and throw this in production. How can I make this better, this auto-complete box? - You could make the results not appear in jTime. - Haha, that's true. I like that one. But this course isn't about HTML formatting. So why don't we come back to that. That's a totally fair request, but I don't want to spend a lot of time with CSS. You can learn that in another class. I'm sure Frontend Masters has got some great classes for it. - People don't often type things incorrectly the first time. They backspace and they go forward and back. - Yeah, yeah? That's interesting. They backspace. What's going to happen if somebody backspaces? - I think it's going to work. It'll probably work, but will it be optimal? If I literally hit the backspace key, what will happen? - It's a keypress. - It's a keypress. So it'll trigger another request. Maybe that's what we want. Because if we went from A-B to A, we should trigger a request for it. But what happens if I go left to right? It'll work. Kind of. It'll also hit my server for a network request for the identical text. Because keypresses are keypresses. How can we make this better? - Caching? - Caching is absolutely true. But the question is where would cache wear? You could cache at the HTTP layer, you could cache in memory. Presumably if the server's caching its search results of the HTTP layer, you might, caching might not be as important. You could probably do a little bit better by caching at memory. But that's not what I'm after. I'm after just making sure we don't issue that request at all. Because I don't know if under the hood it's cached. - Limiting to alphanumeric characters? - Yeah. We could filter for the keyCode. We could absolutely filter for the keyCode. The fact is though there's a lot of those keyCodes. That's a totally reasonable approach. There's actually a really helpful Rx operator. I want to take this opportunity to teach you right now. The way it works is if you think about what we're really doing is, when we take keypresses, I actually want to break this up a little bit, I want to show you guys that maybe this warrants breaking up, as this is as nice and short as this query is, why don't we instead of going directly from the key, directly to the search results? Why don't we just break it into a couple steps? And the reason for that, you'll see in a moment. I'm going to go from the key to the search. Now instead of A-B, we have-- - Now you have stream of input values that you can do the collapsy thing to compare the values to see if it change. - You're right. Now we've turned it from keypresses into searches. With searches, we should be able to do some sort of thing to make sure that we don't get two searches in the same screen. It turns out there's a helpful operator called distinctUntilChanged that will compare successive values in a stream and will de-dupe them for us. Let's try that out. Now that we've got searches, I'm going to make this code work again, because now we need to map search into this and so instead of returning input.value, I'm going to return, or textbox.value, I'm going to just return search. Notice how we went from keypresses to searches, and then we map from searches into searchResultSets, and then we flatten and we're good. Now that I've got the stream of searches, I'm going to run distinctUntilChanged. That turns something like this, if we hit the left arrow, we'd end up with a stream like this, right? Because it's still the same search. It's A-B and then A-B. - And then you'd have the right arrow. - Yes, then you hit the right arrow, exactly, into. If ever you are in doubt of what your stream looks like, try and write it out. Try and write out and visualize each step. It's really important, because let's face it. It's hard to picture all this stuff in your head. Console logging it works great for arrays. But you can't console log an observable data structure. If you have an array in memory, you can console log it and you can see it all in JSON. But you can't console log an observable. You can put break points in your handlers but this isn't where you're going to have to actually sort of live in your head a little bit. It helps to get that stuff out on the page and sort of see what it looks like. That is going to take this stream and turn it into this stream. Because we got two A-Bs in a row in the stream and then we de-dupe them to just A-B. Does that make sense? Now let's try, now I don't think we'll see a result, but we'll know, we'll know that it's worked. We'll see less network request. We could probably see that if we went down the Chrome. Uh-oh. - Did you save? - I might not saved. But I wouldn't have expected I needed to save that actually. - Usually it says locked or something when you click save. - Oh okay. - Or, freeze, as it turns to freeze. - Run would presumably work. I might have a typo or something. Let's comment out distinctUntilChanged to see if that's the problem. Oh, input.value. It's textbox.value. I'll save, and run. And now we have our auto-complete, hopefully. Oops. Has somebody actually been changing the textbox? I think somebody's changing it out from under me. Because before that textbox appeared to the right. Maybe not, maybe I've just made one that's smaller. - You probably did it smaller. - Yeah, that's what it is. I'll leave it like that. Looks better. Textbox.value's not working. Let's just make sure I don't have a, oh, I'm seeing a little error here. Is Plunker helping me out? - Um, map search should be map function search. - Thank you very much. This is why I love the arrow function. I never make that mistake with arrow functions. Thank you Plunker. - It's actually turned out to be a pretty immense tool, huh? - Yeah, it's working out. Especially with the streaming. Hopefully worth the wait. There we go. And if I make a left here now, how can I proof that I'm not making the outgoing network request? Let's just keep me honest here. Always going to keep me honest. Let me bring up the network tab. Plunker might just make an arbitrary network request, I'm not sure, but if we hit-- - You can shrink the size of just that debug tool without a click over here, focus on the debug thing and then I think somewhere down. - We just want to make sure it's readable to the folks at home. - Yeah, that. - I'm going to clear out the network request. We're going to hit B. We should see a network request. Bam! But now if I go left, Hey, no network request, right? Cool! Catching Errors Can we make it better? Is it ready yet to ship our auto-complete box? Example? What do you think? How can I improve on this? - They need a message if there's an error, if there's anything returned. - That's true. And what I would do, what I would do if we weren't happening to use a JSONP API, what I would also be doing and what I should do is adding an error handlers. Let's pretend that we're not using a JSONP because we're never going to get, at least, well technically you can get an error, but when you're using JSONP under the hood, because it's just attaching a script tag, there's no way to get a message that that error happened. Let's imagine though, in practice, you're probably not going to be using JSONP in production. You're going to be using XHRs. Let's imagine that the error that this getWikipediaSearchResults could in fact onError. The observable could onError us. What's going to happen is even though the error's going to occur here, it's the observable's job to forward it on up to the first, to the forEach call, and so we can handle it right here. This is like our catch statement. We don't have to put in any error handling code in here whatsoever. Because the observable will do what the JavaScript runtime does for us with synchronous exceptions. The JavaScript runtime, we don't have, every single time we call a function, we don't have to put a tryCatch around it. We can out tryCatch around a big block of synchronous code, and if an error occurs anywhere in there, it's going to bubble up and then we can catch it using tryCatch. The same thing is done here, but this time by the observable and not the JavaScript runtime. In other words, if the map function, so the map function creates a new observable using another observable as its source. When you forEach over the map observable, it forEaches over its source. And if the source observable onError, and sends and onError message back, it just forwards that onError along. It's only when somebody provides an onError handler right here, like here at the end with the forEach, that that error can be handled once and for all. We can just sort of catch it. An error that occurs anywhere in here, we can catch here. This is a hundred times better than nested callbacks because when you have nested callbacks like you get in node.js for example, if an error occurs you have to be sort of at every point handling that error. If I have some callback like this, callback API, some arguments, and then I have the path to the callback that gets, and this is what it looks like in node.js, and I'll call some other callback because I want to do one thing after another thing. This is how it works in node.js. At every stage here, I have to be prepared for the eventuality of getting an error and then I have to either forward it up or clean up any subscriptions that I've added here. What if I subscribed to an event here? If an error occurs down here I have to remember to unsubscribe from that event. It gets very very complicated very very fast. Whereas with an observable, as soon as it's onErrors, just like with an onCompletes, it will clean up after itself. That's an important key thing that I may not have mentioned before. When an observable stream completes, when it tells onCompleted, that I'm never going to send you anymore data again, it frees your subscription. The same thing applies if it tells you an error occurred. It will free your subscription automatically so it will clean up after itself. Much like when an error occurs in the middle of a JavaScript code and there's a tryCatch, it unwinds the stack and de-allocates all the variables that you created in your function. So all those things were de-allocated and then they can garbage collected if there's no other references to it. The point is, it's just like when you're dealing with synchronous code. You can count on the fact that when an error is forwarded up the chain, all of the individual event subscriptions that were added when you forEached over observables had been unsubscribed. Does that make sense? This dramatically decreases the complexity of working with async code. If you, it's the choice really between learning. This definitely complicated loop-less style of programming. It's definitely hard to learn, than, when you learn it, or dealing with a thousand different callback APIs and subscribing to subscriptions, unhooking the subscriptions, error handling. That's the problem you have to solve every single time you start a new app. Every single time you build an asynchronous app. You've got to deal with handling all these callbacks and making sure you don't have memory leaks. Better to learn five functions and get really good at loop-less programming so that you don't have to worry about that ever again. Because once you've got that skill, it's with you forever. That makes sense? The complexity we're trying to get rid of here. Let's go back. Here, what would I put in here? What would be the thing? The only reason that an error would occur here is that the network error. Sorry, got a question in the back. - Yeah, they're just asking about can you use retry on the JSON calls? - Good, that's just what I was getting to. I love this audience, it's great. Absolutely because when we hit Wikipedia, it could go down. Wikipedia is not up 100% of the time. Netflix has awesome uptime. We've got great uptime. I think it's, I don't want to score a number, because it'll be wrong, but it's up there. It's definitely about 97%. But that still means, when you're looking at millions and millions of request everyday, a large number of request fail. Rather than just throw up our hands if one little server happens to be misbehaving, we should probably retry this request. Now again, in this particular instance, just remember that the JSONP is never going to error but in reality when you're working in network, you'll be working against XHRs and it may well error. Although it's not really specifically practical for this call, and if you were doing this in production you'd want to do this. You'd want to retry. Because if it errors, you'll just automatically schedule another retry. And if it errors, only if it errors three times, will that error actually make it through down here to our error handler. By the time it's failed three times, yeah it's a pretty good bet that the server's down right now. Instead of just continuing to hammer your server, which is probably not a good idea, because if the server's having problem, the last thing it needs is a million more retries, you want to just say. You know. Maybe a little bit nicer language. But you get the picture, right? That's for cases where you're really, it's just like you catch an error, you've done your best. You can't do any better. I like that answer out there. Is there another situation where I wouldn't want to send a network request? Let's try this. First of all, I just want to make sure this still works. I'll fork this right now. Let's run. - Someone's asking what's the retry rate? The default. - Rate? So there's no rate per se. What happens is, retry, from now I'm going to try and anthropomorphize these operations. I'm an observable. Let's imagine I'm an observable. She is a mapped version of me. Another words, because when you call map, you create another observable that's a mapped version of the previous observable. Let's say you want to take everything that I onNext and then she's going to add one to it and onNext it to him. He's going to consume the data from her. What he says to her is he says, forEach. Then she says to me, forEach. Then I say onNext one, and then you say to him, onNext two. That's good, let's keep it going. OnNext two. - OnNext two. - OnNext three, right? You're adding one to each one. OnNext three. - OnNext four. - Yes. So now let's play it differently. Instead of the map, she's going to be the retry function. She's going to be retry function, and you're going to say to her, play along with me here. What are you going to, how are you going to get the data out of her? She's an observable. - forEach. - ForEach. You say to her forEach and I am the network request. She's the retry observable that's created when you call retry on an observable. You're going to say to me what? - ForEach. - ForEach. And I'm going to take my sweet time going off to the network and then eventually I'm going to say, oh no, error. Then what're you going to say to him? - Error. - No. You're the retry operation, so you've got a little counter. So what you're going to say is how many increment that counter from zero to one. Then what are you going to say to me? - Retry. ForEach. - ForEach, right? I take my time, making network request. OnError. - ForEach. OnNext some data. What do you do? - Uh, some data plus one. - Well, you're not an app observable. You're retry. So you're not going to keep transforming the data, you're just sending it. Do we see how this works? It's just each step is just like another person. You just add another person in the chain. When somebody says forEach, the forEach is cascade through all the way to the data source, so that's retry's job, is to just insert itself into the middle of our operation, and just if it gets a few errors, it keeps incrementing a counter, until that counter goes up above a certain level, and then instead of telling me forEach again, she's just going to tell him error. Does that make sense? Handling Empty Searches We're going to do a little more of these exercises as time goes on to understand observable. We might even use up the whole table, I'm not sure. Be ready. There's one more problem with this and then I think we can ship it. What happens if I type, for example, A-B, okay, good, we still got it working, great. And then I hit backspace. We do want to issue a network request here. Hopefully it's going out there. I'm not sure what's happening there. Hm, might have been a failed network request. Hmm. Question? - What is the event of using the distinctUntilChanged while we're using onChange observable? - OnChange observable? I don't think I'm familiar. - DistinctUntilChanged with an on-- - Oh, I think what's being said is, why not just subscribe to onChange on the textbox, is that correct? I believe onChange on the textbox only fires when you tab away from the textbox. I could be wrong about that. If you're listening for the onChange event on the textbox, you're only going to get that event fire when you tab away from the textbox. It's not going to fire every single time you press a key, and so it's not very good for our auto-complete example. But good question. But if I backspace and delete everything here, I'm not sure if Wikipedia, what might be happening under the hood is Wikipedia might not be able to handle an empty search. So that might be why we're not getting some search results here. But, we probably don't want to send a request for an empty search in the first place, do we? Right? Probably not that useful to send an auto-complete box request for an empty search. Question? - A question about this, the retry, they're saying, instantly after the error, and there's another retry sent out. - Instantly after, yes. - So is there anything that you put on there to throttle that to just wait a little bit? - Um, I'm not sure. If not, I think the key thing to understand here is that observable comes on with a ton of operators. A huge number of operators. What I would highly recommend you at home use is rx.lite which is a version that's really stripped down to the most common operators that you would use, and it's kind of relatively small download size once it's zipped and minified, but if you don't have an operator that you need, you guys now know how to write observables. It's nothing to it but a forEach function that returns the subscription. In those cases where there's an operator that you don't want like for example they might be talking about an exponential back off, so one common practice when you have an error, instead of just returning immediately, retrying immediately excuse me, you might want to wait a certain amount of time. And then if you're in another failure, you might want to wait a certain amount of times too. Just to avoid hammering the server. And I don't believe Rx has an operator out, I apologize to the writer of Rx, sorry Matthew, if I'm saying the wrong thing here, but I don't believe it has an operator right out of the box that does an exponential back off but part of the goal here is to be able to teach you as I've shown you how to write observables, if it's an operator you don't have, you can go ahead and write it. Just using the, unfortunately when you're writing observables, you have to deal with callbacks. Remember when we wrote the timeout observable? Or the fromEvent observable? Those methods. In that world, you got to deal with callbacks. But the good thing is, it's only inside of there. You just have to make sure that all that callbacks stuff is taken care of inside of there, and outside you got a nice, clean observable that takes care of itself and cleans after itself. If you're rinsing an operator, you should feel confident, hopefully to go and write that operator. There's nothing wrong with that. That should be easy for us to write an exponential back off retry operator. Won't be much more code than we saw in the fromEvent as a matter of fact. Maybe we can get to that, I'm not sure. We probably don't want to send out an empty search. We probably don't want to send out an empty search. How can we avoid doing that? How do I not send out an empty search, guys? What's the operator I want to use? - Filter? - Filter, right? It's all there is to it. Let me see if I can filter this operator. Maybe I'll use trim, is it, uh. We want larger than zero. Because the filter condition is that, sorry? - Just that in bower.io, trying to find if they have RxJS Lite package. Ironically, the auto-complete search function is failing. - How, on Bower? - Yeah, at bower.io. So let's run that one more time and just confirm that we don't get a search sent off for an empty search term. Let's see if we get a search sent off for A, we should. If I hit backspace, no search. - Clear it, eh? - Sorry? - You should clear the results then. - Oh that's a good point. Yeah, that is actually a good point. Maybe filter is not the right operator here. Because we should clear the searchResultSets. And if we filter it, then the old resultSet is left over. Very good. Very good, so maybe. - And that hasn't changed. - Right, good point. I'm glad we're displaying the results here so that we can see, we can see when a bad decision is being made. If we want to avoid sending out the search results, if we want to avoid sending out empty string or maybe ignore an empty string, there might be a different way we could go about it here. I'll tell you what? Why don't we come back to that? Because I want to move forward and cover some other topics? But that's auto-complete box, anyways, and I think what you'll find is that the same pattern keeps coming up again and again. We have an event, we have an asynchronous request, and then we would have some condition under which we want to stop. You'll pick the right flatten, excuse me, we have an event and asynchronous request, and we want to pick the right flatten for the job. Does that make sense? As soon as we have the event, we'll map it into an asynchronous request and we'll end up with a two dimensional observable. Because that's how we make sure that one thing happens after another, we have an event, and then we use map to say, only after this event occurs do I want to run this function which creates an async, an observable which will cause an asynchronous event, asynchronous request to occur. Because for each item in the event stream we're creating another observable, we end up with a two dimensional observable and then we have to choose the right flattening pattern for it. Does that make sense? Showing the Search Box Let's make this a little bit more interesting. Let's make it actually kind of mirror the types of things that will happen in applications because sometimes applications come up and may have an auto-complete box all the time. It's just up there in the corner. But maybe you've got an application where you, it only, auto-complete box only appears when you open a form. Then it disappears when you close that form. Maybe you don't want to have this auto-complete box around all the time or certainly you might not be, you might not want to hook up to that event all the time. In fact, maybe the auto-complete box doesn't even exist all the time. What would it look like to open a form and then begin listening to the auto-complete box and then close the form and then stop listening to it? Let's zoom out just one level. Because maybe your whole app isn't just this one form. Does that make sense? What if I click a button and then we open a form with an auto-complete box in it, and then there's an X button and we click that X button and then we close that form and make sure that we're no longer subscribed up to that textbox event? What happens if you have an event listener on a textbox and then you remove the textbox from the DOM? - You have a zombie listener. - Right, we're actually, what's happening is you might, you may actually have the DOM objects still in memory because it's holding on to your closure and by extension you. You want to avoid that so what we'd like to do is when we get rid of the textbox, we'd also like to kind of stop listening. When we stop, we want to stop listening to our searchResultSets observable. Let's make this HTML a little more interesting. I'm going to have a button which I click. Again this is not about design, this class. I'm going to trust you guys have the HTML chops to make this a lot prettier. We're going to call this the searchForm. And I'm going to have a button which is going to show this searchForm. Close this. We buy ourselves some room. Now the search area is only going to appear when we click the search button. So what should I do? If I want, the area to only appear when I click the search button, what should we do? What's the first step? What's the asynchronous API I'm going to be working with in order to listen for that search button? - The button clicks. - The button clicks. So what should I do first? Should I just add an event listener to the button click? - Shouldn't subscribe to the button clicks. - I should adapt the button clicks to an observable and subscribe to them. Now, maybe it might seem like overkill, because maybe all I'm doing is I'm just forEach it over this thing and then displaying something. Maybe there's no point in adapting it to an observable. Let's see. Let's find out if there's a win here. I'm going to go into script js. I'm going to grab a searchButton reference using document getElementById. Does anybody know how to make auto-complete work in this? Does anybody out there in the web know how to, because tab isn't working so that would be an awfully nice way to save me from some typing. Now, just like we got keypresses, we can also get searchButtonClicks. What's the last argument to fromEvent? First things first, what do I want to do when search button gets clicked? Now we got it as an observable. What do I want to do, and do is the operative word here. Because I'm changing something. When the search button gets clicked, what am I changing? Well, let's go back to the index HTML. I've got this hidden div which contains the search screen. And I want to show it when the search button is clicked. Because I'm changing the DOM, what I want to use is forEach. Can anybody tell me why I would want to use forEach? - Because you're just going to div once. - Sorry? - You're just going to div once. - I might do it many times. Somebody clicks, every time somebody clicks search I want to open it. My point is, as soon as I am changing something, an object that I don't own, that I didn't create, a global object, I always want to do it inside of a forEach block. Because when you're looking through your code and you want to know where the code is that's causing some side effect to occur, ideally, not always there's some exceptions, and we'll come back to that. But ideally, you should just be able to look at your forEach blocks and then you can see where all of the effects are happening in your application. Why don't we start there? I'm going to, actually I'm going to call forEach, and then every single time the search button is clicked, I'm just going to delete this now. I'm going to go ahead and display the search screen which is named searchForm. What's the opposite of none? Did anyone remember? - Block. - Block. Thank you. Let's see if that works. Search. Didn't work. - If you go up to where you register the button, do you have that in quotes? FromEvent. - Ah, I do, thank you. Let's run it. Now when I click search, we see our box. Now we want to be able to again, not about design, you get the picture, it'll pop up a nice pretty dialog, but you know that's the idea. With a little splash of CSS we can make this look a lot prettier, but, we also, when we pop up dialog, we usually have some sort of button to close it. We want to be able to close that dialog. Usually an X button on the top. But let's just make it a little easier than that. We can worry about the CSS afterwards or not at all. I'm going to put a close button in here. We're going to stop, and we're going to save, we're going to run again. Now let's make this close button work. The interesting thing is, this close button is always there. It's just hidden on the screen. But it doesn't have to work that way. I could actually remove these elements and only create them when somebody clicked search. Rather than just subscribing to the close button, why don't we only subscribe to the close button after we display the form? Because then that gives us the flexibility later on to lazily create these DOM elements. Does that make sense? Rather than have them on the screen all the time. Because that uses a lot of memory potentially. Close Button Observable When searchButtonClicks happen, we know we want to cause a side effect. We want to show something on screen. But we also might want to use that as a stream of information to create our searchResultSets. Why would that be? Remember the goal here is, when the search screen is open, we want to begin listening to the clicks and the text, to the keys and the textbox, and start generating these search results. But as soon as somebody closes the search screen, we don't want to be subscribed up to that textbox anymore. That gives us again, the flexibility later on to remove the DOM elements and we won't have a memory leak. Let's go back to our searchResultSets and let's make it a little bit more complex. Let's listen for the searchButtonClicks and only begin, excuse me, only begin listening for keypresses after a search button click. See how I've used map to order things here. I'm saying now don't subscribe up to keypresses all the time, just subscribe up when somebody has clicked the search button. Remember, map orders things. It means that this function, this map function is only going to get executed when a click happens. It necessarily has to happen that way. This whole function can only get executed when we get a clickEvent. I don't particularly care about the clickEvent, but it will be passed in, so that's why I just left it out. In general, I recommend that, if you don't care about the input from stream, just leave out the identifier, because it causes less cognitive overload when people, rather than introducing identifiers that aren't used. Because some developer might look at that and assume that variable's used somewhere. For the same reason you wouldn't declare a variable and then never use it. It's like an unused variable. Now we see that we're only going to begin listening to keypresses when the search button is clicked. But, how many dimensions is this collection now? What datatype is this? What's this whole sub-expression? What did I hear? - That's an observable. - Yeah, it's an observable. So if we map over an observable, and for every entry inside, return another observable, how many dimensions are we? Two? Right? For the same reason that if I take an array, go back to the old example, and I do this, I get this. So whenever we map and then for every entry, we return another observable, we end up with a two dimensional observable. And so that means we need to do what? - Concat. - Is it concat? So what are our options? Let's talk about the options and which one we should use. - It's always going to be switchLatest. It's always switchLatest. Well I don't know about always. I didn't say always. But most of the time certainly. Now which one would we want to use? It might be switchLatest because obviously the next time somebody clicks the search button, we don't want to listen to keypresses multiple times. Somebody clicks the search button twice, we don't want to be listening to keypresses twice which is effectively what happened if we didn't switch to the latest one. But now we have an interesting case here. We have an interesting case because switchLatest is great for those times. Remember we talked about this yesterday. SwitchLatest is great for those times when you want to stop listening to an inner observable based on the same event that causes another observable to be created. In this case, I don't actually just want to stop listening to the previous, to the textbox, as soon as somebody clicks search again, I actually want to stop earlier. What's the event that happens that makes me want to stop listening to the textbox? - Close. - Close. Exactly. That's when I want to stop listening to this particular observable. That is not the same as the searchButtonClick. And it's the close button inside of search when we display. This is what I want to listen to. How do I cause this searchButtonClicks or technically the sub-expression keypresses. What I really want to do is I want to complete that, I want to complete this when the search button is clicked. So I only want to listen to the keypresses until a search button is clicked. - You mean until the close button is clicked? - Excuse me, close button. Thank you. I only want to listen to keypresses until the close button is clicked. How am I going to do it? What's the right operator to use here? - I mean, we like our drag, right? You start listening at the mouse down, you stop listening at the mouse up. You start listening at the search button, you stop listening at the close button. - Yes, so what's the operator I use? What's the operator we use in drag it up? - TakeUntil. - TakeUntil. So what do I have to do? I have to takeUntil and I don't have an observable for the closes yet. Why don't we just create one of those? I'm actually going to create it here. I'm not going to create it out here. Why does it matter where I create it? So I can put closeClicks here. I can go observable fromEvent. Is it closeButton? - That would affect other functions. - That, I heard, that would affect other functions? - It's not the right scope. - It's not the right scope? - It only matters inside the form. - Yeah, well let's take the thought experiment that we only actually, so instead of just showing, instead of just showing this form, let's say in this code when you click search we actually create the DOM elements. So maybe at this level of scope, close button isn't even there. But now, here inside of here, is close button available? Yeah, because we know we've shown the search button. Now that's technically not entirely true. Because it relies on order. It relies on the order of this event handler just happening to execute before this function and that's a dangerous place to be. We don't want to be in that place. So sometimes a pattern that people will use is they will have an -ing and an -ed observable in their application. Search opening, search opened. When you have a search opening observable, you can do some side effect and then cause a search opened observable to fire. And then when you listen to the search opened observable, you can be sure that all the side effects that needed to occur in order to create those DOM elements have successfully completed. Does that make sense? We'll come back to that, we're going to, we're going to have explain a new concept to explain how that works in just a moment. But for now let's rely on the fact that it just will happen to work this way. Because I believe it will based on subscription order. Now that we've created closeClicks, we can do a takeUntil and closeClicks and now what's the right flattening strategy to use? We still haven't put in one. We could use switchLatest. The truth is all of them are, well actually, that's not true. Notice we still see the search button when we open the form. That's probably not the way it would work in real application. You'd probably click search and then you'd move to this form and you wouldn't see the search button anymore. But who knows, maybe it's in the toolbar and somebody can click search again. It's possible somebody could click search again before clicking close. And then what would happen? Probably not something good, right? It would probably either create the DOM elements twice which we wouldn't want, or we would avoid creating a DOM elements twice but subscribe up to the textbox a second time. So we wouldn't want that. One thing we could do and one thing I would probably do in this instance is inside of searchButtonClicks, I would hide the searchButton. Right, that's one thing we could do. Another, actually let's not go that route. Another route we could go is we can just make sure to clean up the DOM element if it occur, if it's there or do nothing effectively if the, the searchForm is already visible. That's probably a better way to go. Because you probably, you might have search in the toolbar for example. So we can check to see if the display is already visible. I guess I can leave this logic out here because we're not currently dynamically creating the DOM nodes, but you'd imagine if you were dynamically creating the DOM nodes, first you'd check to see if the DOM node existed and then you wouldn't dynamically create it. Does that make sense? Now, under that circumstance, the fact that somebody could click search again, what's the right flattening pattern to use? - SwitchLatest. - SwitchLatest. Yeah. Because then they can click search as many times as they want, and then we were just going to unhook from the event handler and then hook it up again. If I go, now here's a question. Where does the switchLatest go? Just want to make sure we really understand what we're doing here. Where do I put the switchLatest? I put it here? How many dimensions is this observable? This sub-expression right here is actually only one dimension. Because we've already switchLatest it. We took every keypress, we mapped it into an observable that represents a request, and because we represented every item in the observable with another observable, we got a two dimensional observable. And then we have that concurrency problem we wanted to solve and so we use switchLatest to flatten it out so we only got the latest result. So this itself is a one dimensional observable. If I applied switchLatest to a one dimensional observable, what happens? What do we think? What happens if we concatAll a flat array? - Error. - It errors. Yeah, concatAll, switchLatest, mergeAll, they're all expected to work on two dimensional collections. So, I can't apply switchLatest here. Where would I apply it then? - On map? - Yeah, on map down here. Why? Because now since this is an observable. We know this is a nice flat observable. But we're creating it for every single search button click here, because we're creating this observable inside the map function for every single time somebody clicks search. So we create this observable. We actually, at this level, we have a two dimensional observable. Because for every search button click, we're creating the stream of search results from the server. And so, we'll switchLatest. And that'll work regardless of whether the, the search button is visible or not. Let's see if that worked. Now, notice, we still don't have our close button doing anything, right? That would be kind of ideal. We'd want our close button doing something. Completing the Close Button Here's where we get to a place where I want to talk about exceptions to rules. Every rule has an exception. One of the exceptions to rules here is that sometimes you want to trigger side effects when somebody subscribes to an observable. In general, that's going to happen, for example, when we make a network request. Side effects happen sometimes when you subscribe to an observable. Again, you want to delay them. You don't want to do the side effects when you create the observable. You only want to do the side effects when somebody forEaches over the observable. You want to make sure that the observable's lazy. Let's take a look at the getJSON request we made. That actually, notice that that triggers side effects. As soon as you forEach over that observable, what happens? It issues a network request, and sure enough, that changes the global object. So it's not that you can't have side effects, it's that ideally you want to encapsulate them inside of the observable, so that when somebody forEaches over that observable, then and only then you trigger the side effects. What would be nice is, and then instead, in addition to doing this inside of the forEach, we can also use an operator called doAction. What doAction does is it make sure that when you forEach over an observable, it injects some code that you can run to cause side effects to occur. Does that make sense? If I forEach over an array, there's no side effects. They're not really changing any global objects. But, with doAction, I can cause a side effect to occur. So I was saying earlier, sometimes we want to inject side effects inside of an observable when we forEach into it. We want to cause a side effect to occur just by forEaching over an observable. We already saw that happen with making a network request. That's what happens when you forEach over a network request, over the observable that comes back from searchWikipedia, we end up triggering a side effect. Somebody's selecting all the text now. At least it's on my screen, interesting. But it's not showing up there. - Are they able to type in there? - You know what, they are selecting, it's just super faint on the, it's just super faint on the projection. But someone is selecting. Someone selected all the text. Yeah, it's happening. - The text selector. - Whoever out there is selecting all the text, do not select all the text. Pink is, it's nice, but. - Yeah, I guess. I guess we're going to have to roll--- - Is this the My Little Pony theme of text editing. - Yeah, the problem is that you guys let more than user at a time. - They seem to have unselected. Thank you whoever you are out there. We've learned about map, filter, mergeAll, zip, and reduce. We've learned about flattening patterns, so we've learned how to build a collection functionally which is that you keep mapping until you have all the data you need in scope, you create your result, and then you figure out how many flattens you need and where to apply those flattens to get down to a nice flat collection. And then you use forEach to consume it. That's just the functional programming style of building collections. Now, you learn that when we start working with observable, we have an additional question to ask ourselves which is in that last step when we do the flattening, the question is, which flatten to use? Do we use mergeAll? Do we use concatAll? Or do we use switchLatest? Another thing about observables that you don't have with arrays is that when you forEach over an observable, it may cause something to happen. It may actually cause a side effect to occur. That's the case with the observable that we get back from searchWikipedia when we forEach over that, we actually trigger a network request that goes across the wire. That can be something that we can use to our advantage in this particular case. SearchButtonClicks. Remember the problem we had? We said, which is that we're relying on the order in which those event handlers fired. Maybe this event handler fires first and then we display or perhaps even create those DOM elements and then later on this event handler fires but we can't be 100% certain of that. If we want to inject side effects, meaning when you forEach over an observable, something happens, we can use the doAction function. So the doAction function works very, actually has the same APIs for each. The only difference is, it returns an observable. If you put doAction in here, this is technically the action to occur when you get an onNext, so you can take the value, and then here we can do something. In here, we actually want to take the side effect and put it right in here. So that whenever you onNext over searchButtonClicks, it injects the side effect. What I've done here is I've actually just created another observable. I haven't, not like forEach where we run something, what we're doing is we're just saying when somebody calls forEach on this observable, do this action. I'm actually, this is going to return yet another observable which I'm just going to alias to the exact same one as before. Actually, it's better practice to see, let's say, it's always better to be semantic when you're naming things. We've gone from searchButtonClicks to searchFormOpened, or searchFormOpens. I usually name my observables plural because they're streams of things. For the same reason you would name arrays plural because they're collections of things. I'm going to call this searchFormOpens and now I know when we onNext searchFormOpens, that form is already been displayed. Does that make sense? Because that is, this is the code that gets injected in when you call forEach. Now we can be sure that by the time this code runs, this code has already run. Because when we forEach over this, the first thing going to happen is that this occurs. Let's just confirm that this still works. Hopefully. No, because I'm not listening to searchFormOpens here, I'm listening to searchButtonClicks. Now that I switch it to listen to searchFormOpens, we should see that it works. Great. And if I type in A-B, we get my search results, great. Now, we want to do the same thing with closeClicks. When you click close, we actually want to hide this form. This allows us to just inject some side effects into the closeClicks observable such that when somebody clicks close, we can be sure that we're going to hide that form before we get any feedback when you forEach over that. Here inside of closeClicks, we created closeClicks, going to make this smaller, and we're going to create a searchFormCloses. That's going to take closeClicks and add a doAction and this is just, that just means we're going to do something when forEach is called and in this particular case, we're just going to take this and hide it again. Now instead of takeUntil closeClicks, we're going to do a takeUntil searchFormCloses. Yeah, question? - He's asking what if the side effects is asynchronous? - Um, if the side effect is asynchronous, you can't do it with doAction. DoAction is only synchronous. But what he's really saying I think in effect is, what if you want to perform that side effect after an asynchronous operation occurs? And then what you would really need to do is you'd need to use map, because you want to map and then create an observable that causes the asynchronous thing to occur, and then do a doAction on the inner observable. I would take, what he's saying is I want to do some asynchronous action when an item arrives through the stream. Then you would just map over it just like we cause an asynchronous request to occur when we type a key. We will take an observable, and when it onNext us, we'll add a map and then we'll create an observable that represents that asynchronous action. On that inner observable that represents the asynchronous action, that's where we add doAction, and then at that point we know that the asynchronous action's completed and so that we can now synchronously cause a side effect to occur. Almost every side effect in a browser is a synchronous API. Because when you work with a DOM, it's all sync APIs for changing the DOM, removeChild, appendChild. If that's what you want to do, if you want to wait until some asynchronous event occurs and then cause a side effect to do it, you would just map, return that observable and then doAction on that inner observable. Does that make sense? Now we make sure that everything executes in the correct order, any time we want, we can inject side effects into an observable. Because remember that observable's just a forEach method? And it's just like tacking on some code inside of that forEach method which you want to run at the top of the forEach method. Think of that as the mental model for what doAction does. And doAction also has overloads, by the way. If you want to do a side effect on onError, or onComplete, you just pass it, it's got the exact same interface as forEach. So you can just do something when an error occurs or do something when a completion occurs. In this case, we don't need to do that. Does that make sense? It has the same interface as forEach. You can just cause some side effect to occur if an error happens to come through the observable, or when a completion happens to go through the observable. That wasn't clear. I'll put that up one more time. Just the same interface as forEach. DoAction is exactly the same as forEach except forEach actually triggers the observable to do something. Whereas doAction just returns another observable that when forEached will cause those actions to occur. One of these is lazy. DoAction is lazy. We're not doing anything yet. We're just creating an observable that when you forEach over it, something will happen and forEach is eager. By the time you call forEach, you're actually now doing something. Does that make sense? Any time you want a UI action to occur, whenever somebody forEaches over a stream or an event occurs, it's usually best to stick that in a doAction. There's just some side effect that's occurring along the side. We see here that if we run this, it should work. Great. Now how do I know, if I close this, it's gone, but how do I know it's actually unsubscribed from the event handler? Let's try something out. Let's try not hiding it on close. Let's comment out this close here and let's run it again. So now we know it won't be hidden. But, if everything worked correctly, if I type into the textbox after clicking close, I shouldn't see anymore search results go out. Because we've stopped listening to the inner stream of searchResultSets the moment, we were only listening to that stream until the close button is clicked. Now that I've clicked close, right, hopefully, nothing. So that's how you clean up after yourself when you close and open sections of the user interface. You always just think about it in terms of, there's some event which causes this form to open, and there's some event which causes this form to close. Just like sequencing any action with observable, you'd take the thing you want to happen first, you map, and you return the thing you want to have second, and then that gives you a two dimensional observable, because for every item in the stream that you want to happen first, you're substituting in an observable for each one of those items, then you have to pick the right flattening strategy. That's the question you have to ask yourself when you work with observable. Not with array. With array, it's just how deep am I, then I know to apply that many concatAlls. But with observable it's how deep am I and which flattening strategies do I apply at each step to get the right concurrency pattern that I want. Now that we've explained mergeAll, switchLatest and concatAll by visualizing them, hopefully try and capture that mental picture and project that mental picture as you're going through and solving these problems to try and decide the behavior you want. Because there are cases in the UI for all of these things. You'll find cases for mergeAll, you'll find cases for concatAll, but for the most part, you'll find cases for switchLatest. You only want to do the latest thing. I'm going to uncomment closeClicks. We've built something that's very much akin to the type of thing that you're going to build in user interfaces. This is how you will build user interfaces. Someone will open a form, you'll have some asynchronous action occurring while that form is open, and as soon as somebody closes that form, you'll want to completely clean up after yourself. This is exactly the pattern that you use. I'm going to fork this so people can come back and refer to it. So search, uh, no search results for that, no surprise. Audience Questions - Is the close buttons observable gone to, I'm thinking it's still around if you click close. - Good question. What happened to the close observable? Let's see where it lives. It lives inside of this closure. As soon as this closure ends, and, well not technically as soon as this closure ends, because we're listening to searchFormCloses, which is all listening to keypresses. It's very difficult sometimes to figure out if something's still in memory. But notice that as soon as this closure runs once, we're going to begin listening to keypresses. And keypresses is a global event in the browser. If you hook up an event to keypresses, it's there unless you come along and remove it. So even though this whole function completes and we've declared closeClicks inside of this function, you might expect that this observable has gone away, but it hasn't. It hasn't because it's chained off of a closure of a closure of a closure of a closure, holding on to keypresses, to the, excuse me, to yes keypresses on the DOM event that's still inside of the DOM. As long as that object is still inside of the DOM, you're going to have that handler exist and it's going to be holding on to indirectly this right here. You can just follow the chain down and see that because we're still listening to keypresses, we're still listening to searchFormCloses. Does that make sense? Yeah, another question? - When does searchButtonClicks forEach get run so that the doAction is fired? - It will get run as soon as this whole operation is forEached over. If I delete this, we haven't even hooked up to the close button or the search button or anything for that matter. Because all we've done with this whole expression here, I'll get rid of this extra boiler plate code, with this whole expression all we've done is create an observable waiting to be called. So I haven't done a thing, haven't hooked up a single event handler. It's all because of forEach. Remember, we can go through the, go around the table and play the forEach, onNext game again, but that's basically what's happened. We've created an observable but nobody's called forEach yet. It's just an object with a single function waiting to be called. Keep crossing my hands over that microphone. Yep? - What would be the best way to test for open observables? - For observables that have not yet been unsubscribed? Is that the question? - Yeah. - That can be challenging if you're trying to figure out if an observable hasn't been unsubscribed. It's difficult. You could always go inside of an observable that you own and log the subscriptions and log the unsubscriptions so you could use doAction for example to log. You could create an observable that wraps around another observable and every single time forEach is called, it could just forward on the forEach and then log open, and then every single time an onCompleted is called, you could log close. You could use doAction on the onCompleted to log close. DoAction doesn't have a function for onSubscribe but you could just create an observable that just forwards the forEach call onto another observable to do that. Yep? - Could we use take one for the search on close stream to clean up those listeners? - We could, but that's just going to end up being a no-op. It's going to end up being a no-op because that's what takeUntil does under the hood. Effectively it does, as soon as it gets a single item out of searchFormCloses, it calls subscription.dispose on searchFormCloses. Then it stops listening to searchFormCloses. Granted, as soon as somebody calls search again, as soon as this search button is clicked again, this function will run all over again. We will create closeClicks, an observable which is based off of the close button clicks. And then we will create this whole sub-expression all over again. And then we'll be listening to that. But, in the meantime, between a search and a close click, did I break my example? I broke my example. In the meantime, we shouldn't be listening to that event at all. This was working. That should work now. Okay. Between the search and the close click, as soon as we hit close, we're not listening. And we proved that earlier. We're not listening to the textbox anymore. We're creating these observables, but then we're really throwing them away as soon as another item comes through the stream. In other words, as soon as we click search again, we're creating, potentially all those DOM elements again. In this case, we're just hiding them, or showing them but as you can see, wherever we're hiding and showing, we could actually be dynamically creating and putting on screen those elements. It's safe to do that because we're also making sure to unhook from the textbox, from the keypresses event. So we won't just leak event handlers everywhere. You will see the same pattern again and again in user interfaces. Some action causes a form opened. That may contain one or more asynchronous actions. And you all wanted to stop as soon as you hit a close button or that form closes. So that combination of switchLatest and takeUntil will get you that without ever having to worry about explicitly unsubscribing from events. Observable in Depth Creating an Observable Class So what I'd like to do now is I'd like to cover a couple of small concepts. One is a subject. Has anybody ever heard of a subject before? Let's say you want to fire an event when somebody calls a method. Let's say I want to create an observable that fires every single time somebody calls a method on a class for example. Because that can sometimes be useful. Let's say you create an object in JavaScript with a function, and every single time somebody calls that function you want an observable that fires because you might want to do something asynchronous. Well, there's no real way to do that. We know how to adapt an event, a dom event to an observable. We know how to adapt a typical call back API to an observable. But what if you had an object and you wanted to create your own custom event? Such that when somebody calls some method or does something that event fires. Now you could use that event listener, you could expose an at event listener or remove even listener. But you can also use a subject. Let me show you a subject in just a moment here. And we are going to be implementing, observable from the ground up. So, as I do this we're going to sort of learn what's going on under the hood. It's not that, the interesting this is, the magic to make all of this stuff happen is really not that much code. So, all of the stuff that you've been seeing happen, takentil, concatAll, map, filter. There's a surprisingly little amount of code here. And, that really comes down to what this is. It's not about writing a lot of code, it's about thinking differently about events. And that's what going to cause you to be successful when you take on asynchronous programming. Just thinking differently, having a little twist on the way that you already think about events. So, let's start by defining the observable type so to speak, the observable constructor. Now remember, observable only has one method called forEach. So that's actually something that I'm just going to accept as a constructor. And I'm just going to add it as a private member, on the observable. Now remember that observable allows you to call forEach in a whole bunch of different ways, right? You can either pass in an observer object with an onNext and onError and a completed. Or you can just pass in three function arguments one after the other. So we'd like to support the same thing. So if the type of, the first function argument, is a function we'll know that they're passing in functions instead of an observer object. So now I'm just going to call forEach. Actually, I can make this a little more readable. Remember the three handlers that get passed into an observable. Alright so if onNext is a function, there's only two possibilities here. That the very first argument is that object with the three onNext, onError and onCompleted properties, or they're passing in functions. So they're passing in the onNext function, the onError function and the onCompleted function. So we're just going to check to see if the very first parameter is a function in which case we are going to build the observer object ourselves, out of the functions that they pass in. Now, remember onError is optional. So, if it's not passed in we're just going to define an empty function for it. So that, when we work with an observer it's always got a function for onError and onCompleted. Even if that function does nothing. So, does everybody understand what this code does? I just want to clarify to make sure everybody understands what's going on here. What this code effectively does, it's as if onError is null, substitute it for this empty function right here. And same thing on the next line. If onCompleted is null, we'll substitute it for this empty function. So that's one possibility. The one possibility is that they're passing three functions. The other possibility is that they're actually passing in an observer object. In which we don't have anything to do. We're just going to forward on. I forget my return statement here. We're just going to forward on the call to our private forEach implementation, and we're going to pass in onNext. Which in this case is an observer. Alright, so those are the two ways you can call an observable. You can call an observable this way. So by passing in three functions. Or you can call an observable this way. Alright, so we want to support both, and that's all this public forEach function does. It just makes sure that we support both. So if somebody passes in an observer object right here, we just use it and pass it to our internal definition of forEach. And if somebody passes a just three functions one after the other, we construct an observer object with the onNext, onError and onCompleted method. And then we pass that to the internal definition of forEach. So, when you create an observable, right, we're going to do it like this, we're just going to go, new observable, and remember, what's the only function that's defined on observable? The only function you have to define to define an observable? - ForEach. - ForEach. So that's the only thing we have to pass into the observable constructor in order to create an observable. So we actually just have to pass in an in line definition of forEach. So, let's create a function which will fire. Actually, let's create, remember our fromEvent function? Let's create the fromEvent function so that we can work with dom events and turn them into observables. So I'm going to create a fromEvent function on observable, and remember it takes a dom element and an event name, and it returns an observable. Make this little bit smaller. Now let's go through how do we take an event based API and convert it into an observable? Well, when somebody calls the forEach function, we want to add an event lister to the dom. Well actually first we're going to create a handler, and when that handler fires, we're going to take the event object and we are just going to call observer onNext, and thread that event object on to the observer. Remember, an observable is just an object with a function, a forEach function, and when you add in an observer, it's going to call that observer every single time, it receives an event from the dom object. So, now we just have to take the dom element, add an event listener at the event name. We're going to add our handler, and what's the last thing that we have to do? What's the last thing a forEach function has to do, when you call it? What does it have to return? When you call forEach on an array, subscription, right? Because you need a way to say, no more data. So we're going to return an object with dispose, and inside of here we have to make sure by doing something that none of the observers onNext, onError, onCompleted call backs are ever going to get called again. So, I'm going to unhook the event handler, like so. And so this is our subscription object. So, this is actually all the code. Yep, got a question in the back? - So why are we doing, that one up there to observable prototype but this one we're just doing to observable? - So, just let's take a close look at the observable constructor again. Notice that observable accepts an argument which is the definition of forEach, right? But notice that you can also call, we saw there's two different ways you can call an observable. One is where you pass three arguments, which are functions. You pass in the onNext, the onError, and the onCompleted function, is three arguments. But, another overload is that you can call with an object that has an onNext, onError, and onCompleted property. Instead, whenever you create an observable. Instead of you having to worry about handling each of those cases. Right, because otherwise you'd have to really kind of, you'd need to make sure to check each time somebody called your forEach function, if they were passing in an observer object, or three callbacks, we are going to add a public forEach method to observable that does that work for us. So, it's going to first check to see are they passing in functions? In which case, onNext and onError, and onCompleted are all going to be individually functions. And then, if so it's going to create an observer object. It's just going to wrap those things inside of an observer object and it's going to call the actual definition of forEach that we passed in the constructor. So that way, when we create an observable, we don't have to worry about this little step of saying, oh are they passing in an observer, or were they passing in three callbacks, because when we have a public forEach method that gets called, and then, it creates an observer every single time, and then calls our internal definition of forEach. So, inside when we define an observable, we only have to worry about the case where we get an observer. Does that make sense? So that's all that's going on here is saying, hey are they passing an observer object? In which case just go ahead and take whatever it is, the first argument and just thread it down to the internal definition of forEach. Or are they passing in functions? In which case create the observer object, and then pass that into forEach. So, now when we define a forEah method, the only case we have to worry about is getting an observer, okay? So, now when I go in define fromEvent, and I create an observable when you call fromEvent, I only have to worry about ever getting an observer. Does that make sense? So inside of whenever we create an observable, when somebody calls forEach on us, we've got two jobs. Our job is to start whatever action may cause asynchronous data to start flowing through, and to return a subscription object, which when disposed of will stop any asynchronous data from flowing through. So, here's step one and here's step two. This is will cause us to potentially start getting messages from the event. But when somebody disposes of the subscription object, when the caller forEach disposes of the subscription object we'll make sure that we never give them anymore data ever again, because we'll have removed the event listener. Okay, we're good? Observable Map Function - So we've got fromEvent, now let's see if we can write map. Remember when you wrote map over array? I bet you it's not much harder to write map over an observable. So, what's it going to look like to write map over an observable? Well, we've added forEach to the prototype. Let's add map to the Prototype. Now, what does map take? Explain to me how map works again, remind me. What's it's argument? What's the only argument to the map function? - Two Now, map is on the array object, in this case map is on the observable object. What does map accept as it's one and only argument? - A function - A function, a function to apply to ever item that comes through the stream. Remember we said that observables like a hot potato, every single item that comes through, we're going to translate it and hand it off right? We're just going to keep calling callbacks all the way through. So, let's write map. So map accepts what's called a projection function. Which is, long story short, just a function that you apply to every item that comes through. And you take the result and forward that along. So what does map return? An array when it's on an array, but when it's on an observable it returns. - An observable. - An observable. So, right there we know that we need to return an observable. And when the forEach function of that observable is called, what do we do? Well one thing I'm going to do. You guys have probably done this in Javascript before. I'm going to capture the current this object. Who's written code like this in Javascript before? A few people. So, one of the very complicated things about Javascript that is not intuitive, is that functions do not execute necessarily against the object in which they are created but they execute on the object in which they are on if you call that object. So, a.function when you call that function this will be the a object. Not the object whereof the function itself was created. So, when we call this inside of here, this will be this observable, it will not be the observable that we called map on to create this observable. So we need a way, remember what happens. We're going to step through this now, again which human beings. I am the source observable, and when you call map on me you create her, and she is the mapped observable. And now, when you call forEach on here, right? Call forEach on her. - forEach. - forEach. - I say one, and let's say you add one to each item I give you. OnNext one. - onNext two. - Right, onNext five. - onNext six. - onCompleted. - onCompleted. - Yeah, right so it just keeps, we just keep handing it off. We just keep sending messages along. We're going to demonstrate this right now. So the point is, when you call forEach on her, she has to call forEach on me. And that's why I have to capture for this object here. Because, this is the mapped observable. Alright, and when you call forEach on it, her, she has to call forEach on me. So, how else is she going to hold on? Because remember, the observable is the this object, inside of this function, it's the thing that you're calling mpa on right? So we need to capture, so I'm going to call var self equals this. And then, as soon as you call forEach on her, she's going to call forEach on me. Now, forEach takes three callbacks or an observer. In this case we're going to use three callbacks, right? The first callback is the callback we get when we get called with data? So, what is my responsibility at this point? She's calling forEach on me, so this is the part where I say onNext one. What's your responsibility now? When I say onNext one you responsibility is to do what? - onNext two. Right, onNext to him, the result of applying the function to what I just gave you, right? So, what does that look like? Well here this is him, he's listening. He's listening, and so we have to call observer onNext X, but first you're going to apply your projection function. You're going to call the projection function onX before you give him the value. That's almost done. We're almost done now. That's all there is to it right? When you call forEach on her, she calls forEach on me, which in this case is self. I onNext her information, we're going to actually write this out so it's clearer. I'm calling her onNext, and she's calling your onNext, but first she's translating it. She's running it through the function before she pushes it to you. - So, does she call projection function? - Yes, if she is the mapped observable, and you are the person who's calling her for each method. And you're passing in your observer. Right, and then she is, in order to give me, in order to give you my data translated, she has to forEach over me because that's how you get data out of an observable, right? You have to forEach over it. So, she forEaches over me. I'm the self here. I'm the this object. I'm the observable that you called map on. That's to the left of this map. It would be observable dot map, and then you'd create a mapped version of that observable. That's what we're returning at the top level. The mapped observable. And then, when somebody forEached over the map observable the mapped observable has to forEach over the source. And I onNext data, and she onNext it to you. She's just forwarding it along like a hot potato. So onNext, that's really good, but what happens if I say, onError, what are you going to say to him? -So there's no retry. - Right, and mapped observables don't do any retrying right? So, you forward on the error call. And what's the last possible message that I can say to you? - Completed. - Completed. At which point what's the logical thing to say to him? - Completed. - Yeah, because I'm not going to send you any more data, so obviously you're not going to send him any more data right? - So at that point would the observer Unsubscribe or terminate or? - As soon as she sends an onCompleted message to you, yes it's the observables responsibility to technically unsubscribe and make sure that it cleans up after itself. Because, I, the interesting this is, here map doesn't need to clean up after anything. It's actually the source that needs to clean up. All she's doing is forwarding on messages, right? If she makes sure that I'm never going to send a message anymore, then by extension she's never going to send a message anymore. And notice what I'm doing with forEach here. What's coming out of forEach? What's the object that's returned when you call forEach on an observable? - Observe. - No, not an observable. When you call forEach on an observable what pops out? - An array. - No, let's go back to this example. - Subscription. - A subscription! Right, when you call forEach on an observable a subscription object pops up, because that's your way of saying no more data please. And so, what we're doing inside of this map, this mapped observable, it's kind of elegant. Because she doesn't need to create a subscription object to hand to you. She's actually just going to take the subscription object I hand to her, and hand it to you. Right, because if you call dispose, and you cause me to never send any more data to her. Well by extension she's never going to get any more messages to send to you. So, she's just saying, oh thanks very much for that subscription here you go. She's handing it right to you. And that's all there is to that. All we need to do is take every onNext I give to you, and forward along to him after running it through a projection function. And onError and onCompleted, you just forward along. Should we try filter? - Where are we handing that subscription off? - Right here, because we know I'm the source right? I'm going to say onNext one, onNext two, onNext three. As soon as you say forEach, the floodgates are open and I'm just going to say onNext, onNext, onNext. But as soon as you call forEach on me I'm going to hand you what? Every observable hands you what when you call forEach on it? - The subscription. - The subscription object, because you need some way of saying I don't want anymore data. And so, as soon as she calls forEach on me, I'm going to hand her my subscription object. That's what comes back from forEach. Right, that's why we're just returning forEach. So, I hand her my subscription object. And she just turns around and hands it right to him. Because if he disposes of my subscription object, what does that mean? I'm never going to send her another message. And what does that mean? She's never going to send him anther message. Because all she does, is wait for me to send messages and then forwards them along. Observable Filter Function - So there's a mapped observable in ten lines? It's not a lot of code. Should we try a filter? Let's see if you guys can help me write a filter. You know how filter works on array. So, filter is not called a projection function, it's called a predicate, actually who cares about the fancy name, let's just call it what we all know it to be, a test function. It returns true or false, based on the object going in. So, now her job is to be a filtered observable. And I'm the source observable, right? And he's going to call forEach on her. And her job is to make sure not to give you any numbers larger than five, okay? So, you're only going to get numbers smaller than five right? So, let's try it again, forEach her. - Okay so I'm forEaching a function, right? And that function is going to have that logic built in? - Well, so she's the observable that is the result of calling filter on another observable. So when you call filter on an array you get an array back right? When you call filter on an observable you get an observable back. So, let's say you, I'm the source observable, and I'm just going to onNext numbers. Number, number, number. You filter me and you create her. She's now a filtered observable that's holding onto me. She knows how to talk to me, and when you forEach over her, she's going to forEach over me. Because in order to give you my filtered data she has to ask for it, and that's how she asks for it, she forEaches over me. Okay, so let's play it one more time. Filter, what's, we're going to do it live and then like with acting it out, and then we're going to code it up and we'll see the correspondence. So, forEach over her. - So forEach over the observable. - Sorry, oh forEach. - Right, she forEaches over me. I say onNext one. You're only going to give him numbers larger or smaller than five excuse me. So one. - onNext one. - onNext seven, onNext nine, onNext three. - onNext three. - onCompleted. - onCompleted. - So she's only, sorry? - Dispose? - Well, you don't call dispose. When she says to you onCompleted you don't have to call dispose, why would you call dispose? She's never going to call you again, right? It's her job that when she says onCompleted, She cleans up after herself. There's no reason for you to call dispose. - She's going to call dispose? - Under the hood yes, technically she calls dispose. Or at least does everything that would happen when you call dispose. Her job is just to clean up after herself, so that she's never holding on to any of his references. Does that make sense? So, let's try it now in code. Before, this was a mapped observable, and now it's a filtered observable. So, this is the filtered observable. This is the original source observable, which we're capturing, by a var self equals this. And now when you forEach over it, inside of the onNext what are we going to do? When it was a map, we just applied a function to it, and then took the result and threaded it on through. But, what's different about filter for map? - Because you're comparing, there's a test. - There's a test function. So we need to take the data that I onNext'd her, like nine and we need to run it through the test function. And then, how do we decide? How does she decide whether to onNext him? - If text function returns true she will onNext him the thing, whatever it is. The value. - We don't translate the value right, filter doesn't change values, it just decides whether or not to forward them along. So, I got a few errors, they're probably missing semicolons. - So is that basically what we wrote when we implemented filter? - For array, yes. You did the equivalent for an array. If you think back to what you did over an array. You forEach'd over. Yeah and if it was true you ran. If you go back and even look at your definition of array you'll see the correspondence, right? You forEach'd over the source, and if it happened to run through a test function and return true, you would send it along. In your case you'd put it into the new array with result, stop, push. In our case we're not accumulating up data in a big array somewhere. It's actually a good deal more efficient. Instead of, imagine if you take one, two, three, four and then you map over it. And then you just add one to everything, you're creating a whole big copy of that array in memory. Not so with observable. We don't create big copies of streams in memory. Every single time I give her a value, she doesn't store it anywhere. She just forwards it on to you like a hot potato. So when you create map filter. When you create this big expression of an observable you know what you actually really created? Just a tiny little object graph in memory of functions holding onto functions holding onto functions, holding onto functions. It's very little memory compared to if you map over an observable with 12,000 items in it. You get another observable- excuse me. If you map over an array with 12,000 items in it, you'll get a whole other array just as long. This is the notion of streaming data through right? We're streaming, we're not collecting it up in memory. We're streaming it through. So, let's, you want to take this out for a spin? Let's see if this actually works, see if we can maybe we can map mouse positions or something. - Why are we, I'm sorry, I'm confused but forEach because we're saying function, we're declaring a new function named forEach. - Yeah. - Within the function definitions here. And then we're calling self dot forEach underneath that? But, the self dot forEach is a different forEach than the forEach. - Yes. - In which we're defining, and then we have underscore forEach. So, there's three different forEaches? - Right, well the underscore forEach. The underscore forEach isn't too important. The whole point of this wrapper object, this wrapper forEach's job, it's the public forEach. We always call the public forEach, and it's always the same definition. All it does is it checks to see whether you're passing in an observer object or you're passing in the three callbacks. It's whole job is just to always turn it into an observer object. And then it calls this underscore forEach. The underscore forEach is the one that we are passing to the constructor. And that way when we define what an observable is, we don't have to worry about the two different possible overloads, and maybe they're passing in functions, maybe they're passing in observers. We always get to write our observable definitions assuming that what gets passed into our forEach function is an observer. So, that's the whole purpose of this function here. It's just to sanitize the input, to always turn it into an observer. So that when we write new observables, we only ever need to worry about observers. We don't need to worry about, oh maybe they passed three callbacks, or maybe they passed an observable. - Yeah, and that's what we call when we say self dot forEach. - Yes, when we say self dot forEach- Well, I'm going to be clear. When we call self dot forEach. - In that context. - In this context we're calling the outer forEach. This is the forEach right, but then, immediately inside, we invoke the underscore forEach. Which is what we pass to the constructor. This is just an identifier name to help us understand that I'm just. I get to, I could delete this forEach right here. It's just there to help you understand that, that's the definition of this function right here, forEach. Yeah, question in the back. - You can also name it there? - Totally optional, I can name this blah blah. Okay, question in the back. - You might've answered it, but are we naming the function we are passing through the observer construction. - Purely for clarity, just so you guys know. Because otherwise this would be just sort of a function without a name floating around here. It's just so that you know that the function that we pass inside of an observable is the definition of forEach. That's why I'm naming that function. So, otherwise if I don't name that function I've found in the past, people kind of get lost, they're like what function is this? Because let's face it, we're dealing with a lot of functions. We're not dealing with a lot of code. Notice there's very little code here. But we have some nested functions. So, you got to get, and part of this exercise will be getting really good with understanding function scope and what it means in Javascript. But as you can see, some pretty powerful technique. Because we haven't written very much code, but we've written some very powerful functions. So, let's try this out on a real dom object and let's see if this works. Maybe even the first time. But, more importantly, it's not so much about writing code. We've already shown you what you can do with Observable. You can build really powerful asynchronous programs that clean up after themselves. The point of this exercise is to actually walk through the code and see what's happening underneath the hood. Which will really deepen your understanding of what's going on when you use these functions. Using the Observable Class I'm going to show my html, right? And I'm going to put in a button. And give it an id. And we'll pull in this button up here. And now I'm going to create, button clicks, right? Because the first thing we do when we have an asynchronous api is we adapt it to an observable. So, I'm going to create clicks, and I'm going to use my, the fromEvent method that we just wrote. And I'm going to put in the button, and the name of the event, which is click. And let's just see right off the bat if this works. We're going to pass in forEach function. I'm just going to log it to the console. And if it doesn't that'll be a good opportunity for us to debug it and see what's going on. So, run. Uh, oh, unexpected token. Am I in six to five? I may have just forgotten to set my browser to six to, there we go. Six to five. That will do the transpolation, allow me to use es6. It means literally es625. So, it's going to convert it for us, so we get to use the arrow function. Clear. So, oh I forgot to show the html window, or the output window. - You might just be able to add it to the url. - Add it, sorry? - So you see html come up, yeah. - What? - You just type comma output. - That is a good tip, thank you. So now let's see? Is this font size, are we doing okay font size wise? - Yeah - Okay, unfortunately it's kind of blocking my way here. So, I'll click me, and look at that. We're now forEaching over a click stream, without much code. So, we've got an observable. We've written it out to the console. But what we actually wrote out to the console is the event object we got from the click. That's all that code, that's all that stuff in the console you're seeing all the properties on the event object we got from the click. So, let me make, let's print out just one or two of those properties. I think we had some problems with offset X didn't we? When we tried it on firefox? But page X, we might have a page X for this click button so we can see where we're actually clicking. And this is the x and y position on the page where I'm clicking the button, right? So, why don't we map it? Why don't we translate, nevermind this is kind of a silly program, but it's just to demonstrate the code in action. So, I'm going to map over this. - You can remove the html panel from the comma. - Yeah it'll require. Oh good call, that's much nicer. And we can do with a very small console. So, let's map over this now. As soon as I have more than one operator. Notice what I do, I move it to next tab level, so we can line up. And, let's take this event object and why not just pull off the page X and let's just add 50 to it. I know it's a silly program, but we kind of get the point, right? Now we can just display the data, we don't have to pull off pageX. Now we just actually got a number. So, we've just taken pageX and we added 50 to it. Well why don't we just, just to make sure it worked, turn it into a string, and put tx afterwards? Run it. So, we can see that we're now mapping this observable. We're mapping the data comes out. We're translating it right? And I actually only want to see numbers, that are say, larger than 40. Oh, there we got one. - Partially breaking it. - So, let's take a look at what actually goes on, under the hood shall we? So, how are we going to figure it out? Well, let's throw a break point somewhere in this code. In fact, let's throw a break point in right here. Let's see what happens when we call forEach on an observable. Let's walk through the chain and let's see what happens. So, you let me know Mark if I need to bump up this font size. Going to run it. Hit debugger. There's a bunch of stuff in here added by. Anybody know how to get rid of this? - There's a little arrow to the left, I think, yeah. - And on the right hand side, think I can do that? - Does that one work, the gray one? - Gray one, work? - Oh no actually. - No it doesn't. That's not bad. Oh I can break it out, there we go. So, let's step through, on the step over. And now I'm going to step in and see where we end up with. Now we're in that wrapper function. You remember that wrapper function? It's whole job is to sanitize any input. If we pass in functions, we'll turn it into observers. If we pass an observer we'll just thread it on through. So, what'll I do, well I passed in a function. So, we're going to wrap it inside of an observer. And now we're going to delegate to the actual definition of forEach that we passed into the constructor. And here we are inside of the observable. So, here we are inside of our forEach function. And we're going to call maps job. Right, maps is going to call forEach on the source. This is when she calls forEach on me. Right, so we're going to step into that. And once again we find ourselves in that outer wrapper function, it's job is just to make sure that all the output's sanitized et cetera et cetera. So, let's step into that. And now we're in the filtered observable. Right, because we mapped the result of a filtered observable. And so if we step into that, backed into public function. Also, I'll skip past that, and finally we find ourselves inside of the fromEvent observable. The observable that represents the event itself. So, now we create a handler, we step over, and we add. Now, notice, I've been saying this all along, but now we see why we only add an event listener when we call forEach. So, if we hadn't called forEach this code never would have run. We never would've hooked up this event handler. So, I'm going to step over it, and now we're going to return this object right? This subscription object that we call dispose on, right? Now, we're never going to call dispose in our code. So, this code is never actually going to run. So, I am going to run this code. And now I'm going to put a break point, inside of the event handler. So, now we've seen what happens when we're calling forEach. When the message is going this way. Now we're going to see what happens when the message turns around. So, he calls forEach on her, she calls forEach on me, and I call onNext on her, and she calls onNext on him. Now, data is traveling the other direction. So, soon as I click this button. Now we're back in. So, I step into the onNext function. Here we are, I've stepped into the onNext function. Sorry I did that before that was visible, but we went all the way from. Where is it, fromEvent. So, we started here at 56, and I stepped into this code, and we ended up inside of the filtered onNext function. So, I've onNext'd her, and now she is going to check to see if it passes the test. Which it may or may not, actually it might not. It does, great! And so, now she's going to call onNext on him. And since there's actually three steps in there. We got one more person involved, right? Now, we're in the map function. We first filtered and then we mapped. And now she calls onNext on him, he's the mapped observable. And this fellow, sorry what's your name? - Kateo - Kateo - Kadon - Caden? - Yes. - Okay, hopefully I'm close on that one, I'm sorry. He's going to, he's now call forEach on, so he's the filtered observable, right? And when we call, sorry you're the mapped observable, because we're calling the projection function, right? We're going to step into the projection function, and look where we are. Now we're back in our code, right? Now we're at the point where we're taking that item, that I've onNext'd through, and it's made it through the filter and now we're translating it. We're going to make it into a string. We're going to stick px on the end of it. I'm stepping through that, and we're going to onNext it, and finally we forward it to the very end, and we end up in our forEach function. Which has got a bunch of junk in it, because of js bin. But this is basically a forEach function that we wrote. So js bin does this so they can intercept the console log and put it into that pane that we saw earlier. But, if you look at our actual code here, it's just console log. And so we made it all the way out to where we called forEach, and we printed out the results. And so if I do this, we're going to see, some junk that's js bin related. But that's actually our answer. Right there. Okay? So the code drilled all the way up, and then we drilled all the way back down. So, hopefully now we have actually a pretty good idea of what's going on under the hood when we use an observable. There's not much to it. You're just forwarding along data. ForEach goes up, data goes down. Observable Take Function - I have a question on how would you wrap, you know, different things like object.observe, and mutation.observers. - Good question, well there's a galaxy of different callback api's and the browser's only adding more of them. In fact, we can actually play around with object observe, and wrap it as an observable. There is a very cool example, of doing just that. What's the fastest way to get this done? I've actually got an example up on github, of using object observe, an observable. Sorry, first of all, who's familiar with object observe? Not many people. So, there's a feature coming, in Javascript7 where you can walk up to any Javascript object and observe to see if its properties have changed. So, you can get a callback if somebody changes a property on a javascript object. And I believe. I actually have. Just happen to be running it right now, in Chrome. So, because I've got my future features. I forget what exactly what it's called. Expanded web features, flag turn on in Chrome, because I like to play around with these things. So, if I go to a console. We can find out if I've actually got this. Because, it's a cool api to try out. Object dot observe. So you can see it's already in Chrome, if you go and load your Chrome you'll turn on a particular flag. I think it's right here. Experiments. No, not in there. We can google it. I believe it's under about:flags. So if you go to about:flags in Chrome. Ah, here we go. Enable experimental web platform features. So for those who can't see that out there. Enable experimental web platform features. I believe that is the one. So if you enable that, and restart your Chrome, and you start typing in console, you should be able to see object dot observe. And we can check out how object dot observe works. I'm going to create an object. Does everybody see the code I just? I'm going to pump this font size up. Everybody see the code I just wrote there? So, I created a plain little object, I gave a single property, and then I called object observe and I passed in the object and then I passed in an event handler, that is going to fire whenever we change any properties on that object. That make sense? So, let's try it. I'm going to go x.name equals Ben. And it worked, see the console. We wrote out, and we actually get a list of changes. And notice it says the name that changed. It has the old value, which is jim, and I believe the object. Yeah basically the object is just the object that we passed into object observe. So, this gives us the old value and it tells us which property changed. So, that's kind of cool right? - Until you eat up all your memory with a bunch of observers yeah. - Well it could definitely be memory intensive to use object observe. And that is actually one of the criticisms of object observe. So, this is great for listening to one property. But what if you have a property with an object inside of it, and it's got a property with an object inside of it, and it's got a property with an object inside of it? You can just listen at the top level with object observe. You have to listen to every single object underneath. So, if you want to listen to a path changing, a series of properties and have change to any one of those series of properties, you need to recursively add object observe. And yes that does turn out to be expensive. Some mvc frameworks like this style of synchronizing models and views, and so they want to use object observe. But, there is probably a fair criticism to say, this is perhaps a heavy way of doing object observation. And some other mvc frameworks for example, use a different approach with is diffing. Angular uses a diffing approach. As soon as an event handler happens it take a look in an old copy of the model, and a new copy of the model and then goes through and compares them. That's very different than listening for fine grain changes. Well, to be fair object observe is somewhat coarse grain because, if I change 15 properties all at once, it will only call that callback once. And it will list in the array, all of the properties that changed. But that still doesn't solve the problem of multiple event handlers at every level of your object observe. I have an example online of using an observable, to do just that. So, it allows you to observe a path, and it uses observable, and actually it's a pretty cool code sample. But under the hood, basically what it does is it uses switch latest. Now let's say I set an object, instead of you know, my name jim. Is this big enough you think for folks at home to see? - Yeah. - Okay. So first of all how would I unobserve? Remember async api's usually have some way of saying no, I don't want anymore data anymore. Well, there's a handy object dot unobserve method. Now, unfortunately I can no longer unobserve this, because I didn't capture this handler. So, I can't even pass it back to unobserve. Which is a great example of why that's a very annoying api, right? This whole idea of like, oh I got to pass in the handler, but I got to remember the handler because I got to pass it back. It's the same as adding a listener to remove that listener. So, the question was, how would I wrap object observe as an observable? Right, so why don't we just do that? Because I hate using object observe on the observe api. I find using api's very annoying, because what I really like to use is observable, because then I can use things like takeUntil or takeOne for example. Let's say I want to listen for the next change to an object. One of the operators we didn't write right here, in what I think we should write, is take. Take right? We never wrote take. And we tried out take in our exercises very briefly. And what it does, take says, if she's the take observable. And, let's say she has a number, which is the maximum number of items she's going to take. If I onNext her one, two, and three, by the time I get, if her maximum number is three, she's going to say to him onCompleted after three. And more importantly you're also going to dispose of my subscription. Right, so you're going to say to me I don't want any more data, I've got all the data I need. And you're going to say to him, onCompleted. So, let's write that function. So, some of these implementations are a little complicated to do fully correctly, but I'm cheating a little bit here to give you the whole idea. There's a couple of things to call your attention to, like for example, when you call an onNext on an observer. If you're an observable and you call an observer's onNext it could throw. And so actually to be fully correct, you should be trying and patching, because if somebody's observer, if I call onNext on you and you throw, right? I should then turn around and call onError. I should be trapping errors, right? Because if nobody's around like with async code your code may not be on the stack at all. So, you can't just let an error propagate up with throw. It's observables job to catch errors and then push them with onError, right? If I'm being fully correct I would do that, we just don't have time, and it would complicate the code. So, we'll just focus on take. So, what does take have to do? Take has to return observable, that only onNext a certain number of items. So, the first thing we do obviously is we got to return an observable. And to create an observable all we have to do is define a forEach function. So, what's the is observable going to do? What's it going to do when you forEach over it? Well, I'm going to do this, because we need to make sure to always capture the source observable, right? That's what this line of code does. When you call take on an observable, that take observable needs to still have access to the source so that it can call forEach on me, so that I can send the data. So, we've trapped a reference to the source observable. And we're creating the take observable. So, just like every other example, or almost like every other example, in the previous example we did this. I'm actually going to start doing something that I know is wrong just to demonstrate what's going on. So, we're going to pass in, we're going to call forEach on the source, right, that's the very first thing. If he forEach's her she has to forEach me. So, we're going to call forEach on the source. Going to pass in the onNext handler. And when it receives a value, what's going to happen? Well, notice take needs a counter. We need to keep a counter inside of the take observable. Because after we delivered a certain number of onNext we want to stop delivering onNext. We want to say onCompleted. So, we're going to declare a counter to keep track of how many onNext's we've delivered inside of the take observable. How many onNext methods have we called, and we'll initialize it to zero. So, when an onNext comes through, well we want to first check to see if the counter. What I'm actually going to do is I'm actually just going to call, I'm going to forward on the onNext, so now she's onNext'ing him. She's forEach'ing me, I'm onNext'ing her, and she's onNext'ing him, and incrementing a counter. Okay. But what happens if the counter gets up to the number? What she would like to do is say to me, no more data, right, she wants to say, never send me another value, because I don't want to send him any more values. He only asked for three, so I'm not going to send any new. So how is she going to say to me no more data? How do you stop listening for items that you get out of an observable when you call forEach on it? - Unsubscribe. - Unsubscribe did I hear? Yeah, how do you unsubscribe? - You dispose. - You dispose of the subscription. Now we see that I'm just, she's actually just returning my subscription to you, but that's not good enough now. She needs to hold on to that subscription so that she can call dispose on it. So, we're going to assign the subscription to a variable. And she's still going to return that subscription object. But, she will call subscription.dispose as soon as that number of onNext she's delivered gets up to her maximum number. So, she'll tell me, no more data please. As soon as I give her, in this case, maybe three onNext. And, what else does she have to do? I'm going to add the typical boilerplate, which is that you're always forwarding the onError and the onCompleted messages. So, we forward along error. So, if I tell her error, she tells him error. And we forward along completions. But, what does she have to do? What's the last missing piece here? So, let's say she's a take three observable. She's only going to take three. You call forEach on her. Do me a favor, forEach. - ForEach. - She calls forEach on me. - forEach. - I call, I say onNext one. Right, you're take three, maximum you take is three. I call onNext one. - onNext one. - onNext two. -- onNext two. - onNext three. --onNext three. - onNext four. Well after the onNext three what are you supposed to say to him? - onCompleted. - Yes, and you have to tell me, don't give me any more data. So, we've only done one of those things here. We've got the section of code which says, you tell me don't give me anymore data. What's the last piece that we have to do? That we haven't done here. Alright, she has to tell you something very explicitly. - Dispose? - No, that's what we did right here. That's you saying no more data please. - You have to call in complete. - Right! How are you going to tell him that it's done? You have to call onCompleted. So if the counter gets up to that number, you need to tell him that no more data's ever coming ever. And as soon as she calls subscription.dispose I am not going to say onError, or onCompleted. As soon as you dispose of the subscription you're never getting another message from me again. And so now, we have take. Implementing a Better Object.observe Now let's see if we can take object observe. We did it with fromEvent. Can we take this new fangled es7 feature object observe, and turn it into an observable? I bet we can. So, instead of, in addition to fromEvent, what if we say observable.observations, right? Because it's plural, now instead of thinking of like events as this like add event listener, remove event listener, now we're thinking about events in terms of collections. Right, it's just another collection. It's the stream of observations we get from an object. So, what's it going to accept? Well it's going to accept the Javascript object. And what does it return? Yell it out. What will this return when you call object observations on an object, what kind of type? - Observable. - An observable! It's the same answer every time so far. Return a new observable, and when you forEach over this observable what do we want to have happen? What's the side effect that occurs? What's the thing that happens that causes us to start getting events pushed at us? What api are we going to call this new es7 api? - Object.observe. - Object.observe! So we call object.observe. So, we're going to create a handler, just like we did with fromEvent, look up here, we created a handler. I'm just going to copy this code, it's going to be faster. Instead of dom add event listener, it's object.observe pass in the object, and then the not necessarily grammatically correct object.unobserve handler. Notice a pattern? It's always the same thing. We hook up the thing that starts the push stream events coming to us, and then inside of the dispose method of the subscription we make sure that we never get any more events ever again. By unhooking, right? So, let's see if we can get this working. This is going to be cool. Got a missing semicolon somewhere, that's fine. Just going to leave this up here. What if for example we wanted to listen for a click, and then open a form, and then we wanted to listen to changes for an object, until somebody closed that form, and then maybe update the view. Because, that's what object observe is all about. It's about allowing you to listen to models, and then when they change updating views. That's how you know a change happened to the model, so you can synchronize those changes with your view. So, when a form opens, you can start listening to changes to an object, and then synchronizing them with your view. And when that form closes, you can stop listening to changes to that object. And that's exactly the thing we did before, with the auto complete box. When the search form opened, we listened for changes to the textbox, for search result sets, and as soon as it closed we stopped listening for changes. So, I'm going to listen for, I'm going to create an object called, person. I'm going to give it the name of jim. And we are going to listen for all the observations on person. And, since it's an observable, nothing's going to happen if I just create it, right? We have to do what in order to start getting the data out of that observable? - forEach. - Yeah, forEach absolutely. And what actually comes out is this changes array. It's the array of all the changes we made to the person. So, in case it wasn't clear before, if I change one property on the object, it's going to eventually asynchronously give me a change record. But if I change 15 properties in the object synchronously, one after the other, it will then asynchronously give me a change record detailing all 17 changing. So, that's more efficient that way, to wait until, it's going to be all the changes in a particular turn, and then onNext- and then deliver them, rather than just keep calling a callback every single time a property changes. That's how object.observe works. So change is actually gone be an array of every change that we make to this object. So, now that we're listening for the changes if we turn around and go person name equals don. Person age equals 23. We should, get only one onNext, of changes. So let's check it out with our little observable lite library. How do I clear this console, anybody know? Usually there's a little button. I don't see a run button either. Okay. Undefined is not a function, it's complaining about- - I think you use observable.observation - Ah, it's thank you. It's observable.observations, it's not object.observations. - And maybe should we call it fromObservations, since we have fromEvent kind of that parallel. - Yeah, sure, sure why not? Sounds fair, and I'm going to follow my own rules here, and do formatting. As soon as we have more than one verb on a noun, we put each on its own line. It's actually already showing the right results. So, if we clear it and I run it. Oh look, we got an object with two records here. Notice it says that, age change? It's hard to see from the formatting, but it gave us a record inside of that array that says the age changed? And it gave us a record inside of that array that said that the name changed. So we patched it on up, right? So, now we can actually make a form, like, we can actually make a text box where you can enter in the name. And a text box where you can enter in the age, and we can set that object. And then, we could sort of display, when a change was made to that object. Or, for example we could make an asynchronous network request to save the data. Right, does that make sense? So I got a forum with a text box for the name, a text box for the age. Every single time we change the name, or the age for that matter, we might want to kick off at asynchronous request to sync it up with the server and save it. Does that make sense? What would that look like I wonder. Let's try it, so I'm going to open up html. Now we're going to make it a save button, and we're going to add a name box. And an age box. And this is the type of thing. Now you're seeing what the benefit of observable is. It's not so much observable if you're just listening to one data source and your forEach'ing over it, and you're doing something. In practice, you're almost never doing that in user interfaces. When an event happens you're usually kicking off some sort of network request, right? Or doing something asynchronous. And if you're trying to do that with callbacks, you're in callback hell. Because now you've got to manage errors yourself, you've got to manage subscriptions yourself. But let's take this as a real example, okay. This is actually a very feasible example. Somebody's editing a form, and then, maybe we wait until after a certain batch of changes is finished, not every single time a property is changed for that matter. And then we go off and we make a network request to save it. And we're going to, I'm going to try that out right here. So, I'm going to stop using our observable lite library. Because for this example I want to actually be able to use the more advanced operators that we haven't defined yet like concatAll and mergeAll. I'm just going to open up a new one and we're going to jsbin this. And start from, and so I'm actually going to go, add library, add the rx library. And now in Javascript I'm just going to copy over our definition of observable.observations. But now I'm going to make a couple changes. So, up until now in our simple library, we used a constructor to create observables. But, rx comes with a helper function, for creating observables, that lets us write a little bit less boilerplate. So, instead of going new observable like we're doing here. Get rid of this html. Rx comes with an observable.create function. And it works pretty much identically, almost identically as in passing forEach to the constructor. The only difference is, instead of returning this object with a dispose method. Which is, if you think about it is always boilerplate. The only thing you're ever defining in here is the definition of dispose itself. It just allows you to just pass back, the definition of the dispose function. So, if I was to write this out the long way, you'd see that this was just the definition. And under the hood it wraps it in a subscription object for you. Does that make sense? So, observable.create takes the forEach and wraps it in an observable. And it takes whatever the dispose function you return, and it wraps that in a subscription. So, that's all, that's the only difference. It's just a little bit of a helper function, to let you write slightly less code. So, this should work as is, with, now we have rx in here, for observation. So x is the changes array that occurred to the person. And I'm going to go, now age equals 23 and we should see something in the console. Unexpected token because I'm not in six to five. I'll switch over to use six to five, and it'll understand my arrow function. And then we'll run, and it'll observable not defined. Just have to, now that I'm using rx we have to make sure to pull it out of the rx namespace here. So, now that I do that. Yes. So, we got out changes record, right? So, we swapped over to using the real rx, but I can tell you, you know, definition very very similar to what we saw earlier. So, now that we've got object.observations. Let's say we just want to listen to the next change to an object. And we don't want to have to like call unobserve, right, inside of our observe handler. You know that thing when you, imagine the code you'd have to write if you wanted to list only one dom event. You'd have to like, inside of your handler, call remove event listener. We don't have to do that. So, let's just listen for one observation, okay, using take one, I'm going to add in formatting here. I don't need the output window either. So, this isn't going to be too impressive looking, because we're going to see the exact same thing. And the difference is, under the hood, our handler is no longer attached to that object. So, if we make more changes to it they're just not going to show up. Our callback is not going to get called. So we have to listen for the next change to an object and never for another change to an object. Binding Between Views and Models So the thing we actually wanted to try out I believe, Unfortunately, jsbin keeps losing the toolbar so I can't add it but, I think if we paste it into another screen it'll come right up. So, let's keep our html toolbar open, and let's try and do that server example we talked about earlier. So, we're listening for changes to a form. We're going to have a form, and that's going to make changes to an object, and we're going to listen to changes to the object, and then make a change to the server. Does that make sense? So, I'm going to add a name and age tag. And I'm going to give them id's. Now, going to clear the console. Let's grab a reference to our name and id. Right, now we need to listen for changes, to name and age. And we want to update it on a Javascript object, a model if you will of a person. And the advantage of doing this, is that if we make all these changes to this model person, we might have six or seven different windows in our view. All bound to the same model. So instead, in view, in addition to a view showing information about a person, you might have another pane on your view showing the location of that person. So, in other words the same model of an application, can be viewed in many many different ways. And so, by using one object to sync up and to store the state of that object, we can have many many different views listening for changes to that object. And you only have to change the object in one place. And all of the views, reflect the difference in that object because they're all listening using object observe. Does that make sense? So, I'm going to listen for changes to age. Now, remember when somebody's asking about change, the on change event. This is the ideal case for the on change event. We don't want to make this change after every key press. We only want to make this change after somebody tabs away from the text box. Then and only then do we want to update the person object. So, I'm going to create person object which I'm just going to keep in memory here. And I'm going to take, just going to listen for. What I'll do is I will listen for changes to the person object. And update the view. So, if we ever change the person object directly, we're going to update the text box and the age of that person. So, I'm going to use object observations on the person. And we are going to change the text box of name if changes includes the property that name has changed. Right, that's one thing we can do. Well before we do that, let's just do it the simplest way, Which is always update. Always update, with the current value. This is less efficient, because, maybe I've just changed the name on the model object, and every single time any change occurs on the model object we're going to update all the fields on the dom. This is the less efficient approach, but it works. Now look, it's already filling in stuff here. Name.value, I'm not sure why the name isn't making it in there. Ah, because it's waiting for the first change to do it. So, if I go person dot, it should actually get the name if I initialize it with the name. So, I'm not sure why our name property is not getting filled in. Anybody have any idea out there on the web? We've defined name and we've pulled it in with getElemetnById just like age. And every single change in object observation is being picked up and written to the dom. So, I'm not sure why I'm not getting the name property here. - Someone said age is a string value. I don't know if that matters. - Is name an observed word? - Should be. - Html. - In the html yes, oh well not inside of a string, I wouldn't expect so, but maybe I think you're right. Because that's the only possibility I can think of. - When you're defining that person you have jim and an age tim, or is that just because. - He's ten years old. - Yeah that's definitely a typo. Okay, I change it everywhere to be first name. - Now it's working. Try rerunning it. - Hey, okay cool. So, now we can listen for changes to a model and update our view, right? Another thing we can do, but now notice we also haven't gone the other way right? We're not going both ways, which would be kind of nice you know, if you change a text box we'd ideally like to update the model right? But here, we might be able to have many different views of the same information, right? But then, only one place in the application, where we change that model, and then we update all of the views. Does that make sense? So, we have this person object in memory. And many many views of it. And we change the person object in only one location and update it everywhere. But another thing we might want to do. We haven't gone the other way yet. We haven't set it up so that if you change the text box you then change the object. The nice thing is we're never getting into an infinite loop, right? Where I change the text box and then I change a property to the same thing it already is, and then it causes an object observe, because, the reason we're not getting into that infinite loop is for one thing object observe won't fire if the property has the exact same name. So, we don't have to worry about that. So, let's see if we can go the other way around. Now in practice you'd use an mvc framework to do this. But, this is exactly what an mvc framework will do under the hood if it uses object.observe, right? So, to listen for name, now I'm going to convert the first name change in forEach. And I don't even care what I get. I'm just going to sync up the person.firstname with value. And I'm going to do the exact same thing with person.age. And let's go be good and do parseInt on that. Okay so now we should be syncing up changes both ways. So, I'm not sure quite how to figure that out but I think if we log when we get an object.observations then we'll know if that's worked successfully. So we'll console log changes, and now we're only going to get a log, when we actually change the person object. So, if I change this form, we should update the person object if everything's working correctly we should get an object.observations callback. We should get these changes printed out to the console. So, I'll run. We've already got one, because I've changed one in here explicitly. Now, let's see if we change this to. I'm going to hide our html pane because we don't need it anymore. To clear the console. No not yet. - You could just tab away right? - Ah good call! Yes I do, but that was not the source of the problem. So, we should be listening to change on, age, I forget to put age instead of first name there. So, let's see if that solves the problem. Yep! Okay, so now every single time we change the form we're changing our model object right? And hopefully vice versa as well. Let's see. So I'm going to use a set time out. And I'm going to change our person name to john and I'm going to do it after five seconds. So we're going to patiently await here. And hopefully if we do this we should see the text box name change. Oh, we didn't see a change, but we did see a console log happen, so that's a little bit odd. Something definitely happened, but we didn't update the form. So let's run it again. So, we got a console log. Then after five seconds we should see another console log. Great, but we're not updating the name. So, why is that? So, let's go back to our, observations, where we're forEach'ing over observations. - Do you have the first name? Person dot first name? - Yes it does, thank you. Collaborative coding I love it. So run it, and we wait. Time elapses, and bam! So now we've got code syncing both ways from our view to our model, but also from our model back out to our view. And that's great because if any of the views change the model they only have to change it in one place. There's one model object, and then you might have n different views of the same model. All the changes are going to get propagated back to those views. Syncing Data with the Server What were we going to try and do originally? We were going to also try and set it up so if we change the model we would write the data to the server. Right, we want to just have it implicitly syncing back to the server every single time somebody changes the model So object observe can be our way of doing that. So, instead of directly forEach'ing over this I'm going to create an intermediate observable, person changes. And then we are going to subscribe, or forEach over person changes. But now I also have this observable that I can forEach over, that I can use as part of a larger observable expression. So, now we're going to create the stream of data that we want from the stream of data that we have. So, I'm going to create a fake function here. Just a function that, you know, save to database. And it's going to accept and, actually it's not going to do anything because it's just going to use the person object every single time. It's going to save the person object to the database. Now it's not really going to do that because I don't have any database open. But, we know that operation would be asynchronous right? It would return an observable. So, I can kind of fake it, because observable has a little timer method. I believe it's time out actually. And you can just give it a time and now what does it do? It'll fire, so under the hood- Yes, a question? - How can you wait until both name and age is changed before reflecting that on the model? - Well the good news is, well actually that depends, what do you mean? Because if you change first name and age, right? All synchronously, then you're only going to get one object observe callback. But, if we do it through the form, in other words we change name, and then we change age, then yes we'll actually get two object observe callbacks. So, we'll write to the database every single time. So, that's a good question. What if we only want to do something after every field is changed? That's kind of a tricky one? Any ideas? So, we only want to make a change, after every single- I don't know how realistic that case is, because in practice you probably only want to either wait for a save to occur, right that's somebody explicitly clicking save. Or you probably want to sync after every single field changes. But it's kind of a fun thought experiment. You guys already know how to do it actually. - It's very similar doing an autocomplete isn't it? - Kind of similar yes. But you know the operator, exactly the operator to use for this case. Let's imagine that we could create a stream of name changes and a separate stream of age changes. Right now we have object.observations, which fires regardless of whether name or age fires. Alright, it's one stream. But can we split it into two streams? One which only fires if the name changes, and the other one which only fires if the age changes? - Could you use zip? - Yes you could, if you took that object.observations and you split it into two. One where it was only name changes, and one where it was only age changes. And you only wanted to save the database when they change the name and age, you could use zip. Because then you'd zip those two things together and then it would only fire after, when you got one from name and one from age. Now there's actually a better operator to use though. So, zip would work but it would be a little bit weird. Because let's say I go to name, and then I tab away, and then I change name again, and then I tab away, and then I change name again, and then I tab away. What zip will do is in order to onNext to a value it needs to have both one from both name and one from age. So, it actually buffers, it buffers up all the names. So, then what would happen is if you change name three times, and then if you change age once, it would save the database. And then you change age again, it would save the database, you change age again, it would save the database, and then the left hand side, all the items would be removed and it would be stopped. So, zip is great when you're working with observable for those cases where you have two network requests. And usually two observables of one, or and observables of one and you want to wait until they all finish. But for cases like this, sorry? - Merge all? - Merge all only works on observables of observables. And with this case we would have two observables, in our hands at the same time. We'd have the stream of name changes, and the stream of age changes. But there's actually a perfect operator in rx designed for exactly this type of thing. Where we want to change, we want to save to the database regardless, would either of these fields change. But if you change name twice, well let me see if that's actually the correct operator. See, this is sort of a weird case, because it's like only when you change both of these operators. Technically, probably what you'd want to do, is you'd want to wait until name changed, and then age changed before saving. And then, if either name or age changed in the future you'd probably want to save right? It would be very weird to wait for them to change all at, it's just a weird case. We could probably make it work, it's just not realistic. So, I'd rather focus on the realistic cases. And my guess is the most realistic case is if any of these things change, you probably want to save to the database. - Updating password would be a good example, if you need both fields changed. - Right, because you have your password in the confirm.. - But even in the confirm would you change it as soon as they synced up their confirm? Like to me that would actually be a very bad UX, because I would actually want to take a very explicit action to change my password. I wouldn't want to just happen to enter something into a text box. - You'd want validation. - You'd want validation, et cetera, right. But one thing that's very, I want to bring your attention to because it's a very common pattern in UI's is. It's generally speaking, if somebody takes an explicit action to save ______save. Or if any field changes, you want to save. Those are the two actions. Question in the back. - What's the point of rx.subject? - So, a subject is a topic I was hoping to get to today but I'm not going to be able to get to. That's okay, because for the most part rx developers don't need to use them. So, I don't know how deep I want to get into that. We can leave that for another time or you can check that out in the documentation. But in practice you shouldn't find yourself using subjects very much, so I don't want to focus on it too much. So, I'm a little reluctant to answer that question. It's a more advanced topic, and as you can see we've done a heck of a lot without ever using a subject. So, that's kind of why I'm not super eager to introduce the topic right now, if that's okay. But we can always handle that question by email after the fact or something like that. Maybe we can explain that. - Or in your next round when you come back. - Or in my next round when I come back we do an advanced rx, yeah. So, I do want to introduce you to one more operator because it is very useful for exactly this case. And it's like the cousin of zip. So, how does zip work. Zip, remember the zipper, where you need to zip up and you need a tooth on either side, otherwise the zipper gets stuck? Combine latest is very close to zip, except the way it works is instead of waiting for an item from each side in order to advance, if a new item comes in either of the left hand side or the right hand side, it immediately- As long as it has at least one item from the left hand side and one item from the right hand side, then if another item comes down from the left hand side instead of waiting for a corresponding item from the right hand side, it just onNext with the last item that it got from the right hand side. So, the very first time it needs to wait for at least one item on the left and at least one item on the right. So, combine latest here would actually be close to what we need. Imagine we create a stream of names, and a stream of age changes, and then we did a combine latest on it. Well, we wouldn't save to the database until somebody changed both name and age. But then after that if they changed either name or age, we would send off a save to the database. Because what happens is the function gets called with the last value of name if you change age. Or if you changed name it gets called with the last value of age. Hence the term combine latest. So, the latest value that comes on the left hand side, it just takes the old value on the right hand side, and immediately calls, whereas zip would wait for another value to arrive on the right hand side before calling the function. Combine latest. So, that's very useful when you tend to work with UI's. And you're doing that type of really reactive application, where every single time a field changes you want to go and sync up with the database. So, why don't we do something a little bit more, well to me the most realistic case here is when any any field changes we want to save to the database right? It's either that or it's save button. And so, let's go with the other kind where any field is changing and we want to save to the database. So, we're going to make a fake function and all it's going to do is just create an observable that takes 500 milliseconds to complete right, and then returns nothing. I believe, I'm not sure if it returns an onNext with like null in it and then nothing or just is empty. We'll find out one way or the other. So, if I call this save to db function. And then I forEach over it, we'll see if it actually has a value in here, or it's just an empty observable. Now it's totally possible that time out could just be an empty observable. One that looked like this. Nothing wrong with an observable like that. In fact, if I was to model an animation of an observer, which it's too bad we didn't get to in this class, but it would be fun. Next time you want to try animations out, and you want to coordinate them. Actually I definitely want to talk a little bit about that, because there's some very cool stuff, right? I'll come back to that. So, an observable. If I was to model an animation as observable, sometimes an observable just doesn't really have any data. It's just a duration of time, and then it completes. An animation is a good example of that, right? It's just sort of like, you would model most animations as an observable that just go on until they're done. You just need to know that the animation's done. Now another way of handling that would be like imagine you were animating a box from left to right, x left to right. You could keep onNext'ing like the individual x and y positions as the animation went on, but in practice that's probably not very efficient number one, number one nowadays we typically do animations using things like CSS Transitions. And the way CSS Transitions work is the don't keep calling you with their intermediary values, because it's not very fast, right? They just call you, they can usually call you to tell you when they're done though. And so that's why it's very convenient to model a CSS Transition as an observable. Observables as Animations But I want to call your attention to those three flattening functions we talked about earlier. Does everybody, tell me what the three flattening functions are? - mergeAll, concatAll. SwitchLatest. - Right, so let's say, and this happens all the time in user interfaces. I start an animation, and then another animation comes along. What are the three options I have when it comes to that previous? So, let's actually imagine spatially. Can we get a whiteboard going? That would be super helpful right now. We're about to talk about how broadly applicable these three flattening patterns are, mergeAll, concatAll and switchLatest. And now we actually can use them to coordinate animations. So, let's just do a quick review of how these flattening patterns work. So, I've got an observable of observables here. Here's the beginning of the outer observable, and within we have an inner observable, another inner observable, and an inner observable. MergeAll is like lanes merging on a highway. So we get five, six, nine, and 77. concatAll, each one of these shifts to the end because we always handle them sequentially, top to bottom, left to right. So we get this very long time period. Because if we don't start forEach'ing over and therefore generating these results until this is finished. And that's why we shift it over here, and so on and so forth. We get this long observable. And finally, switchLatest, where we begin listening to an observable, and as soon as another observable comes along we switch to it. And actually I got this wrong because there's no five here. This is the right observable for switchLatest. Because, we started listening to an observable, and before the second one came along, this one came long. Sorry I, my time's a little bit off here, but the point is this is the time axis. And so another observable came along before you even got five. Which means we only ended up with nine. So, if I was to visualize what these operators would do to two animations. So, let's say we started an animation that animated a box from this zero zero to 100. And all it looks like, is it literally just looks like this. It looks like an empty observable, because all it does is it starts animation and then as soon as that animation's done it says onCompleted. So there's no data inside. So, if we model our observables as animations, we can do them this way. And then, by the time this reaches here, one second will have elapsed. So, let's say this is one second of duration right here. 1,000 milliseconds, okay? To move 100 pixels. So, let's say we start the animation and halfway through we start an animation that moves the y position of the box to 100. So, halfway through we start that animation. So, that's actually an observable of observables. The first observable, that can be the first inner observable that comes along, starts the animation going this way, but then half a second later another inner observable comes along that starts pushing the y up this way. Now I'm going to draw a path for this box, and you guys are going to tell me which flattening pattern accomplishes that path. It got halfway, and it started moving up, at the same time as it's completing moving this way. And then eventually it got to the top. - mergeAll. - mergeAll absolutely! Right, because we're listening to the current observable, and as soon as the next observable comes alone, we immediately start playing it. And now we're pushing the box x and y up at the same time. Until finally x animation completes, but we still go half a second left on the y animation, and so we get that structure right there. When you're coordinating animations, you can either play them at the same time, you can do the next thing which is... Which flattening pattern would this be? - Concat. - concatAll! We fully finish the x animation, before we even start the y animation, and so we get that. And, obviously by process of elimination, we get this. Technically it's going to look a little more like this. We got halfway through the x animation and we stopped. And then we moved all the way up the y animation. When you dispose of the observable you freeze the animation just right in place. Now you can make your decision about what that does when you dispose of an observable, when you're designing the api that wraps an animation as an observable, you can decide that when somebody disposes of it you could snap it back to its original position. You could also decide that you could snap it to its end position. But that's how I would do it, by default anyway, is you dispose of the observable that represents that animation, it just freezes in place. So, that's why. This is a nice visual explanation of how flattening patterns work and how you can use them to coordinate animations. Right? If you treat all your animations as observables, you can coordinate them just using these three flattening patterns. Does that make sense? Okay, so, I don't know how much more we can cover today. Well I think we're going to cover the one last thing, which is saving to database. Which is every single time that model changes we're going to save to the database effectively. But I want to ask you guys a question. So, before we get, turn the projector back on. I want to get your intuition about it. So, we've got this stream of changes to the object right? And we're going to map it, because this is always what happens, we map it into attempts to save the database. So, for every single time we get a change to the object we are going to issue a save to the database command which is going to return an observable. So, right off the top, how many deep is that collection? If for every change to the object we create an observable which save the current state to the database? - Two. Two, right, because for every little of those change record, those arrays of change record, we're substituting in an observable that will cause us to save the current state to the database. Which flattening pattern would we use to flatten it? What's the right behavior? - switchLatest. - Why switchLatest? - Because you don't need the older data. -Why would you want to keep trying to save old data right? You wouldn't, you'd want to save the latest data, absolutely. So, not really that surprising in user interfaces mostly you want to do the latest thing. So, I'm not necessarily going to type that all out now, because every single example I've shown you up to this point has followed the exact same pattern. Let's map over some event, let's issue an asynchronous request, we'll take a look and we'll see that sure enough we've created two dimensional collection and then we'll choose the right flattening pattern, which on the UI 90% of the time is switchLatest. But, I've just shown you, that's for asynchronous requests. There's that last step, which unfortunately I couldn't get to today. Which is, fine now I've saved to the database, maybe I want to show some animation that indicates that's complete. Maybe want to fade a dialogue box in, that says save. Then that's one more asynchronous action to the chain. And you can bet that'll involve going to a three dimensional observable. And then you want to decide, what flattening pattern to use for that third level. Maybe it's switchLatest or maybe you want to smoothly show and hide animations one after the other. When you're doing animations switchLatest isn't always the best pattern. Do you want to start an animation, and then flicker and switch to another animation. I can tell you Netflix, for our Xbox one application wrote a drop down list, and we used the exact same technique that you saw there. But here was one extra step, was that when we showed the results we didn't just flick them in, we animated them smoothly fading in coming in from the right hand side. And then as soon as you click the key we faded it out to the left hand side. And then when the search results came in we faded it back out from the right hand side, and so on and so forth. What we definitely didn't want to do. Is start fading that animation in from the right hand side and then have you click, and then immediately have it jerk and then fade out from the left hand side. You see what I'm saying? So, maybe you'll want to use concatAll to make sure you're animations are smooth. So, I'm sorry we didn't get to animations, asynchronous request and events, but it's the same thing again and again. You're going to map over a stream, you're going to pick the right flattening strategy, that's the hard part. You learned how to flatten streams with arrays. The only question that now you have to deal with when you deal with asynchronous programming. We've really summed up the challenges involved in asynchronous programming to one question. Which flatten do I use? What's the right way to coordinate this concurrency? That's what we doing here. We're coordinating concurrency, things happening at the same time. That's all it is and it turns out there's only three common patterns. So, when you look at asynchronous programming from the right level. When you take enough of a step back and look at it, sure enough it's actually really simple. But if you walk all the way up to it and look at these little callbacks, and these handles and these, all the sudden it seems impossibly complex. And so most problems are really about, zooming out and finding the right level of abstraction to tackle them. And I suggest to you guys, that events as collections is the right level of abstraction, to tackle asynchronous problems in user interfaces. And so now it's up to you to go out and master these skills. Drill them, try them in real applications, because I can tell you that everybody at Netflix who learns this technology, very often they come up to me within two or three months, because that's often how long it takes of thinking about it, playing with it, and say. I've completely changed the way that I write user interfaces. So, I hope the same for you. Thanks very much. Course author Jafar Husain Jafar Husain is the Cross-Team Technical Lead for the Netflix UI's. He is the architect of Netflix's UI data platform, and specializes in building reactive, event-driven systems. Course info LevelIntermediate Rating (55) My rating Duration9h 39m Released6 Apr 2018 Share course