What do you want to learn? Leverged jhuang@tampa.cgsinc.com Skip to main content Pluralsight uses cookies.Learn more about your privacy Sweet.js: Get Started by Aaron Powell This course provides an introduction to Sweet.js, a macro engine on top of JavaScript which allows you to extend the language as you desire. Start CourseBookmarkAdd to Channel Table of contents Description Transcript Exercise files Discussion Learning Check Recommended Overview of Sweet.js Introduction Welcome to the Pluralsight course Sweet.js: Get Started. In this module we'll have an overview of what Sweet.js is and why it could be useful in the projects that you're building. First off, what is Sweet.js? Compile to languages are becoming a lot more popular with Javascript. These are often referred to as transpilation languages or use of them is done by a transpile. Some popular variants of these are CoffeeScript and TypeScript, which are different languages that can compile down into resulting Javascript. The main reason that these types of languages are becoming popular is that they can be used to address issues or at least perceived issues with the Javascript programming language and different communities of Javascript developers have different approaches to the way that they think these problems should be solved. Sweet.js, on the other hand, is not a full compile to language like TypeScript or CoffeeScript could be considered. Instead, it gives you the building blocks to make your own specified languages for the problems that you try to solve. These are done through the use of hygienic macros. Under the hood Sweet.js is still just plain old Javascript. You just write your normal Javascript as you would do as though you were not using a transpile. The only time that Sweet.js would come in is when you actually want to use a macro with inside of what you're actually building and to do something specific to the problem you're trying to solve. Sweet.js was initially started as an open-source project by a Mozilla intern and it is still sponsored by them to a certain degree. A Sweet.js is available as a Node.js package, which we'll be using throughout the rest of this course. What is Sweet.js? First off I want to talk a bit about other transpilers and why Sweet.js could be different to them. Let's start by talking about CoffeeScript. I'm not going to go too deep into CoffeeScript as a programming language. If you're interested in CoffeeScript itself, I recommend you check out the Pluralsight courses on CoffeeScript. What CoffeeScript is is a completely new programming language that sits on top of Javascript, so the resulting output is Javascript that you would be used to reading and writing if you were just using plain old Javascript, but instead you have an entire new programming language that you start on top of. This language is very familiar to people coming from a Ruby background and it introduces a lot of new programming concepts, so things such as the fat arrow syntax or the class syntax are things that have become very popular with CoffeeScript developers, as these new language features that they have introduced. Some other things that CoffeeScript does is it introduces some changes to programming language syntax that Javascript developers might be familiar with, so in CoffeeScript we only have the concept of a double equality statement, whereas Javascript has both a double and a triple equality if you're doing type unsafe or type safe equality comparisons. CoffeeScript removes this feature from the language and, to an extent, changes the way the symantics work. As a Javascript developer you might be used to double equals, meaning a type and safe comparison, whereas in CoffeeScript you don't have that concept. It also does some other things through some of the language features it introduced, such as changing the way the lexical context is managed through the this value, so functions can inherit the this value of their parent functions. TypeScript is another good example of a transpilation language, but instead of being an entirely new programming language that is very different from the programming language of Javascript, TypeScript is acting more as an extension on top of Javascript. It too brings a whole series of new programming language features, such as static typing or it also brings in some of the features that available in CoffeeScript, things like the fat arrow syntax or classes. These are both provided by the TypeScript compiler, but they're done using a different syntax. These two examples of transpiler languages gives us an idea that we've got this eco system of being able to address problems that we either perceive or are actually with the Javascript programming language, and to be able to add new features, but we do this by bringing in this whole set of other features that we may not be interested in. What Are Macros? Sweet.js, on the other hand, brings in the concept of hygienic macros from other programming languages, such as Scheme and Rust to Javascript to allow you to start building on top of the Javascript programming language, and make the specific adjustments to the language that are specific to the problem spaces that you're working with, but not necessarily bring in the entire weight of an entirely new programing language, as you would do if you were using a more feature-based and language-based transpiler, such as CoffeeScript or TypeScript. What are macros? Macros are the building blocks that we have for Sweet.js, so it's a good idea to start having a look at exactly what they _____ to be, and how they're going to make working with Sweet.js potentially a better option than some of these other compile languages. Macros give you snippets in the programming language that allow you to augment the way the language works. Rather than just writing functions to continuously hide away some syntax that you might not like or to perform a function or perform an operation that you continuously do within your programming, you might want to do this with just a small snippet that could be implemented in line. This could be implementing an entirely new feature into our programming language or can also be used to override existing syntax. By overriding existing syntax with macros we can change the way something works. Say we don't like the fact that we have a double and triple equality statement within Javascript. We always wanted to be a type safe equality check, so we could override the syntax for double equality statement to make it produce a triple equality statement in the generated output for the Sweet.js compiler, and that means that it might be a lot more approachable to someone coming from a non-Javascript development background that isn't familiar with the fact that there are two different kinds of equality that can happen. By being able to produce snippets that will augment our language by introducing something entirely new or to override a syntax that we may not like or we might want to work differently with inside of our problem spaces, we could use this to build a programming language that is much more specifically designed for the problem space that we're working within. You can almost consider this as a domain-specific language because that's kind of what we're building, is a language that is specific to the domain problems that we're solving, but rather than introducing an entirely new programming language, such as we would with something like CoffeeScript, we're still using the base language of Javascript, we're just adding the small changes that are required to make it work better for our scenarios. How Can Macros Be Hygienic? When I was talking about Sweet.js a few slides ago I mentioned that the language introduces hygienic macros, so what specifically makes a macro hygienic? Let's say we have this macro here. This is just some pseudo code of a make believe programming language, it's not actually Sweet.js syntax. The way that it works in our pseudo code here is we define that we've created our macro with the #macro syntax. We then give the macro a name or a construct of the way that we expect it to work, so inc and then some parentheses, an i, and then we close the parentheses, and then the output of that is a particular template. In this case, where I define a new variable called x, we're assigning a value of 0 and then we're incrementing the value of i. Here's how we could be using our pseudo coded macro from our make believe programming language. We create a value called x, we assigned it the value of 1, and then we call the increment macro. This goes through a macro compiler and becomes the following output, so you might see that there is a potential problem that's happening here. Let's have a look at it in another way. What we've done is we've created a macro that is going to change our syntax. It's going to introduce something new. In this case, it introduced a new variable, and that variable is called x, but what if we'd previously defined a variable called x. In the case of our simple example that we had there, we hadn't defined a variable called x, so we've introduced a variable called x, but we already defined it, so we've had a change to our code that was something that was completely unexpected. We didn't expect it to be introducing something that's clashing with existing code that we've got. Hygienic macros aim to solve this problem by doing some intelligent renaming of the syntax that they are introducing, so that it doesn't collide with anything that's existing with inside our code. Let's go back to _____ looking at our example and assume that the macro engine that we're passing through this time is a hygienic macro engine instead of just a plain macro engine. We've got the same code that we had it before. We've defined our variable called x, we've assigned it the value of 1, and then we've passed through our hygienic macro engine and this time, instead of just using the x as it was originally defined in the macro up on the first line, instead that variable has been renamed to be something that should be a lot more unique for the scope that it's working in. In this pseudo code example we've just appended the name of the macro, which is inc, and a $ to the variable. We could assume that it might also include a numerical counter, so that if we use this macro multiple times the counter increases, so we don't collide with previous usages of this macro, but again, this was more about a simple pseudo code example of how hygienic macro could be different to a standard macro, and this is the concept that Sweet.js does. It will be introducing new variable names over the top of the variables that we initially design to make sure that we don't have scope collisions or we accidentally redefine variables that we've already defined within side of our programming language. There are ways that we can tell the Sweet.js compiler that we don't want it to be introducing these more intelligent renaming. Instead, we want it to make the names a lot more friendly of the variables that it's generating, but this can also lead to problems, as it doesn't necessarily pick out every usage of the previously defined variables. Why Use Sweet.js? Why would we want to use Sweet.js then? At this call Sweet.js is just plain old Javascript. If you don't opt in to using any particular macros, you don't actually have any differences to just writing Javascript. The Sweet.js compiler will take your plain Javascript, read it in, trap it in to transformer, find that there's nothing to transform, and give you back the original Javascript. This means that we can add Sweet.js to our programs without having to take full weights on top of another programming language or take dependencies off things that we'd not necessarily go to use. By introducing a hygienic macro syntax we're able to produce augmentations to the language that we're working with, in this case Javascript, that can transform for the specific scenarios that our program has. Maybe we're building something that has a lot of complex math in it and we want to have a nicer way that we can represent some of the mathematical operations that we're performing without having to continuously call out to external functions. We want to have it on a single line, so we could read that as a mathematical equation, rather than a mathematical equation combined with live remethods. Finally, what Sweet.js allows us to do is composition of a programming language instead of inheritance, whereas something like CoffeeScript or TypeScript gives us this very full fictured programming language. There might be only a couple features inside of it that we want to use. Maybe we're only interested in the fat arrow syntax or class-based programming. We're not interested in static typing if we were using TypeScript. We might only want one or two features from these other languages, so rather than taking this large dependency on this programming language that we don't really need, we can start using macros to build up the specific scenarios that we want to solve. We can also start using macros that other people have defined by bringing them in through the node.js package manager and including them into our program, so we don't necessarily have to build a brand new programming language and a whole new series of macros every single time we want to use it. We can take the advantages that other people are also trying to solve the same problems that we want to solve, so we can find people that are also working in the same problem domain that we have and use their knowledge as well to combine and make a program language that's better solved for both of our domains. Macros vs. Functions So Why would we want to use macros? The first question you might think of when wondering why a macro language might be interesting is, why don't we just use standard functions? First off, the advantage of using macros is that we can inline the problems that we're solving, so rather than having something that keeps calling out to an external function, so that we have this augmented call stack, particularly difficult to the one we're starting to read complex call stacks inside of our program just from an initial construction level, we might not want to keep stepping into these functions to work out what does that function do or what does the next function with inside of that one do? Instead, we can pull these up to a single line statement that we use with inside of our program and then at the compile time we generate those multiple depths of functions, so the initial way that we would write our program is a lot easier to read. We can also add new syntax, so there might be problems that we don't have any ways to natively solve with inside of our programming language. Say we wanted to work with promises, which is something that's very popular in Javascript at the moment. Well to do that we have all these functions that return and then we call it the then method on them and then we pass in _____ as the continuation, so on and so forth. What if we wanted to change that to look more like just a procedural programming language where we have some kind of a syntax that will take care of the fact that we're using a promise library. This is something that we could introduce using a macro, but it's very difficult to introduce in any other approach, but we're not actually changing the programming language we're working with. Finally, macros can allow us to polyfill or _____ prolyfil new features in a language, so you might be familiar with polyfilling. It's the idea of adding a feature to something that doesn't exist, so polyfilling is fairly commonly used if you want to add features to a browser that might not support them. Prolyfilling on the other hand is prototyping a new proposed feature and being able to test that out in the browser or in whatever runtime you're working with, so we could use a macro to define the fat arrow syntax, and this would then look exactly like what it would look with inside of Javascript, and we could propose that as a new language feature and say hey, this is how I think it should work, and this is how I think it would look. Because we're not introducing an entire new programming language on top of it, we're just augmenting a small portion by introducing this new feature, we can see how that looks with inside the larger eco system of the programs that we might be working with. These are some of the reasons why macros can be more beneficial than just functions on top of the programming language that we're working with. Summary So in conclusion, we've had a look that Sweet.js is not necessarily a complete language replacement like some other transpilers that we find out there. Some examples I used were CoffeeScript and TypeScript as examples of languages that have a lot more features that you may or may not require in the programs that you're building. Sweet.js introduces the concept of hygienic macros. We've had a look at what makes a macro different from a hygienic macro by the use of intelligent renaming to make sure that we don't have scope collisions and the fact that macros can be used to introduce specialized programming language features that might only exist for the problem domains that we're working with, but by having these features inside our language they can make it easier to solve the problems that we are working with inside of our programs. Lastly, we talked about how macros can be modular. Instead of taking this large extended programing language and having all the features, we can pick and choose the features that are specific for what we want to do, and be able to build a program language that works better for the kinds of problems that we're solving. Writing Your First Macro Macro Syntax Welcome back to Sweet.js Get Started. In this module we're looking at how to write some simple macros in Sweet.js and just introducing the constructs that we use to build macros within Sweet.js. The first kinds of macros that I want to talk about are rule-based macros. There's another kind of macros that we'll be looking at later on in this module called case-based macros, but we'll start with the basic constructs of rule-based macros, as they're generally the simplest types of macros that you'll be using within Sweet.js. They're also incredibly powerful for what you get with them and they could be used for a lot of the common scenarios we're building with macros is Sweet.js. This is the syntax for building a macro with Sweet.js. The first thing we do is we use the macro keyword to say that this is going to be a Sweet.js macro. This is one of the few places that Sweet.js introduces new syntax to the Javascript programming language. It also introduces another keyword that we'll look at in another module, but macro is the most common one that you'll be working with. This is nice compared to some of the other transpilation languages, as all the new syntax that has been introduced is easily identifiable by the first new keyword that we saw, which is macro. After the macro we need to give it a name. The name is what we will use with inside of our code to invoke the particular macro and say to the Sweet.js compiler, hey we want to use this macro at this particular location. Because this is a rule-based macro inside of the parentheses we start with the word rule. This is to identify the type of macro that we're working with and could be used with either rule or case. As I said, we'll look at case a bit later. Inside the parentheses for the rule we have to define a pattern. This is what Sweet.js will be matching on when it's creating the macro itself, so this is how we can make sure that the particular rule that we're using is matched for the particular scenario that we're wanting it for, so we can have a different pattern to look the way that we want to look for the usage scenarios. Finally, we have a template. This is what is output by Sweet.js in the replacement of the macro itself. This is what will be resulting in our Javascript code and, in a rule-based macro, what we put with inside the curly brackets for the template, is exactly what will be inserted in line for the macro usage, obviously with hygiene taken into account. Let's have a look at actually writing a simple rule-based macro. Demo: Writing a Simple Macro Here I've got my text editor open and on the left hand side I've got my simple.sjs file. Sjs is just a common convention to indicate that this particular file is a Sweet.js file as opposed to a plain Javascript file, which I have open on the right. The right hand side will be the compiled output of executing the Sweet.js compiler against our simple macro. I won't be looking into actually into how to use the Sweet.js compiler in this module, we'll leave that for a later one, as I want to focus specifically on how to create our macros, and what happens when they are output as plain Javascript in the end. The first thing I need to do is define my simple macro. I'm going to use what's referred to as an ID macro, which is a simple macro that will just return the value that's directly passed into it. First off I create my macro and I give it a name, in this case id, because it's an id macro, and then I need to define that it's a rule-based macro and give it a pattern. For this macro, because it's an id macro I want to just take some simple input, so in this case I've defined an input that is independent called $x. This can be defined in whatever variable name you want to call it, so $x represents a variable, something that is captured by the Sweet.js compile and available then with inside of our template. For the template I just want to return the value that is being passed in, so let's have a _____ for that and pass that through the Sweet.js compiler. You will notice on the right hand side that after our build is completed we've gotten our output. That's because we haven't actually used this macro, we've just defined that we've got a macro to be used with inside of our program, but we haven't actually used it anywhere, and because the macro is not Javascript in the true sense, it's something that is built on top of Javascript, there's no resulting output for it until we actually invoke it, so let's go ahead and actually use our macro. For the usage I just specify the name of the macro and then I implement the pattern to the right hand side of that. Because this pattern is just, it expects a value to be on the right hand side of it or it expects something to be on the right hand side of it, and that's going to be captured as $x with inside of our pattern. I'm just using the value of 42 and if I pass that through the Sweet.js compiler we'll say that we get 42 on the right hand side in our resulting Javascript. This is because that's exactly what I told the template to do, was just to output the value that I provided to it. Demo: Consuming a Macro I can do interesting things with a simple macro like this because it's capturing everything from id across to the right, so I can put some into the left hand side of that. Say I could assign that to a variable, so now I'm using my id macro, we're sending it to a variable, and if I pass that through the Sweet.js compiler, you'll see that the right hand side has updated to have my variable is then assigned the value of 42. You'll also notice that the variable I defined as x on line 5 in my simple.sjs file, but on the right hand side it is named x$511. This is because, as I referred to in the very first module in this course, Sweet.js is a hygienic macro engine, so it will do its best to ensure that the variable names don't collide with anything else. I believe in this example we don't have any other variables that could be called x. We're not using any variables with inside of our macro itself that has been called x. We're not redefining x as _____ with inside that program, so it's not really needed to do the hygienic renaming for us, but it's still good that it does that regardless of whether we needed it or not. It just means that it's not something we have to opt into, it's immediately done on our behalf. Demo: Understanding Macro Patterns I can do something else with a macro pen and I've called it x, so if I was to go up and change it from being x inside the pattern, and called it a, if I save that, but I don't update my template, and then we run this you'll notice on the right hand side the output is $x, so now it is varx$511 = $x because that's what our template is saying. We've said that we want to output something called $x. We haven't necessarily defined $x and, as far as --- the macro engine is concerned $x is something that might be coming elsewhere in scope, so Sweet.js isn't really concerned about where $x came from, it's just that's exactly what you told it to output. We can obviously change that to $a in our template. Do it again, and we'll be back to having the value of 42. The reason that I've prefixed these variables with $ is because that is kind of the common pattern to be used with Sweet.js, that pattern components that we're matching on inside of our rule or case pattern, we prefix with the $ just to indicate that they're something that has come from Sweet.js itself, not something that we're going to solely define with inside of our program. I can change the pattern on the macro itself to look a little bit different. I actually don't like the fact that I have id and then some white space and then the values being captured. Maybe I wanted this to look a bit more like a function, so let's come back to the pattern, and I was to wrap the $a in parentheses. Now if we pass that through the Sweet.js compile you'll see that we've actually got an error. If I just scroll up here in my apple window, you'll see that we've received a syntax error and the syntax error says that the macro id for the macro with the name that we've got specified, did not match its usage, which was 42 on line 5. This is because the pattern that we've defined expects parentheses, so now I would need to update my usage to include the parentheses. Now if I pass that back through the compile it's happy, we get our value, and again, we don't have any additional changes to it because we haven't changed the apple template, we've just changed the way that we're trying to match on this. You'll notice that I've still got white space between the id and then the parentheses. Sweet.js doesn't concern itself with how much white space. White space isn't something that we can match on with inside of our rule, so we can have as much white space kind of between the name of the macro that we're using, and then where the pattern starts, with the exception of things like new lines. Those could be captured in other manners, but we don't have to make sure that our rule takes into account that there has to be only one character of white space or it has to be no characters of white space. White spaces kind of collapse down with inside of the Sweet.js engine. Demo: Values and Macro Patterns Other things that I could do with my macro is, because this is just a simple value, I could pass it through to another function. Here I've said that I want to use my id macro and pass it through to console.log, which is a function available in both node.js and inside of the browser. When we run that through the compiler you'll see that we end up with console.log and 42, so the value is just returned from the macro engine. The template has said that we want to output the value directly, and it will just output it inline, so obviously we don't have to necessarily assign this to a variable. We can use it in whatever user scenario is useful for what we want to do. I also don't necessarily need to use a numerical identifier or a numerical value here for id macro. Instead I can make this an object, so now that it's an object that will be at port inline, as we would expect. I could make this an array, I could make this another macro, anything like that because we haven't put any kind of restrictions on the type of values that we're trying to match with inside of our pattern. We're just saying that our pattern needs fixed something with inside of parentheses and we're going to return whatever that something was, so what is passed in is kind of irrelevant. It's not struggling any particular type or any expected uses because I haven't opted into that. We'll have a look at how we can specify components of a pattern to be of particular types later on in the course. Recapping Macro Workflows Now that we've seen how we can write a simple Sweet.js macro. Let's just do a bit of a recap to make sure that we've got all the principles down. The first thing we'll need to do is we need to give our macro a name, so the name can be anything that is a valid identifier with inside of Javascript, so this could be a series of characters in _____ we used ID, so it's something that could look like a variable name or a function name or anything like that, but we can also use special characters, such as equal signs, but the way we would use is slightly different and we'll look at later in the course. There are a few restrictions that we can and can't use inside of our macro name and those are mostly related to what are referred to as delimiters by Sweet.js and within Javascript itself, so things like parentheses, curly brackets, and square brackets are not things that we can use to match on with inside of our macro name. The main reason for this is we don't want to change the way those particular things work because they're very core to the way Javascript program language works, so we don't want to be overriding the way the curly bracket or said white space would work because then we could get very large and unintended side effects with inside of our program. Once we've created a name for our macro we then have to give it a pattern. This is what we're going to match on with inside of our location, so that sweet.js knows that this particular macro is the one that we want to use in this case, yet macro patterns can be as simple or as complex as we desire, so we can have a very simple one, which we started off with, which was just matching anything after the name or we can start putting in delimiters that we can then restrict the usage of our particular macro and if we don't use the macro in the way that the pattern is defined, then we'll receive an error with inside of our application. Perhaps the patterns defined, we then need to define a template. A template is what will be inserted in place of the Sweet.js macro, so this is what we'll be getting in our expected output from the Sweet.js compiler. Obviously this will go through a hygienic rename to make sure that anything that we introduce doesn't accidentally override anything that's previously been used with inside of our Javascript program, so Sweet.js will do its best to ensure that all that is tracked. Then we need to consume the macro because once we've defined a macro without an actual consumption of it we don't have anything displayed. There's not particular usage that is provided out of the box and if we don't consume the macro, then we have no outputting Javascript from it. When we've finished writing all of our Javascript that then consumes ur Sweet.js macros we then pass it through the Sweet.js compiler. I've got an extension for Sublime Text, which is the text editor I'll be using that calls the Sweet.js compiler when I pass it through the Sublime Text build engine. There are many different ways that we can use the compiler and we'll look at those in a later module, but at the moment I just want to look at how we can use macros themselves. Finally, we end up with an output Javascript file that we can then use with inside of a node.js application or inside of the browser or in any other runtime environment that we've got for Javascript. Macros With Multiple Rules Sometimes when we're creating rules for our Sweet.js macros we might not want to match on every single scenario or attempt to try and match every single scenario in a single pattern. In this case, we might want to have multiple patterns that we can do slightly different operations based off of the way the pattern was used. In this case, we might want to have multiple rules for a single macro and Sweet.js allows us to do that. The way it looks is we have a macro with the name and the same way that we did with the simple macro we looked at earlier on in this module that we can have multiple of these rule definitions and we can have a different pattern for each time that they're used. Each different pattern will then have its own template output, so that we can do slightly different things or some scenarios we might want to do very different things based off of the way that the pattern was matched, so this is how we can have different layers to our patterns and different layers to our macro that allow us to do different things. The patterns are matched for a best fit scenario, so it's not like it's a top down approach where the very first one that could potentially match the usage scenario is the one that's used. Sweet.js will try and find the one that matches exactly to the pattern that you've provided it and if it ultimately doesn't find any one that matches it will then raise an error, the same that we saw earlier on in this module, so we don't have to concern ourselves with the order in which the rule patterns are defined. They can be in whatever order works best for us and the way that we want to define that that ultimately it will just come down to what is the best pattern to match the usage scenario that we have. Demo: Creating Macros With Multiple Rules Let's have a look at how a macro with multiple rules would look. We're going to create a simple rule-based macro that has some multiple patterns here and we'll have a look at how that works with inside of our Sweet.js application. I'll start with a simple macro definition again, and I'll give it our initial pattern. This'll look very similar to the adding macro that we had previously, so I'm creating an addition macro here, so I might want to be adding some values together, and ultimately that will substitute down to the _____ with inside of Javascript, but I started off with, well if we've only got one argument I actually don't want to do anything in terms of addition that's just going to act like the adding macro we previously had. Let's have the usage of that (Typing) and we'll pass that through the Sweet.js compiler. You'll see that we output just the value that we get because that's the expected match that we filled and say we wanted to add two numbers together (Typing). Passing that through the Sweet.js compiler brings us up with our error and that's because this time we don't have any pattern that matches one caller to, we only have one that expects just a value called x, so there's a delimiter here, bit in the comma, that doesn't match with the rule that we've got defined. If I was to go up and create a new rule I could create a rule that looks something like this, so in this case I've got $x and then I've got a comma delimiter to indicate that there might be some more stuff to come, so in this case it is part of the pattern that I want to match is the comma, and then I have $y. This means that --- we'll be able to capture two values in this case, so if I pass this back through the compiler this time we get 1 + 2, so that's the template that we've output for the second usage, which is 1, 2. First we come back up to the first rule and change the pattern for it. You'll see that in the outputting Javascript from the compiler that we've definitely matched on the very first rule because this time we've got $x + 0 is what I expect from the output of the first rule macro, so that's what has happened here. We've got 1 + 0 or 1 + 2, depending on the usage. Now in my second pattern here I choose to use the comma as a delimiter, but I don't necessarily need to use the comma as the delimiter, I could also use just a space or a shared white space between the two parts of the pattern that I want to match, so first I remove the comma from both lines three and seven, and then pass it through the compiler again. You see that we still get the same resulting Javascript, so the pattern matches that we have are specific to the usage and we can, as I said earlier, we can make that usage as complex or as simple as we ultimately want. Recursive Macros The next way we might want to use a macro that I want to have a look at is the idea of making a recursive macro. Sometimes we want a macro that can depend on other macros, so this way we have a each macro is very specific to doing a single task and if there's other things that it might want to be able to do or we might want to inject other syntax, then we could use other macros that can do that. Alternatively, we want on our _____ macro that actually recurses into itself, so this would be like a recursive function essentially, but recursing through the macro definition, so it can keep calling itself again, and again. These can be good scenarios that we can use to make sure that we have macros that are targeting a specific pattern and a specific template output that are very very small and isolated, so that they only have a single responsibility, and that if there's responsibilities beyond what we can do with inside of a small logical concept, we can break those out into separate macros. Now we'll go back to our code and have a look at how we can create a macro that is recursive. I've got my add macro from our last little code example here again, and I want to take this and make it into a recursive macro because I've got this bit of a problem. If I add on value or if I want to add two values together, then I've got patterns defined for that, but what if I was wanting to add a third value? I could always create an entirely new rule to match that, but it's starting to look a bit complex. What if I wanted to add 4 values, 5 values or 10 values together? The more values I want to add then obviously the more rules that I --- would have to create because they're all different patterns essentially, so this is where the idea of a recursive macro can start becoming very beneficial because I don't want to have to create a pattern to match every single usage scenario of my addition macro here, I want to be able to use it kind of for n number of arguments. Now we update my macro so that it is recursive, so what I've done is I've changed on line three to match on the pattern of $x $y … The … indicates that there is going to be 0 or more arguments expected after the y inside of this pattern that we can then pass through as the … again, through _____ template, so this time in our template we've got, instead of having just x + y, we go $x + add($y …). What this says is to the macro engine --- if we match on more than two arguments it's going to go x + and then it will call the add macro again, with y and then 0 or more arguments passed in, so if it was to just do on what we've got on line 6, which is add 1 and 2, the _____ that you'll see on the on the right hand side has not changed because what has happened it's gone and seen that the first time that we've run through we've hit the second macro template, so we've got $x $y and then that has gone through to the template and has gone $x + --- add and then $y and then … indicating all the arguments that appear after $y. In this case, there's actually no argument after $y, so this time we're just calling add with a single argument, which matches then the first rule in our template, so the rule on line 2, which then just returns the value of $y, which then results in 1 + 2. If I was to start putting some more arguments into our app and pass this again, through the Sweet.js compiler you'll see that now add all the arguments together because it's just become a recursive function essentially or realistically it's a recursive macro, so it's matched x, y, and then … has represented the 3, 4, and 5 on the first pass through, so this has become x + add 2 3 4 5, so again, this has matched the second rule usage of the add macro, which is this time x now equals the value of 2, y equals the value of 3, and then … represents 4 and 5. Then we go back through, and because we've got more than two arguments, oh I'm sorry, more than one argument, we've matched the second macro rule again, and we keep going until we get down to just the five being the value of y and nothing else after it, and then that will result in matching the very first rule pattern. We can see that this happens by if were to change the first rule again, to have + 0 to it and then run that through the macro engine, you'll see the + 0 is added after the 5, so it's only until we get to the 5 that we've actually hit the first part of our macro, so the very first rule pattern that we've defined, and the $0 is added to 5, which is then added to 4, which is 3, 2, 1, etc. etc. We've made our macro recursive by the usage of the …, so this is a special bit of syntax that Sweet.js allows you to use with inside of a raw pattern to indicate 0 or more arguments are matching after a particular section of a pattern. Avoiding Macro Recursion Because macros in Sweet.js are recursive by nature, we have to be careful of unintended impacts of doing that. Here I've got a code example where I'm wanting to override something that's built into Javascript itself. In this case, I'm actually overriding the function keyword in Javascript. What I'm doing here is I want to do some kind of debug profiling on functions with inside of my Javascript program because I might be concerned about the performance of one or more of them. What I've got here is I've created a macro with the name of function. Now this is perfectly legal within Sweet.js, is we're able to override built-in components of Javascript and, in this case, we're overriding the function keyword, so that we can do our own things when that is in use. What I've done is I've created a macro, it's got a rule that matches on a name after the keyword that we've defined, so this would be the name of the particular function. We then expect some parameters to be provided to the function that we've got, so this could be 0 or more as indicated by the … and then finally we expect a body. We have curly brackets to indicate the start and close of our function script and then we have a body, so something happening with inside of that, and that again, is a … to indicate that we might have 0 or more statements with inside of our function. My usage down starts on line 11. The usage of it is I've got a function, it's got a name, and it hasn't got any arguments in this case, and then we're just going to be iterating through a for loop and just basically counting up to a number and then outputting the result at the end, so it's not something that's particularly real world, what we've got here in our usage of our macro, it's just an example of a function that might take a long period of time that we might want to profile, so what I've done inside of the template for my macro setting up on line 3 is I've output, obviously, the function keyword because again, this is resulting in a function, and then we have the name that was captured as part of the rule, the parameters, and then our …, so this just basically says take the parameters that were passed in and then we'll output them directly in line. We are going to close our curly brackets for our function and then I'm inserting a console.time and console.time end method calls. I ran the body that has been provided to our function, so by doing body … inside of our template it just says that that output the 0 or more lines that represented the body, so I'm wrapping these inside of our timing scope here, so I could see how long that has taken when I've executed it through the browser or through node.js or through any _____ through runtime and then I get results from that. If I was to run this macro as it currently stands I will actually crash the _____ node.js _____, but because I am using the node.js compiler for Sweet.js, but ultimately you crash the Sweet.js compilation engine because what happens is Sweet.js goes I have a macro, it's got a name function, so every time I see that I want to match on something, match on a rule that's defined in the where function would look, and then I start outputting a template. This template output has the function keyword inside of it and Sweet.js goes oh that is a macro, so I've to backup and start this macro, so then it goes and realizes that it's matched on that the usage that we've output, so then it goes into this macro template and then it realizes it's got a function keyword, so it matches on the function keyword again, and then we site our macro. You see that this has been an unintended side effect of the recursive nature Sweet.js macros. We're recursing into something that we didn't really want to recurse into, we only want to execute this once, we only want to wrap the console time start and end around the body of the original macro usage. I want to actually run this through the compiler of the _____ because, as I said, this will end up in an infinite loop and ultimately it will just crash it in the background, so the output of that will never actually see anything on the right hand side. The question is, how would we fix this so we don't get the unintended side effects of doing the recursion? To avoid this unintended recursion I want to actually change the way that I'm defining the name for my macro, so instead of doing macro name I can use a different way that we can define macros, which his let name = macro. Demo: Avoiding Macro Recursion I've updated my macro definition, so instead of having the name after the macro cubit I have it before, so essentially this is like an assignment operator, so I'm using the language as a keyword that's introduced for Sweet.js macros, but it's also part of the next version of the Javascript programming language in ES6, but in terms of Sweet.js it could operate a little bit differently, but if you are using Sweet.js with the next version of Javascript ES6, it won't have any unintended side effects. Sweet.js will only invoke itself with let-based macros if you have a macro assignment to the bracket side of the equals statement. What let does is it says that the name of this macro is going to be at top of a macro only, so it's not something that we'll recurse through, so the macro name can only be used from invocations outside of other macros. We can't actually use it from the invocation of a template within a macro, so this time if I was to run it you'll see that on the right hand side we have got the correct output that we were expecting and we haven't crashed the Sweet.js compile engine, we've just executed the code in the way that we expected it to be executed. You'll see that the name of our function, longRunning, has been hygienically renames, as have all the variables that were defined with inside of that. You'll notice that we have our console.time and console.timeEnd statements that have wrapped around the body of the function that we defined, and we've got our invocation at the end, so this has done what we were expecting our macro to do and we haven't had any unintended side effects of the fact that this is naturally recursive macro with inside the Sweet.js. Case Macros So far we've only looked at how to create macros that are rule-based, so using the rule keyword to define the way the pattern works, so I want to talk about the other kind of macros that we can create with Sweet.js, These are case-based macros. With rule-based macros we've had a fairly simple usage model. We have our macro definition, which contains a pattern, and it has a template, and that template is directly output exactly as it was written inside of our macro definition. This is how a rule-based macro works. We've got input and we've got direct output. We don't really have the ability to do any kind of manipulation in between, but that's where case-based macros come into Sweet.js. We have our macro definition, the same or at least very similar to the way we've had with our rule-based macros, but then in our template we're able to execute some custom Javascript code, so this could be we want to do something specific like introduce a new variable, create some new syntax or manipulate the parameters that have been passed in in ways that the resulting --- usage of those parameters is different to what we could just do by directly outputting them. We might want to change the value of them or change the structure of them before they're output and then finally we're able to output our particular template that works the same way a template would have worked wherever I put from a rule-based macro, but instead we're able to manipulate it, kind of have that step between the macro being invoked and the output being directly written. This is where we might want to be using a case-based macro, is where we've already kind of intercept that pipeline between the macro definition and the output of it. This is how a case-based macro looks. It's defined in a very similar manner to the way that we have a rule-based macro. We have macro name and then the parentheses or what name equals macro and if we want to _____, actually there's a recursion or correct macros that are kind of top level only, once we are in the macro definition itself we have the case keyword instead of the rule keyword and then we define a pattern. The main difference from the usage is the way that we output the template at the end of the day. Where a rule-based macro allowed it to just write the template directly with inside of the curly brackets we have to, with a case-based macro, explicitly return a syntax object from the macro itself. What I mean by syntax object is something that Sweet.js is able to track and knows is a series of syntax variables and objects that have been identified through the Sweet.js engine, so that it is capable of tracking them through hygienic renames. The way that we tell Sweet.js that the result is a syntax object is by using the hash symbol before the curly brackets that result in the template. This is just so that Sweet.js knows that the following is kind of the resulting syntax that we want to output into the usage of the macro and it's capable of tracking everything that is with inside of that template and do all the kind of hygiene that we want to be done for it. Demo: Simple Case Macro Now let's have a look at how we can create a case-based macro with Sweet.js. Back in my editor here I'm going to start with creating the id macro that we had in the very first rule-based macro that we looked at, so it looks very similar to the way the rule-based macro worked. We have our case, we have a pattern, and then we have the return, and then we have our usage of it. You'll notice though that in the pattern that I've defined I actually have another character before the x. I've used the underscore character to represent what's being captured here and what is in the underscore character is actually the name of the macro itself, so this will contain id. Generally speaking, you're probably not going to want to use the name of the macro with inside of the template itself that we're outputting, and as a common convention in Sweet.js we tend to replace the name of the macro if we wanted to kind of discard it as an underscore just so we can, it's kind of easy to overlook when we're looking at that pattern, and we also know that if as a usage underscore is something that you have kind of disinterested in, but after the name it works the same way as the pattern that we previously had, so I just have my $x and then I break into my template where I'm explicitly returning a syntax object that contains $x. If it passed through the Sweet.js compile you'll see that we get the value of 42 output on the right hand side. I can do all the same kinds of usages that I have in my rule-based macro, like assigning the result to a variable, and we'll see that that works the same way as we expected it to work from the rule-based macro where you get our hygienically renamed variable of x on the right hand side, which equals the result of calling our macro. Demo: Manipulating Syntax in Case Macros Now let's look at creating another, slightly more complex, case-based macro. Something that we're actually going to do some Javascript manipulation with inside of our template before we return the final syntax objects, so we're going to delete this macro that we've got here, and I want to create a macro that allows me to generate a random number, but instead of it being a random number that is generated every time that the program runs I want the random number generator to kind of at compile time, so it's kind of a constant random number. Every time we recompile we get a different random number, but the random number will be the same for every time that this instance or this version of my program is run, so I've scaffolded out my macro here. At the moment it looks exactly like my case-based macro did previously, but I've given it the name or rar because the way I want to use this is I want to be able to assign a variable from the creation of it, so I want to use it like rar x, so this will create me a random variable or a random var in the location and assign it to a variable that is called x with inside of my application. Inside of my template I'm going to start _____ creating a random number. Now this random number I've assigned to a variable named r and this random number exists only within the scope of my particular macros execution, so I now want to change the resulting template to generate me a variable assignment statement as well, so I'm going to do var $x, so the identifier that was captured as part of the pattern will equal r. We'll save that and pass it through the Sweet.js compiler. You'll see that we've got something a bit unexpected has happened on the right hand side. We haven't actually got the output of a random number. Instead we've got var $x has equaled r$506, so you might be wondering why that's happened. The reason that that has happened is because what we've done is we've created a variable inside of a new scope, which is the scope of that particular template that was running that Sweet.js has hygienically renamed, and then we just told it to output x as an assignment to that, so x actually references the variable that was created with inside of our Sweet.js macro scope, but that variable doesn't exist outside of that macro scope, so unless we somehow manage to magically create a r$506 and somehow magically knew what the hygienic rename was going to give us, we wouldn't actually have this as a working function. We'd end up with a _____ because r$506 doesn't exist anywhere. What we need to do is tell Sweet.js that this variable that we've created represents as a syntax value and then once they actually capture what the assignment of the r with inside of our --- template is and then output the value that it equals, not a direct reference to the variable that was created. Now macro template has now been updated to tell Sweet.js to create a new syntax object that we're tracking. What I've done is I've used the letstx, which is let syntax and is a macro itself, so I let the syntax be assigned to a variable called $r in this case because, as I said earlier on in this module, things that represent Sweet.js syntax objects a common convention is to prefix it with a $ and then I say that I want to make a value of the case, this is a value from a variable that exists within scope, so r, so the first value that we're passing in is the value that we're willing to make a syntax object from, and then we have to tell it a scope that we're working within, so this is what the $x inside of a template represents. We can use templates with inside of a Sweet.js case-based macro that don't actually represent a return value, but just represent a scope, so that's how this particular usage works. We say, make value r equal to make value with the value of r and then inside of the scope that this particular macro exists in. I then put that into an array because syntax objects in Sweet.js are expected to be array objects that ultimately is just going to be an array of a single value, and then inside of my template I do $x = $r instead of r itself, so I want the variable that we're introducing to equal the syntax object or the result of that syntax object in this case. Now if we were to pass that through the Sweet.js compiler you'll see that we get a hygienically renamed variable of x and we get the value of our random number generator. If I was to, instead of using r, change that to 42 and then run that back through, you'll see that we get the value of 42 because makeValue, as I said, the first argument that we pass into that is the value that we want to create of this syntax object. Let me just reset that to r again, and I'll introduce a couple more variables (Typing). I've introduced two more variables that will run through the compiler. I will say that each of them gets their own randomly generated number. The reason for this is each time the macro runs it has its own scope and because we're telling it to create new syntax objects they definitely reference the previous usage, it's always a brand new scope or a brand new syntax object that is created, so we get a unique value every single time. Also because Sweet.js is intelligent about the renaming of them, we can use these values outside of the initial variable declaration that I've got there, and so I've loaded them out to the console, and then if I pass that through the compiler you'll see that the hygienically renamed variables of x, y, and z was x521, y523, and z525 and that's also the variable usage that we end up with inside of our console.log statement. This is how the variables that we're creating inside of our Sweet.js file, so our .sjs file that get hygienically renamed, the usage is also hygienically renamed, we don't end up having to magically guess what the hygienic naming is going to look like, Sweet.js will take care of that all the way through the pipeline. Summary In conclusion of this module we've had a look at rule-based macros with Sweet.js. Rule-based macros are the simplest kinds of macros that we can create with Sweet.js. We start with a macro and then we give the macro a name. We then have a pattern and our resulting template, so the template is exactly what we output in place of the usage of that particular macro. A pattern can be something as simple as just match everything after the usage of the, usually the macro name, up until at end of line or we can start introducing additional delimiters or other components within it that we want to capture as separate parts of the pattern. Variables that we introduced par the pattern we commonly prefix with a $ to just indicate that they're Sweet.js syntax objects, rather than normal Javascript variables, and these could be then used with inside of our template. We've also created multiple rules for a particular macro pattern, so that we have multiple --- rules for a single macro that we can do different things with them. We also saw that macros can be recursive, that macros are able to call both themselves and to each other, and we also have the potential for unintended side effects from doing that. We might accidentally end up in infinite recursion because we have macros that keep calling themselves, and we can avoid this by using the let syntax to define a macro, so let macro name equals macro definition, rather than a macro name _____ macro then the name, and then the definition. Finally, we looked at case-based macros. Case-based macros are very similar to rule-based macros with the two primary differences that we have to add that we have the date of the macro is provided as part of the pattern and it's always the first argument. Generally speaking it's not something that we're going to need as part of it, so we use a common convention of underscore as the value of something that we want to discard, so in this case, the name of the macro. We then have the --- template that will return. We have to explicitly return our Sweet.js syntax object represented by the hash line and the two squiggly brackets, and then inside of that is exactly what we output. This could be one or more lines of code and our simple example we only output a single line of code, but we also saw that because we're able to do some additional things like manipulate the syntax objects that we're working with, we can run our Javascript with inside of these case-based macros. We can introduce new variables into the Sweet.js scope that we can then assign values to and then extract those values out and then return the resulting values as part of the macro template that we've output. We can also create multiple patterns with this case-based macros, the same as we did with rule-based macros. Add, obviously we'd just add different patterns that we can match on so that we can do different things with inside of our template. Advanced Macros Classifying Macro Tokens Welcome back to our Sweet.js Get Started course. In this third module we're going to be having a look at some advanced macros that we could write with Sweet.js and some advanced concepts that we can combine with Sweet.js macros to make them more powerful and easier to work with. The first advanced concept that I want to introduce is adding restrictions to the patterns that we're creating, so that we can make sure that we only are matching _____ be more specifically on the things that we want to match with inside of our pattern usage. The order of this is that we can create classes for the tokens that we're going to be working with, so the tokens are the variables that we're capturing with inside of our Sweet.js patterns. The reason we might want to do this is that we don't want to treat every single token with inside of our pattern in the same manner. By default we can use any kind of token classification with inside of our pattern and they'll be parsed as valid by Sweet.js, but we might not want that to happen. We might want to be able to treat the usage of a variable with inside of our Sweet.js macro differently to the way we do an assignment statement with inside the macro itself, so this is what we would want to use token classes to make sure that we do some restrictions and their incorrect usages can be caught at compile time. Ultimately the goal is that we can be able to do different things based on the types of tokens that we're capturing with inside of our macro, so some of these that we don't have to create as many different named macros to do what are similar things, instead we can have different types of patterns and classes of those tokens with inside the patterns to make sure that we're capturing and performing the operations we want to on the correct part of our macro definition. Overview of Token Classes There's a number of different classes that we've got with Sweet.js tokens. These could be used to represent different kinds of information that could be passed into the different macros that we're defining. We have the concept of a literal or lit is the name of the class with inside of Sweet.js, so this represents a literal value with inside of Javascript, so this could be a numeric value, it could be a Boolean value, it could be a string. Anything that is just a raw value that we could be using with inside of a Sweet.js macro or just with inside of Javascript itself that would be classed as a literal. We have expressions, so these are the things that have a couple values will be provided to them. Generally two values will be provided to perform an expression and we get a result back from that. An expression could be something like an assignment, so variable name equals a literal. That would be broken down as var and then we have an expression that exists to the right hand side of the var token. Also things like functions that we could be invoking with Javascript, they could be expressed as expressions, and they can be captured as so as Sweet.js tokens. Finally, the last built-in token class that we have is the identifier, so this would be a variable that we might be defining with inside of our code, it might be the name of an argument that is being provided to a function, so if we were overloading the function keyword within Javascript, as we saw in the previous module, we might want to specify that we're expecting idents or identifiers as what is in the parentheses --- that appear after the function name, but it can also be for the names of functions themselves, so if we wanted to capture the name of our function and be able to use that with inside our token, then we could have that as an identifier. Token Class Syntax This is what a Sweet.js macro with a token class looks like. Going back to our initial definition of the id macro. With this definition of the id macro I've defined that I have a token named x, which will be the result that gets returned, but then I've used the token class syntax after that, so that's with the colon and then the class name is. In this case, I'd say that I expect x to be a literal, so you'll see down in the code at the bottom of it that I have indicated that there's a syntax error on the second line. That is because the first usage id(42) is valid because 42 is a literal, whereas the second usage of id(1 + 1) is going to result in a syntax error because 1 + 1 is an expression, it's going to give us a result back, so this is something that is not valid for the rule that we've defined for the current macro. Shortly we'll have a look at how we could modify this macro so that we can capture different types. Custom Tokens Classes Finally, with token classes that we have within Sweet.js there's actually a new feature that's been introduced as of the 0.6 release of Sweet.js and that is the ability to create our own token classes. This will be useful if you've got a particular bit of syntax that you want to capture and then break down the subparts of that as their own tokens and then get those out and use them with inside of your own macros. At the moment, as of version 0.6 of Sweet.js, this is still considered a highly experimental as it's also a new concept that only just made its way into the compiler, so it's something that is subject to change, but it has a lot of potential and we'll have a look at how we can use that with inside of our one code base. To use custom token classes what we do is we use the macro class keyword to define that this is going to be a macro class. After that we provide the name of the token class that we're going to be capturing and then we provide a pattern that matches it in the case that we want to use it. Then down in our macro at the bottom half of this code snippet we see that we use it by doing $x and then :name, so the same way that we would use any of the built-in macro classes, so like :lit to say that we're expecting a literal, I would use :name to say that I'm expecting my particular macro class or my particular token class to be used at this point. Demo: Adding Tokens Restrictions Now that we've got the base understanding of what a token class is let's have a look at how we can use that with inside of Sweet.js. Here I've got my code snippet from the slides before. You'll see that I have my id macro that's got a rule with x and that it says that it's expecting a literal and then I've got my two usages, one which is with a literal and then one which is without a literal. Now if I can run this through the compiler, you'll see that we get the error that we were expecting. If I just scroll up in the error information you'll see that we get a syntax error because, as Sweet.js says on line 7, the usage of the id macro cannot be matched with the pattern that we have defined. As I said, this is because I'm using the literal macro token class, so when I'm going to try to use an expression and that's invalid, so we haven't got a pattern that captures what we're expecting to use it as, so we don't have any way to use it in this example yet. If I was to modify my macro to add another rule to the macro definition I can than make a new macro pattern that is expecting an expression to be there, so this time I've got a second one, which is pretty much the same as the first one, but says that it's a $x:epr, so it is expecting the expression as the token that's valid in this scenario. If I run this through the compiler now you'll see that we get rid of the error that we had there previously and we get the expected output on the right hand side of the screen. Demo: Different Templates With Macro Classes Because we're watching each part of this macro rule differently based off of the argument types that we've provided it, so the different type of token classes, I can do different things with the resulting of the macro token class itself, so let's say I wanted to change the expression that if that is provided, I want to then add 1 to it. I could run this through the compiler and you'll see that the expression now is updated so that we get 1 + 1 + 1, which is what we expect because it was the second of our rules that were matched. From an external usage point of view of the way we consume our id macro, we don't have to know that anything is different about what we're providing to it. All we need to know is that when we provide it with different arguments we can get different results. If we weren't using tokens and token classes in this manner, we would have to create different id macros to represent the different kinds of usage that we wanted, so if we wanted an id where we resulted with a + 1 on the end, we would have to have a different name for that if we didn't want to put a restriction on the token that we had provided. Now let's move on and have a look at how we could create a custom token class with Sweet.js. Demo: Custom Token Classes What I'm wanting to do with my new macro class is I want to introduce the ability to create default values for functions with inside of the Javascript. This is a feature that has been proposed for the next version of Javascript, but at the moment I want to use it in my current Javascript runtime environments. The problem is that no browsers support this and I might want to go back beyond browsers that would put support in maybe in the next six months or so, so I need a way that I could turn this function with default values provided as their arguments into something that I could use in my programs today. Admittedly I could write this in a way where I do the checks and assignments with inside of the functions in themselves, which is obviously the way that we would do it --- in Javascript today normally, but I want to be able to do this in a bit more of an expressive manner, so that it looks like something that is coming in a future version of the syntax. I want to be able to use default values. The way these default values work is that I do = and then the value that I want to perform an assignment against inside of the argument definition for my function, so the first thing I want to do is create a custom macro class. I've decided to call my macro class default_value because that's a name that makes sense, it is representing a default value should be there, so then you'll see that I have a pattern that's defined on line 2 that, after the pattern keyword, I expect it to be id = value. That is --- exactly what I'm looking for with inside of a Sweet.js macro when we go around to writing it, is that I'm expecting the particular token to be in the shape of id or of a token, in this case I'm calling it id, an equal sign, and then another token, which I'm going to capture into value. The names that I have defined here, id and value, are important because we'll be using those with inside of our macro definition momentarily. I've gone ahead and created my function overloading macro that defines that this is going to have default values, so you'll see here on line 6 that I'm expecting, after the function keyword, a name and then open parentheses and then I'm going to be capturing a repeating set, so indicated by the comma and then the … at the end of the line, that is going into a variable name or a token called param, which is then expected to be of the class default value, so this is telling the Sweet.js compiler that I'm expecting default value, which obviously I've defined area is the way that we're going to be using this. Obviously we do have a bit of a restriction with the way we're using token classes, is that I'm now expecting that all the parameters with inside of my function are going to be using the default value token class that I've defined up front, that I can't have any tokens that are just plain literals, but that's okay. For the current usage that I've got that's a perfectly fine scenario, I'm not going to be mixing and matching. Also, because I'm overloading the function keyword, any functions that don't use default values won't be impacted by this usage. Inside the function body, so you'll see that I've start, as to be expected, by writing out the function keyword and then $ name, so the name of the function that's been defined, in case of line 14 that will be add, and then I say $param$id, so this is where I was saying that the name of the tokens that are inside of our token class are important because that's how we reference them with inside of our macro body. They are referenced by the fact that I used the name of the token as captured by the macro itself, so in this case, param being the name of the token that I captured on line 6, and then I want the part of that macro token class that, in this case, I want the id part just from with inside of our token value. I then say that this is going to be repeating, so the bracket comma, so I want to repeat the comma and then a … for all the 0 or more repetitions of the arguments that have been provided to this particular function. On line 8 you'll see that we've done the --- other half of the usage of this to make our default value work, so you'll see that I've $ and then a parentheses that wraps around this entire statement, so if I just scroll across and make the text editor window a little larger, you'll see that we've actually defined that this is going to be a repeating block, so I've defined that everything from the stand in parentheses on line 8 is where I'm going to be repeating indicated by the … on the end of the line. This is how we can make sections of our macro body that we're outputting repeated when we're using repetition capturing inside of the macro pattern itself, so what I want to do is I want to repeat $param$id, so in the case of the function defined on line 14, so the first one will be, I'm sorry the first id will be a, and then that will equal the assignment of the a being undefined and if it is undefined we will return the value section of our token class, otherwise we will just use the id. This means that if we didn't provide an argument or we explicitly provided undefined for the argument to the function, then we'll get back the default value that we've set. If I have provided a value, then instead we will use that one, so this means that we're not accidentally providing values that get ignored. As this is a repetition, --- this line of code will then be repeated for the b argument that is defined for the function on line 14. After we've done all of our default value assignments I then just want to output the body of the function as it was defined previously, so that's $body and then …, so this means that there is 0 or more lines to the particular function that we've captured, and I just want to output all of them as well, so this means that we're kind of just inserted something above the definition of the function body. The last thing that I will need to do with inside of this macro definition is deal with the second scenario that I've got defined on line 18. Instead of defining the name of the function after the fact, I'm defining the function as an assignment to a variable, so var sub = function. The reason I just wanted to include this as an example is to illustrate the way that we can have different patterns with inside of our macro definition that can still ultimately do the same thing, so I just need to _____ that so that we can run this through the compile successfully. The second pattern has been implemented correctly, as you'll see that I've got $param:default_value like we did up on line 6 for our first one, but we don't have the $name capturing beforehand. That's because we're not expecting a name of the function to be defined after the function keyword itself was used. We --- then do this similar kind of iteration through the parameter list on both lines 13 and 14, so to output the names of the parameters and then finally do the undefined check for them, so we setup our default values, and then finally output the body of our macro. I'm just going to make this window a little bit smaller, so we can see the output on the right hand side, and then pass this through the Sweet.js compiler. Here you'll see on the right hand side that we've got a successful output. Just make the window a little bit bigger so we can see it in its entirety. You'll see that we've defined our function called add and add is expecting two arguments, a and b. They've been hygienically renamed, all the variables and function names with inside of this, so you'll notice then on line 2 we have the function variable name that we defined, so $a became a and $529, and then it's doing an assignment of that equal to undefined and then the result of that ternary statement, so either 5 or the value that we provided to the function, and it's being repeated again, for the second argument that was put in, but this would be repeated for all the arguments that it provided to this particular function. Now let's just modify our sjs files, so we actually use these two functions, and just to ensure that the arguments that we're providing are getting set to the default values, so we'll do some sample usages. The few sample usages I've got here is first off I want to invoke the add function without any arguments, so I expect that to be the result of 5 + 6 passing in two arguments or passing in just one argument, so I would expect 5 + 6, 1 + 2, and 1 + 6 as the three usages that we have here. I've done a similar thing with sub, so no arguments, two arguments or one argument. If you pass it through the compiler again, we'll grab the output and jump over to some browser dev tools. We'll just paste this in and execute the code, so you see here that we've executed that code, both that we've passed in and that I've got 11, which is the first that I was expecting, so that's 5 + 6. We get 1 + 2 being 3 and then 1 + 6 being 7 and then similarly for 5 - 7 being -2, etc. etc. You see that our logic that we've got there to do our default values has executed as we were expecting it to and that we've been able to simulate a feature that is coming in a future version of Javascript, but then have a fallback to working in browsers that we have today. Demo: Custom Token Classes With Multiple Patterns As I said previously, we have a limitation with the way that we've defined our macro token class that we're always expecting there to be a default value assigned, so if I was to not want a default value, and kind of just assume that there is always going to be a value passed at a particular position, then I might have an error happening here because I won't be able to run this through the compiler. I'm essentially restricting myself, so I can't mix and match the concept of default and non-default values, but there's a way we can actually implement that with Sweet.js tokens as well, so let's say that I want to create a function that looks like this. In this case, I've expected there always to be a first argument provided being a, but the second argument b is not necessarily going to be provided, and it's going to default to 2, so I'm going to call this add2. If I pass this through the compiler now, you'll see that we result in an error. The reason is that it's not able to find any particular match with the way that we've got our token classes defined, so it doesn't have all the arguments matching the default value token class, but it also doesn't have any or all the arguments matching the initial implementation or function, which is all expecting the arguments to be of a _____ dead type not expressions or literals, so I've tried to make them match here, but I'm not able to do that. What I can do though is update my default value implementation to handle that, so I'll scroll back up to the top of our code file, and what I need to do is supply a secondary pattern for my macro class, so like macros themselves, I can create multiple pattern classes. The new pattern class that I've defined here is that I only expect there to be an id. I could put an additional restriction on this to say that I expect it to be an identifier as well. I could also do that on the previous line. I could say that I expect this to be an identifier and this one to be a literal or I could even make it that I expect it to be an identifier and then I could have a separate one that expects the _____ out of the _____ to be an identifier, I could do expressions, I can do anything I want, so I can add the restrictions as well with inside of macro token classes that I have with inside of the macros themselves. What I've done differently though is because I don't have an =value statement in this token class, I have to do some kind of explicit detection of what the value would be because otherwise I don't have a value that I could use with inside of the macros that are defined a little bit further down in this file. This is where another kind of syntax could be used with the way that we define our macro token classes. You'll notice that after I define the pattern I then have a where keyword and then after the where keyword I have a statement that is creating the missing part of our token class, so in this case, I've got parentheses and then the $value token is going to be equal to the result of executing the syntax block $id. Essentially that's telling it to whatever is going to be id will then be used as the value as well, so the identifier that was provided will also become the $value property that's exposed for our macro token class. Now let's pass that through the Sweet.js compiler and see what happens. You'll see here that we've got our add2 function has been created and the variables have all been hygienically renamed as we expect, but if I expand out the code window you'll see what's happened is that $a is going to be equal to undefined. If it's equal to undefined it will then be assigned either the a$545 or $a545, so it's going to result in something that will be assigned to itself regardless of the outcome of the equality test that's happening on the --- _____ truthy component to our ternary statement. I believe this is a bit of code overhead that we would have executed by the Javascript runtime engine because this statement doesn't make sense, but what it has meant is that we don't have to define multiple different macros to represent optional and non-optional macro classes, and we still get the advantage of being able to kind of mix and match that syntax. We could also put the default value creation at any point with inside of the parameters that are defined with inside of our function, so I could have a, b, and b equal to and then comma c and expect that both a and c have values provided to them, but the b is something that could be optional. Whether or not it would make sense to not just group all your optionals at the end of a function that comes down to a usage scenario, but what we've got here is we've seen how we can do that mix and matching by just defining multiple rules for our macro token classes. Demo: Runtime Type-Checking Finally, I want to have a look at another way we could use our macro token class to do something a little bit different with inside of our usage. I've used typescript before and I've referred to it earlier on in this course as one of the advantages that it gets from that is it has static typing built into it. What I want to do with this token class is kind of simulate the concept of static typing, so what I've done is that I've now created a way that we can define the type that I'm expecting for a token with an argument for a function, so what I've done is I've created a --- token class called typedef and what I expect is an identifier being the name of the parameter for our function and then I expect a literal value on the right hand side of the colon. What I'm going to be doing here then inside of our macro body is I'm going to be looking at the element that's passed in and if the argument is not of the type expected, so I'm doing a typeof check, then I'm going to throw an error, so this gives me some runtime testing of the values that are being passed in. This is the kind of thing that I might want to do in the debug version of my code base, so that I can capture through all the meta testing or just through user testing. Any incorrect usages of the functions without having to take on the full language of typescript to do compiler level testing on these. The reason that I've specified that I want it to be a literal as the right hand side of the colon with inside of my token class is so that I can do a correct type of a closing statement. We could do this in many different ways that I could say that I expect that to be the string number or any of the other ways that we can do type of detection, but in this case I just wanted to do a simple type of comparison. You'll also see that we've used our repeating block that we saw in the previous example, so $ and then a parentheses and then the block that we want to repeat. In this case I want to repeat an if statement, so if I was to have multiple parameters that were provided, then I could do a multiple different type of checks. If you pass this one through the Sweet.js compiler you'll see here that I have my typeof statement has been executed, so I have typeof, the argument that's passed in equals or if it doesn't equal the type of 0, so if the parameter for that is not a numerical value, then it's going to throw an error and provide some information, so it says that the typeof was not valid and it's not what was expected. Seeing the two usages that I've got down on lines 7 and 8, the first one would run successfully, and the second one we'll throw a runtime error. Let's have a look at that in action. Here I am back in my browser again, and if I just paste this into the console and then execute it, you'll see that we got log error, which is what I expected at the console.log statement to have been executed, and then the second one doesn't get looked at because it throws an error saying that the type of string was provided, but it expected the type to be a number, so this could be useful in terms of a debugging version of my code base, so that I could make sure that the parameters were all being used correctly, that I'm not getting strings being passed when I wanted to have numbers. We could also take this idea of doing the top of check to a higher level and rather than just doing a typeof we could maybe make that a Boolean function, --- a function that returns a Boolean value so that we can do something more complex than just saying is this a number, but maybe do a range test that I'm only expecting values between a particular range, so if I was doing pagination with inside of an ajex request, I would expect it to only allow to x number of pages or x number of records with inside of the page set. Let's all look at how we can create restrictions on the types of arguments that we've got or types of tokens that we've got with inside of macro definitions. Infix Macros The next principle I want to introduce is the ability to look backwards with inside of the macros that we're defining. So far the way that we've defined our macros is that they work with this kind of a work flow. We have our macro syntax on the left hand side, so be it the id or the function keyword or anything else like that, and then on the right hand side of that is the tokens that we're going to capture, so this would be Javascript syntax or anything else that we've got that we want to capture within part of that pattern, but the _____ always sits to the right hand side of the name of the macro, but sometimes we want to be able to capture macros that work with the parts of the tokens that are being used to make up the macro pattern sitting on both the left and the right hand side or even just on the left hand side of the definition that we've got, so in this case, we would want to be using a macro that has the ability to look behind the name of itself to do some detection. This is the case where we would want to use an infix macro rule. This could be used with both the rule-based macros and the case-based macros and we talked about the --- differences between those type of macros in our previous module, but what we can do is we can define something that has a rule that then has the infix keyword directly after it. What happens then is with our token we have a popup writer to indicate where the split happens or where the infix is. What happens then is we have something to the left hand side of that, so a in this simple concept over macro that I'm just expecting there to be something that I'll be capturing as a token on the left hand side or we're assigned to something called left, and then on the right hand side I would have the token that gets its sign to the token of right. In the form of a case-based macro the name of the macro is still provided as we talked about in the previous module, but with an infix macro the name of the macro will be the first argument to the right of the pipe uploader. Well if we've captured these different components as they are left in the right sections of the macro token set, we can then perform the same sort of macro templates that we would do with any of the previous macros that we've look at. Demo: Adding Infix to an id Macro Now let's have a look at how we can actually implement some infix macros. Again, we'll revert back to our simple example of the id macro. The id macro that I've got here is our standard one that we've seen many times before and it's not infix macro, so it expects something to be on the left hand side, so in this case the parentheses and the x, so that would have to go after the id name of the macro, but what if we wanted to do that prior to it, so if I wanted to move the parentheses in front of it. If I wanted to change this to here. Now if we pass that through the compiler, unsurprisingly, we've received an error because it doesn't have anything after the id that matches in this case, so what I would want to do is come up to the macro definition and change it so that we can have infix. If I then provide a pipe, and because I don't have anything that I want to capture that is after the name of the macro, I can just leave that blank. First to save that and run through the compiler you'll see that the macro is back working as the way we expected. I can mix and match between infix rules with inside of a macro, so I could define both an infix and a non-infix version of the id macro with inside of the same macro template. Demo: Post-Expression Boolean Statements Now let's have a look at something a little bit more advanced with the idea of an infix macro. You'll see here that what I'm defining is a macro that will execute an expression when a particular expression returns true, so if you've come from the CoffeeScript background or you've seen CoffeeScript before, this is something that is sort of common within CoffeeScript is to be able to define a Boolean comparison and after our statement is expected to be executed and then that same word that we executed in the case of that returns true, so I want to implement this with a Sweet.js macro. If we look at the way that this macro would be defined, so my macro in this case will be the when key _____. That I have an expression to the left and an expression to the right of that, so I want to capture those and then turn that into an if statement. The way that this macro would look is that I have my macro within a when and then I have a rule with an infix applied to it, and then I have something to the left hand side, which I've used a token class to specify to be an expression, so something that I'm going to capture as a token do, so what I want to do, and then on the right hand side I --- expect another expression and then I'll capture that as a condition, so this will then be expanded out by Sweet.js's macro engine into an if statement that if the condition is true, then --- we'll execute the line of code that we expect. If I'm to pass this through the compiler, you'll see that we have a value over this, obviously to 1 = 1, which will always return true, and we would be executing our statement as expected, Obviously this is our standard Javascript expression, so I could have this made up of variables or anything like that that I would expect to be able to return out to our first value or something that could be _____ coerced to true or false. This is just a simple idea of how we can use infix to maybe make our code more expressive that this is the statement that I want to execute, but only when this condition happens. Demo: Null-Guard via Infix The final infix macro that I want to look at is the ability to kind of a null guard with our code base, so something that we quite often do with Javascript because it's a dynamic language is that we might want to do a test to make sure that the properties all exist on an object as we kind of dive through it, so in this case I've define a variable that's called x that then has a property y, z, and then a and they're all nested within each other. I'm already going to be capturing the z component of that object, but I actually don't know whether or not I'll be able to find it. I mean I might not have a y property on there and in that case I might want to do something different. I might want that to be treated as undefined. I don't want it to be an error that happens because I've expected there to be a y that has a z, I want it to kind of fail early in this scenario, so the way I would implement this is by creating a macro that looks at the question mark dollars as the name of the macro and then doing a kind of a recursion across that to make sure that all the descendent properties of that all exist. Here's my implementation of my ?. operator. What I've said is that it's going to be an infix again, and that I expect something to be on the left hand side of that and I'm going to call this head, so this is the start of my object, and then after that I expect one or more properties to be represented by _____ we captured as $rest and then there'll be a dot that continues to separate them. What we're saying here is that we can also use the recursive nature of Sweet.js token capturing to then be combined with an infix operator, so I basically say everything after that will be a repetition of .properties. Internally this macro will then --- use recursive macros, so I've kind of split this out into a separate macro that it does code null_helper that first off we output the star or var object, so in our x component, and then it will pass in the x and then everything else after that to the null_helper that I've got defined at the top. The null_helper is then just going to recursively walk the dot separated version of our object chain and it will continuously output the && operator, so that we say that I expect this property and when that property exists then we'll expect the next property and then so on and so forth and recursively build that up. This just, as we've seen previously with recursive macros, it will just keep calling itself until the different --- token sets match, so initially it will start off on the second set of tokens and it will just keep working that one down until we have processed everything except for the last line and then it will just output all the processed versions, so x and then the && and then x.y && etc., and then finally we'll output the $z property, so that we will either fall _____ laterally because we don't have any more _____ truthy values or we will finally get to the end of our object chain. Now after I pass that through the compiler, just make the window a little bit bigger. You'll see how this has worked out _____ expects if there was a variable defined as x, then check if x has a y property, and then finally if the x's y property has a z property and then we'll assign that to foo. If at any point those properties don't exist it'll break out and we'll get undefined provided to foo and that's a way that we could simulate another check that we might have to manually code up if we wanted to introduce that to our inversion of our Sweet.js implementation. Conclusion In conclusion, we've had a look at token classes. We've seen by using the built-in token classes that come with Sweet.js, such as identifier, literal, and expression, we can add restrictions onto the types of tokens that we're expecting within in the patterns that we're creating for our macros. We've also had a look that we can use the macro class keyword to start defining our own custom token classes, so that we can have particular sections of our pattern that we want to match based off of a specified syntax that we have with inside of our application that we're building. We've also seen that we're able to extract the sub tokens of these token classes out by using a $ prefix and then the name of this sub token with inside of the token class. This means that we can create really custom token classes ourselves that then break down into smaller patterns that we can then extract the little pieces out of them. We've also had a look at the infix operator that we can provide through the Sweet.js macro. We've seen that by using the infix operator we can tell the macro pattern that we want to be able to split that down the middle so we can look at both the left and right hand sides of it based off of where the name lies. This gives us some good parallels, so that we can capture information that may proceed the macro name, so that we can make things that might be a bit more expressive, such as adding a Boolean condition on the end of a statement line, so that we can execute that statement only when a Boolean condition is true. Polyfilling ECMAScript6 Polyfilling ECMAScript 6 Features In this module we're going to be having a look at how we can polyfill some features of the next version of Javascript, that is ECMAScript6, using some Sweet.js macros. The obvious question is why would we want to do this necessarily. The main reason that we want to do this is there are some new language features that are coming into play with ES6 that we might want to use today, but they're actually introduced breaking syntax. We saw that with the default values for parameters that there's several other things that are also being introduced with the language that will be introducing syntax --- changes that old Javascript engines will simply be unable to parse, so we won't be able to use them with inside of browsers that don't support ES6, so we need some way that we either can use these features in browsers today or we have to discard them. Often times we want to continue to use these features in the browsers today because there's advantages to them that we don't want to have to just simply discard them and wait until their only in the browsers that we're going to be supporting because there's a chance that that could be a long time coming. There are two features in particular that I want to look at that we could ideally use today with inside of our Javascript programs, but we can't because they're going to be introducing new --- syntax changes. The first of these concepts is the fat arrow. This is something you might be familiar with if you've used CoffeeScript or TypeScript or many of the other compile to Javascript languages that what this introduces is a new symbol, which is the equals and then the greater than symbol that represents what's essentially our lambda statement. If you've come from a .NET background then this code might look reasonably familiar to you, but what it does is it allows you to create a shorthand for defining functions with inside of your Javascript. The functions that are defined, as fat arrow is, don't necessarily need to have an explicit return statement as we've got here in our second dot _____ that it's implicitly returning if there is only a single line to the statement. The final advantage of using the fat arrow is it changes the lexical scoping of the this value with inside of Javascript. The fact that this value changes within the usage of Javascript can be a lot of confusion, anxious confusion to a lot of new developers to the Javascript programming language, so the ability to have more _____itical usage of the way this works can be of advantage to introducing new developers to Javascript programming. The other feature that I want to look at introducing with a Sweet.js macro is the idea of doing classes. Classes are something that have been hotly debated within Javascript programming for many years. Javascript naturally isn't a class based language, instead it uses prototypal inheritance that with the next version of Javascript there is a fixture proposal that is to introduce a class syntax to the language itself. Again, this looks fairly similar to what you've seen in other transpilation languages, like CoffeeScript and TypeScript, but ultimately this will be built-in to the language, so we get the advantages of it being native, but if we wanted to use that today it's going to be breaking syntax, so we need to work out some way that we could be doing this in our --- programs today. Demo: Fat-Arrow Syntax Now let's have a look at how we can add the fat arrow syntax and classes using Sweet.js macros. I've got a file open here that has a sample usage of something I might want to do with the fat arrow syntax. You'll see that on line 1 I have defined an array that has the 0 to 9 as the values available within it and then on line 3 I want to find only the odd numbers in that array and then produce the squares of them, so I'm using the fat arrow syntax that, if you are coming from languages like C#, would look fairly similar to what you might be doing with lambda statements, enormous functions, and working with link, so we define that we're going to have an argument called x, then we have the fat arrow symbol, so equals then the greater than symbol, and then the expression that we want to evaluate. Because this is only a single line expression, I expect this to be the value that is returned, so the result of that expression is what will be returned by our fat arrow syntax, and then after I finish the filter method, I'm then passing that through to my map method, which does a similar thing, it defines an argument and then it translates this through an expression that I expect to return and I should get back a subset of the original items that we started with. What we're going to do is we're going to use the infix style rules that we looked at earlier in this course and combine them with other Sweet.js syntax we've worked with to produce a new syntax object, which is going to be the fat arrow that we can then produce a function at the end of it because ultimately at the end of the day what I want this to look like is the filter and the map method will both take a function that has an argument of x and then returns the expression that's within it. The first thing I need to do is create my macro definition. The thing that's going to be different about this though compared to other macros that we've done is we're not actually giving it a name which would be considered standard literal characters, so it's not a series of alphanumeric values, which we've used previously like id or overloading the function keyword that's already in Javascript. Instead, we're actually using symbols, an equal and a greater than sign, so to do this we have to define the macro name in a slightly different manner. What you'll notice is that the macro name this time, instead of just being the literal, is actually wrapped in parentheses. This is to help the Sweet.js compiler understand that what I've got inside of here is not some syntax that is coming from outside of Sweet.js or from Javascript itself, but this is going to be the actual macro name. It's going to be a collection of symbols instead of just a collection of literal values. If I was to be overloading --- any special characters with inside of Javascript, so if I was overloading the equality standard or if I wanted to boot up my own macro that is based off of symbols, such as equals, greater than, dots, question marks, etc. then I want to wrap those in parentheses to make sure that the Sweet.js compiler knows that that is the name of the macro, not just some additional syntax that might be, appearing, _____ is there accidentally or not with inside of my current lines of code. Now I'm going to create this as a rule-based macro, so I'm going to do this as a rule macro with an infix on it. I've also defined that the tokens of this infix rule macro are going to have some --- token classes defined on them. That's because I expect that the body of it in particular is going to be an expression. Because this is a single line use of fat arrow, I expect there to be an actual expression. It has to return something because the function that I'm going to be generating has to return a value, which is the result of that expression, and that makes sense because of the usage that we've got here and the way that it is defined in the ECMAScript6 specification. I'm also just specifying that I expect the argument to be provided as a single argument and then I expect it to be an identifier and not a value or anything like that just because otherwise we could end up with invalid Javascript syntax as the resulting generated code from Sweet.js. The final thing that I need to do is to actually define a function that will be going in place of my fat arrow statement, so here's the function that's going to be output from the Sweet.js compiler. You'll notice that I've actually wrapped this inside of a set of parentheses before the function and then after the bind statement that has been called. This is just to make sure that this function and Sweet.js understands that this function is the entire body and that that's kind of where the start and end will be of the scope of this output generated code. I've also defined that this function is going to have a name, it's called x. This is just for convenience that I could potentially use the name of this function inside of the body if I wanted to, but generally speaking you're not going to need to. It also just means that the function itself has a name, which means that they could be better identified by Javascript runtime environments. You'll also notice that I'm doing a bind(this) on line 5, which is after the function has been created. The reason I'm doing that is because the way the fat arrow syntax is defined in the next version of Javascript is very similar to what has come from CoffeeScript and other transpilation languages that have already introduced this syntax. The idea is that the fat arrow represents a change to the way the lexical scope in Javascript works. Whether this value is defined by the way the function has been called traditionally in Javascript, with the fat arrow it says that this scope of the statements inside of the fat arrow function are actually capturing the this of the parent scope, so I do a dot behind this to make sure that if I was using this with inside of my fat arrow statements I'm referring to the correct this as I would be expecting. Ultimately I'm not actually using those with any of the examples that I've got here, but it's still a good idea to make sure that we're following the convention in the macro because I might be using it in a wider set in my application, not just in this little sample that I've got here. If I save this and run this through the Sweet.js compiler you'll see on the right hand side here I've got my successfully generated code that I would be expecting from Sweet.js. First thing it's done is --- redefined the array with our hygienically renamed variable name, so items$511 instead of just items, which it previously was. We'll say the way I've got my indentation rules configured it has split all the items with inside the array onto separate lines, so that it now takes up multiple lines instead of being across a single line. There's ways that we could change this with the Sweet.js compiler and we'll have a look at that in the following module. Finally, on line 13 you'll see the generated or converted fat arrow statements into the statements that we were expecting from the actual running code. We'll see that we've first got a filter statement that has been defined and we've --- renamed a variable called x to be x$515 and then that has become our return statement of that variable mod 2, which is exactly what we expected. Because it was a single line statement we have expected it to be converted that one single line into an explicit return statement. Similarly, with the map statement to make sure that we are able to generate our squared value. Lastly, instead of being able to do squares as I'm doing on the map statement, I actually want to change that so that the map function or the map fat arrow function that I'm using is able to take two arguments. The way I want it to look is x and then comma i, so i being the position of the current item within the array, so that the index of that, and I'm going to instead of multiplying it x by itself, I'm going to multiply x by i. Now if I was to run this through the compiler you'll see that we get an error, because obviously we haven't defined a pattern that matches parentheses and then two parameters. I need to update my macro definition for fat arrow to support the concept --- to do multiple parameters to the function. The way that would look is very similar to what we've previously seen in examples, I've just instead of defining a single parameter, as I have on the first --- macro pattern definition, instead I've wrapped it in parentheses, so that there will _____ identifiers and then it is going to be a comma separated to repeat the 0 or more arguments. This also means that I could have no arguments provided to my fat arrow. I'd just use open and closed parentheses again, very similar to the way we would do our lambda statement with .NET and C#, so I would just have open and closed parentheses immediately and then followed by the fat arrow syntax and then an expression that is immediately returned. We obviously fill out the concept or the fat arrow, so that it is not just dealing with a single --- line to represent the body. Instead we have some curly braces either side and then multiple statements and then we could expect the body to --- have a … after that to do as 0 or more repetitions, so we have multiple statements potentially with inside the body, and that's the way that we could start filling out our fat arrow definition to be almost identically aligned to what we would be getting from what's coming in the next version of Javascript. The reason I say almost identically aligned is because we're still having to use standard functions instead of getting the optimizations that will be coming by actually using them natively with inside the Javascript runtimes, so it will be close to, but not exactly the same as having just the actual fat arrow syntax natively within the browser engine. So far _____ compile off you'll see that it has successfully compiled and if I resize the window on the right hand side you'll see that we've got what we were expecting. We've got the two arguments that have been provided to the map function and then we've got the multiplication has happened correctly with those two. Demo: Implementing Classes Here I've got open a class definition, written in Javascript, using the pattern defined as part of ECMAScript 6. The way that this works is we start off with a class keyword to indicate that this is going to be a Javascript class, followed by the name of that class. We then have curly braces and then inside of that we have constructors or _____ assume constructor and then multiple functions that represent the public instance methods that would be available from this class and the instances that are created from it. My constructor here I have defined that it's going to have two arguments that are passed to it, a firstName and a lastName. They're going to be assigned to public properties that'll be available off this firstName and the lastName property and then lastly, I have a method called sayHello. This takes a single argument called to and then the body of that is it just doesn't cause a _____ long statement using the argument that was passed in plus the firstName public member that we have created as part of our constructor. Down on line 12 you'll see that we've just newed up this instance of this class and passed in the two arguments that we've got and then we've invoked a public method on that, the sayHello method in this case. Let's take a look at how we'd write a Sweet.js macro that does something very complex for this, so it's going to be looking at this large chunk of code and be able to turn that into traditional Javascript. The first thing I need to do is create a macro that represents the class keyword because that's going to be our hook from Sweet.js. The skeleton of my macro will look like this. As I said, we're using the class as our keyword that's going to be breaking into the macro definition, so we start with that as the name of the macro, so the name of the macro will be class in this case. Inside of that we need to specify that we want to capture all the different tokens, so we're going to be interested in the name of the macro, the information that makes up the constructor, and then any functions that have been defined that'll be on the class definition itself. Let's have a look by first off defining the outline structure for the macro itself. The first thing we want to capture after the classes keyword itself is we want to capture the name of the class that we're creating. This is so that we can make sure that we create an object that is exposed using the expected name that has been defined and also so we get the proper hygienic renaming. What I've defined is that we're going to expect a token after the keyword of class that is $name. I then expect the starting and closing curly brackets because that will represent the outline script of the components that will make up the class that we're creating and they still want to capture the constructor of the class that we're creating. What I've done is I've defined that I expect there to be a block that starts with the keyword constructor has open and close parentheses, which has 0 or more arguments, and then curly brackets and then the body that can have 0 or more statements. You notice here something that we haven't seen previously in macros that we've defined, is we're actually introducing an entirely new keyword is expected at a particular location with inside of the macro that we've created. In this case I'm saying that I expect there to be a keyword of constructor somewhere with inside of this macro definition and that's how we're going to have it as a usage. This could be useful so that we can define the way that we want the internals of our pattern to look and how we can match to a specific part of the macro itself based off of other keywords we're introducing, things that we're not really addressing and capturing as tokens, and indicating this by not having the $ in front it, which also helps Sweet.js understand that by not having the $ these aren't actually tokens, these are just things that we're kind of ignoring, but by having these new keywords introduced we can give our code more of an expressive nature. Once I've captured the constructor, I then want to be able to capture one or more functions that would be exposed on the class itself. These functions I will be using in a repeating block, as we've seen previously with $, and then open and closed parentheses to wrap around a section that we expect to be repeating. This section will then be expected in the name of the method name that we've defined, so in the case of the example that we've got that'll be sayHello will be the mname token, and then 0 or more parameters that will be captured as a repetition and then the body for that function as 0 or more statements. I expect 0 or more of these to be created, so I can create a class that has no public methods on it or I can create a class that --- has multiple public methods on it. In the case that I've got in our present example I only have a single one, so we should expect that to be captured as a single method, and then exposed as a single method on the resulting class that we generate. Now let's have a look at how we could implement the template that will output this class for usage with inside of our application. The Person I'm going to use for making the template for this class definition is going to be the similar pattern that you will have been familiar with if you've done class based programming in Javascript before. I'm going to take the constructor that we've defined as part of our class definition and turn that into what is considered a constructor function in Javascript, so our function then would be expecting to invoke with the new operator. This will then create an object that represents the initial starting point for our class definition and then for all the public methods I'm going to be adding those to the prototype of the function. This means that whenever we create new instances of our class the methods that are then available on that are available via the prototypal inheritance chain that we get through Javascript. Also, by modifying the prototype to output these additional instance methods instead of adding them as part of the constructor means that we share the function between all instances of the class, so then if we updated it one place we can update all of those through the standard Javascript inheritance chain. Let's start off by creating the constructor method for our class. As I said, this is just going to be a standard function, but the expectation is that it'll be used using the new operator, so I take the name token that we captured at the start of our macro pattern definition, and create a function using that name. I then use the cparams as 0 or more repetitions to output all the arguments that we're passing through to our constructor function and then the cbody is the body segments that will be appearing as the constructors body itself. You're obviously going to use cparams and mparams and cbody and mbody as in the pattern that we've used to capture the components for this macro. There is enough down there, it's just to make sure that I can easily identify the difference between the parameters and the body that was for the constructor of our files, as opposed to the ones that he used for the methods that are then exposed on the class itself. The next thing to do is implement the methods which will be augmenting the prototype of this new function that we've created as part of our template. Again, we're using the repeating blocks that we've seen previously, so $ and then open and closed parentheses followed by the … syntax to indicate that this is a repetition set, and that repetition will be assigning $name, so the name of the function that we've created is our constructor function, and then .prototype because we're going to be modifying the prototype and then .mname, so the name of this function that we're creating. This means that we'll be building up what is a prototype property of the constructor function and then we're assigning that to an instance of a function that is created using the parameters and the body that was provided for the instance function that we've got. That's how we can take what looks very different from the initial code that we wrote using the macro, so the class definition that we created --- looks quite different to what we're going to have as resulting Javascript, but we're able to follow and output what is a very common Javascript design pattern. The way that we do classes in Javascript is quite often done this way and it's very easy that we can transform one type of syntax to something that is somewhat different, but still has the same logical constructs to it. The final step is to parse this through the Sweet.js compiler. With the compiler output appearing here on the right, I just expand the window open a little bit. You'll see that we've got what we're expecting. We've got a constructor function that is being created, which is person, but hygienically renamed obviously. We then have captured the arguments that are being provided out and expose them as public properties, so the firstName and the lastName. These haven't been hygienically renamed because they're something that we've created as part of the function that is being run rather than something that we're providing to the function itself, and then finally we have our prototype augmentation to create the sayHello property on this particular class and then the internals of that are what we're expecting again, from our usage. To finish off the code table, we've got our example implementation of this, so we've created a new instance of person, we've assigned it to a variable, and then we've invoked the public method that we've got on that. Let's take this code and execute this in some _____ just to make sure that it works the way that we're expecting. With my dev tools open here I'm just going to paste it into the console and now we'll execute that code. There we go. We'll say that we have created our instance of our person class, we've setup the prototype as expected, we were able to create new implementations of that, so the _____ variable that we've created, and then we're able to invoke the method on that and we get the console log statement that we were expecting from it. You'll see here that this is obviously very different code that we had initially to start with, something that is matching what is provided for ES6, but we're able to transpile that down into something that is available in browser runtimes today, so we can easily add --- features of the next version of the language to browsers and Javascript runtimes that currently don't support that. Demo: Creating a TicTacToe Game The next thing I want to have a look at is how we can take our macro syntax for a class and make a much more complex little application. What I've got here is a very simple tic tac toe game. It's not a particularly intelligent tic tac toe game, it uses random numbers to have two computer opponents play against each other, but ultimately at the end of the day we are able to output a potentially winning game. What I've done is I've created two classes. I've got one called Board that we've got on screen at the moment, and then if I scroll down a little bit we've got a class called Player. We'll start at that Player class and have a look at how it's defined. It looks very similar to the person that we had defined in the previous example of using classes, so we've got a constructor that takes a single argument that we assign to a public instance bubble, and then we have a single function that we're exposing from this called findMove. This takes in an argument called area, does some arithmetic inside of that to determine a move that could be played by the current player, and then returns that move back out. Back up in the Board we have, you'll see, something that's a little bit more complex. The first thing I'm doing inside of the constructor for this board is I'm defining what is a convention for a private variable, so this is not really a private variable because we'll still be able to access this if I know the name of it, but instead what I've done is I've used the underscore prefix. This is a common convention in Javascript just to say that hey, this variable or this member that is exposed, it's actually kind of internal and maybe you shouldn't be modifying it. I then defined a couple of public properties, so the size of the board, the area of the board, and then finally the two players that are going to be playing, our X and our O. I then have a couple of different methods that are exposed on this particular board. I have a playMove method, which takes a particular player. It will then find a move for that player and then play that on the board, found a move that that player can play. If the player tries to play into a position that is invalid, so someone else has already played into that position, it will then tell the player to try and find the move again. We have a method that will output the board to the screen and this just formats it with --- all the Xs and the Os in the appropriate location and then finally I have a method called play, which is actually the invocation of the game. First thing it does is it creates an empty board of the correct dimensions that we've specified. It will then start with player X and just go through every potential position on the board, so diagramming from 0 to the maximum area of the board because the area represents the number of available squares, and each player will take turns in playing their move and placing them on the board, and then finally we will print the output of the board and hopefully we'll have a successful result. Now let's pass this through the Sweet.js compiler and you'll see that we've got our generated Javascript as we kind of expected we've got from the previous example. You'll see that we've defined our board with the prototype and we've added the various methods onto the prototype for playing a move, creating the board, playing the game, and then finally I have to find a player that _____ play class and then we've exposed the method on that, and the very last thing I do is I create an instance of the board, our size is going to be 3x3 and then we tell the players to play the game. I'm back over in my browser here and I've posted out the generated content from the Javascript output, see down here in the console, and now let's tell our _____ to play the game. There we go. We've successfully played through our game and you'll see that both X and O have won. I might call this that our O has successfully won because they have the first full row. Like I said, this is not a particularly version of tic tac toe, as it just randomly generates a potential outcome. We can play through this multiple times and you'll see we get different outputs each time. What this is a good illustration of is a much more complex series of classes that are being used in conjunction together to generate a larger more complex application that has all been generated by Sweet.js macros. Demo: Extending Classes Now I want to take this one step further. I don't necessarily like the fact that to define my public and private members I have to use this and I then have to do particular conventions. I don't like that it's kind of not obvious if you're not familiar with Javascript coding conventions that the underscore represents something that is private and then non-underscores represent something that's public, so what I've done here is I've written some modifications to the way my constructor works, so that I have some new keywords. I have the keyword private and the keyword public appearing on lines 45 for private and then public on line 47 through to line 53. What I want this to do is to work the same way as I had with previously of exposing the appropriate methods and either making them public or private or at least as public or private as we can simulate with Javascript. What I'm going to do is I'm going to create a couple of new Sweet.js macros to represent these two new pieces of syntax, so to represent the public and the private, and what we'll see here is the ability to do --- several different Sweet.js macros all being used with inside of the one fire. First off I'm going to start with my private definition. You'll see here that I'm using a case-based macro instead of a rule-based macro and that's because I want to do something a bit more complex inside of the macro template. My definition is that I expect there to be something that I want to make private, that's going to be represented by the what token, and this will be an identifier and then my expected value, so something that is going to be assigned to what is going to be private. For the template I just scroll across a little. You'll see that I'm using the let syntax macro, so letstx that we saw earlier in the course, and what I'm doing with that is I'm creating a value that will represent a string that is the name of the property that we're trying to define as private, so what I'm doing is using the makeValue method that is exposed by Sweet.js for use inside of macro templates, to then create a value from --- the identifier that we've passed in. The way that I do that is then by using another function that is a value with inside of macro templates and that is unwrapSyntax. What that does out of the unwrapSyntax method is it allows us to take a syntax object, represented by the hash, the curly brackets, and then I'm passing through what I want to unwrap, so in this case the what identifier. That will then give me a syntax object that I can then create the value from. This will ultimately just return me the name of the --- string representation of the identifier that I've passed in, so the what_str variable will contain underscore blank as a string, so in the template that I'm returning from our private definition here is I'm going to be using the Object.defineProperty method. Object.defineProperty is something built-in to Javascript that allows me to create properties on a particular object, in this case I'm using the this object, which will be the this context of my constructor. I'm defining a property that will be underscore blank in our first example and then I'm giving it kind of a schema definition for the property I want to define. Because I want this to be a private property, I'm saying that it will not be an innumerable property, so in your rule is false. This means that if I was to do a for in statement against the instance of our class, this property will not become available. It won't be listed as a property that I can iterate through. The last thing I'm doing is I'm saying that the value I want this property to be assigned is the value that is on the right hand side of my --- assignment statement that is captured as part of my macro definition. This means that I'll be able to create a property. It will not be innumerable, so it won't be discoverable. The only way I will know it exists if I know the actual name of the property and it will have the value that I'm expecting. I can then expand this concept out to be able to produce a public keyword as well, so the public keyword looks pretty much identical as we have with the private, the only difference is that I'm not specifying whether or not the properties are _____ by default the property will be the newer property of our property schema, it will be true, which means that I will be out of _____. I could do a for in statement against my instance of whatever generating class I have and I will be able to find that particular property. Now if I was to pass this through the Sweet.js compiler and expand out our right hand side window, jump up to the top of the particular file that we're looking at. You'll see here _____ I have my Object.defineProperty for both my private and public properties and you see that some of these are computer values, such as I have here on line 7, which is size multiplied by size to represent the area or I might have something that represents the invocation of a constructor for our player, so on lines 8 and 9 I'm creating a new player based off of what has been previously defined as another class elsewhere with inside of my application. I'm going to take this, pass it through Javascript runtime, and we would get a similar output, as we will see in previously. The only difference is that I won't be able to grab my instance from my board and easily find the properties that are defined as private. That's it. That's how we can easily create some classes and then combine them with some additional Sweet.js macros to define our classes in a way that we've now gone beyond what would be available as part of ES6, but just made it a little bit easier to work with in the scenarios that we want to work with it. Conclusion In conclusion, we've had a look at how we can use Sweet.js to polyfill features that are coming in the next version of Javascript. We see that the advantage of this is that we're able to use syntax that is being introduced by the next version of Javascript that will be breaking within Javascript runtime environments that we've got at the moment, such as the new syntax for fat arrow and classes, but leverage that today by polyfilling the features that are being written for the next version. We're able to look at the advantages and we're able to deliver these features early on in our application development, be able to test them out to see if they fit for what we want to do with them, and provide feedback to the developers of the language about any issues that we might have. Operators Operators Overview This module we're going to have a look at how we can create our own custom operators with Sweet.js, as well as override existing operators in the Javascript programming language to be able to manipulate the functionality to be able to better suit the needs of the applications that we're building. The concept that I've been talking about a lot throughout this --- course so far is the idea of dealing with the main specific problems using Sweet.js. The scenario I want to put forward with custom operators is that we might be working on a specific domain feature that is unique to the problems that we're trying to solve. Generally what we'll do is we'll have some boilerplate code around this that will make that feature work, something that makes it _____ construe to us as Javascript developers, and then we'll finally use that feature with inside of our programs. What happens in here is we have a bit of a cyclic motion of we have these custom features that are for our problem domains, then we then have all this additional code to make it easier to work with, and we end up with code that is not quite as expressive as we might want it to be and we have all these other things that we have to take in as cognitive overload to understand exactly what is happening in the code that we're working with. With a custom operator or operators in general, what they allow us to do is take domain-specific features and have syntax to make use of, so that we can use those features. Arithmatic operations, for example, are examples that we have of domain-specific features that have been implemented with custom syntax. The way we add two values together, we have a special syntax for that, we have a plus operator. Multiplication, subtraction, division, all of these things are domain-specific features around how we do mathematical operations, but we have specific syntax. We don't have all these extension methods on a math object that we call, we don't call math.add to perform this operation, we have a syntax that will allow us to do that. One of the most common domain-specific problems that we have in Javascript is dealing with asynchronous code. The most common way that we deal with asynchronous code is we use this concept called promises. A promise is something that they've beta ran for a couple of years that I think gaining a lot more in popularity as the years have gone by and as the availability in the different Javascript libraries that we work with is increasing and it's to the point where it's actually something that's proposed for the next version of the Javascript language. The way a promise works is we might be performing something that's asynchronous. This example here I'm calling out to an AJAX endpoint that is returning me a list of people. From that getJSON function I will be returned a promise. The promise exposes a then method that I pass in a function that I want to execute when that asynchronous operation has completed. The then method itself then returns a promise that I can then chain onto another function, so I can add a secondary hold to then, which then takes the result of the first function invocation, so return result.slice(0, 5) can be then passed through to the second callback that we've defined. We have this method chaining that could happen and, as I said, this is a fairly common domain-specific problem we have in Javascript, overusing promises to deal with asynchronous code, but as you can see we have a lot of _____ applied around it. We have this then method that we have to make sure that we call, we have the function keyword to make sure that we know that we're using a function here, we have the curly braces, and then we are obviously chaining that to another function, so this is over quite a number of lines of code to achieve something that is fairly small and simple, but what if we could introduce some new syntax to make this easier? Here's the same concept of code that I've got and instead of calling out to what is obviously the promise library and calling a chain version of then and then and then, so that we can invoke multiple callbacks, what I've don't instead is I've introduced a new operator to represent that. This operator I'm going to call the rocket operator, which is the double equals and then the greater than symbol, and that then itself is expecting an expression, so just the same way as the then method exposed from the promise library expects an expression after it, my rocket operator also expects an expression You'll see here I've also combined it with the fat arrow syntax that we saw in the previous demos, so that I can have a very clean and concise and easy to read set of code about what's happening. I'm getting some JSON back, I'm then executing a couple of expressions against that, so the first is reducing our result set from however large it was to only having five items, and then passing that through to an expression of console.log. Ideally this will compile down to the same code that we had on the previous slide, so that we can then have that working with inside of our browsers. Operator Structure Custom operators have a new keyword that is introduces by Sweet.js, so like the macro and the macro class keyword have been introduces, we have one that is operator. The operator keyword says that the following piece of Sweet.js syntax is going to represent a custom operator that we're introducing, as opposed to a macro. The tokens that we're going to be using to use our new custom operator, so our rocket symbol that we had on our previous slide, that is what happens inside of the parentheses where the name goes, so a name doesn't necessarily have to be special characters, such as equals, greater thans, less thans, etc., we can also use the same kind of naming conventions that we would use with a macro we can also use for an operator. After we've defined the name we have to define a precedence for the operator that we're working with. The precedence of the operator allows the Sweet.js engine to determine when the operator is used and how high up in the chain of other operators that are being used, either custom operators introduced by Sweet.js or operators that have been just native to the Javascript programming language, and where our operator fits into that. We'll see more about how we can use precedence to increase and decrease and manipulate the way our operator works when we have a look at some demos in a moment. Precedence also has to be a numerical value, so a number between 0 and infinity essentially defines the precedence of the operator. The higher the precedence the earlier our operator will be parsed as opposed to other operators that are in the programming language that have Javascript or that we've added customly ourselves. Lastly, we need to specify the associativeness of the operator, so the associativeness works with the precedence of the operator to, when we have an operator of the same precedence, how our operator will be working with that one. Associativeness works particularly with arithmatic operators, so it's where the parentheses will be inserted, and we specify whether it is left or right associative, so when an operator is brought associative and it finds something with a similar precedence, it will then look at putting that to the right hand side of the previous operator, and not take to the left component of our operator will be up until the previous operator was used. If we had a plus and a minus operator the precedence --- is the same with those two operators, but the associativeness is working out which order to put the parentheses to make sure that we get the correct outcome. An associative, as I said, we specify by saying it's either left or right associative. We'll have a look more at that when we come to the demos in a moment. Finally, we need to specify what essentially be our pattern for our custom operator. Unlike --- patterns that we've had with macros in Sweet.js, we have a much more simplified version of what we get. All we can specify is what is going to be the left and right hand side or our operator, so operators are very similar to if we were doing a custom macro that was an infix macro and that both components of the infix macro were expressions, so the left and the right would both be expressions in that case, and that's what we get with our custom operators. We don't have to worry about where the --- name of the macro will appear with inside of our pattern and we also can't introduce anything custom inside of our pattern like we saw with classes where we introduced a constructor keyword. We only have the ability to capture a left and a right token. Finally, we specify a template that will be output as the result of this custom operator's invocation. We don't have any difference between our rule of _____ case space macro concept inside of operators, it's just what we specify in the template is exactly what's output, so it's very close to what we have with rule-based macros as opposed to case-based macros. We don't have the ability to manipulate the syntax tree during the operator processing. Demo: Overriding the Equality Operator Now let's have a look at how we can implement some custom operators in Sweet.js. The first bit of operator I want to have a look at with Sweet.js is the idea of doing operator overloading. Operators in Sweet.js, when we're introducing custom operators, don't necessarily have to be entirely new syntax or entirely new operators, it's simply catching and manipulating existing operators with the ability to change the way they work. As we know from the Javascript programming language, we have both a double and a triple equality statement, so we have type--- unsafe and type save equality that can be done. To a lot of new developers to Javascript this can be a bit of a foreign concept. Equality should represent equality naturally and that they don't realize that there's a difference between double and triple equality statements. What I've decided to do here is I want to make it easier for new programmers to come to Javascript and not have to understand that there is a difference between the two and that in my code I always want the equality statements to be of a type safe nature, so I want all equality statements to be done via the triple equal sign. What of them is _____ when people are using the double equals I want to overload that, so it automatically converts it up to a triple equal statement. What I've done here is I've defined that this is a custom operator on line 1. I've then specified that the operator that I'm creating or, in this case, that I'm overloading, is the double equals. I then said that this is going to be an operator precedence of 9. All operators that are built into the Javascript programming language have an operator precedence already assigned to them by Sweet.js. In the case of an equality statement, such as this, we have our precedence of 9. Some of the mathematical operations, such as multiplication and division, are much higher than that represented by the value 13. They're the highest precedent operators that we have in our programming language. If you have a look at the Sweet.js website, there is a full list of what the precedence of an operator is. We can always change the operator precedence as well as we create our overloading the existing operators. The next thing I'm saying is that I would expect this operator to be a left associative operator. Again, this is the default associativeness of the operator defined by Sweet.js, so I've just left that alone, I don't want to manipulate the way that the operator is treated by Sweet.js or indeed Javascript itself, I just want that to be left by default. I've then defined my pattern is a $l, $r and then my template is going to be $l, so the left hand side of my equality statement will be triple equals to the right hand side of my equality statement. On lines 4 to 8 you'll see that I have the usage of this. I have a variable that has been defined, this could be passed into a function or anything like that, and I'm expecting that I'm doing a _____ truthy operation inside of another part of our code based off of that variable. Now with Javascript the statement that I've got there of 1 == to true will return true because Javascript will do type conversions to make sure that these are both of the same type and one is a _____ truthy value, but because I want to make sure that all my statements are done in a type safe manner, my operator overload is going to change that to be a triple equals statement, so let's pass this through the Sweet.js compiler. You'll see on the right hand side that we've got our generated output. Our variable has been hygienically renamed and you'll see that our --- operator has been run and what was previously a double equality statement has now been overloaded to a triple equality statement. This can be very useful if we want to manipulate the way built-in operators work we want to enforce our own coding conventions based off of them, so in this example I want to do all equality statements as type safe. I could also do a similar thing for inequality statements. I could force them all up to be type safe inequality statements, as well as type safe equality statements in my code base. Demo: Creating a Power-Of Operator Now let's move along and have a look at how we could introduce a fully custom operator. In this case, I'm building something where I have a lot of mathematical operations that are happening, and while there's certain things that could be represented natively by Javascript syntax, such as multiplication, addition, subtraction, and division I don't have any way to represent powers with syntax in Javascript. I have to show up to the static method of Math.pow. While this is not really a problem in the final code, I want my code to look a bit more like a mathematical equation, as I've got on lines 4 and 6. I want 2 and then a symbol to represent a power statement and then to the power of a value. In this case I'm using just performing squared, so 2 to the power of 2, but I also want that to work within the scope of doing something that is already a mathematical equation, so introducing that with our multiplication symbol, so on line 6 I do 2 to the power of 2 multiplied by 2, so that should give me the value of 8. I don't want this to be accidentally to become 2 multiply by 2 and then --- the result of that to the power of 2 because that would be resulting in a potentially different outcome, so that's where our operator precedence and associativeness come into play. We're just running this through the compiler as I've got it currently defined, so you'll see that on the right hand side we've got our output as we were expecting it, our template of Math.pow, and then the left and right values has been executed correctly, so we either have Math.pow just 2, 2, and that's a value of 4 or 8 is 2 multiplied by the result of 2 to the power of 2. Now if I was to change the operator precedence from 14 to 12 because multiplication is a precedence of 13 itself, so this is now going to be a lower precedence than the operator of multiplication, when that runs through the Sweet.js compiler you'll see that the result is slightly different for the second usage of that, so where I was doing 2 multiplied by 2 to the power of 2, so I would expect that reading the statement in Sweet.js before it's been compiled is, I would expect that to become 2 --- multiplied by the result of 2 to the power of 2. Because my operator precedence has changed, now the left hand side of the expression is 2 multiplied by 2, and then that is passed as the all argument to my operator. Now if I was to change the operator precedence to be 13, so it's the same precedence as a multiplication, and pass that through the compiler you'll see that we don't really have any change. The --- resulting output is still incorrect from what our usage is, so I can change that by changing it from being a left associative to right associative. Now the associativeness has changed, if we pass that through the Sweet.js compiler you'll see that the result that we have now looks correct. We now have 2 --- multiplied the result of Math.pow, so that's the expectation that we have, that's where we would expect that parentheses to be inserted if we were just doing this as a pure mathematical operation. This is how we can manipulate both the operator precedence and the associativeness to change the way that our operators work. Demo: Overriding the in Operator I want to go back to revisiting the idea of overloading an existing operator. In this case, I'm actually going to overload an operator of in, so the in operator we can use to determine if a property name exists on an object, but coming from other programming languages, you might expect in to also work with a raise. When we work with in on an array it doesn't work the way that we would expect it, so in my example where I do 7 is in the array, we would expect that to return false because the value of 7 doesn't exist with inside of that array, and while, in this case, it will return false, the reason it returns false is because they're a third property of 7. What I want to do instead is make that if I use in and _____ that inside is actually an array, then I wanted to determine whether or not the value exists with inside of that array, not just whether there is a property of that array that is of that name. For this operator, as I said, I'm overloading in, and this is an example of how we can use uploaders that are not just symbols, so this is more inclined to our macro name, so in being a series of characters is the name of the --- operator that we're overriding. I'm also using the default precedence associativeness of this operator, which is 10 in left by default, and the template of this I'm using the Array.isArray method, so this will tell me whether or not the right hand side of the uploader is an array, and --- based on that I will either pass that through to an indexOf, which I'm then doing a bit right shift and then doing a type conversion to true or false. I could also do this as indexOf is greater than or equal to 0 because if an index doesn't exist for the value that's being passed in, it will return -1. If the value of the right hand side of this operator is not an array, then I just want to use the default usage of in, which is does a property of that name exist with inside of my object that I've got on the right hand side? If we pass this through the compiler _____, you'll see that we have the output Javascript as expected. Because I didn't define a variable for this array, so it has inserted the array three times, first to do the isArray check, then --- for the different sides of the ternary statement that we've got, but obviously if I had a variable we could have had this hygienically renamed and it would make the code a little bit simpler to read potentially, but you'll see that this is one way that we can make our code look a lot more expressive. We could assume that the left and the right hand sides of the in operator that are being passed in are something that are being passed in as the arguments to a function. Again, we're not actually going to know whether or not these are arrays or objects at the time their passed in, so we might have unexpected side effects by the fact that we're using in. We can make our code look a lot more expressive, but this is just one way that we can also overload a built-in operator to give it a very different usage depending on how the values that it defined for the operator work. Demo: Promise Operator The final operator that I want to have a look at is our rocket operator from the slides that we had previously. Here I've got the code that we had within our slides and I've got the operator defined that will work for this. I've also got my fat arrow macro defined at the top of this file, but I've just collapsed that down for brevities sake. For the record operator I've defined this is going to be an operator precedence of 12 because I want it to be fairly high up, but it doesn't have to be the most important operator that is available with inside of the expressions that we're dealing with, and it's also going to be a left associative operator. You'll see our usage down at the bottom is that we have the same as we had within our slide deck. We have our fat arrow that is taking an argument and then we have an operation that is performed inside of that that we return and then finally, we pass it through to console.log as a chained invocation. The way this operator works is very simple. We just expect that the left hand side of this operator is going to be a promise object itself, so we then just do $l and then .then because the then is the method that we expect to be exposed from our --- promise operator, and then we pass in the expression that was captured on the right hand side of this particular operator. We'll pass that through the Sweet.js compiler now and if I expand over the right hand side of the screen, we can see that the code that we've got generated is pretty much exactly what we were expecting to have generated. We have our then methods that are chained twice, we have our function that is converted from our fat arrow --- from an expression of a single line to something that is implicitly returned, and then we have our console.log that's passed through to the second then method, which because console.log itself is an expression, it's passed through as just a function invocation that will then get invoked itself. That's how we can take something that is a very domain-specific programming concept, promises, and introduce syntax that can make that a lot more expressive with inside of the code that we're working with. Conclusion In conclusion, we've had a look at how we can work with operators with Sweet.js from both the perspective of defining our own custom operators in Sweet.js using the operator syntax, providing a name or an identifier for the operator that we want to override giving the operator our precedence, so that Sweet.js knows when to use our operator before any other operators or whereabouts our operator will sit within the tree of operators that need to be evaluated, how we can use associativeness to determine how the groupings of the other paths of the expressions that we're working with are dealt with by Sweet.js, and finally, how we can build operators that can work with our own domain-specific problems that we have with Sweet.js. We've also _____ to use the operator syntax to not just create our own uploaders, but override existing operators that are available from the Javascript programming language, so we could do something like manipulate the way that equality works to ensure that we're always doing type safe equality in our Javascript code. Integrating Sweet.js External Macros Welcome to the final module in our Sweet.js Get Started course. In this module I want to talk about how we can integrate Sweet.js into an existing application that we might already have. The first thing I want to talk about is the area of using some external macros. Up until now the macros that we've created have always existed at the start of the files that we've been using them within, but obviously this has some limitations. One of the first things that I talked about at the very start of this course was that the idea of a macro-based programming language can have advantages over a normal transpilation language by the fact that we can do language composition, that we don't have to take on this full large language just to use one or two features of it. Instead we can build the language based off of the features that we want and the features that we need to design specifically for the problem domains that we're within. The main idea behind this is that macros that we're defining for Sweet.js should be modular. Obviously we're designing a macro for our application that doesn't necessarily mean that we want to use it within a single _____ ploycation in our code base. It wouldn't make sense for a lot of the macros that we'd be writing, particularly some of the ones that we looked at in the last module where we were doing things like overloading the double equality statement to be forcing a triple equality statement to do type safe equality or using things like fat arrows in classes. These are the kinds of things we probably want to be using across our whole application and not just within one file, so we don't want to have to write those every single time. The other thing we might want to do is share the macros that we've created with other users of Sweet.js or consume macros that other users of Sweet.js themselves have created inside of our own codebase via npm. The way that we would be doing this is that we can specify to the Sweet.js compiler, so that's the sjs command, that we have a module that exists and here is a particular location for it, so we do this with the dot chain command line switch. We can have multiple modules specified by multiple dot chain commands and this will allow us to build up a series of modules that we're going to be providing as external macros to a particular file that we're transforming at that given time. There's another way that we can do language composition with Sweet.js and this is by using require js from with inside of the node Sweet.js file. We can then require in the Sweet.js compiler and then load the macros programmatically. There are different reasons that we might want to have a look at these different forms of usage and that would mostly come down to how we want to actually execute the Sweet.js compilation. Do we want to do that as part of a build step or do we want to do that at runtime? Here I've got an example of doing an external macro in a separate file from where we're going to be consuming it. This file _____ was called id.sjs, it contains the id macro that we've been using throughout this course. The thing that's different about this file, as opposed to our other usages, instead of having the usage directly at the end of this file, instead I have an export command that then specifies the id or the name of the macro that I want to export. In this case, I'm saying export id because I want to expose the id macro from this file. The advantage of this is that I'm able to create private macros with inside of this particular file because I have to explicitly export the macros that I want others to be able to construe, so if I was using macro recursion, such as I was doing with the null _____ operator in our previous module, I might not want to have the helper macros there I've created exposed outside of this particular file, so by not --- including them in our export statement at the end these macros that I created will only be available with inside the scope of that one particular file. This can be useful if I don't want to have all the macros that I'm defining exposed publicly, I just want to have a couple that might actually be used as private macros that are only valid within the scope of the current file. Demo: Working With External Macros Now let's have a look at the existing application where we're using Sweet.js as modules instead of just within one file. Here I've got a very simple express.js application that I've created and I'm using Sweet.js as the foundation for the way that I'm developing it. You'll see at the top of my folder list I have a folder called macros and inside of that I've got a number of macros that I've created that are to go use with inside of this application. I've got my fat arrow that we've seen in previous examples. I have a couple of macros to make it easy to work with Sweet.js, which is what will be in --- .sjs. I also have our promise or rocket macro that I have from the previous module and finally, I have the when macro that we saw when we were doing infix operators. I'm just going to start by opening up our app.sjs file. This is the core of my --- express application. You'll see that --- if you've worked with express before that we have the r standard in require express and then we setup our application. I'm also requiring a couple of other things from node.js. Where this gets interesting is that I've actually created a macro to make it easier to work with express.js. Essentially, I've created a small dsl on top of express.js and Javascript to make the way that we work with it a little bit different. Instead of doing at.get as we normally would to specify a get route with express.js, instead I just have a get statement. I then specify a route that I want to get and then I've got a couple of different ways that I can then use that particular route. This first example I am using a using command, which says get this route using, and then essentially a function body that takes a recursive response where I can do something called _____, so for our courses wrap what I'm doing is I'm reading in a file from disk, which is a series of data, and then I'm returning that if it's valid. You also see a usage of the fat arrow on line 12, so instead of having a function that's defined I just use fat arrows to provide our under statements, and then I'm also using my run macro to do some error handling, so I want to return a 500 status when there was an error. If that was all successful, I will eventually return down the JSON that I read out of our particular file from disk. Just going to scroll down a bit in this file and show another example usage of our get macro. Here on line 38 you'll see that I have a different way of using it. Rather than having a using statement as the way that we want to use it, instead I do get the route, so in this case /, and then I'm telling it I want to render a particular file. I'm rendering out index.jade as I'm using the jade template language. What you'll notice about this file is if I scroll back up to the top, is nowhere have I explicitly told Sweet.js what macros are available with inside of this scope. I don't have any macros defined at all within this file in fact, I just have a series of macro usages. If we go over and I'll look it up, the get.sjs file, you'll see that I have a number of different things with inside of this that both are internal macros and internal macro token definitions and then finally, I have the macro that I'm defining and that I'm ultimately exposing, so we'll start down at the bottom with our get macro. The macro itself I've defined with a case-based macro from line 11 and there are a number of different cases in which this could be used. Finally, I'm just exporting that on the very last line of this file, export get with the token class that I defined, and also an internal macro called copy_context are both never exposed outside of this particular file. The way this macro works is that I have a number of different definitions folders, so this time I actually want to capture the name of the macro, so that's --- just the $m of the first of all the different cases that I've got. Then I've got a couple of different way I can use this. That will expect a path, which will be a literal, which is our --- string and then the path of the wrap that we want to get, and then I can either tell the macro that I want to do something, so I've got a $do, and that will be then returned as a direct res.$do statement, so that could be how I do my render or if I just wanted to send down some plain text I could just do get a path and then send and then the string that I want to send down as a response and then finally, on the first macro pattern that I've got as I have a literal as what I want to send down. This second case implementation that I've got here is I expected to have something that I'm going to do, but --- this time I expect the do to be of a particular token class, so I've made this one, in this case, as $what, and then that has a token class of status. This status will have the method that I'm going to do, so this is how I could do a 404, so I could do get and then error /404, and then I would say send and then in brackets 404 because I want to send that as the status code, and then some information to send down, so write Not Found at the end as the final literal. The last example usage of this in my case patterns is with the using statement. This time I have using and then I expect a simplified function body that has our requested response as identifies and then curly braces, which is the body of the grout handling that I want to include. As I said, I am actually using a private macro in here, which is the copy_context that I'm using with inside of each of the different macros templates that I've got defined. What that does is that uses the scope of the where this macro was called to create a variable called app that isn't being hygienically renamed. This is so that I can then use the app variable that was created in my app.sjs file that references the express.js application itself, and it will then ultimately --- boot up something that I will look valid in the resulting code. The way the copy_context macro works is that it's again, a case-based macro and expects some source context that I'm going to start with, and then what I want to generate a variable and then name of that. I start with passing in the original context, so the scope where this macro is initially defined, which is why I am actually capturing the name of the macro as $m. The reason I do this is so that when the Sweet.js compiler runs it's only to understand the parent scope of where the macro was defined, so that it knows that when it generates a variable called app that it also references the same variable called app that is existing in the external scope. We're then using the unwrapSyntax macro helper that we saw in the previous module to unwrap the variable that we've passed in, so that the variable of the external scope, and that is then being passed into a makeIdent function, so this is similar to our makeValue function that we've seen a couple times in previous modules, but instead of just creating a literal value we're creating a new identifier, and that means that we're going to be able to create an identifier call app, which is then added to the Sweet.js scope, and then it is able to track for hygienic renaming. We also have other macros, such as our run macro, which as I said, is an infix macro that allows us to do conditional statements on the end of a line and execute a particular _____ expression when something is passing the truthy statement. Demo: Using the Sweet.js Compiler via the CLI Let's have a look at how we can use the Sweet.js compiler from the command line. I've got a command line here open at the location where my code exists and I've already installed Sweet.js' node.js package. Normally I would do this by using the npm command. Because I want this to be a command that I can write from anywhere or a global command, I would be installing this using the install-g flag from npm. The package from npm that we're going to be installing is Sweet.js, so once that's installed I then have that available as an sjs command from the command line. Just executing it from the command line you'll see that it them prompts me because it doesn't have any commands that are being provided to it with all the available options that are there from the compiler. You'll see that we have the -m command that's available from the compiler, so we can --- specify where the external macros are on disk and make sure that they get loaded appropriately with the file that we're passing through the command. We can also specify if we wanted an output file, data file, Sweet.js we return this to standard _____, so we can pipe that to wherever we want or we could just use the -o command if we want to generate a file from that, and then there are other commands from the compiler, such as _____ to generate just the tokens or the AST, if we want to just get that out of Sweet.js or if we want to use source maps we can also specify a -c command because we want to use sourcemaps for Sweet.js, so that we can get rich debugging experiences with inside of our dev tools. What I'm going to do is execute the sjs command against our app.sjs file. This is the full command that I'll be executing for the app.sjs files to ensure that I load up all the modules that I've got, that contain all of my external macros, so you see that I specified that I have fat arrow, get, and when as the three external macros that I'm going to be using with inside of the app.js. If I execute this, as the default output of this is to start it out for whatever command line we're currently using. I can then also specify a -o flag if I want to generate an actual file. Now I've generated the actual file. If we go back to our text editor I now have an app.js that's available within the file or the files list, it can open that up, and you'll see that I have my outputs being generated by Sweet.js that's using all the macros that have then been compiled down into their valid Javascript. We'll see a lot of hygienic renaming has happened, such as express itself has been hygienically renamed to express$645, and also the app that we're actually using for all of our get routes, such as on line 7, you'll see that that has been hygienically renamed to app$646 and that is the same that was initially generated by calling express as a function on line 2. It's the same variable that's also been used for setting up our static routes and our foo engine and this is because we've been able to capture the context using the copy_context macro inside of the get sjs macro that I defined externally. This makes sure that we don't have to know the name or have Sweet.js, so now hard-coded to know what the name is that it's going to hygienically generate for objects that are exposed in our context, they are actually captured by understanding the full lexical scope that the macro has been used within. Now I can pass this through to node and execute my simple server. My server's up and running. It's listening on port 3000, so if I was to jump over to a browser we'll see that we've brought up our website that's got the default content that's being returned from our view. We still have one more step that would need to be done to make sure that this works correctly and that would be to generate the front-end or the client-side Javascript from Sweet.js. If we have a look here back in our text editor under my public folder I have a site.sjs file. This is a very simple file that is just doing a couple of AJAX requests to load in some information from one of the routes that we've exposed from our Sweet.js server. First thing I've done is I'm using the Q promise library that I've just downloaded from the internet and included in my application and then have created a very simple AJAX call using the default XMLHttpRequest object. Obviously this doesn't have the fallbacks for all the versions of ie, which used ActiveX, it's only using the current standard of how we would get the information out. What we're doing is we're wrapping everything with inside of a promise object, so that I'm using Q to return a promise, so that I can then use that with inside my application. If I scroll down a little bit in the window you'll see that I've got two AJAX events that have been defined. The first one reads in the full list of courses from my /courses wrap that my server has defined, I then pass that through to JSON.parse because I just get back the real string value, and then I pass that through an object that does a map and then a reduce to make sure that I append all the items into a ul that's appearing on screen. At the bottom I then have a click handler that I'm attaching to the ul and again, you'll see that I'm using fat arrow in this case, so here you will see a mix and match of we're using both fat arrows for some callbacks that are happening with inside of AJAX response or inside of a promise handling, and I'm also using a fat arrow with some event handling with inside of my Dom interactions. I've also brought in my rocket Sweet.js macro, so that I can simplify down the way that I'm working with promises, but obviously this is a front-end file, so I need to generate that to be able to have a proper Javascript file that I can consume on the client. To do this I'll just execute the sjs command against my site.sjs file with inside of my public folder and then generate a new file that's at public\site.js. You'll notice here that I've received an error and that error has happened because I've forgotten to include a module from the command line that I'm actually using in the file. Here I haven't defined the rocket operator, so what has happened is Sweet.js has interpreted part of the rocket operator that looks like the fat arrow, and thought that that should be a fat arrow, so it's kind of got confused. It's not realized that there is some additional stuff that I was actually missing from my initial command execution, so if I go back and update the command line invocation to now include the promise.sjs file you'll see that we've been able to successfully generate our public sjs file into our public js file. Let's run our node server back up again, and come back to our browser. If I reload the page you'll see that I've got some additional things that have happened. My AGAX call has gone out to the server and I've got back a list of information of Pluralsight courses that I've published that just list out on the screen. I can then interact with these and that does another AJAX call that brings in some additional information, but this whole process of compiling our different files has a bit of an overhead. As you saw, I had one instance where I missed including one of the external macros that I was wanting to include, but then on top of that I also had to keep rewriting the calls out to the compiler to make sure that I was compiling all the files down appropriately, so let's have a look at how we could simplify that and have a single script that will take care of all of that for us. I've created a new file with inside of my application called build.ps1 because at the moment I'm using partial as my command environment. Obviously we could make a file for whatever command line that we're working with, be it a show command from an OSX or Linux machine or I could create a bat file if I wanted to use a bat file on windows to execute the commands, but ultimately what I want is a single file where I can invoke all the different transformations of sjs files down to their equivalent js file from one particular location. You'll see here in this file that I've got two calls out to the sjs command, but again, because I'm using this on Windows I'm using the .cmd file, but obviously if you were using this on osx or Linux machine, then you would be calling down to the actual show version, and I'm also assuming that this instance of Sweet.js is included as a local node package, so I don't have to rely on that being publicly installed or globally installed on my machine, that I can have this included locally, as I've defined that as a dependency on my package.json file. I've also got all the different macros that I require for the different files that I'm generating, so you see that site.sjs only required the fat arrow and the promise, it's not going to need to get macro because it doesn't have the concept of routing from express because this is happening with inside of the browser. I'm also not using the when macro, so I don't have that on the list of macros that external macros that I'm passing into the path for that instance, and you'll see that for our app.sjs we have a larger set of macros because I have a different set of requirements that are happening on the server to what I've got happening on the client. Now I can run this power _____ strip file from the command line and it will go out and generate the files that we're expecting and that have just been written to disk in the locations that we expected, so we've generated our app.js and our public/slight.js. I can then run up my note application again, and it would look exactly the same as I had in the previous example where we were using the compiler directly from the command line. Demo: Node.js Macro Loader The final way I want to look at using external macros is by using the macro loader from node.js. Let's just reopen our app.sjs file and you'll notice that this time it looks quite different. What I've done is instead of having a command line library that we'd be using to generate our sjs file into our Javascript file is that I'm actually getting that to happen at runtime by node.js. First thing I do is I say require the sjs package, so this is essentially requiring the compiler to be loaded in. I then specify the macros that I want to get loaded into the compiler at this particular instance, so here I'm saying well I want to load the macro for fat arrow, get, promise, and when. The last thing I do is I specify the Sweet.js files that I want to then load in, so what I've done is I've taken all the content that was previously in app.js and moved them into a new file called wrap.js and and here you can see our wrap.js file looks exactly the same as our app.js file used to look. It's got all of the same routes defined and finally we're setting up our server at the end. The difference is instead of using the compiler to generate a file that we then execute, we just tell node.js to execute our app.sjs file and it will then take care of loading in the compiler itself and then executing, so we'll just remove the app.js file from disk, and now back at the command line I'll just use node to invoke the app.sjs file directly instead of app.js as we previously were (Typing). You'll see that our server has started up correctly. If you look in our file list in our text editor we don't have an app.js file on disk because this file has only been generated at runtime, it only exists within memory, and if I was to come to my browser and reload you'll see that our application is still running, so we've still got all the same advantages we had of generating the file and running it out to disk, but instead we've just generated --- our transformations at runtime, so we haven't had to have that step of writing those files to disk. There is a disadvantage of this though, is that this is only designed to work with node.js. If we're using Sweet.js to generate any kind of client-side files, such as we are in this application where we're generating a site.js file from a series of Sweet.js macros, we won't be able to do that because we can't invoke the compiler using the node.js runtime. We don't have the node.js runtime in the browser, so the way we would address this is we would either generate the files for our client using the Sweet.js compiler, either as a build process or as part of the application start or alternatively, we could use the Sweet.js compiler from within the browser. I'm not a particularly big fan of doing that, of using the Sweet.js compiler within the browser because it means that we're adding a considerable amount of overhead to --- our actually being able to run at Javascript client-side. First off we have to load the Sweet.js compiler into the browser. Once the Sweet.js compiler is loaded to the browser we then have to load all the different macros that we want as external macros that are available for a particular file. Once those files have all been loaded we can then finally load the files that we actually want to be doing a transform against, so you can see that this is a considerable amount of overhead that would be much better to do before a user has made a request to a page within our web application. Why Node.js Build Systems? Now that we've seen a way that we can manually invoke the Sweet.js compiler or manually control the compilation while the application is running, I want to talk about ways that we can integrate Sweet.js with a node.js build system. For that I want to talk about two different build systems for node.js, grunt and gulp. First off, why would we want to be doing this? Why would we want to be looking at a node.js build system and what are the advantages of that? Mostly, as I said, we don't want to be managing the compiler calls through Sweet.js ourselves. While our simple application we saw in the previous demo only had two files with inside of it, one that was a series of routes that designed our server for our application, and one which was a single client-side file. You can imagine that in a real world application we could have multiple files of both client and server that we would have to be building a compile step for. This could become a considerable amount of overhead because if we were to have multiple files we'd have to write a compiler call for each of those individual ones. We saw that even with only two files we're still looking at about half a dozen different lines of code that was to make sure that we had all of our different modules loaded, and then we're invoking the compiler gets the right file. This can start becoming a problem with the more macros that we're using and the more we're breaking down our macros into smaller external files that we've got a lot more macros that we have to make sure that we get loaded to every single time that we run our build script. Really we want to be able to reduce the amount of time we spend writing calls to the compiler and manually managing the loading of the different modules that we've also got for our external macros. All of this becomes overhead that we would have to be maintaining with inside of our applications and we want to try and avoid it. The other reason we wanted to look at things like grunt and gulp is if you're building a node.js application there's a good chance that you're already using one of these tools, so we will want to be introducing something that works with inside of our existing tool chain. Both grunt and gulp are very common in node.js applications, as they can be used for things that are very common for transforming and setting up our environments before our application runs. Grunt + Sweet.js First off, I want to talk about grunt. Grunt is a node.js task runner, so not just a transformation engine or anything like that. It's more about running abstract tasks with inside of a build process for a node.js application. This can be used to do things such as generate assets from files that we've got, Sweet.js being a kind of asset that we might want to generate files from. This could also include things like converting our css from a css pre-processor like sass or less or even in combining with a css pre-processor like prefixer if we wanted to make sure that we have all of our proper vender prefixes defined. We could also use grunt to do things like have UglifyJS go over our application and do some Javascript minification, so that we can send down only a single Javascript file that represents our overall response, rather than having it split across multiple files, and we'll also do optimizations like that. Grunt can also be used to automate our test running and run files for changes, so that while we're developing we don't have to continuously restart our build process. These are all the kinds of things that we can do with grunt and why we would want to be introducing Sweet.js into this is so that we have that built-in to our entire ecosystem of I've got a series of code that I've written that we need to transform into the final code that we're going to be running with inside of our application. With grunt we have a Sweet.js package that is called grunt-sweet.js, so we would normally install this via the npm command and then use --save-dev to make sure that we save this as a dependency, but only a development dependency into our package.json file, so that when we're storing this onto our production server we don't bring down all our _____ want modules, instead we should have all of that done as a pre-processor step before that we deployed into our production environment. The way we then use Sweet.js from a grunt application is that we specify inside of the config for grunt that we're using Sweet.js, so we specify a Sweet.js object inside of our grunt config, and then we can provide it with a series of options and then all the different types of --- file packages that we want to deal with. The options would include things like the external macros that we want to include, whether or not we want source maps, both for client and for node.js, and whether or not we want to enforce the hygienic renaming or we want to have it try and do some more intelligent renaming, so that we get more readable variable names. With inside of our sources set we would specify things like our node.js files that we want to transform and where we want to output those, as well as our client-side files and where we want to output those. Demo: Using Grunt Sweet.js Plugin Let's have a look at how we can use Sweet.js with grunt inside of our sample node.js application. Here we are back in our sample application and you'll see that I've added a new file to my file list called Gruntfile. This is the convention with grunt that our file name, Gruntfile, will exist and that is where we define all the tasks that we're going to have with inside of our grunt build system, so if I open up my Gruntfile here you'll see that I've got the Sweet.js config setup, so I've setup that I have Sweet.js and that the options that I've got is here is the different macros that I want to be importing from our external macro set, so I've got my fat arrow, my get, my promise, and my when. You can imagine that you might be using the node.js fs package to then scan a particular directory and then find all the files of a .sjs extension, so you don't have to hard code this. In this simple example I just wanted to illustrate that here's how we can list out the macros that we want to include. I've then defined two different file bundles, a client and a server file bundle, and where those files are located and I've also got some specific Sweet.js options at those particular levels. For the client I'm specifying that I want to generate source maps, so I specify the sourceMaps of true in the options, so this is combined to the overall Sweet.js option set. I say that the files are expected to be found in a public folder and are of a .sjs extension and that the destination is to back to the public folder and the extension in the generated code is to be a .js. I then have a server bundle that, in this case, I only have my app.sjs file, so I'm not worried about having a different set of files to be here or a directory scanner or anything like that, but I'm also combining it with a custom option for server-side modules, and that is that I want node.js source map support, so the grunt Sweet.js command allows you to break down between node and client-side source map generation. The final thing that I need to do is tell grunt that we're going to be using the grunt Sweet.js command, so I load that as from an npm task on line 40, and then finally I register a default task and specify that the tasks that I expect the default one to run are just our Sweet.js task itself. If I jump over to our command line again, I've already gone ahead and installed the grunt command globally, so normally I would do that with npminstall-g grunt-cli. This will then make grunt available from my command prompt, so that I can just execute it straight away. Now that we're ready to go I'll just execute grunt from our command line here and you'll see that what grunt has done is it's executed the Sweet.js common client command and then Sweet.js called the server, so this is where it has detected our different --- sets of files that we want to compile, so our public/site.sjs and then our app.sjs and Done the generation against those, so that it's successfully completed, there are no errors in there, and if you jump back to our text editor you'll see in the file list that we have our generated app.js and we have our generated site.js. I can then run the application from the command line again, (Typing) and our server is up and running. I can jump to the browser and reload and you'll see that the application is still up and running, this time with files that we've generated from grunt instead of from directly calling the command line, so this has greatly simplified the way we've been able to generate the Sweet.js files, so we haven't had to manually make sure that we have all the command lines I can then setup, and that we've made sure we cover off all the files we can use the power of grunt that can do things like directory scaling to make sure it finds all the different files that we want to compile in a single step. Gulp + Sweet.js Now that we've had a look at grunt I want to have a look at another node.js build system, this one called gulp. Gulp is a little bit different from the way grunt works, is instead of being a very arbitrary free command line runner where we define tasks that it just functions that execute, gulp is more designed around doing specific file transforms and piping files through multiple steps. What happens is we would find all of our .sjs files and then we'd pass them through a chain of commands, so we would pass them through the Sweet.js compiler, and then we would pass it through maybe uglify and finally, through another file to rename that and then finally, we pass it through to our destination directory. This is how a sample task would be setup with gulp in Sweet.js, so we have to first off install the gulp-sweet.js package and again, I'm saving that as a developer dependency inside of my package.json file, and then inside of our particular gulp task, so in this case the default task, I say that I'm using gulp and then saying that the source files are in a particular location and of .sjs, and then I use the gulp pipe command to pass that through Sweet.js and then finally, through to our destination folder. This will do our transforms and then pass them all the way through to our generated files and write them out to a particular location. Demo: Using Gulp Sweet.js Plugin Let's have a look at how we can setup gulp for Sweet.js in our simple application. Like grunt, gulp has a convention on the name of the file that all the tasks are going to be built into, so in gulps instance it's called gulpfile, as opposed to Gruntfile, which we had with grunt. You'll see that I've done a similar approach that I had with my gulp file of how we're doing our locating of the particular files. The first thing I'm doing is specifying all the different macros that again, I just have an array that's defined for the list of macros, but obviously you could get this list through various different methods, such as using a node.js package to do some directory scanning. Again, I've just gone and hard-coded for simplicities sake. The first thing I do after I define my list of external macros is I say that the source file I want to work with is the app.js file. I will pipe that through our Sweet.js gulp package and specify the list of external macros that I have and then finally, I pass that through a gulp rename package. The reason I have to do this is by default gulp will generate a file with the same name as the source input that you gave it, so this could be useful if you're outputting to an entirely separate folder. In my case, I'm putting out the generated Javascript files alongside my Sweet.js files, so I use the rename package to change it from a .sjs extension to a .js extension. The second _____ I am using a directory scanning to locate all of the sjs files inside of --- our public folder. Again, I --- pop that through the Sweet.js macro, and then the second time I use the rename macro, this time because I have one or more files that I'm transforming, I don't actually know how many I'll have explicitly. I provide a callback function to rename the change of the extension to be a .js. I then say that the destination will be to our public folder, so that they will sit on alongside the Sweet.js files that are with inside of our public _____ folder as well. Again, _____ we have to install a global command so that we can execute gulp from the command line, and that is using the gulp node.js package, which I do using npm install-g gulp. With gulp installed I can then just run gulp from the command line and it will automatically detect my gulp file and then the default task with inside of that (Typing). You see the output from gulp is a little bit different to what we had with grunt. First thing it does is it tells us what file that it's located, so it's located a gulp file in my current working directory. It's then said that it's executing the default command and this has been successfully completed at the end and we should have had our files generated. We can see back here in my file list that I have my app.js that's been correctly generated and my site.js has also been generated. We can also generate source maps from this package, it's just another property that we pass through as the object set to the Sweet.js command that we pipe through with gulp. Now back in my command line I can run my application again, using the file that's been generated from gulp, (Typing) and now back over in our browser it can reload and again, we still have our application running doing exactly what it's been doing in all the previous examples, but this time with a file that's been generated from gulp. Conclusion In conclusion of the module we've had a look at using external macros to split up the macros that we've defined with inside of our Sweet.js application across multiple files. Most likely, you're going to be creating macros that you want to be able to reuse in more than one file, so having them at the head of the file is not necessarily the best approach. _____ is saying that by doing this we can create private macros, so macros that are only valid within the scope of the file that they've been created. This could be useful if we've got things that we don't really want exposed publicly, that we don't want actually used as macros themselves, they only make sense within the context of that one particular macro file. We've had a look at how we can use the compiler to manually transform files using Sweet.js. We've also seen that we can write our own build scripts to do these kinds of transformations, but there can be some overhead in doing that ourselves that we have to make sure that we specify all the different commands for all the different files we want to transform, and this can become a fair bit of an overhead if we have a lot of files that we want to transform. Finally, we had a look at two different node.js task runners that are quite popular these days with node.js applications and how we can use both grunt or gulp to integrate Sweet.js into an existing application. The application that I was working with we weren't doing anything beyond the transforms of Sweet.js to Javascript, but you can imagine that if you were using a css pre-processor or you were doing some Javascript minification that we could combine either of these things to make sure that we're generating files that have been through our entire build process and then we're combining Sweet.js with the already existing tool chain. We're also seeing how this can reduce the effect that it takes to include external macros by the fact that we can define them once and that they can be used for all the different files that we're doing the transformers against. Course author Aaron Powell Aaron is a Senior Developer and Technical Web Specialist with Readify and Microsoft MVP for Internet Explorer Development. Aaron focuses mainly on front-end web development, with a passion for... Course info LevelBeginner Rating (14) My rating Duration3h 22m Released19 Jul 2014 Share course