What do you want to learn?
Skip to main content
Sweet.js: Get Started
by Aaron Powell
Start CourseBookmarkAdd to Channel
Table of contents
Overview of Sweet.js
What is Sweet.js?
What Are Macros?
How Can Macros Be Hygienic?
When I was talking about Sweet.js a few slides ago I mentioned that the language introduces hygienic macros, so what specifically makes a macro hygienic? Let's say we have this macro here. This is just some pseudo code of a make believe programming language, it's not actually Sweet.js syntax. The way that it works in our pseudo code here is we define that we've created our macro with the #macro syntax. We then give the macro a name or a construct of the way that we expect it to work, so inc and then some parentheses, an i, and then we close the parentheses, and then the output of that is a particular template. In this case, where I define a new variable called x, we're assigning a value of 0 and then we're incrementing the value of i. Here's how we could be using our pseudo coded macro from our make believe programming language. We create a value called x, we assigned it the value of 1, and then we call the increment macro. This goes through a macro compiler and becomes the following output, so you might see that there is a potential problem that's happening here. Let's have a look at it in another way. What we've done is we've created a macro that is going to change our syntax. It's going to introduce something new. In this case, it introduced a new variable, and that variable is called x, but what if we'd previously defined a variable called x. In the case of our simple example that we had there, we hadn't defined a variable called x, so we've introduced a variable called x, but we already defined it, so we've had a change to our code that was something that was completely unexpected. We didn't expect it to be introducing something that's clashing with existing code that we've got. Hygienic macros aim to solve this problem by doing some intelligent renaming of the syntax that they are introducing, so that it doesn't collide with anything that's existing with inside our code. Let's go back to _____ looking at our example and assume that the macro engine that we're passing through this time is a hygienic macro engine instead of just a plain macro engine. We've got the same code that we had it before. We've defined our variable called x, we've assigned it the value of 1, and then we've passed through our hygienic macro engine and this time, instead of just using the x as it was originally defined in the macro up on the first line, instead that variable has been renamed to be something that should be a lot more unique for the scope that it's working in. In this pseudo code example we've just appended the name of the macro, which is inc, and a $ to the variable. We could assume that it might also include a numerical counter, so that if we use this macro multiple times the counter increases, so we don't collide with previous usages of this macro, but again, this was more about a simple pseudo code example of how hygienic macro could be different to a standard macro, and this is the concept that Sweet.js does. It will be introducing new variable names over the top of the variables that we initially design to make sure that we don't have scope collisions or we accidentally redefine variables that we've already defined within side of our programming language. There are ways that we can tell the Sweet.js compiler that we don't want it to be introducing these more intelligent renaming. Instead, we want it to make the names a lot more friendly of the variables that it's generating, but this can also lead to problems, as it doesn't necessarily pick out every usage of the previously defined variables.
Why Use Sweet.js?
Macros vs. Functions
So in conclusion, we've had a look that Sweet.js is not necessarily a complete language replacement like some other transpilers that we find out there. Some examples I used were CoffeeScript and TypeScript as examples of languages that have a lot more features that you may or may not require in the programs that you're building. Sweet.js introduces the concept of hygienic macros. We've had a look at what makes a macro different from a hygienic macro by the use of intelligent renaming to make sure that we don't have scope collisions and the fact that macros can be used to introduce specialized programming language features that might only exist for the problem domains that we're working with, but by having these features inside our language they can make it easier to solve the problems that we are working with inside of our programs. Lastly, we talked about how macros can be modular. Instead of taking this large extended programing language and having all the features, we can pick and choose the features that are specific for what we want to do, and be able to build a program language that works better for the kinds of problems that we're solving.
Writing Your First Macro
Demo: Writing a Simple Macro
Demo: Consuming a Macro
I can do interesting things with a simple macro like this because it's capturing everything from id across to the right, so I can put some into the left hand side of that. Say I could assign that to a variable, so now I'm using my id macro, we're sending it to a variable, and if I pass that through the Sweet.js compiler, you'll see that the right hand side has updated to have my variable is then assigned the value of 42. You'll also notice that the variable I defined as x on line 5 in my simple.sjs file, but on the right hand side it is named x$511. This is because, as I referred to in the very first module in this course, Sweet.js is a hygienic macro engine, so it will do its best to ensure that the variable names don't collide with anything else. I believe in this example we don't have any other variables that could be called x. We're not using any variables with inside of our macro itself that has been called x. We're not redefining x as _____ with inside that program, so it's not really needed to do the hygienic renaming for us, but it's still good that it does that regardless of whether we needed it or not. It just means that it's not something we have to opt into, it's immediately done on our behalf.
Demo: Understanding Macro Patterns
I can do something else with a macro pen and I've called it x, so if I was to go up and change it from being x inside the pattern, and called it a, if I save that, but I don't update my template, and then we run this you'll notice on the right hand side the output is $x, so now it is varx$511 = $x because that's what our template is saying. We've said that we want to output something called $x. We haven't necessarily defined $x and, as far as --- the macro engine is concerned $x is something that might be coming elsewhere in scope, so Sweet.js isn't really concerned about where $x came from, it's just that's exactly what you told it to output. We can obviously change that to $a in our template. Do it again, and we'll be back to having the value of 42. The reason that I've prefixed these variables with $ is because that is kind of the common pattern to be used with Sweet.js, that pattern components that we're matching on inside of our rule or case pattern, we prefix with the $ just to indicate that they're something that has come from Sweet.js itself, not something that we're going to solely define with inside of our program. I can change the pattern on the macro itself to look a little bit different. I actually don't like the fact that I have id and then some white space and then the values being captured. Maybe I wanted this to look a bit more like a function, so let's come back to the pattern, and I was to wrap the $a in parentheses. Now if we pass that through the Sweet.js compile you'll see that we've actually got an error. If I just scroll up here in my apple window, you'll see that we've received a syntax error and the syntax error says that the macro id for the macro with the name that we've got specified, did not match its usage, which was 42 on line 5. This is because the pattern that we've defined expects parentheses, so now I would need to update my usage to include the parentheses. Now if I pass that back through the compile it's happy, we get our value, and again, we don't have any additional changes to it because we haven't changed the apple template, we've just changed the way that we're trying to match on this. You'll notice that I've still got white space between the id and then the parentheses. Sweet.js doesn't concern itself with how much white space. White space isn't something that we can match on with inside of our rule, so we can have as much white space kind of between the name of the macro that we're using, and then where the pattern starts, with the exception of things like new lines. Those could be captured in other manners, but we don't have to make sure that our rule takes into account that there has to be only one character of white space or it has to be no characters of white space. White spaces kind of collapse down with inside of the Sweet.js engine.
Demo: Values and Macro Patterns
Other things that I could do with my macro is, because this is just a simple value, I could pass it through to another function. Here I've said that I want to use my id macro and pass it through to console.log, which is a function available in both node.js and inside of the browser. When we run that through the compiler you'll see that we end up with console.log and 42, so the value is just returned from the macro engine. The template has said that we want to output the value directly, and it will just output it inline, so obviously we don't have to necessarily assign this to a variable. We can use it in whatever user scenario is useful for what we want to do. I also don't necessarily need to use a numerical identifier or a numerical value here for id macro. Instead I can make this an object, so now that it's an object that will be at port inline, as we would expect. I could make this an array, I could make this another macro, anything like that because we haven't put any kind of restrictions on the type of values that we're trying to match with inside of our pattern. We're just saying that our pattern needs fixed something with inside of parentheses and we're going to return whatever that something was, so what is passed in is kind of irrelevant. It's not struggling any particular type or any expected uses because I haven't opted into that. We'll have a look at how we can specify components of a pattern to be of particular types later on in the course.
Recapping Macro Workflows
Macros With Multiple Rules
Sometimes when we're creating rules for our Sweet.js macros we might not want to match on every single scenario or attempt to try and match every single scenario in a single pattern. In this case, we might want to have multiple patterns that we can do slightly different operations based off of the way the pattern was used. In this case, we might want to have multiple rules for a single macro and Sweet.js allows us to do that. The way it looks is we have a macro with the name and the same way that we did with the simple macro we looked at earlier on in this module that we can have multiple of these rule definitions and we can have a different pattern for each time that they're used. Each different pattern will then have its own template output, so that we can do slightly different things or some scenarios we might want to do very different things based off of the way that the pattern was matched, so this is how we can have different layers to our patterns and different layers to our macro that allow us to do different things. The patterns are matched for a best fit scenario, so it's not like it's a top down approach where the very first one that could potentially match the usage scenario is the one that's used. Sweet.js will try and find the one that matches exactly to the pattern that you've provided it and if it ultimately doesn't find any one that matches it will then raise an error, the same that we saw earlier on in this module, so we don't have to concern ourselves with the order in which the rule patterns are defined. They can be in whatever order works best for us and the way that we want to define that that ultimately it will just come down to what is the best pattern to match the usage scenario that we have.
Demo: Creating Macros With Multiple Rules
The next way we might want to use a macro that I want to have a look at is the idea of making a recursive macro. Sometimes we want a macro that can depend on other macros, so this way we have a each macro is very specific to doing a single task and if there's other things that it might want to be able to do or we might want to inject other syntax, then we could use other macros that can do that. Alternatively, we want on our _____ macro that actually recurses into itself, so this would be like a recursive function essentially, but recursing through the macro definition, so it can keep calling itself again, and again. These can be good scenarios that we can use to make sure that we have macros that are targeting a specific pattern and a specific template output that are very very small and isolated, so that they only have a single responsibility, and that if there's responsibilities beyond what we can do with inside of a small logical concept, we can break those out into separate macros. Now we'll go back to our code and have a look at how we can create a macro that is recursive. I've got my add macro from our last little code example here again, and I want to take this and make it into a recursive macro because I've got this bit of a problem. If I add on value or if I want to add two values together, then I've got patterns defined for that, but what if I was wanting to add a third value? I could always create an entirely new rule to match that, but it's starting to look a bit complex. What if I wanted to add 4 values, 5 values or 10 values together? The more values I want to add then obviously the more rules that I --- would have to create because they're all different patterns essentially, so this is where the idea of a recursive macro can start becoming very beneficial because I don't want to have to create a pattern to match every single usage scenario of my addition macro here, I want to be able to use it kind of for n number of arguments. Now we update my macro so that it is recursive, so what I've done is I've changed on line three to match on the pattern of $x $y … The … indicates that there is going to be 0 or more arguments expected after the y inside of this pattern that we can then pass through as the … again, through _____ template, so this time in our template we've got, instead of having just x + y, we go $x + add($y …). What this says is to the macro engine --- if we match on more than two arguments it's going to go x + and then it will call the add macro again, with y and then 0 or more arguments passed in, so if it was to just do on what we've got on line 6, which is add 1 and 2, the _____ that you'll see on the on the right hand side has not changed because what has happened it's gone and seen that the first time that we've run through we've hit the second macro template, so we've got $x $y and then that has gone through to the template and has gone $x + --- add and then $y and then … indicating all the arguments that appear after $y. In this case, there's actually no argument after $y, so this time we're just calling add with a single argument, which matches then the first rule in our template, so the rule on line 2, which then just returns the value of $y, which then results in 1 + 2. If I was to start putting some more arguments into our app and pass this again, through the Sweet.js compiler you'll see that now add all the arguments together because it's just become a recursive function essentially or realistically it's a recursive macro, so it's matched x, y, and then … has represented the 3, 4, and 5 on the first pass through, so this has become x + add 2 3 4 5, so again, this has matched the second rule usage of the add macro, which is this time x now equals the value of 2, y equals the value of 3, and then … represents 4 and 5. Then we go back through, and because we've got more than two arguments, oh I'm sorry, more than one argument, we've matched the second macro rule again, and we keep going until we get down to just the five being the value of y and nothing else after it, and then that will result in matching the very first rule pattern. We can see that this happens by if were to change the first rule again, to have + 0 to it and then run that through the macro engine, you'll see the + 0 is added after the 5, so it's only until we get to the 5 that we've actually hit the first part of our macro, so the very first rule pattern that we've defined, and the $0 is added to 5, which is then added to 4, which is 3, 2, 1, etc. etc. We've made our macro recursive by the usage of the …, so this is a special bit of syntax that Sweet.js allows you to use with inside of a raw pattern to indicate 0 or more arguments are matching after a particular section of a pattern.
Avoiding Macro Recursion
Demo: Avoiding Macro Recursion
Demo: Simple Case Macro
Now let's have a look at how we can create a case-based macro with Sweet.js. Back in my editor here I'm going to start with creating the id macro that we had in the very first rule-based macro that we looked at, so it looks very similar to the way the rule-based macro worked. We have our case, we have a pattern, and then we have the return, and then we have our usage of it. You'll notice though that in the pattern that I've defined I actually have another character before the x. I've used the underscore character to represent what's being captured here and what is in the underscore character is actually the name of the macro itself, so this will contain id. Generally speaking, you're probably not going to want to use the name of the macro with inside of the template itself that we're outputting, and as a common convention in Sweet.js we tend to replace the name of the macro if we wanted to kind of discard it as an underscore just so we can, it's kind of easy to overlook when we're looking at that pattern, and we also know that if as a usage underscore is something that you have kind of disinterested in, but after the name it works the same way as the pattern that we previously had, so I just have my $x and then I break into my template where I'm explicitly returning a syntax object that contains $x. If it passed through the Sweet.js compile you'll see that we get the value of 42 output on the right hand side. I can do all the same kinds of usages that I have in my rule-based macro, like assigning the result to a variable, and we'll see that that works the same way as we expected it to work from the rule-based macro where you get our hygienically renamed variable of x on the right hand side, which equals the result of calling our macro.
Demo: Manipulating Syntax in Case Macros
Classifying Macro Tokens
Welcome back to our Sweet.js Get Started course. In this third module we're going to be having a look at some advanced macros that we could write with Sweet.js and some advanced concepts that we can combine with Sweet.js macros to make them more powerful and easier to work with. The first advanced concept that I want to introduce is adding restrictions to the patterns that we're creating, so that we can make sure that we only are matching _____ be more specifically on the things that we want to match with inside of our pattern usage. The order of this is that we can create classes for the tokens that we're going to be working with, so the tokens are the variables that we're capturing with inside of our Sweet.js patterns. The reason we might want to do this is that we don't want to treat every single token with inside of our pattern in the same manner. By default we can use any kind of token classification with inside of our pattern and they'll be parsed as valid by Sweet.js, but we might not want that to happen. We might want to be able to treat the usage of a variable with inside of our Sweet.js macro differently to the way we do an assignment statement with inside the macro itself, so this is what we would want to use token classes to make sure that we do some restrictions and their incorrect usages can be caught at compile time. Ultimately the goal is that we can be able to do different things based on the types of tokens that we're capturing with inside of our macro, so some of these that we don't have to create as many different named macros to do what are similar things, instead we can have different types of patterns and classes of those tokens with inside the patterns to make sure that we're capturing and performing the operations we want to on the correct part of our macro definition.
Overview of Token Classes
Token Class Syntax
This is what a Sweet.js macro with a token class looks like. Going back to our initial definition of the id macro. With this definition of the id macro I've defined that I have a token named x, which will be the result that gets returned, but then I've used the token class syntax after that, so that's with the colon and then the class name is. In this case, I'd say that I expect x to be a literal, so you'll see down in the code at the bottom of it that I have indicated that there's a syntax error on the second line. That is because the first usage id(42) is valid because 42 is a literal, whereas the second usage of id(1 + 1) is going to result in a syntax error because 1 + 1 is an expression, it's going to give us a result back, so this is something that is not valid for the rule that we've defined for the current macro. Shortly we'll have a look at how we could modify this macro so that we can capture different types.
Custom Tokens Classes
Finally, with token classes that we have within Sweet.js there's actually a new feature that's been introduced as of the 0.6 release of Sweet.js and that is the ability to create our own token classes. This will be useful if you've got a particular bit of syntax that you want to capture and then break down the subparts of that as their own tokens and then get those out and use them with inside of your own macros. At the moment, as of version 0.6 of Sweet.js, this is still considered a highly experimental as it's also a new concept that only just made its way into the compiler, so it's something that is subject to change, but it has a lot of potential and we'll have a look at how we can use that with inside of our one code base. To use custom token classes what we do is we use the macro class keyword to define that this is going to be a macro class. After that we provide the name of the token class that we're going to be capturing and then we provide a pattern that matches it in the case that we want to use it. Then down in our macro at the bottom half of this code snippet we see that we use it by doing $x and then :name, so the same way that we would use any of the built-in macro classes, so like :lit to say that we're expecting a literal, I would use :name to say that I'm expecting my particular macro class or my particular token class to be used at this point.
Demo: Adding Tokens Restrictions
Now that we've got the base understanding of what a token class is let's have a look at how we can use that with inside of Sweet.js. Here I've got my code snippet from the slides before. You'll see that I have my id macro that's got a rule with x and that it says that it's expecting a literal and then I've got my two usages, one which is with a literal and then one which is without a literal. Now if I can run this through the compiler, you'll see that we get the error that we were expecting. If I just scroll up in the error information you'll see that we get a syntax error because, as Sweet.js says on line 7, the usage of the id macro cannot be matched with the pattern that we have defined. As I said, this is because I'm using the literal macro token class, so when I'm going to try to use an expression and that's invalid, so we haven't got a pattern that captures what we're expecting to use it as, so we don't have any way to use it in this example yet. If I was to modify my macro to add another rule to the macro definition I can than make a new macro pattern that is expecting an expression to be there, so this time I've got a second one, which is pretty much the same as the first one, but says that it's a $x:epr, so it is expecting the expression as the token that's valid in this scenario. If I run this through the compiler now you'll see that we get rid of the error that we had there previously and we get the expected output on the right hand side of the screen.
Demo: Different Templates With Macro Classes
Because we're watching each part of this macro rule differently based off of the argument types that we've provided it, so the different type of token classes, I can do different things with the resulting of the macro token class itself, so let's say I wanted to change the expression that if that is provided, I want to then add 1 to it. I could run this through the compiler and you'll see that the expression now is updated so that we get 1 + 1 + 1, which is what we expect because it was the second of our rules that were matched. From an external usage point of view of the way we consume our id macro, we don't have to know that anything is different about what we're providing to it. All we need to know is that when we provide it with different arguments we can get different results. If we weren't using tokens and token classes in this manner, we would have to create different id macros to represent the different kinds of usage that we wanted, so if we wanted an id where we resulted with a + 1 on the end, we would have to have a different name for that if we didn't want to put a restriction on the token that we had provided. Now let's move on and have a look at how we could create a custom token class with Sweet.js.
Demo: Custom Token Classes
Demo: Custom Token Classes With Multiple Patterns
Demo: Runtime Type-Checking
Finally, I want to have a look at another way we could use our macro token class to do something a little bit different with inside of our usage. I've used typescript before and I've referred to it earlier on in this course as one of the advantages that it gets from that is it has static typing built into it. What I want to do with this token class is kind of simulate the concept of static typing, so what I've done is that I've now created a way that we can define the type that I'm expecting for a token with an argument for a function, so what I've done is I've created a --- token class called typedef and what I expect is an identifier being the name of the parameter for our function and then I expect a literal value on the right hand side of the colon. What I'm going to be doing here then inside of our macro body is I'm going to be looking at the element that's passed in and if the argument is not of the type expected, so I'm doing a typeof check, then I'm going to throw an error, so this gives me some runtime testing of the values that are being passed in. This is the kind of thing that I might want to do in the debug version of my code base, so that I can capture through all the meta testing or just through user testing. Any incorrect usages of the functions without having to take on the full language of typescript to do compiler level testing on these. The reason that I've specified that I want it to be a literal as the right hand side of the colon with inside of my token class is so that I can do a correct type of a closing statement. We could do this in many different ways that I could say that I expect that to be the string number or any of the other ways that we can do type of detection, but in this case I just wanted to do a simple type of comparison. You'll also see that we've used our repeating block that we saw in the previous example, so $ and then a parentheses and then the block that we want to repeat. In this case I want to repeat an if statement, so if I was to have multiple parameters that were provided, then I could do a multiple different type of checks. If you pass this one through the Sweet.js compiler you'll see here that I have my typeof statement has been executed, so I have typeof, the argument that's passed in equals or if it doesn't equal the type of 0, so if the parameter for that is not a numerical value, then it's going to throw an error and provide some information, so it says that the typeof was not valid and it's not what was expected. Seeing the two usages that I've got down on lines 7 and 8, the first one would run successfully, and the second one we'll throw a runtime error. Let's have a look at that in action. Here I am back in my browser again, and if I just paste this into the console and then execute it, you'll see that we got log error, which is what I expected at the console.log statement to have been executed, and then the second one doesn't get looked at because it throws an error saying that the type of string was provided, but it expected the type to be a number, so this could be useful in terms of a debugging version of my code base, so that I could make sure that the parameters were all being used correctly, that I'm not getting strings being passed when I wanted to have numbers. We could also take this idea of doing the top of check to a higher level and rather than just doing a typeof we could maybe make that a Boolean function, --- a function that returns a Boolean value so that we can do something more complex than just saying is this a number, but maybe do a range test that I'm only expecting values between a particular range, so if I was doing pagination with inside of an ajex request, I would expect it to only allow to x number of pages or x number of records with inside of the page set. Let's all look at how we can create restrictions on the types of arguments that we've got or types of tokens that we've got with inside of macro definitions.
Demo: Adding Infix to an id Macro
Now let's have a look at how we can actually implement some infix macros. Again, we'll revert back to our simple example of the id macro. The id macro that I've got here is our standard one that we've seen many times before and it's not infix macro, so it expects something to be on the left hand side, so in this case the parentheses and the x, so that would have to go after the id name of the macro, but what if we wanted to do that prior to it, so if I wanted to move the parentheses in front of it. If I wanted to change this to here. Now if we pass that through the compiler, unsurprisingly, we've received an error because it doesn't have anything after the id that matches in this case, so what I would want to do is come up to the macro definition and change it so that we can have infix. If I then provide a pipe, and because I don't have anything that I want to capture that is after the name of the macro, I can just leave that blank. First to save that and run through the compiler you'll see that the macro is back working as the way we expected. I can mix and match between infix rules with inside of a macro, so I could define both an infix and a non-infix version of the id macro with inside of the same macro template.
Demo: Post-Expression Boolean Statements
Demo: Null-Guard via Infix
In conclusion, we've had a look at token classes. We've seen by using the built-in token classes that come with Sweet.js, such as identifier, literal, and expression, we can add restrictions onto the types of tokens that we're expecting within in the patterns that we're creating for our macros. We've also had a look that we can use the macro class keyword to start defining our own custom token classes, so that we can have particular sections of our pattern that we want to match based off of a specified syntax that we have with inside of our application that we're building. We've also seen that we're able to extract the sub tokens of these token classes out by using a $ prefix and then the name of this sub token with inside of the token class. This means that we can create really custom token classes ourselves that then break down into smaller patterns that we can then extract the little pieces out of them. We've also had a look at the infix operator that we can provide through the Sweet.js macro. We've seen that by using the infix operator we can tell the macro pattern that we want to be able to split that down the middle so we can look at both the left and right hand sides of it based off of where the name lies. This gives us some good parallels, so that we can capture information that may proceed the macro name, so that we can make things that might be a bit more expressive, such as adding a Boolean condition on the end of a statement line, so that we can execute that statement only when a Boolean condition is true.
Polyfilling ECMAScript 6 Features
Demo: Fat-Arrow Syntax
Demo: Implementing Classes
Demo: Creating a TicTacToe Game
Demo: Extending Classes
Demo: Overriding the Equality Operator
Demo: Creating a Power-Of Operator
Demo: Overriding the in Operator
Demo: Promise Operator
The final operator that I want to have a look at is our rocket operator from the slides that we had previously. Here I've got the code that we had within our slides and I've got the operator defined that will work for this. I've also got my fat arrow macro defined at the top of this file, but I've just collapsed that down for brevities sake. For the record operator I've defined this is going to be an operator precedence of 12 because I want it to be fairly high up, but it doesn't have to be the most important operator that is available with inside of the expressions that we're dealing with, and it's also going to be a left associative operator. You'll see our usage down at the bottom is that we have the same as we had within our slide deck. We have our fat arrow that is taking an argument and then we have an operation that is performed inside of that that we return and then finally, we pass it through to console.log as a chained invocation. The way this operator works is very simple. We just expect that the left hand side of this operator is going to be a promise object itself, so we then just do $l and then .then because the then is the method that we expect to be exposed from our --- promise operator, and then we pass in the expression that was captured on the right hand side of this particular operator. We'll pass that through the Sweet.js compiler now and if I expand over the right hand side of the screen, we can see that the code that we've got generated is pretty much exactly what we were expecting to have generated. We have our then methods that are chained twice, we have our function that is converted from our fat arrow --- from an expression of a single line to something that is implicitly returned, and then we have our console.log that's passed through to the second then method, which because console.log itself is an expression, it's passed through as just a function invocation that will then get invoked itself. That's how we can take something that is a very domain-specific programming concept, promises, and introduce syntax that can make that a lot more expressive with inside of the code that we're working with.
Welcome to the final module in our Sweet.js Get Started course. In this module I want to talk about how we can integrate Sweet.js into an existing application that we might already have. The first thing I want to talk about is the area of using some external macros. Up until now the macros that we've created have always existed at the start of the files that we've been using them within, but obviously this has some limitations. One of the first things that I talked about at the very start of this course was that the idea of a macro-based programming language can have advantages over a normal transpilation language by the fact that we can do language composition, that we don't have to take on this full large language just to use one or two features of it. Instead we can build the language based off of the features that we want and the features that we need to design specifically for the problem domains that we're within. The main idea behind this is that macros that we're defining for Sweet.js should be modular. Obviously we're designing a macro for our application that doesn't necessarily mean that we want to use it within a single _____ ploycation in our code base. It wouldn't make sense for a lot of the macros that we'd be writing, particularly some of the ones that we looked at in the last module where we were doing things like overloading the double equality statement to be forcing a triple equality statement to do type safe equality or using things like fat arrows in classes. These are the kinds of things we probably want to be using across our whole application and not just within one file, so we don't want to have to write those every single time. The other thing we might want to do is share the macros that we've created with other users of Sweet.js or consume macros that other users of Sweet.js themselves have created inside of our own codebase via npm. The way that we would be doing this is that we can specify to the Sweet.js compiler, so that's the sjs command, that we have a module that exists and here is a particular location for it, so we do this with the dot chain command line switch. We can have multiple modules specified by multiple dot chain commands and this will allow us to build up a series of modules that we're going to be providing as external macros to a particular file that we're transforming at that given time. There's another way that we can do language composition with Sweet.js and this is by using require js from with inside of the node Sweet.js file. We can then require in the Sweet.js compiler and then load the macros programmatically. There are different reasons that we might want to have a look at these different forms of usage and that would mostly come down to how we want to actually execute the Sweet.js compilation. Do we want to do that as part of a build step or do we want to do that at runtime? Here I've got an example of doing an external macro in a separate file from where we're going to be consuming it. This file _____ was called id.sjs, it contains the id macro that we've been using throughout this course. The thing that's different about this file, as opposed to our other usages, instead of having the usage directly at the end of this file, instead I have an export command that then specifies the id or the name of the macro that I want to export. In this case, I'm saying export id because I want to expose the id macro from this file. The advantage of this is that I'm able to create private macros with inside of this particular file because I have to explicitly export the macros that I want others to be able to construe, so if I was using macro recursion, such as I was doing with the null _____ operator in our previous module, I might not want to have the helper macros there I've created exposed outside of this particular file, so by not --- including them in our export statement at the end these macros that I created will only be available with inside the scope of that one particular file. This can be useful if I don't want to have all the macros that I'm defining exposed publicly, I just want to have a couple that might actually be used as private macros that are only valid within the scope of the current file.
Demo: Working With External Macros
Demo: Using the Sweet.js Compiler via the CLI
Demo: Node.js Macro Loader
Why Node.js Build Systems?
Now that we've seen a way that we can manually invoke the Sweet.js compiler or manually control the compilation while the application is running, I want to talk about ways that we can integrate Sweet.js with a node.js build system. For that I want to talk about two different build systems for node.js, grunt and gulp. First off, why would we want to be doing this? Why would we want to be looking at a node.js build system and what are the advantages of that? Mostly, as I said, we don't want to be managing the compiler calls through Sweet.js ourselves. While our simple application we saw in the previous demo only had two files with inside of it, one that was a series of routes that designed our server for our application, and one which was a single client-side file. You can imagine that in a real world application we could have multiple files of both client and server that we would have to be building a compile step for. This could become a considerable amount of overhead because if we were to have multiple files we'd have to write a compiler call for each of those individual ones. We saw that even with only two files we're still looking at about half a dozen different lines of code that was to make sure that we had all of our different modules loaded, and then we're invoking the compiler gets the right file. This can start becoming a problem with the more macros that we're using and the more we're breaking down our macros into smaller external files that we've got a lot more macros that we have to make sure that we get loaded to every single time that we run our build script. Really we want to be able to reduce the amount of time we spend writing calls to the compiler and manually managing the loading of the different modules that we've also got for our external macros. All of this becomes overhead that we would have to be maintaining with inside of our applications and we want to try and avoid it. The other reason we wanted to look at things like grunt and gulp is if you're building a node.js application there's a good chance that you're already using one of these tools, so we will want to be introducing something that works with inside of our existing tool chain. Both grunt and gulp are very common in node.js applications, as they can be used for things that are very common for transforming and setting up our environments before our application runs.
Grunt + Sweet.js
Demo: Using Grunt Sweet.js Plugin
Let's have a look at how we can use Sweet.js with grunt inside of our sample node.js application. Here we are back in our sample application and you'll see that I've added a new file to my file list called Gruntfile. This is the convention with grunt that our file name, Gruntfile, will exist and that is where we define all the tasks that we're going to have with inside of our grunt build system, so if I open up my Gruntfile here you'll see that I've got the Sweet.js config setup, so I've setup that I have Sweet.js and that the options that I've got is here is the different macros that I want to be importing from our external macro set, so I've got my fat arrow, my get, my promise, and my when. You can imagine that you might be using the node.js fs package to then scan a particular directory and then find all the files of a .sjs extension, so you don't have to hard code this. In this simple example I just wanted to illustrate that here's how we can list out the macros that we want to include. I've then defined two different file bundles, a client and a server file bundle, and where those files are located and I've also got some specific Sweet.js options at those particular levels. For the client I'm specifying that I want to generate source maps, so I specify the sourceMaps of true in the options, so this is combined to the overall Sweet.js option set. I say that the files are expected to be found in a public folder and are of a .sjs extension and that the destination is to back to the public folder and the extension in the generated code is to be a .js. I then have a server bundle that, in this case, I only have my app.sjs file, so I'm not worried about having a different set of files to be here or a directory scanner or anything like that, but I'm also combining it with a custom option for server-side modules, and that is that I want node.js source map support, so the grunt Sweet.js command allows you to break down between node and client-side source map generation. The final thing that I need to do is tell grunt that we're going to be using the grunt Sweet.js command, so I load that as from an npm task on line 40, and then finally I register a default task and specify that the tasks that I expect the default one to run are just our Sweet.js task itself. If I jump over to our command line again, I've already gone ahead and installed the grunt command globally, so normally I would do that with npminstall-g grunt-cli. This will then make grunt available from my command prompt, so that I can just execute it straight away. Now that we're ready to go I'll just execute grunt from our command line here and you'll see that what grunt has done is it's executed the Sweet.js common client command and then Sweet.js called the server, so this is where it has detected our different --- sets of files that we want to compile, so our public/site.sjs and then our app.sjs and Done the generation against those, so that it's successfully completed, there are no errors in there, and if you jump back to our text editor you'll see in the file list that we have our generated app.js and we have our generated site.js. I can then run the application from the command line again, (Typing) and our server is up and running. I can jump to the browser and reload and you'll see that the application is still up and running, this time with files that we've generated from grunt instead of from directly calling the command line, so this has greatly simplified the way we've been able to generate the Sweet.js files, so we haven't had to manually make sure that we have all the command lines I can then setup, and that we've made sure we cover off all the files we can use the power of grunt that can do things like directory scaling to make sure it finds all the different files that we want to compile in a single step.
Gulp + Sweet.js
Now that we've had a look at grunt I want to have a look at another node.js build system, this one called gulp. Gulp is a little bit different from the way grunt works, is instead of being a very arbitrary free command line runner where we define tasks that it just functions that execute, gulp is more designed around doing specific file transforms and piping files through multiple steps. What happens is we would find all of our .sjs files and then we'd pass them through a chain of commands, so we would pass them through the Sweet.js compiler, and then we would pass it through maybe uglify and finally, through another file to rename that and then finally, we pass it through to our destination directory. This is how a sample task would be setup with gulp in Sweet.js, so we have to first off install the gulp-sweet.js package and again, I'm saving that as a developer dependency inside of my package.json file, and then inside of our particular gulp task, so in this case the default task, I say that I'm using gulp and then saying that the source files are in a particular location and of .sjs, and then I use the gulp pipe command to pass that through Sweet.js and then finally, through to our destination folder. This will do our transforms and then pass them all the way through to our generated files and write them out to a particular location.
Demo: Using Gulp Sweet.js Plugin
Aaron is a Senior Developer and Technical Web Specialist with Readify and Microsoft MVP for Internet Explorer Development. Aaron focuses mainly on front-end web development, with a passion for...
Released19 Jul 2014