Tag Archives: Automation

Using gulp plugins to transform files

In the last post, I explained how to use tasks, but I didn’t explain yet how to make them useful. Tasks are most useful when they automate certain file operations, and gulp helps with reading and writing files. However, if you need to perform a transformation, then you need to look for plugins that can get that work done for you.

In this post you’ll learn how to set up and use various plugins that can automate some common file operations.

As gulp documentation says: Gulp plugins are Node Transform Streams that encapsulate common behavior to transform files in a pipeline – often placed between src() and dest() using the .pipe() method.

Let’s imagine that:

  • You are creating a control add-in and you are writing a lot of JavaScript
  • You want to isolate your JavaScript features into individual files, one file per feature, and that you want to combine these files all into one file that will be included in your control add-in
  • You want to write modern JavaScript, but still want to make sure all browsers will be able to run it (remember, anybody using earlier versions of Windows than Windows 10, and using the NAV/BC universal client, will be running Internet Explorer in there)
  • You want to minify the resulting file to consume space
  • You want to zip the contents of your control add-in into the Resource.zip file

All valid assumptions, correct? If you are anything like me, this is what you do every day.

Now, some of these things can be done manually, some not really, but all of them certainly can be automated using gulp. Our job would need to contain the following transformations:

  1. Bundle JavaScript files into a single file
  2. Transpile that file into a lower version of JavaScript
  3. Minify that file
  4. Package the contents of the resource directory to produce the Resource.zip file

The more astute of you will immediately notice that all of this cannot be one task. Unless you want to complicate your life, one task should only perform those operations that can be handled with a single pipeline. A single pipeline is what starts with a stream of input files and transforms them until you cannot transform it. This means that as long as the output of the previous operation can be the input into the next operation, you can pipe that output down the pipeline. Packaging the contents of the resource directory is an operation that cannot use the output of the previous operation, so you need to isolate it into a separate task.

Another reason why you may want to separate operations into individual tasks is when an operation makes sense on its own. If you can imagine any of the previous operations as something you’d ever want to run individually, then that operation should also be isolated into a separate task. Tasks can be combined in different ways, and I’ll address that as well in one of the future posts, so don’t be afraid to split up operations that can benefit from splitting.

In my example, packaging the Resource.zip file is an operation that can be done independently of everything else. JavaScript is not the only content of the resource file: you may change CSS, you may add or modify images, configure the manifest, you name it. In all these situations, you may want to package the resource file independent of the JavaScript operation. It just makes sense to turn it into a task of its own.

Good, so we have two tasks then, with the following flow:

  1. Preparing JavaScript file
    1. Collect all JavaScript files into a single stream
    2. Pipe that into a function that combines all the files into one
    3. Pipe that into a function that transpiles your code to a lower version
    4. Pipe that into a function that minifies your code
    5. Store the result into a destination inside the resource subdirectory
  2. Zip the content of your resource subdirectory
    1. Collect all files underneath the resource subdirectory into a single stream
    2. Pipe that into a function that zips the content of the stream
    3. Store the result into a destination directory

Good. Now that we know what steps we need to do, let’s create the two tasks with minimum operations that gulp can do on its own:

So far, so good.

Let’s focus now on JavaScript task first. The first operation we need to do on the files is to bundle (or concatenate) them. If bundling the files was the only operation you need, then you wouldn’t really need a plugin as Node.js contains all the necessary APIs for that. However, if your bunding is just a pass-through operation in a pipeline, you’ll need a plugin. Let’s look for a plugin.

Normally, you’d do a general google search, but gulp maintains its own plugin directory which makes your life much easier. Go to https://gulpjs.com/plugins/ and search for “concatenate”, you’ll find a number of plugins. For my example, I chose gulp-concat. When I need a plugin, I look in its documentation to see whether it can do everything I need, and how simple it is to use. I also look at its github repository (all of plugins I used have a github repository) where I can see how much “alive” it is: how many commits and how often, are there pull requests merged into it, how many forks are out there, is the author responding to issues, etc. All of it can contribute to a perceived reliability rating that finally makes me choose one plugin over another.

Good. Now that we know that gulp-concat plugin can do the job for you, how do you use it?

If you remember the first post in the series, you can remember how you imported gulp itself into your workspace: by using npm. Gulp plugins are typically package, and to import them into your workspace, you’ll also use npm. Just remember, any gulp plugin packages that you import are dependencies in your development environment, therefore you must use the –save-dev option. Let’s import gulp-concat:

Now that gulp-concat is a part of your solution and under the management of npm, you can use it inside your gulpfile.js script:

Now, we can use the concat in the pipeline like this:

The next step is to transpile that using babel. Again, a little search should help you find the gulp-babel plugin that you can then first install into your workspace. However, if you simply use this, you may realize that it does not work.

Some plugins use other packages that they don’t bundle directly, but allow you to choose specific versions of those packages, and sometimes they will provide different install scripts. To install gulp-babel with the necessary dependencies, you should use this:

… then declare inside the gulpfile.js script:

… and finally pipe the concatenated file into it:

Good job! Finally, we are ready for the last step of the first task – the minification. A quick search should reveal any number of possibilities, but I’ll go with this one:

When it’s installed, declare it:

… and then pipe the transpilation results into it:

And you are finished. If you did everything correctly, this is now your first task:

For the second task we need to locate a plugin that can zip a stream of files. Again, search the gulp plugin catalog, and you should discover gulp-zip. You are an expert by now, so you know that you first need to install it:

… then declare it:

… and finally use it in your zip pipeline:

If you did everything correctly, this is the second task:

Perfect. The only thing that’s missing is exporting the tasks from your gulpfile.js:

You can try to see if these two tasks now work:

Take a look inside your resource/Script directory and you should fine the controlAddIn.js script in there.

Then, run this:

Now take a look inside the build directory and you should fine the Resource.zip file in there.

Cool, but now we need another task that first invokes the javaScript task and then the resourceZip task in sequence. Luckily, gulp helps with that, too. The gulp.series() method creates a task that runs specified tasks in a serial sequence, one after another. Just export one more task from your gulpfile.js:

If you now delete the files that previous two tests created, and then run this:

… you will see that it has correctly built your Resource.zip file.

Just in case, here’s my latest state of the gulpfile.js:

In my next post, I’ll cover two more topics: deploying the control add-in to an NAV / Business Central instance using PowerShell from a gulp task, and passing configuration to gulp tasks from JSON configuration files.

Writing gulp tasks

Now that you know how to get gulp up and running in VS Code, and how to export tasks from your gulp file, let’s talk about tasks themselves. What are they, what are they good for, and how to write them.

Gulp – being a task automation tool – is all about tasks. Anything you do about programming, which is not writing the code itself, can be considered a task, and can very likely be automated. Some tasks are automated already, like compiling your code, building your app. In case of AL development, there are many more already automated tasks, like creating a demo workspace or deploying your app to NAV. If you can automate them, tasks are immensely powerful because they save time and eliminate error.

If you are a web developer, your tasks can include bundling your source files and minifying them. For me, when building pre-2018 control add-ins, most important tasks were bundling source files, zipping the resource file, deploying a control add-in and importing the control add-in using PowerShell. All of these I have automated using gulp, and I’ll explain every single one of them in this series.

But for now, let’s simply take a look at how to write gulp tasks.

Continue reading Writing gulp tasks

Exporting tasks from gulpfile.js

In my last post I covered the very basics of getting started with gulp in VS Code. It was all about getting gulp up and running under the hood of VS Code, and about writing your very first gulp task. It didn’t do much, but it showed you what gulpfile.js is good for and how to export tasks from it, that you can later use from VS Code.

However, I mentioned that there is much more to exporting tasks than it seems on the surface, and I promised to blog more about it. So here we are. Tasks.

Continue reading Exporting tasks from gulpfile.js

Getting started with gulp and VS Code

Strange times. I never stop to be amazed with the kinds tools that invaded our pristine development environments. One of those tools is gulp, and if you haven’t heard of it, don’t worry. Why should you hear of it?

A couple of weeks ago I introduced my talk on gulp by explaining how I made it simplify and speed up my development of control add-ins. If you are a real NAV developer, and if you didn’t classify this entire series on gulp into the “who cares” category, then chances are you’d stick with me today and learn how to set up gulp and start using it in VS Code.

In this post, I’ll explain what (and why) you need to do to get gulp up and running, and then I’ll create a very simple gulp task. This is the first post in a series where I hope to build an end-to-end example that you can even put to practical use in your projects.

Let’s get started.

Continue reading Getting started with gulp and VS Code

Automating control add-in development using gulp

Happy new year everyone! Last year was a bit slow for me, until the very end, when I churned out a year’s worth of posts just in two days. I hope this year to be a bit different, and let me kick-start it with a concept I’ve been playing with recently.

If you are into control add-ins development, chances are you’re familiar with my Visual Studio Project Template for control add-ins. It’s cool because it allows you to deploy all of your changes all the way over to NAV just by pressing F6. But it sucks because it’s based on Visual Studio, which is so… not 2019. It’s true, Microsoft never prescribed exactly which tool you should use to build your control add-ins once you’ve created the interface. It was entirely up to you. For me, Visual Studio used to be the tool of choice because in there you create the interface, and then why not just create everything in there.

But recently, I thought – why not using VS Code to develop control add-ins “v1” (that is: the control add-ins that work in pre-AL, pre-extensions “v2”, pre-NAV2018/BC environments)? If you, like me, still have to do those from time to time, but absolutely want to use VS Code instead of Visual Studio, then this post is for you.

Last month, I’ve decided to port all development work on a major control add-in from Visual Studio into VS Code. When I say “major”, I mean it: it’s 16K lines of JavaScript code in 138 source code files, not including 3rd party libraries. To automate that in Visual Studio, I used a hodge-podge of build tasks, Windows batch scripts, PowerShell scripts, external utilities for zipping and minification, and the first thing I did – stupid, I know – when I transferred all my work to VS Code was to automate all of my tasks exactly the same way. So, I took all the same batch scripts, configured them as VS Code tasks, and I was happy. Not. It was slow, and ugly.

And then I remembered gulp.

It’s funny, gulp should have been the first thing to cross my mind. But, while I knew it was there, I never did anything with it. But, how complicated can it be? It turned out, I was able to replace all of the automation I had earlier into one nice gulpfile.js and turn my entire development experience completely upside-down. It’s that good.

So, I decided to share a little bit of my learning path, lessons learned, hints, tips, tricks, and all else in a series of blog posts about how to automate control add-in development using gulp. This is the first post in the series of an unknown number of posts to follow up.

Disclaimer: this particular post is not a NAV 2018/BC/AL. This is (mostly) for pre-BC control add-in developers. However, a lot of gulp concepts I’ll talk about are readily applicable to AL and BC world.

Continue reading Automating control add-in development using gulp