Blog

Happy new year everyone! Last year was a bit slow for me, until the very end, when I churned out a year’s worth of posts just in two days. I hope this year to be a bit different, and let me kick-start it with a concept I’ve been playing with recently.

If you are into control add-ins development, chances are you’re familiar with my Visual Studio Project Template for control add-ins. It’s cool because it allows you to deploy all of your changes all the way over to NAV just by pressing F6. But it sucks because it’s based on Visual Studio, which is so… not 2019. It’s true, Microsoft never prescribed exactly which tool you should use to build your control add-ins once you’ve created the interface. It was entirely up to you. For me, Visual Studio used to be the tool of choice because in there you create the interface, and then why not just create everything in there.

But recently, I thought – why not using VS Code to develop control add-ins “v1” (that is: the control add-ins that work in pre-AL, pre-extensions “v2”, pre-NAV2018/BC environments)? If you, like me, still have to do those from time to time, but absolutely want to use VS Code instead of Visual Studio, then this post is for you.

Last month, I’ve decided to port all development work on a major control add-in from Visual Studio into VS Code. When I say “major”, I mean it: it’s 16K lines of JavaScript code in 138 source code files, not including 3^rd party libraries. To automate that in Visual Studio, I used a hodge-podge of build tasks, Windows batch scripts, PowerShell scripts, external utilities for zipping and minification, and the first thing I did – stupid, I know – when I transferred all my work to VS Code was to automate all of my tasks exactly the same way. So, I took all the same batch scripts, configured them as VS Code tasks, and I was happy. Not. It was slow, and ugly.

And then I remembered gulp.

It’s funny, gulp should have been the first thing to cross my mind. But, while I knew it was there, I never did anything with it. But, how complicated can it be? It turned out, I was able to replace all of the automation I had earlier into one nice gulpfile.js and turn my entire development experience completely upside-down. It’s that good.

So, I decided to share a little bit of my learning path, lessons learned, hints, tips, tricks, and all else in a series of blog posts about how to automate control add-in development using gulp. This is the first post in the series of an unknown number of posts to follow up.

Disclaimer: this particular post is not a NAV 2018/BC/AL. This is (mostly) for pre-BC control add-in developers. However, a lot of gulp concepts I’ll talk about are readily applicable to AL and BC world.

Why did I need automation in the first place?

Consider control add-in development. To get your control add-in from whatever development environment you use all the way to NAV running in the browser, there are a few things you need to do:

• Bundle and minify your source files (optional)
• Zip your control add-in resource
• Deploy your control add-in resource to NAV

For the first step, you are better-off automating it, even though, theoretically, you can do this on foot. Don’t.

The second and the third step can be done manually. The Visual Studio template I built ages ago solves this problem by automating it. The toughest one here is step #3 that requires you to run PowerShell with elevated privileges.

Without automation, doing steps #2 and #3 can take between 20 seconds to a couple of minutes, depending on how efficient you are, but is prone to human error. While 20 seconds may seem cheap, multiply that by the number of times you need to do this. Every time you want to update your changes (to see how they work), you need to do it. If control add-in development is your primary task, you probably do this 20 times per hour, which is at minimum 40 minutes per day. Why not automating this?

How did I automate it in Visual Studio?

The easiest way to check what I did is to go back to the original blog post about the Visual Studio template. If you are lazy, then:

• I did bundling using a very stupid approach: I used post-build tasks that used Windows batch commands to iterate over file contents, then type them into a redirected target file, for example:

  for %%f in (“$(ProjectDir)JavaScript\*.js”) do type “%%f” >> “%target%”

  I did minification using AjaxMin

• I did zipping from PowerShell, by using Ionic.Zip
• I invoked PowerShell using “powershell” command from a post-build event
• To make it possible to even run the needed PowerShell cmdlets, I ran Visual Studio as administrator (there were better approaches to this!)
• All was done from pre-build and post-build events in the project properties.

How did I automate it in VS Code before gulp?

My first step, before gulp crossed my mind, was to convert all of pre-build and post-build events into .cmd batch scripts that I could invoke with parameters. Then I configured VS Code tasks to invoke my batch scripts. Then I bound a VS Code task to the build process, so that when I launch Chrome (with Chrome debugging enabled) from VS Code it first does all of the control add-in automation, and then runs the browser.

I also did some changes here:

• I simplified my PowerShell to remove all but necessary stuff. What absolutely needed to remain in PowerShell was invoking NAV cmdlets to create and/or update the control add-in with a freshly-built resource zip file. Everything else I took out of PowerShell.
• I exchanged Ionic.Zip with 7zip. This was stupid, because it create an additional external dependency. But this was the step that kicked me into all of this gulp story.
• I used runas to invoke PowerShell with elevated permissions, to avoid running VS Code as administrator. I could have done this (I suppose, didn’t check) in Visual Studio, it’s just that I originally configured my shortcut with “run as administrator” option, so I never had any need to do this step.

I kept bundling and minification as it was, with a mental not to “automate this in a better way when I have time”.

However, all this didn’t run as fast as I wanted it to. It turned out that the entire process took just enough to be classified as “too long”. Something like 15-20 seconds per run, which is too much. Like, this is text files, no compilation, a couple of tasks, none of which should take tremendous amounts of time.

So, I started looking how to improve. Obvious thing was to get rid of external dependencies and replace the bundling process with something smarter, and then I remembered gulp.

I knew about gulp, vaguely. I knew web folks used it to automate all sorts of development tasks. Then I thought: well, gulp must be able to bundle my files, probably also zip them, probably also invoke PowerShell. I wasn’t wrong. It turned out that gulp was not only able to do all this, it was the right tool to do all that in VS Code.

How did I automate it with gulp?

VS Code runs on node.js infrastructure, so gulp, that runs on node.js too, is a natural choice for any kind of automation. On top of that, VS Code understands gulp natively – any gulp tasks will be automatically seen by VS Code and readily invokable from the command palette.

The first task I did was bundling. Gulp does it almost natively. I streamed all of my *.js files into the gulp-concat plugin and that was it. Three lines of clean code, literally. I’ll do all code examples in follow-up posts where I explain step by step all of the things I did, together with the why and the how.

The next task I did was zipping. For that, I used the gulp-zip plugin.

Since I had some intermediate tasks that require building some intermediate files (something that I’ll automate better in the future) my entire gulp configuration at this stage consisted of tasks to:

• Create two intermediate temporary *.js files
• Bundle these two intermediate files with all other *.js files into one target file
• Delete the intermediate files

Then I added more tasks to bundle all of my *.css files. And I created a task to zip the control add-in contents together.

Deploying was a tiny bit more complicated. First, I needed to figure out how to invoke PowerShell from gulp. That was fairly easy through node.js built in child_process module, at least to the point of invoking PowerShell. However, apart from having to invoke PowerShell, I had to invoke it with elevated privileges, so I had to invoke it not directly, but also through runas. Since I wanted to avoid any *.cmd files in my workspace, this required some considerations as to the paths of *.ps1 files and resource zip files, because once you run something with runas the paths are not relative to workspace root anymore. But, soon I had this part done, too. However, a lot was hardcoded in my PowerShell script, and in my gulpfile.js, so I used an additional JSON config file to configure the deployment process and gulp tasks to make it more reusable for other control add-ins.

Then, I attached the proper gulp tasks to VS Code task bound to the build process, and tested it, and it worked nicely. However, it still had to do all the things every time, and not all the things need to be done every time.

One nice aspect of moving this all to gulp was that I had modularized the entire automation process. Instead of having monolithic scripts that do everything at once, I had a bunch of very granular, smaller tasks concerned with simple things. Like, bundling CSS was one task, bundling JavaScript was another task, zipping was another task, etc. I could invoke every one of them individually, and good thing is – I could invoke only those that I need only when I need them.

Another nice feature of gulp is that it can set up a watch task. That’s a task that will monitor the changes in your file system and then run a required gulp task when a specific change is detected. I created a watch task to watch changes to *.css and *.js files independently, and then run the necessary bundling process in the background every time a change is made. This wasn’t without issues, and there were some important lessons learned here that I will blog about separately, but eventually this turned out to be a beautiful thing. Now, every time I save a file, it bundles all of my JavaScript together so that it doesn’t have to do it during deployment process. This simplified and sped up the deployment.

What next?

Well, there are two “what next” here. First one is for me, the next one is for you.

For me, next is to blog about individual things I did and how I did them. So I’ll blog about how to configure your VS Code with gulp, how to configure specific tasks in here with gulp, etc. Also, one possible “next” for me is to build a VS Code extension that automates this entire process so that you can simply install the extension and you have it all automated without having to do a single thing.

For you, next is to consider how you can use gulp in general, outside of the control add-in development.

Obviously we can automate a number of control add-in development tasks in AL development contexts using gulp, like transpiling any source code, or minifying, and similar things. But I am pretty sure gulp can be used outside of JavaScript development context, and in pure AL development context to automate all sorts of things. I hope this blog post raised your awareness about a tool that web developers have under their belt already, but that can be useful outside web world, and for general-purpose task automation in VS Code and AL. If you think of a way to put gulp to good use in AL workflows, do let me know.

That’s it for now, and stay tuned for actual examples of how I automated specific things, that will follow soon.