Blog

Okay. There has been a lot of “long time no see” intros to my posts on this blog in the past, but never this long. So – looooooong time no see, folks! We may have seen at other places I have been active – which obviously wasn’t here – so you know that I have not been gone fishing since my last blog, but I think it’s now really about time I restart.

I have been blogging and speaking at events and in general sharing my ideas and experience with this community for more than fifteen years. And anyone who as done something similar must have been in this state of mind for at least once (I know for sure I have been, and not once 😁) is that you’ve done it all – you’ve written it all, you’ve said it all, there is nothing more meaningful that you can contribute. Looking back, in the day I was a kind of go-to guru for all things .NET+C/AL – if you had a problem, you could find a solution on this blog, and I shared a lot of (useful) stuff here. Then JavaScript and control add-ins came along and I did the same. I could do this, because my past includes a lot of both of these technologies, so I knew immediately how to contribute useful content that people could put to use.

But for a while now, I had been feeling growing into a dinosaur. .NET is gone from AL (which is actually a very good thing) for a long time. JavaScript (or better let’s call it control add-in) integration is here and is going to stay for a very long time (if not forever) – but there is not really much more I can say about it – it’s a piece of integration with a grand total of 2 functions with 5 parameters total that people truly use from it, and in all honesty I think I’ve said all that can be said about it. Also, fresh blood has arrived in the last years and there is so much useful content about BC these days, that I really felt there is nothing else left for me to blog about or contribute. I really felt like my days are over, and I should come back here, say “so long and thanks for all the fish” and then just close it.

Until this year.

One thing happened that pushed me in this new direction I took, and I feel there is so much stuff I can share and contribute. And it’s “testing and testability”. For the foreseeable future, perhaps all of my future as an active professional in this community – let’s face it: I am getting old – is going to be about automated tests.

We do testing wrong. Plain and simple. All of us. Every. Single. One. Of. Us. In this BC community, we just don’t get it and that’s it – now I’ve said it.

Me too, by the way 👋. Yes, I mean it. If you met me at any Directions, TechDays, Days of Knowledge or any other event this year, you could say “come on Vjeko, you can’t be serious” – but I am dead serious. We are just doing it wrong.

And I want us to start doing it right.

Some of us are starting to do it right. Me too, now again, yes. But this is a journey. I want all of us to do it right.

This is how it all started.

A year or so ago, I got involved with a project for a team that has a strict code testing rule: no pull request without decent coverage of code you submit. I wish there were more teams like that out there. One entire class of problems in this community is that most of the teams still don’t do any testing at all – and I will address that problem on this blog, too – but I also think it’s in fact the same problem. Teams don’t do tests because the way how we are writing tests as of today is just wrong, demotivating, discouraging, and I truly admire every team that routinely produces tests for their AL code. Anyway, back to this particular team. Simple rule: no code without tests.

My task on that project was to build an almost isolated vertical, so during the entire project I worked almost completely isolated from the rest of the team (and the rest was about 15 developers). It was a fairly long project and the team has built a massive test library, Around 1500 tests and growing. By the time I integrated my work into theirs, they already had a growing concern – the test suite took too much time to run in the CI pipeline. It took between 60 and 90 minutes to execute that suite, depending on whatever parameters most CI worker containers out there depend on. When I contributed my 700 tests to that repo, the lead was deeply concerned about how much time I would at to it. I wasn’t, because I knew it wouldn’t be that much – I have been running all of my test suite locally for all the time, nearly 20 times per day and I knew it took about 2 minutes on my local development environment. It ended up adding 5 minutes of execution.

And then I got this “no way, this can’t be real” kind of reaction – how can it be possible. And then I explained the what’s and why’s and how’s of my testing approach, and then I got this “you must teach us how to do this, all of us should be doing it this way” and then I started. I started talking at events, I started digging into my old tests, I (re)started reading (again) about how people do it in other environments, I started considering how could I make this available to more of the people out there.

One of the things I didn’t mention when listing the areas of contribution where I have felt like I contributed substantial amount of content to this community was interfaces. Most of AL developers don’t get them, documentation is scarce, use examples in the base app are few and far between, and I even got this kind of a “why are you so worked out about interfaces, why do you care so much about them, why all this strange excitement” – and here’s your answer. Testability. Interfaces make testing easier. I have talked about polymorphism – which is where interfaces live – since at least 2015. We had ways to do it before, and then we got more ways to do it, but interfaces allowed us to do it the right way. And #1 place for polymorphism is testability

One of the reasons why my tests ran so much faster was obviously interfaces. But the other reason was that I tried to do unit tests. I dare saying that probably more than 99% of all AL tests out there are not unit tests. They are integration tests.

That suite of 1500 tests that ran for 90 minutes was all integration tests. My suite of 700 tests was about 600 unit tests and about 100 integration tests. The reason why my suite took all those 5 minutes to run was those integration tests – drop them, and it took under 10 seconds.

This was the tipping point for me. I went back into all of my tests I ever wrote, I went back into checking Microsoft’s base app test suite, I checked most of test suites I could lay my hands on, I went back to reading books and articles and forums about testing, only to confirm my conviction that nearly all AL tests we have are truly nothing else but integration tests.

One of the topics I want to cover deeply, inside out, is integration tests vs unit tests in our testing practices. But not today.

What I want to cover today is why I am now officially embarking on this “let’s finally start writing good tests” crusade.

The root cause of the problems is that most of AL developers aren’t even aware that there are different kinds of automated tests. It’s just “tests”. We just “test” our code, we write “tests”, we build “test” suites, and we “test” the code in front of us. And that’s all there is to it.

We’ve all learned how to write tests by looking at Microsoft’s test suite. For base app alone, it comprises more than 37.000 tests so it’s quite a repository of examples if you want to see how test code is written. It takes about 7 hours to run this suite on average on fairly decent hardware. I guess it takes around 10-12 hours or more to run it on a typical pipeline worker container (or whatever it’s truly called, correct me if I am using wrong terminology there). For Microsoft, it takes a lot less, because it’s massively parallelized.

So now we reached the point where I must say it. Is this a good test suite? No. And yes. There is no simple answer. I will be critical – very critical – here on my blog, about how we all, Microsoft included, write our test code. But the answer is not simple because it depends. How you write your test code depends 100% on what the code you test actually looks like. If your code is not written with testability in mind, then you are very limited in what you truly can do with your test code.

So, given the state of actual functional code in the base app, Microsoft’s test suite is actually fantastic. The fact that it runs for only 7 hours is mind blowing! Just for comparison, my 700 tests for that project I mentioned earlier run for 2 minutes (on same hardware where it takes 7 hours to run Microsoft’s suite). Let’s give it 171ms per test. Microsoft’s suite takes around 700ms per test. But it’s comparing apples and oranges! Most of Microsoft’s suite is actually integration tests, and most of my suite is actually unit test. My integration tests take north of 1000ms per test to run!

Really, honestly, if you want to learn how to write good, proper, well thought-out, well-structured integration tests, look no further. Microsoft’s test suite will teach you that, hands down.

So why “no” then? Why did I say that it’s not a good test suite? Because one thing you never learn from it is that you should – in fact – truly aim at writing unit tests. I am sure Microsoft’s test suite contains a lot of unit tests. I know for a fact that Microsoft wants to write a lot more of them. It’s just difficult to find them among thousands upon thousands upon thousands of integration tests.

One other thing I don’t want to get me wrong about is integration tests. I am not after them! I don’t want you to stop writing them. What I am after is the fact that we only write them. We all need integration tests, there are an absolute must for every project, but if all tests you write are integration tests, then don’t be surprised if your suite starts taking 60, 90, or 120 minutes to run.

Yesterday in my session about code coverage here in Lyon at Directions EMEA 2023, I said that over time, slow tests will render your entire test suite completely useless. The longer it takes to run the suite, the less useful it is. When your test suite takes more than 2 minutes to run, people will stop running it locally while they are doing their development work (and this is precisely when most of execution of test suites should happen!) When your test suite takes more than 45 minutes to execute, it will become a serious obstacle to your continuous integration efficiency, and this becomes the more obvious the more people you have on your team. When your test suite takes more than 2 hours to execute, it probably gets shut down from the CI pipelines, and starts happening only before CD – if you do CD. When your test suite takes 7 hours to execute, you stop running it.

Yesterday when I asked the audience (and it was 200+ people) how many of them run Microsoft’s test suite regularly, no hands went up (yes, some did – but that was Microsoft people). And this proved my point. We don’t run this test suite. And it’s absurd! I mean, imagine – we are all writing code that runs on top of the base app, and we never, I mean NEVER run the test suite Microsoft gives us, to test if we broke something in the core, by building on top of that core.

Funnily enough – when I asked the audience “does performance of your tests matter” – very few hands went up. And that also proved my point. People aren’t aware that it should matter. I mean – Microsoft’s test suite is slow, our test code obviously runs slow. So we shouldn’t care. Wrong, wrong, wrong, wrong, wrong!

Tests must run *FAST* or they are useless. There is this stream of thought that I subscribe to 100%, that’s well explained in this blog: https://blog.ploeh.dk/2012/05/24/TDDtestsuitesshouldrunin10secondsorless/

Yes – your test suite should run in under 10 seconds. What does that mean? If your tests run on average 1 second each, then your test suite should not have more than 10 tests! You can afford more if they run faster. If you have a suite of 1000 tests, then they should take no more than 10ms on average! Tests must be fast.

The problem we have right now in AL community is that we can’t possibly begin to imagine how to write tests that take less than 100ms on average, let alone 10ms or less. Why is that?

Testability. I have mentioned it a few times. Testability is not “presence of test code” (which is how most of this community sees it) but “how easy or difficult is it to test this code”. Testability is something that you don’t find in your test suite – no! Testability is something you either have or you don’t in your application, functional code. Before you can write good tests, you need to have good code.

And we mostly don’t know how to write good code. And this is the bottom line, the heart, the root cause of all our problems related to our test code.

Microsoft’s base app is not in a good shape. Everyone is aware of it, and Microsoft has been aware of it from the day they took ownership of it from Navision A/S now more than 20 years ago. Microsoft’s journey all along has been improving on that code base. The problem is – most of the community hadn’t truly cared. People learned how to write code by looking at the base app, and most of the AL code out there resembles the state of base app, my guess is, about 10 years ago or so.

Microsoft introduced test framework in BC I think in 2010, and ever since they have strived to improve on the state of the base app because the moment they started writing tests for the code they had at their hands, they realized it’s a huge issue. Try to remember what Sales-Post codeunit (or codeunit 80 as most of developers still refer to it as) used to look back then. There was this long monolith function of nearly 2000 lines of code, and that function’s name was “Code” – because there was no better name to give it, indeed. How in the earth do you begin to write tests for that? And that was just one example, all the rest was in not much better shape. AL code was spaghetti all over, and very long ones at that. I believe there are chunks of code still there that have been written more than 30 years ago when nobody in the entire industry truly cared about automated testing.

The problem is, when your code is all tightly coupled, as most of AL code is, then you can’t unit test it, plain and simple. So you do what you can – you write this other kind of tests which I called “integration tests” several times already. They aren’t truly integration tests either. For an integration test to be an integration tests it should be envisioned as such from the onset. Most of the tests out there want to be unit tests, but they turn out to be integration tests, and you can read it right from them. You can see that they try to unit test the code, but fail because the code they test is just so tightly coupled that you can’t unit test it.

And this is what this journey I have now been officially on since earlier this year is all about. Testability. I want to share my experience, my lessons learned, my ideas about what we can do to make our functional code better, so that we can (finally) start writing unit tests for that code.

And now we come to this last, burning point. Why now. Where have I been all this time, why do I start talking about it in 2023? Hasn’t all there is to say about automated testing already been said by now? I have been asked this question quite a few times this year.

Well, it’s not really quite like that. When BC TechDays (back then NAV TechDays) started back in 2011, my first submission was about unit testing, because back then it was a fresh, fairly unknown new feature in BC stack. It was accepted, and I started preparing it, but then Luc asked me to do a talk on .NET, because Microsoft wanted to introduce the test framework, so I got Microsoft’s .NET slot, and they got my automated testing slot at that event. I did talks on test-driven development in AL back in 2016 at Directions EMEA and some other events. I did quite a few talks at local events, and also some private workshops on the topic, since 2012. I have myself written test code for my entire professional career, which started in 1999. So I do have quite a lot of experience with testing. Just in other languages. I have written probably 20x more “other” code than AL code. I’ve written C++, C#, JavaScript, TypeScript and I have written tests for all of those.

The reason why I decided to start this journey now is because I wasn’t aware of how much this community really needs this stuff done right. When discussing my concerns about what it is that I can still contribute to this community a year ago with a couple of fellow MVPs during Directions EMEA 2022, one of them said “why not testability, people would love to hear it from other angles, you could do that”. So I tried. I submitted a few “how to write testable code” proposals for Days of Knowledge events and Directions Asia 2023, and they got accepted, and then I got comments like “we need more of this”. Then together with waldo I did a talk at TechDays 2023 about it, and I delivered quite some private workshops on the topic, all from companies who attended my event talks and wanted it implemented right now.

So here we go. I have a lot to share on this topic, and today I start this journey.

Today or tomorrow I’ll follow up with more content, and I’ll share all of my Directions EMEA 2023 demo code on my GitHub. There won’t be much structure, I may do things first that should come later, and vice versa, but I am eager to share what I have right now fresh in my mind from my yesterday’s talk. And I am writing a blog, not a book, after all, topics can come and go not necessarily in a logical order.

Thanks for reading all the way to this, and see you soon 😎