Sorting out the DLL hell, Part 3: The Code

[Update, February 8, 2016: there is a new version of code from this post. Please check http://vjeko.com/dynamically-loading-assemblies-at-runtime]

Okay here we go. In this post I deliver the promised code that handles automatic deployment of all your assemblies to client and server, as needed.

For any of you who haven’t read the last two posts, I am talking about automatically deploying .NET assemblies to clients and server, from the database, on demand, at runtime.

This will be heavy on code, so fasten your seatbelt and brace for impact.

First we need to have that .NET assembly that handles the AssemblyResolve event. Let me show that code first:

image

Yes, I really mean it. This is C# code, compressed (for conserving space), and stuffed into a C/AL string. You bet – I’ll compile it into an assembly right away:

image

The reason why I do it this way is simple – to avoid deployment. I don’t want to deploy anything, I want this to be a zero-footprint solution, as far as .NET is concerned. My goal is to enable you to just import C/SIDE objects into the database, and be able to use it right out of the box, without any external dependencies you need to worry about at any point, now, or later.

That’s why I simply create an assembly on the fly. If you can’t bother de-minifying C#, let me explain what this piece of code does:

  • It has a private dictionary that keeps a map between fully qualified assembly names and assemblies. When .NET runtime asks my class about an assembly, it will primarily look into this dictionary.
  • It has a public method to add an assembly to the map. This method can be called by NAV to feed my class with assemblies from the database.
  • It subscribes to the AssemblyResolve event. Inside it it first looks if the assembly exists in the dictionary, and if it does, it returns it. If it doesn’t, then it raises an event in NAV and allows NAV to resolve the assembly and call the ResolveAssembly method to feed the assembly into the dictionary. After calling C/AL, the event handler looks into the dictionary again, and if it finds the assembly there, it returns it. Since calls from .NET to C/AL and back are synchronous, this works as expected.

Good, now let’s take care of the assembly storage first. I don’t want to use the standard Add-ins table for all the reasons I explained in my first post in this series. It’s disregards the only correct way of identifying assemblies in .NET, which is through fully qualified name, so I provided my own table:

image

The primary key is the first three fields, and let me explain why.

First, I want to identify the assembly through its fully qualified name, that’s the first field. Then, I want to be able to bind an assembly to a user. Maybe, just maybe, I’ll want to load different assemblies for different users. If nothing else justifies it, then testability does – you can bind mock assemblies to user accounts used for testing. And last, I bind it to a tier. It’s an option field with three values: Both, Server, Client. Here I can choose where I want the assembly to be deployed when needed. Client assemblies are deployed to client, Server to server, and Both to both. Duh!

There is one function in this table, as well:

image

No magic in here. It receives a stream containing the assembly bytes, then a System.Reflection.Assembly by reference, to return the assembly to the caller, and name – if for some reason you want to store an assembly under a different name than the one reflection can look up inside – I’ll explain why a bit later.

This function then loads the assembly from the stream, looks up its name,  creates a record for it, and streams the assembly bytes into the BLOB field.

Next thing you need is a page to manage the assemblies:

image

The Import Assembly action does this:

image

It shows the import dialog, retrieves the selected assembly loaded in TempBlob, reads the stream from TempBlob and passes it onto the InstallAssembly function in the table.

Okay, now that you’ve seen the code and objects for keeping assemblies in line, let’s go back to the codeunit that manages all this.

One of my design goals was to make it possible to deploy new assemblies live, without having to restart the service tier, and without having to restart the client, if possible. I also want to minimize unnecessary loading of assemblies. So I created a single-instance codeunit to host my assembly resolver assembly I showed you how to compile.

Just after compiling the assembly, the codeunit will also call the InstallAssembly function on the table, to store the compiled assembly for future use. This means that compilation will occur only if needed – if the assembly resolver assembly is stored in the database, it’ll be used from there, otherwise it will be compiled first, then stored for future use:

image

And of course, this:

image

This function retrieves the BLOB from the database, and converts it to a byte array.

So, here we have three functions: one that decides whether to load or compile the assembly, and two functions to load and compile, respectively.

From OnRun trigger, I retrieve the assembly, and then use it to instantiate the AssemblyResolver class from it

image

I assign the reference to the instantiated object to a System.Object variable.

Now, I need to teach this AssemblyResolver how to resolve assemblies. I start by providing this function:

image

Now, this one is trickier than you expected. It receives a .NET Assembly record, the resolver object by reference, and target tier. Then it loads the assembly bytes into a byte array

Then, based on the desired tier (client or server) it uses reflection to access the ResolveAssembly method and call it. I need to do it twice, once with server-side variables, once with client-side variables. That’s important. The Resolver object received as a parameter does not need to be declared twice – it’s by reference, and it carries its RunOnClient flag too.

Apparently, I’ll need two resolvers – one for client, one fore server, so my OnRun trigger will also have this:

image

Now, this ResolverNst := Object may seem redundant at this point, but it isn’t. It would be if ResolverNst was of System.Object type, but it isn’t. I won’t reveal just yet what it is, because if I do, it’ll twist your brain.

Finally, I want to pre-feed my server and client-side resolvers with the assemblies from the database. This may not be needed for the service tier, but it is absolutely required for the client tier:

image

If it isn’t obvious, it iterates through the .NET Assembly table, applies the user filter, and then calls the ResolveAssembly method with either ResolverRtc or ResolverNst, depending on the Tier option setting of each record.

Now, I said you don’t need this for the service tier, and you absolutely do for client. Why?

Simply because you can’t have client-side .NET events in non-page objects in C/AL. Since we are in a codeunit, a client-side event is not allowed, so the client-side resolver must know about all assemblies up front. Server-side, on another hand, can raise events, and we can subscribe to the OnResolveAssembly event.

And now comes the trickiest part of all. First, we assigned the instance to System.Object, and setting WithEvents to Yes won’t do anything because this class has no events, and even if it had any, it must have the OnResolveAssembly event to be able to respond. So how in the earth do you subscribe to an event from a class that you just compiled at runtime?

First, let me show you another function I have here:

image

This function is hardcoded to resolve an assembly on the server side. I need it to properly apply filters, and to load an assembly using the server-side CASE.

Now, I have this trigger in my codeunit as well:

image

This event will fire when the AssemblyResolver class fires the OnResolveAssembly event and asks C/AL to provide an assembly for the specified name.

So – how did I get this event into my codeunit, because it certainly doesn’t come from System.Object, and I don’t even have my AssemblyResolver class yet, it will be compiled by runtime.

Well, I did it like this – I first compiled the assembly, then deployed it to my RoleTailored Client Add-ins folder to be visible by the development environment, and then I declared the variables:

image

ResolverNst has WithEvents set to Yes, and ResolverNst has RunOnClient set to Yes.

This solves the compile-time problem. But how do I trick the runtime to not fail at execution time, because it is running an object that references a type from an assembly that doesn’t even exist yet. To answer that question, let me explain when runtime resolution of assemblies occurs. In C/AL, since all .NET interop is heavily based on reflection, .NET variables are not really of the exact type you declare them at, but of a generic type provided by NAV. It’s NAV runtime that handles actual types through reflection. Thankfully, no assembly is resolved sooner than the variable of a specific type is first accessed. This means, that no matter that assembly is not there yet, everything will work before this line of code is reached:

image

This will fail, because to properly execute this line, NAV runtime must have access to the assembly containing this type. And since the assembly resolver will not be executed before this line fully executes, this fails at runtime because at this point, type NavHelper.AssemblyResolver.Resolver is unknown.

That’s why I never wrote that line of code. If you remember, I did this:

image

This is legal, because I am assigning to System.Object variable, so at this point only System.Object type must be known – and it always is. Only at this point it contains a valid reference to an instance of NavHelper.AssemblyResolver.Resolver.

Now, if I attempt to do this:

image

It will still fail, for exactly the same reason. Even though I have my assembly resolver instance ready – it is not able to raise any events in C/AL, because it’s bound to System.Object variable that doesn’t see any events. Also, the assembly resolver instance has no clue about any assemblies to resolve, so I need to do this:

image

The first line “teaches” my assembly resolver about resolving this particular assembly that hosts the assembly resolver class itself. Then, at the point when I assign my existing instance (referenced by the Object variable) it works, because at this point, the assembly is known, and variable assignment works.

So, in the end, my whole OnRun trigger will look like this:

image

In a nutshell, it compiles an assembly and then users that assembly to resolve itself for the purpose of initial assignment to the variable of a type which is otherwise unknown at this point.

Obviously, I could have done it even without pulling this stunt, if I wanted to settle for the please-restart-your-client for my service tier to access any assemblies added to the .NET Assembly table after I first loaded this codeunit. Since I want my codeunit to be able to resolve any assembly, even those added live, during runtime, I wanted to expose the OnResolveAssembly event to C/AL to be able to resolve assemblies that were not in the database when the codeunit was first instantiated. This makes it extra useful.

And finally, last piece of code you need to do, and you’ll need to do it on your own – just run this codeunit from codeunit 1, OnCompanyOpen or something. It will install itself and live there for the duration of your session.

Obviously, this solves only the runtime issues – for design-time, you’ll need to have all your assemblies present, but with this little utility here you can achieve this:

  • For development purposes, add all your assemblies to the RoleTailored Client Add-ins folder. This will allow you to compile your code.
  • For runtime, you can deploy assemblies to the .NET Assembly table at any point. They will be loaded as needed, and always the latest version will be loaded. In the worst case, you’ll need to restart your client session to pick up new versions of already resolved assemblies (e.g. if you didn’t change the version property of the assembly)

And this solves the NAV DLL hell. I feel it does a far better job than automatic deployment of assemblies in NAV 2016, and it’ll also run under NAV 2013 and newer.

Download the objects by clicking here, and do let me know what you think of this solution.

58 thoughts on “Sorting out the DLL hell, Part 3: The Code”

  1. Nice and wise solution Vjeko 🙂

    I have only one concern – if we are using NAV over internet with ClickOnce install, using this solution would mean that all DLLs are downloaded each time when session is started. But maybe it’s not too big load.

    1. It’s not much of a concern. This would apply only to client-side dlls (and you probably don’t have many of those anyway), and even when you have to send them over, most dlls will be under 100 kb in size, unless you beef them up with image resources. Plus, it’s up to you – you can always cache them locally if you want, and then instead of streaming them every time from server, you can load them up from local file system.

  2. Very smart solution, Vjeko!

    Having an action ‘Import Assembly’ on a Page still requires to have files available that can be uploaded in the page. It could be extended by automatically initializing that Assembly table. Assemblies can be provided by embedding them in a Codeunit or by making them available via a web service. Company Initialize / Upgrade Codeunits / whatever Initialize Codeunit could then pull the assemblies and automatically populate the Assembly table.

    Only client side control add-ins still need a record in the Control add-in table, don’t they? This solution could also take care of that part, including the zip package for Javascript add-ins.

    1. Yes, of course, all these are possibilities, and all of them add even more on top of this solution. I am glad everyone is seeing so much opportunity here already 🙂

    2. By the way – as I said in the post – this does not take care of client add-ins. As far as I know, you can’t “resolve” them using AssemblyResolve event, as they are not resolved through standard .NET process. Microsoft has invented their own wheel here, the ControlAddInExport, and what Microsoft does here is enumerate through all dlls and looking for a type that’s decorated with ControlAddInExport with a matching name. There certainly is a way to tap into this, as well, just not with the piece of code I put here. I might try to hack my way into it once, or maybe you can as well, but for now this is a limitation of the solution I give here.

  3. (slowly clapping) That is some major C#-fu you have going on!

    Thinking of any possible drawbacks with this, how about the following situation: let’s say we have a bunch of assemblies that we’re using on the client side. Am I right in assuming that using this deployment method, all of them will be downloaded from the mid-tier during each RTC startup? So, if we have NAV deployment with clients connecting directly over WAN and a 100MB worth of custom assemblies, those will get pulled each time RTC reconnects?

    1. Thanks, even though I don’t know what “C#-fu” stands for 🙂
      Anyway, as I said to Viktoras – you are free to cache all client-side dlls locally and stream them over the wire(less) only for the first time. It’s easy to extend the C/AL code to check whether an assembly exists in some local temporary cache before deciding to pull them from the database. The true value here is not pulling from database, it’s generic runtime assembly resolving from a byte array, which can come from any stream. Database, file system, C/AL objects even (as Arend-Jan points out) – sky is the limit. I’ve shown you how to launch your rocket, now you go and fly it any way you want.

  4. This is lovely!

    Ever since I saw Waldo’s solution for building a .NET class on the fly I’ve been thinking about trying to build something like this that meant we could build our .NET DLLs in the appropriate place (Visual Studio) and just deploy on the fly.

    Brilliant!

    1. Thanks! But I have to officially whine for a moment 🙂 I have used this in-memory building trick for a long time before it appeared on Waldo’s blog. With all due respect to Waldo, the trick originally comes from my work for Stratus project, and I first presented it at my TechDays 2012 session. I used it to compile the web service proxy classes in memory, and I blogged about it as well – http://vjeko.com/web-services-black-belt-consuming-nav-web-services-using-pure-cal. Not that I mind Waldo taking credit, it’s just that he owes me a beer and a steak every time he gets credited for my tricks 😛

      1. I originally got the trick from one of my developers .. also a brilliant mind, just like Vjeko. That’s what I said on my session .. I merely shared the trick ;-).

        So what did we learn: I absolutely did NOT come up with this trick, but I seem to be the best in sharing it :p

  5. Hi,

    seems like you messed up the identifiers a bit. In the codeunit you declared NavHelper.AssemblyResolver.Resolver, but the class is actually called NavHelper.AssemblyResolver.AssemblyResolver. I changed the code in GetSource and the global TextConst TypeName then it worked.

    Thanks for the great solution!

      1. BTW you should do a IF NOT GUIALLOWED THEN EXIT in the OnRun of the COdeunit, If the Codeunit is called from a webservice you get an exception.

        1. Actually, the check should be done from the places where the client-side functionality is used. The code should still run under web services, and only GUI specific things should not be accessible. If you download the demo from NAV TechDays 2015, this issue is solved in there.

  6. Hi. Please help.
    I have downloaded your NAV Objects and was able to generate your Helper Class (after modifying your code). After copy the generated file into NAV Client AddIn Folder I am not able to compile, because the helper was not identified.
    Can you please publish the dll for download?

    1. Yes, I have noticed this issue, I’ll make some changes to the NAV code and C# code and then publish this again, then it will work nicely. Sorry for this issue.

      1. Seems you already noticed the issue that the ResolverNst and ResolverRtc variables are of type NavHelper.AssemblyResolver.Resolver while in the GetSource function I see that the class is called AssemblyResolver.
        Also it would be fine to have the dll in the zip. Now I first need to get the code from NAV and paste it in a VS project to get the dll to get started. (And perhaps some words in your blog on how to get started after downloading your zip, would be nice)

        1. Yes, I did notice it, and I fixed it ahead of my NAV TechDays 2015 session – if you take a look at my NAV TechDays 2015 demo code, you’ll see that it’s fixed in there. I’ll try to do what you suggest here, it does make sense, indeed.

  7. In the InstallAssembly function in the table object there is this code:

    IF INSERT() THEN
    MODIFY();

    I guess the intension of this code is that, if the record already exists (INSERT fails), a MODIFY is done. But in this case the MODIFY is done when the INSERT succeeds. So think the code should be:

    IF NOT INSERT() THEN
    MODIFY();

    1. Yes, yes, that’s correct. That was the original intention, and I took the screenshot before I fixed it. The code definitely goes IF NOT INSERT THEN MODIFY, otherwise it makes no sense whatsoever. I should not be lazy and fix the screenshot… 🙂 But I am 🙂

  8. One thing that would make this even more awesome … would be the possibilty to include assemblies right from a NuGet-Package-Source… And an In-NAV-Dependency-Tracker like … a Codeunit that exposes a single function that enables other objects to declare that they require a certain .NET assembly. And your resolver resolves it straight from the NuGet-Source. Maybe a “private” one hosted by the partner … That would then be really ZERO deployment.

  9. Hi Vjeko,

    Shouldn’t the .NET Assembly table be set to “Data per company = No”?
    Doesn’t make sense if that’s Yes, I think.

    Regards, Jaap

  10. Hi.
    ” I’ll make some changes to the NAV code and C# code and then publish this again”.

    Do you have had time to do changes and publish again?

  11. Hi Vjeko, I think I found a major memory leak in your solution. Every session is creating that resolver class and loading the assemblies in memory. .NET needs only the first one, because that will resolve the requested assemblies. But worse is that the resolver instances are never released by the garbage collector, because AppDomain.CurrentAppDomain.AssemblyResolve event has a reference to it.
    Same applies to the client side when switching from company.
    If you drop me an email, I will send you my latest code.

      1. Good idea. Here it is: https://gist.github.com/JaapM/bd20aaf01d3579452b07
        Be aware that it is less functional than the original code of Vjeko, but enough for my purpose. And I use the TryFunction of NAV 2016.
        I don’t have the code of the resolver included in NAV, but just in a regular .NET assembly which must be available in the table in NAV.
        The main difference in architecture is that the codeunit is not singleinstance anymore, but the resolver class has a static property of itself. This way you have a singleinstance at application/service level instead of NAV session level.
        If you need more clearification, leave a comment on the gist.

  12. Hi, it looks like Nav is now doing a similar thing… compiling at runtime c# version of C/AL code, isn’t it?

  13. Hi, I am beginners in Dyn Nav, so my question may be surprising for you.
    During Codeunit (76101) compiling (on DynNav 2013 R2) i get error: ‘NavHelper.AssemblyResolver.Resolver.’NavHelper…’ can’t load..
    I use your DLL from your VS Project ‘Vjeko.Com.AssemblyResolver.sln’.
    Can you tell me where i have to add this library? I’ve tried to add them into AddIn’s directory and later into Table76101, but without luck :(.

    Thanks, best regards
    Cezary

  14. Hey Vjeko,

    We’d really like your tool to get out of the DLL hell 😉
    But currently, after the migration to Nav 2017 we noticed and issue. that is even reproducable in the Cronus DB.
    The issue is, that if you try to Debug something, the Debugger hangs, and leads to an Crashing NAS.

    We tried to solve this, by removing some DLLs (we thought, it’s may something, that is in one of the loaded assamblies) but, even if the table with the Dlls is completly empty (and all the cashes Cleared) the behaviour stays reproducable.

    What we did:
    We “just” imported the objects into the cronous, ran the “init” method of the Codeunit, restarted the app server and than tried to debug the “on Open Trigger” of the “Company Information” Page.

    Has anyone else may already noticed this ? (And in Best case a good solution?)

    What we allready did:
    1) We looked into the Eventlog. The only “partial” usefull information is the hint “Name of the Error Paket: -” / “Relative Path to the Paket: – ”

    2) We Removed all Assamblies from the table, to be sure it’s no side effect of another logic

    3) We Logged all DLLs and instanciations to see if there is anything “not resolvable” or “old Versioned”. But it isn’t.

    For Testing 4)
    We inserted an exit into the “onCompanyOpen” Trigger Method, what leaded to a working debugger.

    What we noticed,
    If the startup (Event) is disabled during the application server startup, and afterwards enabled again (so the client can load the assamblies again), the Debugger works fine. (But the server side dlls dont work :()

    Thank you,
    Torben

    1. Hi Torben! Thanks for the comment and for bringing this to my attention. I am not aware of these issues, and apparently I’ll have to work on making this compatible with AV 2017.

  15. Hey Vjeko,
    in-between i detected the reason for this behaviour myself.
    i found a stack trace in the system, after activating the extended logs in the registry. The stack trace said, that a class could not be loaded (that could be loaded while the CU to dynamically load the dlls is not integrated in the system.) This class is part of the “Microsoft.Dynamics.Nav.Client.CodeViewerTypes.dll” that is normally located on the App Server Directory.

    To solve the issue, we just imported the Assembly into the Table that stores the Assemblies and see… it works fine.
    To finalize the logic, we extended the provided Codeunit from this tutorial, to check at the beginning, if the Dll is already imported and otherwise to “try” to import it directly.

    The dynamic Assembly loading works now fine in Nav20107.

    Thank you,
    Torben

  16. Hi again,

    you can forget about my last post, I mixed did not see that the Nav Objects were included in the files of the Visual Studio Project.
    I will try those now and hopefullly I can go ahead…

  17. Hi,

    firstly I want to say thank you for this great stuff here!

    Secondly I want to ask for help on a problem I have with this.
    I created the DLL with the latest stuff I donwloaded from the GitHub link provided at the beginning of this post.
    I also downloaded the Nav objects (freom the DLL hell part III) and imported those into our Nav 2017 system.
    The table and page compile like a charm but the codeunit throws a type conversion error in function ‘InstallResolverAssembly’:

    InStream := DotNet

    This happens in the line:

    Asmbl.InstallAssembly(MemStr,Asm,AssemblyName);

    ‘MemStr’ is declared (in function InstallResolverAssembly) as DotNet variable and inside the function ‘InstallAssembly’ (this is located in the .NET Assembly table) the parameter ‘InStr’ is declared as InStream.

    What is the correct way to solve this problem now?

    Thanks for any help!

  18. Hi,

    I have a problem if I run Nav 2017 client after installation of Nav objects and AssemblyResolver.dll.

    The error is (translated from German):

    Error on calling System.Reflection.RuntimeMethodInfo.Invoke with following Message: Conflict in number of parameters

    The error happens in function ‘ResolveAssembly’, line:

    MethodInfo.Invoke(Resolver,Params);

    What I am missing?

    Thanks a lot!

Leave a Reply