How to fix the “circular file references are not allowed” Error in ASP.Net

On one of my .Net projects, I came across an interesting issue.  I wasted couple of hours fixing it.  Hence, I decided to share my findings, so that others do not have to waste their time fixing the same issue. 

The issue

The issue was pretty simple, the app would not build.  The error that I was getting while building a ASP.Net web project using MSBuild was:

Of course the error said there is some sort of circular reference in my code.  I looked around to check and recheck, if I have created a circular reference by mistake.  However, if there was any circular reference, the code would not compile.  Code was compiling fine but it was failing when we ran aspnet_compiler.exe.


The ASP.Net Compilation tool (aspnet_compiler.exe) enables you to compile an ASP.Net Web application, this helps application performance because end users do not encounter a delay on the first request to the application.

I checked again, but certainly there was no code related circular dependency then why the aspnet_compiler.exe was complaining about the circular file references?

The Explanation

Googling a little, I found that, by default, in a Website Project, ASP.Net creates one DLL per folder. Hence, if you have the following setup:

• User control ucA.ascx is present in the directory "A".  ucA.ascx refers another user control ucB.ascx
• User control ucB.ascx is present in the directory "B".  ucB.ascx refers another user control ucC.ascx
• User control ucC.ascx is present in the directory "A".

The folder A's DLL will reference the folder B's DLL, which will again reference the folder A's DLL, causing a "circular file reference".

This is the reason why the aspnet_compiler.exe fails with the "circular file reference" error.

The Fix

There are two ways this issue could be fixed

• Rearrange the user controls (or MasterPages) to remove the circular references.  Usually this means moving the user controls in separate directories.  In our example, moving ucC.ascx to a new directory "C" (Preferred Solution).
• Use batch=”false” in the compilation tag of the web.config file.  This will cause a new DLL to be created for each control/page in the site. This should fix the error but is really lousy for performance, so it should be avoided.

I moved the ucC.ascx in a different directory and yes the error went away!

Fixing the java.lang.ClassNotFoundException com.test.model.SomeModelClass in loader dalvik.system.PathClassLoader on Android

Recently, I had faced a very interesting issue on one of my Android project.  The issue was the dreadful java.lang.ClassNotFoundException.

The Android App, that was facing the ClassNotFoundException was tested on multiple devices.  Some of the devices that we tested the app on were

• Motorola Droid
• Motorola Droid X
• HTC Desire
• HTC WildFire
• Dell Streak
• Nexus One
• Samsung Galaxy 3
• LG Optimus One

The app worked fine on all devices without any problems at all.  We decided to go live with the App on Android Market.

Everything looked great till one day when one of the QA's downloaded the app on HTC MyTouch 4G.  The app refused to work on this device.  The app threw ClassNotFoundExceptions.  The stack trace looked like

After looking at the stack trace, I was thinking, Hmm this is weird, the app works on all these devices without an issue.  Moreover, the app cannot compile if the class com.test.model.SomeModelClass was not found.  Then why it doesn't work on HTC MyTouch 4G?

My initial reaction was, may be the app was not downloaded correctly from the Market.  May be the downloaded APK was corrupt?

I requested the QA's to uninstall the app and re-install it again from the market.  I was hoping that this could fix the issue.

But software development has taught me one thing

Nothing changes, If Nothing changes

Yep, the reinstall didnt work.  The app was still throwing the ClassNotFoundException.

We were using ProGuard.  Proguard is a free Java class file shrinker, optimizer, obfuscator, and preverifier.  Another thought that crossed my mind was, Is ProGuard removing the class com.test.model.SomeModelClass?

I created the APK without running ProGuard on it.  Requested the QA to test it.

That didn't work either.  ProGuard was not at fault.

By now, I was really confused.  What could be the issue with the app?  I thought, may be, if I try to load the class explicitly as the first thing in the app, then would it fix the issue?  

I updated the APK, loaded the class explicitly when the app starts, requested the QA to retest the APK.

Nope, that didn't fix the issue.  Interesting thing that I noticed about about this change was, the class loads successfully when I load it explicitly, but the class is not found when the app really needs it at a later stage!

History has shown us, class loader issues are not easy to fix.

I was quickly running out of options.  People were downloading the app on their device from the Market and may be some of them were facing the ClassNotFoundException issue with the app.  I needed a solution and that too quickly!  We didn't want to make a wrong first impression.  What should I try next?

At the back of my mind, I was sure that its some sort of a class loader issue.  When I explicitly load the class its found, but when the class is needed later in the app, its not found.  Surely something to do with a different class loader.  I decided to go back to basics, let's to do it step by step.

First step, 

When does the app need the class?

The app makes a HTTP call to get some information.  The information is returned in form of JSON responses.  As mentioned in my previous post, I was using Gson to parse the JSON response.  Gson is an excellent library with it comes to JSON manipulation.

Gson converts JSON response into Java Model Objects.  This is the point when the Java Model Class com.test.model.SomeModelClass is loaded.  The problem lies here, Gson tries to load the class com.text.model.SomeModelClass dynamically and its unable to find it!

Why Gson does not find the class?

The real confusion in my mind was, why Gson is able to load the class on all other devices but HTC MyTouch 4G?  My reasoning to that was, may be, Gson is loaded in a different class loader in HTC MyTouch 4G?  Is that even possible?  

After some intense googling and trying our numerous things, I finally figured out what was issue and what was solution.

This issue occurs only when all of the following conditions are met

• The App uses Gson to manipulate JSON
• The device is HTC Desire HD or MyTouch 4G on T-Mobile

The issue is HTC - The device manufacturer, also uses Gson library and they have made it public.  

Somehow, when the JVM is loaded on such devices, the Gson is also loaded along with it.  Because of this, the Gson library that is bundled with the app never gets loaded.  

When the global Gson tries to load classes from our domain model, it fails to find them.  This is because, the class loader that loaded the global Gson library does not have our app on its classpath!

That explained what the problem was, let's see how we can fix it!

The Fix

The fix was pretty simple.  We need to change the package name of the Gson library so that the global Gson library does not interfere with the local Gson library.  To change the package name of local Gson library, I used JarJar project.

I followed the following steps to change the package name of the local Gson library

• Download jarjar project.
• Put jarjar-1.1.jar and gson-1.6.jar in the same directory
• Create a new text file in this directory, lets call it rules.txt.
• Write the following line in the rules.txt file: rule com.google.gson.** com.google.myproject.@1
• Open a command prompt and execute the command, java -jar jarjar.jar process rules.txt gson-1.6.jar myproject-gson-1.6.jar.
• Replaced the gson library reference in the project with myproject-gson-1.6.jar
• Update the imports to use the new package name
• Compile and build the new APK
• Requested the QA to test the updated APK

To look at more details visit this link.

And that was it, It worked!

It was such a relief!  I am glad that, I faced this issue, makes me love the job that I am doing!

How to work with JSON on Android - Part - 2

As mentioned in my previous post, Gson is an excellent lightweight opensource library to do JSON manipulations.  I have used this library on the Android platform, it works great!

We have seen that, converting JSON to Java model objects and Java model Objects to JSON is pretty straight forward with Gson.

In this post lets look at the various hooks that Gson provides to deal with not so straight forward scenarios.

Lets take an example.  Lets say you want to convert the JSON string value "08:00" into Java model Time object.  A very valid scenario right?

Basically, we need to serialize and deserialize the Time object in a special way.  Gson provides a way in which we can hookup our own serializer and deserializer for a given type.  This serializer and deserializer will be invoked to convert the Java model object into JSON String and JSON string into Java model object.

The approach taken by Gson to do this task is pretty natural and easy to follow.  Great job guys!

Lets look at the code to put things in perspective.

The Time class has two fields one to represent the hour and other represents the minute of the day.  Let's say that, we want the JSON representation of Time objects to be of in the format "HH:MM".  Let's first look at the code to convert Java model Time object into JSON

Gson provides an interface called JsonSerializer.  We can provide an implementation of JsonSerializer to serialize the Time object in the way we want.  The code to do this would look like

Ok so the serializer does not look very complex. How do we tell Gson to use TimeSerializer when it serializes the Time object?

Turns out, Gson providers a class called GsonBuilder.  This class can be used to register custom serializers and deserializers with Gson.  GsonBuilder class has the responsibility to construct an instance of Gson with registered serializers and deserializers.  Let's look at the code

Does not look too difficult right?

Serialization done, let's look at deserialization.  Any guesses?

I am sure you got it right.  Gson provider an interface called JsonDeserializer.  We have implement this interface to provide a custom deserializer for the Time Java model class.  Let's look at the deserializer code.

The code of deserializer looks pretty simple. The code to integrate the deserializer is exactly the same as the code to integrate the serializer.

Thats all folks! Gson is a really good framework to manipulate JSON.

We have effectively used it in Android projects. If you have to do JSON manipulation in your project, I would suggest to have a look at Gson for sure!

How to work with JSON on Android

Recently, I have been working quite a bit on Android.  In my experience, there are lot of times when one has to deal with JSON while building apps for Android.

In one of my projects, I had a situation where, I had to invoke a web-service which returned JSON response.  JSON response has to be parsed, information has to be extracted from it and shown to the user.

The JSON response returned by the web-service was pretty big, with lots of fields, objects, lists and lists of objects.  In sort, the JSON response was pretty complex, it would have been a lot of work to extract information out of it, construct domain objects and then use it to show the information on the GUI.

Android provides a package org.json which has class like JSONObject, JSONArray, etc which help us parse the JSON responses.  I could have used those class and parsed the JSON response, extracted the information out of it, converted that into domain objects and used them to show it on the GUI.  But I am a lazy developer!  I am always in an hunt for ways to reduce my work!

I was in no mood to write the such a lot of code just to parse the JSON response.  Over and above it, there are multiple problems with writing such code

• Quite some code just to parse the JSON response
• Code has to be tested very well.  We have to make sure that it works in all situations.
• This approach is error prone.  For example, if we decide to change the JSON keys, a lot of code have to be updated and there is no compile time check to find out errors if any while updating the keys.

So what is the alternate?  Simple, search for a library that does the job for you :)

The perfect library for the task is Gson.  Gson is an opensource project that helps us convert JSON to JavaBeans or JavaBeans to JSON using one line of code!

This effectively means we could convert the JSON response to our model Objects with one line of code.  Isn't it awesome!

How do they do it?

An example explains things much faster.  So lets not waste any time and head straight to see Gson in action.

Lets say that the JSON response returned by the service looked like

As we can see, the web services returns us the searched Address and Points of Interest (POI) around the searched address in the JSON response.

We want to convert this JSON response into the POIResponse Java domain object.  Lets look at how the POIResponse class looks like

The model class are pretty standard.  They just have fields with the same name as the keys in the JSON response.  Notice that we do have complex objects like Address and POI and list of POI in POIResponse class.

For Gson to work only one condition has to meet and that is, the name of fields in the model class should be same as the JSON keys.  This is based on the convention over configuration principal.

Also notice that we do not have getters and setters in our domain classes.  Gson uses reflection to set the values directly into the fields.  Gson find private fields as well.

Lets look at the code to convert the JSON response into model Java object.

Thats all we need to do to convert JSON response into POIResponse object!  As promised its just one line of code.

Gson has excellent support for Generics.  Notice that, we did not have to caste the return type of fromJson method to POIResponse.

Gson can convert POIResponse to JSON with the same ease.

The string reference json holds the JSON representation of the poiResponse instance.

Gson is an excellent lightweight library when it comes to working with JSON.

In the next post I will show how Gson provides enough hooks to provide custom serializers and deserializers.

Till then, stay tuned!