asp.net Performance snippet - disable dynamic compilation unless you need it

disable dynamic compilation unless you need it

Should be false

Playing with MongoDB in c#

Took an hour to start playing with MongoDB using c#. It's really easy to get it up and running, practically a matter of minutes on a single machine configuration. Later on I installed the official .net driver from jithub and wrote some code.

my conclusion so far:
1. it's fast. much faster than relation DB on same machine with same data.
2. for CRUD operations it's easy to use (I didn't take the time so far to checkout the map-reduce implementation coming with the driver).

When I tried to bulk insert more than 250,000 items I got an error from the server saying:
"Wed Mar 10 21:37:34 bad recv() len: 53888928
Wed Mar 10 21:37:34 end connection 127.0.0.1:2795"
I opened a Jira bug for this since I didn't find anything about it on google and got the following response:
"messages can't be more than 4MB (plus a little wiggle room for header)
when doing bulk inserts, need to do in batches of 4mb"
Great work by the jira team of this project: http://jira.mongodb.org/browse/CSHARP-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel




These are the results for my machine using single threaded approach with one open connection (intel core2duo , 2.66GHz , 3.25GB Ram, win xp sp2, standard single HD):



Time (ms) to create 1 jsons in memory: 0
Time (ms) to insert 1 jsons to mongo - one by one: 0
Time (ms) to insert 1 jsons to mongo - bulk operation: 0
Time (ms) to insert 1 jsons to mongo - bulk operation (safe mode): 41
Time (ms) to read 1 records: 0
***
Time (ms) to create 10 jsons in memory: 0
Time (ms) to insert 10 jsons to mongo - one by one: 2
Time (ms) to insert 10 jsons to mongo - bulk operation: 0
Time (ms) to insert 10 jsons to mongo - bulk operation (safe mode): 31
Time (ms) to read 10 records: 0
***
Time (ms) to create 100 jsons in memory: 3
Time (ms) to insert 100 jsons to mongo - one by one: 19
Time (ms) to insert 100 jsons to mongo - bulk operation: 0
Time (ms) to insert 100 jsons to mongo - bulk operation (safe mode): 42
Time (ms) to read 100 records: 4
***
Time (ms) to create 1000 jsons in memory: 22
Time (ms) to insert 1000 jsons to mongo - one by one: 197
Time (ms) to insert 1000 jsons to mongo - bulk operation: 4
Time (ms) to insert 1000 jsons to mongo - bulk operation (safe mode): 17
Time (ms) to read 1000 records: 29
***
Time (ms) to create 10000 jsons in memory: 233
Time (ms) to insert 10000 jsons to mongo - bulk operation: 58
Time (ms) to insert 10000 jsons to mongo - bulk operation (safe mode): 175
Time (ms) to read 10000 records: 247
***
Time (ms) to create 100000 jsons in memory: 2294
Time (ms) to insert 100000 jsons to mongo - bulk operation: 693
Time (ms) to insert 100000 jsons to mongo - bulk operation (safe mode): 1410
Time (ms) to read 100000 records: 2360
***
Time (ms) to create 1000000 jsons in memory: 23328
Wed Mar 10 21:37:34 bad recv() len: 53888928
Wed Mar 10 21:37:34 end connection 127.0.0.1:2795


as you can see it, on this very simple configuration the performance is close to linear to the number of items on all types of tested operations

Performance snippet – arrays in c# (for x86 compilers)

Here are some rules for working with array in .net when performance is critical. Some demand from you nothing but knowing about them with no special implementation implications, other require making decisions that require more attention (such as using unsafe code or even unmanaged code).

It's better to use jagged arrays (array or arrays) rather than multidimensional arrays
Within the CLR there is an optimization for loops that has a termination checkpoint against the length property of the collection/array. However this is not implemented for multidimensional arrays.In order to achieve better performance when the need arises to use multidimensional arrays it's better to use Jagged arrays
Example of a jagged array:int[][] arrJagged = new int[][]{new int[]{1,2,3,4,5},new int[]{2,3,4,5,6}};

Always use ascending loops
Well its even a better coding practice. Not only that your loop would be easier to read and understand it will also enjoy the benefit of better performance.
In a regular ascending loop the compiler statically checks that we are in the boundaries of the array, thus preventing the need for further checks on runtime. But, this feature is not implemented for descending loops which means that for each access by index a to the array a dynamic range check would be performed on runtime, reducing code speed…
But, pay attention. If you write this code

Private double[] GenerateAndPopulate(int iSize)
{
   double[] arr = new double[iSize];
   for(int I=0 ; I < iSize ; i++)
   {
      Arr[i] = i;
   }
}

You would not enjoy the termination of dynamic bound checks since. The right way (in terms of performance) is to use arr.Lenght

To be continued soon….

Tools for debugging/analyzing web sites

I've found out a couple of weeks ago a nice Firefox add-on, "tamper data", that allows tampering with out coming http requests (POST, GET). The add-on captures even captures requests coming out of Flash which I honestly did not expect.

This little tool can help debugging web pages on development/production and help testing web sites for security issues.

https://addons.mozilla.org/en-US/firefox/addon/966

I can also recommend great tool called HttpWatch which helps analyze the way your web pages load. This is usually the tool for finding out performance related issues. This is much like the freeware "fiddler" only much faster and intuitive imho.


http://www.httpwatch.com
http://www.fiddlertool.com/fiddler/

Web site performance - don't forget the gzip

if your web server is not in a CPU intensive situation and your bandwidth is high (even for some assets like image/javascript files) you should consider using the HTTP compression - GZIP.

read how to get it done on Microsoft techNet

Web site performance - don't forget the cache-control /content expiration (and a partial explanation about what is browsing from the HTTP perspective)

cache-control is a directive inside the http header instructing the web-browser and/or proxy server how to handle the data transffered. The main use of this directive is for cache handling.

usually when you surf to a web page for the first time the process is as follows at the 7th OSI level:
- the browser sends an http/s request for a specific URI. for example this is the request sent to google when requesting the advanced-search page:

GET /advanced_search?hl=iw HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-icq, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-silverlight, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.590; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
Host: www.google.com
Connection: Keep-Alive
Cookie: PREF=ID=XXXXXXXXXXXXXXXXXX

- the web server gets the request and responds. if everything went well the HTTP header would contain the following response code "HTTP/1.1 200 OK" which is followed by the HTML content of the requested page.

- Once the client finished getting the content the browser will parse the data and extract all of the assets (javascript, flash, images etc.) it needs to get from the server in order to assemble and render the page. Since in the scenario this is the first time the user surfs to this page it has none of the assets stored on his harddrive in the browsers "temporary internet files". the browser opens connections to the server in some level of parallelism (IE opens 2 tcp connection while firefox acts not very politely and opens 6 tcp connections) and requests the assets. this is how it looks like when the browser requests google logo image:
GET /intl/en_com/images/logo_plain.png HTTP/1.1

- The server sends back the image and within the http header of the response it has the expires directive:
Expires Sun, 17 Jan 2038 19:14:07 GMT

- If the user's browser is using the default configuration it will store the google logo png file and will hold the expiration date. if the user will surf to google search page again before the expiration date the browser will not request the image from the web server. It will be taken from the harddrive (or memory - depends on the browser and the settings).

- If the user will surf again the the search page after the date of the expiration the browser will detect that it has the google logo in his cache but its time validity expired. the browser will add to the request the "If-None-Match" or an "If-Modified-Since" header field with the ID of the asset:
If-None-Match "8e9bc4e4e50c71:76f"

- The server will examine the request and will decide if the image on the server has the same ID (meaning not changed) or a new ID. if the asset has already changed the server will send the new asset with the "HTTP/1.1 200 OK" code. if the asset is still the same the server will respond with:

HTTP/1.1 304 Not Modified Date Sat, 09 Feb 2008 11:29:39 GMTEtag "8e9bc4e4e50c71:76f"

this tells the browser that the asset is still valid.


now as you can see the performance hit cause by forgetting the use of cache-control could be severe and lead to many severe problems such as unnecessary round-trips, slowness of the site, throughput problems and overall - a crappy user experience.

So how can you set this cache control?

It can be done within your HTML code by adding meta tags such as meta http-equiv="Expires" or you can do it within your web server. On the IIS console just right click on the asset (for specific asset policy) or on a folder containing the assets and select: properties->HTTP-Headers->enable content expiration and choose the correct policy.
In apache web server you can do it in the httpd.conf file - you can read a much more detailed explanation about apache configuration here

Just remember - use it wisely. If you over cache your site your site visitors would suffer from strange behaviors and not up-to-date data/look-and-feel. You need to organize your assets hierarchy in the web server in a way that it will not be hard for you to set a different policy for each asset type.

www.highscalability.com - very good site about HPC/scalability/architecture

I consider this site a must for everyone who's interested in the realm of HPC/scalability/architecture.

You can find there some detailed posts, debates and links about topics such as youtube/flickr/amazon architecture, thoughts outside the relational database model, introduction to relevant products and much more.

http://www.highscalability.com/

XSS in Imperva :) I found it

funny stuff:
http://www.securityfocus.com/bid/28279

c# Performance snippet - looping through a multidimensional array

Within the CLR there is an optimization for loops that has a termination checkpoint against the length property of the collection/array. However this is not implemented for multidimensional arrays.
In order to achieve better performance when the need arises to use multidimensional arrays its better to use Jagged arrays (Jagged array or arrays)

Example of a jagged array:

int[][] arrJagged = new int[][]{new int[]{1,2,3,4,5},new int[]{2,3,4,5,6}};

c# Performance snippet - checking if a string is null

Best practice for checking if the string is null is by (string.length == 0). It is more than 2 times faster than comparing to string.empty

c# Performance snippet - Using static string.equals

When comparing two strings it is recommended to it using the string.equals static function. The reason is that this function performs some short circuit checks before actually going to compare each character. For example if you are writing some generic code that will possibly get null as one of the strings the function will use this to perform more quickly. Also if you send the same two strings a reference matching will be found saving you the binary character comparison.

c# Performance snippet - Sending value types to function as REF

Something new I've found out is that if you have a function that you call many times and it has value parameters its worth considering sending these parameters as Ref. By doing so you'll send only the memory address and not generate a new value type.