Playing with MongoDB in c#

Took an hour to start playing with MongoDB using c#. It's really easy to get it up and running, practically a matter of minutes on a single machine configuration. Later on I installed the official .net driver from jithub and wrote some code.

my conclusion so far:
1. it's fast. much faster than relation DB on same machine with same data.
2. for CRUD operations it's easy to use (I didn't take the time so far to checkout the map-reduce implementation coming with the driver).

When I tried to bulk insert more than 250,000 items I got an error from the server saying:
"Wed Mar 10 21:37:34 bad recv() len: 53888928
Wed Mar 10 21:37:34 end connection 127.0.0.1:2795"
I opened a Jira bug for this since I didn't find anything about it on google and got the following response:
"messages can't be more than 4MB (plus a little wiggle room for header)
when doing bulk inserts, need to do in batches of 4mb"
Great work by the jira team of this project: http://jira.mongodb.org/browse/CSHARP-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel




These are the results for my machine using single threaded approach with one open connection (intel core2duo , 2.66GHz , 3.25GB Ram, win xp sp2, standard single HD):



Time (ms) to create 1 jsons in memory: 0
Time (ms) to insert 1 jsons to mongo - one by one: 0
Time (ms) to insert 1 jsons to mongo - bulk operation: 0
Time (ms) to insert 1 jsons to mongo - bulk operation (safe mode): 41
Time (ms) to read 1 records: 0
***
Time (ms) to create 10 jsons in memory: 0
Time (ms) to insert 10 jsons to mongo - one by one: 2
Time (ms) to insert 10 jsons to mongo - bulk operation: 0
Time (ms) to insert 10 jsons to mongo - bulk operation (safe mode): 31
Time (ms) to read 10 records: 0
***
Time (ms) to create 100 jsons in memory: 3
Time (ms) to insert 100 jsons to mongo - one by one: 19
Time (ms) to insert 100 jsons to mongo - bulk operation: 0
Time (ms) to insert 100 jsons to mongo - bulk operation (safe mode): 42
Time (ms) to read 100 records: 4
***
Time (ms) to create 1000 jsons in memory: 22
Time (ms) to insert 1000 jsons to mongo - one by one: 197
Time (ms) to insert 1000 jsons to mongo - bulk operation: 4
Time (ms) to insert 1000 jsons to mongo - bulk operation (safe mode): 17
Time (ms) to read 1000 records: 29
***
Time (ms) to create 10000 jsons in memory: 233
Time (ms) to insert 10000 jsons to mongo - bulk operation: 58
Time (ms) to insert 10000 jsons to mongo - bulk operation (safe mode): 175
Time (ms) to read 10000 records: 247
***
Time (ms) to create 100000 jsons in memory: 2294
Time (ms) to insert 100000 jsons to mongo - bulk operation: 693
Time (ms) to insert 100000 jsons to mongo - bulk operation (safe mode): 1410
Time (ms) to read 100000 records: 2360
***
Time (ms) to create 1000000 jsons in memory: 23328
Wed Mar 10 21:37:34 bad recv() len: 53888928
Wed Mar 10 21:37:34 end connection 127.0.0.1:2795


as you can see it, on this very simple configuration the performance is close to linear to the number of items on all types of tested operations

Performance snippet – arrays in c# (for x86 compilers)

Here are some rules for working with array in .net when performance is critical. Some demand from you nothing but knowing about them with no special implementation implications, other require making decisions that require more attention (such as using unsafe code or even unmanaged code).

It's better to use jagged arrays (array or arrays) rather than multidimensional arrays
Within the CLR there is an optimization for loops that has a termination checkpoint against the length property of the collection/array. However this is not implemented for multidimensional arrays.In order to achieve better performance when the need arises to use multidimensional arrays it's better to use Jagged arrays
Example of a jagged array:int[][] arrJagged = new int[][]{new int[]{1,2,3,4,5},new int[]{2,3,4,5,6}};

Always use ascending loops
Well its even a better coding practice. Not only that your loop would be easier to read and understand it will also enjoy the benefit of better performance.
In a regular ascending loop the compiler statically checks that we are in the boundaries of the array, thus preventing the need for further checks on runtime. But, this feature is not implemented for descending loops which means that for each access by index a to the array a dynamic range check would be performed on runtime, reducing code speed…
But, pay attention. If you write this code

Private double[] GenerateAndPopulate(int iSize)
{
   double[] arr = new double[iSize];
   for(int I=0 ; I < iSize ; i++)
   {
      Arr[i] = i;
   }
}

You would not enjoy the termination of dynamic bound checks since. The right way (in terms of performance) is to use arr.Lenght

To be continued soon….

First chance exception

First chance exception is an exception that is handled somewhere in the code. it means that the programmer was able to recover the situation or handle it gracefully. Second chance exception is an exception that was not caught.

erlang snippet (on windows) - anonymous functions

in erlang you can define an anonymous function the following way:


FUNCNAME = fun(X) -> 10*X end.

and you can use it like this:

FUNCNAME(11).

erlang snippets (on windows) - change directory and compile

change directory:

cd ("c:/erlang/2/").

compile:

c("C:\\Erlang\\1\\helloworld.erl").

or you can change directory and comiple:

cd ("c:/erlang/1/").

c(helloworld).

use DebugView to view System.Diagnostics.Trace.Write("boo!");

Microsoft's ie8 InPrivate browsing & google chrome "Incognito" - not as private as you think...

I was thrilled to find out about the new "InPrivate" feature latly introducted in internet explorer 8. This feature is very usefull when you want to buy flowers to your wife or when you accidently roam into transexual midget snuff p0rn. People suppose that when using this feature no one can know where they surfed (nor the sites themselves or other users on the PC).

Actually, google already had this on Chrome with a feature called "incognito"...

however, something poped into my mind - every site can save (without asking anything) a nice file with extention ".SOL" on your machine. these files are located on -> C:\Documents and Settings\{user-name}\Application Data\Macromedia\Flash Player\#SharedObjects\{some-hash-code}\{ site domain name}\{SOL file}

These ".SOL" files are used by macromedia flash movies for user traking and configuration storage. in this folder you can actually see all sites that has this feature (for example "youtube.com" or "dailymotion.com".

I though that the "InPrivate" thingy or "Incognito" takes care of this but I was surprised to see it doesn't. when browsing in "InPrivate" mode I went to both youtube and dailymotion and found that both sites happily written their SOL files on my machine. So, sites can track my actions even on so called private modes and my wife can find where I was buying her flowers...

so, next time you are "buying flowers to the wife" make sure to cover your tracks (Microsft and google only covers some of them).

97 Things Every Software Architect Should Know

It seems that software architects love to write articles. I found this link and must say that some of the articles there gave me some good ideas and nice observations. One of the short but very good articles is about "Continuously Integrate"

protobuf for .net - protocol buffering allows communication between all languages

Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats
http://code.google.com/p/protobuf-net/
this is related to the larger project:
http://code.google.com/p/protobuf/

How to change data on the metabase of IIS 6

First you'll need to enable the metabase editing. You can do it like this
Then you'll need to install the metabase editor

Once you can change metabase data you can do stuff like enabling gzip compression, removing/changing the IIS banner.