rants of a singaporean male

Being a singaporean is tough, a singaporean male is tougher. (No doubt there are nations where, just being a citizen is punishing enough).

To strive for excellence but to give up social life and attention for your loved ones, or to be glad with what you have currently and spend the rest of your life like any other person? I'm going through a hard time, trying to get a degree, serving my nation (NSmen overseas training), working full time in a very harsh environment, appease my parents to make them not worry about me, to cherish the love of my life that i found recently.

I do not know how long i can last before i burn out, i really hope that i do not end up becoming like the sort of people that i've been bitching about.

2 years of my prime-time was spent on protecting my nation, i had to delay my degree programme and i have 2 years lesser working experience than my female peers... not to mention the major influx of Foreign Talents are making my sector very competitive. To start planning for family seems so faraway with my current remuneration, housing price are sky-high and COE (a certificate which you must buy if you want to own a car in Singapore)... looking at friends that come from wealthy family makes me feel so inferior and helpless.

This is one of the rare posts whereby it's got nothing to do with technical stuff, sorry if you are a fellow singaporean male and feel the same (if you don't already feel the same :)), for non singaporeans or maybe females that reads my blog. Be glad if you are not in the same situation as many of us, this is rarely a place for me to rant. But all these emotions have to go somewhere and this place seem to be a decent one.

WSE 3.0 extreme latency

We had a legacy .NET web-service that uses WSE 3.0. We are the service provider and we have an intermediate authentication server as well as the consumer. We had intermittent mysterious socket time out exceptions.

A thorough network snoop was conducted across 4 network teams (we span across 2 intranet and 2 WAN domains). Everything was healthy with <10ms latency between hops.

It was puzzling and a stroke of genius came by and we decided to check the inputtrace.webinfo and outputtrace.webinfo (default logging configs). Nothing special about it except for the size. Monthly scheduled jobs archival results in file size in excess of 600mb. Some team members had argued that logging mechanism should be asynchronous and should not have any influence on the response SOAP. There's no harm trying to flush out the logs and archive and do a re-test. Sure enough everything was back to < 0.5s

Our corrective action was to increase the frequency of the archival from monthly to weekly.

Gridview grouping

I've worked with SPGridView quite a bit lately and know that grouping is achievable, i thought that Gridview being the superclass of SPGridview, it should support grouping as well. I was wrong. My girlfriend had a requirement to group a set of data based on their type. A mock set of the data involved:


Product A - Type A
Product B - Type A
Product C - Type B
Product D -Type B
Product E - Type C
Product F - Type D


My first move was to take a look at the DAL (Data access layer) to see how the objects are related, in order for me to plan on grouping it. They are returned as generic dataset, so i had to create some simple POCO (Plain Old C# Objects), i prefer to work with strongly-typed grids :)


After retrieving the the data, i've modified the DAL to return an IList of the POCO instead of DS.


With the data retrieval and formatting done, all i have to do now is to bind it to a gridview and set the group by option. A little googling, i found a custom gridview which allow grouping:

ZNetControls

Read a little on the page and from the author's documentation, it seem simple enough.

1. I created an ObjectDataSource that hooks up with the DAL GetAllBindableObjects() as the selectmethod. 
2. Added the library (ZNetControls) into my VS and added the ZNetGridview into the page
3. Added the grouping type: DataKeyNames="Category"

Ran the page and i get this:

Pretty easy to implement :)

My analogy of a delivery team

I've recently worked under a project manager that don't do his job holistically, i was tasked to deliver 2 assignments but he didn't gave me the resources i need.

List of resources i requested for:

1. Local intranet account (in order to get internet access)
2. A machine for me to work on (i had to use my own macbook)
3. Someone that have complete understanding of the database (since i'm only on a short attachment)
4. Documentations/specifications (i had to work based on my own analogy)

This dragged on for 4 torturous months, the best thing was, in this 4 months, i only managed to complete (partially) 1 of his task due to all the above reasons. He didn't respond to my desperate pleas (i was dying to finish this assignment so as to move on with another team), he just wanted me to get the things done. Until an indian (no offense) knight, donning a set of bright shining armor came along and did his job for him. He came for barely 3 days and my first assignment was progressing by leaps and bounds, it's near completion, implementation and testing. He set a very defined objective for me to achieve and go all the way out to give me any resource that i require. I had to dedicate this post to him for his professionalism and commitment.

I stayed till almost 10pm tonight (it was a very rare thing for me to do), but i didn't sulk or feel moody over it. He had the charisma which made me willing to stay for his sake and to finish this whole thing and move on with life.

Upon sitting down, this analogy of mine came into my mind:

I can be metaphorically viewed as a Katana, him as a Samurai, my system analysts as a razor (not belittling them)

In a project team, our objective is to slaughter a cow (sorry for hindu folks but i can't think of other animals), for the whole 4 months, my previous PM was asking me to serve the meat to our users, without telling me which cuts they want, how well they want it to be done, and with what sauce. So from day one, i was only able to kill the cow, skin it and thinking of how should i serve the users. When the new PM came in, he told me everything, had all the tools ready. I did my cutting, cooked the beef and is just waiting for the sauce to be ready. (Don't feel hungry yet :))

As the saying goes: a blade is only as strong as its possessor

My job is to cut things up fast and nicely according to your requirements, not to shave the fur or to skin the beast. 

Luckily for me, a skilled wielder came along and we are on track to serve our users with a plate of medium-rare steak with black pepper sauce.

Swollen lymph nodes

Wouldn't be blogging in the coming few days due to cold sores -> swollen lymph nodes. Hurting like hell, will try to set up SPListEngine on MOSS 2010 and update with screens and POCs...

Sharepoint 2010 virtual box

Many of you are aware that microsoft had released a virtualized server with many components (SharePoint 2010, MS Office 2010, SilverLight, Active Directory) all configured nicely (even the AD had 100 test users with complete profiles and photos)

But for a developer like me, i would like to start from a clean server and install all the components one by one.

I am using Sun/Oracle's virtualbox as it allows 32 bit host to have 64 bit guest (free). I have a dreamspark account, which gives me access to a lot of software for free (academic purpose).

To set up your own MOSS 2010,

1. Prepare virtualbox (create win server 2008 64bit)
2. Install win 2008 with active directory
3. Create domain (in MOSS 2010, you can't use a local account as farm administrator anymore)
4. Install MS SQL 2008 developer 64bit
5. Patch MS SQL to SP1 and CU2 (cumulative upgrade)
6. Install MOSS 2010
7. Go to the familiar looking Configuration wizard
8. Start having fun with your new MOSS 2010 vm

I will attempt to put my SPListEngine within to speed up development efforts though, will update on my success (if any)

MOSS 2007 DAL alpha

assembla repository

Please email me for me to add you into assembla team to check out the source codes using SVN. Set up guide is on the website.

Pre-release functionalities:

• AJAX-enabled demo site (Autocompleter/update panel) Ajaxcontroltoolkit.dll
• Using Lists as primary data store
• using SmartPart for UI(visual web part in 2010) (found @ codeplex)
• Model, View, Controller, AJAX (WS), Framework projects (for a robust & scalable application)
• SPGridView (filtering/sorting/paging)
• SPGridView with MenuTemplateItem & SPToolBar (for the nice looking MOSS style menu that allow custom events)
• WebConfig extractor (supporting lazy-loading)
• Predicate builder (for using Lambda expressions as where clause)
• LinQ to sharepoint (SPMetal) credits to Bart Desmet for his excellent contributions
• LinQ/IList extensions: for converting to DataTable for ObjectDataSource (for spgridview filtration) //optional

Can't think of more stuff, it's still in its infancy. Don't worry too much if MOSS 2010 is coming and think that the efforts here are going to be wasted as MOSS 2007 still have it's market share and most stuff in the DAL are interoperable with MOSS 2010.

It's unable to perform joining like SQL, but even MOSS 2010 Microsoft.Sharepoint.Linq doesn't as CAML doesn't support joining. You will have to do a joining at the controller level.

For e.g.

3 Lists:

Products, Products category, products in category

IList<Product> products = SPListEngine.GetIListByClause<Product>(obj => obj.Name.Contains("Demo"));
IList<ProductCategory> productCategories = SPListEngine.GetIListByClause<ProductCategory>(obj => obj.Id.Equals(5);

Perform a LinQ join like:

var joinedProducts = from p in products join pc in productCategories on products.Id == productCategories.ProductId;

foreach(var joined in joinedProducts)
{
     //do what you need in joined IEnumerable<T>
}

I am really hoping someone can help me develop an internal logic class that will remove the 2000 listitem limit. I've conceptualised it, but can't squeeze time out to do it (work & school is taking its toll on me)

Do give it a try in your next MOSS 2007 project or for your own practice.

Sharepoint 2007 memory leak (available temporary workaround)

Todd Carter's blog post

Todd Carter had uncovered a serious memory leak issue that wasn't due to sharepoint developers not disposing properly, it was due to SPHttpApplication’s Init() method. Read up on his blog for more details.

edit: hotfix was released by microsoft, please patch all your sharepoint servers! http://support.microsoft.com/kb/982741

smartpart with AJAX

I've installed smartpart with AJAX, but when i try to add this webpart into the page, MOSS returned an error saying that the webpart is no longer there.

After a bit of searching, i've found in within codeplex discussion itself saying that adding these into the <runtime> node of the web.config. It worked.

<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions" publicKeyToken="31bf3856ad364e35"/>
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions.Design" publicKeyToken="31bf3856ad364e35"/>
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
</dependentAssembly>
</assemblyBinding>

SPListEngine

I'm finishing the final touches of the Sharepoint 2007 to LinQ helper, 2 things that i am unable/don't have time to incorporate:

1. Transaction scope (best way i can think of is to use versioning in SPList to do it, but there is too much overhead and it's still not atomic)
2. Relationships in LinQ (SPLists do not have referential integrity, manipulating manually requires using other frameworks like SLAM! which is not very well supported and widely used, and there's a lot of redundancies)

Will be finding a place to host it and seek feedbacks.

*update*

I've set up a repository @ https://www.assembla.com/ it's still alpha and not public
yet, drop me a mail for me to add you, if you are interested.

Sharepoint: How to achieve "Table of contents" on custom/default web part pages

This is a default web part page with "Table of contents" web part loaded. I assume that you can see the difference between this and the default "TOC".















This is also a default web part page however, without the need for me to add in a "TOC" web part. How do i do it?


First:

Go to your site collection features (note: site collection, not site)


Second:

Activate the publishing infrastructure feature.

Third:

Click on site features.

Fourth:

Activate Office SharePoint Server Publishing feature

Fifth:

When you click on "Site Actions", you will notice that you will have more options, click on "Create Page".

Sixth:

Choose the template you need.

Seventh:

You will see this web part page (note that this page can be found in your "Masterpage gallery" which can be modified with Sharepoint designer, as i know most people do not have that many web parts in a single page, ironically, it's named "BlankWebPartPage". Not very "blank" in my opinion.

Eigth and final step:

Click on publish :)

And there you go, no more users complaining or asking "Hey, why is the table of contents so different in sharepoint??"

I hope that this post will help you lessen the amount of hair you will pull out of your head.

Sharepoint enterprise application development template

I haven't been posting any entries due to my exams and my current personal project. I came up with this idea to design an architectural template for MOSS development as my organization do not have many MOSS developers and expertise.

Things that i already achieved:

Incorporating n-tier architecture
Working with strongly-typed SPList entities generated using SPMetal
Data Access engine using generics and reflection to perform CRUD (Create, read, update, delete)
Utilities methods for config extraction
Changed the way MOSS UI is designed and developed in a more rapid/easy manner

Things that i want to achieve (i've conceptualized ways to do it, just need time):

Support atomic transactions when performing CRUD (Rollback, Commit, isDirty, etc)
Workaround with SPList to get satisfactory performance when exceeding 2,000 items
Improve exception handling
Audit trial using list event handlers (features)
Logging

Things that i still have doubts of feasibility/performance:

Using singleton pattern for heavy MOSS objects like SPSite


My school is starting soon, and i'm still thinking whether is this project of mine worthy of my effort or not. I am not the best architecture designer out there as well, but i will try my best to complete the designs and do proper unit tests and to document the architecture. I'm actually actively looking for help to work on it, if you are interested please contact me.

Encrypting values in web.config

Sometimes user requirements don't allow you to put password or sensitive data in web.config.

There is a really simple to do it:

1. go to "command prompt"
2. type: cd "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\" (which every your drivespace is)
3. type: aspnet_regiis.exe -pef "<Name of node to encrypt>" "<Full path to web.config>"

Your done.



Edit post:

It was really that simple when i do it on my development server. But when i do the same thing on the SIT server, i was thrown with: "The RSA key container could not be opened"


Solution for me:

1. Ensure that you have permission to use the keys in "C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys" 
2. Check the keyContainerName in "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config"
3. go to "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727" type aspnet_regiis -pa "NetFrameworkConfigurationKey" "NT AUTHORITY/NETWORK SERVICE"

I'm assuming you are using the default name for NetFrameworkConfigurationKey.

Then go to the top of the post to do the encryption again.

Good luck.

Resizing array

If you ever had to work with an array that you wouldn't know the size till run-time, you have a couple of options:

1. Clone the old array into the new array (un-necessary overhead)
2. Just proceed with a array with redundant trailing allocated memory
3. You can resize it with Array.Resize(ref <arrayName>, int newArraySize)

int[] myArray = new int[50];
int newArrayCount = 0;

//some biz logic goes here
//determined array size

newArrayCount = myArray.Count;

Array.Resize(ref myArray, newArrayCount);

You can put myArray.Count as the 2nd parameter of the resizing method, up to your preference.

c#: boolean vs bool, String vs string, Int32 vs int

A frequently asked question, what is the difference between a Boolean and a bool object in c#? In short, it's actually the same thing. When compiled to MSIL(Microsoft Intermediate Language), they are referring to System.Boolean, thus there's no implication in performance.

But as a good conventional practice, i always use bool & string as opposed to Boolean & String. Unless you always use System.Int32 instead of int.

IIS logs 400 0 64

From my previous post, i've described how to consume a .NET webservice using netbeans. However, due to the JDK version on my colleague's solaris server, they are not able to import several libraries in order to do so.

Thus, they had use socket programming to send out the SOAP request. Everything was doing fine and well on development server, SIT server, and our local machines.

The wierd thing happens when the swing-over to UAT. When the function was called to send the SOAP request and in the while loop that waits for server response, it kept waiting... waiting... waiting... After several attempts to find faults in the java, web service and JDK versions. There was still no good explanation nor solutions. I went onto the web service machine and dug up the IIS Logs and found a couple of lines that showed the request reaching the web service machine but with an error code 400 0 64. After some research, i found out that the error code represents Bad request. Another vague error message -_-.

I almost wanted to give up, until i went into the HTTPERR folder and open up the logs and saw something that caught my attention. There was a request coming into the server via port 9090, but the response was sent to a random port number.

Bingo! In my client's environment, firewalls will block all the traffic going into the servers unless the need arises and we have to write in service requests to open up the port.

1 more problem resolved and hope this post will help anyone that hit this error.

Business objects VS Typed Datasets

 I always had this feeling that POCO (Plain old C# objects) will outperform typed datasets & datasets. The main reason why people are using it is only because it lessen development time by a very....... small amount in my opinion. Using business objects are easier to understand and in C# context, the compiler actually do a lot of work for you.

Example in other languages:

public myClass()
{
     private String someString;
  
     public String getString()
    {
        return this.someString;
    }
    public void setString(String value)
    {
        this.someString = value;
    }
}

you can do this in C#:
public myClass()
{
    public String someString { get; set; }
}

That only explains how much it takes to write a Business Object class. Next we go to real work.

I created a simple demo project in VS 2010 (beta), you can use any visual studio to compile this if you want to try it out yourself.

in my Business object class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace DemoBizObject
{
    public class BizObject
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Desc { get; set; }
        public decimal price { get; set; }
        public DateTime CreatedDt { get; set; }
        public DateTime ModifiedDt { get; set; }
    }
}

in my aspx code behind, i did this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Collections.ObjectModel;

using DemoBizObject;
using System.Diagnostics;
using System.Web.UI.DataVisualization.Charting;

namespace DemoWebApp
{
    public partial class _Default : System.Web.UI.Page
    {
        Collection<BizObject> BizObjects = new Collection<BizObject>();
        double[] pocotimings = new double[7];
        double[] typeddstimings = new double[7];

        decimal d = 0.00m;
        const int ONE = 1;
        const int TEN = 10;
        const int ONEHUNDRED = 100;
        const int ONETHOUSAND = 1000;
        const int TENTHOUSAND = 10000;
        const int ONEHUNDREDTHOUSAND = 100000;
        const int ONEMILLION = 1000000;
        const string LINEBREAK = "<br />";

        protected void Page_Load(object sender, EventArgs e)
        {
           
        }

        protected void Button1_Click(object sender, EventArgs e)
        {
            this.Label1.Text += "Typed Datasets" + LINEBREAK + LINEBREAK;
            this.Label1.Text += ONE +": "+ demoTypedDS(ONE)+ LINEBREAK;
            this.Label1.Text += TEN + ": " + demoTypedDS(TEN) + LINEBREAK;
            this.Label1.Text += ONEHUNDRED + ": " + demoTypedDS(ONEHUNDRED) + LINEBREAK;
            this.Label1.Text += ONETHOUSAND + ": " + demoTypedDS(ONETHOUSAND) + LINEBREAK;
            this.Label1.Text += TENTHOUSAND + ": " + demoTypedDS(TENTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEHUNDREDTHOUSAND + ": " + demoTypedDS(ONEHUNDREDTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEMILLION + ": " + demoTypedDS(ONEMILLION) + LINEBREAK + LINEBREAK;

            this.Label1.Text += "POCO" + ": " + LINEBREAK + LINEBREAK;
            this.Label1.Text += ONE + ": " + demoPOCO(ONE) + LINEBREAK;
            this.Label1.Text += TEN + ": " + demoPOCO(TEN) + LINEBREAK;
            this.Label1.Text += ONEHUNDRED + ": " + demoPOCO(ONEHUNDRED) + LINEBREAK;
            this.Label1.Text += ONETHOUSAND + ": " + demoPOCO(ONETHOUSAND) + LINEBREAK;
            this.Label1.Text += TENTHOUSAND + ": " + demoPOCO(TENTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEHUNDREDTHOUSAND + ": " + demoPOCO(ONEHUNDREDTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEMILLION + ": " + demoPOCO(ONEMILLION) + LINEBREAK + LINEBREAK;
        }
        private string demoTypedDS(int numOfRecords)
        {
            typedDS ds = new typedDS();
            Stopwatch watch = Stopwatch.StartNew();
           
            for (int i = 0; i < numOfRecords; i++)
            {
                ds.DemoTable.AddDemoTableRow(i, "demo name", "demo desc", d + i, DateTime.Now, DateTime.Now);
            }
            watch.Stop();
            return watch.Elapsed.TotalSeconds.ToString();
        }
        private string demoPOCO(int numOfRecords)
        {
            BizObject tempObject;
            Stopwatch watch = Stopwatch.StartNew();

            for (int i = 0; i < numOfRecords; i++)
            {
                tempObject = new BizObject();
                tempObject.Id = i;
                tempObject.Name = "demo name";
                tempObject.price = d + i;
                tempObject.CreatedDt = DateTime.Now;
                tempObject.ModifiedDt = DateTime.Now;
                tempObject.Desc = "demo desc";
                BizObjects.Add(tempObject);
            }
            watch.Stop();
            return watch.Elapsed.TotalSeconds.ToString();
        }
    }
}

pardon my messy codes, i didn't want to create this project initially, but i want to be someone that talks the talk and walk the walk.Next, create a similar table in your database with following...

 After that, right click on your solution file and click "Add new item" and select a dataset object.
Double click on the XSD file and go to your DB and drag-and-drop the table onto the XSD and you will get the table there..

The codes are very self explanatory and simple, not much re-using of codes especially @ the printing part.

I ran the test and get the results for POCO VS DS

Results are:
Typed Datasets

1: 4.1E-05
10: 5.47E-05
100: 0.0004515
1000: 0.0045912
10000: 0.0747852
100000: 0.6806366
1000000: 7.4931112

POCO:

1: 5.8E-06
10: 9.6E-06
100: 8.26E-05
1000: 0.0008128
10000: 0.0081756
100000: 0.1015654
1000000: 1.1922586

I tested it several times and dare to confirm that with this demo project, POCO is that Datasets in the lower range (1 to thousands) and the ratio justs exponential when the records reaches millions.

In this test, i only did instantiation and assignment, the Big O notation for:

• POCO = n Log n 
• Dataset = C^n

I don't find the need to extract data from database tables and going through the whole cycle as it will end up the same for both, the main difference only occurs when i instantiate and assign values to them. Even with just 6 primitive type objects, typed datasets are giving C^n in terms of performance. That is why my organization's Entity Objects are such a pain in the neck (they used typed datasets plus overriding several events)

Consume .NET webservice with only WSDL file

Objective: Consume .NET web service
Problem: I only have WSDL file (Server is blocked from my dev server)

In the past, a web service consumption was only: Right click on project, click on "Add web reference" and enter URL. But it's different this time as the production server had firewalls that blocked unauthorized/unnecessary access. I know it's possible to generate class files from the WSDL file, but how?

Time saver:.NET Framework Tools

After you generate the .cs file, CSC compile it into a dll and put the dll into your project and you will be able to invoke web methods immediately. For my case, i have to bring it to the production server to try access it, and it did worked perfectly :)

Performance bottleneck? review boxing & unboxing frequencies

There are times where our applications exceed users' response time threshold of 7s (usually), where all action must be completed under 7s.

Page-by-page code reviews are the last resort, but when you are resorting to that, look out of redundant boxing and unboxing of objects, check out: Technet

That's usually my case, or un-necessary objects declaration and recursion.

It's always better to go with iteration than recursion in terms of performance, as well as a typical For loop will outperform a foreach loop due to the boxing and unboxing overhead.

Check the traffic, CPU & memory utilization for resource bottleneck.

The last option is to review the entire architecture of the application (a PM's nightmare as that means a lot of time and resource needs to be pumped into the project). If good architectural designs was laid in place before development start, it wouldn't have to come to this point :)

Consuming .NET webservice using Java (netbeans 6.8) in 2 mins

I'm not a very Java person, but i was tasked to assist a team of Java developers using Solaris server. (A .Net developer's nightmare) I'm a GUI person and didn't like the command line concept, well it's powerful and have very little overhead but i need time to learn before jumping into it.

I told the team that i'm a .NET person, not a java guy. They told me leave the console/terminal/ftp to them, just help them to consume a .NET web service. Good.

I asked them from the WSDL file and went to research how java can consume a web service.

Luckily, there was a lot of resource out there documenting how to consume web service in netbeans.

First, create a New Project in NetBeans, give it a meaningful name.
Second, add a file and select "Web service" tab.
third, browse the wsdl file and click "Finish".

Now NetBeans will auto generate all the dependent files.

Expand the tree under your webservices and you will be able to see the web methods with red dots.

Drag and drop them into your methods and all the objects will be automatically instantiated and invoked with default values.

Using your java DAO(Data Access Objects) or equivalent and assign them the correct values and hit then run button.

There you go, java consumption of .NET web service in under 2 mins.