Silverlight on sharepoint

Recently, i have a silverlight project being handed over to me, i was supposed to evaluate the security concerns that might go against my client's policy. I started off adding the feature into my site, saw the new web part.. added the web part to my site, but all it shows was a blank white space, it can't be clicked, when i right click, there's only a generic silverlight context menu. A bit of searching on msdn/technet, i saw this post: silverlight mime, did as they said and the fanciful silverlight components starting to load :)

File share to SharePoint 2007/2010 migrator

A stable release is available here now! I haven't played much with SharePoint 2010, so glad that it passed my own test for migration, onwards for future enhancements.

WSE 3.0 & inversed murphy's law

I know that murphy's law exist and in short, it's a law that states that whatever might go wrong, will go wrong.

But I was forced to do OT to implement WSE 3.0, a technology that went out of support(?) since 2005? I started the implementation with a I'm-gonna-OT-till-client-gets-pissed attitude. But at the last min after struggling for 2hrs with an exception, "The security token cannot be authenticated or authorized."

I managed to rectify it... A stroke of genius came when I try to use command prompt to run notepad as the user specified in the security contract.. Command prompt returned me: user account is being locked out.


True enough everything went OK after I unlocked the account via "local users and group" -__-

The command:
"runas /user: notepad.exe"

For once.. Murphy's law worked for me instead of against me.

upload file into sharepoint 2007 (copy.asmx)

There are many ways to do it:

1. Customised web services
2. Out-of-the-box web service (copy.asmx)
3. WebDAV

Advantages of using Customised web services:

• Ease of uploading (through personalised codes)
• Ability to do many fine tuning (such as permissions, etc)

Disadvantages of using Customised web services:

• Not all SharePoint owners are comfortable with customised webservices running on their server

Advantages of using Out-of-the-box web service (copy.asmx):

• Using Microsoft "supported" API
• No need to fear if the next service pack is going to screw up all the pre-patch programs

Disadvantages of using Out-of-the-box web service (copy.asmx):

• Limited No way to set permissions of files

Advantages of using WebDAV:

• Very easy to implement

Disadvantages of using WebDAV:

• Unable to set any metadata

I had a recent job to do a migration from File System -> SharePoint Document Library. A short summary of what happened: 

I was told to migrate everything over with the same file names, permissions, taxonomy (folder structures), etc.

Initially, i chose to use SharePoint OOTB web service to do all the above, as i know that i can use it to create folders, upload items, set metadata, etc. Until i hit a problem: I can't fricking set the permissions. After a lot of negotiation with my stakeholders, there's only 2 choice for me (without breaching their microsoft gold support clauses). Deploy a customized web service written by me or skip the whole permission idea (they have to do it manually using cheap interns). Lucky for me, they chose to skip the permission thing, so i proceed to continue with the whole project with user-friendly user interface using just the Web Service of SharePoint. Another advantage of using it is, you can use it anywhere as long as you can achieve a connection to the SharePoint and File System, you do not need to be running on either servers.

I am still tweaking the project source codes for some eye candies before my next new assignment begins.

And for the copy.asmx to be consumed, you only have to do this:

public string UploadFile(string sourceFilePath, string destinationUrl)

{

            string msg = string.Empty;

            Copy c = new Copy(); 

            c.Url = this.spURL + "/_vti_bin/copy.asmx";

            Authenticate(c);

            byte[] myBinary = File.ReadAllBytes(sourceFilePath); 

            string[] destinationUrlColl = { destinationUrl };

            FieldInformation info1 = new FieldInformation(); 

            info1.DisplayName = "Title"; 

            info1.InternalName = "Title"; 

            info1.Type = FieldType.Text; 

            info1.Value = "new title"; 

            FieldInformation[] info = { info1 }; 

            CopyResult resultTest = new CopyResult(); 

            CopyResult[] result = { resultTest }; 

            try 

            { 

                //When creating new content use the same URL in the SourceURI as in the Destination URL argument

                c.CopyIntoItems(destinationUrl, destinationUrlColl, info, myBinary, out result); 

                 if (result[0].ErrorMessage != null)

                 {

                     msg = "Error: " + result[0].ErrorMessage;

                 }

                 else

                 {

                     msg = "Success";

                 }

            } 

            catch (Exception ex) 

            { 

                //TODO: logging & handling

            }

            return msg;

}

I written a simple authentication method to do the authentication properly -> Authenticate(c);

internal CookieContainer AuthenticateFBA()

        {

            CookieContainer cookies = new CookieContainer();

            using (GSPAuthenticationWS.Authentication authSvc = new GSPAuthenticationWS.Authentication())

            {

                authSvc.Url = this.spURL + "/_vti_bin/authentication.asmx";

                authSvc.CookieContainer = new System.Net.CookieContainer();     //create a new cookie container

                authSvc.AllowAutoRedirect = true;

                //set the FBA login information

                try

                {

                    GSPAuthenticationWS.LoginResult result = authSvc.Login(this.spAdminId, this.spAdminPw);

                    if (result.ErrorCode == GSPAuthenticationWS.LoginErrorCode.NoError)

                    {

                        CookieCollection cookieColl = authSvc.CookieContainer.GetCookies(new Uri(authSvc.Url));

                        //Get the specific cookie which contains the security token

                        cookies.Add(cookieColl[result.CookieName]);

                    }

                }

                catch (WebException wex)

                {

                    throw new NotImplementedException("Specified Sharepoint server is not FBA-enabled");

                }

                return cookies;

            }

        }

        internal NetworkCredential AuthenticateIWA()

        {

            return new NetworkCredential(this.spAdminId, this.spAdminPw, this.spAdminDomain);

        }

        private void Authenticate(SoapHttpClientProtocol SoapProtocol)

        {

            if (this.isFBAEnabled)

            {

                SoapProtocol.CookieContainer = AuthenticateFBA();

            }

            else

            {

                ServicePointManager.CertificatePolicy = new TrustAllCertificatePolicy();

                ServicePointManager.ServerCertificateValidationCallback = delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)

                {

                    //Here we can write code for implementing client side validation.

                    // This anonymous function is a call back function which is called once the SSL validation completes on WebServer.

                    return true;

                };

                SoapProtocol.Credentials = AuthenticateIWA();

            }

        }

A neat way to authenticate through the web service for both Form Based authentication and Integrated Windows authentication :)

iOS 4.1 development

I'm currently picking up a little on Objective-C to do some iOS development, for school project though. Since i have a MacBook (late 2009), apple developer account (free) and iPhone 3GS, it should not be that hard.. (i assumed)

I had very little C and C++ experience and knowledge, i'm just a hello world-level developer for both languages. That didn't stop me from downloading the 2.9 GB Xcode IDE and iOS4.1 SDK and try to write my first Hello World.

Followed a few guides like this and this, it wasn't that hard to print a hello world using the Interface Builder:

Just follow the guides step-by-step and try to grasp a few of the IDE (which is rather different from Visual Studio, NetBeans & SharePoint Designer)





After messing around with a few UIKit controls, i met a problem of not able to retract the keyboard on the simulator. Went around some forums and found that Apple gave a very good tutorial: UICatalog, which included most, if not all of the controls available in the API.

However, when i loaded the xcodeproj file, the compiler keep giving me the message: "Code Sign error: The identity 'iPhone Developer' doesn't match any valid certificate/private key pair in the default keychain"

You can bring the build results up by pressing: command+shift+B










After a few attempts to search around the net for the solution, many suggested setting up Keychain Access (windows equivalent of the certificate manager) so that you have Apple Worldwide Developer Relations Certification, and the iPhone developer certificate, iPhone distributor certificate (if you want to put apps into iTunes), there are lots of things to do and pay (for distributing), but hey, i'm just trying to mess around with iOS 4.1, i'm not yet ready to go through all these.

I tried to set the code signing options under the project settings, but it still doesn't let me compile and run the simulator. Then for no reason, i looked at the build results window and saw that the build option was targetted for "device" instead of "simulator". I didn't change it to device (which requires you to do the above certificate signing set-up), it was the default setting, i changed it to "Simulator" then pressed command+B and everything went well, finally.

I'll be messing around more with Xcode and hope to be able to produce something and decide if i'll sign up for a distributor package with Apple @ US$99/yr. Stay tuned.

rants of a singaporean male

Being a singaporean is tough, a singaporean male is tougher. (No doubt there are nations where, just being a citizen is punishing enough).

To strive for excellence but to give up social life and attention for your loved ones, or to be glad with what you have currently and spend the rest of your life like any other person? I'm going through a hard time, trying to get a degree, serving my nation (NSmen overseas training), working full time in a very harsh environment, appease my parents to make them not worry about me, to cherish the love of my life that i found recently.

I do not know how long i can last before i burn out, i really hope that i do not end up becoming like the sort of people that i've been bitching about.

2 years of my prime-time was spent on protecting my nation, i had to delay my degree programme and i have 2 years lesser working experience than my female peers... not to mention the major influx of Foreign Talents are making my sector very competitive. To start planning for family seems so faraway with my current remuneration, housing price are sky-high and COE (a certificate which you must buy if you want to own a car in Singapore)... looking at friends that come from wealthy family makes me feel so inferior and helpless.

This is one of the rare posts whereby it's got nothing to do with technical stuff, sorry if you are a fellow singaporean male and feel the same (if you don't already feel the same :)), for non singaporeans or maybe females that reads my blog. Be glad if you are not in the same situation as many of us, this is rarely a place for me to rant. But all these emotions have to go somewhere and this place seem to be a decent one.

WSE 3.0 extreme latency

We had a legacy .NET web-service that uses WSE 3.0. We are the service provider and we have an intermediate authentication server as well as the consumer. We had intermittent mysterious socket time out exceptions.

A thorough network snoop was conducted across 4 network teams (we span across 2 intranet and 2 WAN domains). Everything was healthy with <10ms latency between hops.

It was puzzling and a stroke of genius came by and we decided to check the inputtrace.webinfo and outputtrace.webinfo (default logging configs). Nothing special about it except for the size. Monthly scheduled jobs archival results in file size in excess of 600mb. Some team members had argued that logging mechanism should be asynchronous and should not have any influence on the response SOAP. There's no harm trying to flush out the logs and archive and do a re-test. Sure enough everything was back to < 0.5s

Our corrective action was to increase the frequency of the archival from monthly to weekly.

Gridview grouping

I've worked with SPGridView quite a bit lately and know that grouping is achievable, i thought that Gridview being the superclass of SPGridview, it should support grouping as well. I was wrong. My girlfriend had a requirement to group a set of data based on their type. A mock set of the data involved:


Product A - Type A
Product B - Type A
Product C - Type B
Product D -Type B
Product E - Type C
Product F - Type D


My first move was to take a look at the DAL (Data access layer) to see how the objects are related, in order for me to plan on grouping it. They are returned as generic dataset, so i had to create some simple POCO (Plain Old C# Objects), i prefer to work with strongly-typed grids :)


After retrieving the the data, i've modified the DAL to return an IList of the POCO instead of DS.


With the data retrieval and formatting done, all i have to do now is to bind it to a gridview and set the group by option. A little googling, i found a custom gridview which allow grouping:

ZNetControls

Read a little on the page and from the author's documentation, it seem simple enough.

1. I created an ObjectDataSource that hooks up with the DAL GetAllBindableObjects() as the selectmethod. 
2. Added the library (ZNetControls) into my VS and added the ZNetGridview into the page
3. Added the grouping type: DataKeyNames="Category"

Ran the page and i get this:

Pretty easy to implement :)

My analogy of a delivery team

I've recently worked under a project manager that don't do his job holistically, i was tasked to deliver 2 assignments but he didn't gave me the resources i need.

List of resources i requested for:

1. Local intranet account (in order to get internet access)
2. A machine for me to work on (i had to use my own macbook)
3. Someone that have complete understanding of the database (since i'm only on a short attachment)
4. Documentations/specifications (i had to work based on my own analogy)

This dragged on for 4 torturous months, the best thing was, in this 4 months, i only managed to complete (partially) 1 of his task due to all the above reasons. He didn't respond to my desperate pleas (i was dying to finish this assignment so as to move on with another team), he just wanted me to get the things done. Until an indian (no offense) knight, donning a set of bright shining armor came along and did his job for him. He came for barely 3 days and my first assignment was progressing by leaps and bounds, it's near completion, implementation and testing. He set a very defined objective for me to achieve and go all the way out to give me any resource that i require. I had to dedicate this post to him for his professionalism and commitment.

I stayed till almost 10pm tonight (it was a very rare thing for me to do), but i didn't sulk or feel moody over it. He had the charisma which made me willing to stay for his sake and to finish this whole thing and move on with life.

Upon sitting down, this analogy of mine came into my mind:

I can be metaphorically viewed as a Katana, him as a Samurai, my system analysts as a razor (not belittling them)

In a project team, our objective is to slaughter a cow (sorry for hindu folks but i can't think of other animals), for the whole 4 months, my previous PM was asking me to serve the meat to our users, without telling me which cuts they want, how well they want it to be done, and with what sauce. So from day one, i was only able to kill the cow, skin it and thinking of how should i serve the users. When the new PM came in, he told me everything, had all the tools ready. I did my cutting, cooked the beef and is just waiting for the sauce to be ready. (Don't feel hungry yet :))

As the saying goes: a blade is only as strong as its possessor

My job is to cut things up fast and nicely according to your requirements, not to shave the fur or to skin the beast. 

Luckily for me, a skilled wielder came along and we are on track to serve our users with a plate of medium-rare steak with black pepper sauce.

Swollen lymph nodes

Wouldn't be blogging in the coming few days due to cold sores -> swollen lymph nodes. Hurting like hell, will try to set up SPListEngine on MOSS 2010 and update with screens and POCs...

Sharepoint 2010 virtual box

Many of you are aware that microsoft had released a virtualized server with many components (SharePoint 2010, MS Office 2010, SilverLight, Active Directory) all configured nicely (even the AD had 100 test users with complete profiles and photos)

But for a developer like me, i would like to start from a clean server and install all the components one by one.

I am using Sun/Oracle's virtualbox as it allows 32 bit host to have 64 bit guest (free). I have a dreamspark account, which gives me access to a lot of software for free (academic purpose).

To set up your own MOSS 2010,

1. Prepare virtualbox (create win server 2008 64bit)
2. Install win 2008 with active directory
3. Create domain (in MOSS 2010, you can't use a local account as farm administrator anymore)
4. Install MS SQL 2008 developer 64bit
5. Patch MS SQL to SP1 and CU2 (cumulative upgrade)
6. Install MOSS 2010
7. Go to the familiar looking Configuration wizard
8. Start having fun with your new MOSS 2010 vm

I will attempt to put my SPListEngine within to speed up development efforts though, will update on my success (if any)

MOSS 2007 DAL alpha

assembla repository

Please email me for me to add you into assembla team to check out the source codes using SVN. Set up guide is on the website.

Pre-release functionalities:

• AJAX-enabled demo site (Autocompleter/update panel) Ajaxcontroltoolkit.dll
• Using Lists as primary data store
• using SmartPart for UI(visual web part in 2010) (found @ codeplex)
• Model, View, Controller, AJAX (WS), Framework projects (for a robust & scalable application)
• SPGridView (filtering/sorting/paging)
• SPGridView with MenuTemplateItem & SPToolBar (for the nice looking MOSS style menu that allow custom events)
• WebConfig extractor (supporting lazy-loading)
• Predicate builder (for using Lambda expressions as where clause)
• LinQ to sharepoint (SPMetal) credits to Bart Desmet for his excellent contributions
• LinQ/IList extensions: for converting to DataTable for ObjectDataSource (for spgridview filtration) //optional

Can't think of more stuff, it's still in its infancy. Don't worry too much if MOSS 2010 is coming and think that the efforts here are going to be wasted as MOSS 2007 still have it's market share and most stuff in the DAL are interoperable with MOSS 2010.

It's unable to perform joining like SQL, but even MOSS 2010 Microsoft.Sharepoint.Linq doesn't as CAML doesn't support joining. You will have to do a joining at the controller level.

For e.g.

3 Lists:

Products, Products category, products in category

IList<Product> products = SPListEngine.GetIListByClause<Product>(obj => obj.Name.Contains("Demo"));
IList<ProductCategory> productCategories = SPListEngine.GetIListByClause<ProductCategory>(obj => obj.Id.Equals(5);

Perform a LinQ join like:

var joinedProducts = from p in products join pc in productCategories on products.Id == productCategories.ProductId;

foreach(var joined in joinedProducts)
{
     //do what you need in joined IEnumerable<T>
}

I am really hoping someone can help me develop an internal logic class that will remove the 2000 listitem limit. I've conceptualised it, but can't squeeze time out to do it (work & school is taking its toll on me)

Do give it a try in your next MOSS 2007 project or for your own practice.

Sharepoint 2007 memory leak (available temporary workaround)

Todd Carter's blog post

Todd Carter had uncovered a serious memory leak issue that wasn't due to sharepoint developers not disposing properly, it was due to SPHttpApplication’s Init() method. Read up on his blog for more details.

edit: hotfix was released by microsoft, please patch all your sharepoint servers! http://support.microsoft.com/kb/982741

smartpart with AJAX

I've installed smartpart with AJAX, but when i try to add this webpart into the page, MOSS returned an error saying that the webpart is no longer there.

After a bit of searching, i've found in within codeplex discussion itself saying that adding these into the <runtime> node of the web.config. It worked.

<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions" publicKeyToken="31bf3856ad364e35"/>
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions.Design" publicKeyToken="31bf3856ad364e35"/>
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="3.5.0.0"/>
</dependentAssembly>
</assemblyBinding>

SPListEngine

I'm finishing the final touches of the Sharepoint 2007 to LinQ helper, 2 things that i am unable/don't have time to incorporate:

1. Transaction scope (best way i can think of is to use versioning in SPList to do it, but there is too much overhead and it's still not atomic)
2. Relationships in LinQ (SPLists do not have referential integrity, manipulating manually requires using other frameworks like SLAM! which is not very well supported and widely used, and there's a lot of redundancies)

Will be finding a place to host it and seek feedbacks.

*update*

I've set up a repository @ https://www.assembla.com/ it's still alpha and not public
yet, drop me a mail for me to add you, if you are interested.

Sharepoint: How to achieve "Table of contents" on custom/default web part pages

This is a default web part page with "Table of contents" web part loaded. I assume that you can see the difference between this and the default "TOC".















This is also a default web part page however, without the need for me to add in a "TOC" web part. How do i do it?


First:

Go to your site collection features (note: site collection, not site)


Second:

Activate the publishing infrastructure feature.

Third:

Click on site features.

Fourth:

Activate Office SharePoint Server Publishing feature

Fifth:

When you click on "Site Actions", you will notice that you will have more options, click on "Create Page".

Sixth:

Choose the template you need.

Seventh:

You will see this web part page (note that this page can be found in your "Masterpage gallery" which can be modified with Sharepoint designer, as i know most people do not have that many web parts in a single page, ironically, it's named "BlankWebPartPage". Not very "blank" in my opinion.

Eigth and final step:

Click on publish :)

And there you go, no more users complaining or asking "Hey, why is the table of contents so different in sharepoint??"

I hope that this post will help you lessen the amount of hair you will pull out of your head.

Sharepoint enterprise application development template

I haven't been posting any entries due to my exams and my current personal project. I came up with this idea to design an architectural template for MOSS development as my organization do not have many MOSS developers and expertise.

Things that i already achieved:

Incorporating n-tier architecture
Working with strongly-typed SPList entities generated using SPMetal
Data Access engine using generics and reflection to perform CRUD (Create, read, update, delete)
Utilities methods for config extraction
Changed the way MOSS UI is designed and developed in a more rapid/easy manner

Things that i want to achieve (i've conceptualized ways to do it, just need time):

Support atomic transactions when performing CRUD (Rollback, Commit, isDirty, etc)
Workaround with SPList to get satisfactory performance when exceeding 2,000 items
Improve exception handling
Audit trial using list event handlers (features)
Logging

Things that i still have doubts of feasibility/performance:

Using singleton pattern for heavy MOSS objects like SPSite


My school is starting soon, and i'm still thinking whether is this project of mine worthy of my effort or not. I am not the best architecture designer out there as well, but i will try my best to complete the designs and do proper unit tests and to document the architecture. I'm actually actively looking for help to work on it, if you are interested please contact me.

Encrypting values in web.config

Sometimes user requirements don't allow you to put password or sensitive data in web.config.

There is a really simple to do it:

1. go to "command prompt"
2. type: cd "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\" (which every your drivespace is)
3. type: aspnet_regiis.exe -pef "<Name of node to encrypt>" "<Full path to web.config>"

Your done.



Edit post:

It was really that simple when i do it on my development server. But when i do the same thing on the SIT server, i was thrown with: "The RSA key container could not be opened"


Solution for me:

1. Ensure that you have permission to use the keys in "C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys" 
2. Check the keyContainerName in "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config"
3. go to "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727" type aspnet_regiis -pa "NetFrameworkConfigurationKey" "NT AUTHORITY/NETWORK SERVICE"

I'm assuming you are using the default name for NetFrameworkConfigurationKey.

Then go to the top of the post to do the encryption again.

Good luck.

Resizing array

If you ever had to work with an array that you wouldn't know the size till run-time, you have a couple of options:

1. Clone the old array into the new array (un-necessary overhead)
2. Just proceed with a array with redundant trailing allocated memory
3. You can resize it with Array.Resize(ref <arrayName>, int newArraySize)

int[] myArray = new int[50];
int newArrayCount = 0;

//some biz logic goes here
//determined array size

newArrayCount = myArray.Count;

Array.Resize(ref myArray, newArrayCount);

You can put myArray.Count as the 2nd parameter of the resizing method, up to your preference.

c#: boolean vs bool, String vs string, Int32 vs int

A frequently asked question, what is the difference between a Boolean and a bool object in c#? In short, it's actually the same thing. When compiled to MSIL(Microsoft Intermediate Language), they are referring to System.Boolean, thus there's no implication in performance.

But as a good conventional practice, i always use bool & string as opposed to Boolean & String. Unless you always use System.Int32 instead of int.

IIS logs 400 0 64

From my previous post, i've described how to consume a .NET webservice using netbeans. However, due to the JDK version on my colleague's solaris server, they are not able to import several libraries in order to do so.

Thus, they had use socket programming to send out the SOAP request. Everything was doing fine and well on development server, SIT server, and our local machines.

The wierd thing happens when the swing-over to UAT. When the function was called to send the SOAP request and in the while loop that waits for server response, it kept waiting... waiting... waiting... After several attempts to find faults in the java, web service and JDK versions. There was still no good explanation nor solutions. I went onto the web service machine and dug up the IIS Logs and found a couple of lines that showed the request reaching the web service machine but with an error code 400 0 64. After some research, i found out that the error code represents Bad request. Another vague error message -_-.

I almost wanted to give up, until i went into the HTTPERR folder and open up the logs and saw something that caught my attention. There was a request coming into the server via port 9090, but the response was sent to a random port number.

Bingo! In my client's environment, firewalls will block all the traffic going into the servers unless the need arises and we have to write in service requests to open up the port.

1 more problem resolved and hope this post will help anyone that hit this error.

Business objects VS Typed Datasets

 I always had this feeling that POCO (Plain old C# objects) will outperform typed datasets & datasets. The main reason why people are using it is only because it lessen development time by a very....... small amount in my opinion. Using business objects are easier to understand and in C# context, the compiler actually do a lot of work for you.

Example in other languages:

public myClass()
{
     private String someString;
  
     public String getString()
    {
        return this.someString;
    }
    public void setString(String value)
    {
        this.someString = value;
    }
}

you can do this in C#:
public myClass()
{
    public String someString { get; set; }
}

That only explains how much it takes to write a Business Object class. Next we go to real work.

I created a simple demo project in VS 2010 (beta), you can use any visual studio to compile this if you want to try it out yourself.

in my Business object class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace DemoBizObject
{
    public class BizObject
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Desc { get; set; }
        public decimal price { get; set; }
        public DateTime CreatedDt { get; set; }
        public DateTime ModifiedDt { get; set; }
    }
}

in my aspx code behind, i did this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Collections.ObjectModel;

using DemoBizObject;
using System.Diagnostics;
using System.Web.UI.DataVisualization.Charting;

namespace DemoWebApp
{
    public partial class _Default : System.Web.UI.Page
    {
        Collection<BizObject> BizObjects = new Collection<BizObject>();
        double[] pocotimings = new double[7];
        double[] typeddstimings = new double[7];

        decimal d = 0.00m;
        const int ONE = 1;
        const int TEN = 10;
        const int ONEHUNDRED = 100;
        const int ONETHOUSAND = 1000;
        const int TENTHOUSAND = 10000;
        const int ONEHUNDREDTHOUSAND = 100000;
        const int ONEMILLION = 1000000;
        const string LINEBREAK = "<br />";

        protected void Page_Load(object sender, EventArgs e)
        {
           
        }

        protected void Button1_Click(object sender, EventArgs e)
        {
            this.Label1.Text += "Typed Datasets" + LINEBREAK + LINEBREAK;
            this.Label1.Text += ONE +": "+ demoTypedDS(ONE)+ LINEBREAK;
            this.Label1.Text += TEN + ": " + demoTypedDS(TEN) + LINEBREAK;
            this.Label1.Text += ONEHUNDRED + ": " + demoTypedDS(ONEHUNDRED) + LINEBREAK;
            this.Label1.Text += ONETHOUSAND + ": " + demoTypedDS(ONETHOUSAND) + LINEBREAK;
            this.Label1.Text += TENTHOUSAND + ": " + demoTypedDS(TENTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEHUNDREDTHOUSAND + ": " + demoTypedDS(ONEHUNDREDTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEMILLION + ": " + demoTypedDS(ONEMILLION) + LINEBREAK + LINEBREAK;

            this.Label1.Text += "POCO" + ": " + LINEBREAK + LINEBREAK;
            this.Label1.Text += ONE + ": " + demoPOCO(ONE) + LINEBREAK;
            this.Label1.Text += TEN + ": " + demoPOCO(TEN) + LINEBREAK;
            this.Label1.Text += ONEHUNDRED + ": " + demoPOCO(ONEHUNDRED) + LINEBREAK;
            this.Label1.Text += ONETHOUSAND + ": " + demoPOCO(ONETHOUSAND) + LINEBREAK;
            this.Label1.Text += TENTHOUSAND + ": " + demoPOCO(TENTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEHUNDREDTHOUSAND + ": " + demoPOCO(ONEHUNDREDTHOUSAND) + LINEBREAK;
            this.Label1.Text += ONEMILLION + ": " + demoPOCO(ONEMILLION) + LINEBREAK + LINEBREAK;
        }
        private string demoTypedDS(int numOfRecords)
        {
            typedDS ds = new typedDS();
            Stopwatch watch = Stopwatch.StartNew();
           
            for (int i = 0; i < numOfRecords; i++)
            {
                ds.DemoTable.AddDemoTableRow(i, "demo name", "demo desc", d + i, DateTime.Now, DateTime.Now);
            }
            watch.Stop();
            return watch.Elapsed.TotalSeconds.ToString();
        }
        private string demoPOCO(int numOfRecords)
        {
            BizObject tempObject;
            Stopwatch watch = Stopwatch.StartNew();

            for (int i = 0; i < numOfRecords; i++)
            {
                tempObject = new BizObject();
                tempObject.Id = i;
                tempObject.Name = "demo name";
                tempObject.price = d + i;
                tempObject.CreatedDt = DateTime.Now;
                tempObject.ModifiedDt = DateTime.Now;
                tempObject.Desc = "demo desc";
                BizObjects.Add(tempObject);
            }
            watch.Stop();
            return watch.Elapsed.TotalSeconds.ToString();
        }
    }
}

pardon my messy codes, i didn't want to create this project initially, but i want to be someone that talks the talk and walk the walk.Next, create a similar table in your database with following...

 After that, right click on your solution file and click "Add new item" and select a dataset object.
Double click on the XSD file and go to your DB and drag-and-drop the table onto the XSD and you will get the table there..

The codes are very self explanatory and simple, not much re-using of codes especially @ the printing part.

I ran the test and get the results for POCO VS DS

Results are:
Typed Datasets

1: 4.1E-05
10: 5.47E-05
100: 0.0004515
1000: 0.0045912
10000: 0.0747852
100000: 0.6806366
1000000: 7.4931112

POCO:

1: 5.8E-06
10: 9.6E-06
100: 8.26E-05
1000: 0.0008128
10000: 0.0081756
100000: 0.1015654
1000000: 1.1922586

I tested it several times and dare to confirm that with this demo project, POCO is that Datasets in the lower range (1 to thousands) and the ratio justs exponential when the records reaches millions.

In this test, i only did instantiation and assignment, the Big O notation for:

• POCO = n Log n 
• Dataset = C^n

I don't find the need to extract data from database tables and going through the whole cycle as it will end up the same for both, the main difference only occurs when i instantiate and assign values to them. Even with just 6 primitive type objects, typed datasets are giving C^n in terms of performance. That is why my organization's Entity Objects are such a pain in the neck (they used typed datasets plus overriding several events)

Consume .NET webservice with only WSDL file

Objective: Consume .NET web service
Problem: I only have WSDL file (Server is blocked from my dev server)

In the past, a web service consumption was only: Right click on project, click on "Add web reference" and enter URL. But it's different this time as the production server had firewalls that blocked unauthorized/unnecessary access. I know it's possible to generate class files from the WSDL file, but how?

Time saver:.NET Framework Tools

After you generate the .cs file, CSC compile it into a dll and put the dll into your project and you will be able to invoke web methods immediately. For my case, i have to bring it to the production server to try access it, and it did worked perfectly :)

Performance bottleneck? review boxing & unboxing frequencies

There are times where our applications exceed users' response time threshold of 7s (usually), where all action must be completed under 7s.

Page-by-page code reviews are the last resort, but when you are resorting to that, look out of redundant boxing and unboxing of objects, check out: Technet

That's usually my case, or un-necessary objects declaration and recursion.

It's always better to go with iteration than recursion in terms of performance, as well as a typical For loop will outperform a foreach loop due to the boxing and unboxing overhead.

Check the traffic, CPU & memory utilization for resource bottleneck.

The last option is to review the entire architecture of the application (a PM's nightmare as that means a lot of time and resource needs to be pumped into the project). If good architectural designs was laid in place before development start, it wouldn't have to come to this point :)

Consuming .NET webservice using Java (netbeans 6.8) in 2 mins

I'm not a very Java person, but i was tasked to assist a team of Java developers using Solaris server. (A .Net developer's nightmare) I'm a GUI person and didn't like the command line concept, well it's powerful and have very little overhead but i need time to learn before jumping into it.

I told the team that i'm a .NET person, not a java guy. They told me leave the console/terminal/ftp to them, just help them to consume a .NET web service. Good.

I asked them from the WSDL file and went to research how java can consume a web service.

Luckily, there was a lot of resource out there documenting how to consume web service in netbeans.

First, create a New Project in NetBeans, give it a meaningful name.
Second, add a file and select "Web service" tab.
third, browse the wsdl file and click "Finish".

Now NetBeans will auto generate all the dependent files.

Expand the tree under your webservices and you will be able to see the web methods with red dots.

Drag and drop them into your methods and all the objects will be automatically instantiated and invoked with default values.

Using your java DAO(Data Access Objects) or equivalent and assign them the correct values and hit then run button.

There you go, java consumption of .NET web service in under 2 mins.

IIS6 - The specified handle is invalid

I've met an issue on a development server whereby the IIS just simply refuses to start. I took a step-by-step diagnostic approach and went to "inetmgr", and all i see is just a IIS without a "+" sign.

When i right click on it and start, it just loads a progress bar and terminates. No error messages, no prompts, no nothing!

I remembered that IIS needs IIS Admin service to be running in order to start, so i checked "services.msc" and i saw the problem. IIS Admin service wasn't running even though it was set to be automatic. When i try to start the service, it gave me a very "helpful" error message

"Error 6: - The specified handle is invalid".

After scouring around the net, i manage to find a couple of solutions, tried all of them and IIS still wouldn't budge. I told the support team to reinstall IIS and we will reconfigure all the sites. Going home and waiting for the reinstallation to finish, i came back when they informed me and found out that the IIS gave me an even better error message now.

"System cannot find the path specified"
Very pissed off, i decided to dig deeply into the root of the problem. I found out from the rest of the team that someone had actually messed around with the server. He was trying to get a X.509 certificate going in the server, but somehow he messed up IIS configs.

After a couple more hours of troubleshooting, i found out that the error message literally implies the truth. IIS is unable to find a couple files that it has direct dependencies. One of which is MetaBase.xml and MBSchema.xml 


True enough, when i navigate to the directory ":\WINDOWS\system32\inetsrv" i saw MetaSchema.xml was there, but MetaBase.xml was missing. I also found out that IIS creates 10 versions of backups whenever there are changes to its configs. I took the latest and renamed it back to MetaBase.xml and everything was up and running again.

MVC architecture in Sharepoint environment

A lot of webpart developers (from what i observed in MSDN forums) always create webparts with the typical

CreateChildControls() and Render() to generate UI, and attempt to use Render, delegates to control the UI

It's very difficult to understand and code as many people know, but it's actually possible to split all this into 3 well-defined layers.

1st you need : SPMetal to create Entity objects, next you can use Linq to Sharepoint, (Sadly, it's out of support or development by the author as Sharepoint 2010 already have EF v4.0 (Entity Framework), which interop nicely between sharepoint and linq.

2nd you will need your own DAL(Data access layer), with Entity objects already catered for, you should have no problem doing the CRUD with it

3rd: you can use features like QuickPart to render UserControls, have a general idea now? With quickpart, you can generate UI easier, as well as validation, multi-views etc, the list just goes on and on.


And there you go, you have QuickPart as (View), DAL (Controller), SPMetal (Entity objects/Model), however, it still requires more effort to do transactional executions, Singleton/doubleton/n-ton SPWeb, SPSite objects (something like a connection pool).

Having problems with CAML?

When i first started MOSS development 1 1/2 years ago, i was still very weak in SQL, let alone querying Lists using CAML. Until i found this tool: CamlQueryBuilder

You can connect directly to your sharepoint site and start querying your lists, no need for modifying your CAML and redeploying your webparts to test your CAML queries. It certainly help speed development up by a few folds.

Generating report from McAfee sidewinder 7.0 log files

Problem:

My colleague was tasked to generate report from the log files churned out by sidewinder 7.0.

Something like this.....



------------------------------------------------------------
SOURCE ADDRESS DESTINATION ADDRESS SERVICE COUNT RULE
------------------------------------------------------------
xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx ping 15000 12345


------------------------------------------------------------
DESTINATION IP COUNT PERCENTAGE
------------------------------------------------------------
xxx.xxx.xxx.xxx 17281 8.3333%


------------------------------------------------------------
SOURCE IP COUNT PERCENTAGE
------------------------------------------------------------
xxx.xxx.xxx.xxx 17281 8.3333%


------------------------------------------------------------
SERVICE NAME COUNT PERCENTAGE
------------------------------------------------------------
ping 17281 8.3333%


------------------------------------------------------------
DATE COUNT PERCENTAGE
------------------------------------------------------------
28-02-2010 17281 8.3333%


------------------------------------------------------------
HOURS COUNT PERCENTAGE
------------------------------------------------------------
12AM - 1AM 17281 8.3333%

Points to note:

There was an existing Perl script that does the conversion from 6.0 -> 7.0, but a monthly log conversion takes roughly 12 hrs to complete one conversion

Solution:

Since i'm more trained in C#, i've chose to write a windows application for him, and for me to put my designing & algorithm skills to test.

Some sample log data:

28-02-2010 00:00:00 Local0.Notice 172.16.20.3 Feb 27 23:56:57 auditd: date="Feb 27 23:56:57 2010 UTC",fac=f_ping_proxy,area=a_proxy,type=t_nettraffic,pri=p_major,pid=63779,ruid=0,euid=0,pgid=63779,logid=0,cmd=pingp,domain=Ping,edomain=Ping,hostname=xxxxxxxxx,event=proxy traffic end,service_name=ping,netsessid=4b89b107000c933b,srcip=xxx.xxx.xxx.xxx,srcburb=dmz2,protocol=1,dstip=xxx.xxx.xxx.xxx,dstburb=external,bytes_written_to_client=40,bytes_written_to_server=40,rule_name=xxxxxxxxx,cache_hit=0,request_status=0,start_time="Sat Feb 27 23:55:51 2010"

The first problem i realised was that the supposed "CSV" (Comma-seperated-values) are not really a CSV, it was part tab delimited and comma delimited. Well, it wasn't that hard to tokenize/split it, just do a string.Split(',') followed by string.Split('\t').

string[] splitByTabArray = new string[5];

string[] splitByCommaArray = new string[30];

if (!string.IsNullOrEmpty(line))

{

    splitByTabArray = line.Split('\t');

}

if (!string.IsNullOrEmpty(OneLineSplitted[4]))

{

    splitByCommaArray = splitByTabArray[4].Split(',');

}

So now, we have 2 collections of string array. The next problem is that i've only shown you one line of sample log, the actual log is much more complicated. Why do i say so?

Because in the auditd: section, the number of string objects to be split varies in quantities. To make things easier, i've asked my colleague what columns are used to generate the reports. 

After that i made a struct.

public struct Report

        {

            public DateTime Time

            {

                get;

                set;

            }

            public string Date

            {

                get;

                set;

            }

            public string Type

            {

                get;

                set;

            }

            public string SrcIp

            {

                get;

                set;

            }

            public string DestIp

            {

                get;

                set;

            }

            public string Pid

            {

                get;

                set;

            }

            public string ServiceName

            {

                get;

                set;

            }

        }

The reason i've created a struct is because i have to group records as you can see from the sample report file. But that doesn't solve the problem with a struct, you might think. Yup it didn't but it gives me IQueryable support so i can use LinQ to do grouping, selecting, joining etc.

A sample query:

var topDest = (from oneReport in allReports

                           where oneReport.DestIp != null && oneReport.Type.Equals(TYPE)                          

                           group oneReport by oneReport.DestIp into groupedReports

                           orderby groupedReports.Count() descending

                          

                          select new { groupedReports.Key, Count = groupedReports.Count(), Total = allReports.Count }).Take(10);

I've decided not to put up the full source code up as, it's still not perfect, optimized and... i have no idea, i just want to keep it till someone out there that really needs it contact me and i would share it.

A brief approach for solution:

1. Tokenizing all the required data
2. Using a struct to give me IQueryable support and ability to work with object

And to top it off, i manage to convert 60MB of logs in 15seconds, and 7GB in less than an hour. Talk about efficiency :)

Microsoft Sharepoint 2007

What exactly is MOSS designed for? In short ECM or enterprise content management, it was never meant to be a platform or framework for developers to develop on. Sadly, my management and clients perceived MOSS as if it was a new language like C#, java, etc.

They are always asking me, "How come i can do that in .NET and Java, but why not in MOSS?". Well first of all, as i said, it's not Application framework like .NET and Java!!

Users are always comparing MOSS with .NET and Java just because to them, anything that can display pictures, buttons, tables is an app. We can't blame them though, in the eyes of a not-so-technically inclined person, that don't and will not understand the true use of MOSS.

I was being tasked to learn how to develop, administer and set up MOSS. It came as a double edged sword. I got my new job due to my skills and knowledge from there but it also hit me very hard when explaining to stakeholders and analysts.

I'm sometimes tired of people saying, "Why SharePoint list have such lousy performance?? 2,000 items limit and it's so slowwwww". Well, things are different in MOSS 2010, there are better support and scalability which boasts millions of list items and still give acceptable performance.

To quote my previous employer, "We are the Elites of the software development world, we are offered projects mainly because we are able to do things that most people said that it cannot be done!" Well, it's very good on the balance sheet but underlings like me are the ones that are struggling to explore the unchartered terrain. It propelled me into more abstract development designs..

OSX vs Windows

I've been using windows since the day 1, and was quite disappointed in certain ways. I guess i do not have to list down everything but these are the most frustrating ones:

1. BSOD (Blue screen of death)
2. Registry
3. Virus, trojans (Malware in general)

I bought myself a MacBook (upgraded to 4gb ram for Virtual machines) on Feb 2010, and was amazed by the way Apple manage to simplify, beautify the entire OS. The most important factor to me was actually the performance. Having Spotlight to easily access any thing on your mac and with all the eye candy, it's still running super fast. I'm a very heavy user as i always run VirtualBox(2GB memory) for my development work, iMail, google chrome browser. I've yet encountered any system error till now and mac never fail to amaze me at how efficiently the entire OS is engineered with their priorities set on visual effects, ease of use and performance. I have never hesitated on downloading mail attachments or random files from the net.

The only problem i face when i switch to a Mac was, the mouse sensitivity. It was wierd to have such acceleration and sometimes when your pointer is right next to what you want to click, it seems like you are moving your mouse in sand. Well, the best solution was actually to use a (ironically) Microsoft IntelliSense mouse. After installing the drivers, you can have a mouse that accelerates like what you have on windows.

Next was the lack of MSN, i use MSN everyday and the best replacement for MSN on mac was Adium, but that didn't have the "Groups" option and i couldn't message my friends & colleagues in native MSN. I'm still hoping that Microsoft will actually release a fully featured, functional MSN for OSX.

The last thing was the File System.

A brief history of Microsoft OS:

In the past, Windows uses FAT16/FAT32, which was replaced by NTFS.

Mac uses HFS+

You might think what does that got to do with you? Well, first of all, Windows with it's gargantuan OS market share had made most people use NTFS for external HDD, that's when the problem starts, OSX have native support to read only (although you can download apps to write into NTFS), but Windows users can't read HFS+, if you use TimeMachine (Apple's backup program), you have to format the HDD to HFS+, then your HDD is officially Mac only.

It took me 4 hours trying to get the best of both worlds as i only have a 500gb HDD, i ended up with a 250GB partition with HFS+, 250GB with NTFS. And i use the HFS+ partition purely for TimeMachine back ups, and NTFS for file transfer with my Windows-using friends.

P.S. 1 major difference i found out is that HFS+ doesn't require you to defragment it (very little fragmentation) whereas NTFS requires you to do it on regularly basis.

The very first post

A short intro of purpose of this site:

I've been doing development for the past 2 years and found a lot of help from fellow engineer's blog and would like to have one myself. I will post my findings, views, solutions and problems encountered in my line of work, school (yes still schooling part-time) and personal development and research work.

My field of interest will be more on architecture design, efficiency and commonly used codes (there's a lot of copy&paste programmers out there).

My preferred language is C#.NET, although i am very limited trained in C++, C and java.

My work involves mostly but not exhaustively Microsoft Office Sharepoint Server 2007, .NET framework, MS SQL database.

I'm also a Mac user, although i have to use VirtualBox to virtualize Visual Studio, (MONO wasn't good enough for me)

Hope to post more along my line of work.