MVC architecture in Sharepoint environment

A lot of webpart developers (from what i observed in MSDN forums) always create webparts with the typical

CreateChildControls() and Render() to generate UI, and attempt to use Render, delegates to control the UI

It's very difficult to understand and code as many people know, but it's actually possible to split all this into 3 well-defined layers.

1st you need : SPMetal to create Entity objects, next you can use Linq to Sharepoint, (Sadly, it's out of support or development by the author as Sharepoint 2010 already have EF v4.0 (Entity Framework), which interop nicely between sharepoint and linq.

2nd you will need your own DAL(Data access layer), with Entity objects already catered for, you should have no problem doing the CRUD with it

3rd: you can use features like QuickPart to render UserControls, have a general idea now? With quickpart, you can generate UI easier, as well as validation, multi-views etc, the list just goes on and on.


And there you go, you have QuickPart as (View), DAL (Controller), SPMetal (Entity objects/Model), however, it still requires more effort to do transactional executions, Singleton/doubleton/n-ton SPWeb, SPSite objects (something like a connection pool).

Having problems with CAML?

When i first started MOSS development 1 1/2 years ago, i was still very weak in SQL, let alone querying Lists using CAML. Until i found this tool: CamlQueryBuilder

You can connect directly to your sharepoint site and start querying your lists, no need for modifying your CAML and redeploying your webparts to test your CAML queries. It certainly help speed development up by a few folds.

Generating report from McAfee sidewinder 7.0 log files

Problem:

My colleague was tasked to generate report from the log files churned out by sidewinder 7.0.

Something like this.....



------------------------------------------------------------
SOURCE ADDRESS DESTINATION ADDRESS SERVICE COUNT RULE
------------------------------------------------------------
xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx ping 15000 12345


------------------------------------------------------------
DESTINATION IP COUNT PERCENTAGE
------------------------------------------------------------
xxx.xxx.xxx.xxx 17281 8.3333%


------------------------------------------------------------
SOURCE IP COUNT PERCENTAGE
------------------------------------------------------------
xxx.xxx.xxx.xxx 17281 8.3333%


------------------------------------------------------------
SERVICE NAME COUNT PERCENTAGE
------------------------------------------------------------
ping 17281 8.3333%


------------------------------------------------------------
DATE COUNT PERCENTAGE
------------------------------------------------------------
28-02-2010 17281 8.3333%


------------------------------------------------------------
HOURS COUNT PERCENTAGE
------------------------------------------------------------
12AM - 1AM 17281 8.3333%

Points to note:

There was an existing Perl script that does the conversion from 6.0 -> 7.0, but a monthly log conversion takes roughly 12 hrs to complete one conversion

Solution:

Since i'm more trained in C#, i've chose to write a windows application for him, and for me to put my designing & algorithm skills to test.

Some sample log data:

28-02-2010 00:00:00 Local0.Notice 172.16.20.3 Feb 27 23:56:57 auditd: date="Feb 27 23:56:57 2010 UTC",fac=f_ping_proxy,area=a_proxy,type=t_nettraffic,pri=p_major,pid=63779,ruid=0,euid=0,pgid=63779,logid=0,cmd=pingp,domain=Ping,edomain=Ping,hostname=xxxxxxxxx,event=proxy traffic end,service_name=ping,netsessid=4b89b107000c933b,srcip=xxx.xxx.xxx.xxx,srcburb=dmz2,protocol=1,dstip=xxx.xxx.xxx.xxx,dstburb=external,bytes_written_to_client=40,bytes_written_to_server=40,rule_name=xxxxxxxxx,cache_hit=0,request_status=0,start_time="Sat Feb 27 23:55:51 2010"

The first problem i realised was that the supposed "CSV" (Comma-seperated-values) are not really a CSV, it was part tab delimited and comma delimited. Well, it wasn't that hard to tokenize/split it, just do a string.Split(',') followed by string.Split('\t').

string[] splitByTabArray = new string[5];

string[] splitByCommaArray = new string[30];

if (!string.IsNullOrEmpty(line))

{

    splitByTabArray = line.Split('\t');

}

if (!string.IsNullOrEmpty(OneLineSplitted[4]))

{

    splitByCommaArray = splitByTabArray[4].Split(',');

}

So now, we have 2 collections of string array. The next problem is that i've only shown you one line of sample log, the actual log is much more complicated. Why do i say so?

Because in the auditd: section, the number of string objects to be split varies in quantities. To make things easier, i've asked my colleague what columns are used to generate the reports. 

After that i made a struct.

public struct Report

        {

            public DateTime Time

            {

                get;

                set;

            }

            public string Date

            {

                get;

                set;

            }

            public string Type

            {

                get;

                set;

            }

            public string SrcIp

            {

                get;

                set;

            }

            public string DestIp

            {

                get;

                set;

            }

            public string Pid

            {

                get;

                set;

            }

            public string ServiceName

            {

                get;

                set;

            }

        }

The reason i've created a struct is because i have to group records as you can see from the sample report file. But that doesn't solve the problem with a struct, you might think. Yup it didn't but it gives me IQueryable support so i can use LinQ to do grouping, selecting, joining etc.

A sample query:

var topDest = (from oneReport in allReports

                           where oneReport.DestIp != null && oneReport.Type.Equals(TYPE)                          

                           group oneReport by oneReport.DestIp into groupedReports

                           orderby groupedReports.Count() descending

                          

                          select new { groupedReports.Key, Count = groupedReports.Count(), Total = allReports.Count }).Take(10);

I've decided not to put up the full source code up as, it's still not perfect, optimized and... i have no idea, i just want to keep it till someone out there that really needs it contact me and i would share it.

A brief approach for solution:

1. Tokenizing all the required data
2. Using a struct to give me IQueryable support and ability to work with object

And to top it off, i manage to convert 60MB of logs in 15seconds, and 7GB in less than an hour. Talk about efficiency :)

Microsoft Sharepoint 2007

What exactly is MOSS designed for? In short ECM or enterprise content management, it was never meant to be a platform or framework for developers to develop on. Sadly, my management and clients perceived MOSS as if it was a new language like C#, java, etc.

They are always asking me, "How come i can do that in .NET and Java, but why not in MOSS?". Well first of all, as i said, it's not Application framework like .NET and Java!!

Users are always comparing MOSS with .NET and Java just because to them, anything that can display pictures, buttons, tables is an app. We can't blame them though, in the eyes of a not-so-technically inclined person, that don't and will not understand the true use of MOSS.

I was being tasked to learn how to develop, administer and set up MOSS. It came as a double edged sword. I got my new job due to my skills and knowledge from there but it also hit me very hard when explaining to stakeholders and analysts.

I'm sometimes tired of people saying, "Why SharePoint list have such lousy performance?? 2,000 items limit and it's so slowwwww". Well, things are different in MOSS 2010, there are better support and scalability which boasts millions of list items and still give acceptable performance.

To quote my previous employer, "We are the Elites of the software development world, we are offered projects mainly because we are able to do things that most people said that it cannot be done!" Well, it's very good on the balance sheet but underlings like me are the ones that are struggling to explore the unchartered terrain. It propelled me into more abstract development designs..

OSX vs Windows

I've been using windows since the day 1, and was quite disappointed in certain ways. I guess i do not have to list down everything but these are the most frustrating ones:

1. BSOD (Blue screen of death)
2. Registry
3. Virus, trojans (Malware in general)

I bought myself a MacBook (upgraded to 4gb ram for Virtual machines) on Feb 2010, and was amazed by the way Apple manage to simplify, beautify the entire OS. The most important factor to me was actually the performance. Having Spotlight to easily access any thing on your mac and with all the eye candy, it's still running super fast. I'm a very heavy user as i always run VirtualBox(2GB memory) for my development work, iMail, google chrome browser. I've yet encountered any system error till now and mac never fail to amaze me at how efficiently the entire OS is engineered with their priorities set on visual effects, ease of use and performance. I have never hesitated on downloading mail attachments or random files from the net.

The only problem i face when i switch to a Mac was, the mouse sensitivity. It was wierd to have such acceleration and sometimes when your pointer is right next to what you want to click, it seems like you are moving your mouse in sand. Well, the best solution was actually to use a (ironically) Microsoft IntelliSense mouse. After installing the drivers, you can have a mouse that accelerates like what you have on windows.

Next was the lack of MSN, i use MSN everyday and the best replacement for MSN on mac was Adium, but that didn't have the "Groups" option and i couldn't message my friends & colleagues in native MSN. I'm still hoping that Microsoft will actually release a fully featured, functional MSN for OSX.

The last thing was the File System.

A brief history of Microsoft OS:

In the past, Windows uses FAT16/FAT32, which was replaced by NTFS.

Mac uses HFS+

You might think what does that got to do with you? Well, first of all, Windows with it's gargantuan OS market share had made most people use NTFS for external HDD, that's when the problem starts, OSX have native support to read only (although you can download apps to write into NTFS), but Windows users can't read HFS+, if you use TimeMachine (Apple's backup program), you have to format the HDD to HFS+, then your HDD is officially Mac only.

It took me 4 hours trying to get the best of both worlds as i only have a 500gb HDD, i ended up with a 250GB partition with HFS+, 250GB with NTFS. And i use the HFS+ partition purely for TimeMachine back ups, and NTFS for file transfer with my Windows-using friends.

P.S. 1 major difference i found out is that HFS+ doesn't require you to defragment it (very little fragmentation) whereas NTFS requires you to do it on regularly basis.

The very first post

A short intro of purpose of this site:

I've been doing development for the past 2 years and found a lot of help from fellow engineer's blog and would like to have one myself. I will post my findings, views, solutions and problems encountered in my line of work, school (yes still schooling part-time) and personal development and research work.

My field of interest will be more on architecture design, efficiency and commonly used codes (there's a lot of copy&paste programmers out there).

My preferred language is C#.NET, although i am very limited trained in C++, C and java.

My work involves mostly but not exhaustively Microsoft Office Sharepoint Server 2007, .NET framework, MS SQL database.

I'm also a Mac user, although i have to use VirtualBox to virtualize Visual Studio, (MONO wasn't good enough for me)

Hope to post more along my line of work.