I’m going to work on a Customer Profiling project (similar but not same to Google Analytics) for our own E-Commerce website using C#. I’m pretty new to this kind of project, and the Customer Profiling project is also a brand new project. Could you give me some suggestion on that?
I think there should be two parts, the first is “tracking visiting data of the website“, the second is “analysis of the tracked data“.
Could you give me some suggestion on that? Thanks:
- What kind of design patterns is most suitable for this kind of project? (Pipeline and Filters? or others?) I’m using C#.
- What kind of database is most suitable? RDBMS or document database?
- How to model the database tables for storing the tracked data?
- What keywords can I use to search this subject on Google?
- Could you suggest me some articles or books to read? (Articles are better, because I don’t have so much time to read a book)
- Any other stuff you think I need to learn.
Thanks in advance!
First, keep in mind that you have more time than you think. You just need to get started soon. Be sure to tell your client/boss (and even yourself) that the first thing to be done is to establish what is needed (requirements gathering).
Needs lead to Design which leads to a Solution (Database, platform etc.) To get started, you mainly need to conceptualize what your client is seeking as the end result.
Let’s take your questions one by one:
Q: What kind of design patterns is most suitable for this kind of project? (Pipeline and Filters? or others?) I’m using C#.
Again, the actual requirements will dictate the design paradigm. You don’t want to shoehorn a design pattern upfront. That said, yes Pipelines (or Funnel Analysis) is probably the most common paradigm for tracking visitors to a site. If you search for “funnel” or “web visitor tracking” you will get many relevant articles. (See also the list of resources mentioned below.)
Q: What kind of database is most suitable? RDBMS or document database?
You weblog will typically text files, with one record (row) recording lots of pieces of information about each visit. If you are planning to perform analysis on a regular basis, you would then build a relational database to store summary stats and detailed reports based on the individual weblogs.
Q: How to model the database tables for storing the tracked data?
Here’s the general principle that you’d follow.:
The design typically includes one set of tables for your site “customers” or “visitors.” Relevant data about the customers would be stored there, with a unique way to identify each visitor. The second set of tables would be around the individual visits to your site by the customers being profiled. You build in any summarized statistics of their visits that will be useful for your future analysis. The two sets of tables are linked via unique visitor/customer ID’s.
(That is a very high level idea. Database design is a huge field in itself, with lots of options each with their own advantages and drawbacks. You will need to experiment before finalizing your design.
Q: What keywords can I use to search this subject on Google?
Try:
* Customer Analytics
* Customer Segmentation
* Web Traffic Analysis
* Website visitor funnel
* Funnel Analysis (SEO)
This SEO Glossary will give you lots more ideas for keywords.
Q: Could you suggest me some articles or books to read? (Articles are better, because I don’t have so much time to read a book)
Articles/Posts:
As always, Wikipedia is a good place to start. It has articles on Traffic Analysis and Web Visitor Tracking
Caveat: There are numerous sites that peddle SEO with big promises. Many of these are okay to browse to see what they offer, but be very wary of their claims.
Books
There are lots of books on Web Analytics. The Dummies book is good, and I can also recommend Avinash Kaushik’s “Web Analytics: An Hour a Day”
Not all books will be at the level you are seeking, so be sure to browse the book (if at all possible) before buying it.
Here are the current bestselling Analytics books on Amazon.
Q: Any other stuff you think I need to learn.
Just keep in mind that many people have done this before. Like all of us, you will learn on the job. Stay confident and keep an open mind.
Hope that helps. Good luck.