Sunday, August 17th, 2008
What it takes to create a CEO-friendly tool allowing to analyze custom data sets?
Article graciously provided by Sylvia Hausnerova, edited by Frank Malina.
Data mining is a process of analysis of a data set aiming
- to answer a predefined question
- to recognize a pattern within a data set
Data-mining process
Application should keep in mind the following user-flow:
- Set a question you need to answer.
- Issue queries on a dataset that will give you insight on your question.
- Take actions to take advantage of the new insight.
OR a more casual work flow:
- Browse the dataset in various ways.
- Try to recognize patterns.
- Take actions to take advantage of the recognized patterns.
What can data mining do?
Data mining is commonly used in analysis of web site traffic data. Traffic data analysis can give you useful marketing and sales related data.
Such as:
- When do people shop in my e-commerce store?
When should I bid on ads displayed in Google to increase traffic to my site for cheaper?
- Where from are my customers?
Where should I invest to off-line marketing to make my brand stronger?
- What browsers my users use?
Should I invest money in testing the design of my site in particular web browser? Will I loose money for not testing my design in IE5.
Most commonly used tool for analysis of web traffic data as mentioned above is Google analytics.
However what it takes to create a CEO-friendly tool allowing to analyze custom data sets such as off-line sales data?
What technology to create Analytics for custom data sets?
Generally there a tree layers of software to be written
- Data warehouse - a database that stores the collected data (MySQL, PostgreSQL, Oracle to name a few). This is probably in place already.
- Interface for data queries - SQL object is a great tool allowing to create interfaces for data mining. Most of the functionality of SQL object is now baked in the Django web application framework.
- Interfaces that display the data retrieved from the warehouse in a human readable form (pie charts, tables, graphs). There a various techniques to make this happen such as open flash chart, CSS bar graphs and HTML tables enhanced with AJAX scripting.
So what do you think?