I have a dataset (where each data is a vector of attributes with their corresponding class label). I want to split the dataset to a training set and testing set. Is there anyway to do this automatically ?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I have a dataset (where each data is a vector of attributes with their corresponding class label). I want to split the dataset to a training set and testing set. Is there anyway to do this automatically ?
The typical way to split a modeling data set into training, validation, and test sets is to use a random sample. That is, assign a random number between 0 and 1. If you want a 40/30/30 split, then rows where the value is between 0 and 0.4 are in training, between 0.4 and 0.7 in validatino, and 0.7 and 1.0 in test. There is nothing magical about the 40/30/30 slpit. It just happens to be the default in SAS Enterprise Miner (in fact, I often change it to 60/30/10).
There are some tweaks and possible improvements to this. If you know there are important features for modeling, such as geography, then you can do a stratified sample. For this, you would sort the data by the columns and then essentially do an “every nth” record sample. I say “essentially”, because this is a little more complicated for a 40% split. To handle this, take the record 10 at a time, choose 4 for the training set, 3 for the validation set, and 3 for the test set. Which particfular ones you choose from the 10 are not important.
A bigger issue is when you have hierarchical data, and virtually all data used for modeling is hierarchies. For instance, you might have customer data with a significant number of columns describing the customer’s census tract. If these variables are important as predictors, then you might consider sampling at the census tract level, rather than the customer level. That is, divide the census tracts into three groups (randomly), so 40% of customers go to the training set, 30% to the validation set, and 30% to the test set.
You want to be sure that you partition the data, and no record falls into more than one group. If you don’t know what training, validation, and test sets are, then I would highly recommend that you get a book on data mining (such as “Data Mining Techniques for Marketing, Sales, and Customer Support, Third Edition” at http://www.amazon.com/Data-Mining-Techniques-Relationship-Management/dp/0470650931/ref=pd_sim_b_5).