Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I have a 2.5 GB dataset, which is quite large for my 4GB memory. I wonder if converting character variables to factors will save space and processing time.
I would imagine that internally, factors will be stored in numeric with a lookup table for levels. But I am not sure how it actually works.
Converting to factor won’t save space because characters are stored in a hash table. See section 1.10 The CHARSXP cache of R Internals.
Converting to factor may improve processing time if your code would need to convert to factor (running a regression, classification, etc.), but it won’t improve processing time if you’re doing string manipulation because it would have to convert the factor back to a character. So it really depends on what you’re doing.