I have a very large file and i need to process each line (each line of file is independent). How can I use goroutines (or should I not use them?) to read the file in the fastest way?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I have a very large file and i need to process each line (each line of file is independent). How can I use goroutines (or should I not use them?) to read the file in the fastest way?
As long as your hard disk is orders of magnitude slower than your CPU, which is still a quite common situation, then you cannot magically make the file reading (domain: from a single HD) any faster by throwing more CPU cycles onto it. (Assuming cold file caches and/or file size much bigger then all available file cache memory).