Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I’m computing data from database (~130 000 000 rows).
Because of big amount of rows I select 1 mln compute them save results then select another 1 mln and so on.
I use select .. orderby .. skip m... take n...ToList()
because I want to have this objects in memory.
When I skip 1 mln then 2 mln then 3 mln … then lets say 6 mln its quite ok but then suddenly query takes very long.
Have you got the same problem ?
Is there any way I can make it work faster ?
Thanks for help,
Bye
If you look at the generated SQL, you will see the problem. SQL Server does not have a native SKIP, so the Entity Framework improvises around this. I’ve explained some of the details in this post.
To do this efficiently, you need to partition your data by a different method, one which can be implemented by the server using an index. Without knowing more about the problem, I can’t say what the best approaches, but look for a means of partitioning your data which could be indexed in a SQL query.