Question

I have a project model that has a FK to a group model. Group has name and two other fields: (bool) is_private indicating if the group is private and FK to django.auth.models.Group to indicate the group membership that this project group is visible to.

There are 100 projects and they are happily fetched in less than 1 second:

def project_list(request):
    parameters = {field_name: value for field_name, value in request.GET.items() if value and field_name in project._meta.get_all_field_names()}
    all_projects = project.objects.select_related().filter(**parameters)
    return  all_projects

there is only one project with a private group and as soon as I add the privacy check, the fetch it taking more than 4 seconds to do:

def project_list(request):
    parameters = {field_name: value for field_name, value in request.GET.items() if value and field_name in project._meta.get_all_field_names()}
    all_projects = project.objects.select_related().filter(**parameters)
    for p in all_projects.filter(group__is_private = True):
        if not request.user.groups.filter(name = p.group.private_group.name).exists():
        all_projects = all_projects.exclude(id=p.id)
    return  all_projects

Is there a way to redo this to take less time?

Answer 1

In the code below you’re fetching single group in every iteration, so you’re making lot of database queries which slows things down:

all_projects = project.objects.select_related().filter(**parameters)
for p in all_projects.filter(group__is_private = True):
    if not request.user.groups.filter(name = p.group.private_group):

If you want to fetch projects without private group or with private group only if user is in that group, then use:

from django.db.models import Q

def project_list(request):
    parameters = {field_name: value for field_name, value in request.GET.items() if value and field_name in project._meta.get_all_field_names()}

    group_ids = request.user.groups.values_list('id', flat=True)
    all_projects = project.objects.select_related().filter(**parameters)
    all_projects = all_projects.filter(Q(group__is_private=False) | Q(group__is_private=True, group_id__in=group_ids))
    return  all_projects