Question

I have a list of users on my php application (using codeigniter). Each user may have completed a form with about 1000 or so total fields. The structure looks similar to this:

users

id|username|...

completed_form_fields

id|formid|userid|fieldkey|data

where field key is just a unique key for that particular form field, ie: “first_name”

I have a user search page where people can filter out specific users by the fields they chose (eye color, race, gender…) Then I need to display these fields so I would love (and currently have) an output like this:

$filteredmembers = array(
[0] = Object(
   [id] => 1
   [username] => kilrizzy
...
   [fields] => Array(
        [fname] => Jeff
        [gender] => Male
        ...

Currently my script is obviously taking forever since I query all the members who filled out this form, then loop through each one to query all of their fields. THEN filter those out based on criteria + page / offset.

I know there needs to be a way to join these together in one query I am not familiar with

Simplified version of my very slow code:

function get_members(){
    $this->db->select('u.*');
    $this->db->from('users AS u');
    $query = $this->db->get();
    if ($query->num_rows() > 0){
        $members = $query->result();
        //Get fields from each user
        foreach($members as $mk => $mv){
            $fields = $this->get_form_fields($mv->id,1,true);
            $members[$mk]->fields = $fields;
        }
        return $members;
    }else{
        return false;   
    }
}
function get_form_fields($uid,$form,$values=false){
    $this->db->where('user', $uid); 
    $this->db->where('form', $form); 
    $query = $this->db->get('form_fields');
    if ($query->num_rows() > 0){
        $result = $query->result();
        return $result;
    }else{
        return false;   
    }
}

Answer 1

There is a way but it gets over expensive the more you add fields. The same thing occurs with many CMS that choose to store additionnal user data in that form.

You can get a working search SQL using this:

SELECT
    users.*,
    firstname.data AS firstname,
    lastname.data AS lastname,
    eyecolor.data AS eyecolor,

FROM
    users
    LEFT JOIN completed_form_fields AS firstname ON firstname.userid = users.id AND firstname.fieldkey = "firstname"
    LEFT JOIN completed_form_fields AS lastname ON lastname.userid = users.id AND lastname.fieldkey = "lastname"
    LEFT JOIN completed_form_fields AS eyecolor ON eyecolor.userid = users.id AND eyecolor.fieldkey = "eyecolor"

WHERE
    firstname.data LIKE '%searchdata%'
    OR lastname.data LIKE '%searchdata%'
    OR eyecolor.data LIKE '%searchdata%'

This method gets very big and expensive for the MySQL server the more you add tables. Therefore, i would recommend not to go more than 10-15 joins like that and then again, i’d profile it to make sure.