Question

I’m still fairly new to SQL and I’m not fully understanding where the problem in my code is coming from. The code below mostly comes from my work so I didn’t write it from scratch. The code gathers a bunch of different information and filters based on it. If you look in the code you’ll see that a student has many observations_students which it is related to. The first version of the code returns the information of all students who have an observations_student with observation_id = 2567. This seems to work correctly with the following code:

SELECT DISTINCT
SUBSTRING(s.osis_id,INSTR(s.osis_id,'-')+1) AS osid,
s.id AS student_id,
CONCAT(s.last_name, ' ',s.first_name) AS sname
FROM students s

# course info
INNER JOIN 
(
    SELECT c.id AS cid,
    c.description AS cname,
    cs.date_end,
    cs.student_id,
    gl.description AS grade,
    c.gradelevel_id
    FROM courses_students cs
    INNER JOIN courses c ON c.id = cs.course_id
    INNER JOIN gradelevels gl ON gl.id = c.gradelevel_id
    WHERE
    IFNULL(cs.date_end, NOW()) >= NOW()
    AND IFNULL(c.date_end, NOW()) >= NOW()
    AND c.school_id = 1509
    AND c.subject_id = 24
) AS cs ON cs.student_id = s.id

# RTI flag info
INNER JOIN 
(
    SELECT os.id,
    os.student_id
    FROM observations o
    INNER JOIN observations_students os ON os.observation_id = 2567
    WHERE
    o.school_id = 1509
) AS os ON os.student_id = s.id

LEFT JOIN schools_students ss ON ss.student_id = s.id
WHERE s.active = 1
AND ss.school_id = 1509
AND IFNULL(ss.date_end,NOW()) >= NOW()
AND cs.gradelevel_id BETWEEN 10 AND 16

What I would like to do after this is for each of these students whom have the 2567 observation, I would like to find the number of 2009 observations that student has. To do this, I am adding another LEFT JOIN and the completed code looks as follows:

SELECT DISTINCT
SUBSTRING(s.osis_id,INSTR(s.osis_id,'-')+1) AS osid,
s.id AS student_id,
CONCAT(s.last_name, ' ',s.first_name) AS sname,
COUNT(fdos.id) AS fd_count
FROM students s

# course info
INNER JOIN 
(
    SELECT c.id AS cid,
    c.description AS cname,
    cs.date_end,
    cs.student_id,
    gl.description AS grade,
    c.gradelevel_id
    FROM courses_students cs
    INNER JOIN courses c ON c.id = cs.course_id
    INNER JOIN gradelevels gl ON gl.id = c.gradelevel_id
    WHERE
    IFNULL(cs.date_end, NOW()) >= NOW()
    AND IFNULL(c.date_end, NOW()) >= NOW()
    AND c.school_id = 1509
    AND c.subject_id = 24
) AS cs ON cs.student_id = s.id

# RTI flag info
INNER JOIN 
(
    SELECT os.id,
    os.student_id
    FROM observations o
    INNER JOIN observations_students os ON os.observation_id = 2567
    WHERE
    o.school_id = 1509
) AS os ON os.student_id = s.id

LEFT JOIN
(
    SELECT fdos.id,
    fdos.student_id
    FROM observations o
    INNER JOIN observations_students fdos ON fdos.observation_id = 2009
    WHERE
    o.school_id = 1509
) AS fdos ON fdos.student_id = s.id

LEFT JOIN schools_students ss ON ss.student_id = s.id
WHERE s.active = 1
AND ss.school_id = 1509
AND IFNULL(ss.date_end,NOW()) >= NOW()
AND cs.gradelevel_id BETWEEN 10 AND 16

If I change the “COUNT(fdos.id) AS fd_count” to “fdos.id AS fdosid” I am returned the correct number of entries. However, the number returned from the COUNT is not the same number and is not correct. Can anyone understand what’s going on here well enough to explain what I’m doing wrong?

Thank you for your time.

Answer 1

I can bet you’re using MySQL.

If you use anything of:

• GROUP BY clause;
• HAVING clause;
• aggregate function, which count() is

then your query is being aggregated one.

This means, that data will be groupped by the fields specified in the GROUP BY clause, such fields should be kept as-is in the select list and elsewhere in the query. All other fields should be arguments of the aggregate functions, otherwise database has no clue which value from the set that matches your group it should return.

All major databases will give you an error for a query contructed the way you did, as there’s no GROUP BY clause for a bunch of fields: s.osis_id, s.id, s.last_name and s.first_name. MySQL will not. Instead, it will implicitly group data. I don’t know what is the grouping criteria, and I don’t want to, as this behaviour is error prone and unreliable.

Instead, your query should be rewritten.
The easiest way is to:

• use your existing query without the count() function, i.e. get a list of fdos.id;
• use the whole query as another subquery, omiting the DISTINCT clause;
• counting the students.

Something like this:

SELECT osid, student_id, sname, count(fdos_id) AS fd_count
  FROM (
    SELECT
        substring(s.osis_id,instr(s.osis_id,'-')+1) AS osid,
        s.id AS student_id,
        concat(s.last_name, ' ',s.first_name) AS sname,
        fdos.id AS fdos_id
      FROM students s
       ...
  ) AS src
 GROUP BY osid, student_id, sname
 ORDER BY osid, student_id, sname;