Question

I have a big query in MySQL for magento which takes way to much time. I tried to optimize it but my MySQL knowledge isn’t enough to get it solved, so maybe someone could take a look and give me some hints in the right direction.

Select Distinct
  `e`.*,
  `cat_index`.`position` AS `cat_index_position`,
  `price_index`.`price`,
  `price_index`.`tax_class_id`,
  `price_index`.`final_price`,
  IF(price_index.tier_price IS NOT NULL, 
      LEAST(price_index.min_price, price_index.tier_price),
      price_index.min_price) AS `minimal_price`,
  `price_index`.`min_price`,
  `price_index`.`max_price`,
  `price_index`.`tier_price`
From
  `catalog_product_entity` AS `e`
    Inner Join
  `catalog_category_product_index` As `cat_index`
    On cat_index.product_id=e.entity_id And
       cat_index.store_id=1 And
       cat_index.visibility In(2, 4) And
       cat_index.category_id='2'
    Inner Join
  `catalog_product_index_price` AS `price_index` 
    On price_index.entity_id = e.entity_id And
       price_index.website_id = '1' And
       price_index.customer_group_id = 0
    Left Join
  `beta_entity_product` AS `beta` 
    On e.entity_id = beta.product_id And
       beta.entity_id In (81558, 81559, ... stupidly long list of ids)
    Left Join 
  `catalog_product_entity_int` AS `is_uni` 
    On e.entity_id = is_uni.entity_id And
       attribute_id = 179 
Where 
 is_uni.value = 1 OR 
 beta.product_id IS NOT NULL

If I just have 1 condition in the WHERE clause everything is fine, but with the OR it takes sometimes some minutes to finish and thats way too long. Which options do I have to get better results? Another problem is that I can’t make more queries from this and just join the results together. Everything has to be in 1 query.

When I do an EXPLAIN on the query I get the following result (copied in JSON format for better overview):

{
        "data":
        [
            {
                "id": 1,
                "select_type": "SIMPLE",
                "table": "e",
                "type": "ALL",
                "possible_keys": "PRIMARY",
                "key": null,
                "key_len": null,
                "ref": null,
                "rows": 213396,
                "Extra": "Using temporary"
            },
            {
                "id": 1,
                "select_type": "SIMPLE",
                "table": "beta",
                "type": "range",
                "possible_keys": "PRIMARY",
                "key": "PRIMARY",
                "key_len": "4",
                "ref": null,
                "rows": 2833,
                "Extra": "Using where; Using index"
            },
            {
                "id": 1,
                "select_type": "SIMPLE",
                "table": "is_uni",
                "type": "ref",
                "possible_keys": "UNQ_CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,IDX_CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,IDX_CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID",
                "key": "UNQ_CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID",
                "key_len": "6",
                "ref": "unc_cpk.e.entity_id,const",
                "rows": 1,
                "Extra": "Using where"
            },
            {
                "id": 1,
                "select_type": "SIMPLE",
                "table": "cat_index",
                "type": "eq_ref",
                "possible_keys": "PRIMARY,IDX_CAT_CTGR_PRD_IDX_PRD_ID_STORE_ID_CTGR_ID_VISIBILITY,15D3C269665C74C2219037D534F4B0DC",
                "key": "PRIMARY",
                "key_len": "10",
                "ref": "const,unc_cpk.e.entity_id,const",
                "rows": 1,
                "Extra": "Using where"
            },
            {
                "id": 1,
                "select_type": "SIMPLE",
                "table": "price_index",
                "type": "eq_ref",
                "possible_keys": "PRIMARY,IDX_CATALOG_PRODUCT_INDEX_PRICE_CUSTOMER_GROUP_ID,IDX_CATALOG_PRODUCT_INDEX_PRICE_WEBSITE_ID",
                "key": "PRIMARY",
                "key_len": "8",
                "ref": "unc_cpk.cat_index.product_id,const,const",
                "rows": 1,
                "Extra": "Using where"
            }
        ]
    }

Answer 1

There are multiple solutions:

1) Split query and use UNION
– possible sorting and pagination problems
2) Add new property that will be used only for this filter so instead of

is_uni.value = 1 OR beta.product_id IS NOT NULL

you would be querying only

feature_filter = 1

3) Decompose query into multiple queries
– I would suggest this as first step since queries as big as this one are really great way to shoot yourself in the foot

BTW. Using distinct on query with so many columns eats a lot of resources (mysql needs to compare each column value!)