Question

Im reciveing 403 User Rate Limit Exceeded error making querys but I’m sure I’m not exceding.
In the past I’ve reach the rate limimt doing inserts and It was reflected in the job list as

[errorResult] => Array
(
[reason] => rateLimitExceeded
[message] => Exceeded rate limits: too many imports for this project
)

But in this case the jobs-list doesn’t reflect the query (nor error or done), and studing the job-list i haven’t reach the limits or have been close to reach it (no more than 4 concurrent querys and each processing 692297 Bytes)

I’ve the billing active, and I’ve make only 2.5K querys in the las 28 days.

Edit: The user limit is set up to 500.0 requests/second/user

Edit: Error code recived

User Rate Limit Exceeded

User Rate Limit Exceeded

Error 403

Edit: code that I use to make the query jobs and get results

function query_data($project,$dataset,$query,$jobid=null){


    $jobc = new JobConfigurationQuery();
    $query_object = new QueryRequest();
    $dataset_object = new DatasetReference();
    $dataset_object->setProjectId($project);
    $dataset_object->setDatasetId($dataset);
    $query_object->setQuery($query);
    $query_object->setDefaultDataset($dataset_object);
    $query_object->setMaxResults(16000);
    $query_object->setKind('bigquery#queryRequest');
    $query_object->setTimeoutMs(0);
    $ok = false;
    $sleep = 1;
    while(!$ok){
        try{
            $response_data = $this->bq->jobs->query($project, $query_object);
            $ok = true;
        }catch(Exception $e){ //sleep when BQ API not avaible

            sleep($sleep);
            $sleep += rand(0,60);
        }
    }
    try{
        $response = $this->bq->jobs->getQueryResults($project, $response_data['jobReference']['jobId']);
    }catch(Exception $e){
        //do nothing, se repite solo
    }

    $tries = 0;
    while(!$response['jobComplete']&&$tries<10){
        sleep(rand(5,10));
        try{
            $response = $this->bq->jobs->getQueryResults($project, $response_data['jobReference']['jobId']);
        }catch(Exception $e){
            //do nothing, se repite solo
        }
        $tries++;
    }

    $result=array();
    foreach($response['rows'] as $k => $row){
        $tmp_row=array();
        foreach($row['f'] as $field => $value){
            $tmp_row[$response['schema']['fields'][$field]['name']] = $value['v'];
        }
        $result[]=$tmp_row;
        unset($response['rows'][$k]);
    }


    return $result;

}

Is there any other rate limits? or it is a bug?

Thanks!

Answer 1

I’ve solved the problem by using only one server to make the requests.

Looking what I was doing different in the night cronjobs (that never fail) the only diference was I was using only one client in one server instead of using diferent clients in 4 diferent servers.

Now I have only one script in one server that manages the same number of querys and now it never gets the User Rate Limit Exceded error.

I think there is a bug managing many clients or many active IPs at a time, althought the total number of threads never exceds 20.