Question

I’m using CLIq and scheduled a task that is supposed to run every x-hours.
My issue is that my SOQL query gets all the items from the database where I only need to extract the stuff that got updated in the last x-hours. How can I limit my query to that ?

This is my process-config.xml file.

<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>
    <bean id="AMP_AIMS" class="com.salesforce.dataloader.process.ProcessRunner" singleton="false">
        <description>Created by Dataloader Cliq.</description>
        <property name="name" value="AMP_AIMS"/>
        <property name="configOverrideMap">


            <map>
                <entry key="dataAccess.name" value="...\AMP_AIMS\write\AMP_AIMS.csv"/>
                <entry key="dataAccess.readUTF8" value="true"/>
                <entry key="dataAccess.type" value="csvWrite"/>
                <entry key="dataAccess.writeUTF8" value="true"/>
                <entry key="process.enableExtractStatusOutput" value="true"/>
                <entry key="process.enableLastRunOutput" value="true"/>
                <entry key="process.lastRunOutputDirectory" value="...\AMP_AIMS\log"/>
                <entry key="process.operation" value="extract"/>
                <entry key="process.statusOutputDirectory" value="...\AMP_AIMS\log"/>
                <entry key="sfdc.bulkApiCheckStatusInterval" value="5000"/>
                <entry key="sfdc.bulkApiSerialMode" value="5000"/>
                <entry key="sfdc.debugMessages" value="false"/>
                <entry key="sfdc.enableRetries" value="true"/>
                <entry key="sfdc.endpoint" value="https://test.salesforce.com/services/Soap/u/24.0"/>
                <entry key="sfdc.entity" value="Agency_Profile__c"/>
                <entry key="sfdc.extractionRequestSize" value="500"/>
                <entry key="sfdc.extractionSOQL" value="Select a.Total_Annual_Sales__c, a.Market_Specialties__c, a.Market_Focus__c,  a.Destination_Specialties__c, a.CreatedDate, a.Agency_Website__c, a.Agency_Business_Email__c From Agency_Profile__c a"/>
                <entry key="sfdc.insertNulls" value="false"/>
                <entry key="sfdc.loadBatchSize" value="100"/>
                <entry key="sfdc.maxRetries" value="3"/>
                <entry key="sfdc.minRetrySleepSecs" value="2"/>
                <entry key="sfdc.noCompression" value="false"/>
                <entry key="sfdc.password" value="blabla"/>
                <entry key="sfdc.proxyHost" value=""/>
                <entry key="sfdc.proxyNtlmDomain" value=""/>
                <entry key="sfdc.proxyPassword" value=""/>
                <entry key="sfdc.proxyPort" value=""/>
                <entry key="sfdc.proxyUsername" value=""/>
                <entry key="sfdc.timeoutSecs" value="60"/>
                <entry key="sfdc.useBulkApi" value="false"/>
                <entry key="sfdc.username" value="bla"/>
            </map>
        </property>
    </bean>
</beans>

Answer 1

I was fine with sth similar to this in the past:

SELECT Total_Annual_Sales__c, Market_Specialties__c, Market_Focus__c,  Destination_Specialties__c, CreatedDate, Agency_Website__c, Agency_Business_Email__c
FROM Agency_Profile__c
WHERE LastModifiedDate = TODAY AND HOUR_IN_DAY(LastModifiedDate) > 9

You’d shedule it to run at 12, then similar one at 3 PM…

Still not ideal because in case of failure of one of runs you’d want to catch up… so some scripting around the loader to save time of last succesful run is neccessary for a failsafe solution.