Question

i got a program that needs to check if a chunk of a file is zeroed or has data. This alg runs for the whole file for sizes upto a couple of gigs and takes a while to run. Is there a better way to check to see if its zeroed?

Platform: Linux and windows

bool WGTController::isBlockCompleted(wgBlock* block)
{
    if (!block)
        return false;

    uint32 bufSize = (uint32)block->size;
    uint64 fileSize = UTIL::FS::UTIL_getFileSize(m_szFile);

    if (fileSize < (block->size + block->fileOffset))
        return false;

    char* buffer = new char[bufSize];

    FHANDLE fh=NULL;

    try
    {
        fh = UTIL::FS::UTIL_openFile(m_szFile, UTIL::FS::FILE_READ);
        UTIL::FS::UTIL_seekFile(fh, block->fileOffset);
        UTIL::FS::UTIL_readFile(fh, buffer, bufSize);
        UTIL::FS::UTIL_closeFile(fh);
    }
    catch (gcException &)
    {
        SAFE_DELETEA(buffer);
        UTIL::FS::UTIL_closeFile(fh);
        return false;
    }

    bool res = false;

    for (uint32 x=0; x<bufSize; x++)
    {
        if (buffer[x] != 0)
        {
            res = true;
            break;
        }
    }

    SAFE_DELETEA(buffer);
    return res;
}

Answer 1

How long is ‘a while’? … I’d say attempting to compare as many values in parallel as possible will help, maybe use some SIMD instructions to compare more than 4 bytes at a time?

Do keep in mind though, that no matter how fast you make the comparison, ultimately the data still needs to be read from the file. If the file is not already in a cache somewhere in memory, then you may be limited to in the order of 100-150 MB/s at a maximum before the bandwidth of the disk is saturated. If you have already hit this point, then you may first need to look at an approach that avoids having to load the file, or just accept the fact that it’s not going to be faster than that.