Question

I am trying to create a bulk loader to upload data into the datastore. However, when I try to upload it, I am facing this error and stacktrace:

[INFO    ] Starting import; maximum 10 entities per post
[ERROR   ] [WorkerThread-0] WorkerThread:
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/adaptive_thread_pool.py", line 150, in WorkOnItems
    status, instruction = item.PerformWork(self.__thread_pool)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 691, in PerformWork
    transfer_time = self._TransferItem(thread_pool)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 846, in _TransferItem
    self.content = self.request_manager.EncodeContent(self.rows)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1267, in EncodeContent
    entity = loader.create_entity(values, key_name=key, parent=parent)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 382, in create_entity
    return self.dict_to_entity(input_dict, self.bulkload_state)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 133, in dict_to_entity
    self.__run_import_transforms(input_dict, instance, bulkload_state_copy)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 230, in __run_import_transforms
    value = self.__dict_to_prop(transform, input_dict, bulkload_state)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 188, in __dict_to_prop
    value = transform.import_transform(value)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/bulkloader_parser.py", line 93, in __call__
    return self.method(*args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/transform.py", line 420, in transform_function
    return bool(re.match(regexp, value, flags))
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
[INFO    ] [WorkerThread-1] Backing off due to errors: 1.0 seconds
[INFO    ] Unexpected thread death: WorkerThread-0
[INFO    ] An error occurred. Shutting down...
[ERROR   ] Error in WorkerThread-0: expected string or buffer

[INFO    ] 1020 entites total, 0 previously transferred
[INFO    ] 0 entities (1325 bytes) transferred in 10.1 seconds
[INFO    ] Some entities not successfully transferred

Does anyone have some suggestions?

I believe it is getting hung up on translating the CSV to boolean. The field has a “y” if true, and nothing if not. I can’t find documentation on transform.regexp_bool, however.

Answer 1

I actually figured it out – the names of the fields are case sensitive! This is very important. They must match the csv file explicitly!