Question

Whenever I try and run zobbpack I generate the error:

psycopg2.IntegrityError: null value in column “zoid” violates not-null constraint

Any ideas what is causing this and how to fix it?

relstorage 1.5.1, postgres 8, plone 4.2.1.1

2012-12-03 13:18:03,485 [zodbpack] INFO Opening storage (RelStorageFactory)...
2012-12-03 13:18:03,525 [zodbpack] INFO Packing storage (RelStorageFactory).
2012-12-03 13:18:03,533 [relstorage] INFO pack: beginning pre-pack
2012-12-03 13:18:03,533 [relstorage] INFO pack: analyzing transactions committed Mon Nov 26 12:31:54 2012 or before
2012-12-03 13:18:03,536 [relstorage.adapters.packundo] INFO pre_pack: start with gc enabled
2012-12-03 13:18:03,759 [relstorage.adapters.packundo] INFO analyzing references from objects in 97907 new transaction(s)
2012-12-03 13:18:03,761 [relstorage.adapters.scriptrunner] WARNING script statement failed: '\n        INSERT INTO object_refs_added (tid)\n        VALUES (%(tid)s)\n        '; parameters: {'tid': 0L}
2012-12-03 13:18:03,761 [relstorage.adapters.packundo] ERROR pre_pack: failed
Traceback (most recent call last):
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 486, in pre_pack
    conn, cursor, pack_tid, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 580, in _pre_pack_with_gc
    self.fill_object_refs(conn, cursor, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 387, in fill_object_refs
    self._add_refs_for_tid(cursor, tid, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 459, in _add_refs_for_tid
    self.runner.run_script_stmt(cursor, stmt, {'tid': tid})
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/scriptrunner.py", line 52, in run_script_stmt
    cursor.execute(stmt, generic_params)
IntegrityError: null value in column "zoid" violates not-null constraint

Traceback (most recent call last):
  File "zodbpack.py", line 86, in <module>
    main()
  File "zodbpack.py", line 78, in main
    skip_prepack=options.reuse_prepack)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/storage.py", line 1114, in pack
    adapter.packundo.pre_pack(tid_int, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 486, in pre_pack
    conn, cursor, pack_tid, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 580, in _pre_pack_with_gc
    self.fill_object_refs(conn, cursor, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 387, in fill_object_refs
    self._add_refs_for_tid(cursor, tid, get_references)
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 459, in _add_refs_for_tid
    self.runner.run_script_stmt(cursor, stmt, {'tid': tid})
  File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/scriptrunner.py", line 52, in run_script_stmt
    cursor.execute(stmt, generic_params)
psycopg2.IntegrityError: null value in column "zoid" violates not-null constraint

Answer 1

Running zodbpack.py without GC enabled (thanks Martijn) worked and enabled the database to pack down from 3.8Gb to 887Mb.

So my zodbpack-conf.xml looks like:

<relstorage>
  pack-gc false
  <postgresql>
    dsn dbname='zodb' user='user' host='host' password='password'
  </postgresql>
</relstorage>

and i run it with:

python zodbpack.py -d 7 zodbpack-conf.xml

Note: after the pack completes you still need to vacuum the database to get the space back. I run this from the command line as:

psql zodb postgresuser
zodb=# SELECT pg_database_size('zodb');
zodb=# vacuum full;
zodb=# SELECT pg_database_size('zodb');

Interestingly, when i run the pack command from the ZMI Control Panel, I still get the same error as previously:

Site Error

An error was encountered while publishing this resource.

Error Type: IntegrityError
Error Value: null value in column "zoid" violates not-null constraint 

So i am assuming the ZMI pack uses GC enabled, and that there is still an issue with null values in my site database…

Should I try and run some SQL to clean it out? If an object has no ‘zoid’ value then is it effectively inaccessible junk? A SQL example for this would be great too 🙂