Question

I have a function written in PostgreSQL, to go over a large table and insert a load of values into a different table. The output is fine, with loads of lines apparently being inserted, but no values are actually inserted into the target table (“resources” table in my code).

I have tried putting the insert statement inside a transaction, to no avail. Is there some sort of fudgy access or permission settings that I am missing? I have found several examples on the web that do this like I am doing, so I am pulling a little hair on this one…

Here is my function:

DECLARE
datatype_property record; 
property record;
new_resource_id bigint;
BEGIN  
    RAISE NOTICE 'Starting...';
    FOR datatype_property IN  
      SELECT * FROM datatype_properties
    LOOP  
        RAISE NOTICE 'Trying to insert';


        if not exists(select * from resources where uri = datatype_property.subject_resource) then
              SELECT INTO new_resource_id NEXTVAL('resources_id_seq');  
              INSERT INTO resources (id, uri) VALUES(  
                    new_resource_id,    
                    datatype_property.subject_resource
              );   
            RAISE NOTICE 'Inserted % with id %',datatype_property.subject_resource, new_resource_id;
        end if;
    END LOOP; 

 FOR property IN  
      SELECT * FROM properties 
 LOOP  

        if not exists(select * from resources where uri = property.source_uri) then
                SELECT INTO new_resource_id NEXTVAL('resources_id_seq');
              INSERT INTO resources (id, uri) VALUES(  
                        new_resource_id,
                        resource.source_uri
              ) ;   
                RAISE NOTICE 'Inserted % with id %',resource.source_uri, new_resource_id;
        end if;
        if not exists(select * from resources where uri = property.destination_uri) then
                SELECT INTO new_resource_id NEXTVAL('resources_id_seq');
              INSERT INTO resources (id, uri) VALUES(  
                        new_resource_id,
                        resource.source_uri
              ) ;   
        RAISE NOTICE 'Inserted % with id %',resource.source_uri, new_resource_id;
        end if;
 END LOOP;  
 RETURN;  

END;

EDIT: I’ve activated the plpgsql language with the directions from the following link:

http://wiki.postgresql.org/wiki/CREATE_OR_REPLACE_LANGUAGE

EDIT 2:

this code:

DECLARE
datatype_property record; 
property record;
new_resource_id bigint;
BEGIN  

    insert into resources (id, uri) values ('3', 'www.google.com');
END

does not work either :O

Answer 1

Your problem does sound like you are not comitting your transaction (as Pavel pointed out) or the tool which you use to check the rows is e.g. using REPEATABLE READ as its isolation level or some kind of caching.

But your function isn’t a good solution to begin with. Inserting rows one by one in a loop is alway a bad idea. It will be much slower than doing a single insert (and will be less scalable).

If I’m not mistaken, the two loops can be rewritten into the following statements:

insert into resource (id, uri)
select NEXTVAL('resources_id_seq'),
       dt.subject_resource
from datatype_properties dt
where not exists (select 1
                  from resources r
                  where r.uri = dt.subject_resource);


insert into resources (id, uri)
select nextval('resources_id_seq'),
       p.source_uri
from properties p
where not exists (select 1 
                  from resources r 
                  where r.uri = p.source_uri
                     or r.uri = p.destinatioin_uri);