Question

I have recently changed to ubuntu from windows and am having difficulty with the permissions on my postgresql database which I am trying to recreate.

My problem is as follows:
I have a script which imports csv files into the postgresql database.
The data is located in multiple csv files and I am using psycopg2 to handle the importing with a wrapper.

However, there is an issue with user permissions.

To run the file I need to manually change the permissions of each csv file (can do in bulk) so that the “other” permissions are set to read and write.

Is this some issue with how I have set up the database initially? And if so, how can I rectify it.

To clarify:

I wish to know how to change my database settings to not require a manual permissions change to the “other” settings in order to allow import.

Scripts added by request

import psycopg2 as psy
import sys

conn = psy.connect("dbname = 'name' host='localhost' user = 'postgres' password ='password')
curs = conn.cursor()
tablename = 'tabname'
filename  = 'filename'

Input = "COPY %s FROM %s WITH CSV HEADER" % (tablename, filename)
curs.copy_expert(Input, sys.stdin)

conn.commit()
curs.close()
conn.close()

This then has a wrapper around it to simply move between files and tables

I know it works, however, it requires that I manually change the permissions of the csv files so that “others” may read and write them.

I wish to know why this is the case

Answer 1

Unix systems enforce filesystem permissions. No user process can access data owned by another user, unless the permissions on the data allow the access. The only user not subjected to these constraints is the root account; but because root is incredibly powerful, you should not run any processes as root unless it is absolutely necessary.

There are several options, depending upon how restrictive you want to keep the data:

• You can make the files owned by the postgres user. This is a bit blunt-force and might become annoying if you must regularly work with the CSV files as a user, as well. It’s about the same as your current approach but requires that root do the chown(1) calls. (You cannot give away files on modern systems.)

  This is akin to find /path/to/top -type f -print0 | xargs -0 chown postgres

• You can make a new group; add the group to the list of supplementary groups that PostgreSQL runs as, make that group the group owner of the files, and make sure that group has read permissions on these files. This might be pretty simple, especially if all the files live in one directory or one hierarchy. (The BSD groups behavior can set the group ownership of a file in a clever way.)

  This approach takes a few steps:

  • use groupadd(8) or a similar command to add the group.
  • modify the PostgreSQL configuration to add the group to the supplementary groups list.
  • find /path/to/top -print0 | xargs -0 chgrp group_name
  • If you want to use BSD groups:
    • read the mount(8) manpage’s section on the bsdgroups mount(8) option.
    • add the bsdgroups mount option to your /etc/fstab file as needed.
    • remount the filesystem to include the bsdgroups mount option: mount -oremount,bsdgroups /path/to/mount/point
    • set the setgid bit on your directories: find /path/to/top -type d -print0 | xargs -0 chmod g+s

• You can grant read access to the files to every user on the system. This requires trusting every process on the computer with the data — which might not be a good decision if the machine hosts unrelated services, such as web, ftp, mail, database services for unrelated clients or systems, etc.

  You may be able to modify the umask(2) setting of the program that creates the CSV files. This mask is applied when files are created and most programs do not try to over-ride the umask(2) setting. Most shells make it easy to change the umask(2) for itself and all child processes — try running umask 0022 in the shell that you use to create the CSV files, and see what that does. (It turns off the write permission for group and other. You won’t need the write permission unless PostgreSQL also updates those files, but you only said you needed to import them.)

Try to pick the solution that is both easiest and respects the principle of least privilege — grant as little permission as necessary to each service on a system to restrict the potential consequences of insecure programs or configurations.