I am getting invalid product identifiers back from the store, however I believe I have set up everthing correctly.
I think this might be due to the fact that I dont have a paid contract in effect.

I was allowed to set up the app’s product, in iTunes connect, without the contract completed which suggests to me completing the paid contract might not be needed (I would rather not since this is a proof of concept app).
Does anyone know for sure if you can or can not test in the sandbox before this is set up?
It turns out you only need to request the contract and enter tax information. I.e. you dont need to enter banking info for the Sandbox.
From my experience it worked as followed:
Request the paid apps contract.
You can now set up the in-app-purchase on your apps. However, you will get ‘invalid product identifier’ back from StoreKit if you try to use it.
Enter tax information.
It now works.