At the moment the software is build each time we deploy to Windows Azure.
We deploy the same software version to every customer.
The only difference is some config files with for example a different database connection string.
Is it possible to use a prebuild Windows Azure package, which is the same for every deployment, and somehow inject the config files?
Look at using CloudConfigurationManager class
Basically it looks in the Azure service configuration first and then in the web.config if not specified in Azure configuration file. You can deploy the same site to different Cloud services and just need to change the Azure configuration. You can also change it without re-deploying
See this question
Azure configuration settings and Microsoft.WindowsAzure.CloudConfigurationManager