In our application, we have a variable number of dockwidgets because some of them are added by plugins that are loaded at runtime. Not all dockwidgets need to be visible at the same time necessarily. This depends strongly on what the user is working on and what plugins are active.
However, if too many dockwidgets are added programmatically with addDockWidget(...), they start to overlap each other (not in terms of tabs, but in terms of content of one being painted on the area of a different one, which obviously looks broken).
The user can move the dockwidgets to dockareas that still have space left, but the layout/main window successfully prevents (untabbed) re-addition to the “crowded” dockarea.
We do allow tabbed docks to allow the user to arrange the dockwidgets a required, but we don’t want to enable QMainWindow::ForceTabbedDocks since this would constrain the number of simultaneously visible dockwidgets too much (one per dock area).
How can I prevent this or better control how dockwidgets are added?
Not answering your question directly but it might be worthwhile to actually forget about Qt and actually think of how the whole interaction should work. What are the user expectations? What should actually happen if 10 different plugins become active? Should they be docked or should they be floating or should they become pin-able docking windows with initial state as a small button on the MainWindow edges? I think once you do that ground work and come up with user interface mock-ups, you can then start looking at Qt and figure out if Qt provides a direct way to develop that interface and if not what additional components you will need to develop to get that interface working.
From my own experience, I had developed a similar interface long back but in MFC. The way we did it was that some of the docked windows were deemed to be must have and they would come up as docked. Then there were a set of windows that didnt need to be visible always but should be quickly available and their initial state was as hidden pin-able dock window which meant they came up as buttons on the MainWindow edge. Finally there was a third set that was not required by the user always and could be called in from File->View Menu. Once the user made it visible, the user typically would assign it to one of the first two groups or keep it afloat. This whole configuration was saved in a config file and from there onwards whenever the plugin was loaded/became active the last used state of the associated docking window was used. It though involved quite a bit of extra work but the end result was to the satisfaction of all users.