I have a web config with the following:
<location path="toolbar/apps/loginservice/login">
<system.web>
<authorization>
<allow users="?"/>
</authorization>
</system.web>
</location>
if <allow users="?"/> doesn’t this means we can omit this from the web.config.
Not necessarily. This section may override other authorization configuration that exists higher in the application structure. The settings you cited would allow anonymous access at this level, thus overriding any inherited authorization.
I infer this is a login page by the URL, therefore the likelihood is high that other pages do NOT allow anonymous access but this one does.