I’m struggling with a weird Azure problem after disabling remote desktop connections. I started upgrading from the Azure SDK 1.2 to 1.4 and had enabled remote desktop for easier debugging. However, once I had finished with the upgrade and everything was working smoothly when deployed, I did one final deployment with remote desktop connections disabled. To my surprise, my web role instances were stuck in the “Waiting for role to start” phase. Interestingly enough, worker role instances started normally. About 20 deployments later the problem is 100% reproducible:
- remote desktop enabled: everything works fine
- remote desktop disabled: web role instances unable to start
Enabling Intellitrace doesn’t seem to help because trying the get the logs results in “No IntelliTrace logs are available” error. As expected, my source control tool shows that the only difference between enabling and disabling remote desktop is in the service definition and config files (.cscfg, .csdef). Re-imaging and restarting the VMs also doesn’t seem to do any good.
Please, if anyone has had a similar problem, help a desperate guy!
UPDATE: Here are the service definition and config files:
ServiceDefinition.csdef
<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="####" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WebRole name="####.Web" enableNativeCodeExecution="true">
<Runtime executionContext="elevated" />
<Startup>
<Task commandLine="StartupTasks\FixACLs.cmd" executionContext="elevated" />
</Startup>
<Sites>
<Site name="Web">
<Bindings>
<Binding name="HttpIn" endpointName="HttpIn" />
</Bindings>
</Site>
</Sites>
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" />
</ConfigurationSettings>
<LocalResources>
<LocalStorage name="Index" cleanOnRoleRecycle="false" sizeInMB="10240" />
</LocalResources>
<Endpoints>
<InputEndpoint name="HttpIn" protocol="http" port="80" />
</Endpoints>
<!-- BEGIN Remote desktop addition -->
<Imports>
<Import moduleName="RemoteAccess" />
</Imports>
<!-- END Remote desktop addition -->
</WebRole>
<WorkerRole name="####.Worker" enableNativeCodeExecution="true">
<Runtime executionContext="elevated" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" />
</ConfigurationSettings>
<LocalResources>
<LocalStorage name="Index" cleanOnRoleRecycle="true" sizeInMB="10240" />
</LocalResources>
<!-- BEGIN Remote desktop addition -->
<Imports>
<Import moduleName="RemoteAccess" />
<Import moduleName="RemoteForwarder" />
</Imports>
<!-- END Remote desktop addition -->
</WorkerRole>
</ServiceDefinition>
ServiceConfiguration.cscfg
<?xml version="1.0"?>
<ServiceConfiguration serviceName="####" osFamily="2" osVersion="*" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
<Role name="####.Web">
<Instances count="2" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" value="DefaultEndpointsProtocol=https;AccountName=####;AccountKey=####" />
<!-- BEGIN Remote desktop addition -->
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="####" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="####" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="####" />
<!-- END Remote desktop addition -->
</ConfigurationSettings>
<!-- BEGIN Remote desktop addition -->
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="####" thumbprintAlgorithm="sha1" />
</Certificates>
<!-- END Remote desktop addition -->
</Role>
<Role name="####.Worker">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" value="DefaultEndpointsProtocol=https;AccountName=####;AccountKey=####" />
<!-- BEGIN Remote desktop addition -->
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="####" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="####" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="####" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />
<!-- END Remote desktop addition -->
</ConfigurationSettings>
<!-- BEGIN Remote desktop addition -->
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="####" thumbprintAlgorithm="sha1" />
</Certificates>
<!-- END Remote desktop addition -->
</Role>
</ServiceConfiguration>
Finally found the culprit thanks to the comments of Steve and smarx 🙂 Thank you guys! You got me digging in the right direction.
Turned out the PowerShell script that my startup task fired was using the Microsoft.WindowsAzure.ServiceRuntime snapin, which, as noted by smarx, was quietly added with the RemoteAccess module when remote desktop connections were enabled. And this subtle inclusion made me overlook the dependency on the snapin and hence my further headaches.
After finding the problem I actually saw that I didn’t even need the snapin, so the solution was to simply remove the Add-PSSnapin Microsoft.WindowsAzure.ServiceRuntime call from my PowerShell script. In case I really needed the snapin, this article shows a way to install the snapin before using it, since it isn’t installed by default on Azure VMs.
Another helpful resource that helped me debug the startup task was this article by smarx.
Hopefully this will spare someone all the time I lost with this problem. Thank you again Steve and smarx!