Question

I need to gracefully shutdown mongod.exe that is started with System.Diagnostics.Process in RoleEntryPoint.OnStop() method.

I was inspired by an article Running MongoDb on Microsoft Windows Azure with CloudDrive.
All seems to work fine, however after WorkerRole restart mongod says:

**************
old lock file: .\mongod.lock.  probably means unclean shutdown
recommend removing file and running --repair
see: http://dochub.mongodb.org/core/repair for more information
*************

So I created simple Console Application, code below and simulated same result, when mongod.exe is killed. Lock file is released only when console window (parent process) is closed. Because of CloudDrive is unmounted earlier than parent process is terminated (RoleEntryPoint), mongod.lock file is never released in Windows Azure WorkerRole environment.

static void Main(string[] args)
{
    StartMongo();

    Console.ReadLine();

    _mongoProcess.Close();
}

private static void StartMongo()
{
    _mongoProcess = new Process();
    var startInfo = _mongoProcess.StartInfo;
    startInfo.UseShellExecute = false;
    startInfo.CreateNoWindow = false;
    startInfo.FileName = @"mongod.exe";
    startInfo.WorkingDirectory = Environment.CurrentDirectory;
    startInfo.Arguments = "--dbpath .";

    startInfo.RedirectStandardError = true;
    startInfo.RedirectStandardOutput = true;
    _mongoProcess.ErrorDataReceived += (sender, evt) => WriteLine(evt.Data);
    _mongoProcess.OutputDataReceived += (sender, evt) => WriteLine(evt.Data);

    _mongoProcess.Start();
    _mongoProcess.BeginErrorReadLine();
    _mongoProcess.BeginOutputReadLine();
}

How I realized that parent process is keeping the lock? I simply changed process to run in new shell window, where no output was redirected (startInfo.UseShellExecute = true). Two console windows started and when mongod was closed, it released lock before main application was terminated. I need to achieve this behavior to use it in RoleEntryPoint in Windows Azure.

Does anyone know how?

EDIT:

I realized, that maybe it’s the parent process, that has the listeners to ErrorDataReceived and OutputDataReceived that holds proper closing/flushing of mongod output stream to mongod.lock … can it be?

Answer 1

In the OnStop method you can invoke the shutdown command. You could do something like

  var server = MongoServer.Create("mongodb://host:port");
  server.Shutdown();

If you are using the official 1.0 driver the shutdown command hangs even though it has shutdown the server. Azure will recycle this role instance in spite of the hang since you get only around 30 seconds in OnStop. This bug has been fixed in the latest version of the driver in GitHub https://github.com/mongodb/mongo-csharp-driver.

Additionally use mongodb 1.8.1 with journaling enabled. You would not need the repair then. This is needed if for some reason Azure recycles the role instance before the shutdown is completed and is not clean. More information on journaling can be found at http://www.mongodb.org/display/DOCS/Journaling