Question

I’m have an instance of graphite configured on one server, and i’m using it to monitor my environment which sadly consists of both linux and windows machines. I want monitor the well-being of my servers so i have opted for collectl on my linux machines which does a nice job collecting system stats.

Sadly for windows there don’t seem to be so many solution for getting system stats and sending them to graphite, but i’ve manage to handle this situation with powershell. I’m using the a script which was suggested here http://josh.behrends.us/2012/07/windows-powershell-graphite-awesome/ for the graphite connection and also for getting metrics out of the computer
i’m using get-counter comandlet which, surprisingly, can gather a lot of info.

My script looks like this:

while (1 -eq 1)
{
$carbonServer = "hostname"
$carbonServerPort = 2003 
$epochtime = [int][double]::Parse((Get-Date -UFormat %s))

$socket = New-Object System.Net.Sockets.TCPClient
$socket.connect($carbonServer, $carbonServerPort)
$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter($stream)
#Write out metric to the stream.

$DiskQue =  [int]((get-counter -Counter "\PhysicalDisk(1 e: f:)\Current Disk Queue Length" ).countersamples | select -property cookedvalue).cookedvalue
$BytesReceived = [int]((get-counter -Counter "\Server\Bytes Received/sec" ).countersamples | select -property cookedvalue).cookedvalue
$BytesSent =  [int]((get-counter -Counter "\Server\Bytes Transmitted/sec").countersamples | select -property cookedvalue).cookedvalue
$MemAvail = ((get-counter -Counter "\Memory\Available Bytes").countersamples | select -property cookedvalue).cookedvalue
$MemCommit = ((get-counter -Counter "\Memory\Committed Bytes").countersamples | select -property cookedvalue).cookedvalue
$ReadOp =  [int]((get-counter -Counter "\System\File Read Operations/sec").countersamples | select -property cookedvalue).cookedvalue
$WriteOp = [int]((get-counter -Counter "\System\File Write Operations/sec").countersamples | select -property cookedvalue).cookedvalue

$metric7 = ("hostname.metrics.WriteOp " + $WriteOp + " " + $epochTime+"`n")
$metric7

$writer.WriteLine($metric7)
$writer.Flush() #Flush and write our metrics.
$writer.Close()
$stream.Close()
sleep 5
}

Now this script outputs what it’s supposed to at first glance. The output has the format hostname.metrics.name $metric_value $epochTime but there are no graphs being graphed. They apear in the graphite dashboard but they are empty. I’ve inspected the output sent to the graphite server with wireshark. It seams that in windows the message gets appended with CRLF as opposed to linux where you only have LF. I have added \n by hand and for a small period of time it did the trick but now it stopped working.

My question is am i doing something wrong in the transmission because i’ve been analysing the trafic and the only difference between traffic coming form linux machines that gets graphed and from windows that doesn’t get graphed is the line terminators. In linux its LF(0a) and in windows is CRLF(0d0a), but again i’ve tried to send from linux LFCRLF(0a0d0a)
hoping the graphite server will read only until the first LF and than interepret the message but still i’m not geting graphs.

Also when I transmit from linux i have only one message, and when i transmit form powershell i have three messages. From what i’ve saw with strace on the carbon-cache process i have one recvfrom system call with the message i want, and the I have an other one which is empty and the the write syscall(when transmitting from powershell) as opposed yo anly one recvfrom with the message and a write(when transmitting with netcat on linux) ,

Answer 1

Seems that nobody provided any suggestions and as such i have followed another approach. I am still very curious as to why Graphite doesn’t interpret my sent message. The solution to this problem was using a UDP socket instead of a TCP one and sending the metrics to statsd which in term sends them to Graphite. This seems to work without any complications, and in if you think about it it’s better for your network because it minimizes traffic(if you keep statsd and Graphite on the same node).

I’m posting the script in case anyone encounters my problem and his environment resembles mine. Here is the script

[int] $Port = 8125
$IP = "here is your IP" 
$Address = [system.net.IPAddress]::Parse($IP) 
$End = New-Object System.Net.IPEndPoint $address, $port 
$Saddrf   = [System.Net.Sockets.AddressFamily]::InterNetwork 
$Stype    = [System.Net.Sockets.SocketType]::Dgram 
$Ptype    = [System.Net.Sockets.ProtocolType]::UDP 
$Sock     = New-Object System.Net.Sockets.Socket $saddrf, $stype, $ptype 
$sock.Connect($end) 

function Send_Graphite
{param ($Metric)

$Enc     = [System.Text.Encoding]::ASCII 
$Buffer  = $Enc.GetBytes($Metric) 
$Sent   = $Sock.Send($Buffer) 
"{0} characters sent to: {1} " -f $Sent,$IP 
"Message is:" 
$Metric
sleep 1

}  

while (1 -eq 1)
{
    $DiskQue =  [int]((get-counter -Counter "\PhysicalDisk(1 e: f:)\Current Disk Queue Length" ).countersamples | select -property cookedvalue).cookedvalue
    $BytesReceived = [int]((get-counter -Counter "\Server\Bytes Received/sec" ).countersamples | select -property cookedvalue).cookedvalue
    $BytesSent =  [int]((get-counter -Counter "\Server\Bytes Transmitted/sec").countersamples | select -property cookedvalue).cookedvalue
    $MemAvail = ((get-counter -Counter "\Memory\Available Bytes").countersamples | select -property cookedvalue).cookedvalue
    $MemCommit = ((get-counter -Counter "\Memory\Committed Bytes").countersamples | select -property cookedvalue).cookedvalue
    $ReadOp =  [int]((get-counter -Counter "\System\File Read Operations/sec").countersamples | select -property cookedvalue).cookedvalue
    $WriteOp = [int]((get-counter -Counter "\System\File Write Operations/sec").countersamples | select -property cookedvalue).cookedvalue

    $Message1 = "MAIN.OSmetrics.DiscQueue:"+$DiskQue+"|c"
    $Message2 = "MAIN.OSmetrics.BytesReceived:"+$BytesReceived+"|c"
    $Message3 = "MAIN.OSmetrics.BytesSent:"+$BytesSent+"|c"
    $Message4 = "MAIN.OSmetrics.MemAvail:"+$MemAvail+"|c"
    $Message5 = "MAIN.OSmetrics.MemCommit:"+$MemCommit+"|c"
    $Message6 = "MAIN.OSmetrics.ReadOp:"+$ReadOp+"|c"
    $Message7 = "MAIN.OSmetrics.WriteOp:"+$WriteOp+"|c"

    $Mesages = $Message1, $Message2, $Message3, $Message4, $Message5, $Message6, $Message7
    foreach($Message in $Mesages)
    {
        Send_Graphite $Message 
    }
}

The script is extensible and you can monitor a lot of things, just run get-counter -ListSet * and you will see. I’ve installed the script with TaskScheduler, and you can control the frequency with the while loop by inserting a sleep.