I’m doing a cluster problem, and the proxy package in R provides both dist and simil functions.
For my purpose I need a distance matrix, so I initially used dist, and here’s the code:
distanceMatrix <- dist(dfm[,-1], method='Pearson')
clusters <- hclust(distanceMatrix)
clusters$labels <- dfm[,1]#colnames(dfm)[-1]
plot(clusters, labels=clusters$labels)
But after I ploted the image I found that the cluster result is not the way I expecte it to be, since I know what it should look like.
So I tried simil instead, and the code is like:
distanceMatrix <- simil(dfm[,-1], method='Pearson')
clusters <- hclust(pr_simil2dist(distanceMatrix))
clusters$labels <- dfm[,1]#colnames(dfm)[-1]
plot(clusters, labels=clusters$labels)
This code computes a similarity matrix using simil, then convert it to distance matrix using pr_simil2dist, then I plot it and get the result I expected !
I’m confused about the relationship between dist and simil. According to the relationship described in the documentation, shouldn’t the two code snippet has the same result?
Where am I wrong ?
Edit:
You can try my code with dfm of the following value, sorry for the bad indentation.
Blog china kids music yahoo want wrong
Gawker 0 1 0 0 7 0
Read/WriteWeb 2 0 1 3 1 1
WWdN: In Exile 0 2 4 0 0 0
ProBlogger Blog Tips 0 0 0 0 2 0
Seth's Blog 0 0 1 0 3 1
The Huffington Post | Raw Feed 0 6 0 0 14 5
Edit:
Actually the sample data is taken from a very big data frame using tail, and I get completely different matrix using dist and simil+pr_simil2dist. The full data can found here.
In case I made other silly mistakes, here’s the full code of my function:
The code I use to read in data:
dfm<- read.table(filename, header=T, sep='\t', quote='')
Code for clustering:
hcluster <- function(dfm, distance='Pearson'){
dfm <- tail(dfm)[,c(1:7)] # I use this to give the sample data.
distanceMatrix <- simil(dfm[,-1], method=pearson)
clusters <- hclust(pr_simil2dist(distanceMatrix))
clusters$labels <- dfm[,1]#colnames(dfm)[-1]
plot(clusters, labels=clusters$labels)
}
Matrix using dist:
94 95 96 97 98
95 -0.2531580
96 -0.2556859 -0.4629100
97 0.9897783 -0.1581139 -0.2927700
98 0.8742800 -0.2760788 -0.1022397 0.9079594
99 0.9114339 -0.5020405 -0.2810414 0.8713293 0.8096980
Matrix using simil+pr_simil2dist:
94 95 96 97 98
95 1.25315802
96 1.25568595 1.46291005
97 0.01022173 1.15811388 1.29277002
98 0.12572004 1.27607882 1.10223973 0.09204062
99 0.08856608 1.50204055 1.28104139 0.12867065 0.19030202
You can see that corresponding elements in the two matrices add up to 1, which I think is not right. So there must be something I’m doing wrong.
Edit:
After I specify names in the read.table function to read in the data frame, the dist way and simil+pr_simil2dist way give the same correct result. So technically problem solved, but I don’t know why my original way of handling data frame have anything to do with dist and simil.
Any one has a clue on that ?
I’m not sure what you mean by not as per expected. If I compute the distance/similarity matrix via
proxy::dist()or viasimil()and convert to a dissimilarity I get the same matrix:and
The last line yields:
which is telling us that
h1andh2are exactly the same except for the matched function call (obviously as we usedd1andd2in the respective calls).The figure produced is:
If you set your object up correctly, then you won’t need to fiddle with the labels. Look at the
row.namesargument toread.table()to see how to specify a column be used as the row labels when the data are read in.All of this was done using: