How can do this with Mongo in one [atomic] statement:
insert mytable set MyUniqueKey = ?, X = ? on duplicate key update Y = ?
When this statement is executed for first time it will set X value but after that it will only update Y value.
Only MyUniqueKey is part of unique key and should be looked up for duplicates.
I asked this question on mongodb user group, the answer was that it is not possible and this is an open issue.