Question

I am trying to cache friends from social media in the user’s doc and am having issues storing/getting them back.
First, I blindly strip the their friends cache and would just fill it with my fresh fetch of the data, but through that I found out that the embedded Documents must be unique among all the users (I have a couple of friends that are the same in my test accounts and I get:

{"name":"MongoError","err":"E11000 duplicate key error index: test-dev.users.$friends.outsideID_1  dup key: { : \"1205314313242\" }","code":11001,"n":0,"connectionId":274,"ok":1} 

so from this I know that because that embedded document exists in my other already registered and updated account it can’t create that document (outsideID is index because I plan on searching using it later))
So then I started trying to OutsideFriend.find({‘outsideID’:{$in:friendsIDs}}(where friendsIDs is an array of the ids I got from the SM query), etc.. and then in the callback I go through and try added the new friends and just add the existing docs to the user in question so that it’s not trying to duplicate it in the system. However, for whatever reason OutsideFrind.find() never returns any docs… which makes me think that the embedded docs aren’t centralized… but then why would the first attempt fail?

How am I supposed to do this? (schema below, please let me know if you need any other info!)

current schema:

//External friend model                                                                                                                                                                                              
var OutsideFriend = new Schema({
         outsideID:{type:String, index:true, unique:true}
        ,username:String
        ,location:String
        ,avatarURL:String
});

//User model                                                                                                                                                                                                         
var User = new Schema({
        avatarURL:String
       ,mySMID:String
    //Social Media info                                                                                                                                                                                                   
        ,smID:{type:String, index:true, unique:true}
        ,tok:String
        ,tokSec:String
        ,friends:[OutsideFriend]
};

Answer 1

There’s a lot going on in your question, but if you want to query OutsideFriend independent of User, then you shouldn’t be embedding them in the User model. Instead, just store ObjectId references to the friends and then populate that in User when you need the full details. Don’t prematurely optimize with embedded copies unless you find you need to as that introduces consistency challenges.

So your User model would change to:

var User = new Schema({
    avatarURL:String
    //Social Media info
    ,smID:{type:String, index:true, unique:true}
    ,tok:String
    ,tokSec:String
    ,friends:[{type: ObjectId, ref: 'OutsideFriend'}]
};