Question

I have this scenario in which memory conservation is paramount. I am trying to read in > 1 GB of Peptide sequences into memory and group peptide instances together that share the same sequence. I am storing the Peptide objects in a Hash so I can quickly check for duplication, but found out that you cannot access the objects in the Set, even after knowing that the Set contains that object.

Memory is really important and I don’t want to duplicate data if at all possible. (Otherwise I would of designed my data structure as: peptides = Dictionary<string, Peptide> but that would duplicate the string in both the dictionary and Peptide class). Below is the code to show you what I would like to accomplish:

public SomeClass {

       // Main Storage of all the Peptide instances, class provided below
       private HashSet<Peptide> peptides = new HashSet<Peptide>();

       public void SomeMethod(IEnumerable<string> files) {
            foreach(string file in files) {
                 using(PeptideReader reader = new PeptideReader(file)) {
                     foreach(DataLine line in reader.ReadNextLine()) {
                         Peptide testPep = new Peptide(line.Sequence);
                         if(peptides.Contains(testPep)) {

                            // ** Problem Is Here **
                            // I want to get the Peptide object that is in HashSet
                            // so I can add the DataLine to it, I don't want use the
                            // testPep object (even though they are considered "equal")
                            peptides[testPep].Add(line); // I know this doesn't work

                            testPep.Add(line) // THIS IS NO GOOD, since it won't be saved in the HashSet which i use in other methods.

                         } else {
                            // The HashSet doesn't contain this peptide, so we can just add it
                            testPep.Add(line);
                            peptides.Add(testPep);
                         }
                     }   
                 }
            }
       }
}

public Peptide : IEquatable<Peptide> {
     public string Sequence {get;private set;}
     private int hCode = 0;

     public PsmList PSMs {get;set;}

     public Peptide(string sequence) {
         Sequence = sequence.Replace('I', 'L');
         hCode = Sequence.GetHashCode();             
     }

     public void Add(DataLine data) {
         if(PSMs == null) {
             PSMs = new PsmList();
         } 
         PSMs.Add(data);
     }

     public override int GethashCode() {
         return hCode;
     }

     public bool Equals(Peptide other) {
         return Sequence.Equals(other.Sequence);
     }
}

public PSMlist : List<DataLine> { // and some other stuff that is not important }

Why does HashSet not let me get the object reference that is contained in the HashSet? I know people will try to say that if HashSet.Contains() returns true, your objects are equivalent. They may be equivalent in terms of values, but I need the references to be the same since I am storing additional information in the Peptide class.

The only solution I came up with is Dictionary<Peptide, Peptide> in which both the key and value point to the same reference. But this seems tacky. Is there another data structure to accomplish this?

Answer 1

Basically you could reimplement HashSet<T> yourself, but that’s about the only solution I’m aware of. The Dictionary<Peptide, Peptide> or Dictionary<string, Peptide> solution is probably not that inefficient though – if you’re only wasting a single reference per entry, I would imagine that would be relatively insignificant.

In fact, if you remove the hCode member from Peptide, that will safe you 4 bytes per object which is the same size as a reference in x86 anyway… there’s no point in caching the hash as far as I can tell, as you’ll only compute the hash of each object once, at least in the code you’ve shown.

If you’re really desperate for memory, I suspect you could store the sequence considerably more efficiently than as a string. If you give us more information about what the sequence contains, we may be able to make some suggestions there.

I don’t know that there’s any particularly strong reason why HashSet doesn’t permit this, other than that it’s a relatively rare requirement – but it’s something I’ve seen requested in Java as well…