Question

I have a matrix (5×10000) with the fifth line contains values ​​between 1 and 50 corresponding to different events of an experiment. My goal is to find the columns of the matrix which are the same for different events. In other words, I want the columns results for all possible combinations of different events (subsets of {1,2, .., 50}) (For example: {1,3,7} and {7,1,3} are of course the same combination).
It sounds like a problem of intersection of sets that each contains all possible outcomes for a given event. I hope also that the computation time is reasonable.

example with a matrix (5×20):

A =

20     4     4    74    20    20     3     1     1     4     3     3     3     7     4     1    20     3     3    74
36     1     1    11    36    36     3     3     3     1     3     3     3     9     4     3    36     4     3    11
77     1     1    15    77    77     1     3     3     1     1     1     1    10     3     2    77     4     1    15
 9     4     4    40     9     9     2     4     4     4     2     2     2    40     1     4     9     3     2    40
 3     4     2     6     7     3     4     5     2     7     4     2     7     6     7     2     5     5     1     3

in this case we have seven different events from 1 to 7: line 5

for example:

the intersection of the results of events 3, 5 and 7 is the vector: [20 36 77 9]’

the intersection of the results of events 1, 2, 4 and 7 is the vector: [3 3 1 2]’

the intersection of the results of events 3 and 6 are the vectors: [20 36 77 9]’ and [74 11 15 40]’

So what I want is the common columns for a specified number of different events between 1 and 50. For example, how to get the columns common to 20 different events? The problem becomes more complicated for me when I think to find this result for all possible combinations of 20 events in the set {1,2,…, 50}.

I want the common columns for all possible combination for a given number of different events, but I gave the number 20 just as an example on which to base one solution.

I’ll rephrase my question that to make it clearer:

the following matrices are sub-matrix of A, each corresponding to a given event:

A1= [3;3;1;2;1]

A1 corresponds to results of the event 1

A2= [4 1 3 1;1 3 3 3;1 3 1 2;4 4 2 4;2 2 2 2]

A2 corresponds to results of the event 2

A3= [20 20 74;36 36 11;77 77 15;9 9 40;3 3 3]

A3 corresponds to results of the event 3

A4= [4 3 3;1 3 3;1 1 1;4 2 2;4 4 4]

A4 corresponds to results of the event 4

A5= [1 20 3;3 36 4;3 77 4;4 9 3;5 5 5]

A5 corresponds to results of the event 5

A6= [74 7;11 9;15 10;40 40;6 ]6

A6 corresponds to results of the event 6

A7= [20 4 3 4 7;36 1 3 4;77 1 1 3;9 4 2 1;7 7 7 7]

A7 corresponds to results of the event

my goal is to find intersection along the columns of the matrix Ai (1:4,:) i = 1,2, … 7

in other words:

intersection(Ai,Aj)(1:4,:) for i and j different

intersection(Ai,Aj,Ak)(1:4,:) for i,j and k different

intersection(Ai,Aj,Ak,Al)(1:4,:) for i,j,k and l different

intersection(Ai,Aj,Ak,Al,Am)(1:4,:) for i,j,k,l and m different

intersection(Ai,Aj,Ak,Al,Am,An)(1:4,:) for i,j,k,l,m and n different

intersection(Ai,Aj,Ak,Al,Am,An,Ao)(1:4,:) for i,j,k,l,m,n and o different

intersection(Ai,Aj,Ak,Al,Am,An,Ao,Ap)(1:4,:) for i,j,k,l,m,n,o and p different

when I say “intersection (Ai, Aj)(1:4,:) for i and j different,” I want the columns common to the matrix Ai(1:4,:) and Aj(1:4,:)

the result for each intersection can be many column vectors, not necessarily one, depending on the columns of the matrix A.

I hope that each result contains the vector column of the matrix Ai(1:4,:) followed by the corresponding values ​​of events, such as: if [3 3 1 2]’ is the intersection of A1, A2, A4 and A7, I want to get as a result the vector [3 3 1 2 1 2 4 7]’

for example: intersection(A1,A2,A3,A4)(1:4,:): my goal is to avoid the following loop:

[n1 m1] = size(A1);
[n2 m2] = size(A2);
[n3 m3] = size(A3);
[n4 m4] = size(A4);

k=1;

for i1=1:m1
    for i2=1:m2
        for i3=1:m3
            for i4=1:m4
                if A1(1:4,i1)==A2(1:4,i2) && A2(1:4,i2)==A3(1:4,i3) && A3(1:4,i3)==A4(1:4,i4)
                    intersection1234(:,k) = [A1(1:4,i1);A1(5,i1);A2(5,i2);A3(5,i3);A4(5,i4)];
                    k=k+1;
                end
            end
        end
    end
end

Answer 1

If I understand correctly, you want to find columns whose events are different. Building on John Colby’s answer:

n = 1e3

tic

% Simulate data
% (Here we've split off the 5th row into a separate variable)
data = randi(5, [4 n]);
exptEvents = randi(50, 1, n);

% Find repeats
[b,i,j] = unique(data', 'rows');

% Organize the indices of the repeated columns into a cell array
reps = arrayfun(@(x) find(j==x), 1:length(i), 'UniformOutput', false);

% Find events corresponding to these repeats
reps_Events = cellfun(@(x) exptEvents(x), reps, 'UniformOutput', false);

U = cellfun(@unique, reps_Events, 'UniformOutput', false);
repeat_counts = cellfun(@length, U);
k=20;
rep_data = b(repeat_counts>=k,:);

toc

U in the code above has in every cell a group (or “combination”) of unique events. Each cell also corresponds to a unique data column. If you need something else, please give an example. rep_data contains results that repeat in k or more events.