I have a component which contains outgoing changes :

When I right click this component and compare against the ‘initial baseline’ I get a list of change sets which comprise this component. This is displayed in the ‘Change Explorer’ view :

Is this the list of change sets over time that have been delivered to this component or change sets that have been delivered to a baseline, or is it both ?
By right clicking on each change set I see the option to ‘accept’ the change set. Should this copy the selected changeset into my workspace ?
If you see accept on a changeset, that means it is part of the incoming changes, done and delivered over time on the stream.
If you accept those, and if your component is loaded in your sandbox, then yes, those files will be copied over on your hard drive.
To be clearer, the “Change Explorer” (compared to your classic “Pending Changes” view) is about source control exploration (ie finding, and if needed, accepting old changeset that you need to see back in your repo workspace, and, if the corresponding component is loaded, in your sandbox or hard-drive)
See “Practicing source control archaeology with Rational Team Concert“