Let’s say my library class (in assembly A) have a method GetFoo() returns a type Foo. Foo implements several interfaces, including IBar in assembly B. When a client code calls GetFoo() (the client code does not reference assembly B), the compiler will gives an error, because IBar “is defined in an assembly that is not referenced. You must add a reference to assembly B”. Note that my code only needs to know about Foo, and it contains no reference to the interface.
Is it possible to not have to reference B? As far as I know there is no need for the runtime to know about it.
That scenario works in native languages like C++ with COM because the interface simply specifies a convention that is checked at compile time but not enforced at runtime. However, in .NET the CLR enforces type safety at runtime and so it needs to reference the assembly in which the interface is defined.
If your situation allows it, you may want to break out the interface into a separate “contracts” assembly and have both assemblies reference it. It will be a very small assembly and should load very quickly.