Question

How do you test pages with single sign-on (SSO) login during integration tests (for instance by using caybara or cucumber)? For a normal login, you would write a method which visits the login page, fills out the form, and submits it. This is a bit difficult if the login form comes from an external SSO server like Shibboleth or OpenAM/OpenSSO. How is it possible to write integration tests for pages protected by SSO?

A similar problem is integration testing with a separate search server (Solr or Sphinx). You would probably solve it by using some form of mocks or stubs. Can someone give a good example how to mock or stub a SSO for cucumber or capybara? If this is too difficult, then a comparable example for a search server would be helpful, too.

Answer 1

Integration testing of a SSO application is a
special case of a more general problem: testing
distributed applications. This is a difficult
problem and there does not seem to be a magic
bullet for it. There are various ways to combine
a set of different servers or services and test them
as a whole. The two extremes are

a) Test an instance of the whole system. You don’t
need any mocks or stubs then, but you need
a complete, full-fledged setup of the entire stack. This includes
a running instance of every server involved.
For each test, setup the entire application stack,
and test the whole stack, i.e. test the
entire distributed system as a whole with all
the components involved, which is difficult
in general. This whole thing works only if each
components and all connections are working well.

b) Write an integration test for each component,
treat it as a black box, and cover the
missing connections by mocks and stubs.
In practice, this approach is more common for
unit testing, one writes tests for each
MVC layer: model, view, and controller
(view and controller often together).

In both cases, you have not considered
broken connections. In principle one
has to check for each external server/service
the following possibilities

• is down
• is up and behaves well
• is up and and replies wrong
• is up, but you send it wrong data

Basically, testing of distributed apps is difficult.
This is one reason why distributed applications are hard to develop.
The more parts and servers a distributed application has, the more difficult it is to setup many full-fledged environments like production, staging, test and development.
The larger the system, the more difficult the
integration testing becomes. In practice,
one uses the first approach and creates a small
but complete version of the whole application.
A typical simple setup would be App Server + DB Server + Search Server.
On your development machine, you would have
two different versions of a complete system:

• One DB Server with multiple databases (development and test)
• One Search Server with multiple indexes (development and test)

The common Ruby plugins for search servers (Thinking Sphinx for Sphinx
or Sunspot for Solr) support cucumber and integration
tests. They “turn on” the search server for certain portions of
your tests. For the code that does not use the search server,
they “stub” the server or mock out the connection to avoid unneeded
indexing.

For RSpec tests, it is possible
to stub out the authentication methods,
for example for a controller test by

  before :each do
    @current_user = Factory(:user)
    controller.stub!(:current_user).and_return(@current_user)
    controller.stub!(:logged_in?).and_return(:true)
  end

It also works for helper and view tests, but
not for RSpec request or integration tests.

For cucumber tests, it is possible to stub
out the search server by replacing the connection
to the search server with a stub (for Sunspot and
Solr this can be done by replacing the Sunspot.session,
which encapsulates a connection to Solr).

This all sounds well, unfortunately it is a bit hard to
transfer this solution for a SSO Server. A typical minimal
setup would be App Server + DB Server + SSO Server.
A complete integration test would mean we have to setup one SSO Server with
multiple user data stores (development and test).
Setting up a SSO Server is already difficult enough,
setting up a SSO Server with multiple user data
stores is probably not a very good idea.

One possible solution to the problem is maybe somewhere in the
direction of Fakeweb. FakeWeb is a Ruby library written by
Blaine Cook for faking web requests. It lets you decouple
your test environment from live services. Faking the response
of a SSO server is unfortunately a bit hard.

Another possible solution I ended up using is to use a fake login, i.e.
add a fake login method that can be called within the integration
test. This fake login is a dynamic method only added during the
test (through a form of monkey patching). This is a bit messy, but
it seems to work.