Question

Currently, it is possible to mark tests and then run them (or not run them) using -m argument. However, all tests are still collected first and only then are deselected

In the below example all 8 are still collected, and then 4 are run and 4 are deselected.

============================= test session starts ==============================
platform win32 -- Python 2.7.3 -- pytest-2.3.2 -- C:\Python27\python.exe
collecting ... collected 8 items

test_0001_login_logout.py:24: TestLoginLogout.test_login_page_ui PASSED
test_0001_login_logout.py:36: TestLoginLogout.test_login PASSED
test_0001_login_logout.py:45: TestLoginLogout.test_default_admin_has_users_folder_page_loaded_by_default PASSED
test_0001_login_logout.py:49: TestLoginLogout.test_logout PASSED

==================== 4 tests deselected by "-m 'undertest'" ====================
================== 4 passed, 4 deselected in 1199.28 seconds ===================

QUESTION: Is it possible to not collect marked/unmarked tests at all?

The problems are:

1) I’m using some test when the database already has some items in it (like my device) and the code it have:

@pytest.mark.device
class Test1_Device_UI_UnSelected(SetupUser):

    #get device from the database
    device = Devices.get_device('t400-alex-win7')

    @classmethod
    @pytest.fixture(scope = "class", autouse = True)
    def setup(self):
    ...

I run the test explicitly excluding and device tests: py.test -m "not device" however, during collection I get the errors, because device = Devices.get_device('t400-alex-win7') is still being executed.

2) Some of the tests are marked time_demanding because there are around 400 generated tests. To generate those tests is also takes time. I exclude those tests from the general tests, however they are generated and collected and then deselected <- just a wait of time.

I know there is a solution for (1) problem – to use pytest.fixtures and pass them to the tests, however I really like autocompletion that PyDev provides.

timedemanding class is:

import pytest
#... other imports


def admin_rights_combinations(admin, containing = ["right"]):
    '''
    Generate all possible combinations of admin rights settings depending
    on "containing" restriction
    '''
    rights = [right for right in admin.__dict__.iterkeys() if any(psbl_match in right for psbl_match in containing)]
    total_list = []
    l = []
    for right in rights: #@UnusedVariable
        l.append([True, False])
    for st_of_values in itertools.product(*l):
        total_list.append(dict(zip(rights, st_of_values)))
    return total_list

@pytest.mark.timedemanding
class Test1_Admin_Rights_Access(SetupUser):

    user = UserFactory.get_user("Admin Rights Test")
    user.password = "RightsTest"
    folder = GroupFolderFactory.get_folder("Folders->Admin Rights Test Folder")
    group = GroupFolderFactory.get_group("Folders->Admin Rights Test Group")
    admin = UserFactory.get_admin("Admin Rights Test")

    @classmethod
    @pytest.fixture(scope = "class", autouse = True)
    def setup(self):
        ...

    @pytest.mark.parametrize("settings", admin_rights_combinations(admin, containing=['right_read', 
                                                                                      'right_manage_folders',
                                                                                      'right_manage_groups']))  
    def test_admin_rights_menus(self, base_url,settings):
        '''
        test combination of admin rights and pages that are displayed with 
        this rights. Also verify that menu's that are available can be opened 
        '''

As you can see, by the time pytest hits @pytest.mark.parametrize it should be already aware that it’s in Class with @pytest.mark.timedemanding. However, collection still occurs.

Answer 1

The problem is not py.test, but the fact that the class code is executed when the file is imported, so you get the error before the decorator is even called.

The only way(without modifying the logic of the code) to avoid this is to completely ignore the whole file.

Anyway, I do not understand why you set the device class attribute there. Use the class-level setup! If you put that code in the setup your problem should be solved, because, since the test is not run, the setup is not called either and you do not get the error.

The same goes for the time_demanding tests. Set them up in the class level setup, so that py.test does the class creation does not take so much time(even though, without a sample code, I can’t say much about this).

If you want to keep things like this, and have PyDev autocompletion, then, as I said, just ignore the whole file with some regex(and eventually you’ll have to split up the tests).