Home/ Questions/Q 204927
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-11T17:30:40+00:00

I am currently developing a library and a set of programs using this library,

  • 0

I am currently developing a library and a set of programs using this library, in python.
Unit testing dictates that I import each module from the library, and test the classes and routines within. No problem with that. I have a separate test directory containing all my tests and importing the library modules, which I run while developing.

However, when it comes to testing the programs, things change. To be tested, the programs must run as a whole. The programs assume to find the library installed (which could actually be the case, albeit wrong, if I installed a previous version of it on my machine, adding further trouble). At the moment, my programs are run by a testsuite with a PYTHONPATH definition that I perform by hand, before the deployment (IOW, I don’t perform the install), but I don’t think I am doing it right. I feel like in general, a program should be tested for functionality when fully deployed, but this would mean that I have to install it every time I want to perform functional testing.

What is your experience and suggestions concerning functional testing of whole programs ? do you do it before or after deployment, and how?

Thanks

Note that I don’t include the python tag on purpose. Although my problem is python specific, and I would prefer python-related answers, I think that contribute can be brought also from experts in other languages.

Edit: as reported in a comment, the fact is that my program, when installed, has to import modules whose path can be found only when deployed (I download and install the dependencies on the fly, they are not installed on my machine). I cannot manipulate sys.path from the test, because that would imply that I modify the sys.path of a program (my executable) from another program (the testsuite, which runs and spawn a system() call).

In other words, the only way I have to test the program without deploying is to execute the program with PYTHONPATH set to the dir containing the deps and the library that program uses installed by the make script (which, as I said, downloads, compiles and “installs” everything in a temporary directory).

At deployment, the deps and the executables are packaged together in a “OSX bundle”-like structure, which is fully executable and relocatable.

Edit:

added a 150 bounty to see if I can get a little more feedback.

Edit:

I appreciated all the answers and voted up all of them. The choice has been a hard call for me, but I’ve been recalled by LudoMC of the V-model approach to testing I studied long ago. Thanks to all for the very good answers.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    In our company, we are using a pretty commonly used V-Model as development process, where the unit tests are done within the implementation phase, the integration tests are done against the architecture/design phase, and the system tests against the requirements phase.

    So in your case, from what I understand, you want to test your application as a whole, on a functional level. So it has to be done against the requirements.

    Thus, you need a requirement document, whether a full-text scenario or (better) a UML use case diagram covering (ideally extensively) the use cases of your application (usually one of the first phase to be achieved).
    You then have to write test cases covering every use case and pass these test cases. It can be done through manual or automatic testing with well-known (and pretty expensive) tools.

    For the when, we usually do these System tests after deployment (test team is using the installer provided by the development team), because we also test the installer itself, where Integration tests are before or after deployment depending on the case.

    In your case, if the installer is error-free, and you’re 100% sure than testing before the deployment using your PYTHONPATH variable will never bring a bug once deployed, then you can choose to test before deployment. It’s pure risk-management and it’s your call, because you’re the one who knows the best the pro and cons of this for your applications. (Personnaly, I don’t see why bugs cannot exist there, they are everywhere 🙂 )

    Hope that helps and I’m not off-topic

    • 0