RetailCRM project started in 2013. Then it was internal startup with the limited functionality, not requiring the testing.
One year later the project became larger and we have started to think about the project testing. In parallel there was a task to update versions of Symfony framework, on which the product is developed. That time the system were working on Symfony 2.1, when Symfony 2.5 was already launched.
It was needed to update Symfony for keeping up to date and not to allow the project code to become outdated. Updating was followed by serious risks, since versions have differences and there was a possibility to get a lot of errors, so it was the cause for writing the first set of tests.
First, the basic tests were written, which just passed by all system pages, sent forms, filters, switched the sorting and checked the absence of errors 500, it allowed to find out the most obvious breakdowns after updating.
Next series of tests were already checking the work correctness of crucial pages and forms. The logic, which independently passes by all sortings of columns and lists, all filters, was added, it fills in the form with different data and checks their work correctness.
Also there were added the tests for business logic, for statuses transition, checking of rights for access to sections. All this functionality was written for one month. In July 2015 it was decided that all written tests (around 300 at that moment) are enough for system updating.
After Symfony updating we started to write the tests mandatorily for all tasks, in passing creating tests for old tasks.
The more tests were created, the longer they were executed. When it was only first series of tests, they were executed just for several minutes. They were executed locally.
When the tests` execution began to take tens of minutes, waiting for execution were very inconvenient for developer. It was decided to use PHPCI, the system, which allows to execute the tests automatically. PHPCI integration with git and Redmine, which allows to work autonomously, was made.
AUTOTESTS — are the scenarios, which check the work correctness of small blockes of code or imitate the user interaction with service. Autotests allow to avoid manual testing and to exclude the code breakdown when adding new functionality to the system.
Meaning of three-sided integration of PHPCI with git and Redmine
1. There is a task, executing by developer in new branch at git.
2. The same number, as a task has, is assigned to this branch.
3. Developer realizes the task, adds tests on it, then executes commit and push to the central repository on server.
4. PHPCI monitors automatically the appearance of new branches and new code on the server.
5. PHPCI catches up the new branch, takes project from there, opens it and starts to execute tests.
6. If all tests are passed, then PHPCI notifies about successful tests passing: it changes task status to "Feedback" and 80% of readiness automatically.
7. If the result is negative (at least one test was not passed), status changes automatically that tests were not passed. There is a link in message from PHPCI.
8. PHPCI executes tests for all PHP versions, the project must work at, and notes the result of each execution in the task.
Task realization cycle from definition till deployment to working CRM-system with using of autotests