Automated test

3 minute read

Automated testing can be run after the build stage has been completed and the code is deployed. You can run all types of tests such as Unit tests, UI tests, and integration tests at this stage. The tests help to maintain quality in your software delivery. If you run tests at different stages, it brings transparency on the code quality. You can find the bugs at an early stage and prevent delays in fixing the issues. It becomes important for you to execute tests in the CI/CD pipelines to accelerate the testing and deployment process.

Automated testing configuration

Enter the following details:

  • Test Configuration Name*: Specify any name for your test configuration.

  • Tool Name*: Specify the tool name.

  • Jobs: Click Add New to add a test job and define the parameters

  • Test Result Formats: Specify the test report format.

    • Test Result Format*: Select the test Mapping Configuration defined in Settings page.

    • Test Result Location: Enter the location to access the test result.

  • Quality Gate: A quality gate is defined by DevOps teams to stop or fail a build by setting up a certain threshold value against whose detection, the build can be halted or continued. Specify any one of the following:

    • Yes: Enable the Yes radio button to allow pipeline to proceed even if the tests fail (at a specific tolerance level such as 90%).

      • Failure Tolerance: Define the failure tolerance such as 90%. If 90% of tests are successful and 10% failed, then the pipeline will still proceed.

    • No: Enable the No radio button to stop the pipeline from execution if even one of the tests fail.

Ability to override the quality gate

On test execution if your Tests are failed and the QualityGate threshold is met the Pipeline will be failed. Enable Ability to override the Quality Gate, If you want the pipeline to RUN even if the Quality gate threshold is met.

Schedule

  • Enable the check box for a test to run at a specific time when a commit happens. You can schedule in the following ways:

    • Force Start: Enable the force start if you need to run the pipeline before Schedule time.

    • Every: Enable the checkbox and select the time (in minutes or hours) to run a test.

    • Time: Enter a specific time. When a commit happens, the test step waits until the defined time and runs at the specified time.

    • Time Zone: Enter the time zone. The test runs at the defined time and in the specified time zone.

    • Recurrence: Enable the check box and define a weekly recurring interval for a test to run on a specific day of the week, when a commit happens.

Notification

  • Enable the check box to notify the user upon the completion of a test.

    • Success: Enable the check box to notify the user when a test is successful.

    • Failure: Enable the check box to notify the user when a test has failed.

    • Notify Committer: Enable the check box to notify the user who made the code change and committed. The platform automatically identifies the user who has committed the code change and notifies the committer.

    • Mode: Select the mode of notification

      • Email ID: Select the mail ID of any specific user from the drop down to send the notification.

      • Slack: When you want to get notification on the Slack tool, you can select Slack → Slack endpoint and Slack Channel.

      • Send Event to Observability Tool: You can also choose to send notifications on the Observability tool by selecting the Send event to Observability tool→ Tool name*.

All the fields marked in Asterisk (*) are mandatory.