CI Integration Guides
Integrating CodeComet with your CI system is crucial for automating the collection of test suite results. This guide covers the integration process for popular CI systems like CircleCI and GitHub Actions, as well as general advice for integrating with other CI systems.
To integrate CodeComet with CircleCI, follow these steps to modify your
.circleci/config.yml using the CodeComet Orb. Note that if you use the Orb, you do not need to download and call the
codecomet executable yourself. The Orb will do this for you.
Set API Key
Securely store your CodeComet API Key in CircleCI's Environment Variables (Project Settings > Environment Variables). Add it as
CircleCI Support For JUnit
If you are using a test framework like
vitest that outputs JUnit, here is how you would configure the CodeComet Orb:
GitHub Actions Integration
For GitHub Actions, the integration process involves adding steps to your workflow in
Configure API Key
Add your CodeComet API Key as a secret in your repository's settings (Settings > Secrets) and name it
GitHub Actions Support for JUnit
If you are using a test framework that outputs results to JUnit, you can ingest these results as follows:
Integrating with Other CI Systems
For CI systems other than CircleCI and GitHub Actions, you should set up the following environment variables:
CODECOMET_BRANCH- Your repo branch name
CODECOMET_REPOSITORY- The name of your repository (without the owner)
CODECOMET_REPOSITORY_OWNER- The owner of your repository
CODECOMET_COMMIT_HASH- The current commit hash you are testing
CODECOMET_SEQ_BUILD_ID- This will be used as your Run ID - you should use a sequential, unique Run ID. You can typically get such an ID from your CI system. For example, in CircleCI, this is
CIRCLE_BUILD_NUM, in GitLab it is
CI_JOB_ID, and in Jenkins it is
Installing the CLI
Include a step in your CI pipeline to download and install the CodeComet CLI using the appropriate method for your environment. Below are sample commands for v1.1.0 of our Linux release (please modify as required):
Configuring the API Key
Securely add your CodeComet API Key to your CI system's way of handling secrets or environment variables.
Adjusting Test Commands
If you are invoking the
codecomet CLI directly (e.g. not using the CircleCI Orb or the GitHub Action), then please follow the instructions below to capture the test results.
For Go and Python, use the
codecomet command to wrap your test execution steps, as follows:
codecomet --suite "Sample Test Suite" -- go test ./...
pip install pytest-cov pytest-reportlog && codecomet --suite "Sample Test Suite" -- pytest
For other test frameworks that output JUnit XML, ingest the test results in
codecomet as follows:
jest --ci --reporters=jest-junit && codecomet --suite "Sample Test Suite" junit junit.xml
Refer to your CI system's documentation for specific instructions on adding steps, configuring secrets, and modifying your test execution commands.
CodeComet API Key
The CodeComet API Key is essential for authenticating your CI system with the CodeComet platform. Here's how to manage it:
Retrieval: Log into your CodeComet account, navigate to Settings > API Key, and copy your key
Storage: Store the API Key in your CI system's secure environment variables or secrets management system
Usage: Set the API Key in your CI pipeline before running test commands with CodeComet
Proper management of your API Key ensures secure and seamless integration between CodeComet and your CI workflows.