The CodeComet Command Line Interface (CLI) tool is an integral part of the CodeComet test observability toolkit. It facilitates the integration of CodeComet with your automated testing environment, enabling you to easily upload test suite results to the CodeComet platform for analysis and monitoring.
Command Line Interface Basics
The CodeComet CLI is designed to be user-friendly and integrates easily into existing test workflows. Before you use the CLI, ensure that it is properly installed on your system.
Installing the CLI
Navigate to https://github.com/codecomet-io/cli/releases, then download the latest release zip file for your operating system. Unzip the file to your desired directory. This will create a
codecomet executable. Here are the sample commands for downloading and installing the v1.1.0 of our Linux release (please modify as needed):
To verify that the CodeComet CLI is installed, run:
This command will return the version of the CodeComet CLI installed on your machine, indicating that the installation was successful.
Integrating with Test Commands
Integrating the CodeComet CLI with your test commands is straightforward. The CLI tool wraps around your existing test commands, capturing test output and relevant metrics, then uploads them to your CodeComet account.
For Go and Python, prepend your test command with
codecomet. For other languages and test frameworks, first generate the test results in the JUnit XML format, and then ingest that using
codecomet junit <file.xml>. Below are examples of how to capture test suite results using the CLI. For all of these examples, please make sure you have properly set the
CODECOMET_API_KEY environment variable (prior to running these commands). You can get your API Key from the CodeComet web app, under Settings.
Configuration Options and Environment Variables
The CodeComet CLI offers several configuration options to customize its behavior and integration with your test suites.
Test Suite Name: Specify the name of your test suite using the
--suiteflag to organize and identify your test results within the CodeComet test platform.
Test Suite Run Id: Specify a Run ID for this particular test suite run. While the test suite name should be constant for repeated runs of this test suite, your Run ID should change every time you start a CI job. If you don't provide an ID, we will make the best attempt to generate one from your CI system's environment variables. You can pass in a Run ID with the
Other environment variables to set prior to running the CLI:
CODECOMET_API_KEY- Set this to your CodeComet API Key. You can obtain this from the Settings menu in the CodeComet web app. This step is required.
If you're not using one of the natively supported CI systems (CircleCI or GitHub Actions, at this time), you should set up the following environment variables as well:
CODECOMET_BRANCH- Your repo branch name
CODECOMET_REPOSITORY- The name of your repository (without the owner)
CODECOMET_REPOSITORY_OWNER- The owner of your repository
CODECOMET_COMMIT_HASH- The current commit hash you are testing
CODECOMET_SEQ_BUILD_ID- This will be used as your Run ID - you should use a sequential, unique Run ID. You can typically get such an ID from your CI system. For example, in CircleCI, this is
CIRCLE_BUILD_NUM, in GitLab it is
CI_JOB_ID, and in Jenkins it is
Below is a reference guide for the CodeComet CLI commands:
codecomet version: Display the current version of the CodeComet CLI
codecomet [options] -- <test_command>: Run your test command wrapped by the CodeComet CLI. Options include:
--suite: Name of the test suite for identification in the CodeComet platform.
--runid: The particular Run ID for this particular test run. CLI invocations that have the same suite name and Run ID will create one single test suite run in the CodeComet app.
codecomet [options] junit <file_path>: Upload a JUnit XML test report to the CodeComet platform. Use the
--suiteoption to attach a suite name to this JUnit file. Use the
--coverageoption to attach a test coverage file to the suite as well.