CodeComet CLI

The CodeComet Command Line Interface (CLI) tool is an integral part of the CodeComet test observability toolkit. It facilitates the integration of CodeComet with your automated testing environment, enabling you to easily upload test suite results to the CodeComet platform for analysis and monitoring.

Command Line Interface Basics

The CodeComet CLI is designed to be user-friendly and integrates easily into existing test workflows. Before you use the CLI, ensure that it is properly installed on your system.

Installing the CLI

Navigate to https://github.com/codecomet-io/cli/releases, then download the latest release zip file for your operating system. Unzip the file to your desired directory. This will create a codecomet executable. Here are the sample commands for downloading and installing the v1.1.0 of our Linux release (please modify as needed):

# Download the v1.1.0 release for Linux
export CCVERSION=1.1.0
curl -L -o codecomet-cli.tar.gz https://github.com/codecomet-io/cli/releases/download/v${CCVERSION}/codecomet-v${CCVERSION}-linux-amd64.tar.gz

# Decompress the CLI tool
tar -xzvf codecomet-cli.tar.gz -C

Installation Check

To verify that the CodeComet CLI is installed, run:

This command will return the version of the CodeComet CLI installed on your machine, indicating that the installation was successful.

Integrating with Test Commands

Integrating the CodeComet CLI with your test commands is straightforward. The CLI tool wraps around your existing test commands, capturing test output and relevant metrics, then uploads them to your CodeComet account.

Basic Integration

For Go and Python, prepend your test command with codecomet. For other languages and test frameworks, first generate the test results in the JUnit XML format, and then ingest that using codecomet junit <file.xml>. Below are examples of how to capture test suite results using the CLI. For all of these examples, please make sure you have properly set the CODECOMET_API_KEY environment variable (prior to running these commands). You can get your API Key from the CodeComet web app, under Settings.

Go

codecomet --suite "Sample Test Suite" --

Python

pip install pytest-cov pytest-reportlog && codecomet --suite "Sample Test Suite" --

Other languages that support JUnit (like Java, Javascript / Typescript, Ruby)

jest --ci --reporters=jest-junit && codecomet --suite "Sample Test Suite"

Configuration Options and Environment Variables

The CodeComet CLI offers several configuration options to customize its behavior and integration with your test suites.

• Test Suite Name: Specify the name of your test suite using the --suite flag to organize and identify your test results within the CodeComet test platform.

• Test Suite Run Id: Specify a Run ID for this particular test suite run. While the test suite name should be constant for repeated runs of this test suite, your Run ID should change every time you start a CI job. If you don't provide an ID, we will make the best attempt to generate one from your CI system's environment variables. You can pass in a Run ID with the --runid flag.

Other environment variables to set prior to running the CLI:

• CODECOMET_API_KEY- Set this to your CodeComet API Key. You can obtain this from the Settings menu in the CodeComet web app. This step is required.

If you're not using one of the natively supported CI systems (CircleCI or GitHub Actions, at this time), you should set up the following environment variables as well:

• CODECOMET_BRANCH - Your repo branch name

• CODECOMET_REPOSITORY - The name of your repository (without the owner)

• CODECOMET_REPOSITORY_OWNER - The owner of your repository

• CODECOMET_COMMIT_HASH - The current commit hash you are testing

• CODECOMET_SEQ_BUILD_ID - This will be used as your Run ID - you should use a sequential, unique Run ID. You can typically get such an ID from your CI system. For example, in CircleCI, this is CIRCLE_BUILD_NUM, in GitLab it is CI_JOB_ID, and in Jenkins it is BUILD_NUMBER.

CLI Reference

Below is a reference guide for the CodeComet CLI commands:

• codecomet version: Display the current version of the CodeComet CLI

• codecomet [options] -- <test_command>: Run your test command wrapped by the CodeComet CLI. Options include:

  • --suite: Name of the test suite for identification in the CodeComet platform.

  • --runid: The particular Run ID for this particular test run. CLI invocations that have the same suite name and Run ID will create one single test suite run in the CodeComet app.

• codecomet [options] junit <file_path>: Upload a JUnit XML test report to the CodeComet platform. Use the --suite option to attach a suite name to this JUnit file. Use the --coverage option to attach a test coverage file to the suite as well.