Test Frameworks Guides

CodeComet is designed to integrate seamlessly with a wide range of test frameworks, enhancing your test observability and analysis capabilities. This guide provides an overview of integrating CodeComet with go test, pytest, and frameworks that support JUnit test results, thereby covering many programming languages.

go test

The go test tool automates testing the packages of your Go projects. CodeComet enhances this process by capturing test results and relevant metrics for monitoring and analysis.

Integrating CodeComet with go test

Install CodeComet CLI

Make sure the CodeComet CLI is installed on your system. You can find the latest releases here: https://github.com/codecomet-io/cli/releases

Follow the instructions in Quickstart Guide or CodeComet CLI.

Set CodeComet API Key

Retrieve the CodeComet API Key from Settings > API Key in the web application. Set the environment variable CODECOMET_API_KEY using a command like the following:

export CODECOMET_API_KEY=<your_api_key>

Modify Your Test Command

Wrap your go test command with CodeComet to capture and upload test results. For example:

codecomet --suite "Go Test Suite" --

This command executes all tests in the current project, captures the coverage information, and uploads the results to CodeComet under the "Go Test Suite" name.

pytest

pytest is a powerful testing framework for Python that makes it easy to write simple tests, yet scales to support complex functional testing. Integrating pytest with CodeComet allows for detailed test observability.

Integrating CodeComet with pytest

Install CodeComet CLI

Make sure the CodeComet CLI is installed on your system. You can find the latest releases here: https://github.com/codecomet-io/cli/releases

Follow the instructions in Quickstart Guide or CodeComet CLI.

Set CodeComet API Key

Retrieve the CodeComet API Key from Settings > API Key in the web application. Set the environment variable CODECOMET_API_KEY using a command like the following:

export CODECOMET_API_KEY=<your_api_key>

Install pytest-cov and pytest-reportlog

For coverage reporting, install pytest-cov and pytest-reportlog:

Run Your Tests with CodeComet

Prepend your pytest command with codecomet. For example:

codecomet --suite "Python Test Suite" --

This command will run all tests and upload the results to CodeComet under the "Python Test Suite" name.

JUnit Support

CodeComet's support for JUnit test results format extends our compatibility to numerous languages and test frameworks, including:

• Java: JUnit

• JavaScript/TypeScript: Jest, Vitest, Mocha

• PHP: PHPUnit

• C#: NUnit, xUnit

• Ruby: RSpec (with JUnit formatter)

• Scala: ScalaTest

• Kotlin: Kotest

• Swift: XCTest (with JUnit formatter)

• Rust: Cargo test (with JUnit formatter)

Integrate your tests with CodeComet by first generating JUnit XML output and using the CodeComet CLI to upload the results using codecomet —suite "Sample Test Suite" junit <test_results.xml>.

Integrating codecomet with jest

Here is how you would modify your existing jest command and ingest the results into codecomet:

# Install the jest-junit test reporter
npm install --save-dev jest-junit

# Run Jest and generate a JUnit XML report
jest --ci --reporters=jest-junit

# Ingest the JUnit XML report into CodeComet
codecomet --suite "Sample Test Suite"

Integrating codecomet with vitest

In similar fashion, here is how you would generate the test results file in vitest and ingest it into codecomet:

# Run Vitest and generate a JUnit XML report
vitest run --reporter=junit --outputFile=junit.xml

# Ingest the JUnit XML report into CodeComet
codecomet --suite "Sample Test Suite"

Stay tuned for updates on our website and documentation. If you have a specific framework or language you'd like to see supported by CodeComet, please reach out via Support.