Introduction

CodeComet is designed to deliver a better test management experience for modern engineering teams. Mature software orgs have already embraced automated testing and CI/CD workflows. However, not enough has changed in the way development teams monitor and manage their tests. Fairly simple questions like "Why did this test fail?", "Why do our tests take so long?" and "What changed recently that is making this test suite so unstable?" are still hard to answer. We aim to change that with CodeComet.

What is CodeComet?

CodeComet is a test observability toolkit. It can be utilized to get better visibility into your team's automated test suites. Relevant metrics like time taken, pass/fail status, and logs are stored for every test. The web app interface offers a concise view of test suites' historical and aggregate data, with drill-down options for detailed log analysis of individual tests.

Ultimately, CodeComet's goal is to help modern engineering teams ship quality code, faster.

CodeComet Components

CodeComet has a few components that work together to help teams better manage their tests. At this time, the two main components are:

• CLI tool: A command-line tool that is designed to be called from within a CI workflow. It ingests all relevant test data and records it in your CodeComet account.

• Web application: A browser-based interactive dashboard for easily viewing current and past test suite execution results. It enables you to quickly drill down into a test suite to find details about specific tests, and test failures.

Supported Languages and Frameworks

CodeComet natively supports Go and Python, and through JUnit support, most major languages like Java, Javascript/Typescript, PHP, C#, Ruby, Rust and others.

go test

The CodeComet CLI tool acts as a wrapper around your existing go test command. You will need to add an additional flag to name your test suite: --suite "Sample Test Suite". Here is an example invocation:

codecomet --suite "Sample Test Suite"

Please note the -- separating the codecomet arguments from the rest of the go test command.

pytest

Similar to our support for Go, the CodeComet CLI tool wraps around your existing pytest command. But first, you will have to install pytest-cov and pytest-reportlog using pip install or similar. Here is an example:

pip install pytest-cov pytest-reportlog && codecomet --suite "Sample Test Suite" -- pytest

CI Integration Overview

CodeComet integrates into your existing CI system through the CLI tool. We natively support CircleCI and GitHub Actions, but the CLI tool can be deployed within any CI system as long as you can invoke the CodeComet CLI download+install command and modify your CI workflow to have the tool ingest test suite results.

CircleCI

Here's an example of how you would modify your .circleci/config.yaml to add the CodeComet Orb. Note that if you use the Orb, you do not need to download and call the codecomet executable yourself. The Orb will do this for you.

orbs:
  codecomet-orb: codecomet/codecomet-orb@1.0.0
# ...
# Then, inside your build job
  - codecomet-orb/install
  - codecomet-orb/ingest:
      # Your test command should go here:
      test_command: go test ./...
      # Your test suite should have the same unique name every time it's run:
      test_suite_name

For other CI systems, including GitHub Actions, please visit our CI Integration Guides section.