CodeComet is designed to deliver a better test management experience for modern engineering teams. Mature software orgs have already embraced automated testing and CI/CD workflows. However, not enough has changed in the way development teams monitor and manage their tests. Fairly simple questions like "Why did this test fail?", "Why do our tests take so long?" and "What changed recently that is making this test suite so unstable?" are still hard to answer. We aim to change that with CodeComet.
What is CodeComet?
CodeComet is a test observability toolkit. It can be utilized to get better visibility into your team's automated test suites. Relevant metrics like time taken, pass/fail status, and logs are stored for every test. The web app interface offers a concise view of test suites' historical and aggregate data, with drill-down options for detailed log analysis of individual tests.
Ultimately, CodeComet's goal is to help modern engineering teams ship quality code, faster.
CodeComet has a few components that work together to help teams better manage their tests. At this time, the two main components are:
CLI tool: A command-line tool that is designed to be called from within a CI workflow. It ingests all relevant test data and records it in your CodeComet account.
Web application: A browser-based interactive dashboard for easily viewing current and past test suite execution results. It enables you to quickly drill down into a test suite to find details about specific tests, and test failures.
Supported Languages and Frameworks
The CodeComet CLI tool acts as a wrapper around your existing
go test command. You will need to add an additional flag to name your test suite:
--suite "Sample Test Suite". Here is an example invocation:
Please note the
-- separating the
codecomet arguments from the rest of the
go test command.
Similar to our support for Go, the CodeComet CLI tool wraps around your existing
pytest command. But first, you will have to install
pip install or similar. Here is an example:
CI Integration Overview
CodeComet integrates into your existing CI system through the CLI tool. We natively support CircleCI and GitHub Actions, but the CLI tool can be deployed within any CI system as long as you can invoke the CodeComet CLI download+install command and modify your CI workflow to have the tool ingest test suite results.
Here's an example of how you would modify your
.circleci/config.yaml to add the CodeComet Orb. Note that if you use the Orb, you do not need to download and call the
codecomet executable yourself. The Orb will do this for you.
For other CI systems, including GitHub Actions, please visit our CI Integration Guides section.