CI Integration Guides

Integrating CodeComet with your CI system is crucial for automating the collection of test suite results. This guide covers the integration process for popular CI systems like CircleCI and GitHub Actions, as well as general advice for integrating with other CI systems.

CircleCI Integration

To integrate CodeComet with CircleCI, follow these steps to modify your .circleci/config.yml using the CodeComet Orb. Note that if you use the Orb, you do not need to download and call the codecomet executable yourself. The Orb will do this for you.

orbs:
  codecomet-orb: codecomet/codecomet-orb@1.0.0
  
  # Then, inside your build job
  - codecomet-orb/install
  - codecomet-orb/ingest:
      # Your test command should go here:
      test_command: go test ./...
      # Your test suite should have the same unique name every time it's run:
      test_suite_name

Set API Key

Securely store your CodeComet API Key in CircleCI's Environment Variables (Project Settings > Environment Variables). Add it as CODECOMET_API_KEY.

CircleCI Support For JUnit

If you are using a test framework like vitest that outputs JUnit, here is how you would configure the CodeComet Orb:

orbs:
  codecomet-orb: codecomet/codecomet-orb@1.0.0
  
  # Then, inside your build job
  - run: npx vitest --reporter=junit --outputFile=junit-test.xml
  - codecomet-orb/install
  - codecomet-orb/ingest_junit:
      junit_file: junit-test.xml
      test_suite_name

GitHub Actions Integration

For GitHub Actions, the integration process involves adding steps to your workflow in .github/workflows/workflow.yaml:

  # Add an action to run your tests with the CodeComet action:
  - name: Run tests
    uses: codecomet-io/setup-and-run@v1.3
    env:
      # You can save the CODECOMET_API_KEY in your repo's secrets:
      CODECOMET_API_KEY: ${{ secrets.CODECOMET_API_KEY }}
    with:
      # Your test command
      test_command: pytest
      test_suite_name

Configure API Key

Add your CodeComet API Key as a secret in your repository's settings (Settings > Secrets) and name it CODECOMET_API_KEY.

GitHub Actions Support for JUnit

If you are using a test framework that outputs results to JUnit, you can ingest these results as follows:

  # A previous action that generates a JUnit report:
  - run: npm run test:src -- --reporter mocha-junit-reporter
  # Add an action to ingest your JUnit file with the CodeComet action:
  - name: Ingest JUnit
    uses: codecomet-io/setup-and-run@v1.3
    env:
      # You can save the CODECOMET_API_KEY in your repo's secrets:
      CODECOMET_API_KEY: ${{ secrets.CODECOMET_API_KEY }}
    with:
      junit_file: test-results.xml
      test_suite_name

Integrating with Other CI Systems

For CI systems other than CircleCI and GitHub Actions, you should set up the following environment variables:

• CODECOMET_BRANCH - Your repo branch name

• CODECOMET_REPOSITORY - The name of your repository (without the owner)

• CODECOMET_REPOSITORY_OWNER - The owner of your repository

• CODECOMET_COMMIT_HASH - The current commit hash you are testing

• CODECOMET_SEQ_BUILD_ID - This will be used as your Run ID - you should use a sequential, unique Run ID. You can typically get such an ID from your CI system. For example, in CircleCI, this is CIRCLE_BUILD_NUM, in GitLab it is CI_JOB_ID, and in Jenkins it is BUILD_NUMBER.

Installing the CLI

Include a step in your CI pipeline to download and install the CodeComet CLI using the appropriate method for your environment. Below are sample commands for v1.1.0 of our Linux release (please modify as required):

# Download the v1.1.0 release for Linux
export CCVERSION=1.1.0
curl -L -o codecomet-cli.tar.gz https://github.com/codecomet-io/cli/releases/download/v${CCVERSION}/codecomet-v${CCVERSION}-linux-amd64.tar.gz

# Decompress the CLI tool
tar -xzvf codecomet-cli.tar.gz -C

Configuring the API Key

Securely add your CodeComet API Key to your CI system's way of handling secrets or environment variables.

Adjusting Test Commands

If you are invoking the codecomet CLI directly (e.g. not using the CircleCI Orb or the GitHub Action), then please follow the instructions below to capture the test results.

For Go and Python, use the codecomet command to wrap your test execution steps, as follows:

codecomet --suite "Sample Test Suite" -- go test ./...

or

pip install pytest-cov pytest-reportlog && codecomet --suite "Sample Test Suite" -- pytest

For other test frameworks that output JUnit XML, ingest the test results in codecomet as follows:

jest --ci --reporters=jest-junit && codecomet --suite "Sample Test Suite" junit junit.xml

Refer to your CI system's documentation for specific instructions on adding steps, configuring secrets, and modifying your test execution commands.

CodeComet API Key

The CodeComet API Key is essential for authenticating your CI system with the CodeComet platform. Here's how to manage it:

• Retrieval: Log into your CodeComet account, navigate to Settings > API Key, and copy your key

• Storage: Store the API Key in your CI system's secure environment variables or secrets management system

• Usage: Set the API Key in your CI pipeline before running test commands with CodeComet

Proper management of your API Key ensures secure and seamless integration between CodeComet and your CI workflows.