Publish line and branch coverage reports in CI (#1564)

* Report line and branch coverage in CI

* docs: explain coverage reports and behavioral tests

* Calculate coverage percentages from counts for older versions

.github/workflows/pytest.yaml

@@ -53,6 +53,7 @@ jobs:
           core-version: ${{ matrix.core-version }}
 
       - name: Run pytest
+        id: pytest
         timeout-minutes: 60
         run: |
           export PYTHONPATH=${PYTHONPATH}:${PWD}
@@ -64,7 +65,51 @@ jobs:
             --timeout=9 \
             --durations=10 \
             --cov=homeassistant.components.adaptive_lighting \
+            --cov-branch \
+            --cov-report=term-missing \
             --cov-report=xml \
+            --cov-report=json \
+            --cov-report=html \
             -o console_output_style=count \
             -p no:sugar \
             tests/components/adaptive_lighting
+
+      - name: Write coverage summary
+        if: ${{ !cancelled() }}
+        env:
+          CORE_VERSION: ${{ matrix.core-version }}
+          PYTHON_VERSION: ${{ matrix.python-version }}
+          PYTEST_OUTCOME: ${{ steps.pytest.outcome }}
+        run: |
+          {
+            echo "### Coverage: Home Assistant ${CORE_VERSION}, Python ${PYTHON_VERSION}"
+            echo
+            if [[ -f core/coverage.json ]]; then
+              echo "| Metric | Executed | Total | Coverage |"
+              echo "| --- | ---: | ---: | ---: |"
+              jq -r '
+                def percent(covered; total):
+                  if total == 0 then 100 else (covered / total * 10000 | round) / 100 end;
+                .totals
+                | "| Lines | \(.covered_lines) | \(.num_statements) | \(percent(.covered_lines; .num_statements))% |\n"
+                + "| Branches | \(.covered_branches) | \(.num_branches) | \(percent(.covered_branches; .num_branches))% |"
+              ' core/coverage.json
+            else
+              echo "Coverage JSON was not generated. See the pytest step for details."
+            fi
+          } >> "${GITHUB_STEP_SUMMARY}"
+
+          if [[ ! -f core/coverage.json && "${PYTEST_OUTCOME}" == "success" ]]; then
+            exit 1
+          fi
+
+      - name: Upload coverage reports
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v7.0.1
+        with:
+          name: coverage-${{ matrix.core-version }}-py${{ matrix.python-version }}
+          path: |
+            core/coverage.xml
+            core/coverage.json
+            core/htmlcov/
+          if-no-files-found: warn

tests/README.md

@@ -1,8 +1,16 @@
 # Developer notes for the tests directory
 
-To run the tests, check out the [CI configuration](../.github/workflows/pytest.yml) to see how they are executed in the CI pipeline.
+To run the tests, check out the [CI configuration](../.github/workflows/pytest.yaml) to see how they are executed in the CI pipeline.
 Alternatively, you can use the provided Docker image to run the tests locally or run them with VS Code directly in the dev container.
 
+## Coverage reports
+
+Open a `pytest` workflow run in GitHub Actions to see line and branch coverage in each job's summary. Download its `coverage-<Home Assistant version>-py<Python version>` artifact for the XML and JSON reports and the browsable HTML report. After extracting it, open `htmlcov/index.html` to inspect missing lines and branches.
+
+Coverage measures executed code, not whether assertions would catch a bug. Add tests for observable behavior: emitted light commands, final states, manual-control events, and timer expiry. The integration suite runs inside Home Assistant with simulated lights; it does not establish physical-device behavior. It also does not execute every documentation generator included in the package's coverage total.
+
+The tests in `test_automation_examples.py` load YAML directly from `README.md` and execute it through Home Assistant's automation and script engines. Edit the README source when changing those examples, then run `./scripts/update-generated-content` to update the documentation pages.
+
 ## Prerequisites
 
 Before running tests with Docker, you need a local Home Assistant core checkout with symlinks: