Blog

The term content engineer means different things depending on who is hiring.

If you have encountered the term in job postings, community discussions, or vendor content and wondered how it differs from technical writing, you aren’t alone. Many technical writers already do much of what companies describe when hiring content engineers, so the roles can appear to overlap.

That raises a reasonable question. Is content engineering a new name for technical writing, or does it describe a genuinely different role?

This article compares the two based on how companies define content engineering in job postings. It examines what each role does, what employers expect, where they overlap, and whether technical writing has ever fully captured the work the role has required.

By the end, you’ll have a clearer line between what each role produces, where they overlap, why the distinction matters, and who you should hire based on your company’s structure and needs.

First, here are some common misconceptions about content engineering.

Misconceptions about Content Engineering

These are three common ways people define content engineering or describe what the role involves:

1. A content engineer is an engineer who writes content. The definition is technically accurate in the same way “a software engineer is someone who writes software” is accurate, but it doesn’t tell you what the person actually does.

By that definition, a mechanical or electrical engineer who writes installation manuals is also a content engineer. The description fits so broadly that it creates confusion about what the person does.

If a software engineer does software engineering, then a content engineer should do content engineering.

1. Content engineering is a new name for technical writing. Companies have repeatedly renamed technical writing as a discipline. Titles like content developer, information developer, and knowledge engineer have appeared, and each time the scope stayed the same while only the name changed.

Mark Baker, author of Every Page is Page One, noted as far back as 2013 that “the constant and uniformly unsuccessful attempts to rename technical writing left many in the field hypersensitive” to yet another name for the same work.

Many technical writers look at what people describe as the requirements for a content engineer and see things they already do, so the obvious conclusion is that the new role is another name. However, it raises the question of whether technical writing was ever a complete title for the role.

1. Content engineering is about scaling content production. This is more common within the search marketing and B2B SaaS industry. Some vendors such as AirOps and another content marketing vendor define content engineering exclusively as building AI production pipelines for scaling content.

In the developer tool space, content goes beyond blog posts. It also includes documentation. Content engineering could involve building drift detection systems, quality gates, review automation, and maintenance workflows.

That work has nothing to do with producing more marketing pages.

What a Technical Writer Does

A technical writer’s primary output is content, including API documentation, integration guides, SDK references, tutorials, release notes, and changelogs.

They research the product, talk to engineers, test the software themselves, and translate that understanding into content that developers can use.

Beyond producing new content, technical writers maintain what already exists. When they spot inaccuracies or style issues in shipped content, they update the affected pages. When the product ships a new version, they revise the docs to match. This ongoing maintenance work is a significant part of the role and often takes more time than writing new material.

Technical writers work within content systems. They open pull requests, follow the review process, and publish through the Docs as Code pipeline. The pipeline itself, however, isn’t their core responsibility. They operate inside the workflow rather than building or maintaining it.

That said, the industry has been shifting. As companies adopted Docs as Code workflows, roles like documentation engineer emerged. These were people who owned the system for producing and maintaining documentation, rather than the content alone. Some technical writers moved into that work, a shift that predates the current content engineering conversation.

What a Content Engineer Does

A content engineer’s primary output is systems, not content. They build the infrastructure that content moves through, including pipelines, automation, quality gates, and workflows.

Their work can involve building documentation systems, including drift detection that flags outdated docs when product code changes, prose linters, automated reviews in CI, maintenance workflows, and reusable templates that reduce effort per update. This is documentation engineering work.

However, content engineering also covers marketing content systems, including production workflows, quality enforcement, topic research workflows, and maintenance automation that keep published blog posts and guides aligned with the product as it evolves.

While documentation engineering covers docs infrastructure, content engineering extends that work to marketing systems as well.

The Overlap and Difference

Content engineers and technical writers share the same content environment, tooling, and quality concerns.

Both roles use the same publishing environment, review process, and quality standards, which is why the distinction can feel blurry, especially in small teams where the same person may write content, maintain workflows, and improve the tooling around publication.

If the primary responsibility is producing content, that’s technical writing. If the primary responsibility is building and maintaining the systems that content moves through, that’s content engineering.

A technical writer improves the content inside the workflow. A content engineer improves the workflow itself, including the review rules, automation, quality gates, publishing process, and maintenance systems around the content.

For practitioners who were already building pipelines, configuring automation, and designing workflows alongside their writing, “technical writer” was never the most accurate description of the full scope.

Content engineering names that systems work more precisely.

Deciding Who to Hire

A technical writer’s job is as valuable as ever. The question is which role gets you to your goal faster, given your team’s structure.

For teams with significant content volume, building systems that produce and maintain content delivers more leverage than managing each piece manually.

If you already have a technical writer who’s been building tooling and automation alongside their content work, you could formalize that as their primary responsibility. They already understand the content, and now they’ll own the infrastructure as well.

If the systems work is complex enough and your writer’s time is better spent on content, hiring a dedicated content engineer is the better move. That way, your writer isn’t split between deadlines and infrastructure, and your systems get the dedicated attention they need to stay reliable.

Or you outsource the engineering work.

TinyRocket builds content engineering systems for developer tool companies. Content pipelines, quality automation, drift detection, and production workflows keep the systems work off your team’s roadmap, so they can focus on producing and maintaining content. Whether you have a technical writer who needs systems support or no dedicated content person at all, book a workflow audit to map your current content workflow and identify which parts automation can handle.

According to Linear, teams using AI agents now ship 5x as many pull requests as two years ago, while Cursor’s data shows developers produce 2.5x as much code as 18 months ago.

More code and faster shipping make documentation drift from the product. Documentation describes one behavior, while code produces another. This hinders adoption, breaks onboarding workflows, and can drive prospective users to competitors or frustrate existing ones.

Tools such as Mintlify offer features to catch documentation drift and automatically update your docs. However, Mintlify’s implementation may not fit your existing workflow, and it locks you into its platform. This guide shows you how to build the same capability without locking into any specific product.

This guide walks through the GitHub Actions workflow we use to catch documentation drift on pull requests with Pi, an open source agent harness. You can replicate the workflow on your own repository.

We’re already running this setup on VectorLint, an open source content review harness we maintain at TinyRocket.

Let’s get started.

What You’ll Build

By the end of this guide, you’ll be able to detect documentation drift in CI. You’ll have three pieces wired together:

1. A GitHub App installed on the repository that gives the workflow permission to post comments
2. An agent skill that tells the agent what counts as documentation drift
3. A GitHub Actions workflow that orchestrates the check on demand, builds the agent’s routing message inline, and posts the result as a PR comment

Note: The workflow flags documentation drift; it doesn’t create a PR or issue. That’s intentional. There could be false positives where the agent flags drift that isn’t user-facing. Automatically creating a pull request for each false positive only adds review work. You can build a downstream workflow that takes this workflow’s output and creates an issue or PR with an agent.

Before You Start

You need a few things in place:

• A GitHub repository following a Docs as Code approach (documentation alongside the code, for example, /docs or a README markdown file)
• Owner or member permission on the repository
• A GitHub account with permission to create GitHub Apps (or an organization where you can create Apps)
• An API key from an AI provider that Pi supports (Anthropic, OpenAI, Google, and others). This guide uses Amazon Bedrock. If you use a different provider, the only change is the authentication parameters.

Pi is an open source coding agent that you run from the command line. It’s lightweight, provider-agnostic, and highly customizable, which is why we chose it for this guide. If you want to test the skill before committing it to CI, install Pi locally.

Note: You can replace Pi with any other agent and get similar results. You’ll need to adapt the setup to how your agent works.

Step 1: Create the Doc-Drift Skill

Copy the doc-drift skill from the VectorLint repository into your project at .agents/skills/doc-drift/. The directory contains:

• SKILL.md: the main instruction the agent follows
• references/user-facing-criteria.md: defines what counts as a user-facing change
• references/comment.md: defines the output format for drift reports

The skill tells the agent to extract user-facing behavioral changes from a PR diff, search the documentation for claims those changes invalidate, and write a report for each finding.

A user-facing change could be a renamed command-line flag, a new environment variable, a changed config key, or a rewritten error message. The skill ignores internal refactors, test changes, and anything with no observable effect on the user.

Note: We built this skill for VectorLint, so it has hardcoded scope paths and repository references. To adapt it, change the scope paths and repository name in SKILL.md and update the criteria in references/user-facing-criteria.md to match your product. You can also hand the skill to an agent and ask it to adapt the references to your codebase.

Step 2: Create and Install a GitHub App

The workflow needs to post comments on pull requests. You can authenticate with either a personal access token or a GitHub App. A GitHub App is safer because you scope its permissions to exactly what the workflow needs. A personal access token tends to carry more permissions than necessary, which increases the blast radius if it’s ever exposed.

Use GitHub’s guide on creating a GitHub App and grant it these permissions:

• Issues: Read & write to post PR comments
• Pull requests: Read & write for the eyes reaction and gh pr diff
• Contents: Read for checkout

After creating the App, grab the App ID and download your private key (a .pem file). Then install the GitHub App on the repository where the workflow runs. You’ll add both values as repository secrets in Step 4.

Step 3: Build the GitHub Actions Workflow

Create a file at .github/workflows/doc-drift.yml. Each section below covers one block of this file. The full workflow appears at the end for copy-pasting.

The Trigger and the Authorization Guard

To guard against cost and security overhead, a documentation check should only run when someone triggers it on a pull request, not on every push.

name: Doc Drift Check

on:
  issue_comment:
    types: [created]

jobs:
  authorize:
    name: Check authorization
    runs-on: ubuntu-latest
    # Only run on PR comments that start with /check-docs (skip bots)
    if: |
      github.event.issue.pull_request != null &&
      startsWith(github.event.comment.body, '/check-docs') &&
      !contains(github.event.comment.user.login, '[bot]')

    permissions:
      issues: write
      pull-requests: read
      contents: read

    outputs:
      authorized: ${{ steps.auth.outputs.authorized }}

The on: issue_comment trigger fires whenever a comment is created. The if: condition narrows triggering to comments on pull requests that start with /check-docs and skips any commenter whose login contains [bot], so bot accounts cannot trigger the workflow. The permissions block grants the minimum access the job needs; write access for comments comes from a GitHub App token (covered below), so GITHUB_TOKEN only needs read access to pull requests.

To guard against unauthorized reviews, only repository owners and members should be able to trigger the drift check. This limits the cost of running AI models and reduces the surface area for prompt injection.

    steps:
      - name: Generate GitHub App token
        id: app-token
        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
        with:
          app-id: ${{ secrets.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}

      - name: Check authorization
        id: auth
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            const association = context.payload.comment.author_association;
            if (!['OWNER', 'MEMBER'].includes(association)) {
              await github.rest.issues.createComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: context.issue.number,
                body: `@${context.payload.comment.user.login} Only repo owners or members can trigger doc drift checks.`
              });
              core.setOutput('authorized', 'false');
            } else {
              core.setOutput('authorized', 'true');
            }

The step reads author_association from the comment payload and allows OWNER and MEMBER. If the commenter is neither, it posts a rejection comment and sets authorized to false. Otherwise, it sets the output to true, allowing downstream jobs to proceed.

Why a GitHub App token, not GITHUB_TOKEN: As covered in Step 2, issue_comment triggers on public repositories downgrade GITHUB_TOKEN to read-only. The App token you created in Step 2 bypasses this restriction, so every write step in the workflow uses it instead.

This mitigates the risk of prompt injection since pull request titles, bodies, and comments are user-controlled strings.

Note: Running an agent in a workflow carries some security risk. A malicious actor could create a pull request containing instructions to get your agent to expose secrets. To avoid that, only grant the minimum permissions your agents need to run. For more on this, read GitHub’s guide on mitigating cloud agent risk.

Check Out the PR and Fetch the Diff

The agent needs the diff to know what changed in the pull request and to check for behavioral changes.

  check-docs:
    name: Check documentation drift
    runs-on: ubuntu-latest
    needs: authorize
    if: needs.authorize.outputs.authorized == 'true'

    permissions:
      issues: write
      pull-requests: read
      contents: read

    steps:
      - name: Generate GitHub App token
        id: app-token
        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
        with:
          app-id: ${{ secrets.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}

      - name: React to comment
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            await github.rest.reactions.createForIssueComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              comment_id: context.payload.comment.id,
              content: 'eyes'
            });

      - name: Get PR head SHA
        id: pr
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            const pr = await github.rest.pulls.get({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number
            });
            core.setOutput('head_sha', pr.data.head.sha);

      - name: Checkout main branch
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
        with:
          ref: ${{ github.event.repository.default_branch }}
          path: main
          fetch-depth: 0

      - name: Checkout PR branch
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
        with:
          ref: ${{ steps.pr.outputs.head_sha }}
          path: pr
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
        with:
          node-version: '22'

      - name: Install Pi
        run: npm install -g @earendil-works/pi-coding-agent@0.79.6

      - name: Show Pi version
        run: pi --version

      - name: Fetch PR diff
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          DIFF_PATH="$GITHUB_WORKSPACE/pr/.doc-drift-input.diff"
          gh pr diff --repo ${{ github.repository }} ${{ github.event.issue.number }} > "$DIFF_PATH"
          echo "Diff size: $(wc -l < "$DIFF_PATH") lines"

The eyes reaction gives the reviewer immediate feedback that the command landed, before the run finishes. The Get PR head SHA step resolves the pull request’s current head commit, so the checkout targets the right ref.

The workflow checks out two copies of the repository: main (the default branch, into path: main) holds the doc-drift skill and the current documentation the agent checks against; pr (the pull request’s head commit, into path: pr) holds the changed code. fetch-depth: 0 gives the agent the full repository history, which it uses to reason about how code and documentation relate. setup-node installs Node version 22, which Pi requires (@earendil-works/pi-coding-agent@0.79.6 needs Node ≥22.19.0). Pi is installed at a pinned version (@0.79.6) for reproducibility. The diff is written to pr/.doc-drift-input.diff so the agent can read it from the PR checkout.

All actions are pinned to full commit SHAs for supply chain safety.

Build the Agent Message

You need to tell Pi where to find the diff, which skill to invoke, and where to write its reports. Rather than maintaining a separate script, the workflow builds this routing message with an inline bash heredoc.

      - name: Build agent message
        run: |
          WORKSPACE="${{ github.workspace }}/pr"
          DIFF_PATH="$WORKSPACE/.doc-drift-input.diff"
          cat > /tmp/pi-message.txt <<EOF
          You are running a doc drift check on a pull request in the VectorLint repository.

          The pull request checkout to inspect is located at:
            ${WORKSPACE}

          Read the PR diff from this file:
            ${DIFF_PATH}

          Use the doc-drift skill. When you have finished, write one report file
          per behavioral change you identified, named sequentially:
            ${WORKSPACE}/.doc-drift-1.md
            ${WORKSPACE}/.doc-drift-2.md
            ... and so on.

          If there are no issues to report, write a single file ${WORKSPACE}/.doc-drift-1.md
          containing the no-issues-found report.

          Do not post anything to GitHub directly. The workflow will handle posting.
          EOF

The heredoc produces a routing message that tells Pi where the PR checkout lives, where to read the diff, and where to write its reports. The variables expand at runtime against github.workspace.

The doc-drift skill it references lives at .agents/skills/doc-drift/, the same skill you created in Step 1.

Run the Drift Check

With the diff on disk, the routing message ready, and Node.js available, the workflow can run Pi and hand it the routing message. This is where the agent reasons about whether documentation needs updating.

A CI runner has no interactive terminal, so Pi runs in a headless, non-interactive mode.

      - name: Run doc drift check
        working-directory: pr
        env:
          AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.PI_BEDROCK_API_KEY }}
          AWS_REGION: ${{ secrets.PI_BEDROCK_REGION }}
          PI_MODEL: ${{ secrets.PI_MODEL }}
        run: |
          pi --no-session -p \
            --provider amazon-bedrock \
            --model "$PI_MODEL" \
            --skill ../main/.agents/skills/doc-drift \
            "/skill:doc-drift $(cat /tmp/pi-message.txt)"

The step runs with working-directory: pr so Pi operates in the PR checkout. --skill ../main/.agents/skills/doc-drift points at the doc-drift skill in the main checkout (one directory up from pr). --no-session makes sure the session isn’t saved, and -p prints the response and exits. Both flags give you non-interactive execution.

When the agent finishes, it writes one report file per behavioral change to numbered paths like .doc-drift-1.md, which the next step picks up. AWS_BEARER_TOKEN_BEDROCK, AWS_REGION, and PI_MODEL come from repository secrets you configure in Step 4.

Post the Findings as PR Comments

The agent is a language model, inherently nondeterministic. Letting it post directly to GitHub would make posting unpredictable. It could emit partial output, retry mid-run, or fail halfway through a comment. A file-based handoff sidesteps that.

      - name: Post report comments
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          REPORT_FILES=$(ls "$GITHUB_WORKSPACE"/pr/.doc-drift-*.md 2>/dev/null | sort -V)
          if [ -z "$REPORT_FILES" ]; then
            gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} \
              --body "Doc drift check completed but produced no output. Check the [Actions log](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details."
          else
            for file in $REPORT_FILES; do
              gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} --body-file "$file"
            done
          fi

      - name: Post failure comment
        if: failure()
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} \
            --body "Doc drift check failed. Check the [Actions log](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details."

The step lists all .doc-drift-*.md files in the pr checkout and loops through them, posting each one as a PR comment with gh pr comment --body-file. The agent never posts to GitHub itself. It only writes files, and the shell loop posts the comments to the PR, producing the same behavior on every run. The workflow passes the App token to gh pr comment for the same reason as every other write: GITHUB_TOKEN would 403 on issue_comment triggers for public repositories.

The workflow owning all GitHub writes keeps the agent’s permissions minimal and makes every comment traceable.

Two fallback paths handle edge cases. If the run produces no report files, a comment tells the reviewer and links to the Actions log. If the run fails outright, the failure() step catches the failure and posts a failure notice with the same log link.

The Complete Workflow

Here’s the full workflow for copy-pasting.

name: Doc Drift Check

on:
  issue_comment:
    types: [created]

jobs:
  authorize:
    name: Check authorization
    runs-on: ubuntu-latest
    # Only run on PR comments that start with /check-docs (skip bots)
    if: |
      github.event.issue.pull_request != null &&
      startsWith(github.event.comment.body, '/check-docs') &&
      !contains(github.event.comment.user.login, '[bot]')
    permissions:
      issues: write
      pull-requests: read
      contents: read
    outputs:
      authorized: ${{ steps.auth.outputs.authorized }}
    steps:
      - name: Generate GitHub App token
        id: app-token
        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
        with:
          app-id: ${{ secrets.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}
      - name: Check authorization
        id: auth
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            const association = context.payload.comment.author_association;
            if (!['OWNER', 'MEMBER'].includes(association)) {
              await github.rest.issues.createComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: context.issue.number,
                body: `@${context.payload.comment.user.login} Only repo owners or members can trigger doc drift checks.`
              });
              core.setOutput('authorized', 'false');
            } else {
              core.setOutput('authorized', 'true');
            }

  check-docs:
    name: Check documentation drift
    runs-on: ubuntu-latest
    needs: authorize
    if: needs.authorize.outputs.authorized == 'true'
    permissions:
      issues: write
      pull-requests: read
      contents: read
    steps:
      - name: Generate GitHub App token
        id: app-token
        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
        with:
          app-id: ${{ secrets.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}
      - name: React to comment
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            await github.rest.reactions.createForIssueComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              comment_id: context.payload.comment.id,
              content: 'eyes'
            });
      - name: Get PR head SHA
        id: pr
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
        with:
          github-token: ${{ steps.app-token.outputs.token }}
          script: |
            const pr = await github.rest.pulls.get({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number
            });
            core.setOutput('head_sha', pr.data.head.sha);
      - name: Checkout main branch
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
        with:
          ref: ${{ github.event.repository.default_branch }}
          path: main
          fetch-depth: 0
      - name: Checkout PR branch
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
        with:
          ref: ${{ steps.pr.outputs.head_sha }}
          path: pr
          fetch-depth: 0
      - name: Setup Node.js
        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
        with:
          node-version: '22'
      - name: Install Pi
        run: npm install -g @earendil-works/pi-coding-agent@0.79.6
      - name: Show Pi version
        run: pi --version
      - name: Fetch PR diff
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          DIFF_PATH="$GITHUB_WORKSPACE/pr/.doc-drift-input.diff"
          gh pr diff --repo ${{ github.repository }} ${{ github.event.issue.number }} > "$DIFF_PATH"
          echo "Diff size: $(wc -l < "$DIFF_PATH") lines"
      - name: Build agent message
        run: |
          WORKSPACE="${{ github.workspace }}/pr"
          DIFF_PATH="$WORKSPACE/.doc-drift-input.diff"
          cat > /tmp/pi-message.txt <<EOF
          You are running a doc drift check on a pull request in the VectorLint repository.

          The pull request checkout to inspect is located at:
            ${WORKSPACE}

          Read the PR diff from this file:
            ${DIFF_PATH}

          Use the doc-drift skill. When you have finished, write one report file
          per behavioral change you identified, named sequentially:
            ${WORKSPACE}/.doc-drift-1.md
            ${WORKSPACE}/.doc-drift-2.md
            ... and so on.

          If there are no issues to report, write a single file ${WORKSPACE}/.doc-drift-1.md
          containing the no-issues-found report.

          Do not post anything to GitHub directly. The workflow will handle posting.
          EOF
      - name: Run doc drift check
        working-directory: pr
        env:
          AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.PI_BEDROCK_API_KEY }}
          AWS_REGION: ${{ secrets.PI_BEDROCK_REGION }}
          PI_MODEL: ${{ secrets.PI_MODEL }}
        run: |
          pi --no-session -p \
            --provider amazon-bedrock \
            --model "$PI_MODEL" \
            --skill ../main/.agents/skills/doc-drift \
            "/skill:doc-drift $(cat /tmp/pi-message.txt)"
      - name: Post report comments
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          REPORT_FILES=$(ls "$GITHUB_WORKSPACE"/pr/.doc-drift-*.md 2>/dev/null | sort -V)
          if [ -z "$REPORT_FILES" ]; then
            gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} \
              --body "Doc drift check completed but produced no output. Check the [Actions log](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details."
          else
            for file in $REPORT_FILES; do
              gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} --body-file "$file"
            done
          fi
      - name: Post failure comment
        if: failure()
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
        run: |
          gh pr comment --repo ${{ github.repository }} ${{ github.event.issue.number }} \
            --body "Doc drift check failed. Check the [Actions log](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details."

Step 4: Configure the Secrets

Add five repository secrets under Settings → Secrets and variables → Actions in your GitHub account.

GitHub App secrets:
– APP_ID: the numeric App ID from the App’s General settings page
– APP_PRIVATE_KEY: the contents of the .pem private key file

Provider secrets:
– PI_BEDROCK_API_KEY
– PI_BEDROCK_REGION
– PI_MODEL

For other providers, change the --provider flag in the workflow and the environment variable names to match. For example, Anthropic needs --provider anthropic and only two secrets ANTHROPIC_API_KEY and PI_MODEL. Check the Pi provider documentation for the environment variable your provider expects.

Step 5: Test the Workflow

On any pull request, leave a comment starting with /check-docs as a repo owner or member. The workflow confirms you’re authorized, runs the agent against the PR diff, and posts one comment per behavioral change the agent found. Here’s a drift finding:

## ⚠️ Doc drift - renamed the `--output` flag to `--format`

### `docs/guides/cli-reference.md` - Output formats

**What the doc claims:** "Run `vectorlint doc.md --output line` for terminal output."
**What's now true:** The flag is `--format`, not `--output`.

Fix prompt:
~~~
`docs/guides/cli-reference.md`, Output formats: "--output" is no longer accurate - the flag is now `--format`. Update it. Keep all existing structure, tone, and style.
~~~

Every surfaced drift finding has a fix prompt. The check tells you what broke, and the prompt gives you a starting point to fix it.

Not every PR drifts documentation. When the agent finds a new user-facing change that isn’t documented yet, the comment names the change and its location. When nothing needs attention, you get a confirmation comment with the files and changes it checked.

Handling False Positives and Negatives

AI models are probabilistic, so there’s bound to be some false positives and negatives.

The agent might flag product changes as documentation drift when they aren’t, or miss drifts when a PR genuinely invalidates documentation.

When you notice either problem, update the instruction in the docs-drift skill. Add a concrete instruction that names the pattern, states what the agent should do differently, and includes an example if the rule isn’t obvious. Then test it locally on the same code change to verify it covers the edge case.

These misses are less frequent with more capable models. If you observe consistent false positives or negatives, consider upgrading to a more capable model.

Resolving the Doc Drift

At this point, your workflow catches documentation drift at the PR level. When a PR changes documented behavior, the agent flags it in the PR itself, no platform required.

The simplest approach to resolving the drift is to update the documentation in the same PR. You can hand the fix message to an agent to update the relevant docs, or build an agent workflow to create a PR based on detected drift.

You’ll need guardrails, constraints, and verification signals to keep the agent’s output accurate, on-style, and useful. These would let you quickly confirm the agent’s work is correct and provide feedback so the agent can self-correct.

Building those guardrails is a project in itself and requires iteration. You need style-aware instructions, verification checks, and feedback loops that let you quickly assess the agent’s output and improve performance over time.

For teams without a dedicated docs engineer, hiring an external documentation engineering service is often more cost-effective and efficient.

At TinyRocket, we build automated workflows, agent skills, verification loops, and tools that help teams keep documentation in sync with their product. Our agentic workflows let your team focus on building while reducing documentation work to a matter of quick judgment. Book a call today to get a free documentation workflow audit.

Related Reading

• What Is Content Engineering for Developer Tools?
• How to Automate Your Content Approval Workflow with Quality Gates
• The Easiest Way to Maintain Doc Quality With Several Contributors
• You Need an AI Prose Linter in Your Docs as Code Workflow

Content engineering, or content operations engineering, has a slightly different definition depending on who you ask.

In enterprise content management, it means applying engineering principles to structure content into reusable modules, manage metadata and taxonomies, and deliver content across channels.

In marketing and search engine optimization (SEO), it means building workflows, automation, and AI to create, update, and distribute content at scale.

For developer tool companies, content engineering is an extension of documentation engineering.

Documentation engineering is the practice of building systems that keep docs aligned as the product evolves. Content engineering extends it to marketing content by building workflows, quality gates, and automation that keep blog posts, tutorials, guides, and launch pages in sync with the product as it ships.

Regardless of how each industry defines it or the tools involved, the common goal is to treat content as a systems problem rather than a series of one-off deliverables.

Content engineering is emerging now because teams want to use AI to gain efficiency and scale content without a proportional increase in effort. AI tools can produce content quickly, but teams have realized that speed without a system leads to inconsistent quality.

This is especially important for developer tool companies as AI increases development velocity.

Why Developer Tools Need Content Engineering

More features shipping faster means more content to produce and maintain, more frequently. For developer tools that pose a risk of content drifting from the product.

Documentation is part of the product, and developers judge your tool by how well your docs help them ship. So when content is out of sync with your product, developer experience degrades.

A stale code sample, a deprecated API reference, or a tutorial that no longer reflects the current UI would cause developers to spend more time verifying your tool instead of building with it.

It also affects how you get surfaced in AI search. AI assistants that read your content use your documentation and marketing pages as a retrieval context. If your docs and marketing pages are factually inconsistent or riddled with terminology inconsistencies, they can surface contradictory product explanations or omit your product altogether.

The good thing is that developer tool companies already use foundational infrastructure such as Git and running CI/CD pipelines, so adopting content engineering practices is a natural extension.

If you’re already practicing Docs as Code, content engineering builds on that foundation.

What Content Engineering Looks Like in Practice

Whatever happens repeatedly in your editorial workflow is a candidate for automating away in a content system. It could be fixing broken links, handling drift, or fixing terminology inconsistencies.

For instance, if your docs live in the codebase, you can detect documentation drift by using a code review tool that can inspect pull requests for changed features and flag affected content for review. Tools such as Qodo and CodeRabbit does this quite well. If your docs live outside the repository, however, you would need to build a system that monitors releases or change logs, cross-references them with the content, and surfaces outdated pages.

Surfacing outdated pages could then automatically create a GitHub issue or a Slack notification to update the team.

AI can reduce manual work, but it shouldn’t make decisions for you. An agent can read and identify the sections that need changes, suggest revisions, and draft a first pass, but a human should still make the final decision to publish.

Although AI workflows can speed up work, a human still needs to remain accountable.

How Content Engineering Supports Developer Marketing

Content engineering supports developer marketing by helping you decide what to publish, write it, review it, and keep it consistent as your product changes.

You can build a repeatable workflow with a topic backlog, a brief template, a source-gathering step, quality checks, and a publishing process to bring in users, create search entry points, and explain your product well enough to earn trust.

This is how we built our demand generation workflow. We use AI for topic research and prioritization, while a human (just me for now) signs off on what to write, and briefs before handing it off to an agent who writes the first draft.

After that, the draft goes through automated quality checks, the agent revises until it meets our quality benchmark, and the human gives the final review. This setup enables one person to keep content moving while also handling software engineering work.

It could also be an event-driven system.

A feature launch can trigger an event that creates a content backlog, generates a brief from the changelog and source docs, drafts a tutorial or comparison post, and then runs checks for claims, terminology, and links before human review.

Whatever workflow is implemented, it must make content creation faster without sacrificing quality.

Benefits of Adopting Content Engineering Practices

Implementing a content system reduces content drift and maintains a good developer experience.

A content system also enforces consistency, making it easier for AI systems to trust and cite. Consistent, technically grounded content gives AI systems an unambiguous representation to work from, improving search visibility.

It also ensures that content quality doesn’t slip as you scale volume. Enforcing quality checks at the writing and submission stages helps you expand your topical coverage while keeping every piece aligned with the same standards.

Building a content system is also cost-effective. Not only does it reduce the workload on already overloaded technical writers or engineers, but it also enables you to scale production without a proportional increase in effort and resources.

Snyk was able to scale its developer marketing strategy by adopting content engineering practices, scaling its developer content production, and driving hundreds of thousands of additional web sessions

In-House vs Outsourcing Content Engineering

You could hire a content engineer to build your content system, or have an in-house technical writer fill the role.

Building a content system is ongoing work, though. The content engineer builds workflows, connects product development to content systems, adds quality gates, sets up knowledge bases, and improves the AI-assisted components when new tasks break the system. AI outputs aren’t deterministic, so the workflow needs ongoing calibration against your quality standards.

Technical writers already juggle documentation, reviews, and contributor support. Adding content engineering to their tasks forces them to context-switch between writing and systems work, which can lead to more content debt or an inefficient system.

At TinyRocket, we build content systems for developer tool companies when your team doesn’t have the capacity to take it on without diverting from the roadmap. We connect pull requests, releases, and change logs to your content inventory, route updates to the right owner, and set up quality-gated production workflows that keep documentation and marketing content aligned with product development.

Book a free workflow audit for an analysis of your editorial process to see where you could automate or apply AI to keep your docs aligned or produce more developer content without sacrificing quality.