Category: Docs as Code

  • Using VectorLint to Catch AI Content Patterns in Docs as Code

    Using VectorLint to Catch AI Content Patterns in Docs as Code

    When you use AI to draft documentation, whether tutorials, guides, or API references, you save hours of writing time.

    The problem is that AI-generated content often contains AI patterns that could erode developer trust:

    “It’s important to note that…”
    “In the landscape of software development…”
    “This isn’t just X; it’s Y.”

    Even when your content is technically accurate, these patterns make it sound lazy.

    You could manually review every piece of content to catch these patterns, or you could automate your review process to catch them instantly.

    This guide shows you how to use VectorLint to automatically check for AI patterns in your Docs as Code workflow.

    VectorLint, AI-Assisted Editing

    VectorLint is an AI-powered prose linter that enables natural-language standard enforcement.

    It uses LLM-as-a-Judge to evaluate content and catch quality issues, making it possible to catch issues like terminology and spelling errors that only require pattern matching, and also those that require contextual understanding, such as AI patterns, Search Engine Optimization (SEO) problems, and technical accuracy.

    As a command-line tool, it runs in CI/CD pipelines, enabling a shared quality gate across your teams, preventing errors from reaching production.

    To get started, install VectorLint on your computer.

    Installing VectorLint

    1. Install VectorLint: To install VectorLint globally, run:
       npm install -g vectorlint

    Verify installation:

       vectorlint --version

    Alternatively, you can run it directly using npx:

       npx vectorlint

    Configuration

    Before you can review content with VectorLint, you need to connect it to an LLM provider.

    1. Initialize VectorLint: Run the initialization command to generate your configuration files:
       vectorlint init

    This creates two files, .vectorlint.ini which contains project-specific settings
    and ~/.vectorlint/config.toml where you configure your LLM provider settings.

       # VectorLint Configuration
       # Global settings
       RulesPath=
       Concurrency=4
       DefaultSeverity=warning
    
       # Default rules for all markdown files
       [**/*.md]
       RunRules=VectorLint

    This configuration tells VectorLint to check all Markdown files using the bundled VectorLint preset.

    1. Configure your API keys: Open your global config file (~/.vectorlint/config.toml) and uncomment the section for your preferred LLM provider. VectorLint supports OpenAI, Anthropic, Google, and Azure models.
       # --- Option 1: OpenAI (Standard) ---
       # LLM_PROVIDER = "openai"
       # OPENAI_API_KEY = "sk-..."
       # OPENAI_MODEL = "gpt-4o"
       # OPENAI_TEMPERATURE = "0.2"
    
       # --- Option 2: Azure OpenAI ---
       # LLM_PROVIDER = "azure-openai"
       # AZURE_OPENAI_API_KEY = "your-api-key-here"
       # ...
    
       # --- Option 3: Anthropic Claude ---
       # LLM_PROVIDER = "anthropic"
       # ANTHROPIC_API_KEY = "your-anthropic-api-key-here"
       # ...
    
       # --- Option 4: Google Gemini ---
       # LLM_PROVIDER = "gemini"
       # GEMINI_API_KEY = "your-gemini-api-key-here"
       # ...

    Uncomment your preferred provider and add your API key. See the configuration guide for full details.

    1. Create a test file: Create docs/getting-started.md with some content containing AI patterns:
       # Getting Started
    
       In the world of software development, getting started with any new tool can be daunting.
    
       This guide is not just a tutorial; it is a comprehensive resource for developers.
    1. Configure AI pattern detection: VectorLint comes bundled with a VectorLint preset that includes AI pattern detection rules. The init command automatically configures this in your .vectorlint.ini file. The VectorLint preset includes these rules: AI-Pattern, Directness, PseudoAdvice, and Repetition. Rules in rule packs are automatically enabled unless explicitly turned off. For this guide, you only need the AI-Pattern rule, so turn off the others in your .vectorlint.ini:
       [VectorLint]
       Directness = disabled
       PseudoAdvice = disabled
       Repetition = disabled

    Running Your First VectorLint Check

    To run VectorLint on a file, use the command:

    vectorlint docs/getting-started.md

    VectorLint should output a quality report in your terminal:

    Adding VectorLint to Your CI/CD Pipeline

    Integrate VectorLint into your GitHub Actions workflow to automatically check documentation on every pull request.

    To add VectorLint to your CI/CD pipeline, create .github/workflows/lint-docs.yml:

    name: Lint Documentation
    on: [pull_request]
    
    jobs:
      vectorlint:
        runs-on: ubuntu-latest
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        steps:
          - uses: actions/checkout@v3
    
          - name: Setup Node
            uses: actions/setup-node@v3
            with:
              node-version: '18'
    
          - name: Run VectorLint
            run: npx vectorlint docs/*.md

    This workflow runs VectorLint on every pull request, checking all modified Markdown files against your quality rules. If VectorLint finds quality issues, the build fails, and the quality report appears in the PR comments.

    Next Steps

    You’ve now automated the detection of excessive hedging in your documentation. But there are more AI patterns worth catching:

    Expand your rule pack:

    • Define rules that match your company’s style guide
    • Share your rule packs across repositories
    • Use VectorLint scores as a metric for documentation quality

    By creating more rules, you can automate more quality checks and save time on content review. This helps you keep up with development velocity while maintaining the quality and trust your brand depends on.

  • You Need an AI Prose Linter in Your Docs as Code Workflow

    You Need an AI Prose Linter in Your Docs as Code Workflow

    With LLMs now capable of creating and reviewing content at scale, your Docs as Code workflow is incomplete without an AI prose linter.

    Although traditional prose linters can catch many errors, their syntactic approach means they can’t catch errors that require contextual judgment.

    To solve this problem, many teams use LLM-powered apps like ChatGPT or Claude. However, this remains outside the team’s shared automated testing workflow, resulting in inconsistent quality.

    These apps aren’t tuned for consistent evaluations, and different team members use different prompts and processes. Even with a shared prompt library, you’re still relying on each contributor to use it correctly.

    An AI prose linter solves this by providing AI reviews and suggestions in your Docs-as-Code workflow. You can achieve reliable automated quality checks by setting the LLM to low temperatures, using structured prompts, and configuring severity levels.

    Making AI Prose Linters Reliable With Severity Levels

    AI prose linters inherit the non-determinism of their underlying technology, which means they will occasionally generate false positives.

    Because the whole point of a CI pipeline is to deliver reliable builds, this is a bad recipe for your pipeline. The solution is to configure them as non-blocking checks that highlight potential issues and suggest fixes without failing your build.

    Just like traditional prose linters aren’t perfect, AI prose linters don’t need to be either.

    Even if you get 50% accuracy on quality flags, you’d be saving half the time you’d otherwise spend hunting for them yourself.

    With that out of the way, here are four reasons you should adopt an AI prose linter in your Docs as Code workflow.

    1. It Reduces Time Spent on Reviews

    AI prose linters reduce the time spent on manual content reviews by catching contextual issues that typically require human judgment.

    While traditional prose linters can catch terminology and consistency issues, the bulk of review time is typically spent on editorial feedback. This involves identifying issues that require contextual judgment, such as whether there is repetition of concepts across sections or if content directly answers the reader’s question.

    By codifying these editorial standards into AI prose linter instructions, you can catch these issues locally or in the CI pipeline, and get suggested fixes. This reduces the mental load on reviewers and saves time.

    2. It Enables Broader Team Contribution

    AI prose linting enables developers, engineers, and product managers to contribute high-quality documentation by providing them with immediate, expert-level editorial feedback as they write.

    Technical writers are often stretched, with some teams operating at a 200:1 developer-to-writer ratio. To get documentation up to date promptly, non-writers often need to contribute. While you can save a lot of time with traditional linters catching typos and broken links, you can make contributing even easier by using AI prose linting.

    Not only does it broaden the scope of issues you catch, but it also helps contributors learn the reason behind the flags and provides them with suggestions to fix them, making them more confident in their contributions.

    3. It Lowers the Barrier to Docs as Code

    Teams who don’t have a dedicated documentation engineer often refrain from adopting a Docs as Code workflow because of its maintenance overhead. It often requires an ongoing effort to create and maintain rules as the team creates more content.

    While traditional linters often have preset style rules that you can start with, you’ll still need to maintain them to deal with false positives that block merges, or to catch new issues that come up.

    AI prose linters solve this problem by using natural language instructions to define rules. This enables you to catch a wide range of issues with fewer instructions, reducing the maintenance overhead.

    For instance, if you wanted to catch hedging language using Vale, you’d need to write a regular expression covering as many variations as you can think of such as `appears to`, `seems like`, `mostly`, `I think`, `sort of`, etc.

    With an AI prose linter, you can simply write:

    `Check for any phrase that connotes uncertainty or lack of confidence (for example, “appears to”, “seems like”)`

    And it can catch variations you never thought to list.

    The trade-off is that natural language tends to leave room for edge cases, and so without precise instruction, you can get false positives. However, the cost of maintaining a wide library of rules or trying to envisage every edge case far outweighs the cost of filtering out false positives.

    4. It Accelerates Productivity For Solo Writers

    To achieve high-quality, error-free content, solo writers still have to review their own work. However, the biggest hurdle isn’t a lack of skill; it’s the human factor. When you’re the only person writing and editing thousands of lines of documentation, you lose the “fresh eyes” benefit that teams take for granted.

    After the fifth hour of editing a technical guide, fatigue sets in, making it easy to miss quality issues. An AI prose linter serves as a peer reviewer, turning the review process into simple “yes” or “no” decisions.

    The AI highlights the issues, and you decide whether they’re valid quality issues or not. This is less mentally taxing and faster than if you had to find the issues yourself.

    Knowing you have an automated editorial pass gives you confidence, allowing you to focus on providing value rather than worrying if you’ve missed a subtle stylistic error.

    Using VectorLint, an Open Source AI Prose Linter

    VectorLint is the first command-line AI prose linting tool.

    We built it to integrate with existing Docs-as-Code tooling, giving your team a shared, automated way to catch contextual quality issues alongside your traditional linters.

    You can define rules in Markdown to check for SEO optimization, AI-generated patterns, technical accuracy, or tone consistency, practically any quality standard you can describe objectively.

    Like Vale or other linters you already use, VectorLint runs in your terminal and CI/CD pipeline as part of your standard testing workflow.

    Check it out on Github

  • Content as Code: Why Technical Teams Should Lint Prose Like They Lint Code

    Content as Code: Why Technical Teams Should Lint Prose Like They Lint Code

    Your engineering team already relies on linters like ESLint, Prettier, or Rubocop to catch code issues before they reach production.

    These tools flag style violations, enforce consistency, and reduce code review time by automating the tedious aspects of quality control, leading to quicker code reviews, consistent output, and a lower mental burden.

    Unfortunately, most technical teams don’t apply this same practice to their technical content.

    Docs, guides, and tutorials undergo entirely manual reviews, where reviewers spend time flagging missing Oxford commas, inconsistent terminology, and overly long sentences.

    If you’re a small content team supporting dozens of engineers or managing contributions from external developers, this manual review process becomes a bottleneck.

    Reviewing technical content manually results in slow publishing cycles, inconsistent docs quality, and frustrated contributors.

    This is where prose linting comes in.

    Prose linting solves these issues by automating style and consistency checks the same way code linters do. It’s a significant part of the Content as Code approach: treating all technical content with the same rigor as code.

    What Is Content as Code?

    Content as Code extends the Docs as Code methodology beyond docs to all technical content, meaning guides, tutorials, and blog posts, treating them with the same rigor as software code.

    This means writing content in Markdown, storing it in version control, conducting reviews through pull requests, and running automated quality checks as part of your publishing pipeline.

    Everything sits in the same repository and follows the same development workflow.

    This approach works well for engineering teams because it integrates seamlessly with the tools they use daily. There’s no need to switch to a separate editor or learn a new publishing system. Writing, reviewing, and releasing docs becomes part of the same workflow as writing code.

    This methodology provides proper version history for all content, supports collaborative workflows across roles, and supports the addition of automated quality checks as part of the publishing pipeline.

    Organizations like Google, Microsoft, GitHub, Datadog, and ContentSquare already use Docs as Code. Extending it to all technical content enables faster growth.

    You Grow Faster with Automated Content QA

    Ship Content Faster

    Automated linting handles the first-pass review for style, terminology, and consistency, so human reviewers can focus on technical accuracy instead of policing syntax.

    Authors get feedback immediately in their IDE or when they open a pull request, allowing issues to be corrected before review, shortening review cycles, and increasing merge speed without reducing quality.

    At Datadog, a 200:1 developer-to-writer ratio still supports reviewing around 40 docs pull requests per day. In 2023, the team merged over 20,000 pull requests across dozens of products and integrations.

    Automated checks made this volume possible by catching repetitive style issues early and reducing the mental load on writers and reviewers without adding headcount.

    When a prose linter like Vale runs in CI, it catches most style guide violations even before a human reviewer sees the PR, so reviewers spend less time pointing out minor fixes, and pull requests move through review faster.

    Scale Your Contributor Program

    External contributors often submit content with inconsistent style and terminology because they haven’t memorized your style guide. Without automated checks, reviewers spend time explaining these standards through manual feedback on every PR.

    However, automated prose linting communicates these standards through immediate, actionable feedback, so contributors see and grasp “what good looks like” without reading lengthy style guide documents.

    ContentSquare saw measurable growth in engineering contributions after implementing Vale for prose linting.

    Engineers reported feeling more confident contributing because they received clear guidance about what to fix. This lowers the barrier to entry for developers who are technical experts first and writers second.

    Build Developer Trust

    Consistent terminology and style signal professionalism and reliability to your users.

    Docs quality directly shapes product trust and adoption, especially with technical audiences who quickly lose confidence when they encounter inconsistent terminology or explanations.

    If one page says “authenticate” and another says “log in,” or if the same concept is described in different ways across pages, developers notice.

    Automated quality checks prevent these issues before they reach production by enforcing consistency across the entire docs set, which helps you maintain credibility as the product evolves.

    Furthermore, automated checks ensure your docs remain current as your products evolve. When docs lag behind releases, developers lose confidence in their accuracy.

    Automated checks solve this by allowing teams to ship updates quickly without compromising quality standards. The combination of consistent quality delivered at speed becomes a competitive advantage for developer adoption.

    What Vale Does

    To implement automated prose linting, most engineering teams start with Vale, an open source prose linter that checks content against style rules.

    It runs locally on your machine, inside your IDE, via pre-commit hooks, in pull request checks, or as part of your CI/CD pipeline.

    Vale understands the syntax of Markdown, AsciiDoc, and reStructuredText, so it can parse your content correctly and ignore code blocks or other markup.

    Alternative prose linters include textlint, proselint, and alex, but Vale is the most common choice among engineering teams.

    Basic Setup Steps

    • Install Vale via your package manager (Homebrew on macOS, Chocolatey on Windows, or apt/yum on Linux)
    • Generate a .vale.ini default configuration file at your project root. You can use existing style packages like Google or Microsoft, or create custom rules.
    Vale-installation-1
    • Run vale sync to download the style package
    • Run vale [path/to/content] to lint your files

    What Automated Feedback Looks Like

    Vale flags issues with line numbers and provides suggested fixes for problems like overly long sentences, filler words, inconsistent terminology, and style violations like missing Oxford commas.

    This feedback appears whether you run Vale in your terminal, your IDE, or as automated comments on pull requests.

    Once you see how Vale flags issues, you’ll be tempted to enable every available rule. Resist that urge.

    Start with 5 Rules, Expand Over Time

    The Progressive Rollout Approach

    Don’t launch with 50 rules; start with 5-10 focused on your biggest pain points. Pilot Vale on one docs set, such as your API reference, getting started guide, or README files.

    Gather feedback from your team and iterate on the rules based on what they find helpful or frustrating. Expand coverage gradually as the system proves its value.

    Teams that roll out too many rules at once face resistance and overwhelm from contributors. A progressive rollout builds team buy-in and confidence in the system.

    You’ll see initial value within weeks, with fewer style inconsistencies flagged during review, and faster merge times. The benefits compound over months as your team internalizes the standards and your rule set matures. 

    ContentSquare implemented this approach and saw a growth in contributions as engineers became more confident in contributing to docs.

    This progressive approach works, but it’s not a “set it and forget it” solution.

    You Need Maintenance To Succeed

    Setting up prose linting requires an initial time investment for installation and rule tuning.

    Some rules will need adjustment based on your specific content type, as what works for API docs may be too strict for tutorial content or technical guides.

    Team buy-in matters more than perfect configuration from day one, and the system works best when integrated into your existing workflow.

    Expect to refine rules as your team discovers edge cases, add exceptions for product-specific terminology, and occasionally tune thresholds for readability metrics.

    The good news is maintenance gets easier over time as your rule set stabilizes and your team adapts to the workflow.

    Many engineering teams at companies like Datadog and ContentSquare automate their content quality checks, reporting significant improvements in review speed and output consistency.

    The challenge for smaller teams is implementing and maintaining the automation pipeline.

    That’s where TinyRocket helps: we implement production-ready prose linting systems tailored to your workflow, so you can focus on building your product, not maintaining docs tooling.