All posts
Cloud Nation Tech Brief #01

From ADF webhook alerts to AI-assisted incident reports.

A failed data run deserves more than “Pipeline failed.” A modern Azure architecture can turn technical telemetry into a structured incident report with context, suggested actions, and a reliable fallback.

AzureAzure Data FactoryAzure OpenAIMicrosoft TeamsData Engineering
Reference architecture

From failure signal to actionable message.

AI is an optional enrichment layer. Telemetry, alerting, and a deterministic fallback remain the reliable core.

  1. 01
    Azure Data Factory

    Pipeline or activity failure

  2. 02
    Log Analytics

    Resource-specific ADF logs

  3. 03
    Monitor + Action Group

    Log search alert and trigger

  4. 04
    Logic App / Function

    Validate, filter, orchestrate

  5. 05
    Azure OpenAI

    Optional structured assessment

  6. 06
    Teams Workflow

    Adaptive Card plus fallback

Many Azure Data Factory environments rely on a familiar pattern:

Pipeline failure → Web activity → Teams webhook

It was simple and often sufficient for a basic notification. But an alert that only says “Pipeline failed” leaves the real work to the on-call or Data Platform team: What failed? Is a retry safe? Who should take ownership?

Why replace more than the webhook?

Microsoft completed the retirement of the former Microsoft 365 Connectors in Teams in May 2026. The supported webhook path now leads through Power Automate-based Teams Workflows.

That makes this a useful moment to improve more than an endpoint. The migration can turn a technical failure message into a consistent first incident report.

The target idea is:

ADF failure → Log Analytics → Azure Monitor alert → Logic App or Function → Azure OpenAI → Teams Adaptive Card

Azure OpenAI is not the source of truth in this design. The model structures and explains. The authoritative inputs remain captured telemetry, curated ownership metadata, and established runbooks.

What actually happens in the data flow

  1. Capture telemetry. Diagnostic settings send relevant pipeline, activity, and trigger logs to Log Analytics. In the recommended resource-specific mode, these include ADFActivityRun, ADFPipelineRun, and ADFTriggerRun.

  2. Detect the failure. A KQL query identifies relevant failed runs. An Azure Monitor log search alert invokes an Action Group—Log Analytics does not call a Logic App or Function on its own.

  3. Prepare the context. The Logic App or Azure Function reads allowlisted fields, removes sensitive content, and optionally adds curated metadata such as service, criticality, owner, and runbook.

  4. Enrich with structure. Azure OpenAI receives a controlled prompt and returns a strict JSON schema: short summary, possible cause, recommended actions, retry guidance, and proposed owner. The response is validated before it continues.

  5. Deliver to Teams. A Teams Workflow receives the HTTP request and posts the information as an Adaptive Card to the intended channel or chat.

  6. Guarantee the fallback. If AI enrichment fails, a conventional card is still posted with pipeline, activity, error code, error message, run ID, and runbook link.

Microsoft’s ADF diagnostic configuration notes that logs can take time to arrive in Log Analytics. This is therefore a near-real-time pattern whose latency must be checked against the operational requirements of each environment.

From signal to useful next step

A conventional alert often contains little more than:

Pipeline failed.

An enriched incident report could instead read:

Summary

Customer ingestion failed during the Copy activity after the source API timed out.

Suggested next step

Retry once, check source API availability, and escalate to the Data Platform team if the second run fails.

The operations team receives a starting point for analysis rather than an isolated signal. Any proposed cause or action remains an AI-generated assessment and must be clearly labelled as such.

Which facts are actually available?

The two main resource-specific tables complement each other:

  • ADFActivityRun can include PipelineName, ActivityName, Status, ErrorCode, ErrorMessage, PipelineRunId, and ActivityRunId.
  • ADFPipelineRun can include PipelineName, Status, ErrorCode, ErrorMessage, and RunId.

Business impact and the responsible team cannot be inferred reliably from those fields alone. They should only be included when curated business and ownership metadata is available. Otherwise, the honest output is “not determined from the available telemetry.”

Guardrails for controlled use

  • Allowlist instead of log dumps: Send only required and redacted telemetry fields to the model.
  • Treat error text as untrusted: Separate external content from system instructions and design for indirect prompt injection.
  • Never include secrets or tokens: Secure identities and access through managed identity and least privilege where possible.
  • Require a strict output format: Use Structured Outputs or another validated JSON schema.
  • Separate facts from assessments: Preserve telemetry values and clearly label possible causes and recommendations.
  • Keep a deterministic fallback: Timeouts, throttling, content filtering, or schema failures must never suppress the underlying alert.
  • Use bounded retries: Apply backoff with a clear upper limit.
  • Design for ownership: Teams Workflows have named owners. Assign at least one co-owner so a production flow does not become orphaned when a person leaves.

Microsoft documents both the available Action Group targets and security guardrails for AI workloads. Both belong in the architecture from the beginning, not in a later hardening phase.

Start pragmatically

A useful proof of concept does not need to explain every failure on day one:

  1. Choose one relevant pipeline and a clearly defined failure scenario.
  2. Stabilize the non-AI alerting and fallback path first.
  3. Define allowlisted telemetry, ownership, and the runbook.
  4. Limit AI output to a small set of structured fields.
  5. Review results with the operations team and record misclassifications.
  6. Only then add more pipelines, failure classes, or automated actions.

The real opportunity

The opportunity is not merely replacing an old webhook. Implemented carefully, the pattern can accelerate initial triage, make incident messages more consistent, and shorten the route to the responsible team.

It does not replace monitoring or experienced operations teams. It makes existing telemetry easier to act on—and turns a notification into a more useful next step.

Sources and further reading

From idea to controlled proof of concept

Want to turn Azure alerts into actionable incident reports?

Cloud Nation can help you design a secure reference architecture for your ADF and Teams environment and validate it against real operating requirements.

Talk to us