Azure Monitor: The Enterprise Guide to Full-Stack Observability, Log Analytics, and KQL
You cannot manage what you cannot observe. Azure Monitor provides the complete observability stack for cloud-native and hybrid enterprise environments — collecting metrics, logs, traces, and dependencies from every layer of the infrastructure and application stack. This guide covers enterprise Azure Monitor architecture, Log Analytics workspace design, Application Insights for APM, alert strategy, KQL query patterns, Azure Workbooks, and cost optimization — based on EPC Group's experience across 200+ Azure enterprise deployments.
Azure Monitor Enterprise Observability Guide 2026
Azure Monitor is Microsoft's unified observability platform for cloud and hybrid environments. It collects metrics, logs, and traces through Log Analytics, Application Insights, Alerts, and Workbooks. EPC Group implements Azure Monitor as part of every enterprise Azure deployment — observability is a foundational layer, not an afterthought. We have completed 200+ enterprise Azure Monitor deployments.
Key facts
- Azure Monitor pricing: Log Analytics charges ~$2.76/GB ingested (pay-as-you-go). First 5 GB/month free. 100 GB/day commitment tier: ~$1.70/GB (38% savings).
- Data retention: free for first 31 days. Additional retention: $0.10/GB/month. Long-term archive: $0.02/GB/month.
- Enterprise deployments ingesting 50–200 GB/day: expect $4,000–$16,000/month.
- Commitment tiers and DCR filtering reduce monitoring costs by 30–50%.
- EPC Group has migrated enterprises from 20+ fragmented workspaces to 2–3 consolidated workspaces, reducing monitoring costs by 40%.
- Alert rules: $0.10–$1.50/rule/month depending on type.
The Azure Monitor Observability Stack
Enterprise observability requires three pillars:
- Metrics — what is happening right now (lightweight, time-series numeric values)
- Logs — what happened and why (detailed structured records in Log Analytics)
- Traces — how requests flow through distributed systems (Application Insights)
Azure Monitor provides all three pillars in a unified platform with a common query language (KQL), a common alerting engine, and a common visualization layer.
Azure Monitor includes these core components:
- Log Analytics — log collection and KQL querying
- Application Insights — application performance monitoring (APM)
- Metrics Explorer — real-time metric visualization
- Alerts — proactive notification and automated response
- Workbooks — interactive dashboards and reports
- Azure Monitor Agent — data collection from VMs
- Network Insights — network monitoring and diagnostics
Log Analytics: Architecture and Workspace Design
Log Analytics workspaces are the central data stores for Azure Monitor Logs. All log data flows into workspaces where it is indexed, queryable via KQL, and retained for the configured period.
The most common mistake is creating too many workspaces — per application, per team, or per subscription. This creates data silos and disqualifies commitment tier pricing.
EPC Group recommends a centralized architecture:
- Centralized workspace (recommended): One workspace per Azure region. Supports cross-resource correlation, simplified RBAC, and commitment tier pricing. Most enterprises use 1–2 workspaces.
- Dedicated security workspace: A separate workspace for Microsoft Sentinel accessible only by the SOC team. Provides data isolation and independent retention policies (security data often needs 1–2 years for compliance).
- Workspace per environment (dev/staging/prod): Useful for cost chargeback by environment. Sacrifices cross-environment correlation — use only when chargeback is required.
Avoid per-resource or per-application workspaces — they create data silos and prevent cross-resource correlation.
Data Retention and Archiving
Azure Monitor Log Analytics provides three retention tiers:
- Interactive retention: Immediately queryable via KQL. First 31 days free, then $0.10/GB/month. Configure at the table level.
- Long-term retention (archive): Queryable via restore or search jobs. $0.02/GB/month. Best for compliance data you need to keep but rarely query.
- Export to storage: Continuous export to Azure Storage for indefinite retention at blob storage pricing.
Configure table-level retention policies. High-value tables (SecurityEvent, AppExceptions) may need 365 days of interactive retention. High-volume tables (Heartbeat, Perf) may only need 31 days of interactive retention with 90 days of archive.
Data Collection Rules and Azure Monitor Agent
Data Collection Rules (DCRs) control what data is collected, how it is transformed, and where it is sent. DCRs replace legacy workspace-based agent configuration and give granular control over data ingestion.
The Azure Monitor Agent (AMA) is the unified agent for Windows and Linux VMs. It replaces the legacy Log Analytics Agent (MMA/OMS), Diagnostics extension, and Telegraf agent. AMA uses DCRs for configuration, supports multi-homing (multiple workspaces), and uses managed identity authentication.
DCRs support KQL-based transformations that filter or modify data before it reaches the workspace. Dropping unnecessary columns and filtering noisy logs reduces ingestion volume by 30–60% without losing operational value.
Application Insights: APM
Application Insights is the APM component of Azure Monitor. It collects telemetry from web applications: requests, dependencies, exceptions, traces, page views, and custom events.
Three instrumentation options:
- Auto-instrumentation (recommended for Azure PaaS): Enable on Azure App Service, Azure Functions, or AKS without code changes. Supported runtimes: .NET, Java, Node.js, Python.
- SDK instrumentation: Add the Application Insights SDK to your application code for maximum control. Enables custom telemetry (TrackEvent, TrackMetric, TrackTrace) and complex dependency tracking.
- OpenTelemetry: Use the Azure Monitor OpenTelemetry Exporter for multi-cloud or vendor-neutral observability strategies.
Key features: Application Map, Transaction Search, Failures and Performance dashboards, Smart Detection (ML-based anomaly alerts), and Availability tests (synthetic monitoring).
KQL: The Query Language for Azure Observability
KQL (Kusto Query Language) is the query language for Log Analytics, Application Insights, Microsoft Sentinel, and Microsoft Defender. It uses a pipe-based syntax where data flows through a series of operators:
source table | filter | transform | aggregate | sort | render
Example — show which VMs reported in the last hour:
Heartbeat | where TimeGenerated > ago(1h) | summarize count() by Computer | order by count_ desc
KQL proficiency is a core skill for any Azure operations or security team. It is the query language for all Azure observability and security platforms.
Azure Monitor Alerts: Key Concepts
An Azure Monitor alert has three components:
- Alert rule — defines the condition (scope, signal type, threshold, frequency)
- Action group — defines who gets notified (email, SMS, webhook, Logic App, Azure Function, ITSM) and what automated actions to take
- Alert processing rule — optional; controls notification routing and suppression during maintenance windows
Alert severity levels: 0-Critical, 1-Error, 2-Warning, 3-Informational, 4-Verbose.
Best practices:
- Alert on symptoms (response time exceeds SLA, error rate spikes) not causes (CPU high) alone.
- Use dynamic thresholds — ML-based baselines eliminate manual threshold maintenance.
- Suppress during planned maintenance windows using alert processing rules.
- Connect action groups to Azure Automation runbooks for auto-remediation.
Cost Optimization for Azure Monitor
Azure Monitor costs grow with data volume. EPC Group implements these controls on every enterprise deployment:
- Commitment tiers: If ingesting 100+ GB/day, switch from pay-as-you-go to a commitment tier. 100 GB/day: 38% discount. 500 GB/day: 50% discount.
- DCR filtering: Drop verbose informational logs and filter performance counters to only the metrics you monitor. Reduces ingestion by 30–60%.
- Table-level retention: Configure shorter interactive retention for high-volume, low-value tables (Heartbeat, Perf). Archive data to long-term tier ($0.02 vs. $0.10/GB/month).
- Application Insights sampling: Adaptive sampling automatically adjusts the sampling rate based on telemetry volume. Reduces cost on high-traffic applications with no material accuracy loss.
- Basic logs tier: For high-volume tables rarely queried (verbose diagnostic logs), use Basic Logs at $0.50/GB vs. $2.76/GB for Analytics tier. Limited KQL, 8-day retention, ideal for troubleshooting-only data.
Frequently Asked Questions
What is Azure Monitor and what does it include?
Azure Monitor is Microsoft's comprehensive observability platform for Azure and hybrid environments. It includes Log Analytics, Application Insights, Metrics Explorer, Alerts, Workbooks, Azure Monitor Agent, and Network Insights. Together they provide full-stack observability across infrastructure, applications, and networks.
How much does Azure Monitor cost?
Log Analytics charges ~$2.76/GB ingested (pay-as-you-go) with the first 5 GB/month free. Data retention is free for the first 31 days. Alert rules cost $0.10–$1.50/rule/month. For most enterprise deployments (50–200 GB/day), expect $4,000–$16,000/month. Commitment tiers and DCR filtering cut costs by 30–50%.
What is the difference between Azure Monitor Logs and Metrics?
Metrics are lightweight, numeric time-series values (CPU %, request count) stored in a fast time-series database. Logs are detailed, structured records stored in Log Analytics and queried with KQL. Use Metrics for real-time alerting; use Logs for investigation, troubleshooting, and complex analytics.
What is KQL and why is it important?
KQL (Kusto Query Language) is the query language for Log Analytics, Sentinel, Defender, and Azure Data Explorer. It is a read-only language optimized for large volumes of semi-structured telemetry. Proficiency in KQL is essential for any Azure operations or security team.
What is the recommended Log Analytics workspace architecture?
One primary workspace per Azure region for all resources in that region. Add a dedicated security workspace for Microsoft Sentinel data. Avoid per-application or per-resource workspaces — they create silos and prevent cross-resource correlation.
Partner with EPC Group
EPC Group was a Microsoft Gold Partner (2016 to program retirement, the oldest in North America) and is currently a Microsoft Solutions Partner. Our Azure cloud practice implements comprehensive observability from Log Analytics workspace architecture and data collection optimization through custom workbook development, alert strategy design, and ongoing monitoring operations.
Frequently Asked Questions
What is Azure Monitor and what does it include?
Azure Monitor is the comprehensive observability platform for Azure and hybrid environments. It collects, analyzes, and acts on telemetry data from cloud and on-premises resources. Azure Monitor includes: Log Analytics (log collection and KQL querying), Application Insights (application performance monitoring/APM), Metrics Explorer (real-time metric visualization), Alerts (proactive notification and automated response), Workbooks (interactive dashboards and reports), Azure Monitor Agent (data collection from VMs), and Network Insights (network monitoring and diagnostics). Together, these components provide full-stack observability across infrastructure, applications, and networks.
How much does Azure Monitor cost?
Azure Monitor pricing is primarily based on data ingestion and retention. Log Analytics charges approximately $2.76/GB ingested (pay-as-you-go) with the first 5 GB/month free. Commitment tiers provide discounts: 100 GB/day tier is ~$1.70/GB (38% savings). Data retention is free for the first 31 days; additional retention costs $0.10/GB/month. Application Insights charges the same Log Analytics ingestion rates since it stores data in a Log Analytics workspace. Alert rules cost $0.10-$1.50/rule/month depending on type. For most enterprise deployments (ingesting 50-200 GB/day), expect $4,000-$16,000/month. Using commitment tiers and data collection rules to filter unnecessary data reduces costs by 30-50%.
What is the difference between Azure Monitor Logs and Azure Monitor Metrics?
Azure Monitor Metrics are lightweight, time-series numerical values collected at regular intervals (typically 1-minute granularity). They are stored in a time-series database optimized for fast queries and real-time dashboards. Examples: CPU percentage, memory usage, request count, response time. Metrics are ideal for real-time alerting and dashboards. Azure Monitor Logs are detailed, structured or semi-structured records stored in Log Analytics workspaces. They include event logs, trace logs, performance counters, custom application logs, and security events. Logs are queried using KQL (Kusto Query Language) and support complex analysis, correlation, and long-term retention. Use Metrics for real-time monitoring and simple threshold alerts; use Logs for investigation, troubleshooting, and complex analytics.
What is KQL (Kusto Query Language) and why is it important?
KQL (Kusto Query Language) is the query language for Azure Monitor Logs, Microsoft Sentinel, Microsoft Defender, Azure Data Explorer, and Microsoft Fabric Real-Time Analytics. It is a read-only language optimized for querying large volumes of semi-structured telemetry data. KQL uses a pipe-based syntax where data flows through a series of operators: source table | filter | transform | aggregate | sort | render. For example: "Heartbeat | where TimeGenerated > ago(1h) | summarize count() by Computer | order by count_ desc" shows which VMs have reported in the last hour. KQL proficiency is essential for any Azure operations or security team because it is the query language for all Azure observability and security platforms.
How do I set up alerting in Azure Monitor?
Azure Monitor alerts consist of three components: the alert rule (defines the condition), the action group (defines who gets notified and what automated actions to take), and the alert processing rule (optional, controls notification routing and suppression). To create an alert: define the scope (resource, resource group, or subscription), select the signal type (metric, log, or activity log), configure the condition (threshold, frequency, aggregation), attach an action group (email, SMS, webhook, Logic App, Azure Function, ITSM), and set the severity (0-Critical through 4-Informational). Best practice: use log-based alerts for complex conditions (KQL queries) and metric-based alerts for simple threshold monitoring.
What is the recommended Log Analytics workspace architecture for enterprise?
For most enterprises, a centralized Log Analytics workspace architecture is recommended: one primary workspace per Azure region for all resources in that region, with Azure Lighthouse for multi-tenant scenarios. This simplifies cross-resource correlation, reduces management overhead, and enables cost optimization through commitment tiers. Exceptions that warrant separate workspaces include: compliance requirements that mandate data sovereignty (different workspaces per country/region), security isolation (a dedicated workspace for security logs accessed only by the SOC team), and cost allocation (separate workspaces for development and production when chargeback is required). Avoid per-resource or per-application workspaces — this approach creates data silos and prevents cross-resource correlation.