Azure Monitor: The Enterprise Guide to Full-Stack Observability, Log Analytics, and KQL
You cannot manage what you cannot observe. Azure Monitor provides a full observability stack for cloud-native and hybrid enterprise environments. It gathers:
- Metrics
- Logs
- Traces
- Dependencies
This data is collected from every layer of the infrastructure and application stack.
This guide covers:
- Enterprise Azure Monitor architecture
- Log Analytics workspace design
- Application Insights for APM
- Alert strategy
- KQL query patterns
- Azure Workbooks
- Cost optimization
All insights are based on EPC Group's experience across 200+ Azure enterprise deployments.
Azure Monitor Enterprise Observability Guide 2026
Azure Monitor is Microsoft's unified observability platform for cloud and hybrid environments. It gathers metrics, logs, and traces using Log Analytics, Application Insights, Alerts, and Workbooks.
EPC Group integrates Azure Monitor into every enterprise Azure deployment. We view observability as a core component, not an afterthought.
So far, we have successfully completed over 200 enterprise Azure Monitor deployments.
Key facts
- Azure Monitor pricing: Log Analytics charges ~$2.76/GB ingested (pay-as-you-go). First 5 GB/month free. 100 GB/day commitment tier: ~$1.70/GB (38% savings).
- Data retention: free for first 31 days. Additional retention: $0.10/GB/month. Long-term archive: $0.02/GB/month.
- Enterprise deployments ingesting 50–200 GB/day: expect $4,000–$16,000/month.
- Commitment tiers and DCR filtering reduce monitoring costs by 30–50%.
- EPC Group has migrated enterprises from 20+ fragmented workspaces to 2–3 consolidated workspaces, reducing monitoring costs by 40%.
- Alert rules: $0.10–$1.50/rule/month depending on type.
The Azure Monitor Observability Stack
Enterprise observability requires three pillars:
- Metrics — what is happening right now (lightweight, time-series numeric values)
- Logs — what happened and why (detailed structured records in Log Analytics)
- Traces — how requests flow through distributed systems (Application Insights)
Azure Monitor provides all three pillars in a unified platform with a common query language (KQL), a common alerting engine, and a common visualization layer.
Azure Monitor includes these core components:
- Log Analytics — log collection and KQL querying
- Application Insights — application performance monitoring (APM)
- Metrics Explorer — real-time metric visualization
- Alerts — proactive notification and automated response
- Workbooks — interactive dashboards and reports
- Azure Monitor Agent — data collection from VMs
- Network Insights — network monitoring and diagnostics
Log Analytics: Architecture and Workspace Design
Log Analytics workspaces are the primary data stores for Azure Monitor Logs. All log data flows into these workspaces. In these workspaces, the data is indexed and can be queried using KQL. It is retained for the specified duration.
The most common mistake is creating too many workspaces — per application, per team, or per subscription. This creates data silos and disqualifies commitment tier pricing.
EPC Group recommends a centralized architecture:
- Centralized workspace (recommended): One workspace per Azure region. Supports cross-resource correlation, simplified RBAC, and commitment tier pricing. Most enterprises use 1–2 workspaces.
- Dedicated security workspace: A separate workspace for Microsoft Sentinel accessible only by the SOC team. Provides data isolation and independent retention policies (security data often needs 1–2 years for compliance).
- Workspace per environment (dev/staging/prod): Useful for cost chargeback by environment. Sacrifices cross-environment correlation — use only when chargeback is required.
Avoid per-resource or per-application workspaces — they create data silos and prevent cross-resource correlation.
Data Retention and Archiving
Azure Monitor Log Analytics provides three retention tiers:
- Interactive retention: Immediately queryable via KQL. First 31 days free, then $0.10/GB/month. Configure at the table level.
- Long-term retention (archive): Queryable via restore or search jobs. $0.02/GB/month. Best for compliance data you need to keep but rarely query.
- Export to storage: Continuous export to Azure Storage for indefinite retention at blob storage pricing.
Configure table-level retention policies to manage data effectively. High-value tables, such as SecurityEvent and AppExceptions, may require 365 days of interactive retention.
In contrast, high-volume tables like Heartbeat and Perf may only need 31 days of interactive retention, along with 90 days of archive.
Data Collection Rules and Azure Monitor Agent
Data Collection Rules (DCRs) manage the collection, transformation, and destination of data. They replace older workspace-based agent configurations. DCRs provide detailed control over data ingestion.
The Azure Monitor Agent (AMA) is the unified agent for Windows and Linux VMs. It replaces several older agents, including:
- Log Analytics Agent (MMA/OMS)
- Diagnostics extension
- Telegraf agent
AMA offers several key features:
- Uses DCRs for configuration
- Supports multi-homing (multiple workspaces)
- Utilizes managed identity authentication
DCRs enable KQL-based transformations to filter or change data before it enters the workspace. This process helps in:
- Removing unnecessary columns
- Filtering out noisy logs
These actions can reduce ingestion volume by 30–60% while maintaining operational value.
Application Insights: APM
Application Insights is the APM component of Azure Monitor. It collects telemetry from web applications: requests, dependencies, exceptions, traces, page views, and custom events.
Three instrumentation options:
- Auto-instrumentation (recommended for Azure PaaS): Enable on Azure App Service, Azure Functions, or AKS without code changes. Supported runtimes: .NET, Java, Node.js, Python.
- SDK instrumentation: Add the Application Insights SDK to your application code for maximum control. Enables custom telemetry (TrackEvent, TrackMetric, TrackTrace) and complex dependency tracking.
- OpenTelemetry: Use the Azure Monitor OpenTelemetry Exporter for multi-cloud or vendor-neutral observability strategies.
Key features: Application Map, Transaction Search, Failures and Performance dashboards, Smart Detection (ML-based anomaly alerts), and Availability tests (synthetic monitoring).
KQL: The Query Language for Azure Observability
KQL (Kusto Query Language) is the query language used for several Microsoft products. These include Log Analytics, Application Insights, Microsoft Sentinel, and Microsoft Defender.
KQL employs a pipe-based syntax. This allows data to flow through a series of operators:
source table | filter | transform | aggregate | sort | render
Example — show which VMs reported in the last hour:
Heartbeat | where TimeGenerated > ago(1h) | summarize count() by Computer | order by count_ desc
KQL proficiency is a core skill for any Azure operations or security team. It is the query language for all Azure observability and security platforms.
Azure Monitor Alerts: Key Concepts
An Azure Monitor alert has three components:
- Alert rule — defines the condition (scope, signal type, threshold, frequency)
- Action group — defines who gets notified (email, SMS, webhook, Logic App, Azure Function, ITSM) and what automated actions to take
- Alert processing rule — optional; controls notification routing and suppression during maintenance windows
Alert severity levels: 0-Critical, 1-Error, 2-Warning, 3-Informational, 4-Verbose.
Best practices:
- Alert on symptoms (response time exceeds SLA, error rate spikes) not causes (CPU high) alone.
- Use dynamic thresholds — ML-based baselines eliminate manual threshold maintenance.
- Suppress during planned maintenance windows using alert processing rules.
- Connect action groups to Azure Automation runbooks for auto-remediation.
Cost Optimization for Azure Monitor
Azure Monitor costs grow with data volume. EPC Group implements these controls on every enterprise deployment:
- Commitment tiers: If ingesting 100+ GB/day, switch from pay-as-you-go to a commitment tier. 100 GB/day: 38% discount. 500 GB/day: 50% discount.
- DCR filtering: Drop verbose informational logs and filter performance counters to only the metrics you monitor. Reduces ingestion by 30–60%.
- Table-level retention: Configure shorter interactive retention for high-volume, low-value tables (Heartbeat, Perf). Archive data to long-term tier ($0.02 vs. $0.10/GB/month).
- Application Insights sampling: Adaptive sampling automatically adjusts the sampling rate based on telemetry volume. Reduces cost on high-traffic applications with no material accuracy loss.
- Basic logs tier: For high-volume tables rarely queried (verbose diagnostic logs), use Basic Logs at $0.50/GB vs. $2.76/GB for Analytics tier. Limited KQL, 8-day retention, ideal for troubleshooting-only data.
Frequently Asked Questions
What is Azure Monitor and what does it include?
Azure Monitor is Microsoft's all-in-one observability platform for Azure and hybrid environments. It offers several key features:
- Log Analytics
- Application Insights
- Metrics Explorer
- Alerts
- Workbooks
- Azure Monitor Agent
- Network Insights
These tools work together to provide complete observability across infrastructure, applications, and networks.
How much does Azure Monitor cost?
Log Analytics uses a pay-as-you-go pricing model. It charges about $2.76 per GB ingested. The first 5 GB each month is free.
Additionally, data retention is free for the first 31 days.
Alert rules cost between $0.10 and $1.50 each per month. For most enterprise deployments, the monthly costs vary based on data size:
- 50 GB daily: approximately $4,000
- 200 GB daily: up to $16,000
You can lower costs by:
- 30 to 50% with commitment tiers
- DCR filtering
What is the difference between Azure Monitor Logs and Metrics?
Metrics are simple, numeric time-series values. Examples include CPU % and request count. They are stored in a fast time-series database.
Logs are detailed records that are structured and stored in Log Analytics. You can query them using KQL.
Use Metrics for:
- Real-time alerting
Use Logs for:
- Investigation
- Troubleshooting
- Complex analytics
What is KQL and why is it important?
KQL (Kusto Query Language) is the query language for Log Analytics, Sentinel, Defender, and Azure Data Explorer. It is built for analyzing large volumes of semi-structured telemetry data.
Mastering KQL is essential for any Azure operations or security team.
What is the recommended Log Analytics workspace architecture?
Each Azure region should have one main workspace for all resources in that region. Additionally, you should create a dedicated security workspace for Microsoft Sentinel data.
Avoid creating workspaces for individual applications or resources. This practice can lead to silos and make it difficult to correlate data across resources.
Partner with EPC Group
EPC Group was a Microsoft Gold Partner from 2016 until the program's retirement. We were the oldest partner in North America and are now a Microsoft Solutions Partner.
Our Azure cloud practice focuses on comprehensive observability. We cover:
- Log Analytics workspace architecture
- Data collection optimization
- Custom workbook development
- Alert strategy design
- Ongoing monitoring operations
Frequently Asked Questions
What is Azure Monitor and what does it include?
Azure Monitor is the comprehensive observability platform for Azure and hybrid environments. It collects, analyzes, and acts on telemetry data from cloud and on-premises resources. Azure Monitor includes: Log Analytics (log collection and KQL querying), Application Insights (application performance monitoring/APM), Metrics Explorer (real-time metric visualization), Alerts (proactive notification and automated response), Workbooks (interactive dashboards and reports), Azure Monitor Agent (data collection from VMs), and Network Insights (network monitoring and diagnostics). Together, these components provide full-stack observability across infrastructure, applications, and networks.
How much does Azure Monitor cost?
Azure Monitor pricing is primarily based on data ingestion and retention. Log Analytics charges approximately $2.76/GB ingested (pay-as-you-go) with the first 5 GB/month free. Commitment tiers provide discounts: 100 GB/day tier is ~$1.70/GB (38% savings). Data retention is free for the first 31 days; additional retention costs $0.10/GB/month. Application Insights charges the same Log Analytics ingestion rates since it stores data in a Log Analytics workspace. Alert rules cost $0.10-$1.50/rule/month depending on type. For most enterprise deployments (ingesting 50-200 GB/day), expect $4,000-$16,000/month. Using commitment tiers and data collection rules to filter unnecessary data reduces costs by 30-50%.
What is the difference between Azure Monitor Logs and Azure Monitor Metrics?
Azure Monitor Metrics are lightweight, time-series numerical values collected at regular intervals (typically 1-minute granularity). They are stored in a time-series database optimized for fast queries and real-time dashboards. Examples: CPU percentage, memory usage, request count, response time. Metrics are ideal for real-time alerting and dashboards. Azure Monitor Logs are detailed, structured or semi-structured records stored in Log Analytics workspaces. They include event logs, trace logs, performance counters, custom application logs, and security events. Logs are queried using KQL (Kusto Query Language) and support complex analysis, correlation, and long-term retention. Use Metrics for real-time monitoring and simple threshold alerts; use Logs for investigation, troubleshooting, and complex analytics.
What is KQL (Kusto Query Language) and why is it important?
KQL (Kusto Query Language) is the query language for Azure Monitor Logs, Microsoft Sentinel, Microsoft Defender, Azure Data Explorer, and Microsoft Fabric Real-Time Analytics. It is a read-only language optimized for querying large volumes of semi-structured telemetry data. KQL uses a pipe-based syntax where data flows through a series of operators: source table | filter | transform | aggregate | sort | render. For example: "Heartbeat | where TimeGenerated > ago(1h) | summarize count() by Computer | order by count_ desc" shows which VMs have reported in the last hour. KQL proficiency is essential for any Azure operations or security team because it is the query language for all Azure observability and security platforms.
How do I set up alerting in Azure Monitor?
Azure Monitor alerts consist of three components: the alert rule (defines the condition), the action group (defines who gets notified and what automated actions to take), and the alert processing rule (optional, controls notification routing and suppression). To create an alert: define the scope (resource, resource group, or subscription), select the signal type (metric, log, or activity log), configure the condition (threshold, frequency, aggregation), attach an action group (email, SMS, webhook, Logic App, Azure Function, ITSM), and set the severity (0-Critical through 4-Informational). Best practice: use log-based alerts for complex conditions (KQL queries) and metric-based alerts for simple threshold monitoring.
What is the recommended Log Analytics workspace architecture for enterprise?
For most enterprises, a centralized Log Analytics workspace architecture is recommended: one primary workspace per Azure region for all resources in that region, with Azure Lighthouse for multi-tenant scenarios. This simplifies cross-resource correlation, reduces management overhead, and enables cost optimization through commitment tiers. Exceptions that warrant separate workspaces include: compliance requirements that mandate data sovereignty (different workspaces per country/region), security isolation (a dedicated workspace for security logs accessed only by the SOC team), and cost allocation (separate workspaces for development and production when chargeback is required). Avoid per-resource or per-application workspaces — this approach creates data silos and prevents cross-resource correlation.