agentleFS
SCStelzSCStelz/security-investigator/.github/skills/ca-policy-investigation/SKILL.md

ca-policy-investigation

Skill252 starsChanged 11 days agoMITView on GitHub

Use this skill when asked to investigate Conditional Access policy changes, sign-in failures related to CA policies (error codes 53000, 50074, 530032), or suspected policy bypass/manipulation. Triggers on keywords like "Conditional Access", "CA policy", "device compliance", "policy bypass", "53000", "50074", or when investigating why a user was blocked then suddenly unblocked. This skill provides forensic analysis of CA policy modifications correlated with sign-in failures.

The file

We split this skill into the rules it gives an agent

12 rules. About 0 are common, 9 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When investigating sign-in failures (error codes 53000, 50074) with CA policy correlation:

    line 35
  2. Distinctive:

    Verify logical consistency: Ask "does this make sense?"

    line 42
  3. Distinctive:

    Order: Always chronological (oldest first) to see cause/effect

    line 142
  4. Very specific:

    Look for "state" property in the JSON

    line 163
  5. Very specific:

    Document: enabled → disabled → enabledForReportingButNotEnforced

    line 165
  6. Very specific:

    Create chronological list: enabled → disabled → enabledForReportingButNotEnforced

    line 185
  7. Distinctive:

    No approval or change request documented

    line 240
  8. Distinctive:

    Verify user's device compliance status

    line 263
  9. Distinctive:

    Document incident for security review

    line 264
  10. Distinctive:

    Verify device meets compliance requirements

    line 268
  11. Distinctive:

    Create alert for policy state changes (enabled → disabled/report-only)

    line 273
  12. Distinctive:

    Consider PIM for Security Administrator role

    line 275

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Conditional Access Policy Investigation - Instructions

Purpose

This skill investigates Conditional Access (CA) policy changes in correlation with sign-in failures to detect:

  • Legitimate troubleshooting (authorized policy changes to resolve access issues)
  • Security control bypass (unauthorized policy modifications to circumvent blocks)
  • Privilege abuse (users with admin rights weakening security controls)

The key distinction is whether policy changes were authorized and necessary vs self-service bypass of security controls.


📑 TABLE OF CONTENTS

  1. Critical Investigation Rules - Mandatory workflow steps
  2. Common Error Codes - Sign-in failure reference
  3. CA Policy States - Understanding policy modes
  4. 5-Step Investigation Workflow - KQL queries and analysis
  5. Real-World Example - Complete walkthrough
  6. Critical Mistakes - What NOT to do
  7. Security Recommendations - Remediation guidance

Critical Investigation Rules

When investigating sign-in failures (error codes 53000, 50074) with CA policy correlation:

⚠️ MANDATORY STEPS - DO NOT SKIP:

  1. Query ALL CA policy changes in chronological order (±2 days from failure time)
  2. Parse policy state transitions from the JSON (enabled → disabled → report-only)
  3. Compare failure timeline with policy change timeline
  4. Verify logical consistency: Ask "does this make sense?"

Key Questions to Answer:

  • Was the user blocked BEFORE the policy change?
  • Did the policy change resolve the block?
  • Who initiated the policy change? (same user = suspicious)
  • What was the business justification?

Common Error Codes

Error CodeDescriptionTypical Cause
53000Device not compliantDevice not enrolled in Intune or failing compliance checks
50074Strong authentication requiredMFA not satisfied
50074User must enroll in MFAMFA not configured for user
530032Blocked by CA policyGeneric CA policy block
65001User consent requiredApplication consent needed
53003Access blocked by CA policyExplicit block condition met
70044Session expiredUser needs to re-authenticate

Error Code Investigation Priority

PriorityError CodesInvestigation Focus
HIGH53000, 530032, 53003Device compliance, CA policy blocks - check for policy manipulation
MEDIUM50074MFA requirements - check if MFA was bypassed
LOW65001, 70044Consent/session issues - usually not security-related

CA Policy State Meanings

StateWhat It MeansSecurity Impact
enabledPolicy actively enforcingBlocks non-compliant access (intended behavior)
disabledPolicy not enforcingSecurity control bypassed - all access allowed
enabledForReportingButNotEnforcedReport-only modeLogs violations but doesn't block - defeats purpose

State Transition Risk Assessment

TransitionRisk LevelInterpretation
enabled → disabledHIGHComplete security bypass
enabled → enabledForReportingButNotEnforcedMEDIUM-HIGHPartial bypass (monitoring only)
disabled → enabledLOWSecurity restored (good)
enabledForReportingButNotEnforced → enabledLOWSecurity strengthened (good)

Investigation Workflow Pattern

Step 1: Identify Sign-In Failures

Query sign-in failures with CA context:

// Get failures with CA context
union isfuzzy=true SigninLogs, AADNonInteractiveUserSignInLogs
| where TimeGenerated between (datetime(<START>) .. datetime(<END>))
| where UserPrincipalName =~ '<UPN>'
| where ResultType != '0'
| where AppDisplayName has '<APPLICATION>'  // e.g., "Visual Studio Code"
| project TimeGenerated, IPAddress, Location, ResultType, ResultDescription, 
    ConditionalAccessStatus, UserAgent
| order by TimeGenerated asc

What to Look For:

  • ResultType values: 53000, 50074, 530032, 53003
  • ConditionalAccessStatus: "failure", "notApplied"
  • Pattern of repeated failures followed by success

Step 2: Query ALL CA Policy Changes in Timeframe

CRITICAL: Query ±2 days from the first failure time

let failure_time = datetime(<FIRST_FAILURE_TIME>);
let start = failure_time - 2d;
let end = failure_time + 2d;
AuditLogs
| where TimeGenerated between (start .. end)
| where OperationName has_any ("Conditional Access", "policy")
| where Identity =~ '<UPN>' or tostring(InitiatedBy) has '<UPN>'
| extend InitiatorUPN = tostring(parse_json(InitiatedBy).user.userPrincipalName)
| extend InitiatorIPAddress = tostring(parse_json(InitiatedBy).user.ipAddress)
| extend TargetName = tostring(parse_json(TargetResources)[0].displayName)
| project TimeGenerated, OperationName, Result, InitiatorUPN, InitiatorIPAddress, 
    TargetName, CorrelationId
| order by TimeGenerated asc  // CRITICAL: Chronological order

Critical Analysis Points:

  • InitiatorUPN: Who made the change? Same user as blocked = suspicious
  • TargetName: Which policy was modified?
  • TimeGenerated: Did change occur AFTER sign-in failures?
  • Order: Always chronological (oldest first) to see cause/effect

Step 3: Parse Policy State Changes

For each CorrelationId from Step 2, get detailed changes:

// Get detailed property changes for a specific policy modification
AuditLogs
| where CorrelationId == "<CORRELATION_ID>"
| extend ModifiedProperties = parse_json(TargetResources)[0].modifiedProperties
| mv-expand ModifiedProperties
| extend PropertyName = tostring(ModifiedProperties.displayName)
| extend OldValue = tostring(ModifiedProperties.oldValue)
| extend NewValue = tostring(ModifiedProperties.newValue)
| project TimeGenerated, PropertyName, OldValue, NewValue

Key Properties to Extract:

  • Look for "state" property in the JSON
  • Parse OldValue and NewValue for state transitions
  • Document: enabled → disabled → enabledForReportingButNotEnforced

Step 4: Extract Policy State from JSON

Manual JSON Parsing:

The OldValue and NewValue fields contain JSON. Look for the "state" field:

{
  "state": "enabled",
  "conditions": { ... },
  "grantControls": { ... }
}

Build the Timeline:

  1. Extract "state" from each OldValue and NewValue
  2. Create chronological list: enabled → disabled → enabledForReportingButNotEnforced
  3. Correlate with sign-in failure timeline

Step 5: Security Assessment

Compare timelines and assess intent:

PatternInterpretationRisk Level
Failures → Policy DisabledUser bypassed security control to unblock selfHIGH - Privilege abuse
Failures → Policy Changed to Report-OnlyUser weakened security controlMEDIUM-HIGH - Partial bypass
Policy Disabled → Failures ContinueCached tokens (5-15 min propagation delay)INFO - Expected behavior
Policy Changed → No More FailuresPolicy change resolved issueContext-dependent - May be legitimate troubleshooting
Different user made changeAdmin assisted with access issueLOW - Likely legitimate (verify authorization)

Risk Escalation Criteria:

CriteriaRisk Level
Same user blocked AND made policy changeHIGH
Policy disabled within 30 minutes of first failureHIGH
Multiple policies modifiedHIGH
Change made outside business hoursMEDIUM-HIGH
No change request ticket/approvalMEDIUM-HIGH
Admin made change for blocked user (with ticket)LOW

Real-World Example Analysis

Scenario: User blocked by device compliance policy, then modifies policy

Timeline

TimeEventDetails
19:05Sign-in failureError 53000: device not compliant
19:06Sign-in failureError 53000: device not compliant
19:07Sign-in failureError 53000: device not compliant
19:09Policy changeenabled → disabled
19:09Policy changedisabled → enabledForReportingButNotEnforced
19:12Sign-in failureError 53000 (cached token)
19:14Sign-in successAccess granted

Analysis

  1. ✅ Policy was correctly blocking non-compliant device
    • Device compliance policy was enforcing as intended
    • User's device failed compliance checks (not enrolled or failing policy)
  2. 🚨 User disabled security control to bypass block
    • Same user who was blocked made the policy change
    • Change occurred within 4 minutes of repeated failures
    • No approval or change request documented
  3. ⚠️ User partially reversed by enabling report-only
    • Shows some awareness that disabling was too aggressive
    • But report-only still defeats the purpose (doesn't block)
  4. ❌ Report-only mode is NOT a valid security posture
    • Logs violations but allows non-compliant access
    • Creates false sense of security (policy "exists" but doesn't protect)

Assessment

FieldValue
Risk LevelMEDIUM-HIGH
FindingSelf-service security bypass using privileged role
Root CauseUser's device is non-compliant (not enrolled/failing compliance)
Policy ImpactDevice compliance checks now ineffective for all users

Recommendations

  1. Immediate Actions:
    • Restore policy to enabled state
    • Verify user's device compliance status
    • Document incident for security review
  2. User-Specific:
    • Enroll user's device in Intune
    • Verify device meets compliance requirements
    • Review if user needs Security Administrator role
  3. Process Improvements:
    • Implement approval workflow for CA policy changes
    • Create alert for policy state changes (enabled → disabled/report-only)
    • Review all users with permission to modify CA policies
    • Consider PIM for Security Administrator role

Critical Mistakes to Avoid

❌ DON'T:

MistakeWhy It's Wrong
Query only ONE policy change eventYou'll miss the sequence of changes
Read policy changes in reverse chronological orderConfuses cause/effect relationship
Assume policy was already disabledMust check starting state from OldValue
Skip verifying "does this make logical sense?"Disabled policies can't block users
Ignore the initiator identitySame user = suspicious, different admin = verify authorization
Focus only on final stateThe transition sequence reveals intent

✅ DO:

Best PracticeWhy It Matters
Query ALL policy changes in the timeframeComplete picture of modifications
Order chronologically (oldest first)See cause/effect sequence
Parse the full JSON for state transitionsExtract exact policy states
Cross-check: blocked user → policy must be enabledLogical consistency verification
Ask: "Why would user disable this policy?"Usually to bypass a legitimate block
Check if initiator had authorizationTicket, approval, documented reason

Security Recommendations

When CA Policy Changes Are Detected

1. Determine Legitimacy

  • Was the policy change authorized?
  • Was there a valid business reason?
  • Did the user have approval to make this change?
  • Is there a change request ticket?

2. Assess Impact

  • How many users affected by policy change?
  • What applications/resources are now unprotected?
  • How long was the policy disabled/weakened?
  • Are there compliance implications (regulatory requirements)?

3. Remediation Actions

ActionPriority
Restore policy to enabled state if unauthorizedIMMEDIATE
Investigate root cause (why was user blocked?)HIGH
Fix underlying issue (device compliance, MFA enrollment)HIGH
Review who has permission to modify CA policiesMEDIUM
Implement approval workflows for policy changesMEDIUM
Create alerts for future CA policy modificationsMEDIUM

4. Long-Term Improvements

ImprovementBenefit
Use PIM for Security Administrator roleRequires approval for elevated access
Implement CA policy change alertsReal-time notification of modifications
Require multi-admin approval for state changesPrevents single-person bypass
Document approved proceduresClear guidance for legitimate troubleshooting
Regular access reviewsEnsure only necessary users have CA admin rights

Prerequisites

Required MCP Servers

This skill requires:

  1. Microsoft Sentinel MCP - For KQL queries against SigninLogs and AuditLogs
    • mcp_sentinel-data_query_lake: Execute KQL queries
    • mcp_sentinel-data_search_tables: Discover table schemas

Required Data Sources

  • SigninLogs - Interactive sign-in events with CA status
  • AADNonInteractiveUserSignInLogs - Non-interactive sign-in events
  • AuditLogs - CA policy modification events

Required Permissions

To view CA policy changes in AuditLogs, ensure:

  • Sentinel workspace has AuditLogs ingestion enabled
  • User has appropriate RBAC to query the workspace

Integration with Other Skills

CA Policy Investigation often follows a user-investigation:

  1. Run user-investigation skill → Identifies sign-in failures
  2. Notice CA-related error codes → 53000, 50074, 530032
  3. Run ca-policy-investigation skill → Correlate failures with policy changes
  4. Document findings → Security assessment with remediation recommendations

Key Integration Points:

  • Sign-in failure data comes from user investigation
  • CA policy changes are NEW queries specific to this skill
  • Assessment combines timeline correlation with policy state analysis

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.