Azure DevOps Service security monitoring using Azure Sentinel

by: Ronald Prasad, Microsoft Services Principal

With cyberthreats on the rise, teams that build and operate applications are facing new, strong challenges every day and calls for the adoption of new concepts like DevSecOps.

DevSecOps, sometimes called Secure DevOps, builds on the principles of DevOps but adds an increased focus on security on and around your development environment. With Secure DevOps, security becomes a central part of the entire SDLC process of the application. We call this concept “shift-left security.” Shift-left security brings a focus on security earlier in the development process. This also includes incorporating security monitoring and governance at every stage of planning, development, and testing.

The Security Features of Azure DevOps

Azure DevOps provides developer services for support teams to plan work, collaborate on code development, and build and deploy applications. The service has many built-in security features like the capabilities to generate audit logs. Audit logs are created when a user or service identity within the Azure DevOps organization edits the state of an artifact.

Audit events can be any of the following occurrences:

• permissions changes
• deleted resources
• branch policy changes
• accessing the auditing feature
• and much more

These logs can be streamed directly to an Azure Log Analytics Workspace which has Azure Sentinel connected. Setting up the stream is very straight forward, and Microsoft has provided so awesome documentation on this Create audit streaming for Azure DevOps – Azure DevOps | Microsoft Docs

Once you get these logs streaming into the workspace, you will need to enable built-in Azure Sentinel Analytics Rules (detection rules) for Azure DevOps. Below are the 15 rules currently available:

• Azure DevOps Agent Pool Created Then Deleted (High)

As well as adding build agents to an existing pool to execute malicious activity within a pipeline, an attacker could create a completely new agent pool and use this for execution. Azure DevOps allows for the creation of agent pools with Azure-hosted infrastructure or self-hosted infrastructure. Given the additional customizability of self-hosted agents, this detection focuses on the creation of new self-hosted pools. To further reduce false-positive rates, the detection looks for pools created and deleted relatively quickly (within 7 days by default), as an attacker is likely to remove a malicious pool once used to reduce/remove evidence of their activity.

• Azure DevOps Personal Access Token (PAT) Misuse (High)

This Alert detects whenever a PAT is used in ways that PATs are not normally used. This may require Allow listing and baselining[GM1] . [GM2] Reference – https://docs.microsoft.com/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page Use this query for baselining: AzureDevOpsAuditing | distinct OperationName

• Azure DevOps Audit Stream Disabled (High)

Azure DevOps allows for audit logs to be streamed to external storage solutions such as SIEM solutions. An attacker looking to hide malicious Azure DevOps activity from defenders may look to disable data streams before conducting activity and then re-enabling them after (as not to raise data threshold-based alarms). Looking for disabled audit streams can identify this activity, and due to the nature of the action it’s unlikely to have a high false-positive rate.

• Azure DevOps – PAT Used with Browser (Medium)

Personal Access Tokens (PATs) are used as an alternate password to authenticate into Azure DevOps. PATs are intended for programmatic access for use in code or applications. Given this, they can be prone to attacker theft if not adequately secured. This query looks for the use of a PAT in authentication but from a User Agent indicating a browser. This is not normal activity and could be an indicator of an attacker using a stolen PAT.

• Azure DevOps Pipeline Created and Deleted on the Same Day (Medium)

An attacker with access to Azure DevOps could create a pipeline to inject artifacts used by other pipelines or create a malicious software build that looks legitimate by using a pipeline that incorporates legitimate elements. An attacker would also likely want to cover their tracks once conducting such activity. This query looks for Pipelines created and deleted within the same day; this is unlikely to be legitimate user activity in the majority of cases.

• Azure DevOps Service Connection Addition/Abuse – Historic Allow List (Medium)

This detection builds an allow list of historic service connection use by Builds and Releases and compares to recent history, flagging growth of service connection use in non-manually allow listed, non-historically allow listed Build/Release runs. This is to determine if someone is hijacking a build/release and adding many service connections to abuse or dump credentials from service connections.

• Azure DevOps – Variable Secret Not Secured (Medium)

Credentials used in the build process may be stored as Azure DevOps variables. To secure these variables they should be stored in KeyVault or marked as secrets. This detection looks for new variables added with names that suggest they are credentials but where they are not set as Secrets or stored in KeyVault.

• New PA, PCA, or PCAS added to Azure DevOps (Medium)

For an attacker to be able to conduct many potential attacks against Azure DevOps they will need to gain elevated permissions. This detection looks for users being granted key administrative permissions. If the principle of least privilege is applied, the number of users granted these permissions should be small. Note that permissions can also be granted via Azure AD groups and monitoring of these should also be conducted.

• Azure DevOps Service Connection Abuse (Medium)

This flags builds/releases that use a large number of service connections if they aren’t manually Allow listed. This is to determine if someone is hijacking a build/release and adding many service connections in order to abuse or dump credentials from service connections.

• External Upstream Source Added to Azure DevOps Feed (Medium)

This detection looks for new external sources added to an Azure DevOps feed. An Allow list can be customized to explicitly allow known good sources. An attacker could look to add a malicious feed to inject malicious packages into a build pipeline.

• Azure DevOps Pull Request Policy Bypassing – Historic Allow list (Medium)

This detection builds an Allow list of historic PR policy bypasses and compares to recent history, flagging a non-manually Allow listed, non-historic pull request bypass.

• Azure DevOps Pipeline modified by a New User (Medium)

Several potential pipeline steps could be modified by an attacker to inject malicious code into the build cycle. A likely attacker path is a modification to an existing pipeline that they have access to. This detection looks for users modifying a pipeline when they have not previously been observed modifying or creating that pipeline before. This query also joins events with data to Azure AD Identity Protection (AAD IdP) to show if the user conducting the action has an associated AAD IdP alerts, you can also choose to filter this detection to only alert when the user also has AAD IdP alerts associated with them.

• ADO Build Variable Modified by New User (Medium)

Variables can be configured and used at any stage of the build process in Azure DevOps to inject values. An attacker with the required permissions could modify or add to these variables to conduct malicious activity such as changing paths or remote endpoints called during the build. As variables are often changed by users just detecting these changes would have a high false-positive rate. This detection looks for modifications to variable groups where that user has not been observed modifying them before.

• Azure DevOps Administrator Group Monitoring (Low)

This detection monitors for additions to project or project collection administration groups in an Azure DevOps Organization

• Azure DevOps – Retention Reduced to Zero (Low)

AzureDevOps retains items such as run records and produced artifacts for a configurable amount of time. An attacker looking to reduce the footprint left by their malicious activity may look to reduce the retention time for artifacts and runs to 0.

• Azure DevOps – New Extension Added (Low)

Extensions added additional features to Azure DevOps. An attacker could use a malicious extension to conduct malicious activity. This query looks for new extensions that are not from a configurable list of approved publishers

After these rules are enabled, Azure Sentinel will create incidents that your SOC can rapidly investigate and respond to any threats and attacks against your organization’s development environment. Following these rules will help you adopt a secure DevOps approach to application development.