Azure/login is slow to execute with Hosted Compute Networking

[CpuID]

Possibly related to any of these issues (some open, some closed):

#456
#449
actions/runner-images#10110

---

We're running Azure/login@v2 on Github-hosted Actions Runners using the Github provided/default Ubuntu 24.04 image.

It seems when using Hosted Compute Networking, the Azure/login@v2 action takes a long time to complete.

My first theory was "maybe there's an NSG rule doing something weird", but then when I saw just running an az version beforehand speeds things up so much, I feel like something else is at play?

Added context:

• for Hosted Compute Networking we're using the default NSG rules listed in here: https://docs.github.com/en/enterprise-cloud@latest/admin/configuring-settings/configuring-private-networking-for-hosted-compute-products/configuring-private-networking-for-github-hosted-runners-in-your-enterprise#prerequisites The important part: we don't have a "block all Internet" NSG rule
• we're using allow-no-subscriptions: true as this setup is relying on the Flexible Federated Identity Credentials (Preview) which isn't supported on User-Assigned Managed Identities yet, only App Registrations.

Any additional debugging steps that can be suggested would be appreciated - I can run further tests to try isolate the root cause with maintainer input.

---

Examples:

• just running login, nothing else, 43-49 seconds
  • note the runs-on

on:
  push:
    branches:
      - main

permissions:
  id-token: write

jobs:
  azure-cli-startup:
    runs-on: ubuntu-latest-2core--azure-vnet
    steps:
      - name: Azure login
        uses: Azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          allow-no-subscriptions: true

• if we run az version before a login, that drops to 7 seconds aggregate, 4 seconds for az version and 3 seconds for the login 🤯

on:
  push:
    branches:
      - main

permissions:
  id-token: write

jobs:
  azure-cli-startup:
    runs-on: ubuntu-latest-2core--azure-vnet
    steps:
      - name: Run az version
        run: |
          az version

      - name: Azure login
        uses: Azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          allow-no-subscriptions: true

• if we run this without Hosted Compute Networking, its a 5 second execution time
  • note the runs-on

on:
  push:
    branches:
      - main

permissions:
  id-token: write

jobs:
  azure-cli-startup:
    runs-on: ubuntu-latest
    steps:
      - name: Azure login
        uses: Azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          allow-no-subscriptions: true

[MoChilia]

Hi Nathan Sullivan (@CpuID), I’ve also noticed that running an az command for the first time can be slow. This is likely because the Python interpreter and necessary libraries are being loaded into RAM from disk, which is unavoidable on the initial run.

Regarding Hosted Compute Networking, I’m not sure why running az version outside of the Azure login action would affect performance so significantly. As az version doesn’t make any network requests, any networking configuration should only impact the az login command. Is the delay you’re seeing consistent, or is it a random behavior?

[CpuID]

Hi Shiying Chen (@MoChilia)

Consistent, not random.

Thanks for investigating 👍 lmk if you need any additional debug info, I'm in UTC+10 btw

[CpuID]

Actually I just did another test, and az version took 49 seconds to run 😢 so that backs up your theory re:

This is likely because the Python interpreter and necessary libraries are being loaded into RAM from disk, which is unavoidable on the initial run.

I feel like there has to be some better way to start the Azure CLI than having to wait 49 seconds on the start of every job though? 🤔 that's a fairly big overhead... I know the AWS CLI is instant to start FWIW :)

[CpuID]

Is the delay you’re seeing consistent, or is it a random behavior?

Aha, so the faster runtimes are probably when a runner is reused for a future workflow job, whereas the first time a runner runs an az command that's when it would be slow...

[CpuID]

This is likely because the Python interpreter and necessary libraries are being loaded into RAM from disk, which is unavoidable on the initial run.

Filed Azure/azure-cli#31541 upstream for the Azure CLI to try see if we can solve the underlying issue there :) More debug info provided over there too

[CpuID]

I also think #531 might be a possible speed workaround here :)

Feel free to refactor it to be less .exec and more native JS/TS of course...

The key is I think az version loads all underlying command modules, whereas you don't need them all just to get the CLI version...

[epot]

FTR it can be way worse than 1min in my experience. Here is a case where it took... 8 min for instance:

[epot]

Is there an update on this? It's still terribly slow on our end.

[mikem-of]

Same issue here: az login runs just under a minute for us...