Upload File From URL To Azure Blob Storage Using Azure CLI With Bash And PowerShell

The Azure CLI az storage blob upload command can be used to upload a local file into Azure Blob Storage. However, there are times when it would be easier to upload a file straight from the HTTP(S) URL into Azure Blob Storage. Unfortunately the Azure CLI does not have a command for this specific scenario, it requires the file to be downloaded locally first. This article looks at the Bash and PowerShell commands to download a file from URL and upload it to Azure Blob Storage. Plus, if you run these commands inside the Azure Cloud Shell, then you don’t need to worry about your local network bandwidth speeds or limits if you’re uploading a large file into Azure Blob Storage.

Table of Contents

Uploading Files from URL to Azure Blob Storage

Let’s first look at the steps for uploading a file to Azure Blob Storage using the Azure CLI. Consider in this scenario for the file we want to upload we have a URL for the file as it’s hosted on the Internet somewhere. We want to take the HTTP(S) URL of the file and upload it to Azure Blob Storage.

Step 1: Download the File

Before we can upload the file to Azure Blob Storage, we first need to download it. This can be done using either the wget or curl commands in bash, or `Invoke-WebRequest` in PowerShell.

Here are examples of each of the bash commands:

wget "https://build5nines.com/file.zip" -O "localfile.zip"curl "https://build5nines.com/file.zip" --output "localfile.zip"

Here’s an example of the PowerShell command:

Step 2: Upload the File to Azure Blob Storage

Now that the file has been downloaded, we can use the Azure CLI to upload it to Azure Blob Storage using the az storage blob upload command.

Here’s an example:

az storage blob upload --account-name <storage-account-name> \ --account-key <storage-account-key> \ --container-name <container-name> \ --name <blob-name> \ --file "localfile.zip"

Be sure to set the command arguments appropriately for your Azure Blob Storage Account:

--account-name – The Name of the Azure Storage Account.
--account-key – The Azure Storage Account Key, so the Azure CLI can authenticate and have access to upload the blob.
--container-name – The Container name where the blob will be uploaded within the Azure Storage Account.
--name – The name of the Blob. This can be different than the source / local file name.
--file – The file path and name of the source / local file to be uploaded to blob storage.

Step 3: Delete the Local File

Once the downloaded file has been uploaded to Azure Blob Storage, the file saved on your local machine (or in Azure Cloud Shell if you’re using that) can now be deleted.

The bash rm command can be used to delete a file in bash:

Full Bash Script to Automate HTTP(S) URL File Upload to Azure Blob Storage

To use bash to automate the uploading of a file from it’s HTTP(S) URL into Azure Blob Storage you’ll want to write a script.

Here’s a full bash script that contains all the above commands with some variables for ease of use:

#!/bin/bash// file download informationurl="http://build5nines.com/file.zip"temp_file="localfile.zip"// Azure Blob Storage informationstorage_account_name="<storage-account-name>"storage_account_key="<storage-account-key"container_name="<container-name>"blob_name="<blob-name>"// download file with wget or curlwget "$url" -O "$temp_file"// curl "$url" --output "$temp_file"// upload file to Azure Blob Storageaz storage blob upload --account-name "$storage_account_name" \ --storage-account-key "$storage_account_key" \ --container-name "$container_name" \ --file "$temp_file" \ --name "$blob_name"// delete local file after uploadrm "$temp_file"

Be sure to replace the placeholders in this script with your own file URL, local file name, and Azure Blob Storage Account information.

Full PowerShell Script to Automate HTTP(S) URL File Upload to Azure Blob Storage

To use PowerShell to automate the uploading of a file from it’s HTTP(S) URL into Azure Blob Storage you’ll want to write a script.

Here’s a full PowerShell script that contains all the above commands with some variables for ease of use:

// file download information$url = "http://build5nines.com/file.zip"$temp_file = "localfile.zip"// Azure Blob Storage information$storage_account_name = "<storage-account-name>"$storage_account_key = "<storage-account-key"$container_name = "<container-name>"$blob_name = "<blob-name>"// download file with wget or curlInvoke-WebRequest -Uri $url -OutFile $temp_file// upload file to Azure Blob Storageaz storage blob upload --account-name $storage_account_name ` --storage-account-key $storage_account_key ` --container-name $container_name ` --file $temp_file ` --name $blob_name// delete local file after uploadRemove-Item -Path $temp_file

Be sure to replace the placeholders in this script with your own file URL, local file name, and Azure Blob Storage Account information.

Conclusion

In this article we looked at how to take a file with a known HTTP(S) URL and upload it into an Azure Blob Storage container. While the Azure CLI can’t take a URL and save it directly to Azure Blob Storage, we need to download the file locally first, then upload it to Azure Blob Storage using the Azure CLI. Also, if you run the commands within the Azure Cloud Shell, then you don’t need to do this locally and can save bandwidth of your local network.

Happy Azure CLI scripting!

About the Author:

Chris Pietschmann is a Microsoft MVP, HashiCorp Ambassador, and Microsoft Certified Trainer (MCT) with 20+ years of experience designing and building Cloud & Enterprise systems. He has worked with companies of all sizes from startups to large enterprises. He has a passion for technology and sharing what he learns with others to help enable them to learn faster and be more productive.

Upload File From URL To Azure Blob Storage Using Azure CLI With Bash And PowerShell | Build5Nines (2024)