Search This Blog

Saturday, July 19, 2014

SMB File Share for Azure Virtual Machine Using Azure Files Services

 

In Azure Iaas, we can attach multiple disk into VM with each disk up to 1TB depend on your Tier choice.

Here is a summary sizes of virtual machines depend on Tier

Basic Tier

Size CPU cores Memory Max data disk (1 TB each) Max IOPS (300 per disk)
A0 Shared (0.25) 768 GB 1 1 X 300
A1 1 1.75 GB 2 2 X 300
A2 2 3.5 GB 4 4 X 300
A3 4 7 GB 8 8 X 300
A4 8 14 GB 16 16 X 300

Standard Tier

Size CPU cores Memory Max data disk (1 TB each) Max IOPS (500 per disk)
A0 Shared (0.25) 768 GB 1 1 X 500
A1 1 1.75 GB 2 2 X 500
A2 2 3.5 GB 4 4 X 500
A3 4 7 GB 8 8 X 500
A4 8 14 GB 16 16 X 500
A5 2 14 GB 4 4 X 500
A6 4 28 GB 8 8 X 500
A7 8 56 GB 16 16 X 500
A8 8 56 GB 16 16 X 500
A9 16 112 GB 16 16 X 500

Sizing information captured from here.

Let say for an example:- You would like to create a file share and hold data more than 1TB (example 4TB), then you may encountered a problem of hitting size limit as per disk maximum size is 1TB.

There are several way available that we can think of to solve this problem.

Option 1:- Create software RAID on dynamic disk.

By using Windows operating system, we can enabling disk striping (RAID 0), disk mirror (RAID 1) and parity set (RAID 5). To achieve this, we need to select A2 / A3 Size. That would provide us 4 disk (using A2) and 8 disk (using A3). Once create the dynamic disk with volumes, it would provide a volume with enough size to hold your 4TB data.

However based on previous testing on IOPS, the performance on using software RAID is not great.

Example:-

a) Use Basic Tier and test IOPS using Iometer- Test result was 280+ IOPS

b) Use software raid – Test result was 180+. It seem the performance was bad.

* anyway, this test was subjective and varies *

Option 2:- Use Azure Files Services

Azure Files allows Virtual Machines (VMs) in an Azure Data Center to mount a shared file system using the SMB protocol. These VMs will then be able to access the file system using standard Windows file APIs (CreateFile, ReadFile, WriteFile, etc.). Many VMs (or Platform as a Service (PaaS) roles) can attach to these file systems concurrently, allowing you to share persistent data easily between various roles and instances.

It’s using SMB 2.1 protocol and compatible to access from Linux System and Windows in VM on Azure.

VMs on Azure must be in same region with the Azure Storage to use for Azure Files.

The following is the scalability of Azure Files:-

  • Up to 5TB per share
  • A file can be up to 1 TB
  • Up to 1000 IOPS (of size 8 KB per share)
  • Up to 60 Mbps per share of data transfer for large IOs

Configuration to Setup

1. Create a new Storage Account

2. Use Windows Azure Powershell to create Azure Files

# import module and create a context for account and key
import-module .\AzureStorageFile.psd1
$ctx=New-AzureStorageContext ms4uteststorage01 7JjWiR04dUAhf5jSRRnswyL5LCfAuYgjIOAd/s6xKqgmHfOLFPF2oPL3vZW9gT3mbfncpWBqxfrbB5sD0/XXX
 
# create a new share
$s = New-AzureStorageShare ms4ushare -Context $ctx
 
# create a directory in the test share just created
New-AzureStorageDirectory -Share $s -Path testdir
 
# upload a local file to the testdir directory just created
Set-AzureStorageFileContent -Share $s -Source D:\Cert\DataEncryptionCertificate.pfx -Path testdir
 
# list out the files and subdirectories in a directory
Get-AzureStorageFile -Share $s -Path testdir

af1

New “Files” services available on Azure Storage Account.

Format:-

# import module and create a context for account and key
import-module .\AzureStorageFile.psd1
$ctx=New-AzureStorageContext <account name> <account key>
# create a new share
$s = New-AzureStorageShare <share name> -Context $ctx
# create a directory in the test share just created
New-AzureStorageDirectory -Share $s -Path testdir
# upload a local file to the testdir directory just created
Set-AzureStorageFileContent -Share $s -Source D:\upload\testfile.txt -Path testdir
PS-AzureFile

Script taken from here.


3. Go to your VM (same region), map Azure Files by using net use and start copying files into Azure Files.





net use z: \\ms4uteststorage01.file.core.windows.net\ms4ushare /u:ms4uteststorage01 7JjWiR04dUAhf5jSRRnswyL5LCfAuYgjIOAd/s6xKqgmHfOLFPF2oPL3vZW9gT3mbfncpWBqxfrbB5sD0/XXX

Format:-


net use z: \\<account name>.file.core.windows.net\<share name> /u:<account name> <account key>


af3


The map drive has a maximum size of 5TB as displayed. Do take note that Azure Files services was able to access by multiple VMs as long it is on the same region.


Option 3:- Setup Storage Spaces


Storage Spaces is introduced on Windows Server 2012/2012 R2 which allow us to combine multiple disk to a pool and then create virtual disk. Since Azure Files (under Preview) had a limit of 5TB, Storage Spaces allow us to scale more higher TB (till 16 TB).


image


At this moment, we only can think of 3 options as listed above. If you have any solution/ways, please leave a comment and share with other readers.

2 comments:

  1. This post will talk about how I boiled down the information in the blog to connect my Azure VMs to this Azure File Server.online file storage

    ReplyDelete
  2. As of 2015-05-21, you still cannot have spaces anywhere in the path to the file you're uploading via Set-AzureStorageFileContent.

    Fails: Set-AzureStorageFileContent -Source 'c:\somePath\some file with spaces.txt' -Share $share -Path $pathInYourAzureStorageAccount

    Works: Set-AzureStorageFileContent -Source 'c:\somePath\somefilewithnospaces.txt' -Share $share -Path $pathInYourAzureStorageAccount

    Hope this helps others running into this issue.

    (Just turn your brain back to 1981, when 8.3 filenames were the norm and you won't encounter any errors).

    ReplyDelete