Statêra is now a Salesforce- and QTC-focused consulting agency. An exciting announcement about our digital transformation practice — including Microsoft consulting services — is forthcoming. More info »

Moving Data to the Cloud Using AWS Snowball

As more and more companies realize the value of the cloud, the conversation quickly moves from should we use the cloud, to how do we get the initial data into the cloud to begin the migration? The AWS Snowball is the solution we used for one client, but this is by no means the only solution available for this task.

 

First thing’s first. What is AWS Snowball?

Before we get into how we used the Snowball, let us take a second to discuss what the Snowball is. A quick search will give the following result:

“AWS Snowball is a service that accelerates transferring large amounts of data into and out of AWS using physical storage devices, bypassing the Internet. There are many options for transferring your data into AWS. Snowball is intended for transferring large amounts of data.”

To sum up the AWS Snowball, it is a device that is about the size of the old-school computer tower.  It has optional 50tb, 80tb or 100tb storage and a small Amazon Fire tablet enclosed in the device to interface with the Snowball.  It features Cat-5 and SFP (10 GB) network connections.  Basically, the Snowball is a large commercial storage device used for transferring terabytes of data.

 

Where do you get it?

The device is ordered through the AWS management console.  The process is easy: Create a new snowball request and input the name and address so that the device can be delivered correctly. It should be mentioned that there are a finite number of these devices, and if they are all checked out, a wait list would be necessary.  Our last order took about ten (10) days to arrive at the office.

Once the device arrives, the “fun” starts.  We coordinated with our network team and prearranged a 10 GB switch to be installed where the device was going to be placed.   Once the device arrives, open up the side panels and use the provided power plug and SFP connector to connect to the right outlet and switch.

 

Turn it on

Power on the device, and the connection instructions will appear on the Fire tablet in the enclosure.   Once the device has powered up, go into the Amazon management console to get the manifest and access keys.  Follow the instructions that will be a guided tour on how to unlock the device for use.

 

Ready for data movement

Now that the device is wired up, powered on and unlocked, we are ready to copy data onto the device.  For our use case, we had 65tb of data to move, and that took a week to do.  One note here is, once the Snowball is unlocked we only have 30 days of time to move data to the device before it locks out on its own.  AWS put this restriction/security measure on the device, and it can not be overridden.

The commands are relatively simple for the standard Snowball.  The commands are like bash on a standard *nix system but the start with the word snowball.

 

To make a directory on the snowball:

We type snowball mkdir [OPTIONS…] “s3://SOMEBUCKET/SOMEPATH.”  Let us say we need a directory named “media” in our S3 bucket called “statera-archives.” To copy all media data and files to the cloud, the command would be “snowball mkdir s3://statera-archives/media.”

 

To copy the data from a network drive:

We have mounted on a computer a path named /server1/archives/media. The command to copy the data would be “snowball cp –recursive /server1/archives/media  s3://statera-archives/media”.

For a full list of available commands for the Snowball, please see:

https://docs.aws.amazon.com/snowball/latest/ug/using-client-commands.html.

 

What next?

Once the files have all been copied to the Snowball, the device must be locked to secure it for transport.  This is done mainly so that no one can access it until it is returned to AWS.  To secure the device, simply type “snowball stop” at the command line. The stop command locks the device after a few minutes.  The screen on the front of the device will turn into a shipping label. After the device has returned to AWS, the Snowball team unlocks the encrypted data and moves it to the AWS S3 bucket that was initially requested in the onset of the order.

 

Finally

As a final step, AWS will send the Snowball requestor an email stating the data has been put into S3, and the ticket is closed. The “media” data is now in its new forever home in S3.

In summary, the AWS snowball is a cost-effective, secure and reasonably easy solution to get data to the cloud.  If you have time constraints, this may not work in every scenario.  However, if time is not an issue or there is plenty of lead time, there is no better way to get bulk data to the cloud.

 

About Riley Webb

Riley is a Cloud Architect with Statera. He moved from Houston, Texas to Denver in 2011. He went to Southern Methodist University – Applied Sciences and Technology Campus (Houston). After finishing his MS Certified Systems Engineer (MCSE) certifications in Windows NT 3.51 and NT 4.0, he reached the Microsoft Certified Trainer (MCT) level and instructed Microsoft engineering classes at SMU. Through the years Riley has worked Security and Systems Engineering positions in oil & gas, manufacturing, financial, healthcare, telecommunications, and media companies.