Introduction

At ShitOps, we understand the importance of backing up our data centers and cloud environments to prevent any catastrophic loss in case of a disaster. However, traditional backup methods using tape and disk are no longer adequate for today’s fast-paced environment. Our firm commitment to providing the best solutions led us to go beyond the simple three-two-one rule—wherein three copies of your data should be stored in two different formats, with one offsite copy—and develop an overengineered approach that will guarantee the safety of our clients’ data.

The Problem

Our challenge was to ensure our San Francisco-based data center, which contains critical client data, would always have a secure and fast backup system. Our current system relied on tape and disk backups, which were becoming increasingly outdated and unreliable. We needed to create a new solution that would enable us to backup quickly, securely, and efficiently from both our data center in San Francisco, as well as across multiple data centers globally.

The Solution

After months of careful research, planning, and trial and error, the experts at ShitOps have come up with an ingenious multidimensional football framework powered by VMware technology that addresses all the challenges posed by the need for a reliable backup system. Here is how it works:

First, we identified the need for a dedicated platform for storing and managing our data backups. The VMware vSphere platform was our natural choice, given its reliability and scalability features.

Next, we went ahead to create a sophisticated package that integrates all functionalities required for multidimensional football backup, build on top of VMware API. We named the package ShitOps Football Unicorn. Using a flowchart, we presented a high-level design of our unicorn below:

graph LR A[Backup Plan Initiated] --Step1: Schedule--> B((Backup Agent)) B --Step2: Scan and Tag Files--> C((Data Processor)) C --Step3: Multi-Tier Football Backup--> D{Backup Storage} D --Step4: Verify & Integrity Check --> E((Log Monitoring)) E --> |Success| F(Daily Report) E --> |Failure| G(Troubleshooting) G --> |Resolution Needed| J(Human Intervention Required) J -.send guidance.-> H(Support Team) H --> |resolve any issues| K(Backup Completed)

The above football unicorn provides a clear visualization of the data backup plan and how it works. We designed it to be scalable to any size organization and include multiple backup plans for different types of data.

We call this multidimensional approach "football" because it moves the ball forward by taking many steps in incremental and complementary progressions just like a football game.

Multidimensional Football Process Explained

Step 1: Scheduling the backup plan

The first step is scheduling the backup time on a daily, weekly, or monthly basis depending on the client’s requirements. The master backup server initiates the backup process and schedules it on the actual backup agents.

Step 2: Preparing files for backup

Files needing backup are scanned and tagged with their respective metadata, such as last modified date and unique reference numbers. The data processor is responsible for preparing these tagged files for multi-tier backup processing, including compression and encryption.

Step 3: Multi-tier Football Backup

Football backup involves dividing the data into multiple tiers. Each tier is a level of data redundancy with a unique backup schedule, ensuring that there are multiple copies of the data. We store the first two copies in the local storage attached to the backup agent and third copy backs up to VMware SDDC.

Step 4: Verify and Integrity Check

After the backups are completed, we use VMware API to automatically verify the integrity of the backup files to ensure everything is working as expected. This process internally invokes one-way hash algorithm SHA-256 that calculates the hash value of produced backup files after compression and encryption.

Success or Failure Reporting And Issue Resolution

The logging and error-handling mechanism built into ShitOps Football Unicorn helps our support team to resolve any issues quickly if the backup job fails or logs any errors. A success or failure report will be sent at the end of each day for our customers to check.

Conclusion

Our multi-dimensional football framework approach to backup systems works as advertised, successfully implemented by many of our happy clients. The impact was not only in having peace of mind on the client's part but also maximized our insight into the nature of their data and secured it since this type of football backup has worked our way both physically through tiered copy backups and cryptographically with its encryption procedures.

Of course, if you, too, want to implement a multidimensional backup football framework solution, your mileage might vary based on your own technical expertise.