September 11th, 2001 is a date which will live in American history forever: a moment frozen in infamy. The 9/11 terrorist attacks were a tragedy which set in motion an era of change throughout the United States and across the global community as a whole. For those of us who witnessed the events of that day firsthand, glued to our TV screens as the reports and rumors poured in, the footage of the planes striking the Twin Towers is an image we may never forget. But while most of us can recall exactly where we were when we heard the news, for Robbie Byrd, his relationship with the events of 9/11 is a little more unique.
In 2001, Robbie was an 18-year-old college sophomore, attending classes, working for the school paper, and dreaming of a future in journalism. When he awoke late on the morning of Tuesday the 11th to news of the World Trade Center attacks, there was no way Robbie could have known the effect that day would have on him, the United States, or the world as a whole; no one did. “My perspective on the world changed in two and a half hours,” says Robbie, reflecting on his memories of the day. Throwing on his clothes, Robbie raced to the office of the school newspaper. While many of us were still questioning what had just happened, Robbie was already in the process of compiling a special edition report on the incident, trying to analyze and make sense of a crisis still unfolding around him. In the twenty years since that day, Robbie’s work as an investigative journalist has found him covering other major stories of crime and tragedy, but like many Americans, the events of 9/11 have stuck with him. In many ways, they remain fresher in his mind than the average person’s.
Robbie is the creator and host of the 9/11 in Realtime project, a not-for-profit educational aid designed to compile and preserve audio and video data from 9/11 into a unified, interactive resource. Its goal? To ensure future generations understand the role 9/11 has played in American history and to give them a glimpse into the dreadful atmosphere of that day, as felt by millions of Americans.
“My perspective on the world changed in two and a half hours.”
First launched in conjunction with the 20th anniversary of 9/11 and originally hosted on AWS, Robbie’s project was intended for a small, specific audience: educators; students; a few curious individuals. But when a sudden surge in exposure resulted in 36,000 visitors over a two-day period, Robbie unexpectedly found himself on the hook for a $10,000 data egress bill from Amazon. On September 13th, 2021, just two days after the launch of his website, Robbie was forced to take down the page or face even higher fees. In the days and weeks which followed, Robbie found himself on a journey that would eventually bring him to Hivelocity, the new home of the 9/11 in Realtime project.
But how did it all happen? Could Amazon’s fees really be that high? Is Robbie’s experience with AWS unique?
To answer these questions, we need to go back to the beginning: to the early days of the 9/11 in Realtime project and the unforeseen events that led Robbie to Hivelocity.
What is 9/11 in Realtime?
Intended as a teaching aid, the 9/11 in Realtime project is a non-linear timeline designed to capture the anxious climate of 9/11 by compiling audio and video data from a multitude of sources into a single, multi-faceted asset. “Imagine sitting in a giant control room, with every piece of media from that day on monitors surrounding you, including coverage from every major network from 12am that morning until 12pm that night,” says Robbie in reference to the project. By selecting a TV channel and a time of day, users can jump back and forth between broadcasts leading up to, during, and after the attacks. Even the website’s user interface is an homage to the era, with panels and features designed to mimic the layout of old Macintosh computers.
Beginning roughly three years ago, the early days of the project were born out of a mix of curiosity, frustration, and a desire to preserve a slice of history at risk of being lost to time. “The project actually started as a way to learn more about how video players and video distribution work,” says Robbie, whose background is in both Journalism and Technology. Initially, the idea just represented a strong cross-section of interests and a means of experimenting with content delivery optimization. It wasn’t until one of the developers helping Robbie with the project admitted that he was too young to remember much from that day, that Robbie realized there was a need here he could meet. Inspired by the Apollo in Real Time project, a similar website which compiles interactive data for the Apollo 11, 13, and 17 launches, Robbie wondered if he could provide a similar experience focused solely on the events of 9/11. What started as a small experiment with just three videos quickly grew into a massive collection of data, housing nearly 72,000 news items.
“Imagine sitting in a giant control room, with every piece of media from that day on monitors surrounding you.”
With the 20th anniversary of the attacks approaching, Robbie found himself disappointed by the lackluster efforts being put forth by the major news outlets: reused footage and repackaged tributes recycled for posterity but providing no new insight or interactivity. “The experience had become kind of stale, almost sanitized. It was hard to not become numb.” Beyond this, many of the 9/11 memorial websites were in bad shape, lacking necessary updates and falling behind on their hosting fees. Thinking of his mother-in-law who works as a teacher, Robbie found himself wondering how today’s students, those too young to have witnessed 9/11 firsthand, could possibly understand the importance of this day without access to the necessary resources and artifacts.
Thus, the 9/11 in Realtime project was born.
2021 and the Early Days with AWS
In the beginning, Robbie never thought his work would draw much attention. The Internet is filled with hundreds of thousands of websites, and while a need for this resource existed, the likelihood that many would ever see Robbie’s pet project were slim. Designed as a tool to help educators supplement their curriculums, Robbie was more focused on getting his project live than optimizing video formats for cost-effective delivery. It just wasn’t the priority.
Seeking convenience and hoping for some leniency in pricing, Robbie turned to AWS for his website’s hosting needs. He had experience building with the platform, and while Robbie points out that setting up infrastructure through Amazon can be difficult, keeping it up and running once it’s established is relatively painless. Beyond that, Robbie hoped that by utilizing some connections he had to Whole Foods Market, a subsidiary of Amazon, he might be able to keep pricing for his website reasonably affordable. After all, the 9/11 in Realtime project is a free teaching aid designed to spread awareness, not reap a profit. Unfortunately, like many ex-AWS customers who end up abandoning the service, Robbie’s experience did not go as intended.
It all started in the days leading up the 20th anniversary of 9/11. Hoping his project would be noticed by teachers and educational professionals, Robbie posted a link to his new memorial site in Reddit’s r/Education subreddit. After the post was blocked by the page’s moderators, Robbie ended up reposting the link to a different subreddit, r/InternetIsBeautiful. The post started gaining traction, and before Robbie knew it, traffic to his website had spiked significantly. That’s when HackerNews took notice. Featuring Robbie’s 9/11 in Realtime project on the front page of their website, HackerNews got thousands of new eyes on the site. From there, traffic exploded and in just two days Robbie had over 36,000 visitors, resulting in nearly 100 TB of outbound data transfer. Because these levels of traffic were completely unexpected, Robbie hadn’t optimized his video format and caching for delivery.
Suddenly, two days of data egress had resulted in a $10,000 bill from Amazon.
Robbie attempted to speak with Amazon on multiple occasions. He applied for promotional credits hoping to bring down his bill but was denied. After several failed attempts, AWS simply advised Robbie to close out his account, and on September 13th, the site was pulled offline.
In the end, the only party claiming a profit was Amazon.
Suddenly, two days of data egress had resulted in a $10,000 bill from Amazon.
Robbie tried to mitigate the damage, relaunching his site as a static, single-page website through Google Cloud in the hopes of getting the content back online. While these efforts were an effective stopgap, he still had 600 gigs of video content to contend with. Without cloud storage through AWS to rely on, Robbie was forced to utilize records available from the Internet Archive, whose older MP4 format videos proved less than ideal for the dynamic needs of live streaming.
Unsure of what to do next, Robbie was stuck between two bad options: underperforming content or overpaying for hosting. Thankfully, that’s when an unexpected third option appeared, offering Robbie a way to complete his vision for the site without breaking the bank in the process.
The 9/11 in Realtime Project Moves to Hivelocity
In September of 2021, when HackerNews featured the 9/11 in Realtime project on their homepage, the abundance of traffic generated unintentionally resulted in the project’s removal from AWS. In a twist of fate, this exposure is also indirectly responsible for the website’s eventual preservation.
By chance, one of the individuals who took notice of the frontpage feature was Mike Architetto, founder and CEO of Hivelocity. Bookmarking the 9/11 in Realtime project, Mike returned a few days later hoping to explore the site in depth, only to discover the page had been taken offline. Concerned and hoping to assist, Mike reached out to Robbie himself, inquiring about the website’s fate and offering Hivelocity’s services as a solution. It was exactly the break Robbie needed. Now, with Hivelocity’s sponsorship and two new servers positioned in strategic geographic locations from among Hivelocity’s 40 available edge-ready data centers, the 9/11 in Realtime project is up and running in its new home, yielding better performance while operating at a fraction of the price.
Since making the switch to Hivelocity, Robbie has found three solutions to his biggest pain points: a sustainable cloud storage option, great bandwidth for video streaming, and the superior computing power of dedicated servers to help optimize video quality. By re-encoding his videos using multi-bandwidth options, Robbie is able to more efficiently optimize his content delivery around the end user, avoiding the shortcomings of the MP4 format, and providing a faster, smoother streaming experience. By keeping his data closer to his users, Hivelocity’s edge solutions are helping Robbie improve the time to first byte on his videos, a crucial component of user-engagement. With testing underway for a new global server load-balancing system, Hivelocity may soon be able to provide even faster results for streaming media providers, decreasing load times further.
With his videos operating more effectively, Robbie now has time to focus on other areas of the project, including UI updates to allow for a more mobile-friendly user experience and filtering through the remaining audio content he’s collected. Robbie’s ultimate goal is to utilize the 9/11 in Realtime project as a framework for a larger learning management tool. His hope is to create a resource where teachers can take any period in history, compile audio and video resources from places like YouTube, mix in images from the Internet and their own notes into an interactive timeline, creating a means to experience these historical artifacts in a more immersive way.
Since making the switch to Hivelocity, Robbie has found three solutions to his biggest pain points: a sustainable cloud storage option, great bandwidth for video streaming, and the superior computing power of dedicated servers to help optimize video quality.
Now, thanks to Hivelocity’s aid, not only is the 9/11 in Realtime project back online, but Robbie’s even planning out future projects, including a similar website built around data covering last year’s January 6th insurrection. Unlike 9/11, which occurred 20 years ago, the January 6th riots are much more recent, and the stockpile of available data is massive. With 2TB of raw video already compiled, Robbie’s biggest need is a team of content reviewers who can help sift through and analyze the data. Those interested in volunteering are welcome to contact Robbie through his email at: [email protected].
Lastly, Robbie says he’s, “humbled and grateful that people found the site and are using it,” and encourages anyone who’s interested in donating to instead send their donations to the 9/11 Memorial Museum. A large portion of the audio and video data his website utilizes originally came from the 9/11 Memorial Museum, and in Robbie’s opinion, they do an even better job of telling the story of that day than his project does.
The Takeaway
While Robbie’s story and use case are unique, his experience with AWS, unfortunately, is not. What the public cloud promises in convenience and scalability, it makes up for in data transfer fees. Oftentimes, it’s only after an organization or individual has become reliant on the provider’s services that these hidden fees begin to add up. At that point, it can be nearly impossible to extricate yourself from the provider, leading to many finding themselves trapped in vendor lock-in. With Hivelocity covering the data transfer fees for the 9/11 in Realtime project, Robbie’s solution is unique, however even at full price Hivelocity’s data transfer fees can save users thousands of dollars on their monthly IT bills. For those interested in a more concrete analysis of our services and pricing, check out our article, Is Your Cloud Too Expensive?, where we cover in-depth comparisons of the three major public cloud providers vs. our own services.
At the end of the day, the best hosting and digital infrastructure solution is the one which best serves you. If you’re currently drowning in public cloud fees and unsure of where to turn, it might be time to consider your other options. A bare metal cloud from Hivelocity might be the exact solution you’re looking for.
At Hivelocity, we believe that when our customers succeed, we succeed as well. Call or chat with an expert today and see what the Hivelocity difference can mean for your organization.
