A Yottabyte of storage

From SusoSight

In my life I've seen us go from Kilobytes to Megabytes to Gigabytes to Terabytes and now just barely large arrays are now hitting Petabytes. Just the other day I was telling someone about how in the 90s we would joke at the concept of a terabyte. And now we are here. So when will we see a Yottabyte of storage. Here is my estimate. Note that in these numbers I'm mostly talking about consumer grade equipment.

  • 1960s - (Megabytes in mainframe use)
  • 1970s - Kilobytes (Megabytes in mainframe use)
    • Harddrives were never below a few megabytes in size, but things like floppy drives and volatile memory were.
    • The Illiac IV supercomputer from 1972 apparently had a special laser based metal film memory that could store up to 1 terabit (125 Gigabytes) of data. However this was write once memory.
    • 1972 - In the documentary "Computer Networks - The Heralds of Resource Sharing", Lawrence G. Roberts refers to weather files that were "10 to the 11th bit". This would be a 12.5 GB file, which seems insane for 1972, but lends credibility to the metal film memory mentioned previously.
  • 1980s - Megabytes (KB/MB era for memory)
    • (1981) 10MB Hard drive ($3398)
    • (1989) ~600MB Hard drives
  • 1990s - Gigabytes (MB era for memory)
    • (1990) - IBM 4GB hard drive ($20,000)
    • (1993) - 9GB hard drive (I think they were about $3000)
    • (1999) - 20GB hard drives
    • 1999 - Suso's backups took up maybe 10GB.
  • 2000s - Giga/Tera era (MB/GB era for memory)
    • (2004) - 5.625 petabytes Google server farm capacity [1]
    • (2004) - 128MB microSD card is available
    • 2006 - Suso's backups took up about 500GB
    • (2007) - 1TB drives
    • (2007) - Place I worked at had a 40TB SAN that cost over $250,000
    • (2009) - 2TB drives ($450)
  • 2010s - Tera era (GB era for memory and maybe TB by 2015)
    • 2010 - 2TB drives available (as low as $200)
    • 2010 - Suso's backups take up 1.6TB
    • 2010 - Highest memory capacity seen for a motherboard is about 192GB (Supermicro X8DTN+-F), but they don't make the 16GB DIMMs yet that it would take and 18x8GB (144GB) dimms would cost $10,800 alone.
    • 2012 - Someone on the BLUG list talking about a small scale SAN for a lab being 5-10TB.
    • 2014 - You can buy 128GB microSD cards for about $190 (SanDisk 128GB 128G microSDXC Ultra microSD micro SDHC SDXC Class 10 UHS-I C10 Memory Card)
    • 2014 (Dec) - Western Digital 6TB hard drives available for as cheap as $250 on Newegg
    • 2014 (Dec) - Sandisk 960GB SSD drives for $400 on Newegg, but various models go as high as $1200.
    • 2014 (Dec) - Sandisk 128GB class 10 microSD cards for $105
    • 2014 (Dec) - Nice home desktop on Dell's website comes with 32GB of RAM and 3TB hard drive for $2000. An Alienware laptop $(4800) has as much as 32GB of RAM and a 512GB SSD drive. Its possible to get a desktop motherboard with 64GB of RAM.
    • 2014 (Dec) - Supermicro 3U servers adversed as having up to 1.5TB of RAM, however only 32GB DIMMS could be found ($350 each) (24 slots on server).
    • 2015 (Oct) - 8TB Internal Hard drives for $600, 1TB "green" Internal hard drive for $50. 2TB Samsung SSD drive $709, 512GB SSD drive $130, 128 MicroSD card $50, 2x16GB RAM Dimms for $250, most laptops these days come with 8GB of RAM and a 1TB hard drive.
    • 2016 (Mar) - Samsung announces 15TB SSD drive, leap frogging the capacity of current magnetic hard discs (still only 8TB available) for the first time.
    • 2017 - It was around 2017 that we started to see SSD drives match the capacity of magnetic drives on the market at the time.
    • 2018 (Nov) - At Supercomputing '18, a vendor was showing a 16TB M.2 drive and a 1U server that could hold 36 of them for a total of 576GB in 1U.

The future

A lot depends on the industry's ability to invent something beyond high capacity magnetic storage. SSD drives are still in their infancy as far as capacity goes (although they are growing fast) and we still have yet to see a consumer grade holographic storage device. I think that some kind of chemical storage medium will be required to get us beyond the 2020s and maybe then we'll see another surge of invention that could take us to Yottabyte levels.

  • 2020s - Petabytes (TB for memory)
  • 2030s - Petabytes/Exabytes (Also the end of 32-bit timespace for unix)
  • 2040s - Exabytes
  • 2050s - Zettabytes
  • 2060s - Zettabytes/Yottabytes
  • 2070s - Yottabytes
  • 2090s - ????bytes (1024^9 so maybe a nonubyte, novembyte or enneabyte? They haven't been named yet but previous prefixes were based on the greek or latin name of the exponent of 1024^N, Novembytes, the official data size of your Thanksgiving dinner.)

No official prefixes to use after Yotta at this time.

So if I live to be 85 I MIGHT get to see yottabytes of storage in use. Of course if they don't fix the data transfer bottleneck we've been in, it will take until the rise of the next intelligent species on Earth to copy the data (1).

Uses for a Yottabyte

In the article data powers of 10, the author states as an example that all words ever spoken by humans would take up 5 exabytes. This is presumably all words spoken in text form (not audio data).

So following along those lines, I thought a good example for demostrating how big a yottabyte would be is to ask "How much storage space would be required for a video recording of the whole lifetime of every person's that ever lived. This surely would be the ultimate expression of data storage as if you recorded someone's life completely, you'd pretty much cover everything they cared about as well. And since you'd be recording every person ever, you'd have all the data. It would basically be more data than anyone could do anything with. Nobody would be able to watch even one whole recording of someone else's life without wasting their own life.

So first we need to know how many people have ever lived. A quick search on google reveals an estimate of 100 billion people, which sounds about right.

We'll say that the average lifetime is 40 years, which kinda accounts for the fact that over the past century or two lifetimes have increased in length dramatically. 40 years is 1,262,269,440 seconds.

Now for a format, let's go ahead and use a modern format of 1920x1080 HD. Quite a bit of data alone just for a 2 hour movie. The data rate of HD is close to 1MB per second, so we'll use that.

So we have this simple equation

1,262,269,440 seconds X 1MB (220) X 100,000,000,000 = 132,358,544,031,744,000,000,000,000 = 132 YB = 109 YiB

Of course this is just for a single copy of all the recordings without backups, so there you go. Humanity will probably find some need for 1000YiB drives anyways. Probably everyone would need to have a copy of everyone else's lifetime, even if they didn't have time to watch it all. Which is kinda what people who hoard massive amounts of media do right now.

  • (1) - 1 YB / 150MB a second / 86400 sec in day * 365 days in year = 243,726,007 years (and 150MB/sec is being generous)