Amazon handling our national security: Worse than you think

The U.S. Department of Defense has more than 3,000 datacenters and countless applications.  The DOD smartly want to consolidate to save money.  In March, the DOD put out a request for a proposal about a compute cloud infrastructure.  Business Insider reports that Amazon is currently a shoe-in for the $10-billion contract.  One reason is because Amazon has already signed a $600M contract with the CIA to run a more secure datacenter in Washington, D.C.

Bryan Crabtree wrote an article talking about why it is a mistake for the U.S. military to choose Amazon.  Many don't realize there is already a free solution to this problem, called OpenStack.

One of the benefits of the free software movement is that you can download the code and run it on your own computer.  You can leverage the advancements of industry and academia inside your datacenter.

There are great free codebases for managing a private cloud, such as OpenStack.  It is a system written in Python that NASA also uses and has contributed to.

With OpenStack, the U.S. military could cheaply and quickly consolidate its existing computing datacenters into a more reasonable number.  Presumably, some of the remaining datacenters will be under mountains.

Free and open-source software is taking over the world.  It is a better way to write software, in the tradition of science.  It is actually an idea both Marxists and libertarian economists can agree on.

The problem is that Amazon is mostly a parasite of the free software movement.  For example, look at Amazon's repositories for Alexa.  What you see are free Javascript samples to plug into their proprietary system.  The underlying code and data to understand your speech, and predict what you wanted, are not built in an open way.  This is ironic because Amazon Web Services is largely a packaging of free software.

In the case of the cloud, Amazon's open-source repositories mostly contain ways to connect to its servers.  It's primarily a free SDK, in many different programming languages, to talk to Amazon's infrastructure.

People have built cross-platform cloud APIs.  There's a popular library written in Python called libcloud, which supports 30 cloud providers.  Amazon on its website recommends the custom ones Amazon built.

Moving applications to an external cloud is a security risk.  Even if the data is encrypted over the wire, and on the hard drive, the application server processing the data has all the information un-encrypted.

The Defense Department can always afford soldiers to guard and maintain its computers.  It's easy to run a datacenter compared to an aircraft-carrier.  With OpenStack, a small team of people can manage hundreds of computers.

The RFP mentions a requirement for 50 petabytes (or 50,000 terabytes) of online storage.  This is a huge number to anyone in the computer industry, who usually deals in megabytes or gigabytes, but you can buy a 6T drive for $114 on Amazon.  You could buy all the storage for $800,000.

Note that 8,300 hard drives is probably not enough.  You will want more for redundancy, and even more for performance.  Some drives could be so busy that you want to make copies or split the data up to handle the load.  So you could triple the cost to be safe.  It's still far less than one F-35 airplane.

The RFP mentions needing 46,000 compute cores.  On Lenovo's website, you can purchase a 16-core server for $4,000.  It would take 2,800 of those servers, and cost about $11 million.  That is still less than an F-35.

These are just partial estimates, but it does put the $10 billion in perspective.

A datacenter in D.C. means that the application servers that used to be close to the soldiers will now go much farther.  What used to be one millisecond to a server nearby will take 100 milliseconds one way.

Even assuming that there are no security risks by sending all traffic this extra distance, it will definitely slow down performance, no matter how fast the servers.  Instead of having local servers and small pipes to talk between datacenters, they will have to build big pipes all around the country.

Moving to a commercial cloud won't even save that much money on I.T.  The big expenses of an I.T. budget have to do with the number and complexity of the applications and the people to maintain them.

The U.S. Army has more than ten command and control systems.  It also presumably has ten companies or teams it's paid to create and maintain the software.  It could move all its servers to the cloud, and it'll still have ten proprietary command and control systems to maintain.  The U.S. military could give Amazon $10B and still have a broken computing infrastructure.

The U.S. Department of Defense has more than 3,000 datacenters and countless applications.  The DOD smartly want to consolidate to save money.  In March, the DOD put out a request for a proposal about a compute cloud infrastructure.  Business Insider reports that Amazon is currently a shoe-in for the $10-billion contract.  One reason is because Amazon has already signed a $600M contract with the CIA to run a more secure datacenter in Washington, D.C.

Bryan Crabtree wrote an article talking about why it is a mistake for the U.S. military to choose Amazon.  Many don't realize there is already a free solution to this problem, called OpenStack.

One of the benefits of the free software movement is that you can download the code and run it on your own computer.  You can leverage the advancements of industry and academia inside your datacenter.

There are great free codebases for managing a private cloud, such as OpenStack.  It is a system written in Python that NASA also uses and has contributed to.

With OpenStack, the U.S. military could cheaply and quickly consolidate its existing computing datacenters into a more reasonable number.  Presumably, some of the remaining datacenters will be under mountains.

Free and open-source software is taking over the world.  It is a better way to write software, in the tradition of science.  It is actually an idea both Marxists and libertarian economists can agree on.

The problem is that Amazon is mostly a parasite of the free software movement.  For example, look at Amazon's repositories for Alexa.  What you see are free Javascript samples to plug into their proprietary system.  The underlying code and data to understand your speech, and predict what you wanted, are not built in an open way.  This is ironic because Amazon Web Services is largely a packaging of free software.

In the case of the cloud, Amazon's open-source repositories mostly contain ways to connect to its servers.  It's primarily a free SDK, in many different programming languages, to talk to Amazon's infrastructure.

People have built cross-platform cloud APIs.  There's a popular library written in Python called libcloud, which supports 30 cloud providers.  Amazon on its website recommends the custom ones Amazon built.

Moving applications to an external cloud is a security risk.  Even if the data is encrypted over the wire, and on the hard drive, the application server processing the data has all the information un-encrypted.

The Defense Department can always afford soldiers to guard and maintain its computers.  It's easy to run a datacenter compared to an aircraft-carrier.  With OpenStack, a small team of people can manage hundreds of computers.

The RFP mentions a requirement for 50 petabytes (or 50,000 terabytes) of online storage.  This is a huge number to anyone in the computer industry, who usually deals in megabytes or gigabytes, but you can buy a 6T drive for $114 on Amazon.  You could buy all the storage for $800,000.

Note that 8,300 hard drives is probably not enough.  You will want more for redundancy, and even more for performance.  Some drives could be so busy that you want to make copies or split the data up to handle the load.  So you could triple the cost to be safe.  It's still far less than one F-35 airplane.

The RFP mentions needing 46,000 compute cores.  On Lenovo's website, you can purchase a 16-core server for $4,000.  It would take 2,800 of those servers, and cost about $11 million.  That is still less than an F-35.

These are just partial estimates, but it does put the $10 billion in perspective.

A datacenter in D.C. means that the application servers that used to be close to the soldiers will now go much farther.  What used to be one millisecond to a server nearby will take 100 milliseconds one way.

Even assuming that there are no security risks by sending all traffic this extra distance, it will definitely slow down performance, no matter how fast the servers.  Instead of having local servers and small pipes to talk between datacenters, they will have to build big pipes all around the country.

Moving to a commercial cloud won't even save that much money on I.T.  The big expenses of an I.T. budget have to do with the number and complexity of the applications and the people to maintain them.

The U.S. Army has more than ten command and control systems.  It also presumably has ten companies or teams it's paid to create and maintain the software.  It could move all its servers to the cloud, and it'll still have ten proprietary command and control systems to maintain.  The U.S. military could give Amazon $10B and still have a broken computing infrastructure.