Google, Amazon, Microsoft, and Facebook buy
more networking hardware than practically anyone else on earth. After all,
these are the giants of the internet. But at the same time, they’re buying less
and less gear from Cisco, HP, Juniper, and the rest of the world’s largest
networking vendors. It’s an irony that could lead to a major shift in the
worldwide hardware market.
Over
the past few years, the giants of the web have changed the way they purchase
tens of thousands of the network switches inside the massive data centers
driving their online services, quietly moving away from U.S.-based sellers to
buy cheaper gear in bulk straight from China and Taiwan. According to J.R.
Rivers — an ex-Google engineer — Google has built its own gear in tandem with
varous Asian manufacturers for several years, and according to James Liao — who
spent two years selling hardware for Taiwan-based manufacturer Quanta —
Facebook, Amazon, and Microsoft are purchasing at least some of their
networking switches from Asian firms as well.
“My
biggest customers were these big data center [companies], so I know all of them
pretty well,” Liao says. “They all have different ways of solving their
networking problems, but they have all moved away from big networking companies
like Cisco or Juniper or [the Dell-owned] Force10.”
The
move away from U.S. network equipment stalwarts is one of the best-kept secrets
in Silicon Valley. Some web giants consider their networking hardware strategy
a competitive advantage that must be hidden from rivals. Others just don’t want
to anger their business partners in the hardware sector by talking about the
shift. But cloud computing is an arms race. The biggest web companies on earth
are competing to see who can deliver their services to the most people in the
shortest amount of time at the lowest cost. And the cheapest arms come straight
from Asia.
J.R.
Rivers is one of the arms dealers. He runs a company called Cumulus Networks
that helps the giants of the web — and other outfits — buy their networking
hardware directly from “original design manufacturers,” or ODMs, in China and
Taiwan. And he’s worked in this world for an awfully long time. He’s one of the
Google engineers who secretly designed a new breed of networking switch for the
company’s data centers, the massive computing facilities that drive its search
engine and the rest of its web services.
Rivers
joined Google in October 2005, after five ears as a distinguished engineer at
Cisco, the company that dominated the worldwide market for networking gear. At
the time, Google was still connecting its servers using standard networking
switches from the likes of Cisco and Force10 Networks. But these mass-market
switches just didn’t suit Google’s unusually large operation.
“When
Google looked at their network, they need high-bandwidth connections between
their servers and they wanted to be able to manage things — at scale,” Rivers
says. “With the traditional enterprise networking vendors, they just couldn’t
get there. The cost was too high, and the systems were too closed to be
manageable on a network of that size.”
So Google
drew up its own designs — working alongside manufacturers in Taiwan and China —
and cut the Ciscos and the Force10s out of the equation. The Ciscos and the
Force10s build their gear with many of those same manufacturers. Google removed
the middlemen.
The
search giant does much the same with its servers, buying custom-built machines
straight from Asia rather than going through traditional sellers such as Dell
and HP. Because its web services were used by such an enormous number of
people, Google faced all sorts of data center problems no one else faced —
problems of power and space as well as cost and logistics. So it built all
sorts of custom hardware to solve those problems.
Now,
the other giants of the web are running into the same issues, and they too are
going straight to Asia for hardware. Following closely behind are companies
that run large internal server farms, including financial houses and healthcare
outfits.
As J.R.
Rivers serves this market with Cumulus Networks, James Liao is doing much the
same thing with a second startup called Pica8, offering networking gear that
comes straight from the ODMs. Pica8 is a spinoff of Liao’s former employer,
Quanta — one of the companies that manufactured Google’s original networking
switches, according to Rivers.
According
to Liao, tens of thousands of switches are already being sold by the Asian ODMs
directly to the likes of Amazon, Facebook, and Microsoft. And that doesn’t
include the gear Google has bought over the past seven years. “This is just the
beginning,” Liao says, pointing out that these buyers operate the biggest data
centers on earth. These companies account for only a part of the
$7-billion-a-year Ethernet switch market, but as more and more outfits move
their operations into the proverbial cloud, the influence of these web giants
will only grow.
Liao
estimates that Amazon, Microsoft, Facebook and others have bought Asian network
switches spanning “millions” of network ports — i.e., connections to servers —
and he guesses that in 2011, about 60 percent of these ports provided 10Gigabit
Ethernet connections. According to Matthias Machowinski — a directing analyst
with Infonetics, a research firm that tracks the networking market — the
official market for 10Gigabit Ethernet spanned about 9 million ports in 2011.
J.R.
Rivers declines to name the companies he’s working with at Cumulus Networks,
but he confirms that some of the big-name web outfits are already buying
networking switches from ODMs in Asia. In all likelihood, these companies are
also purchasing switches from other sources as well. Cisco says it has a
“significant presence and mindshare” in the big-name web market, and Juniper
says it has a relationship with all of the top five web players, pointing out
that data center networks require more gear than just switches. But the market
is on the move.
The Future of ‘Web Giant 3.0′
“We are
continuously exploring new infrastructure technologies that may evolve further
efficiencies across our portfolio. We normally have discussions with ODMs and
large and small OEMs to better understand their capabilities and evaluate their
products,” reads a statement sent to Wired by a Microsoft spokesperson and
attributed to Dileep Bhandarkar, a distinguished engineer who oversees the data
centers driving Microsoft’s online services. But the statement did not
specifically address the purchase of networking gear.
Amazon
did not respond to a request for comment about its hardware practices, and a
Google spokeswoman sent us a one-sentence statement: “We work with a variety of
vendors to manufacture the equipment we use in our data centers,” she said.
These two companies — particularly Google — are rather tightlipped about their
data center practices.
Facebook
declined to discuss how it purchases networking gear, but in response to
secretive approach of Amazon and Google, the company has openly discussed some
of its other practices, and it has actually shared its server and data center
designs with the rest of the world. It purchases its servers directly from Quanta
and Wistron, another Taiwanese ODM.
Martin
Casado — the chief technology officier of a third Silicon Valley networking
startup, Nicira — confirms that the hardware market is shifting to Asia.
Offering a software platform that virtualizes networking gear in much the same
way that VMware virtualized servers, Nicira helps some of the big web players
build their networks. The Nicira platform was designed specifically for
companies along the lines of Google that want to use cheap commodity switches
to physically construct their network but then do all the complex management in
software.
“If
you’re building web giant 3.0, you can go to Quanta in Taiwan and buy crates …
of switches,” he says. “This supply chain change is nascent. But it’s the most
exciting thing going on in Silicon Valley right now.”
Google Goes to Asia
According
to J.R. Rivers, Google began work on its custom-built networking switches in
early 2005, before he arrived at the company. In the beginning, River says,
Google worked in tandem with Quanta and other Asian ODMs. But eventually, he
says, Google took all the engineering work in house. Basically, he says, the
company wasn’t happy with the work the ODMs did at the time. Google engineers
would design the switches, and then they would bring the completed designs to
contract manufacturers in Asia, outfits along the lines of Foxconn, the Asian
company that builds Apple’s iPhones and iPads.
Google
has never discussed its practices publicly, but rumors have long indicated that
the company built its networking switches in this way. In 2007, research
analyst Andrew Schmitt noticed that certain manufacturers were producing
enormous numbers of chips for 10Gigabit Ethernet switches but that the switches
themselves weren’t actually turning up on the market. “It didn’t make sense to
me why someone would be building so much of a given component if there were no
customers that could use it,” he says. “What I was able to determine is that
Google was purchasing switch chips straight from the comment suppler.”
The
switches Google was building typically sat at the top of a rack of servers in
the data center, connecting the servers to the rest of the network. As Juniper
points out, this is only part of the networking hardware used in the data, but
it’s a large part.
Google,
Rivers says, is a unique company. It has the wherewithal and the talent to
built its own switches, but other companies may not be up to the task. With
Cumulus Networks, J.R. Rivers and his partner, Nolan Leake, are trying to
grease the wheels. “[The other web players] are trying to figure out what the
best model is, and that’s one of the reasons we started up,” Rivers says.
“Google is unique in its willingness to build something just because they know
it can be done. Most other people see a risk/reward trade-off. We seek to
minimize that risk.”
Though
Rivers declined to name the ODMs his company is working with, he says that
these are well-known manufacturers in Taiwan and China. “We’ve been working for
the last year on opening up a supply chain for traditional ODMs who want to
sell the hardware on the open market for whoever wants to buy,” he says. “For
the buyers, there can be some very meaningful cost savings. Companies like
Cisco and Force10 are just buying from these same ODMs and marking things up.
Now, you can go directly to the people who manufacture it.”
This
has become possible in recent years, Rivers says, because the ODMs have slowly
acquired more and more engineering talent. You can now buy commodity gear from
more places. “Networking is opening up much like the transition from mainframes
to RISC machines and later to x86 servers,” says Rivers’ partner, Nolan Leake.
“We’re moving towards a world where customers have more control over their
destiny.”
‘The Arms Dealer’
Before
spinning Pica8 out of Quanta, James Liao was already selling similar networking
switches to the big web players. Nicira’s Martin Casado refers to James Liao as
“the arms dealer” in this networking revolution. “He’s the conduit between the
rest of the world and Quanta. He knows this space better than anyone,” Casado
says. “And I love him because he talks like he’s part of organized crime.”
From
July 2009 to September 2011, Liao was the senior director at Quanta in charge
of product strategy for network switching and data center products. He was
based in Silicon Valley, and his job was to serve the giants of the web. He
declines to go into much detail about how these companies acquire their
hardware, but he’s unequivocal in saying that the other big companies — Amazon,
Microsoft, and Facebook — are now following Google’s lead in going directly to
Asia for their gear.
Networking
switches, he says, have become a commodity. “They all use the same chips. They
have to same latency. They have the same bandwidth. This is a clear signal that
the hardware platform is commoditizing,” he says. “You can actually find a lot
of [ODM] suppliers that have the capability to manufacturer and design this
kind of platform.”
Like
Cumulus Network, Liao’s new venture, Pica8, brings this low-cost networking
hardware to a much larger market. In the past, one of the problems with buying
directly from the ODMs is that you had build your own software to drive your
switches. But Pica8 aims to provide software for those companies that don’t
want to build their own. The company has open sourced an early version of this
software — known as Picos — and it plans to open source a more extensive
version of the platform next month.
“We
give you the hardware and the software,” Liao says. “If you take our platform
and compare it to Cisco, the protocol features we provide and the hardware
performance are all in the same range. The only difference is that the price is
40 percent to 60 percent lower.”
Though
Pica8 spun off of Quanta, Liao says that the company will also sell switches
from other ODMs. But he declined to name them. But he does say Pica8 is selling
gear to Japanese telecom giant NTT and Baidu, the company that dominates the Chinese
search engine market.
Matthias
Machowinski, of research firm Infonetics, says he is “very much aware” of this
trend, though he adds that it is extremely hard to track. He says that the big
web giants account for only a part of the overall switch market — “the number
of customers that might choose to go down this route are very limited. Today,
you can count them on one hand, and maybe over the next two years, two hands
might be enough” — but he also acknowledges that as businesses move their applications
onto services such as Amazon EC2 and Microsoft Azure — rather than running
stuff in their own data centers — these web giants will account for an even
larger part of the switch market.
Like Server, Like Switch
This
shadow networking market is a repeat of what happened in the server world.
Years ago, Google started building its own servers in tandem with the Asian
ODMs, and other web giants followed. These companies are looking to save cost,
but they’re also looking to reduce their power consumption, customizing
machines so they’re far more efficient than their mass-market brethren.
In
2009, Google revealed some server designs it produced several years before.
But, as with networking practices, the company says very little about its
server gear. Amazon operates in much the same way. But Facebook had taken a
different approach. Last year, after building its own data centers and working
with various manufacturers to build its own servers, the social networking
giant open sourced these designs to the rest of the world, hoping that others
across the industry can help improve those designs, buy more hardware based on
the designs, and ultimately drive down the price of the hardware.
This
Open Compute Project already has several other big-name backers, including
Texas-based cloud computing outfit Rackspace and Japan’s NTT. And it doesn’t
stop at data centers and servers. Last month, Frank Frankovsky — the ex-Dell
man who oversees hardware design at Facebook — told us that the company is in
the midst of building its own storage hardware and that these designs will be
open sourced in early May.
In
these cases, Facebook and Amazon and Google and others bypassed “original
equipment manufacturers,” or OEMs, such as Dell and HP. The servers sold by the
likes of HP and Dell are actually manufactured by those same ODMs in Taiwan and
China.
James
Liao, of Pica8 and formerly of Quanta, does not work with servers. But he says
that it’s common knowledge that — like Google and Facebook — Amazon purchases
at least some of its servers from ODMs in Asia. “For servers, Facebook and
Amazon are taking almost exactly the same approach,” he says. “Amazon also has
some very high power designers, but they don’t do the design themselves. They
come up with a certain architecture and they tell the ODMs: ‘This is my vision.
These are the goals. And I want help designing the hardware.’”
Now,
Liao says, this same sort of thing is happening with, well, everything. “All of
the data center hardware is bought this way,” Liao says. “You can refer to
Facebook as an example, where one of the big projects inside the Open Compute
effort is storage. Even the storage side is being commoditized. Servers,
storage, and networking — all of them are going to this way.”
Howard
Wu — the president of greater China for Joyent, an Amazon-like cloud computing
outfit based in San Francisco — agrees. “If you’re a small business and you’re
going to buy five servers, you’re going to Dell or HP, because of the support
services. But if you’re a data center operator and you’re going to buy 10,000 servers,
you’re going straight [to the ODMs],” he says. “It’s kind like buying couches.
If you buy one, you go to a retail store. If you buy 10,000 couches, you go
straight to the factory.”
That
said, Joyent is not yet buying its gear from the ODMs.
“We are
definitely in talks, but it hasn’t actually happened yet,” he says. “We have
other contractual obligations right now.” The market has not completely shifted
to Asia. It’s moving in stages. These web companies have many suppliers —
that’s just good for businesses — and in some cases, they’re still buying
hardware from the traditional players — perhaps because they still have
contracts in place. Facebook, for instance, is still buying some servers from
Dell and HP. And Amazon is still buying custom servers from Rackable, a
stateside manufacturer, and apparently other outfits based here in America.
The
hardware supply chain is vast and varied. But it’s consolidating. Now that they
have the engineering talent, J.R. Rivers says, the ODMs are transforming into
OEMs. “The market is maturing to the point where anyone can buy directly from
ODMs,” he says. “You don’t have to be Google.”
Cade
Metz
wired.com
Business & Investment Opportunities
YourVietnamExpert is a division of Saigon Business Corporation Pte Ltd, Incorporated in Singapore since 1994. As Your Business Companion, we propose a range of services in Strategy, Investment and Management, focusing Healthcare and Life Science with expertise in ASEAN. We also propose Higher Education, as a bridge between educational structures and industries, by supporting international programmes. Many thanks for visiting www.yourvietnamexpert.com and/or contacting us at contact@yourvietnamexpert.com
No comments:
Post a Comment