This link has been bookmarked by 24 people . It was first bookmarked on 30 May 2008, by Akihiko Komada.
-
17 May 10
-
25 Mar 10
-
23 Apr 09
-
18 Feb 09
-
12 Jan 09
-
Dean seemingly thinks clusters of 1,800 servers are pretty routine, if not exactly ho-hum.
-
the software company runs on top of that hardware, enabling a sub-half-second response to an ordinary Google search query that involves 700 to 1,000 servers, is another matter altogether
-
Google clearly believes its technological destiny is best left in its own hands.
-
Server makers pride themselves on their high-end machines' ability to withstand failures, but Google prefers to invest its money in fault-tolerant software.
-
Our view is it's better to have twice as much hardware that's not as reliable than half as much that's more reliable
-
In each cluster's first year, it's typical that 1,000 individual machine failures will occur; thousands of hard drive failures will occur; one power distribution unit will fail, bringing down 500 to 1,000 machines for about 6 hours; 20 racks will fail, each time causing 40 to 80 machines to vanish from the network; 5 racks will "go wonky," with half their network packets missing in action; and the cluster will have to be rewired once, affecting 5 percent of the machines at any given moment over a 2-day span, Dean said. And there's about a 50 percent chance that the cluster will overheat, taking down most of the servers in less than 5 minutes and taking 1 to 2 days to recover.
-
Google required Intel to create custom circuit boards
-
Google likes multicore chips, those with many processing engines on each slice of silicon
-
they already have made the jump to parallelism
-
multicore machines look like lots of little machines with really good interconnects.
-
"Single-thread performance doesn't matter to us really at all," Dean said. "We have lots of parallelizable problems."
-
three core elements of Google's software: GFS, the Google File System, BigTable, and the MapReduce algorithm.
-
Some incarnations of GFS are file systems "many petabytes in size"--a petabyte being a million gigabytes.
-
GFS stores each chunk of data, typically 64MB in size, on at least three machines called chunkservers
-
To provide some structure to all that data, Google uses BigTable.
-
On any given day, Google runs about 100,000 MapReduce jobs; each occupies about 400 servers and takes about 5 to 10 minutes to finish, Dean said.
-
MapReduce, like GFS, is explicitly designed to sidestep server problems.
-
the company isn't satisfied and has a long to-do list
-
-
12 Nov 08
-
27 Jul 08
-
01 Jul 08
-
28 Jun 08
-
It puts 40 servers in each rack, Dean said, and by one reckoning, Google has 36 data centers across the globe. With 150 racks per data center, that would mean Google has more than 200,000 servers
-
In each cluster's first year, it's typical that 1,000 individual machine failures will occur; thousands of hard drive failures will occur; one power distribution unit will fail, bringing down 500 to 1,000 machines for about 6 hours; 20 racks will fail, each time causing 40 to 80 machines to vanish from the network; 5 racks will "go wonky," with half their network packets missing in action; and the cluster will have to be rewired once, affecting 5 percent of the machines at any given moment over a 2-day span, Dean said. And there's about a 50 percent chance that the cluster will overheat, taking down most of the servers in less than 5 minutes and taking 1 to 2 days to recover.
-
-
24 Jun 08
-
22 Jun 08
-
11 Jun 08
David NaughtonStephen Shankland: 2008-05-30: CNET News.com
architecture google datacenter scalability web dldl-geek-book-club for:libsys for:jtrammell for:gormsby for:greevar for:olegmmiller
-
03 Jun 08
-
31 May 08
-
30 May 08
-
Akihiko KomadaI'm sure a number of server companies are sour about it, but Google clearly believes its technological destiny is best left in its own hands. Co-founder Larry Page encourages a "healthy disrespect for the impossible" at Google, according to Marissa Mayer, vice president of search products and user experience, in a speech Thursday.
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.