Google Compute Engine and MapR Technologies Crush MinuteSort Record

Related: Geeks, News

MapR Technologies, Inc., the Hadoop technology leader, announced at the O'Reilly Strata Conferencea new world record for MinuteSort, sorting 15 billion 100-byte records (a total of 1.5 trillion bytes) in 60 seconds using Google Compute Engine and the MapR Distribution for Apache Hadoop. The benchmark, often referred to as the World Cup of data sorting, demonstrates how quickly data can be sorted starting and ending on disks.

MapR's record-setting benchmark was completed on 2,103 virtual instances in the Google Compute Engine. Each virtual instance consisted of four virtual cores and one virtual disk, for a total of 8,412 virtual cores and 2,103 virtual disks. The new record surpassed the previous record of 1.4TB. The new record is also almost three times the amount of data processed by the previous Hadoop MinuteSort record which was 578 GB.

"The record is significant because it represents a total efficiency improvement executed in a cloud environment," said Jack Norris, VP of marketing, MapR Technologies. "In an era where information is increasing by tremendous leaps, being able to quickly scale to meet data growth with high performance makes business analytics a reality in situations previously impossible."

While the previous MinuteSort record was achieved with custom hardware, MapR set the record using commercially available Google Compute Engine, Hadoop MapReduce and the MapR Distribution. Google Compute Engine is currently in Limited Preview but will soon be available so any business or developer can benefit from the scale, performance and value of Google's infrastructure.

Page 2 >>