Anandtech Reviews the Calxeda ECX-1000: “Calxeda’s ECX-1000 server node is revolutionary technology”
I’d like to point everyone over to a great review of the Calxeda-powered Boston Viridis box by Anandtech that just went live, here. First of all, big thanks to Johan De Gelas over at Anandtech and Wannes De Smet at SizingServers for doing a top notch job pulling together an in-depth review of our gear as well as the team at Boston Limited for taking care of the hardware. Since we launched the ECX-1000 we’ve been beating the streets to get real results and metrics out into customers’ hands and show that the technology delivers as promised. With quotes like “Calxeda really did it”, “nothing short of remarkable” and “revolutionary technology”, we’re all excited to see these results posted on a site like Anandtech.
Written by Shawn Kaplan, General Manager – Financial Services, TELX
Advances in multi-core computing have allowed far greater compute densities such that nearly all datacenter racks run out of available power far sooner than physical space. Traditional High Performance Computing (HPC) X86 clusters can consume upwards of 400W per rack unit (U), this means that a typical data center rack with a 5KW – 8KW circuit can be maxed out in as little as 1/4 or 1/2 of the available space. Many of today’s forward thinking IT leaders are asking “Why can’t I have both extremely dense computing and better power efficiency?”
IEEE held their annual fest for uber-techies at SuperComputing ’12 this week in Salt Lake City. With over 8000 attendees flocking to the snowy site in spite of the economy and impending fiscal cliff, this event has become a mecca for anyone seeking the next great technology in computing hardware for serious work. In the old days, it was all about (Tera)Flops and Fortran. These days it is about Big Data, hardware acceleration, interconnect fabrics, storage, and green computing. Wandering around in the massive exhibit hall, one could see name badges from companies like eBay, Amazon, Peer One Hosting, and Dreamworks, right alongside the traditional attendees from leading universities, National Labs, and the Departments of Defense and Energy.
So, what’s a little core like ARM doing in a place like this? Its all about the data. “Data Intensive Computing” in HPC is pronounced “Big Data” in the enterprise. And the two communities have another thing in common: both are seeking more energy efficient solutions to large computations challenges. So naturally, they are turning to ARM with great hopes for the future.
Today’s Dell announcement of their donation to the Apache Software Foundation is a huge milestone not only for Calxeda but the entire ARM server ecosystem. Supporting and engaging the open-source community has always been a high priority for the Round Rock based company, evidenced by their contributions and leadership in multiple open-source projects like OpenStack and other Apache projects. But this particular announcement is more than just a generous donation to a non-profit foundation.
Back in June, Calxeda published web-serving benchmarks that claimed a significant advantage in performance per watt over x86-based servers. Using ApacheBench, a single 5.26 watt Calxeda EnergyCore server delivered 5500 transactions per second, compared to a 102 watt (TDP) Intel E3-1240 that saturated the network at 6950 TPS. About 2 months later, Intel spoke with Timothy Pricket Morgan at The Register to provide their response.
You have to hand it to Intel; they make really fast processors, which are appropriate when maximum compute performance is needed. But Intel’s argument is missing the point, the very reason why Extremely Efficient Servers are a promising trend: by right-sizing the compute, memory, and networking infrastructure to meet real workload requirements, one can save a great deal of money and power. Intel’s response is classic PC-Server era thinking: use a faster CPU, and then feed it like a force-fed goose being prepped for foie gras. In this case they added a 10G ethernet port to try to close the gap. But if 5000 transactions per second is all your website needs, or you use load balancing to handle the peak loads above normal usage, Calxeda is dramatically more efficient. That is the point.
It is a bit surprising Intel went to these lengths when Intel’s own math shows that Calxeda maintains a 4-5X performance/watt advantage versus the solution most websites would use. Apparently not satisfied, Intel then upped the ante and added an expensive 10 Gb network infrastructure to keep their uber-fast processor busy. With this configuration, Calxeda is still some 30% more efficient than the significantly more expensive* 10Gb Ivybridge solution. But small-medium web sites rarely use or need a 10Gb ethernet port; a 1Gb interface is usually sufficient for typical demand. Moreover, Intel’s proposed alternative would require two 10Gb top of rack switch (TORS) ports in addition to the 3 NICs (2 for data, 1 for management). Those TORS ports alone could add 10-15 watts per server for the 10Gb solution that were not included in Intel’s math. But hey, it won the benchmark (well, almost)!
Calxeda is focused on providing energy-efficient solutions for real-world problems and we believe that bigger and faster is not always better. Leaner and cleaner can be less expensive and far less power hungry, lowering costs for real-world workloads which can be highly variable. Which is more representative of your real real-world environment? You be the judge.
* Based on comparing the servers w/o disks to isolate the server-power, and adding 1 watt to each 5.26 watt Calxeda node to estimate wall power, assuming a modest 24 nodes in a chassis share the power supply and fans. Note that each Intel server equipped as Intel suggests would require a PCI extension with 10 Gb NICs, and switch ports; 2 for data and 1 for management. These are costly additions ($700 per 2 ports, plus the required 10Gb TORS ports) to the IvyBridge server, and of course consume even more power. We are still optimizing our platform and Calxeda will publish a slew of benchmarks and wall-power measurements in the coming weeks.
Yesterday, Oracle made announcements regarding a few product lines including a subtle “update” release for Java SE 7 Update 6. While only an “update” for the Java community, this release is a significant milestone not only for Calxeda, but the entire ARM ecosystem. Java SE 7 Update 6 now introduces a general-purpose port of the JDK to Linux ARM. Here are a few of the highlights and some commentary on what this means for Calxeda:
- With the addition of the JDK port to ARM (previously only the JRE was available), Oracle is showing their support and belief in the emerging ARM server market.
- There is a 32-bit binary for the ARMv6 and v7 instruction sets, with both client (C1) and server (C2) compilers. That means that it a) natively supports the Cortex-A9 cores in our SOC, and b) provides a server optimized compiler.
- The ARM architecture is now treated as a “first class citizen” by the Oracle Java SE team, which means it is treated like all general-purpose JDK and JRE binaries from both a licensing and distribution perspective (under the Oracle Binary Code License):
- The ARM JDK is free for development and production use on general-purpose platforms.
- The binaries can be redistributed for free with applications targeting a general-purpose computer/server.
- With a fully supported version of Oracle Java now available for ARM servers, customers should feel confident about their Java apps running on Calxeda hardware. (While OpenJDK is available, we have seen Oracle’s JVM to be up to 5-6X faster in some instances.)
- The one caveat that remains is that Oracle Java SE 7 remains “softfloat ABI” only, which means that it will only run on Ubuntu today with Calxeda hardware. We will have updated installation instructions for Ubuntu available by end of this week.
Oracle’s commitment to the ARM architecture is a great sign of what’s to come for this ecosystem. Looking into their crystal ball, they clearly see the opportunity before them and the alignment with their strategy. Henrik Stahl, Sr. Director of Product Management in the Java Platform Group at Oracle, said it best in his blog post — when asked why Oracle is investing in an ARM port and then giving it away for free, he replied: “We have a super-secret agenda. The idea is to enable Java developers so that Java can continue to thrive, and maybe sell some middleware on ARM servers down the line.“
Want to test some cloudy code on an ARM server?
Now you can! For Free!
Calxeda, OpenStack, HP, Canonical, and Core NAP hosting have now donated the hardware, software, and facilities to provide FREE access to HP’s Calxeda-based Redstone servers using the TryStack sandbox. This is so cool; you can provision a free server instance and play to your hearts content. Upload code or images. Develop and test your software. Use the OpenStack API’s on ARM and realize “Its just Linux!”. And pay nothing. For more information on how to access this, see the OpenStack Blog
Obviously, there has been a lot of interest about this new class of servers that can dramatically reduce power and space requirements for scale-out workloads. Thats why you came to this site! Some call these “micro-servers” (Intel). Some call them “Extreme Low Power Servers” (Gartner’s analysts). And yes, some have affectionately called them “Wimpy Nodes” (See Carnegie Mellon University’s “FAWN” paper.)
We spent a lot of time at various tradeshows around the world in June and the #1 question we were asked was “when can I get my hands on a Calxeda-based server?” I am happy to tell you the wait is over.
We have been working with Boston Limited in the UK, a highly respected solution provider, for about a year to bring an excellent Proof of Concept (POC) platform to market called “Viridis”. Boston currently has about 20 customers lined up for beta testing and a pipeline of hundreds of others interested in evaluating the platform. Boston is taking orders now from users in Europe, Asia and the US with shipments beginning later this month.
The Register published a great article today highlighting the features of the Boston Viridis platform:
Boston Viridis is a perfect option for those users who want to port their code, run benchmarks, and optimize their workloads for ARM. This highly configurable solution allows users to create their ideal initial testing environments with options ranging from 4 to 48 Calxeda EnergyCore server nodes in a 2U form factor.
We look forward to working with Boston and other systems providers to enable the market with Calxeda-based POCs. Stay tuned as we learn about success stories users experience with Calxeda EnergyCore-based solutions over the coming months.
Jon Masters, Principal Software Engineer at RedHat, just finished his presentation (“Hyperscale Cloud Computing with ARM Processors”) at RedHat Summit ’12 and concluded with one of the cooler demos I’ve seen in quite some time! We sat down with Jon afterwards to get his thoughts and more detail into this very interesting topic. He’s graciously allowed us to publish his blog post here first.
Hyperscale computing is a truly exciting emerging technology that, I feel, promises great things over the next few years. It will take advantage of such technology innovations as System-on-Chip (SoC), distributed fabric technologies, and integrated systems management. These are all features that are available today in Calxeda’s EnergyCore, which powers HP’s Redstone ARM servers. A complete Calxeda server node requires only three components: the SoC, memory, and storage. Together, these can be combined into extremely dense Cloud servers. Density is important because the era of cheap single-core processor performance growth is over. While the 80s and 90s saw a 52% year-on-year growth in compute performance per core on average, we’ve since reached a limit and returned to a growth rate half of that. So, the future is multi-core, and at phenominal levels of scale. With such scale comes an opportunity to rethink the conventional server design. Not only can we integrate fabric technologies (and obviate the need for discrete networking components), but we can also redesign server systems at the rack-level to take advantages of the efficiencies of scale.
The use of ARM technology in these systems is key in another way. It brings a new level of energy efficiency to datacenter server designs, such as the Redstone. A fully loaded ARM-based Calxeda server node (including memory) draws only 5W of power. Contrast this with conventional server designs using hundreds of Watts across many discrete components, bringing significant overhead in the form of power generation, distribution, and HVAC requirements. Since the future datacenter is all about high density at scale, it will be important to design server systems with energy efficiency in mind. These systems will use less energy and so will require less energy. They will generate less heat, and will have a greatly reduced overhead in terms of the traditional infrastructure, which has been designed to run legacy servers drawing hundreds of Watts.
Low energy computing has interested me personally for a number of years. At first in the embedded space, where I have worked with devices requiring under 1W of power and running for days or weeks on batteries, but now increasingly in the enterprise server space. When we have server nodes that require only 5W of power, we open up whole new avenues of exploration – both in terms of technology, and in terms of fun! With this in mind, it seemed only natural to find a way to truly visualize the low energy aspect of these emerging hyperscale server systems. Solar power is certainly an option. It’s a well-known, tried-and-tested technology that many people are familiar with. But I wanted to find something more novel and unique, more directly connected with the user and audience. The idea was suggested: why not use a bicycle? Bicycle generators have been used to power all kinds of things over the years, but to my knowledge they’ve never been used to power servers.
With this in mind, I designed and built what I think is the world’s first bicycle powered hyperscale server rig. The rig was used during my HP Redstone Server demo at the 2012 Red Hat Summit. It consists of a bicycle, attached to a generator via a friction bearing, the output of which is fed into a repurposed solar-charging circuit. The bike generator easily produces up to several hundred Watts, which trickle charges a battery that powers the server. Some smoothing circuitry is also added to prevent damage to the server as the pedal power is applied and removed, and a fan is attached to divert any excess power produced (which happens frequently), cooling the rider down. Finally, a series of multi-meters and some custom software is used to graph the instantaneous power generated by the rig as the rider pedals away. Using this rig, I have successfully powered the HP Redstone server while generating up to 200W of power on an average bicycle. At 5W per server node, that’s a lot of ARM server nodes that can be powered by one bicycle!
We couldn’t have said it better ourselves! Check out the pictures from the demo below!
Update: Some people have asked us what was actually running on the server for the demo. Each of the four nodes was running Fedora 17 GA and a distributed Mandelbrot demo using OpenMPI across 32 cores (8 Calxeda EnergyCores). Subtle, but this was the first public demo of Fedora on ARM that we are aware of…all powered by a bicycle!