Hail Mary - Parallelism

iTunes is so cool! Now, in addition to music, podcasts, movies, even PDFs, you can check out university courses and programs, some of which deal with information technology. This morning, I took in the first in a series of lectures from Stanford University entitled, "Computer Architecture is Back: The Berkeley View of Parallel Computing". This specific lecture was given by Dave Patterson in January of this year and is part of what Stanford calls their Computer Systems Colloquium, a series of 14 lectures in "movie" format, available free on iTunes. If you have the 60GB "video" iPod, like me, then you can download these lectures and view them while traveling on the train or plane (uh, forget the car, please, if you're doing the driving!). To get more information, simply head over to SoiT, Stanford on iTunes U at itunes.stanford.edu.

So, what about the lecture and what is the message for computing and information technology in a manufacturing environment?

Patterson's high-level message in this lecture is that everything is changing in computer architecture, that the conventional wisdom of doubling processing speed every 18 months is over, that both hardware and software design need to invest in parallelism, and that a multiplicity of perspectives in the computer industry is required to find solutions for the new predicaments we face. The message for IT in a manufacturing environment (at least a non-computer-related manufacturing environment) is that it looks like the near future will offer far less performance improvement than we might otherwise have hoped for assuming the conventional wisdom.

The metaphor Patterson used is the "Hail Mary", itself a metaphor in football for a desperate play in which the quarterback throws a long pass hoping that somehow somebody on his team will catch it and rescue the team in the end zone. In the computer industry, the same appears to be true. Everyone is betting the farm on parallel computing to solve problems like the power wall we read about daily in the computer press. The "hail Mary" part is that the industry itself doesn't seem to know who or how the pass will be caught. And if the industry doesn't know, how the hell are IT managers supposed to figure this out?

As the computer industry moves uniformly in this new direction, IT guys like me are trying to figure out the implications for designing our systems. Although Patterson tries to make it easier by talking about the "13 dwarfs" - the areas in which multiple disciplines see the future evolving - and the computing world "interest groups" (things like high-performance computing, machine learning, games, embedded systems, and databases) where the 13 dwarfs are mapped, guys like me may feel somewhat overwhelmed.

But I think viewing this material a second and possibly third time brings on the aha experience.

One take home is that programming will have to change dramatically in the next few years. No longer will performance improvements be relegated to hardware. Programmers and developers will have to figure out how to program for parallel processing (Patterson calls this the end of "La-Z-Boy Programming"). Programmers will also have to invest heavily in the psychology of programming, in other words understanding how people interface with computers and making adjustments to make those interfaces more appropriate.

Another take home is that IT managers will have to understand their environment and make changes necessary to find incremental improvements. We can't depend any longer on new servers providing us with huge performance dividends. For example, in a database-driven environment such as ERP systems, IT managers will have to push software vendors in the direction of optimizing database structures, algorithms, reporting tools, querying tools, and integration.

Yet another take home is that knowledge of virtualization will be critical for IT managers, especially as we realize that we need to deconstruct our bloated operating systems, adding only the functions and services necessary to meet a computing objective which is, in turn, linked to a business goal.

Dave Patterson says, "No one is building faster processors." So, given the end of Moore's Law for single-core processing and given the industry's "hail Mary" approach to parallelism, why should IT managers feel optimistic? I mean, after all, are we now relegated to incremental improvements and lobbying with database and ERP vendors?

Patterson admits that the future could be one of failure. But it might also be one in which 1000 cores could be manufactured on a single, economical chip. It might be one in which experts from multiple disciplines work together to design human-centric solutions. It might be one in which the open source programming movement transforms software development by fostering development of minimalist services and functions based on minimalist operating systems and made available to IT managers in a secure virtual package. In other words, there is reason for optimism.

View the lecture for more detail, and check out the innovative Research Accelerator for Multiple Processors (RAMP).