Jim Whitehurst, the former CEO of Red Hat who led the firm through a $34 billion acquisition by IBM and became the No. 2 exec at Big Blue, is stepping down.