International Business Machines Corp. reported disappointing revenue in two key software categories, including its closely watched Red Hat unit, sparking concerns among investors who see those businesses as essential to the company’s growth.