The goal is to make Microsoft function more cohesively and be more efficient and innovative so it can better compete against rivals like Apple, Oracle, IBM and Google.
The reorganization, which is being implemented now, dissolved the company's five business units -- the Business Division, which housed Office; Server & Tools, which included SQL Server and System Center; the Windows Division; Online Services, which included Bing; and Entertainment and Devices, whose main product was the Xbox console.
Those business units are being replaced by four engineering groups organized by function, around OSes, applications, cloud computing and devices, and by centralized groups for marketing, business development, strategy and research, finance, human resources, legal and operations.
However, the plan has also met with skepticism among those who believe that it will lead to less accountability, less clarity and ultimately less agility.
Others maintain that the "One Microsoft" mantra at the center of the reorganization is misguided because the opposite approach is needed, namely to reorganize it into more independent operating companies because it now houses businesses and products that are too different -- like the SQL Server enterprise database and the Xbox console.
Another question concerns how Ballmer's successor handles the company's Dynamics ERP (enterprise resource planning) and CRM (customer relationship management) software business, which has been built up through a number of acquisitions.
Ballmer carved out a special place for Dynamics in his reorganization plan, saying Microsoft would "keep Dynamics separate as it continues to need special focus and represents significant opportunity." He also kept Kirill Tatarinov in place as head of Dynamics.
While Microsoft lags behind the likes of Oracle and SAP in enterprise applications, analysts see Dynamics as a conduit of sorts for selling a wide array of other Microsoft services into enterprises. Given the strategic nature of ERP and CRM systems to a business, it also gives sales representatives an opportunity to speak with CEOs, chief marketing officers and chief financial officers, as opposed to IT managers.
"Dynamics is key to Microsoft's enterprise cred," said analyst Ray Wang, CEO of Constellation Research. "I think that it was a visionary move to bring Dynamics in when he did."
However, Dynamics didn't start becoming well-integrated and aligned with other Microsoft products such as SharePoint, SQL Server and Azure until more recently, he said. "If it had happened earlier, they'd be further ahead."
There has often been talk of Microsoft selling off the Dynamics business. "I think that's up in the air," Wang said. "Dynamics set up as a stand-alone unit could be possible given how the company is currently structured. The CRM side would go to Office and the ERP stuff would be spun out."