We do not like the IRS because they are after our money and there are so many, many stupid tax laws. But they are chronically under-funded and mismanaged. The under-funding is intentional- our fine lawmakers do not want to enforce the wretched laws they pass, at least on the rich and influential. The mismanagement is in keeping a program going to audit little guys as part of managing "compliance." Meaning the little guy needs to fear the knock on the door, the boots kicking in the door. This is not good for the PR.
In a rational world, the IRS would pursue cases on a cost/benefit metric. But don't expect that. As for tax simplification, which is always talked about by every damn politician since the year zero, if that ever happens I promise to shoot myself immediately, as I will have reached the apex of happiness. Well I won't have those Redstone problems anyway.