They report to the POTUS, but that doesn't mean the POTUS could not direct the VPOTUS to coordinate the efforts of dismantling the bureaucratic Deep State and have the cabinet secretaries report to the VPOTUS on their progress in reducing their departments. It's just a matter of delegation. And, it would give the VPOTUS something meaningful to do.
The essential American soul is hard, isolate, stoic, and a killer. It has never yet melted. ~ D.H. Lawrence