Should a mid-term cabinet replacement need a fresh Senate confirmation?

Vacancies are often filled by acting officials who never face a confirmation vote. Should the Senate always get a say?