I'm bemused by the idea that Peter Hollingworth cannot or should not be dismissed. I'm not a lawyer so I invite correction if I'm wrong at law, but really it seems to me the problem is political rather than legal.
The Constitution's only reference to appointment and dismissal provides:
2. A Governor-General appointed by the Queen shall be Her Majesty's representative in the Commonwealth, and shall have and may exercise in the Commonwealth during the Queen's pleasure, but subject to this Constitution, such powers and functions of the Queen as Her Majesty may be pleased to assign to him.
The Queen acts on the advice of the prime minister of Australia on this as on any other matter in relation to Australia. Similar language appears with respect to the ministers (including the prime minister) at Sections 62 and 64.
Talk of legal grounds for dismissal actually turns quite quickly into statements like Ken Parish's:
Creating a precedent for dismissal of a Governor-General for conduct (short of proven moral turpitude) committed otherwise than in office would be best avoided,
I know it's not Ken's positions but a lot of commentary is getting close to the deeply weird position that the governor-general should not be dismissed because the dismissal would confirm the arbitrary powers we already know the prime minister holds. The solution is first to terminate this governor-general so we can find someone capable of representing Australia and second reviewing the whole shoddy system by constitutional amendment. Somehow I suspect the 'ain't broke, don't fix it' vote might be significantly lower at the next referendum.