Peter Dutton has doubled down on his controversial election pledge to remove Indigenous flags from official events, saying he will "work with the state government" in NSW to take down the Aboriginal and Torres Strait Islander flag from Sydney Harbour Bridge.

The Opposition Leader drew heat from Indigenous leaders when he first made the promise last week, arguing multiple flags divided the country.

Mr Dutton repeated his justification on Thursday morning.

"The argument is how can you be united as a country if we're asking people to identify under different flags?" he told Seven.

"No other Western democracy does that, so I believe very strongly we should have an enormous amount of respect for the Indigenous and Torres Strait Islander flags but they are not our national flag."

Mr Dutton said he wanted "practical reconciliation" with Indigenous Australians.

Asked directly if he would try to remove the Aboriginal and Torres Strait Islander flag from the Sydney Harbour Bridge, he said the Coalition, if it won next year's election, would push for a change.

"We'll work with the state government … and obviously the state government has made a decision to put the flag up there, but for us, at a federal level, I'm not going to pretend that our country can be united when we're asking people to identify in different ways," he said.

"We have an incredible migrant story in this country. We should be talking more about it.

"Should be very proud of our Indigenous heritage, but I don't believe that serving under three flags can unite the country."

But pressed further on the bridge, Mr Dutton conceded the NSW state government had the final call.

"I think it's a decision for the NSW government," he said.

Joseph Olbrycht-Palmer - NCA NewsWire