Uluru Dialogue representative and former First People's Assembly of Victoria co-chair Marcus Stewart has slammed opposition leader Peter Dutton for "playing a cruel game" with his proposal for a second referendum.

On September 3 Mr Dutton flagged a commitment to hold a second referendum for the constitutional recognition of Aboriginal and Torres Strait Islander people should the current proposal fail and the Coalition win the next federal election.

His plan does not include an enshrined Voice to Parliament.

Mr Dutton has previously endorsed a legislative Voice model if his party were to oust Labor from Government in 2025.

However his timeline to send Australians to the polling stations again remains unclear.

Speaking to Sky News he cited the Coalition's history of advocating constitutional recognition.

"My first preference is that this referendum doesn't go ahead at all because it's clear the Prime Minister is dividing the country, one in three Labor voters now aren't voting for the Voice," he said on Wednesday.

"I think most Australians want to see a better outcome for Indigenous Australians - particularly in places like Alice Springs, want to see kids go to school, want to see crime rates reduce, want to see job outcomes, people work.

"I do think there is overwhelming support for constitutional recognition. It's been the policy of the Liberal Party back to John Howard's days…it's a policy we took to the last election as well. In that sense there's nothing remarkable in what I said on the weekend."

Speaking on behalf of the Uluru Dialogue, Mr Stewart, a Nira illim bulluk man of the Taungurung Nation, said Mr Dutton is "determined to destroy" the opportunity to enshrine the Voice in just over a month's time.

He said a second referendum ignores the processes and work put into the Voice proposal, would be a waste of taxpayer money and pushes a plan with "no power to make a positive difference in the lives of Aboriginal and Torres Strait Islander people".

"Above all, it is cruel because it deliberately silences our people - again. It is Peter Dutton playing the tired, wasteful game of overriding the advice of people in communities in favour of a position that favours his own Canberra-centric political view," he said.

Anthony Albanese labelled Mr Dutton's idea "absurd", while Marcia Langton, co-author of the Calma-Langton report, told the National Press Club "there's no point" and that "it is not what we asked for".

Shadow Minister for Indigenous Affairs and Jacinta Nampijinpa Price stopped short of endorsing Mr Dutton's suggestion.

"There are a couple of bridges we've got to cross. The first bridge is getting past this referendum. And that is certainly the focus between now and October 14. There needs to be obviously further discussion as to a second referendum within party rooms and determinations made that brings everyone together in agreeance with that," she told Sky News.

"The polls are certainly showing that Australians don't want the Voice. There are a lot of Indigenous Australians who don't want this unknown entity known as the Voice, whereas recognition is a completely separate issue."

Mr Stewart noted that Senator Price did not back Mr Dutton's proposal.

"The fact that Dutton does not even have the full support of his own party including his own Indigenous Affairs spokesperson, reveals that the future of Indigenous people's lives are once again an ideological football with no rules," he said.

"The Coalition and the No campaign have no alternative, no solutions and no clue."