Victoria's peak Aboriginal health body has urged the State government to take immediate action to address deaths in custody after the death of a 38-year-old Gunditjmara and Wiradjuri man at Loddon Prison in Castlemaine.

The call follows a string of deaths nationwide in recent weeks, including Clinton Austin this month, and a 32-year-old Aboriginal man at Port Phillip Prison in August.

Mr Austin's family said they could not believe he was dead.

"We don't know how we will face the next days," they said.

"Clinton was meant to be getting out of prison. He had his heart set on being an Aboriginal art teacher. His heart was for his son and for his art."

Victorian Aboriginal Community Controlled Health Organisation chairman Michael Graham said the community-controlled health sector should be used as primary providers for vulnerable Indigenous people in custody.

"As a result of this tragedy yet another Aboriginal family is left to deal with the consequences of a system that continues to see members of our community imprisoned at rates disproportionate to those of the general population," he said.

"It is completely unacceptable that we continue to see our loved ones and community members die in custody on a regular basis. This cannot and must not be accepted as the norm.

"This latest tragedy is a reminder of the need for urgent work to be done to change a system that is clearly failing Aboriginal and Torres Strait Islander people."

Mr Graham said that culturally safe healthcare was the "heart" of much-needed reform.

Victorian Aboriginal Legal Service chief executive Nerita Waight called for an inquiry into Mr Austin's death.

"We hope that the coroner's court of Victoria will thoroughly investigate Clinton's death and will be respectful towards and prioritise the voices of Clinton's family throughout the coronial investigation process," she said.

"VALS has serious concerns about the quality of healthcare provided to Aboriginal and Torres Strait Islander people in prisons in Victoria."

A report into Mr Austin's death which will examine contributing factors is being prepared.

A Victorian Government spokesperson said more needed to be done to address over-representation of Indigenous people in the justice system.

The 2022-23 Victorian Budget outlined more than $76m for programs to address the causes of offending and the over-representation of certain groups in the adult corrections system.

The programs include expanding the family visits support and improved rehabilitation and reintegration services.