Turnbull said the titles were "not appropriate" in modern Australia, and that Queen Elizabeth had accepted the cabinet's recommendation to drop them.
Former PM Tony Abbott reintroduced knighthoods and damehoods in 2014.
His controversial decision to grant Prince Philip a knighthood in January was widely seen as one of the factors which ended his term as leader.
Abbott at the time said Australia was honouring the Duke of Edinburgh's lifetime of "service and dedication", but later said the decision was "injudicious".
He was ousted as leader of the centre-right Liberal Party by Turnbull in September.
Analysis: Jon Donnison, BBC News, Sydney
The decision to give Prince Philip a knighthood was hugely unpopular in Australia and seen as one of the biggest blunders of Tony Abbott's brief time in office.
Now the new Prime Minister Malcolm Turnbull, who in 1999 led Australia's unsuccessful campaign to abandon the monarchy and become a republic, has dropped knights and dames.
The prime minister will have the chance to discuss the matter with the Duke of Edinburgh's son later this month when Prince Charles comes to Australia on a royal visit.
Turnbull has recently said he remains a republican but says he does not regard the issue as a priority for his government.
End of 'Knightmare' poses republican questionIn a statement, Turnbull said the Order of Australia awards were "an important way of honouring the achievements and service of many Australians, including those unsung heroes who might not otherwise be recognised outside their local communities".
But he said his cabinet had recently reviewed the system and decided that the knighthoods and damehoods were "not appropriate in our modern honours system".
Existing knights and dames would not be affected by the change, he said.
The opposition Labor party welcomed the move, with shadow treasurer Chris Bowen describing the titles "a national disgrace".
But he said it was "not appropriate" for Australia to be "clinging onto imperial Britain through our honours system".
"We shouldn't be celebrating the fact that knights and dames are gone, we should be lamenting the fact that they came back under this government," ABC News quoted him as saying.
Alongside Prince Phillip, three people have received the honour since 2014 - former News South Wales governor Marie Bashir, former Governor General Peter Cosgrove and the former Defence Force chief Angus Houston.
Australia's honours system
- Australia began awarding its own honours in 1975 - the awards eventually replaced the existing British honours system.
- Anyone can nominate an Australian citizen for an award for service, excellence or achievement.
- The awarding of knighthoods and damehoods was discontinued in 1976 but brought back very briefly in 1986 - Tony Abbott reinstated them in 2014.
- Republicans had said the honours system was an outdated remnant of colonialism.