Cut off from the nearest town by some 10-20 kilometres of thick forests all around, the people of Kailakuri and adjacent villages didn't have access to trusted medical care in any emergency.
Many would die without treatment, some without even proper diagnosis. And even if some would make it to some distant hospitals, they barely could afford the treatment.
Against this backdrop, Dr Baker came to Kailakuri three decades ago.
"God had sent Daktar Bhai to us as a blessing," said Sabitre Hagidok, an indigenous woman of Kailakuri.
Ever since he came to the village, not a single person died there without treatment, she said a few days after Baker's death.
Puleen Barman, a 45-year-old day labourer from Sherpur's Nalitabari, had been suffering from diabetes and tuberculosis for over a year now.
"I went to many hospitals but never got proper treatment. I couldn't go to work. With five mouths to feed at home, how long could I continue like this?
"Then last year, I came to Daktar Bhai. He diagnosed me and gave me medicines free of cost. When I came for follow-up treatment three months ago, he advised me to be admitted here.
"I feel a lot better now. But I feel sad to think that the man who did so much for my recovery is no more," Puleen said, sitting on a bench at Kailakuri Healthcare Centre.
For regular check-ups, Baker charged only Tk 10 from the patients living within a three kilometre-radius of the centre and Tk 20 from those living beyond. After check-up, patients got the required medicines, and they still do, from the centre whether they could afford it or not.
Every day, around 100 outpatients get treatment and medicines for diabetes, TB, fever, cough, burn injuries, stomach problems and complications related to pregnancy, among others.
Those needing long-time treatment are admitted to the 35-bed centre. In addition, Dr Baker trained up 97 young boys and girls as health assistants and paramedics who visit the neighbouring villages to give treatment to the sick people, especially pregnant women and newborns.
The death of Baker, who had become an integral part of their everyday life, has created a deep void in their heart.
"If I hear a knock on the door," nurse Saleha Begum said, pointing to the entrance to the hospital's outdoor, "I feel like Daktar Bhai would now appear before me and ask, 'Is any patient waiting for me?' ... He was always so careful not to keep anyone waiting."
"With Daktar Bhai's death, I feel as if I had lost a father," said Sabera Khatun of Kailakuri.
Who will fill Dr Baker's shoes now? A question that has been buzzing around since he passed away.
A doctor couple from the USA are supposed to join the centre, said Pijon Nomgin, manager of the Kailakuri project.
But the wife is expecting now and the delivery date is in February next year. After that, the family will shift here, he said.
"Besides, Gonosasthya Kendra sends medical interns here," he added.
About 85 percent of the centre's annual cost comes from donations from abroad, 10 percent from home and five percent from patients' contribution, said Noor Amin Ratan, assistant director (programme) of the project.
Baker used to collect the money from private donors, including his friends and well-wishers in New Zealand, the US and the UK.
He used to go to New Zealand once a year for visa extension and to collect money for the centre, Ratan added.
Pijon said the donors have assured them of continuing their support.
“Besides, Daktar Bhai once told us, 'Man can do everything if he has dedication. And dedication comes from faith.' We have faith in God, in people, in ourselves," he added.