Fears have been raised that Welshpool will become a "town of old people".
This comes as the developers behind converting Newydd Maldwyn into an 'extra care' facility addressed Welshpool town councillors.
Dylan Owen, from Social Services at Powys County Council, explained why care facilities are needed for the over 60s in Powys, and in Welshpool specifically, at an online meeting of Welshpool Town Council on Wednesday.
"The population is getting older," he said. "Young people are leaving Powys and not returning.
"In the next 18 years there will be five thousand fewer people of working age."
Mr Owen said that there's particular demand for property among people who don't want to go into residential care homes.
Welshpool, Montgomery, and the North East of Powys have been pinpointed as the areas most in need of the accommodation.
"Welshpool is one of the main places in Powys with the demand," Mr Owen said.
"In Powys there's 300 to 400 extra rooms in demand and Welshpool is at the very heart of it."
However, Councillor Alison Davies said the council was being "complacent" in its handling of young people leaving the area.
She said: "We have young people living in the community who are leaving. Young people providing something for our economy and keeping our schools opens. And Powys's reaction to this is to make more capacity for people who do not."
Cllr Davies raised concerns that the development would "draw lots of old people in" from Montgomery and the north of the county. "We're going to become a town of just old people," she said.
However, Craig Sparrow, from Clwyd Alun Housing, argued that the care facility will be "intergenerational" and that it will provide jobs for younger people.
Mr Owen said that Welshpool might have the greatest need for this accommodation not just in Powys, but across all of Wales.
Councillor Hazel Evans said: "Why does the council care so much about elderly people when there's no jobs for young people? I just want Powys to think about what they're doing."
"It's not intergenerational. It's not what many would refer to as a healthy vibrant community," added Cllr Davies.
"Maybe in terms of the over 60s, but that's a very small part of the community. That is not good for the town as a whole."
The meeting also heard discussions about the impact of the development on GP services, and changes to the plans which mean one of the buildings will no longer be demolished.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here