Around 2,000 refugees have arrived at Austria's border from Hungary, and the numbers could more than double through the day, Austrian police said on Saturday.
"Our biggest problem is that the Hungarians - after checking back with Budapest - are refusing to let our buses enter their territory and pick up the refugees," said Hans Peter Doskozil, chief of the police in the Austrian province of Burgenland.
"We offered them that they can bring the refugees directly to the trains, or to the shelter (on the Austrian side), but they just stop the buses on the Hungarian side, everyone has to get off in the rain."
Two special trains taking migrants from the Austrian border town of Nickelsdorf to Vienna were about to depart, he added.
The highway was blocked for all traffic into Austria.
Migrants are arriving in Austria after Hungary's surprise move to provide buses for them, BBC adds.
For days, Hungary had blocked migrants from travelling by train to Western Europe, saying it was obliged to register them, sparking angry scenes.
Austria says it has agreed with Germany to let the arrivals cross their borders.
It comes as European Union states are struggling to agree on how to deal with an unprecedented surge in migrants.
Buses began arriving at Keleti station, central Budapest, late on Friday, which has become a makeshift camp for thousands of migrants.
Vehicles were also sent to collect hundreds who broke away from the camp to walk on foot along the main road to Austria.
When the buses arrived to collect them, some of the migrants argued with officials, fearful they would be arrested rather than sent to Germany, the BBC's Matthew Price reported.
But later, small groups began crossing the border, with some Austrians putting up welcome signs.
The BBC's Bethany Bell saw the first group of about 30 migrants cross into Austria on foot, after the buses dropped off them off on the Hungarian side of the border.
She says some were limping, but many were smiling with relief. Austrian Red Cross workers at a makeshift centre greeted them with blankets and tea.
"I feel [at] home," said Ayaz Morad, one of the first to arrive. "This is a great land - nice people, nice government."
Mohammad, a Syrian refugee, said he was happy to leave but warned other Syrians against travelling to Hungary because the situation there was "ugly".
Many of the migrants hope to travel on to Germany, which has said it expects to take in 800,000 people this year.
The Hungarian government said it had despatched the buses so that the country's transport network would not become "paralysed".
Austria's Chancellor Werner Faymann said that after talks with his German counterpart Angela Merkel, the two countries would allow in the migrants due to the "emergency situation" in Hungary.
But he said he expected Hungary to respect any EU quotas for asylum seekers - something Hungary, along with the Czech Republic, Poland and Slovakia, has rejected.
Hungary has become a major transit nation for people fleeing the Middle East and Africa who are seeking to reach north and west Europe.
The Hungarian parliament on Friday approved tougher border controls and penalties for migrants, underlining divisions within the EU on how to tackle the crisis.
Hungary's Prime Minister, Viktor Orban, has said the surge in arrivals was "Germany's problem", since that was where most people wanted to go.
But Chancellor Merkel has called for refugees to be fairly divided among EU members.
At the scene: Matthew Price, BBC News
Our correspondent has been walking with hundreds of migrants who attempted to travel on foot to Austria.