Under blood-red skies in Mallacoota town, residents fled to the beach or took up shelter in fortified homes when they heard the warning siren go off at 8 a.m., the BBC reported.
The fires swept through the town destroying numerous buildings, but was kept back from the shore by the change in wind.
Firefighters had gathered at the shore as a last line of defence.
Victoria's state emergency commissioner Andrew Crisp told reporters there were "4,000 people on the beach".
Steve Warrington from the fire service said: "It is pitch black, it is quite scary... The community right now is under threat but we will hold our line and they will be saved and protected."
He said there had been "significant property losses" across the entire East Gippsland region in the past days.
Victoria Premier, Daniel Andrews said on Tuesday that navy ships might be called upon to provide food, water and power to the cut-off townships.
Currently, over a dozen "emergency-level" blazes span a 500 km stretch across two Australian states - from Batemans Bay in New South Wales (NSW) to Bairnsdale in Victoria.
Several holiday spots along the coast have been cut off and the main road in the region - the Princes Highway - has been closed.
Residents in the NSW holiday towns of Bermagui and Batemans Bay also fled on Tuesday morning to the waterfront or makeshift evacuation sites near the shore.
Locals told the BBC they had "bunkered in" as the front approached, raining ash on the beaches.
"It was bloody scary. The sky went red, and ash was flying everywhere," a resident in Batemans Bay said.
Meanwhile, officials confirmed another two people had been killed by fires in NSW, bringing the fire-linked death toll to 12.
Authorities have said that four people were missing in Victoria and another in NSW.
The bodies of the latest victims - believed to be a father and son - were found in the NSW town of Corbargo, which was hit by a massive blaze on Tuesday.
"Very tragic set of circumstances," the BBC reported citing NSW Deputy Commissioner of Police Gary Worboys as saying.
"(They were) obviously trying to do their best with the fire as it came through in the early hours of the morning."
Hundreds of massive blazes have destroyed millions of hectares in the two states since September.
Temperatures exceeded 40 degrees Celsius in every state and territory at the start of the week, with strong winds and lightning strikes bolstering the flames.
( With inputs from IANS )