Protesters were on Elizabeth and Lonsdale streets in the CBD.
Flares were set off and multiple arrests made as police moved in on the rally.
READ MORE: Statewide lockdown for Victoria from 1pm
Earlier today, at a press conference announcing the expansion of the lockdown into regional Victoria, Chief Commissioner of Victoria Police Shane Patton also slammed plans for the protest.
"It's tagged as a freedom demonstration, it's anything but," he said.
"It's just selfish."
He said encrypted communications involved in the organisation of the event made it difficult to know how many people would join in, but said it was "between 200 and 5000".
He warned anybody who attended and was intercepted by police would face a massive fine.
READ MORE: Record COVID-19 case spike for NSW
READ MORE: Police flood Sydney CBD amid protest warning
Premier Daniel Andrews warned that if people ignored lockdown restrictions this weekend "well then next week it will be like Sydney", where cases have boomed to hundreds of new infections every day.
"Protesting does not work against this virus," he said.
Attendees face a fine of $5500, with several already handed out this morning to suspected would-be protesters who had been too far from home, Commissioner Patton said.