Havelock Road was closed to traffic and pedestrians were advised to avoid the area after the brickwork fell from the side of Queensbury House.
Sussex Police and East Sussex Fire were called to the scene.
East Sussex Fire said: “At 13:11 yesterday, we were called to attend a dangerous structure on Havelock Road, Hastings.
“Crews from Bohemia Road attended. On arrival, firefighters found a large amount of brick work had fallen from the 5th floor of a building. Crews used an aerial ladder platform to inspect and ensure the building is safe. The incident was handed over to a building inspector.”
Town centre manager, John Bownas, said: “We’ve been in touch with the contractor currently carrying out building works and steps are in place to make the building safe as quickly as possible.”
Stagecoach said most of its services within the town centre were diverted due to the debris falling from the building.
Sussex Police added: “Police were called to reports of a dangerous structure on Havelock Road, Hastings, around 1.10pm on Tuesday (August 3).
“Officers supported East Sussex Fire and Rescue Service with a road closure while the area was made safe.
“Police stood down from the scene once road closures were handed over to Highways.”