Sixty homes were evacuated after two military grenades and “unidentified chemicals” were found at a property in Lowestoft.
Officers set up a 100m cordon at Normanshurst Close while an Army bomb disposal team made the site safe.
Suffolk Police made the discovery while executing a warrant just before 13:00 BST on Sunday. They are being supported by Suffolk Fire and Rescue crews.
A man in his 50s was arrested on suspicion of possession of a firearm.
The force said the outer cordon had since been lifted and only the property where the items were discovered, as well as three other neighbouring properties, were inaccessible to the public.
Marion Rabbatts, 70, said it was “a bit scary” to hear grenades were found over the street from where she lives.
She said she was about to have dinner with her daughter when the police knocked on her door and they had to “dash around” and pack up their belongings for the night.
She stayed overnight at her daughter’s house nearby.
A man who lives in the close, who did not want to be named, said he slept in his car overnight after police told him he could not return home.
Michael Grove, 71, who lives with his wife, said they were in the middle of cooking their evening meal when they were evacuated.
“We just had to grab everything and go. The chicken had come out of the oven and went with us,” he said.
Mr Grove, who has lived on the estate since it was built, said the arrested man had lived there for a number of years.
“It is concerning because you don’t expect it,” he said.
William Coleman, 91, who lives above the arrested man’s flat, went to stay with his daughter Bridget Porter.
Ms Porter, 60, said her father was “very stressed out over the whole affair”.