Boris Johnson has said he is facing accusations about his personal conduct because he is viewed as the person who is “helping to deliver Brexit”.
The prime minister said it was “inevitable” he would face “shot and shell” because of his stance on Brexit.
He also continued to deny allegations he squeezed a female journalist’s thigh at a lunch 20 years ago.
He said it was “very sad that someone should make those allegations”, adding they were “not true”.
Journalist Charlotte Edwardes, writing in the Sunday Times, accused Mr Johnson of touching her thigh, and that of another woman, at a lunch in at the offices of the Spectator magazine.
He has also faced questions in recent days over his ties to US businesswoman Jennifer Arcuri during his time as London mayor – he insists he acted with “full propriety”.
It is alleged Ms Arcuri received favourable treatment due to her friendship with Mr Johnson, but the PM has said there was “no interest to declare”.
Mr Johnson was speaking on the third day of the Conservative conference in Manchester.
Asked about stories regarding his personal conduct, he told BBC Breakfast: “I’ve said pretty much what I have to say on all those things.
“This is a very difficult time…Brexit is about to be done and a lot of people don’t want Brexit to be done.
“And I think, rightly or wrongly, they conceive of me as the person who is helping to deliver Brexit, and it is inevitable that I’m going to come under a certain amount of shot and shell.
“I don’t mind that in the least,” he added.
Asked in a later interview with LBC radio whether he thought stories about his behaviour were politically motivated, Mr Johnson said he did not “want to impugn people’s motives or to minimise the importance of the issue”.
But he added: “I think, generally – you asked me about why is all this shot and shell raining down on the government – I think it is because we’re going to get on and deliver Brexit by October 31.”
In her first column for the Sunday Times last weekend, Ms Edwardes said the incident took place at a lunch in 1999 – when Mr Johnson was editor of the Spectator.
“More wine is poured; more wine is drunk. Under the table I feel Johnson’s hand on my thigh. He gives it a squeeze,” she wrote.
“His hand is high up my leg and he has enough inner flesh beneath his fingers to make me sit suddenly upright.”
Ms Edwardes said another woman at the lunch later told her he had done the same to her.
On Monday, Spectator magazine commissioning editor Mary Wakefield, who is married to the prime minister’s adviser Dominic Cummings, issued a statement to say she was “not the woman referred to in Charlotte Edwardes’s column”.
“Boris was a good boss and nothing like this ever happened to me. Nor has Charlotte, who I like and admire, ever discussed the incident with me.”
After No 10 first denied the accusation on Sunday evening, Ms Edwardes tweeted: “If the prime minister doesn’t recollect the incident then clearly I have a better memory than he does.”