I don't want to give the impression that I don't like "ma'am" or that I prefer "my friend."
I just thought that "my friend" was a good choice when the person isn't in a position to know (or remember at that moment) the other person's name, but "ma'am" isn't the right choice for some reason (like, you've already said it three times) or the person is looking for a more familiar approach, in an appropriate setting.
It is interesting to learn that in the UK no one says anything! They just get to the point.
I'm surprised that anyone finds "my friend" in a context like this (it was a nurse giving me a COVID-19 test, in my case) to be overly familiar. To me, it just hits my ear as a general positive all-purpose term, like "Dear ____," "Sincerely," and "Yours truly" in letters -- not meant literally, just as a conventional pleasantry. Some people even use "Neighbor." I've often heard people giving speeches addressing the audience as "My friends."
I thought it was terrific when once in Uganda a young man working at a restaurant called me "Mother." What could be more familiar than that? But it felt super respectful.