Interviews (in-person, phone, or video) are just as much the company interviewing the candidate as the candidate interviewing the company. The interviewer saying, "I am available on Tuesday at 2pm" (ie. take or leave it style) would not bother me. Usually that is done to prevent all the back and forth and to get the interview done as quickly as possible. It feels a bit presumptuous to me (like we expect the candidates to drop everything for us), but I see it enough to acknowledge that it's how business is done. (In most cases when the managers do this, they are willing to suggest other times if the individual is not available.)
The not calling when scheduled? Big red flag. Minus a major issue followed by a prompt apology, to me this means that the manager does not value the time of others. The receptionist confirmed to you that this was not a one off thing. I don't know that I would read into the receptionist's attitude. She may not be in a position to change or comment on the practice. And she didn't say that's how the company does things, but just this manager. I would probably not want to work for this manager and I would proceed cautiously with this company in the future, but I wouldn't rule them out from this one interaction.
I don't think that you're wrong to be put off. This might cause me to cut ties and not want to work for this manager (and maybe the company). But depending on other things, I may have continued with the process to see what else I could learn. I will say that this sort of thing almost always ends with me deciding that the manager is not going to mesh well with me.