My latest thought was to set up an email-based one-time password system so people could log in, RSVP, and select whether they want their RSVP to appear publicly or not.

I will probably experiment with this on my own site, but I could also see it as a Bridgy-like service that generates an RSVP post with mf2 and sends a webmention on your behalf.