Mac seems a bit backwards where you cannot close a dialog by default with the keyboard. You need to go into settings, keyboard, shortcuts.
In the "Full keyboard access" section, choose the radio "All Controls"
After this, you can close without saving by using the space bar, use escape to cancel or enter to save with a prompt.