Is it Okay to Break the Rules When Writing Short Stories?

Some writing related rules are really just suggestions.

On the day I decided I was finally going to take a short story writing course, I cautioned myself with the following words:

At least learn all the rules first, before you start breaking them.