Most of the distinction between fantasy and SF is the I-know-it-when-I-see-it type. The technical distinction that some have proposed is that science fiction doesn't break any known physical laws. Unfortunately, most of the stuff that gets classified as science fiction does break known physical laws, and the authors knew it when they wrote it. That includes such big-name writers as Heinlein, Asimov, and Niven.
I think the real-world distinction that writers, publishers, and readers use is that if it contains elves, swords, unrationalized magic (unexplained by psi, or something), vampires, or dragons, it is fantasy. That is why the vast majority of readers consider Pern and Glory Road fantasy, even though they are given SFnal rationalizations. If it contains space ships, time travel (but no other magic, other than FTL), techno-babble, or alternate history, it is science fiction. If it has swords and space ships, it is science fantasy (or space fantasy).
I used to consider SF and fantasy to be separate branches of a larger category called "speculative fiction," but considering what I have written above, as time has passed I have concluded that SF, technically, is actually a subcategory of fantasy and that speculative fiction and fantasy, properly understood, are the same thing. Fantasy is merely any type of fiction that does not take the real world, either past or present, as a starting assumption. Magic realism is also a branch of fantasy.