Crystal sets sometimes actually worked better in some places using the "Ground" as the "Antenna". Your "crystal" (if it wasn't a diode) survived because the current reversed... and it did what it was suppose to. If it was a true galina (lead crystal) then I've seen them survive a lightning strike on the antenna that fried the "cat's whisker". (Yes it was spark gapped. Jumped the gap and vaporized the spark plug... or at least blew it somewhere.) At about age 12 my life changed when a Lafayette store opened in town. My career in serious electronics (besides changing tubes in TV's) started by building their kits. At the time Radio Shack was second banana.And, the crystal radio I had clipped to it didn't get fried, either. Weird. (No, I don't know why I had a crystal radio clipped to it. Likely because that was "what I had" and a Mad Scientist uses what is at hand.) The fuses finally tripped, though.
What killed Layayette was they were too into vacuum tubes.
Ahhh the good old days when I could drag a neighbor's TV into the garage and nine out of ten bring it back to life. Know what I mean?