In the middle of Key West Harbor, there’s a modern-day Gilligan’s Island just a short dinghy ride from Schooner’s Wharf. It’s 21 acres of wild green space with a rusting barbecue grill and an empty beer keg swinging from a tree.

And if you thought the fictional island on the 1960s TV show harbored a quirky assortment of characters, get a load of the ones who’ve figured in Wisteria Island history:

There was Key West luxury developer Pritam Singh, who lived on the rent-free paradise as a hippie.

There were the pot-smokers, partiers, poets and picnickers — as well as artists, vagrants and nature-seekers — who used it to get away from civilization for hours, days or even weeks at a time.

Navy SEALs trained for secret missions on its terrain. Thieves used it, too, as a chop shop for stolen outboard motors.

And don’t forget Dennis Walsh, a guy who sold dirty jokes for $1 to tourists on Duval Street and enjoyed walking his dog on the island’s beach.

Over the past four decades, it seems like just about everyone has used the island except its acknowledged owners: the Bernsteins, a well-to-do family with New York roots who developed most of nearby Stock Island. “We kind of ignored it,” Roger Bernstein said.

But in 2007, Bernstein and his brother, Jordan, decided it was time to turn their “trophy property” into a luxury resort. Those who loved the scruffy island — created during a Navy channel-dredging project around the turn of the last century — had other ideas.

Among them were Naja and Arnaud Girard, who own a Key West marine assistance and boat salvage company. The couple have fond memories of their two children playing on the island with kids from other live-aboard families who grew up on boats anchored off its shore.

The Girards started poking around Washington, D.C., archives and online databases, trying to find a way to keep at least some of the island undeveloped. They hit the jackpot: documents that suggested the Bernsteins could not own the island because the Navy had not given up title to Wisteria until 1982. And, when it did, it transferred it to the U.S. Department of the Interior, not the state of Florida.

“This came out of nowhere,” said Barry Richard, an attorney with Greenberg Traurig, which is representing the Bernsteins. “I have to say it came out of a relatively low administrative level . . . and may well be, at some point, reversed administratively. But it is not a risk we can take.”

As a result, Key West’s local drama has turned into a federal case: F.E.B. Corp. vs. the United States of America.

The ownership controversy prompted the Monroe County property appraiser to slash the island’s valuation to $17,900 (it once was $700,000), and it has brought to a halt the Bernsteins’ effort to turn Wisteria into a resort that mirrors its twin spoil island, Sunset Key.

The Bernstein brothers are shocked that the feds are disregarding the warranty deed their late father, Ben, obtained for $155,000 in 1967, the four decades worth of property taxes they’ve paid and the Submerged Lands Act of 1953, which they believe confirms their ownership.

“This is a land grab by the federal government of epic proportions,” Roger Bernstein said.

But according to Naja Girard, the “real land grab” occurred in 1951, when the state auctioned off the island. She dug up documents that showed the Navy claimed ownership at the time and objected to the auction. The state went ahead anyway.