"middle finger held up in a rude gesture," slang derived from 1860s expression give the big bird "to hiss someone like a goose," which was kept alive in vaudeville slang with sense of "to greet someone with boos, hisses, and catcalls" (1922), and transferred 1960s to the "up yours" hand gesture (the rigid finger representing the hypothetical object to be inserted) on the common notion of defiance and contempt.

In theatrical slang, by 1818, to be goosed meant "be hissed."

"He was goosed last night, he was goosed the night before last, he was goosed to-day. He has lately got in the way of being always goosed and he can't stand it." [Dickens, "Hard Times"] 

On Michaelmas-day, 1808, a piece called The Fortune Teller was produced and damned at the shortest notice. Harris said to Dibdin, "My dear fellow, I did not think it would do? but who would have expected it to be goosed? (hissed.) "Why, what could you expect, but goose on Michaelmas-day," said Tom. ["Oxberry's Anecdotes of the Stage, &c. &c.," London: 1827]

The gesture itself seems to be much older (the human anatomy section of a 12c. Latin bestiary in Cambridge describes the middle finger as that "by means of which the pursuit of dishonour is indicated").

type of thorny shrub with hairy fruit, cultivated in northern Europe, 1530s, with berry, but the first part is of uncertain origen; no part of the plant seems to suggest a goose. Watkins points to Old French grosele "gooseberry," which is from Germanic. Or perhaps from German Krausebeere or Kräuselbeere, related to Middle Dutch croesel "gooseberry," and to German kraus "crispy, curly" [Klein, etc.]. By either path it could be related to the Germanic group of words in kr-/cr- and meaning "to bend, curl; bent, crooked; rounded mass." Under this theory, gooseberry would be folk etymology. But OED editors find no reason to prefer this to a literal reading, because "the grounds on which plants and fruits have received names associating them with animals are so commonly inexplicable, that the want of appropriateness in the meaning affords no sufficient ground for assuming that the word is an etymological corruption."

As slang for a fool, 1719, perhaps an extended form of goose (n.) in this sense, or a play on gooseberry fool in the cookery sense. Gooseberry also meant "a chaperon" (1837) and "a marvelous tale." Old Gooseberry for "the Devil" is recorded from 1796. In euphemistic explanations of reproduction to children, babies sometimes were said to be found under a gooseberry bush.