I've thought about this a lot myself. Please enjoy my ramblings on the topic and debate/discuss!

I think you'd find the reasons are as varied as the participants, but perhaps there are core motivators which can be identified.
For both men and women there is a societal taboo factor which may attract.
We also have a strong big dick/tight twat image culture that people may feel they need to live up to. If the fit isn't so tight then perhaps either party could find "alternate" ways to live up to the ideal?
For men you can cite "vagina envy" although I think that's a very narrow label and not very useful.
Obviously I can only speculate from a woman's perspective but I might imagine the core motivation being the parallels with childbirth, the primary motivator and ultimate goal of sex for a woman.
Of course many do say that bigger is better and I guess there's not really any other part of their partner which can match for size or dexterity than the hand(s). So there's the physical reasoning - perhaps bigger and more dexterous simply is better for the pure stimulation factor, getting past the very general and PC "any size is fine".
From the male perspective there is also the innate size factor you could argue is based in biology before society: bigger is better in the male brain at least (penises, cars, muscles, bank accounts) in all cultures so there is a strong psychological drive to make their possessions, embodiments, status symbols bigger to increase their perceived value. Perhaps this also applies somewhat to their or their partner's vagina/anus although I think there are stronger/more obvious motivators here. (I think it neatly helps to explain the stereotypical "male pig" though, taking their capacities to the extreme with pumping/stretching whereas most women don't push the boundaries as much I find)
The evolutionary spanner I'll throw in here with point form key ideas:
1) Human infant's head circumference is comparatively huge in animal kingdom - our intelligence speaks for itself in terms of advantages it has brought us
2) Human pelvic diameter is also large (obviously necessary to accommodate the large infant's head with a side effect of a increased vaginal capacity and perhaps even contributes to cultural factors seeing big babies come out of there)
3) Human penis is largest amongst primates compared to body size suggesting it has been selected for, logically either because a bigger one is more effective for reproduction or that females prefer it bigger for "some reason"
From an evolutionary perspective, big breasts and big vaginas are desirable and attractive to males because they suggest a female who will be successful at childbirth and mothering.
Perhaps though I'm just taking a bunch of bullshit

I think there is some biological reasoning amongst my ramblings for the "bigger is better" camp. Personally I believe that it's true up to a point, and that our penises are big but perhaps not big enough to have caught up to our big human vaginas!

I'm sure through surgery or genetic engineering or what have you, men will all end up with what amounts to two coke cans stacked up and perhaps that is closer to "ideal" (for pleasure purposes) at least for an open minded and well initiated woman... LOL
My opinion is that the penis is not ideal for pleasure, simply a compromise between that and reproductive success. IMO the hand is a superior sex organ to the penis for the receiver, at least in terms of potential pleasure it can give. Further, the tongue is a strong contender!
Personally, I first tried playing around with my own arse because I reasoned that gay guys enjoyed it "for some reason" and women liked to be fucked so why shouldn't I similarly enjoy the sensation? Feelings that I was doing something wrong or damaging myself held me back but over time I loosened up physically and figuratively to discover that being fisted by the woman I love is one of the most enjoyable experiences in my life, and research in various areas confirmed the mechanical explanations why that might be so. Funnily enough I believe as a man I enjoy it largely because the anatomical differences between genders aren't so significant as people imagine.
I want to give her the best experience possible and I genuinely believe that may lie within the palm of my hand so to speak
