I’m not the most popular man in libertarian circles at the moment. However I am going to go ahead and assume this is because I haven’t presented enough evidence yet. I think I can still make a reasonable case that a lot of Libertarianism is Identity Politics for selfish white men.
We’ll start with a quote from Ludwig von Mises writing to Ayn Rand:
“You have the courage to tell the masses what no politician told them: you are inferior and all the improvements in your conditions which you simply take for granted you owe to the effort of men who are better than you.”
Perhaps not a typical definition of selfish, but arguing that the proles owe nothing but gratitude for what their Randian overlords deign to provide them implies a rather bloated sense of self-worth which may err towards selfishness. If your view of production is that some superman directs the little people to do useful things and that absent the superman they would loll around lazy and unproductive, you are going to begin to argue that what little you (clearly you are the superman) is yours, and then some more too.
But what about the white bit? Surely that implies Libertarianism is a bit…racist. That is a classic way to shut down discussion and seems very unfair, but I think justifiable. I’d hesitate to use it against specific Libertarians, unless I have some pretty solid evidence. Though if you think this is completely unfair do spend some time in Guido Fawkes of Old Holborn’s comment threads and then get back to me.
There is a sizeable chunk of Libertarian opinion, especially active in American that take an objectively pro-racist position. I mean objectively pro-racist in the same way that Orwell called mid-century pacifists objectively pro-Hitler. Moreover, many Libertarians seem to hold their principles absolute when an issue involves privileged wealthy white men, but are happy to compromise when an issue involves foreign or poor people.
We’ll begin with The Civil Rights Act in the US. This was enacted to infringe people’s property rights, it said if you own a hotel, even though it is your property, you cannot exclude blacks. Your property is yours only if you use it in ways the state deems acceptable.
Ron Paul, probably the most famous Libertarian in the world (which is to say, not very famous at all) has made his principled objection to the Civil Rights Act very clear. The right of racist property owners trump the rights of black people to live free and full lives. Despite the legislation being incredibly effective, the US is significantly less racially divided than it was even a generation ago,  Libertarian principles of freedom are too important to compromise.
Got that, no compromise with reality.
Even where the results of the policy you would have opposed have been very beneficial, the only group to really suffer were vile racists and it makes you very unpopular with those (black) people who benefited from its enactment and enforcement.
However, some Libertarians are prepared to compromise with reality. I say some, I mean Libertarians like Ron Paul himself are prepared to compromise when it means those getting stiffed are foreign people of a somewhat mocha complexion. Will Wilkinson writes mournfully of Ron Paul’s willingness to sell the primacy of individual rights down the river as soon as foreigners are involved rather than wealthy Americans.
So too does compromise come easily to Libertarians who hate taxes. Taxation is, after all, merely legitimised theft.  So no tax increase, yes? The Bush tax cuts for wealth creators must stay. So how about a tax hike which will hit many more people? Surely that will meet principled opposition from the most right-wing congress this side of the Second World War? Nope, the temporary payroll tax is to expire adding a couple of percent tax to every working American’s tax bill. Even Alex Taborrok is forced to agree with Paul Krugman’s arguments that they aren’t “right-wing” they’re just supporting their constituency, privileged selfish white men:
How can [Republicans not want to cut the payroll tax], when Republicans love tax cuts? The answer is, they don’t. They love tax cuts for the rich. Tax cuts for ordinary workers, many of whom will be those hated lucky duckies whose incomes are too low to pay income tax, are if anything something Republicans dislike.
So, lets recap, no compromise with reality when the property rights of the privileged might be infringed is happily married to a willingness to stiff the poor, the old the alien.
There is also of course that quip of Peter Theil of the Cato Institute that since women lean more leftward than men, giving them the vote was a retrograde step for freedom. That might, just possibly, be reason to call some Libertarians’ Libertarianism more than a little sexist. Will Wilkinson suggests that this sort of thing is what gives Libertarians its image problem:
Libertarianism does have public relations problems, and it’s not because most people are stupid or immoral. It’s because libertarians have done a terrible job countering the widespread suspicion that it’s a uselessly abstract ahistorical ideology for socially retarded adolescent white guys. The sadly common libertarian-conservative penchant for “brave” counter-PC truthiness (e.g., “Women do love the welfare state!” “Blacks really do have lower IQs!”) certainly doesn’t help.
I’d have to disagree, actually existing Libertarianism is asymmetrically Libertarian. It is, often, more Libertarian when the rights of wealthy, domestic, male people face oppression than when the rights of poor, foreign, “others” are affected. Of course, the temptation for Libertarians is to invoke the No True Scotsman defense:
“These people aren’t Libertarian,” they’ll splutter, “look at all the illiberal things they support.”
“Ahem, yes,” I’ll respond, “just look.”
 I think some left wing anti-Americans need to take this on board. In a generation America has reinvented itself. Racism is still a problem but in half a century we’ve gone from blacks being banned from white’s toilets  to blacks being able to become president. Not many other countries have managed such a complete turnaround in such a short space of time.
 Interestingly this toilet segregation has had some egaliratian side effects. Women’s loos need to be larger than men’s loos for the queueing time for each to be equal. Having mens toilets cover the same floor space as women’s toilets is an inegalitarian use of space because women need longer to pee (and fix their make up to look pretty for us quick whizzing lads). The only official building I know of in the world to have this egalitarian toilet design is the Pentagon because men use the old, smaller, ex-black only loos while women get the larger, ex-white only loos.
 Indeed historically this is pretty much exactly what Taxation is and how it has developed. I recommend the late Charles Tilly on the subject.