For instance: if you have an adult tumblr, now you can’t even search your own blog to find an old post.

I’m getting panicky emails from people with huge adult Tumblrs, thousands of posts. Apparently internal Tumblr search has never worked well (you can search for one tag, or for blog names, but not for post content and there are no multi-keyword searches) and it’s impractical to scroll back very far in your own Tumblr dashboard. So they were in the habit of typing [keyword] [their own tmblr url] into Google, and hey presto! There would be the post they were looking for.

Now their blogs have a non-consensual robots.txt file that excludes Google, and all of those search results are gone from Google.

Worse yet? Tumblr blogs flagged “adult” aren’t searchable even with Tumblr’s own internal search. You can test this yourself. Log into your Tumblr dashboard, go to your settings, and make sure you haven’t checked the “Browse tag pages in Safe Mode (Hide content from NSFW blogs)” setting:

Unlike the one that doesn’t actually “allow search engines to index your blog”, this checkbox appears to actually work in the narrow sense that if it is not checked, you can search for blogs flagged “NSWF” within the Tumblr tag search interface. But this checkbox lies by omission. You’ve got the option to search tag pages of NSFW blogs (or not) but opting to search them does not let you search blogs that have the deeper-level-of-perdition “adult” flag.

My test for this is to search for a recent post at Wicked Knickers, which I used as my “adult” flagged example in the Thou Shalt Not Search Adult Tumblr Blogs post:

The post we will be looking for in the Tumblr dashboard tag search has a time stamp of 9:30pm yesterday, May 18, and is tagged “ziegfeld” which makes it a nice handy and recent thing to search for.

We already know that Google no longer has access to the posts on an adult-flagged Tumblr like this:

So, what happens in the Tumblr tag search interface? If you’re logged in, this is what you see when you search for tumblr posts with the “ziegfeld” tag. The posts returned are listed in date order (most recent first) and dates are visible as tooltips on the live page, so I’ve added them in the margin with red arrows and white text. You’ll see that the Wicked Knickers post is not returned by the Tumblr search:

Interestingly, that logged-in Tumblr dashboard search result is displayed at a URL ( http://www.tumblr.com/tagged/ziegfeld ) that returns something very different (but still no sign of our Wicked Knickers post) if you navigate to it as a not-logged-in person:

Try it yourself if you’ve got an adult-flagged Tumblr blog. Log in and try to search for your own posts in the search box on your own dashboard. You will, sadly, fail.

So, what is to be done? How can you search your own Tumblr blog?

The answer is, quite simply, you cannot — not while it’s on Tumblr’s server behind their robots.txt that you do not have the power to alter or remove.

But, all is not lost. Be ye not in despair. If you could only back up your adult Tumblr blog — make a complete copy of it, on your local hard drive — you could search it there with any file searching tool. Or, if you have a web server of your own, you could upload that copy (mirror it) onto your own web space, where it would once again be indexed and searched by Google.

That’s all I got. It’s the only way. It’s also a very good idea, because eventually The Pornocalypse Comes For Us All, and because Anything Worth Doing On The Internet Is Worth Doing At Your Own Domain That You Control.

Your next logical question is “But how do I do that? How do I back up a Tumblr blog?”

It’s not a simple question. The answer isn’t simple either. But, it can be done. So, that’s my next post.

Similar Sex Blogging:

• Yahoo To Buy Tumblr?
• Thou Shalt Not Search Adult Tumblr Blogs
• The Pornocalypse Comes For Us All

Well, here’s a Time report that Tumblr is in talks to be acquired by Yahoo for big bucks. A potential acquisition like that would certainly explain the urge to scratch kitty litter hastily over the porn that made your system big enough to sell in the first place:

Internet icon Yahoo! is in talks to buy New York-based social blogging platform Tumblr for as much as $1 billion, according to multiple reports. At that price, Tumblr would be pretty expensive, given that it reportedly only booked $13 million in revenue last year, but the deal could still make sense for Yahoo! That’s because Tumblr is extremely popular with the 18-to-24 year-old-set, precisely the demographic CEO Marissa Mayer is targeting as she attempts to turn the purple-hued Internet pioneer around following a multi-year slump.

It’s like I said in The Pornocalypse Comes For Us All:

But Tumblr is, famously, a popular platform in search of a revenue-generating business model. And we’ve learned that the suits have no loyalty to the porn users who made their platform popular. So, my bold prediction is that as Tumblr casts about for a business model, one of their steps will be to “clean this place up” (for the VCs, for the advertisers, for the potential buyers, for somebody). A lot more porn tumblrs will go away when that happens.

The pornocalypse comes for us all.

Note well: Yahoo itself is no friend to adult content. As early as 2001 Yahoo started hiding adult Yahoo Groups from its own directories and site search, making them very hard to find. And adult Yahoo Groups used to be (it’s been some years since I stopped paying attention) frequently deleted, seemingly at random and without any notice or hope of appeal, forcing the group members to reconstitute themselves on other services or in new, temporary, Yahoo groups. If Yahoo buys Tumblr, the adult Tumblr ecosystem is in for a rough ride.

Update:

According to sources close to the situation, the Yahoo board plans to meet Sunday night to decide whether to approve a $1.1 billion all-cash offer for New York-based blogging site Tumblr.

Update, via Violet Blue’s sex news:

If Yahoo Buys Tumblr, What Will It Do With All That Porn? (Businessweek)

The sales copy is perhaps even more eye-opening:

If you were ever curious about whether you can milk a bull, this device answers with a roaring “YES!”

The Stud Finder™ is newest addition to our stable of prostate stimulation devices, but unlike the more conventional silicone, rubber, or plastic p-spot toys, this massive metal bad boy is daunting to look at as well as feel. 12 inches of solid stainless steel end in a heavy curving bulb designed to activate the male prostate gland and get your juices flowing, whether you want them to or not!

Prostate milking is a technique permitting the expression of fluids without necessarily triggering male orgasm, so this device is great for keeping your male slave healthy without offering him orgasmic relief – though using it doesn’t exclude pleasure at all! This heavy duty rod is destined to be all up in your arsenal for years to come!

Similar Sex Blogging:

• Home-3D-Printed Sex Toys: Not Yet, And Not On Thingiverse
• Douching For Fun
• Giant Penis Plug
• ZOMG, It's An Anal Pear!
• Prostate Milking: What It Feels Like
• Half-Cocked Canadian

My generation doesn’t treat consent as a lodestar merely because consent permits pleasurable sexual activity that more traditional sexual codes would prohibit. The ethos of consent is regarded as a lodestar because its embrace is widely seen as an incredible improvement over much of human history; and because instances when the culture of consent is rejected are superlatively horrific. The average 30-something San Franciscan has had multiple friends confide to them about being raped, and multiple friends confide about participating in consensual BDSM. Only the former routinely plays out as extreme trauma that devastates the teller for decades. Little wonder that consent is treated as the preeminent ethos even by many who suspect that transgressive sex like what Witt describes is ultimately unwise or even immoral.

Let us imagine that, 50 years hence, we have a society where the ethos of consent and attendant norms of sexual conduct have triumphed so completely that rape is as rare as cannibalism. Everyone would regard that as a civilizational triumph. Would it be a bigger or smaller triumph of sexual mores than a culture where consent was valued exactly as much or little as it was in 1950, but BDSM and kink, extreme or tame, was so widely rejected as to render it as rare as cannibalism? That I’d strongly prefer the former triumph explains why I cannot agree with Alan Jacobs when he writes of the San Francisco pornographers, “I do not believe that it is possible to be more uncivilized than they are, though one might be equally uncivilized in different ways.”

I think rapists are far more uncivilized, and that every champion of consent, however myopic they are about other moral norms they ought to follow, are trying to build “structures of thought and practice that harness humankind’s sexual instincts and direct them in socially up-building ways.” Consent isn’t, after all, entirely separable from other widely accepted norms of civilized behavior. Taking it seriously means refusing to watch certain types of porn (the hidden up-skirt camera, for example); it means being forced to conceive of every potential sexual partner as an autonomous individual with inherent worth and desires so important that they frequently trump yours; it means, in at least that one respect, treating other people as you’d want to be treated.

None of that means one must approve of the acts described in the San Francisco basement. I happen to think it doesn’t in fact threaten civilization, that transgressive sex cannot, by definition, become the norm. Others may differ, and I’m just guessing there; but it is to say that, whatever you think of the porn shoot, the scattered, unconsensual sex that went down in the Bay Area that night was more worthy of condemnation, more uncivilized, more destructive and less moral.

To me, the fact that Friedersdorf felt consent culture needed defending in the conversation says rather a lot about the conversation itself. Friedersdorf himself is at pains to disclaim any suggestion that his interlocutors “are insufficiently horrified by rape” — but how else are we to parse that “impossible to be more uncivilized” remark by one of them?

Similar Sex Blogging:

• Yet Another "Shooting For Kink.com" Story
• Consent Every Time
• Head-In-The-Toilet Sex
• Radical Feminists Of Gor
• Evil Porn Werewolf Enslavers Debunked

Two weeks ago in The Pornocalypse Comes For Us All, I wrote:

Who is next? My guess would be Tumblr. Tumblr is, of all the big platforms, perhaps the most porn friendly; there’s lots of porn on there and the Terms of Service do not prohibit it… But Tumblr is, famously, a popular platform in search of a revenue-generating business model. And we’ve learned that the suits have no loyalty to the porn users who made their platform popular. So, my bold prediction is that as Tumblr casts about for a business model, one of their steps will be to “clean this place up”…

And now, guess what? I’ve discovered that Tumblr uses robots.txt to bar all search engine access to blogs flagged as adult. If you’ve got an adult Tumblr, go look at your own settings. Do you see that first checkbox, the one that says “allow search engines to index your blog”?

That checkbox is a lie. It’s nicely checked, it’s not greyed out, but if your blog is flagged “adult” it’s a lie. Do you see the “Learn more about what this means” link under “Your blog was flagged NSFW” selector? It leads to this page, where Tumblr requests users to appropriately self-flag their blogs:

Please respect the choices of people in our community and flag your blog as NSFW or Adult from your blog Settings page.

• NSFW blogs contain occasional nudity or mature/adult-oriented content.
• Adult blogs contain substantial nudity or mature/adult-oriented content.

If you’re not sure if you should flag your blog you can leave it unflagged, but keep in mind that we might flag it later if we see a lot of mature/adult-oriented content.

To answer the question “What happens to blogs that are flagged NSFW or Adult?” Tumblr offers this handy chart. The key piece of information is the white space indicated by my red superimposed arrow:

That’s right — where the “Blog indexed by Google” row intersects the “Adult Blogs” column, we find a ringing silence.

Would you have noticed? None of the adult Tumblr bloggers I know ever did. I knew from my porn researching that adult Tumblrs tended to be poorly represented in Google search results, but I chalked it up to the sheer scale of Tumblr and Google’s growing bias against returning porn search results. Nope, I found out the truth in one stark moment of astonishment, summed up by this image:

Let’s click the “See wickedknickers.tumblr.com robots.txt page” link:

From me: Aghast. Fucking. Gulp.

In robot, that means, roughly “All robots: stay out!” No search spiders allowed. No Internet Archive crawler. The Wicked Knickers tumblr is there, but you have to know about it, or you have to be linked to it. You won’t find it in Google, you won’t find it in any other search engine that honors robots.txt, and when Tumblr decides to stop hosting it, you won’t find the pages in the Wayback Machine — it will be gone for good, lost to humanity unless somebody with the technical chops and outlaw sensibilities of Archive Team finds a way to archive it anyway, robots.txt be damned.

Wicked Knickers is just an example, one that has some meaning to me because it’s one of the first Tumblr blogs I ever noticed, and I’ve been linking to it since 2010. That’s almost 6,000 vintage erotica posts since January 2009, and none of those pages are in Google or the Wayback Machine. It was only when I twigged to that anomaly that I finally understood what Tumblr is doing to adult blogs.

In all the years that I’ve been preaching Bacchus’s First Rule (“Anything worth doing on the internet is worth doing on your own domain that you control”), I’ll confess that I never considered the power of robots.txt, or what it means to be putting stuff on an internet site where somebody else controls what robots.txt says. Not only do they control your visibility to search engines, they control whether history will remember what you said. That strikes me as a high price to pay for a “free” blogging platform.

It’s worth noting that there’s still rather a lot we don’t know about the Tumblr robots.txt blockade on adult Tumblr sites. Unanswered questions include:

• Does Tumblr have any flexibility on this? Would their support, if asked, remove or modify the robots.txt barrier in specific cases?
• When did Tumblr start using robots.txt to block Google from adult blogs? Has it always been like this, or is it a recent innovation?
• Why does Tumblr display the misleading checkbox that falsely implies that search engines can see flagged adult blogs?
• What is the actual reason for excluding adult Tumblrs from search engine and (especially) archive crawls?

In an unusual move for me, I actually reached out to press@tumblr.com, told Tumblr I was going to write this post, and asked them for answers to those questions. That was on May 11th. No response so far. If they ever do answer, I’ll be sure to update this post.

Similar Sex Blogging:

• Google Glass: No Fuck For You!
• The Pornocalypse Comes For Us All
• Son Of "Anything Worth Doing..."
• Web 2.0: Spammed Into Oblivion
• Seven Things Google Doesn't Mind You Searching For
• Cory Doctorow On The Value Of Links
• Why Blogging Services Suck

It’s a work in progress. A worthy one.

Similar Sex Blogging:

• Reading Sex Tarot With Annie Sprinkle
• Tentacles Are All The Rage
• Tentacle Sex, With Suckers
• Tentacle Sex, Pinup Style
• Tentacle Sex, Shunga Style
• Yet Another Sort Of Tentacle Sex
• The Fruits of A Good Hard Drive Cleaning