Friday, July 19th, 2013 -- by Bacchus
I was among the first to discover back on May 15 that Tumblr was using an exclusionary robots.txt file to hide the contents of blogs flagged “Adult” from all search engines, the Internet Archive crawler, and any other internet service that respects robots.txt files. A few days later when I was poking at that unpleasant fact, I also discovered that Tumblr was excluding these blogs from its internal tag-search function:
Worse yet? Tumblr blogs flagged “adult” aren’t searchable even with Tumblr’s own internal search. You can test this yourself. Log into your Tumblr dashboard, go to your settings, and make sure you haven’t checked the “Browse tag pages in Safe Mode (Hide content from NSFW blogs)” setting:
Unlike the one that doesn’t actually “allow search engines to index your blog”, this checkbox appears to actually work in the narrow sense that if it is not checked, you can search for blogs flagged “NSWF” within the Tumblr tag search interface. But this checkbox lies by omission. You’ve got the option to search tag pages of NSFW blogs (or not) but opting to search them does not let you search blogs that have the deeper-level-of-perdition “adult” flag.
I even proved it with a careful set of screenshots:
My test for this is to search for a recent post at Wicked Knickers, which I used as my “adult” flagged example in the Thou Shalt Not Search Adult Tumblr Blogs post:
The post we will be looking for in the Tumblr dashboard tag search has a time stamp of 9:30pm yesterday, May 18, and is tagged “ziegfeld” which makes it a nice handy and recent thing to search for.
So, what happens in the Tumblr tag search interface? If you’re logged in, this is what you see when you search for tumblr posts with the “ziegfeld” tag. The posts returned are listed in date order (most recent first) and dates are visible as tooltips on the live page, so I’ve added them in the margin with red arrows and white text. You’ll see that the Wicked Knickers post is not returned by the Tumblr search:
Thus you can imagine my surprise when Twitter started blowing up yesterday with outraged Tumblr-users who had only just discovered that their adult blogs had gone missing (2-3 months ago) from the Tumblr tag search interface. Of course it was all over Tumblr as well. I didn’t pay it any mind; I was busy yesterday and figured it was just one of those moments when public consciousness crystallizes about a long-established injustice.
It wasn’t until this morning, when I finally had time to get caught up, that I discovered what had caused the moment of crystallization. At first I thought it was the Daily Dot article that appeared yesterday, which I only skimmed at the time due to it being such ancient news to me: New NSFW content restrictions enrage Tumblr users. But then this morning Violet Blue published a similar (but much better, and I’d say that even if she hadn’t quoted my May reportage in detail) article at ZDNet: Adult Tumblr blogs now removed from every form of search possible. Violet’s too good a reporter to jump on a bandwagon just because it’s starting to speed up, so I took a second look at both articles to see what, if anything, had actually happened recently and wasn’t old news. Finally I twigged to it. Although the Tumblr internal search (which has always been a tags-only search) hasn’t revealed content from blogs flagged “adult” for months, the “new thing” is that Tumblr’s cryptic little internal checklist has finally (yesterday? Nobody seems to know just when) been updated to reveal that fact. Here’s the box as it existed on May 15, with my red arrow:
And here’s the box as it is today, which Violet linked to and the Daily Dot printed:
I’ve outlined what’s actually new in the graphic. Once again, it isn’t new policy at Tumblr; these are the rules since (near as I can tell) some time in February. What’s new is that Tumblr is now admitting what the rules have been for some time.
Now we come to the happy fun-time “evasions and denials” section of this post!
Remember, first, that Tumblr’s internal search has always been a tags-and-titles-only affair. So, look again at that line in the first red box of mine: “Posts appear in tag pages and search pages for logged-out users.” They are playing games with multiple pairs of yes-no variables here, for maximum confusion. The one checked box is for unflagged blogs; the unstated obverse is that “posts on flagged Adult and NSFW blogs do not appear for logged-out users”. Fair enough so far. What does the statement “do not appear for logged-out users” imply? Well, it implies that the posts do appear for logged-in users, which would make this a fairly benign (if still nanny-ish) attempt to make sure everybody who sees porn has opted-in to see it.
But that benign implication is false. Remember the other new (red boxed) line in the graphic? It says “Blog indexed by Tumblr search” and shows an unchecked box under “Adult”. So, in the first red box they say “you can’t see adult adult or NSFW blogs when logged out”, implying that logged in users can see them. But then in the second red box they carve the adult ones away, because they admit that Tumblr search (which is tag search) won’t index these blogs at all, meaning that logged-in users won’t see them either. NOBODY WILL SEE THEM except your logged-in followers, and that appears to mean legacy followers only, because how would anybody not already a follower of an adult-flagged blog ever discover it now? If your blog is flagged adult on Tumblr, you’re blogging inside a sealed black box, and you have been for months.
So much for evasion — now for the outright false denials. The reporters for the Daily Dot sought comment from Tumblr, and they got this from Tumblr Head of Communications Katherine Barna. I’m including my commentary inline [in italicized brackets like this]:
Tumblr’s longstanding policy regarding NSFW content has not changed [true, if a few months is “longstanding”] and emphasizes the importance of free expression. [Bullshit!] As addressed in these policies, we are constantly taking measures to ensure our users can avoid this content [true so far] unless they’d like to see it. [A lie — if they’d like to see it, they still can’t find it because Tumblr doesn’t index it.] You can read about some of these features here: http://www.tumblr.com/docs/en/nsfw
Adult and NSFW content will be visible to anyone who has opted-in via their Settings page. [“Visible” only if you know the link already, but not searchable for anyone. In other words, more bullshit. Unless the “will be” future phrasing means Tumblr plans to change this? Do not die holding your breath.]
Different app environments have different requirements that we do our best to adhere to. [Presumably true, but freighted with the false implication that this is why Tumblr restricts adult content in its apps, given that they restrict it elsewhere likewise] Users can also find all content with Tumblr search in their mobile web browser. [Flat lie. Tumblr’s own policy chart newly shows that Tumblr search does not index the adult-flagged blogs.]
So, there you have it, folks! Months after making a policy change to exclude adult-flagged blogs from its internal search, Tumblr updates its docs to disclose the change. Internet goes wild, so their spokeswoman sends email that contradicts the new policy document and falsely denies the change.
Similar Sex Blogging:
