Google and The Online Slang Dictionary
Last update to this page: October 13, 2020
August 14, 2020: The Signal phone number listed in the secure contact methods section was not functioning but should now be fixed.
August 14, 2020: Google's penalty has been going on for years. Since they have finally almost killed this site, I am going to start releasing details on Monday August 17 of my conversation with the Google employee who told me in secret about the penalty. This will culminate in my release of an MBOX file including full headers.
August 17, 2020: Since Google has finally almost killed this site, I have started releasing details of my conversation with the Google employee who told me in secret about the penalty. More detail is to come.
If you find this page interesting, I'd love to chat with you. If you work for Google, then we should speak soon. I can be reached at .
– Walter (editor)
Table of contents
Summary
This website has been penalized by Google under suspicious circumstances. The penalty causes our web pages to appear at lower positions in Google's search results than what they've earned on their merit. It is an intentional, hidden, manual penalty executed by Google against this site.
Publicly, the company says that websites can discover and fight these manual penalties via a tool they provide. What they don't disclose is that Google can and does execute hidden penalties against sites. Since they're hidden, there's no way for site owners to respond to - or even know about - the fact that Google is intentionally limiting visitors to their sites. Websites live or die based on people visiting the site. This means that Google kills websites in private, with no recourse for the site owners. And there is no oversight or accountability.
Urban Dictionary's owner and former Google employee Aaron Peckham with Google's former head of penalties Matt Cutts.
Photo by Julie Pimentel (licensed CC BY-NC 2.0.)
To my knowledge, Google's ability to destroy websites in this manner has not been disclosed before.
I was only made aware of the penalty after a whistle-blower informed me of it.
Separately, when confronted about this hidden penalty, a Google employee lied to me and to the public about it. That Google employee is Matt Cutts, the former head of the Google team that executes penalties against websites. He quietly left the company after an extended "sabbatical" and, tellingly, no one has been hired to replace him.
This is an initial information drip. I will be releasing more information over time. For example, the entire entire communication history with the whistleblower - including associated metadata.
Please help me in spreading the word. If you've found The Online Slang Dictionary to be useful, please let your friends and colleagues know about the site. The only way to save it is to get the word out.
About The Online Slang Dictionary
Started by Walter Rader in 1996, The Online Slang Dictionary is the oldest slang dictionary on the web and the first to accept submissions from visitors. Serving 1.5 million visitors each month, it provides more than 28,000 real definitions for over 19,000 actual slang words and phrases. The site offers the only slang thesaurus on the web, which is fully integrated with the dictionary. Other unique features include showing you how common each word is, how vulgar it is, and where geographically it's used. Logged in visitors can add words and definitions.
The site isn't perfect: I totally acknowledge that. I just want my site to be able to compete on its own merits, rather than being intentionally crippled by Google.
The first penalty: automated
This site has more than 5,700 citations curated by hand from published sources like TV programs, films, news publications, and magazines - and added by hand to the appropriate definitions. Each is a short 1 - 3 sentence excerpt, with proper attribution. This site is the only freely available slang dictionary on the web that has citations.
The first penalty executed against this site by Google was automated. That is, the penalty was applied by an algorithm - not a human. This algorithm has a bug: it mistakenly thinks that the citations on this site are an indication of a low-quality site that should be penalized.
Further, Google hides automated penalties from website owners, so there was no way for me to know the cause of the penalty.
Sidebar: Why are citations important?
Real dictionaries have citations because that's how they show that their definitions are real. Being a dictionary of real slang terms, the same applies to us. Citations serve the same purpose in dictionaries as they do on Wikipedia: they provide evidence for stated facts. Unlike encyclopedias such as Wikipedia, dictionary citations are inline excerpts rather than links to outside works.
After months of this algorithm at Google intentionally limiting the traffic to this site, and after months of experimentation and thousands of changes made to the site, I removed the citations - more than 5,700. This was a desperate experiment, having tried everything else I could think of to try to eliminate the penalty. This caused the penalty to be undone, again automatically, by the algorithm.
The current penalty: manual
A Google employee later executed a manual penalty against the site. Recall that a penalty against a web site causes web pages on that site to appear at a lower position in Google's search results than the position they have earned on their merit. A manual penalty is one that has been activated by a human, by hand.
On October 9, 2011, a Google employee set an internal flag that executed a penalty against this site in a way that is both impossible to undo, and impossible to detect by the site owner (me.)
I was only made aware of the penalty after a whistle-blower informed me of it.
There is no truthful justification for this penalty. However, when a manual penalty is activated, a reason can be recorded. The Google employee recorded the reason as being related to intellectual property law. But that makes no sense as there are no - and have never been any - intellectual property law violations on this site.
If you work for Google, then we should speak soon. I can be reached at .
The automated penalty in more detail
This section describes the automated penalty at length. In this section:
(
Show this sectionHide this section.)
Introduction
In 2011, Google released a change to their ranking algorithms designed to penalize low-quality websites. One thing Google now believes is an indicator of low quality is displaying excerpts of content that exists elsewhere. Despite citations being a hallmark of a high-quality dictionary, they get interpreted by this change (called Panda) as the complete opposite, because by definition citations are content that exists elsewhere.
Sidebar: Citations on The Online Slang Dictionary
This site has more than 5,700 citations curated by hand from published sources like TV programs, films, news publications, and magazines - and added by hand to the appropriate definitions. Each is a short 1 - 3 sentence excerpt, with proper attribution. This site is the only freely available slang dictionary on the web that has citations.
In addition to the 5,700 citations on this website, we have another 11,400 to be added, which would bring the total citation count to more than 17,000.
But we can't show them to you. That's why they appear blacked out, as in a redacted classified document. When Google started penalizing this site, it was because we have citations. There is a bug in one of Google's algorithms, related to a set of changes they call "Panda 2.0". This bug incorrectly interprets the citations as evidence that the site is low quality, despite citations being a sign of the exact opposite: a high quality dictionary. This penalty was automated: it didn't require any human intervention. Removing the citations resulted in the penalty being removed, also automatically.
So since we can't show you citations, there's no reason to add the additional 11,400 citations mentioned above.
The following shows the consequence of this penalty, measured in visits to this site from Google searches.
The rest of this section explains the data in more detail.
Timeline
2011
- April 11: Google releases the Panda algorithm changes globally to all English-speaking users, and starts penalizing this site.
- November 5: I remove all citations, largely as an experiment. I couldn't believe Google would be penalizing a dictionary website because of the presence of citations.
- November 13: Google confirms that the citations are the cause by retracting the penalty.
Google penalizes The Online Slang Dictionary for showing citations
Google's Panda changes - designed to penalize what Google's algorithms believe are "low-quality" sites - were released globally to all English-speaking users on April 11, 2011. Visits to The Online Slang Dictionary that were the result of a Google search dropped markedly on April 11, 2011.
This chart only shows traffic from Google searches.
The loss in traffic was limited to visits from Google searches. No other traffic sources (other search engines, direct traffic, or referral traffic) were affected. Such a drop is unprecedented for the site since I began tracking site analytics in 2007.
Specifically, traffic declined because Google moved this site to a lower position in its search results. Reports from Google Webmaster Tools show that the position of The Online Slang Dictionary in Google search results fell on April 11, the date of the Panda penalty.
The reports that I have access to cover a time period of a month, rather than a single day. So the effects of the penalty only start to become evident in the report dated March 12 - April 11. The next dated report goes through April 12, so it includes 2 days of penalty. The subsequent report includes 3 days of penalty, and so forth. As more days of the penalty are within the date range of the report, the amount of the decline is increasingly revealed. (Why the uptick at the end? Because the reports show a range, the final report that includes April 11 is dominated by subsequent dates. There was a small uptick in ranking during those dates - which did not restore traffic - then the ranking dropped back down.)
So as the data shows, The Online Slang Dictionary started receiving a penalty on April 11, 2011, and that's the date that the Panda changes were released to all English-speaking users of Google.
Citations are removed
During the next seven months, I made dozens of structural changes and thousands of content changes to the site - but these changes had no effect on the amount of traffic referred by Google queries.
Largely as an experiment, I removed every citation from the site on November 5, 2011. I couldn't believe Google would be penalizing the site because of the presence of citations. After all, high-quality dictionaries have citations, and Google Panda was designed to penalize low-quality websites.
A week and a day later - on November 13 - the penalty was rescinded.
The following chart shows only traffic from Google searches.
The manual penalty in more detail
This section describes the manual penalty at length. In this section:
(
Show this sectionHide this section.)
Timeline
2012
- October 6: Believing I had identified a way to restore the citations without incurring the penalty, I add them back to the site.
- October 9: A Google employee used an internal tool to execute a penalty against this site.
- November 16: To be able to compete with dictionary websites that don't have citations, I once again remove citations.
- November 16, 2012 - October 18, 2020: The penalty continues without relief.
Google penalizes The Online Slang Dictionary again
The citations were then missing from the site for almost a year. In early October 2012, I identified what I believed was a way to restore citations to the site without being penalized by Google. The full inventory of citations - approximately 4,800 at that time - returned to the site on October 6, 2012.
The penalty returned 3 days later.
This chart only shows traffic from Google searches.
This second penalty executed by Google against this site was manual. Recall that a penalty against a web site causes web pages on that site to appear at a lower position in Google's search results than the position they have earned on their merit. A manual penalty is one that has been activated by a human, by hand.
On October 9, 2011, a Google employee set an internal flag that executed a penalty against this site in a way that is both impossible to undo, and impossible to detect by the site owner (me.)
I was only made aware of the penalty after a whistle-blower informed me of it.
There is no truthful justification for this penalty. However, when a manual penalty is activated, a reason can be recorded. The Google employee recorded the reason as being related to intellectual property law. But that makes no sense as there are no - and have never been any - intellectual property law violations on this site.
I tried a second approach to restore the citations, but it did not eliminate the penalty. At this time, I was unaware that the penalty was manual. (Recall that this manual penalty was set to be hidden from me.) Had I known, I wouldn't have continued to attempt to show this valuable information to my visitors.
Citations are removed again
On November 16, 2012, in order to be able to compete with dictionary websites that don't have citations, I once again removed all citations.
I had hoped that it would cause the penalty to be lifted. That is, I hoped that the Google employee that had executed the penalty against this site would see that I removed the citations, and would undo the penalty.
That has not happened.
Matt Cutts lies, and confirmation of the manual penalty
Matt Cutts is an ex-Google employee. When he was at Google, he ran the Webspam team - the team that is enirely responsible for both automated and manual penalties. They create the algorithms that execute automated penalties and write the policies for executing manual penalties.
On March 21, 2013 Matt Cutts wrote the following in response to someone asking how the owner of a small site could get a penalty resolved when Google had de-listed Digg.com in its entirety. Cutts responded, in part:
...
In general, when a member of the webspam team directly applies a manual webspam action against a site, we also drop a note to the site owner at https://ift.tt/136dOuK . That helps the site owner tell whether something is going on with manual spam vs. just algorithmic ranking. Then any site can do a reconsideration request at the same place or post in our webmaster forum at https://ift.tt/16ohg1Z .
...
I responded saying that though there appeared to be a manual penalty against my website, I received no notification of any form that there was a manual penalty. On March 22, 2013 Cutts replied with the following:
Hi Walter, the only manual webspam action I see regarding onlineslangdictionary.com is from several years ago (are you familiar with a company called Web Build Pages or someone named Jim Boykin?), but that no longer applies here.
You're affected by a couple algorithms in our general web ranking. The first is our page layout algorithm. See http://googlewebmastercentral.blogspot.com/2012/01/page-layo... or http://searchengineland.com/google-may-penalize-ad-heavy-pag... for more context on that. In particular, comparing a page like http://onlineslangdictionary.com/meaning-definition-of/compy to a page like http://www.urbandictionary.com/define.php?term=Pepperazzi... , your site has much more prominent ads above the fold compared to Urban Dictionary.
Your site is also affected by our Panda algorithm. Here's a blog post we wrote to give guidance to sites that are affected by Panda: http://googlewebmastercentral.blogspot.com/2011/05/more-guid...
I'll respond to each of his claims.
Hi Walter, the only manual webspam action I see regarding onlineslangdictionary.com is from several years ago (are you familiar with a company called Web Build Pages or someone named Jim Boykin?), but that no longer applies here.
I don't know who Web Build Pages or Jim Boykin are.
(If you're familiar with web ranking and SEO companies, an interesting thought experiment is: If Matt Cutts were telling the truth, how would a manual penalty against my site - or any site - be tied to both a specific person and a spefific company? For example, did the owner of an SEO company have a change of heart, provide Google with a complete customer list, and Google - without any analysis whatsoever - just manually penalized the website of every one of those customers? a. That's not how it works. b. I've never heard of the person or company. What's a legitimate way that a manual penalty could have been in place against my site - or any site - tied in some way to a specific person and a specific company?)
There was never any manual webspam action against the website. Analysis of traffic referred by Google searches shows smooth growth until Matt Cutts decided to kill the website. There is zero evidence of a manual penalty, which would be obvious in the traffic analysis.
You're affected by a couple algorithms in our general web ranking. The first is our page layout algorithm. See http://googlewebmastercentral.blogspot.com/2012/01/page-layo... or http://searchengineland.com/google-may-penalize-ad-heavy-pag... for more context on that. In particular, comparing a page like http://onlineslangdictionary.com/meaning-definition-of/compy to a page like http://www.urbandictionary.com/define.php?term=Pepperazzi... , your site has much more prominent ads above the fold compared to Urban Dictionary.
Despite Matt Cutts' claim to the contrary, Urban Dictionary actually had more screen area devoted to ads than this site.
Nevertheless, I removed all ads for approximately 6 months. There was no change in the amount of search traffic that Google referred to my site. (Recall that when the previous automated penalty was removed, search traffic was restored - this is how automated penalties work.) Automated penalties are removed when the issue is fixed. It doesn't happen immediately, but it happens far more quickly than 6 months. What does it mean if the issue was fixed and the penalty was never removed? It means that the penalty didn't exist in the first place.
Your site is also affected by our Panda algorithm. Here's a blog post we wrote to give guidance to sites that are affected by Panda: http://googlewebmastercentral.blogspot.com/2011/05/more-guid...
Just as I removed the ads, I removed all of the citations. There was no change in the amount of search traffic Google that referred to my site. (Recall that when the previous automated penalty was removed, search traffic was restored - this is how automated penalties work.) Automated penalties are removed when the issue is fixed. It doesn't happen immediately, but it happens far more quickly than 6 months. What does it mean if the issue was fixed and the penalty was never removed? It means that the penalty didn't exist in the first place.
So the question remains: how exactly do I know that Matt Cutts was lying? How do I know that there was (and is) a manual penalty against my website?
Because a Google employee told me so.
I am going to start releasing details of my email conversation with the Google employee, culminating in my release of an MBOX file including full headers.
Around the time of my exchange with Matt Cutts (see above) I had an email conversation with a Google employee. It went something like this. (The below is heavily paraphrased. As time goes on, the below will come more and more to resemble the actual conversation.)
Subject: What's with your article about the penalty?
From: redacted@google.com
Date: redacted 2013Hey, I saw the article on your website. Is there anything I can do to help?
Subject: Re: What's with your article about the penalty?
From: Walter Rader
Date: redacted 2013Yes. Could you please ask the web spam team what's up with the manual penalty against my website?
Subject: Re: What's with your article about the penalty?
From: redacted@google.com
Date: redacted 2013There is and has been a manual penalty against your site for some time. I can't help you.
In closing
The penalty is ongoing as of October 2020, and is killing The Online Slang Dictionary.
Please help me in spreading the word. If you've found The Online Slang Dictionary to be useful, please let your friends and colleagues know about the site. The only way to save it is to get the word out.
If you work for Google, then we should speak soon. I can be reached at .
Thanks for your time.
Any suggestions?
I'd love to hear them. My email address is .
Frequently asked questions
I just did a Google search and your site appeared in the first few results. Does that mean that the penalty has been removed?
No.
"Google Juice" is an informal term for how favorably Google views a website and pages on that website. There are a lot of factors that go into how much juice websites and pages earn.
Every time you do a search, Google's algorithms evaluate every page on the web to decide how relevant each page is to your query - how much Google Juice each of the pages has for your query. Then they show the search results, which are ordered by the amount of Google Juice each page has.
What the penalty does is subtract an amount of Google Juice from every page on this site. But whether that means we appear 1st, 2nd, or 307th in the search results depends on how much juice each of the other pages on the web have for your query.
You could think of it as a foot race. The penalty doesn't work as in: however you finish in the race, Google will drop you down by 9 places. It works as in: Google attaches a 20 pound weight to your foot, and whether that means you finish 1st or 307th depends on how good the other runners are.
So sometimes pages from this site appear towards the top of search results. That just means that those specific pages have enough Google Juice - and other pages on the web have so little Google Juice - that even with the penalty, we can appear towards the top. But overall, the penalty drags our pages down far enough that we would get about three times as many visitors if the penalty weren't in place.
(This answer is written for a general audience. If you have technical questions, feel free to email me at .)
Why are the 5,700 citations on your site blacked out?
This site has more than 5,700 citations curated by hand from published sources like TV programs, films, news publications, and magazines - and added by hand to the appropriate definitions. Each is a short 1 - 3 sentence excerpt, with proper attribution. This site is the only freely available slang dictionary on the web that has citations.
But we can't show them to you. That's why they appear blacked out, as in a redacted classified document. When Google started penalizing this site, it was because we have citations. There is a bug in one of Google's algorithms, related to a set of changes they call "Panda 2.0". This bug incorrectly interprets the citations as evidence that the site is low quality, despite citations being a sign of the exact opposite: a high quality dictionary. This penalty was automated: it didn't require any human intervention. Removing the citations resulted in the penalty being removed, also automatically.
The penalty is no longer an automated one, but rather a manual one. A Google employee set an internal flag that penalizes our site, and it won't be removed until someone within Google removes it. But if that happens, then having citations on the site would likely lead to another automated penalty.
Hence the blacked-out citations.
Why not remove them entirely? Because the only way to get Google to remove the manual penalty against this site is for the cost of that penalty to be visible.
Encrypted email
You may use PGP / GPG to encrypt email sent to .
Be sure to include your public key.
Be aware that only the body of the email will be encrypted. Email metadata will not be encrypted, such as sender, subject, when the email was sent, and the email provider used to send the email.
Unfortunately, due to the time suck of fighting Google's penalty, this site is not yet served via SSL. The hypothetical risks involved in publishing, receiving, and using a public key over a non-SSLed connection is low, limited to:
- a MITM attack replacing my actual public key with a fraudulent key that contains an exploit of the encryption/decryption software used to securely communicate
- my simply not being able to decrypt email that was encrypted using the fraudulent public key, since I don't have the corresponding private key.
Out of an abundance of caution, my public key is available on a separate domain which I control and which is SSLed.
My public key is available here.
Links to my public key and/or the public key itself are also available via this site's official blog, Facebook, and Twitter accounts. Note that the official blog is hosted on Google's Blogger service.
Signal
Signal is an encrypted instant messaging and voice and video calling platform. Apps are available for iPhone, Android, and desktop operating systems.
I can be reached via Signal at: +1 530 727-
Due to the time suck of fighting Google's penalty, this site is not yet served via SSL. The Signal number above is also available via this site's official blog, Facebook, and Twitter accounts. Note that the official blog is hosted on Google's Blogger service.
Postal mail
I can be reached by postal mail at the following address.
Walter Rader
PO Box
Washington, DC 20016
Due to the time suck of fighting Google's penalty, this site is not yet served via SSL. The address above is also available via this site's official blog, Facebook, and Twitter accounts. Note that the official blog is hosted on Google's Blogger service.
In closing
The penalty is ongoing as of October 2020, and is killing The Online Slang Dictionary.
Please help me in spreading the word. If you've found The Online Slang Dictionary to be useful, please let your friends and colleagues know about the site. The only way to save it is to get the word out.
If you work for Google, then we should speak soon. I can be reached at .
Thanks for your time.
Any suggestions?
I'd love to hear them. My email address is .
from Hacker News https://ift.tt/2kuTfYO
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.