Simple Google-based cheat sites often beat humans, but a big one was just shut down.
The daily trivia game attracts millions of players to battle for real money 12 times a week by answering 12 multiple-choice questions sent via live video stream. In recent days, though, the app makers have been locked in their own battle with sites like HQuack. These bot sites use optical character recognition and Google to try to figure out the answers to the game’s questions and feed them to players before the game’s ten-second timer is up.
Bots like these are still imperfect—HQuack advertises only “up to 82 percent accuracy,” which is often not enough in a game where a single wrong answer leads to elimination. But if and when they work, they have the potential to ruin a game that’s becoming a bona-fide phenomenon.
To see just how much of a problem these question-answering bots can cause, look no further than the 3:45 HQ Trivia game on Tuesday, February 6. Of the 786,883 who started the game, a full 9,046 answered all 12 questions correctly, a number host Scott Rogowsky confirmed was a record. Each of those winners got a grand total of 23¢.
For context, just a week before, in an exceedingly average January 31 game, a playing crowd of 798,796 had been reduced to just 81 winners, who each won $30.86. That’s a more than hundredfold increase in the win rate, from 0.01 percent to over 1.1 percent.
It’s probably no coincidence that the AI bot at HQuack happened to get every single answer right during Tuesday’s show. The week before, the site had barely launched to the public and had yet to hit the mainstream via an Outline article. (There are anecdotal reports that a reusable extra lives bug may have also contributed to the record result.)
You can see the HQuack effect even when the bot misses a question or two. On a Thursday, February 8 game, 11,581 players managed to answer the first 11 questions correctly, just as the HQuack bot had (the bot actually missed the second question about the Olympic logo, but it was so simple that about 98 percent of the playing humans managed to get it right anyway).
For question 12, though, HQuack’s algorithm suggested that Karl Marx was born with the same name as a famous ketchup, and over 8,800 of the remaining 11,500 players agreed. Only the 2,000 who knew the correct answer was Henry Kissinger (born Heinz Kessinger) ended up winning the prize. A similar pattern of seemingly HQuack-led answer grouping could be seen in the final few questions of Sunday’s Super Bowl halftime edition of HQ Trivia, where the bot and its loyal players eventually fell to a question about the Reimann hypothesis.
Dedicated HQ players are beginning to notice this growing problem. “Will this even be worth it if they cant [sic] cut down on programs googling?” Redditor Vikemin1 asked somewhat rhetorically on the HQ Trivia subreddit this week. “There’s really no reason to keep playing this game until they find a way to take care of the bots,” redditor cbooz added by way of answer in another thread.
HQ players have been manually speed-Googling their way to HQ wins for a while, and a number of hackers have talked up their own automated methods for finding answers since December at least. But it was the recent launch of HQuack (and similar sites like the members-only HQHelp) that seems to have upped the prevalence and effectiveness of HQ cheaters.
HQuack creator Jake Mor told Ars the site started as a simple program on his computer, Googling questions and giving confidence intervals to the possible answers based on the number of results. After throwing the program on a website for his friends to use, Mor told Ars that “word of the site kind of just spread.”
Mor wouldn’t tell Ars how many people are using his site these days, but he told The Outline last week that he was already getting “a thousand unique visitors a week” a few weeks ago and that he’d been seeing “20 percent day over day growth.”
Mor says he doesn’t make money off of HQuack and never intends to. “This just started as a side project out of curiosity,” he told Ars. “I never expected it to pick up the way it did.” And while he acknowledged that the site “may have played a part” in recent record-setting win rates, he said he doesn’t want to destroy the legitimate HQ experience. “If it does start to truly ruin the game, I think I will take it down.”
While HQuack and other automated tools are prohibited by HQ’s terms of service, preventing people from using them would seem difficult as a practical matter. After all, how can the app tell the difference between someone who just knows a lot of trivia answers and someone who’s being told a lot of trivia answers by a website? Even pattern-matching algorithms could be thrown off if players just go against the bot every so often. (HQ has not responded to a request for comment from Ars.)
HQ’s best protection for now might be the bots’ imperfection—with the aforementioned 82 percent accuracy, HQuack will be perfect or near-perfect for some 12-question games and then miss four questions in a row on others. Mor says HQuack currently has the most trouble with “double questions” where the answer requires knowing two things at once. (One recent example that tripped up HQuack: “If you capitalize it, what Disney character name doubles as a healthcare program?” Answer: Chip.)
Changing up the question formats could help, too. While a bot can search for the answer to a text question pretty easily, questions based on pictures or audiovisual snippets could prove more resistant to simple Googling. Google also struggles with more abstract questions like “Why did…” and relational questions like “How close is [place] to [place]” according to HQ bot maker Stephen Cognetta.
But Google’s question-answering algorithms are getting better every day, and more sophisticated AI approaches could perform even better at the game. In a world where IBM’s Watson can easily beat Jeopardy champions, a multiple-choice quiz app seems like easy pickings. And with HQ offering real money for winners 12 times a week, the incentive for hackers to continue to try to beat the app will remain high.
HQuack creator Mor, though, doesn’t think you should count HQ out of the battle with the trivia bots just yet. “There are still other bots attempting to do the same thing, but I trust that HQ will figure it all out,” he said. “They’re a really smart and capable group of people.”
[Update: Shortly after this article was published, cheating bot site HQuack was taken down before the start of Friday’s 3pm HQ Trivia game. “The Quack is gone. But something big is coming soon — and it’s by invite only =)” the site now reads, with a link to sign up for updates.
For what it’s worth, today’s HQuack-free game finished with 81 winners out of nearly 900,000 who started the game.
Site creator Jake Mor was not immediately available to respond to a request for comment from Ars. As noted in the original story, though, he previously told Ars, “if it does start to truly ruin the game, I think I will take it down.”]
[Further update (February 12): Mor tells Ars that “it became increasingly clear that HQuack was starting to affect the game, and I wasn’t really interested in that happening. For now, I’m focusing my attention on a new project that I think tons of people will enjoy. I’ll keep you posted on my progress!”
When creating Ultima Online, Richard Garriott had grand dreams. He and Starr Long planned on implementing a virtual ecology into their massively multiplayer online role-playing game. It was an ambitious system, one that would have cows that graze and predators that eat herbivores. However, once the game went live a small problem had arisen…