About 10 — of 310 — contributions to The Wall Street Journal, the Washington Post and the New York Times over the past month appear to have been written by AI models.

Humans are still writing the vast majority of guest columns published in the opinion pages of The Wall Street Journal, The Washington Post, and The New York Times, a Semafor analysis found — but large language models were creeping in well before a high-profile Journal op-ed provoked controversy this week.

Over the past month, 10 out of 310 guest submissions to the three publications were labeled as at least 80% “AI” by AI detector Pangram, suggesting AI is not yet making a meaningful dent in human contributions to the most prestigious publications.

Another 40 articles were partially AI-generated. Pangram uses custom AI models to analyze text and other content for AI and, according to a spokesperson, has a .01% false positive rate. If Pangram finds more than 80% of the text is AI generated, it is labeled as AI. In informal testing by Semafor, Pangram was very accurate in detecting AI-written text, though some complain of false positives.

  • dhork@lemmy.worldEnglish
    21·
    13 hours ago

    I’m not too confident in the AI detection tools myself. They’re just as flawed as the rest of it. I used to have a boss who “wrote like an AI” years before Generative AI took off, we just didn’t know to describe it back then.

    It’s up to editors to discern whether an op-ed is good for publication. Often times, it’s based on the reputation and credentials of the author. A well credentialed author can use AI as a reference and still create a piece that has their voice, and gets their point across, even if some analysis bot somewhere thinks it finds a familiar pattern.

  • Sibshops@feddit.cl
    1·
    7 hours ago

    Semafor has come out of no where and became one of the best news organizations as of late. They even seperate reporting from analysis within the same article.

  • Entropy_Pyre@lemmy.ca
    6·
    13 hours ago

    I feel like one problem is that AI trained on real people. So I find it hard to believe that real people will never sound like AI. I think these sorts of analyzing tools can be useful, but shouldn’t be the sole source of evidence that something is generated.

    • Em Adespoton@lemmy.ca
      3·
      13 hours ago

      That said, LLMs tend to contain “tells” from a number of prominent writers and style guides, in combinations that a human would never use on their own.

      • DomeGuy@lemmy.world
        7·
        12 hours ago

        The most certain thing about humans is that predictions about “humans would never” will inevitably be falsified.

        Especially amateur writers without an editor, which is what editorial pages have largely devolved into in the era of social media.