• MonkderVierte@lemmy.zip
    link
    fedilink
    English
    arrow-up
    11
    ·
    edit-2
    2 months ago

    How does archive get the unpaywalled version? I don’t think they pay the subscription for every single tabloid out there?

    Asking for a friend.

    • stoly@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      2 months ago

      The paywall is JavaScript but the content is still in plaintext below. The crawlers don’t read the JavaScript.

      • MonkderVierte@lemmy.zip
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 months ago

        Disabling 3rd-party js has no paywall, but only the first paragraph too. Crawlers get full access?