MyNixOS website logo
Description

Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls.

Metadata

Version

2.0.1

Maintainers (1)

Platforms (31)

    Darwin
    FreeBSD
    Linux
    Windows
Show all
  • aarch64-darwin
  • aarch64-linux
  • armv5tel-linux
  • armv6l-linux
  • armv7a-darwin
  • armv7a-linux
  • armv7l-linux
  • i686-cygwin
  • i686-darwin
  • i686-freebsd
  • i686-linux
  • i686-windows
  • loongarch64-linux
  • m68k-linux
  • microblaze-linux
  • microblazeel-linux
  • mips-linux
  • mips64-linux
  • mips64el-linux
  • mipsel-linux
  • powerpc64-linux
  • powerpc64le-linux
  • riscv32-linux
  • riscv64-linux
  • s390-linux
  • s390x-linux
  • x86_64-cygwin
  • x86_64-darwin
  • x86_64-freebsd
  • x86_64-linux
  • x86_64-windows