{"id":"PYSEC-2021-363","details":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","aliases":["CVE-2021-41125","GHSA-jwqp-28gf-p498"],"modified":"2023-11-01T05:43:46.749808Z","published":"2021-10-06T18:15:00Z","references":[{"type":"FIX","url":"https://github.com/scrapy/scrapy/commit/b01d69a1bf48060daec8f751368622352d8b85a6"},{"type":"WEB","url":"https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.http.basic_auth_header"},{"type":"ADVISORY","url":"https://github.com/scrapy/scrapy/security/advisories/GHSA-jwqp-28gf-p498"},{"type":"WEB","url":"http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth"}],"affected":[{"package":{"name":"scrapy","ecosystem":"PyPI","purl":"pkg:pypi/scrapy"},"ranges":[{"type":"GIT","repo":"https://github.com/scrapy/scrapy","events":[{"introduced":"0"},{"fixed":"b01d69a1bf48060daec8f751368622352d8b85a6"}]},{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.1"},{"introduced":"2.0.0"},{"fixed":"2.5.1"}]}],"versions":["0.10.4.2364","0.12.0.2550","0.14.1","0.14.2","0.14.3","0.14.4","0.16.0","0.16.1","0.16.2","0.16.3","0.16.4","0.16.5","0.18.0","0.18.1","0.18.2","0.18.3","0.18.4","0.20.0","0.20.1","0.20.2","0.22.0","0.22.1","0.22.2","0.24.0","0.24.1","0.24.2","0.24.3","0.24.4","0.24.5","0.24.6","0.7","0.8","0.9","1.0.0","1.0.0rc1","1.0.0rc2","1.0.0rc3","1.0.1","1.0.2","1.0.3","1.0.4","1.0.5","1.0.6","1.0.7","1.1.0","1.1.0rc1","1.1.0rc2","1.1.0rc3","1.1.0rc4","1.1.1","1.1.2","1.1.3","1.1.4","1.2.0","1.2.1","1.2.2","1.2.3","1.3.0","1.3.1","1.3.2","1.3.3","1.4.0","1.5.0","1.5.1","1.5.2","1.6.0","1.7.0","1.7.1","1.7.2","1.7.3","1.7.4","1.8.0","2.0.0","2.0.1","2.1.0","2.2.0","2.2.1","2.3.0","2.4.0","2.4.1","2.5.0"],"database_specific":{"source":"https://github.com/pypa/advisory-database/blob/main/vulns/scrapy/PYSEC-2021-363.yaml"}}],"schema_version":"1.7.3"}