!h , F S r SSKJr SSKrSSKJr SSKJrJrJ r J
r
SSK J r SSK
r
SSKJrJrJrJrJr SSKJr SS KJr SS
KJrJr SS KJ r \( a( SS K J r J r J r SS
K!J"r" SSK#J$r$J%r%J&r& SSK'J(r( SSK)J*r*J+r+ \," 5 r- " S S\5 r. " S S\/5 r0SS jr1g)z
This module implements the TextResponse class which adds encoding handling and
discovering (through HTTP headers) to base Response class.
See documentation in docs/topics/request-response.rst
) annotationsN)suppress)
TYPE_CHECKINGAnyAnyStrcast)urljoin)html_body_declared_encodinghtml_to_unicodehttp_content_type_encodingread_bomresolve_encoding)strip_html5_whitespace)Response)memoizemethod_noargs
to_unicode) get_base_url)CallableIterableMapping)Failure) CallbackTCookiesTRequest)Link)Selector SelectorListc ^ \ rS rSr% Sr\r/ \R QSP7rS\ S' SU 4S jjr
SU 4S jjr \ SS j5 r
S S jrS S
jr\ SS j5 rS S jr\S S
j5 rSS jrS S jr\S S j5 r\S S j5 r\ S S j5 rS!S jrS!S jrS"S jr S# S$U 4S jjjr S% S&U 4S jjjr Sr U =r $ )' TextResponse* asciiencodingztuple[str, ...]
attributesc ~ > UR SS 5 U l S U l S U l S U l [
TU ] " U0 UD6 g )Nr" )pop _encoding _cached_benc
_cached_ubody_cached_selectorsuper__init__)selfargskwargs __class__s `/root/1688_scrapy/alibaba-scraper/venv/lib/python3.13/site-packages/scrapy/http/response/text.pyr+ TextResponse.__init__0 s? %+ZZ
D%A(,)-15
$)&) c > SU l [ U[ 5 ( aP U R c" [ S[ U 5 R S35 eUR U R 5 U l g [ TU ]% U5 g )Nr2 z Cannot convert unicode body - z has no encoding)
_body
isinstancestrr& TypeErrortype__name__encoder* _set_body)r, bodyr/ s r0 r; TextResponse._set_body7 sl
dC ~~% 4Dz**++;= T^^4DJ G d #r2 c P U R 5 =( d U R 5 $ N)_declared_encoding_body_inferred_encodingr, s r0 r" TextResponse.encodingC s &&(JD,H,H,JJr2 c U R =( d> U R 5 =( d' U R 5 =( d U R 5 $ r? )r&
_bom_encoding_headers_encoding_body_declared_encodingrB s r0 r@ TextResponse._declared_encodingG sC NN
.!!#
.%%'
. ++-
r2 c U R [ L a% [ R " U R 5 U l U R $ )zH
.. versionadded:: 2.2
Deserialize a JSON document to a Python object.
)_cached_decoded_json_NONEjsonloadsr< rB s r0 rL TextResponse.jsonO s3 $ $ -(,
499(=D %(((r2 c U R nU R c" SU 3n[ X R 5 S U l U R $ )zBody as unicodezcharset= )r" r( r r< )r, benccharsets r0 textTextResponse.textY sG
}} % 'G!0))!DQ!GD !!!r2 c , [ [ U 5 U5 $ )zgJoin this Response's url with a possible relative url to form an
absolute interpretation of the latter.)r r )r, urls r0 r TextResponse.urljoind s |D)3//r2 c | [ [ U R R SS5 5 n[ [ USS95 $ )N Content-Typer2 latin-1r" )r bytesheadersgetr r )r, content_types r0 rF TextResponse._headers_encodingi s1 E4<<#3#3OS#IJ )*\I*VWWr2 c U R cj [ [ [ U R R SS5 5 SS9n[
UU R U R U R S9u p#X l X0l
U R $ )NrY r2 rZ r[ )auto_detect_fundefault_encoding) r' r r r\ r] r^ r r< _auto_detect_fun_DEFAULT_ENCODINGr( )r, r_ rQ ubodys r0 rA $TextResponse._body_inferred_encodingn sy $%UDLL , ,_c BCi L * $ 5 5!%!7!7 KD !% !& r2 c U R SS4 H n UR U5 [ U5 s $ g ! [ a M2 f = f)Nzutf-8cp1252)re decode UnicodeErrorr )r, rS encs r0 rd TextResponse._auto_detect_fun} sP **GX>C
C $C( ( ?
s 5
AAc , [ U R 5 $ r? )r
r< rB s r0 rG $TextResponse._body_declared_encoding s *49955r2 c 2 [ U R 5 S $ )Nr )r
r< rB s r0 rE TextResponse._bom_encoding s "1%%r2 c Z SSK Jn U R c
U" U 5 U l U R $ )Nr )r )scrapy.selectorr r) )r, r s r0 selectorTextResponse.selector s) , ($,TND !$$$r2 c SSK Jn [ U R S5 ( d [ S5 e[ X0R R " U40 UD65 $ )Nr r jmespathz6Please install parsel >= 1.8.1 to get jmespath support)rs r hasattrrt AttributeErrorr rx r, queryr. r s r0 rx TextResponse.jmespath sG 0t}}j11 H
L--"8"8"I&"IJJr2 c Z SSK Jn [ X0R R " U40 UD65 $ Nr rw )rs r r rt xpathr{ s r0 r TextResponse.xpath s$ 0L--"5"5e"Fv"FGGr2 c V SSK Jn [ X R R U5 5 $ r )rs r r rt css)r, r| r s r0 r TextResponse.css s 0L--"3"3E":;;r2 c > [ U[ R 5 ( a [ U5 nO*[ U[ R 5 ( a [ S5 eUc U R OUn[ TU ]! UUUUUUUUU U
U U U
S9
$ )a^
Return a :class:`~.Request` instance to follow a link ``url``.
It accepts the same arguments as ``Request.__init__()`` method,
but ``url`` can be not only an absolute URL, but also
* a relative URL
* a :class:`~scrapy.link.Link` object, e.g. the result of
:ref:`topics-link-extractors`
* a :class:`~scrapy.Selector` object for a ```` or ```` element, e.g.
``response.css('a.my_link')[0]``
* an attribute :class:`~scrapy.Selector` (not SelectorList), e.g.
``response.css('a::attr(href)')[0]`` or
``response.xpath('//img/@src')[0]``
See :ref:`response-follow-example` for usage examples.
z SelectorList is not supported)
rV callbackmethodr] r< cookiesmetar" priority dont_filtererrback cb_kwargsflags) r5 parselr _url_from_selectorr
ValueErrorr" r* follow)r, rV r r r] r< r r r" r r r r r r/ s r0 r TextResponse.follow s @ c6?? + +$S)C
V 0 0
1
1 <= =$,$44==(w~ #
r2 c > X U4 Vs/ s H
nUc M UPM nn[ U5 S:w a [ S5 eU( d0 U( a U R U5 nU( a U R U5 n[ U[
R 5 ( a? Un/ nU H5 n[ [ 5 UR [ U5 5 SSS5 M7 [ TU ]1 [ SU5 UUUUUUUU U
U U U
S9
$ s snf ! , ( d f Mp = f)aK
A generator that produces :class:`~.Request` instances to follow all
links in ``urls``. It accepts the same arguments as the :class:`~.Request`'s
``__init__()`` method, except that each ``urls`` element does not need to be
an absolute URL, it can be any of the following:
* a relative URL
* a :class:`~scrapy.link.Link` object, e.g. the result of
:ref:`topics-link-extractors`
* a :class:`~scrapy.Selector` object for a ```` or ```` element, e.g.
``response.css('a.my_link')[0]``
* an attribute :class:`~scrapy.Selector` (not SelectorList), e.g.
``response.css('a::attr(href)')[0]`` or
``response.xpath('//img/@src')[0]``
In addition, ``css`` and ``xpath`` arguments are accepted to perform the link extraction
within the ``follow_all()`` method (only one of ``urls``, ``css`` and ``xpath`` is accepted).
Note that when passing a ``SelectorList`` as argument for the ``urls`` parameter or
using the ``css`` or ``xpath`` parameters, this method will not produce requests for
selectors from which links cannot be obtained (for instance, anchor tags without an
``href`` attribute)
NrP zFPlease supply exactly one of the following arguments: urls, css, xpathzIterable[str | Link])
urlsr r r] r< r r r" r r r r r )lenr r r r5 r r r _InvalidSelectorappendr r*
follow_allr )r, r r r r] r< r r r" r r r r r r r x arguments selectorsselr/ s r0 r TextResponse.follow_all s R "&E 2D 21aQ 2 D y>Q X
xx}zz%( dF / / 0 0 ID ./KK 23 78 0/ ! w!,d3 # "
! E 0/s C-C-#C22
D )r4 r' rJ r) r( r& )r- r r. r )r< zstr | bytes | NonereturnNone)r r6 )r
str | None)r r )rV r6 r r6 )rS r\ r r )r r )r| r6 r. r r r )r| r6 r r ) NGETNNNNNr FNNN) rV z str | Link | parsel.Selectorr CallbackT | Noner r6 r] :Mapping[AnyStr, Any] | Iterable[tuple[AnyStr, Any]] | Noner< bytes | str | Noner CookiesT | Noner dict[str, Any] | Noner" r r intr boolr Callable[[Failure], Any] | Noner r r list[str] | Noner r )NNr NNNNNr FNNNNN) r z1Iterable[str | Link] | parsel.SelectorList | Noner r r r6 r] r r< r r r r r r" r r r r r r r r r r r r r r r r zIterable[Request]) r9
__module__ __qualname____firstlineno__re rK rJ r r# __annotations__r+ r; propertyr" r@ rL rS r r rF rA rd rG rE rt rx r r r r __static_attributes__
__classcell__)r/ s @r0 r r * s "DH$7$7"D"DJD*
$ K K
) " "0
X X
! 6 6 & & % %KH
< &*NR#'#'&* # !37+/"& 3
)3
#3
3
L 3
!
3
!3
$3
3
3
3
13
)3
3
3
3
n CG%)NR#'#'&* # !37+/"& !G
?G
#G
G
L G
!
G
!G
$G
G
G
G
1G
)G
G
G
!G
"
#G
G
r2 r c \ rS rSrSrSrg)r i) z6
Raised when a URL cannot be obtained from a Selector
N)r9 r r r __doc__r r r2 r0 r r ) s r2 r c [ U R [ 5 ( a [ U R 5 $ [ U R S5 ( d [ SU 35 eU R R S; a# [ SU R R S35 eU R R S5 nUc% [ SU R R SU 35 e[ U5 $ ) NtagzUnsupported selector: )alinkz1Only and elements are supported; got <>href element has no href attribute: )r5 rootr6 r ry r r r^ )r r s r0 r r / s #((C %chh// 388U # # !7u =>>
xx||=( ? ~Q O
88<< D | 388<<.0QRUQV WXX !$ ''r2 )r zparsel.Selectorr r6 )2r
__future__r rL
contextlibr typingr r r r urllib.parser r w3lib.encodingr
r r r
r
w3lib.htmlr scrapy.http.responser scrapy.utils.pythonr r scrapy.utils.responser collections.abcr r r twisted.python.failurer scrapy.http.requestr r r scrapy.linkr rs r r objectrK r r r r r r2 r0 r sz # 3 3
. ) @ .;;.@@ 6 |
8 |
~z
(r2