hZSrSSKJr SSKJr SSKJrJrJrJ r J r J r J r J r JrJr SSKJr SSKJr SSKJrJr SSKJr SS KJr SS KJr \(aSS KJr SS jr"S S5r g)z; Item Loader See documentation in docs/topics/loaders.rst ) annotations)suppress) TYPE_CHECKINGAnyCallableDictIterableListMutableMappingOptionalPatternUnion) ItemAdapter)Selector) extract_regexflatten)wrap_loader_context)Identity) arg_to_iter)Selfc[[5 SUR;aURsSSS5 $SSS5 U$!,(df  U$=f)zz Allow to use single-argument functions as input or output processors (no need to define an unused first 'self' argument) .N)rAttributeError __qualname____func__)methods [/root/1688_scrapy/alibaba-scraper/venv/lib/python3.13/site-packages/itemloaders/__init__.pyunbound_methodr$sH . ! f)) )?? " ! ) " M " ! Ms A Ac(\rSrSr%Sr\rS\S'\"5r S\S'\"5r S\S'S)S*S jjr \ S+S j5r \ S,S j5rS-S jrS.S jrSS.S/SjjrSS.S/SjjrS0SjrS0SjrSS.S1SjjrS,SjrS2SjrS3SjrS4SjrS4SjrS5S6SjjrS7SjrS8SjrSS.S9SjjrSS.S9SjjrSS.S:Sjjr S;Sjr!SS.SS#jr%SS.S?S$jjr&SS.S@S%jjr'SS.SAS&jjr(SBS'jr)S(r*g)C ItemLoader/a Return a new Item Loader for populating the given item. If no item is given, one is instantiated automatically using the class in :attr:`default_item_class`. When instantiated with a :param ``selector`` parameter the :class:`ItemLoader` class provides convenient mechanisms for extracting data from web pages using parsel_ selectors. :param item: The item instance to populate using subsequent calls to :meth:`~ItemLoader.add_xpath`, :meth:`~ItemLoader.add_css`, :meth:`~ItemLoader.add_jmes` or :meth:`~ItemLoader.add_value`. :type item: :class:`dict` object :param selector: The selector to extract data from, when using the :meth:`add_xpath` (resp. :meth:`add_css`, :meth:`add_jmes`) or :meth:`replace_xpath` (resp. :meth:`replace_css`, :meth:`replace_jmes`) method. :type selector: :class:`~parsel.selector.Selector` object The item, selector and the remaining keyword arguments are assigned to the Loader context (accessible through the :attr:`context` attribute). .. attribute:: item The item object being parsed by this Item Loader. This is mostly used as a property so when attempting to override this value, you may want to check out :attr:`default_item_class` first. .. attribute:: context The currently active :ref:`Context ` of this Item Loader. Refer to for more information about the Loader Context. .. attribute:: default_item_class An Item class (or factory), used to instantiate items when not given in the ``__init__`` method. .. warning:: Currently, this factory/class needs to be callable/instantiated without any arguments. If you are using ``dataclasses``, please consider the following alternative:: from dataclasses import dataclass, field from typing import Optional @dataclass class Product: name: Optional[str] = field(default=None) price: Optional[float] = field(default=None) .. attribute:: default_input_processor The default input processor to use for those fields which don't specify one. .. attribute:: default_output_processor The default output processor to use for those fields which don't specify one. .. attribute:: selector The :class:`~parsel.selector.Selector` object to extract data from. It's the selector given in the ``__init__`` method. This attribute is meant to be read-only. .. _parsel: https://parsel.readthedocs.io/en/latest/ typedefault_item_classCallable[..., Any]default_input_processordefault_output_processorNc JX lURUS9 UcUR5nXlXS'X@lX0l0Ul[U5R5HAupVURRU/5 URU==[U5- ss'MC g)Nselectoritem) r)updater# _local_itemcontextparent _local_valuesritems_values setdefaultr)selfr*r)r.r- field_namevalues r__init__ItemLoader.__init__zs-5 ) <**,D18 ,2 35!,T!2!8!8!: J LL # #J 3 LL $ E(: : $";c`URbURR$UR$N)r.r1r/r3s rr1ItemLoader._valuess( ;; ";;&& &%% %r8c`URbURR$UR$r:)r.r*r,r;s rr*ItemLoader.items( ;; ";;## ### #r8c UR5 URceURRU5nURUS9 UR"SUR US.UD6nU$)a= Create a nested loader with an xpath selector. The supplied selector is applied relative to selector associated with this :class:`ItemLoader`. The nested loader shares the item with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`, :meth:`add_value`, :meth:`replace_value`, etc. will behave as expected. r(r*r.)_check_selector_methodr)xpathr+ __class__r*)r3rCr-r) subloaders r nested_xpathItemLoader.nested_xpathsd ##%}}(((==&&u-)NNJ $J'J r8c UR5 URceURRU5nURUS9 UR"SUR US.UD6nU$)a: Create a nested loader with a css selector. The supplied selector is applied relative to selector associated with this :class:`ItemLoader`. The nested loader shares the item with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`, :meth:`add_value`, :meth:`replace_value`, etc. will behave as expected. r(r@rA)rBr)cssr+rDr*)r3rIr-r)rEs r nested_cssItemLoader.nested_csssd ##%}}(((==$$S))NNJ $J'J r8)recUR"U/UQ7SU0UD6nUcU$U(d,UR5HupgURXg5 M U$URX5 U$)a} Process and then add the given ``value`` for the given field. The value is first passed through :meth:`get_value` by giving the ``processors`` and ``kwargs``, and then passed through the :ref:`field input processor ` and its result appended to the data collected for that field. If the field already contains collected data, the new data is added. The given ``field_name`` can be ``None``, in which case values for multiple fields may be added. And the processed value should be a dict with field_name mapped to values. :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader Examples:: loader.add_value('name', 'Color TV') loader.add_value('colours', ['white', 'blue']) loader.add_value('length', '100') loader.add_value('name', 'name: foo', TakeFirst(), re='name: (.+)') loader.add_value(None, {'name': 'foo', 'sex': 'male'}) rL) get_valuer0 _add_valuer3r4r5rL processorskwkvs r add_valueItemLoader.add_valuesgBu?z?b?B? =K %&  OOJ . r8cUR"U/UQ7SU0UD6nUcU$U(d,UR5HupgURXg5 M U$URX5 U$)z Similar to :meth:`add_value` but replaces the collected data with the new value instead of adding it. :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader rL)rNr0_replace_valuerPs r replace_valueItemLoader.replace_valuesju?z?b?B? =K ##A)&     2 r8c[U5nURX5nU(a=URRU/5 URU==[U5- ss'ggr:)r_process_input_valuer1r2)r3r4r5processed_values rrOItemLoader._add_valuesQE"33JF  LL # #J 3 LL $ O(D D $ r8c^URRUS5 URX5 gr:)r1poprO)r3r4r5s rrXItemLoader._replace_values" T*  *r8cr^T(a [U5n[U4SjU55nUH)nUc U$Un[XPR5nU"U5nM+ U$![aRn[ SUR R<SU<S[U5R<S[U5<S3 5UeSnAff=f)a Process the given ``value`` by the given ``processors`` and keyword arguments. Available keyword arguments: :param re: a regular expression to use for extracting data from the given value using :func:`~parsel.utils.extract_regex` method, applied before processors :type re: str or typing.Pattern[str] Examples: >>> from itemloaders import ItemLoader >>> from itemloaders.processors import TakeFirst >>> loader = ItemLoader() >>> loader.get_value('name: foo', TakeFirst(), str.upper, re='name: (.+)') 'FOO' c3<># UHn[TU5v M g7fr:)r).0xrLs r 'ItemLoader.get_value..&s@%QM"a00%sNzError with processor  value= error=': ') rrrr- Exception ValueErrorrD__name__r"str)r3r5rLrQrRproc_proces ` rrNItemLoader.get_value s4 &E@%@@ED} E&t\\:D U     //Q8H8H#a&R s A B6$A B11B6c[UR5n[UR5HnUR U5nUcMX1U'M UR$)z Populate the item with the data collected so far, and return it. The data collected is first passed through the :ref:`output processors ` to get the final value to assign to each item field. )rr*tupler1get_output_value)r3adapterr4r5s r load_itemItemLoader.load_item6sO dii( -J))*5E &+ #. ||r8c .URU5n[X R5nURR U/5nU"U5$![ a>n[ SU<SU<S[U5R<S[U5<S3 5UeSnAff=f)z Return the collected values parsed using the output processor, for the given field. This method doesn't populate or modify the item at all. z#Error with output processor: field=rhrirjrkN) get_output_processorrr-r1getrlrmr"rnro)r3r4rpr5rrs rrvItemLoader.get_output_valueDs ((4"46   R0 ;  ud1g&6&6A@  sA B9BBc:URRU/5$)z0Return the collected values for the given field.)r1r|)r3r4s rget_collected_valuesItemLoader.get_collected_valuesTs|| B//r8c[USU-S5nU(dURUSUR5n[U5$)Nz%s_ininput_processor)getattr_get_item_field_attrr%rr3r4rps rget_input_processorItemLoader.get_input_processorXsCtWz148,,-t/K/KDd##r8c[USU-S5nU(dURUSUR5n[U5$)Nz%s_outoutput_processor)rrr&rrs rr{ItemLoader.get_output_processor`sCtX 2D9,,.0M0MDd##r8cl[UR5RU5nURX#5$r:)rr*get_field_metar|)r3r4keydefault field_metas rrItemLoader._get_item_field_attrhs,!+:::F ~~c++r8c*URU5nUn[X0R5nU"U5$![aVn[ SUR R <SU<SU<S[U5R <S[U5<S3 5UeSnAff=f)NzError with input processor z: field=rhrirjrk) rrr-rlrmrDrnr"ro)r3r4r5rprqrrs rr\ItemLoader._process_input_valuens'' 3"46 ;  OO,,G$$F   s2 BAB  BcbURc"[SURR-5eg)NzFTo use XPath or CSS selectors, %s must be instantiated with a selector)r) RuntimeErrorrDrnr;s rrB!ItemLoader._check_selector_methods6 == 79=9P9PQ  !r8cXUR"U40UD6nUR"X/UQ7SU0UD6$)ax Similar to :meth:`ItemLoader.add_value` but receives an XPath instead of a value, which is used to extract a list of strings from the selector associated with this :class:`ItemLoader`. See :meth:`get_xpath` for ``kwargs``. :param xpath: the XPath to extract data from :type xpath: str :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader Examples:: # HTML snippet:

Color TV

loader.add_xpath('name', '//p[@class="product-name"]') # HTML snippet:

the price is $1200

loader.add_xpath('price', '//p[@id="price"]', re='the price is (.*)') rL)_get_xpathvaluesrUr3r4rCrLrQrRvaluess r add_xpathItemLoader.add_xpaths7:&&u33~~jK:K"KKKr8cXUR"U40UD6nUR"X/UQ7SU0UD6$)z Similar to :meth:`add_xpath` but replaces collected data instead of adding it. :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader rL)rrYrs r replace_xpathItemLoader.replace_xpaths9&&u33!!*OzObOBOOr8cXUR"U40UD6nUR"U/UQ7SU0UD6$)a Similar to :meth:`ItemLoader.get_value` but receives an XPath instead of a value, which is used to extract a list of unicode strings from the selector associated with this :class:`ItemLoader`. :param xpath: the XPath to extract data from :type xpath: str :param re: a regular expression to use for extracting data from the selected XPath region :type re: str or typing.Pattern[str] Examples:: # HTML snippet:

Color TV

loader.get_xpath('//p[@class="product-name"]') # HTML snippet:

the price is $1200

loader.get_xpath('//p[@id="price"]', TakeFirst(), re='the price is (.*)') rL)rrN)r3rCrLrQrRrs r get_xpathItemLoader.get_xpaths76&&u33~~f?z?b?B??r8c ^^TR5 TRce[U5n[UU4SjU55$)Nc3z># UH0nTRR"U40TD6R5v M2 g7fr:)r)rCgetall)rdrCrRr3s rrf.ItemLoader._get_xpathvalues..s1UfUt}}**57B7>>@@fs8;rBr)rr)r3xpathsrRs` `rrItemLoader._get_xpathvaluess< ##%}}(((V$UfUUUr8cTURU5nUR"X/UQ7SU0UD6$)an Similar to :meth:`ItemLoader.add_value` but receives a CSS selector instead of a value, which is used to extract a list of unicode strings from the selector associated with this :class:`ItemLoader`. See :meth:`get_css` for ``kwargs``. :param css: the CSS selector to extract data from :type css: str :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader Examples:: # HTML snippet:

Color TV

loader.add_css('name', 'p.product-name') # HTML snippet:

the price is $1200

loader.add_css('price', 'p#price', re='the price is (.*)') rL)_get_cssvaluesrUr3r4rIrLrQrRrs radd_cssItemLoader.add_csss2:$$S)~~jK:K"KKKr8cTURU5nUR"X/UQ7SU0UD6$)z Similar to :meth:`add_css` but replaces collected data instead of adding it. :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader rL)rrYrs r replace_cssItemLoader.replace_csss4$$S)!!*OzObOBOOr8cTURU5nUR"U/UQ7SU0UD6$)as Similar to :meth:`ItemLoader.get_value` but receives a CSS selector instead of a value, which is used to extract a list of unicode strings from the selector associated with this :class:`ItemLoader`. :param css: the CSS selector to extract data from :type css: str :param re: a regular expression to use for extracting data from the selected CSS region :type re: str or typing.Pattern[str] Examples:: # HTML snippet:

Color TV

loader.get_css('p.product-name') # HTML snippet:

the price is $1200

loader.get_css('p#price', TakeFirst(), re='the price is (.*)') rL)rrN)r3rIrLrQrRrs rget_cssItemLoader.get_csss24$$S)~~f?z?b?B??r8c^TR5 TRce[U5n[U4SjU55$)Nc3v># UH.nTRRU5R5v M0 g7fr:)r)rIr)rdrIr3s rrf,ItemLoader._get_cssvalues..3s,G$3t}}((-4466$69r)r3cssss` rrItemLoader._get_cssvalues/s: ##%}}(((4 G$GGGr8cTURU5nUR"X/UQ7SU0UD6$)aV Similar to :meth:`ItemLoader.add_value` but receives a JMESPath selector instead of a value, which is used to extract a list of unicode strings from the selector associated with this :class:`ItemLoader`. See :meth:`get_jmes` for ``kwargs``. :param jmes: the JMESPath selector to extract data from :type jmes: str :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader Examples:: # HTML snippet: {"name": "Color TV"} loader.add_jmes('name') # HTML snippet: {"price": the price is $1200"} loader.add_jmes('price', TakeFirst(), re='the price is (.*)') rL)_get_jmesvaluesrUr3r4jmesrLrQrRrs radd_jmesItemLoader.add_jmes5s28%%d+~~jK:K"KKKr8cTURU5nUR"X/UQ7SU0UD6$)z Similar to :meth:`add_jmes` but replaces collected data instead of adding it. :returns: The current ItemLoader instance for method chaining. :rtype: ItemLoader rL)rrYrs r replace_jmesItemLoader.replace_jmesTs4%%d+!!*OzObOBOOr8cTURU5nUR"U/UQ7SU0UD6$)aX Similar to :meth:`ItemLoader.get_value` but receives a JMESPath selector instead of a value, which is used to extract a list of unicode strings from the selector associated with this :class:`ItemLoader`. :param jmes: the JMESPath selector to extract data from :type jmes: str :param re: a regular expression to use for extracting data from the selected JMESPath :type re: str or typing.Pattern Examples:: # HTML snippet: {"name": "Color TV"} loader.get_jmes('name') # HTML snippet: {"price": the price is $1200"} loader.get_jmes('price', TakeFirst(), re='the price is (.*)') rL)rrN)r3rrLrQrRrs rget_jmesItemLoader.get_jmeses24%%d+~~f?z?b?B??r8c^TR5 TRce[U5n[TRS5(d [ S5e[ U4SjU55$)Njmespathz6Please install parsel >= 1.8.1 to get jmespath supportc3v># UH.nTRRU5R5v M0 g7fr:)r)rr)rdrr3s rrf-ItemLoader._get_jmesvalues..s,Ot}}--d3::<<r)rBr)rhasattrrr)r3jmesss` rrItemLoader._get_jmesvaluess] ##%}}(((E"t}}j11 H OOOOr8)r,r/r-r.r))NNN)r*rr)zOptional[Selector]r.zOptional[ItemLoader]r-r)returnzDict[str, List[Any]])rr)rCror-rrr)rIror-rrr) r4 Optional[str]r5rrQr$rLUnion[str, Pattern[str], None]rRrrr)r4ror5rrNone) r5rrQr$rLrrRrrr)r4rorr)r4ror List[Any])r4rorr$r:)r4rorrrrrr)r4ror5rrr)rr) r4rrCUnion[str, Iterable[str]]rQr$rLrrRrrr) rCrrQr$rLrrRrrr)rrrRrrr) r4rrIrrQr$rLrrRrrr) rIrrQr$rLrrRrrr)rrrr) r4rrrorQr$rLrrRrrr) r4rrrrQr$rLrrRrrr) rrrQr$rLrrRrrr)rrrr)+rn __module__r__firstlineno____doc__dictr#__annotations__rr%r&r6propertyr1r*rFrJrUrYrOrXrNrxrvrrr{rr\rBrrrrrrrrrrrr__static_attributes__rAr8rr r /shDL $#2:*/<3;:0='+'+ ;;%;% ;  ;*&& $$  (.2 )!))( ) + )  ) )`.2 !(  +    2E+.2 **(* + *  * *X  0$$9=,,$',25, , &.2 L!L)L( L + L  L LJ.2 P!P)P( P + P  P P,.2 @(@(@ + @  @ @<V/V7:V V.2 L!L'L( L + L  L LJ.2 P!P'P( P + P  P P,.2 @ &@(@ + @  @ @:H.2 L!LL( L + L  L LH.2 P!P(P( P + P  P P*.2 @'@(@ + @  @ @:Pr8r N)rr$rr$)!r __future__r contextlibrtypingrrrrr r r r r r itemadapterrparselr parsel.utilsrritemloaders.commonritemloaders.processorsritemloaders.utilsrtyping_extensionsrrr rAr8rrsN #   $/2+)&[ P[ Pr8