o LDi%D@s"dZddlmZdZdgZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZmZmZddlmZmZmZmZmZmZddlmZmZdd lmZmZm Z m!Z!dd l"m#Z#erodd l$m%Z%dd lm&Z&dd l'm(Z(m)Z)m*Z*dZ+e e e,e,fe,e,gdfZ-GdddeeZ.Gddde Z/dS)zCUse the HTMLParser library to parse HTML files that aren't too bad.) annotationsMITHTMLParserTreeBuilder) HTMLParser) AnyCallablecastDictIterableListOptional TYPE_CHECKINGTupleTypeUnion) AttributeDictCDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution UnicodeDammit)DetectsXMLParsedAsHTMLHTMLHTMLTreeBuilderSTRICTParserRejectedMarkup) BeautifulSoup)NavigableString) _Encoding _Encodings _RawMarkupz html.parserNc@seZdZUdZded<dZded< edd6ddZd ed <ded<ded<d7ddZd8ddZ d9d:dd Z d9d;d"d#Z dd/d0Zd. F)handle_empty_elementN)handle_starttag handle_endtag)r6r=r>r7r7r8handle_startendtags z*BeautifulSoupHTMLParser.handle_startendtagTr@boolc Cs|}|D]3\}}|durd}||vr5|j}||jkrq|d|jfvr)|||<qtt|}||||q|||<q|jjjrF| \}} nd}} |jj |dd||| d} | durl| j rl|rl|j |dd|j ||jdurx||dSdS)zHandle an opening tag, e.g. '' :param handle_empty_element: True if this tag is known to be an empty-element tag (i.e. there is not expected to be any closing tag). N) sourceline sourceposF)check_already_closed)r2r+r)r'r_DuplicateAttributeHandlerr,r1store_line_numbersgetposrAis_empty_elementrBr4append_root_tag_name_root_tag_encountered) r6r=r>r@ attr_dictkeyvalueon_duperFrGtagObjr7r7r8rAs2         z'BeautifulSoupHTMLParser.handle_starttagrHcCs.|r||jvr|j|dS|j|dS)zHandle a closing tag, e.g. '' :param tag: A tag name. :param check_already_closed: True if this tag is expected to be the closing portion of an empty-element tag, e.g. ''. N)r4remover,rB)r6r=rHr7r7r8rBs z%BeautifulSoupHTMLParser.handle_endtagdatacCs|j|dS)z4Handle some textual data that shows up between tags.N)r, handle_datar6rVr7r7r8rWsz#BeautifulSoupHTMLParser.handle_datanamecCsh|drt|dd}n|drt|dd}nt|}t|\}}|r-d|j_||dS)zHandle a numeric character reference by converting it to the corresponding Unicode character and treating it as textual data. :param name: Character number, possibly in hexadecimal. xXTN) startswithintlstriprnumeric_character_referencer,contains_replacement_charactersrW)r6rY real_namerVreplacement_addedr7r7r8handle_charrefs  z&BeautifulSoupHTMLParser.handle_charrefcCs0tj|}|dur |}nd|}||dS)zHandle a named entity reference by converting it to the corresponding Unicode character(s) and treating it as textual data. :param name: Name of the entity reference. Nz&%s)rHTML_ENTITY_TO_CHARACTERgetrW)r6rY characterrVr7r7r8handle_entityrefs z(BeautifulSoupHTMLParser.handle_entityrefcCs&|j|j||jtdS)zOHandle an HTML comment. :param data: The text of the comment. N)r,endDatarWrrXr7r7r8handle_comment s  z&BeautifulSoupHTMLParser.handle_commentdeclcCs6|j|tdd}|j||jtdS)zYHandle a DOCTYPE declaration. :param data: The text of the declaration. zDOCTYPE N)r,rilenrWr)r6rkr7r7r8 handle_decls  z#BeautifulSoupHTMLParser.handle_declcCsN|drt}|tdd}nt}|j|j||j|dS)z{Handle a declaration of unknown type -- probably a CDATA block. :param data: The text of the declaration. zCDATA[N)upperr]rrlrr,rirW)r6rVclsr7r7r8 unknown_decl s  z$BeautifulSoupHTMLParser.unknown_declcCs0|j|j||||jtdS)z\Handle a processing instruction. :param data: The text of the instruction. N)r,rirW_document_might_be_xmlrrXr7r7r8 handle_pi/s   z!BeautifulSoupHTMLParser.handle_piN)r,rr-rr+r.r/r)r9r&r:r;)r=r&r>r?r:r;)T)r=r&r>r?r@rDr:r;)r=r&rHrDr:r;)rVr&r:r;)rYr&r:r;)rkr&r:r;)__name__ __module__ __qualname__r'__annotations__r)r3r<rCrArBrWrdrhrjrmrprrr7r7r7r8r$=s*     >     r$cseZdZUdZdZded<dZded<eZded<ee e gZ d ed <d ed <dZ ded <  d&d'fdd Z   d(d)dd Zefd*d$d%ZZS)+rzA Beautiful soup `bs4.builder.TreeBuilder` that uses the :py:class:`html.parser.HTMLParser` parser, found in the Python standard library. FrDis_xmlT picklabler&NAMEz Iterable[str]featuresz$Tuple[Iterable[Any], Dict[str, Any]] parser_argsTRACKS_LINE_NUMBERSNOptional[Iterable[Any]] parser_kwargsOptional[Dict[str, Any]]r/rc spt}dD]}||vr||}|||<qtt|jdi||p#g}|p'i}||d|d<||f|_dS)aConstructor. :param parser_args: Positional arguments to pass into the BeautifulSoupHTMLParser constructor, once it's invoked. :param parser_kwargs: Keyword arguments to pass into the BeautifulSoupHTMLParser constructor, once it's invoked. :param kwargs: Keyword arguments for the superclass constructor. r*Fconvert_charrefsNr7)dictpopsuperrr3updater{)r6r{r~r/extra_parser_kwargsargrR __class__r7r8r3Ks  zHTMLParserTreeBuilder.__init__markupr#user_specified_encodingOptional[_Encoding]document_declared_encodingexclude_encodingsOptional[_Encodings]r:DIterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]]ccst|tr|dddfVdSg}|r||g}|r!||t|||d|d}|jdur3td|j|j|j|jfVdS)a2Run any preliminary steps necessary to make incoming markup acceptable to the parser. :param markup: Some markup -- probably a bytestring. :param user_specified_encoding: The user asked to try this encoding. :param document_declared_encoding: The markup itself claims to be in this encoding. :param exclude_encodings: The user asked _not_ to try any of these encodings. :yield: A series of 4-tuples: (markup, encoding, declared encoding, has undergone character replacement) Each 4-tuple represents a strategy for parsing the document. This TreeBuilder uses Unicode, Dammit to convert the markup into Unicode, so the ``markup`` element of the tuple will always be a string. NFT)known_definite_encodingsuser_encodingsis_htmlrzPCould not convert input to Unicode, and html.parser will not accept bytestrings.) isinstancer&rMrunicode_markuproriginal_encodingdeclared_html_encodingra)r6rrrrrrdammitr7r7r8prepare_markupis4     z$HTMLParserTreeBuilder.prepare_markup _parser_classtype[BeautifulSoupHTMLParser]r;c Cs|j\}}t|ts J|jdusJ||jg|Ri|}z |||Wnty:}zt|d}~wwg|_dS)z :param markup: The markup to feed into the parser. :param _parser_class: An HTMLParser subclass to use. This is only intended for use in unit tests. N) r{rr&r,feedcloseAssertionErrorrr4)r6rrr-r/parserer7r7r8rs    zHTMLParserTreeBuilder.feed)NN)r{r}r~rr/r)NNN) rr#rrrrrrr:r)rr#rrr:r;)rsrtru__doc__rwrvrx HTMLPARSERryrrrzr|r3rr$r __classcell__r7r7rr8r:s     ! H)0r __future__r __license____all__ html.parserrtypingrrrr r r r r rrr bs4.elementrrrrrr bs4.dammitrr bs4.builderrrrrbs4.exceptionsrbs4rr bs4._typingr!r"r#rr&rIr$rr7r7r7r8s(  4    ~