HEX

File: //usr/local/lib64/python3.8/site-packages/lxml/html/__pycache__/diff.cpython-38.pyc
U

O��i݀�@s zddlZWn(ek
r4Gdd�d�Ze�ZYnXz8ddlmZddlZe�ej�r`ed��ddl	m
Z
Wn.ek
r�ddlZejs�ddlm
Z
YnXddlZddl
Z
ddlZddlZddlmZdd	lmZdd
lmZddgZe
jeje�d�d
�Zejd�eeed�dd��Zej�s.ddlmZdd�Z e fdd�Z!dd�Z"dd�Z#dd�Z$dd�Z%eje&d�dd ��Z'd!d"�Z(d#d�Z)d$d%�Z*d�d'd(�Z+e&d)�d*d+�Z,ejeed,�d-d.��Z-ej�s�eed,�d/d.�Z-Gd0d1�d1�Z.Gd2d3�d3�Z/e&d)�d4d5�Z0e&d6�d7d8�Z1eje&d9�d:d;��Z2Gd<d=�d=e�Z3Gd>d?�d?e3�Z4Gd@dA�dAe3�Z5d�dCdD�Z6d�dEdF�Z7e�8dGej9ej:B�j;Z<e�8dHej9ej:B�j;Z=e�8dIej9ej:B�j>Z?dJdK�Z@dLdM�ZAdNdO�ZBe�CeDejE�ZEe�CeDeDdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_d`dadbdcdddedfdgg��ZFe�CeDeDdhdidjdkdldmdndodpdqg
��ZGe�CeeeHeFeGB���ZId�drds�ZJe�8dtejK�jLZMdudv�ZNe�8dw�jOZPdxdy�ZQdzd{�ZRd|d}�ZSd~d�ZTd�d��ZUd�d��ZVd�d�d��ZWejd�d���ZXejd�d���ZYejd�d���ZZd�d��Z[ej\ej]Gd�d��d�e
���Z^e_d�k�rdd�lm`Z`e`�a�dS)��Nc@s0eZdZdZdd�Zdd�Zdd�Zdd	�Zd
S)�fake_cythonFcCs|S�N���self�funcrr�0/tmp/pip-install-gdmoqxf6/lxml/lxml/html/diff.py�cfunc�zfake_cython.cfunccCs|Srrrrrr�cclass	r
zfake_cython.cclasscCs|Srr)r�_�valuerrr�declare
r
zfake_cython.declarecCsdS)N�objectr)r�	type_namerrr�__getattr__r
zfake_cython.__getattr__N)�__name__�
__module__�__qualname__�compiledr	rrrrrrrrs
r�)�_difflibzLEmbedded difflib is not compiled to a fast binary, using the stdlib instead.)�SequenceMatcher)�etree)�fragment_fromstring)�defs�
html_annotate�htmldiff)�key�z&amp;z&lt;z&gt;z&quot;z&#x27;)�text�_escapes�returncCs�dgd}|D]h}|d|dkO<|d|dkO<|d|dkO<|d	|d
kO<|d|dkO<qtd�D] }||r�|�d
|||�}q�|S)NF�r�&r�<��>��"��'z&<>"')�range�replace)r r!r-�ch�irrr�html_escape-s
r0)�escapecCsdt|�|fS)Nz<span title="%s">%s</span>�r0)r �versionrrr�default_markupEs�r4cCsVdd�|D�}|d}|dd�D]}t||�|}q"t|�}t||�}d�|���S)a
    doclist should be ordered from oldest to newest, like::

        >>> version1 = 'Hello World'
        >>> version2 = 'Goodbye World'
        >>> print(html_annotate([(version1, 'version 1'),
        ...                      (version2, 'version 2')]))
        <span title="version 2">Goodbye</span> <span title="version 1">World</span>

    The documents must be *fragments* (str/UTF8 or unicode), not
    complete documents

    The markup argument is a function to markup the spans of words.
    This function is called like markup('Hello', 'version 2'), and
    returns HTML.  The first argument is text and never includes any
    markup.  The default uses a span with a title:

        >>> print(default_markup('Some Text', 'by Joe'))
        <span title="by Joe">Some Text</span>
    cSsg|]\}}t||��qSr)�tokenize_annotated)�.0�docr3rrr�
<listcomp>cs�z!html_annotate.<locals>.<listcomp>rrN�)�html_annotate_merge_annotations�compress_tokens�markup_serialize_tokens�join�strip)ZdoclistZmarkup�	tokenlistZ
cur_tokens�tokens�resultrrrrIs�

cCs t|dd�}|D]
}||_q|S)zFTokenize a document and add an annotation attribute to each token
    F��
include_hrefs)�tokenize�
annotation)r7rEr@�tokrrrr5qsr5cCsVt||d�}|��}|D]8\}}}}}|dkr|||�}	|||�}
t|	|
�qdS)z�Merge the annotations from tokens_old into tokens_new, when the
    tokens in the new document already existed in the old document.
    ��a�b�equalN)�InsensitiveSequenceMatcher�get_opcodes�copy_annotations)Z
tokens_oldZ
tokens_new�s�commands�command�i1�i2�j1�j2Zeq_oldZeq_newrrrr:ysr:cCs4t|�t|�kst�t||�D]\}}|j|_qdS)zN
    Copy annotations from the tokens listed in src to the tokens in dest
    N)�len�AssertionError�ziprE)�src�destZsrc_tokZdest_tokrrrrM�srMcCsV|dg}|dd�D]:}|jsF|djsF|dj|jkrFt||�q|�|�q|S)zl
    Combine adjacent tokens when there is no HTML between the tokens,
    and they share an annotation
    rrN���)�pre_tags�	post_tagsrE�compress_merge_back�append)r@rArFrrrr;�s
��r;)r@cCsd|d}t|�tk	s t|�tk	r,|�|�n4||j|}t||j|j|jd�}|j|_||d<dS)zY Merge tok into the last element of tokens (modifying the list of
    tokens in-place).  rZ�r[r\�trailing_whitespaceN)�type�tokenr^r`r[r\rE)r@rF�lastr Zmergedrrrr]�s�r]ccsF|D]<}|jEdH|��}|||j�|j}|V|jEdHqdS)zz
    Serialize the list of tokens into a list of text chunks, calling
    markup_func around text to add annotations.
    N)r[�htmlrEr`r\)r@Zmarkup_funcrbrdrrrr<�sr<c
Csht|�}t|�}t||�}zd�|���}Wn2ttfk
r^}zt|�d}W5d}~XYnXt|�S)a� Do a diff of the old and new document.  The documents are HTML
    *fragments* (str/UTF8 or unicode), they are not complete documents
    (i.e., no <html> tag).

    Returns HTML with <ins> and <del> tags added around the
    appropriate text.

    Markup is generally ignored, with the markup from new_html
    preserved, and possibly some markup from old_html (though it is
    considered acceptable to lose some of the old markup).  Only the
    words in the HTML are diffed.  The exception is <img> tags, which
    are treated like words, and the href attribute of <a> tags, which
    are noted inside the tag itself when there are changes.
    r9N)rD�htmldiff_tokensr=r>�
ValueError�	TypeError�print�fixup_ins_del_tags)Zold_htmlZnew_htmlZold_html_tokensZnew_html_tokensrA�excrrrr�s
cCs�t||d�}|��}g}|D]�\}}}}}	|dkrN|�t|||	�dd��q|dks^|dkrxt|||	��}
t|
|�|dks�|dkrt|||��}t||�qt|�|S)z] Does a diff on the tokens themselves, returning a list of text
    chunks (not tokens).
    rGrJT)rJ�insertr-�delete)rKrL�extend�
expand_tokens�merge_insert�merge_delete�cleanup_delete)Zhtml1_tokensZhtml2_tokensrNrOrArPrQrRrSrTZ
ins_tokensZ
del_tokensrrrre�s
reFccs@|D]6}|jEdH|r|js.|��|jV|jEdHqdS)zeGiven a list of tokens, return a generator of the chunks of
    text for the data in the tokens.
    N)r[�hide_when_equalrdr`r\)r@rJrbrrrrns

rn)r7cCs�tt|��D]�\}}dd�|D�}|dkr�|rL|d�d�sL|dd7<|�d�|�|�|d�d�r�|ddd�|d<|�d�q|�|�qdS)	z| doc is the already-handled document (as a list of text chunks);
    here we add <ins>ins_chunks</ins> to the end of that.  cSsg|]}|d�qS)rr�r6�itemrrrr8sz merge_insert.<locals>.<listcomp>rIrZ� z<ins>Nz</ins> )�group_by_first_item�mark_unbalanced�endswithr^rm)Z
ins_chunksr7�balancedZ
marked_chunks�chunksrrrro
s


ro)�chunkr"cCsp|ddkrdSd}t|�D]F\}}|dkr2d}q|dkrJ|||�S|��r|||�Sq||d�S)Nrr%r9r�/r&r')�	enumerate�isspace)r{�	start_posr/r.rrr�tag_name_of_chunk+sr�cCs|�dd�d�d�S)Nrrz<>/)�splitr>)r{rrrr�@sc@seZdZdS)�	DEL_STARTN�rrrrrrrr�Gsr�c@seZdZdS)�DEL_ENDNr�rrrrr�Isr�cCs"|�t�|�|�|�t�dS)z� Adds the text chunks in del_chunks to the document doc (another
    list of text chunks) with marker to show it is a delete.
    cleanup_delete later resolves these markers into <del> tags.N)r^r�rmr�)Z
del_chunksr7rrrrpMs

rp)rzcCst|�}d}z|�t|�}Wntk
r6Y�qYnX|�t|d�}d}}d}}t||d|��}	|	D]�\}
}|
dkr��q"|d7}t|�}t|d|�D]z}
||
tkr�qp||
}|ddks�|ddkr�qpt|�}|dkr�qp|dk�std|����||k�rqp|d7}q�qpt	|	�D]�\}
}|
d	k�rB�q�|d7}t|�}t|dd
d
�D]~}
||
tk�rz�q*||
}|ddk�r�|ddk�r��q*t|�}|dk�s�|dk�r��q*||k�r��q*|d7}�qb�q*||}t|d||d�D]}
||
||<|d7}�q|�rN||d�
d��sN||dd7<d||<|d7}t||d||�D]}
||
||<|d7}�qt||d�
d��r�||dd
d
�||d<d||<|d7}t|||�D]}
||
||<|d7}�q�||||d�=|}qd
S)a� Cleans up any DEL_START/DEL_END markers in the document, replacing
    them with <del></del>.  To do this while keeping the document
    valid, it may need to drop some tags (either start or end tags).

    It may also move the del into adjacent tags to try to move it to a
    similar location where it was originally located (e.g., moving a
    delete into preceding <div> tag, if the del looks like (DEL_START,
    'Text</div>', DEL_END)
    rr�usr%r|�ins�delzUnexpected delete tag: �uerZruz<del>Nz</del> )rU�indexr�rfr�rwr�r,rV�reversedrx)rzZchunk_countrZ	del_startZdel_endZshift_end_leftZshift_start_rightZunbalanced_startZunbalanced_endZdeleted_chunksryZ	del_chunkZunbalanced_start_namer/r{�nameZunbalanced_end_name�posrrrrqWs�




rq�r"c	Cs$g}g}|D]�}|�d�s*|�d|f�qt|�}|tkrJ|�d|f�q|ddkr�|r�|��\}}}||kr�|�d|f�|�|�|�d|f�|}d}q�qV|�d|f�|�|�|}qV|dk	r�|�d|f�q|�|||f�g}q|�r |��\}}}|�d|f�|�|�|}q�|S)Nr%rIrr|r�r�)�
startswithr^r��
empty_tags�poprm)	rzZ	tag_stackZmarkedr{r�Z
start_nameZstart_chunk�parentsrrrrrw�sB



rwc@s.eZdZdZdZddd�Zdd�Zd	d
�ZdS)rba8 Represents a diffable token, generally a word that is displayed to
    the user.  Opening tags are attached to this token when they are
    adjacent (pre_tags) and closing tags that follow the word
    (post_tags).  Some exceptions occur when there are empty tags
    adjacent to a word, so there may be close tags in pre_tags, or
    open tags in post_tags.

    We also keep track of whether the word was originally followed by
    whitespace, even though we do not want to treat the word as
    equivalent to a similar word that does not have a trailing
    space.FNr9cCs:t�||�}|dk	r|ng|_|dk	r*|ng|_||_|Sr)�str�__new__r[r\r`)�clsr r[r\r`�objrrrr�)s
z
token.__new__cCsdt�|�|j|j|jfS)Nztoken(%s, %r, %r, %r))r��__repr__r[r\r`�rrrrr�2s�ztoken.__repr__cCst|�Sr)r�r�rrrrd6sz
token.html)NNr9)rrr�__doc__rrr�r�rdrrrrrbs

	rbc@s*eZdZdZd
dd�Zdd�Zdd	�ZdS)�	tag_tokenz� Represents a token that is actually a tag.  Currently this is just
    the <img> tag, which takes up visible space just like a word but
    is only represented in a document by a tag.  Nr9cCs4tj|t�d|��|||d�}||_||_||_|S)Nz: r_)rbr�ra�tag�data�	html_repr)r�r�r�r�r[r\r`r�rrrr�?s�ztag_token.__new__cCs d|j|j|j|j|j|jfS)NzRtag_token(%s, %s, html_repr=%s, post_tags=%r, pre_tags=%r, trailing_whitespace=%r))r�r�r�r[r\r`r�rrrr�Js�ztag_token.__repr__cCs|jSr)r�r�rrrrdRsztag_token.html)NNr9)rrrr�r�r�rdrrrrr�9s�
r�c@seZdZdZdZdd�ZdS)�
href_tokenzh Represents the href in an anchor tag.  Unlike other words, we only
    show the href when it changes.  TcCsd|S)Nz	 Link: %srr�rrrrd\szhref_token.htmlN)rrrr�rrrdrrrrr�Usr�TcCs2t�|�r|}nt|dd�}t|d|d�}t|�S)ak
    Parse the given HTML and returns token objects (words with attached tags).

    This parses only the content of a page; anything in the head is
    ignored, and the <head> and <body> elements are themselves
    optional.  The content is then parsed by lxml, which ensures the
    validity of the resulting parsed document (though lxml may make
    incorrect guesses when the markup is particular bad).

    <ins> and <del> tags are also eliminated from the document, as
    that gets confusing.

    If include_hrefs is true, then the href attribute of <a> tags is
    included as a special kind of diffable token.T��cleanup)�skip_tagrC)r�	iselement�
parse_html�
flatten_el�fixup_chunks)rdrCZbody_elrzrrrrD`s

rDcCs|rt|�}t|dd�S)a
    Parses an HTML fragment, returning an lxml element.  Note that the HTML will be
    wrapped in a <div> tag that was not in the original document.

    If cleanup is true, make sure there's no <head> or <body>, and get
    rid of any <ins> and <del> tags.
    T)Z
create_parent)�cleanup_htmlr)rdr�rrrr�ysr�z	<body.*?>z
</body.*?>z</?(ins|del).*?>cCsFt|�}|r||��d�}t|�}|r8|d|���}td|�}|S)z� This 'cleans' the HTML, meaning that any page structure is removed
    (only the contents of <body> are used, if there is any <body).
    Also <ins> and <del> tags are removed.  Nr9)�_search_body�end�_search_end_body�start�_replace_ins_del)rd�matchrrrr��s
r�cCs$t|���}|d|�||d�fS)zP
    This function takes a word, such as 'test

' and returns ('test','

')
    rN)rU�rstrip)�wordZstripped_lengthrrr�split_trailing_whitespace�sr�c	CsRg}d}g}|D�]}t|t�r�|ddkrf|d}t|d�\}}td||||d�}g}|�|�q|ddkr|d}t||dd	�}g}|�|�qt|�r�t|�\}}t|||d	�}g}|�|�qt|�r�|�|�qt	|��r |r�|�|�n&|�st
d
||||f��|j�|�qdst
�q|�s>td|d
�gS|dj�|�|S)zM
    This function takes a list of chunks and produces a list of tokens.
    Nr�imgrr&)r�r[r`�hrefru)r[r`z4Weird state, cur_word=%r, result=%r, chunks=%r of %rFr9)r[rZ)
�
isinstance�tupler�r�r^r��is_wordrb�is_start_tag�
is_end_tagrVr\rm)	rzZ	tag_accumZcur_wordrAr{rXr�r`r�rrrr��sR

�


��
r��address�
blockquote�center�dir�div�dl�fieldset�form�h1�h2�h3�h4�h5�h6�hr�isindex�menu�noframes�noscript�ol�p�pre�table�ul�dd�dt�frameset�li�tbody�td�tfoot�th�thead�trccs�|s0|jdkr&d|�d�t|�fVn
t|�V|jtkrR|jsRt|�sR|jsRdSt|j�}|D]}t|�Vq`|D]}t	||d�EdHqt|jdkr�|�d�r�|r�d|�d�fV|s�t
|�Vt|j�}|D]}t|�Vq�dS)a Takes an lxml element el, and generates all the text chunks for
    that tag.  Each start tag is a chunk, each word is a chunk, and each
    end tag is a chunk.

    If skip_tag is true, then the outermost container tag is
    not returned (just its contents).r�rXNrBrHr�)r��get�	start_tagr�r rU�tail�split_wordsr0r��end_tag)�elrCr�Zstart_wordsr��childZ	end_wordsrrrr�s$




r�z\S+(?:\s+|$)cCs|r|��sgSt|�}|S)z_ Splits some text into words. Includes trailing whitespace
    on each word when appropriate.  )r>�_find_words)r �wordsrrrr�#sr�z
^[ \t\n\r]cCs,d�dd�|j��D��}d|j�|�d�S)z=
    The text representation of the start tag for a tag.
    r9cSs&g|]\}}d|�dt|��d��qS)ruz="r)r2)r6r�r
rrrr82s�zstart_tag.<locals>.<listcomp>r%r')r=�attrib�itemsr�)r��
attributesrrrr�.s
�r�cCs,|j}|rt|�rdnd}d|j�d|��S)zg The text representation of an end tag for a tag.  Includes
    trailing whitespace when appropriate.  rur9�</r')r��_has_start_whitespacer�)r�r��extrarrrr�8sr�cCs|�d�S)Nr%�r��rFrrrr�?sr�cCs
|�d�S)Nr�r�r�rrrr�Bsr�cCs|�d�o|�d�S)Nr%r�r�r�rrrr�Esr�cCs$t|dd�}t|�t|dd�}|S)z� Given an html string, move any <ins> or <del> tags inside of any
    block-level elements, e.g. transform <ins><p>word</p></ins> to
    <p><ins>word</ins></p> Fr�T)�
skip_outer)r��_fixup_ins_del_tags�serialize_html_fragment)rdr7rrrriHsricCsdt|t�rtd|����tj|ddd�}|r\||�d�dd�}|d|�d��}|��S|SdS)	z� Serialize a single lxml element as HTML.  The serialized form
    includes the elements tail.

    If skip_outer is true, then don't serialize the outermost tag
    z1You should pass in an element, not a string like rd�unicode)�method�encodingr'rNr%)r�r�rVr�tostring�find�rfindr>)r�r�rdrrrr�Qs�r�cCs:t|�dd��D]$}t|�sqt||jd�|��qdS)z?fixup_ins_del_tags that works on an lxml document in-place
    r�r�)r�N)�list�iter�_contains_block_level_tag�_move_el_inside_blockr�Zdrop_tag)r7r�rrrr�ds
r�cCs|jt�D]}dSdS)zPTrue if the element contains any block-level elements, like <p>, <td>, etc.
    TF)r��any_block_level_tag)r�rrrr�psr�c	Cs�|j}|jt�D]}||k	rqXq||�}|j|_d|_|�t|��|g|dd�<dSt|�D]\}t|�r�t||�|jr�||�}|j|_d|_|�	|�q`||�}|�
||�|�|�q`|jr�||�}|j|_d|_|�d|�dS)zt helper for _fixup_ins_del_tags; actually takes the <ins> etc tags
    and moves them inside any block-level tags.  Nr)
�makeelementr�r�r rmr�r�r�r�Zaddnextr-r^rk)	r�r�r�Zblock_level_elZchildren_tagr�Ztail_tagZ	child_tagZtext_tagrrrr�ys4
r�cCs�|��}|j}|j}|rFt|�s.|p&d|}n|djp:d||d_|�|�}|r�|��}|dkrv|jpld||_n|jp~d||_|��|||d�<dS)z�
    Removes an element, but merges its contents into its place, e.g.,
    given <p>Hi <i>there!</i></p>, if you remove the <i> element you get
    <p>Hi there!</p>
    r9rZNr)Z	getparentr r�rUr�Zgetprevious�getchildren)r��parentr r�r��previousrrr�_merge_element_contents�s
r�c@s(eZdZdZdZejed�dd��ZdS)rKzt
    Acts like SequenceMatcher, but tries not to find very small equal
    blocks amidst large spans of changes
    r&r�csFtt|j�t|j��}|j�t�|d��t�|�}�fdd�|D�S)Nr*cs$g|]}|d�ks|ds|�qS)r&rrs��	thresholdrrr8�s�zBInsensitiveSequenceMatcher.get_matching_blocks.<locals>.<listcomp>)�minrUrIr�r�get_matching_blocks)r�size�actualrr�rr��s

z.InsensitiveSequenceMatcher.get_matching_blocksN)	rrrr�r��cythonr	r�r�rrrrrK�srK�__main__)�_diffcommand)r)F)T)T)F)F)br�ImportErrorrr9r�difflib�inspect�
isfunction�get_close_matchesZ"cython.cimports.lxml.html._difflibrr�	itertools�	functools�operator�re�lxmlrZ	lxml.htmlrr�__all__�partial�groupby�
itemgetterrvr	r�r�r0rdr1r4rr5r:rMr;r�r]r<rrernror�r�r�rprqrwrbr�r�rDr��compile�I�S�searchr�r��subr�r�r�r�r�	frozensetr�Zblock_level_tagsZblock_level_container_tags�sortedr�r��U�findallr�r�r�r�r�r�r�r�r�rir�r�r�r�r��finalrrKrr�mainrrrr�<module>s�(
(

5!

6��
�
	
	



#