HEX

File: //opt/saltstack/salt/lib/python3.10/site-packages/pygments/__pycache__/lexer.cpython-310.pyc
o

;jR��@s�dZddlZddlZddlZddlmZmZddlmZddl	m
Z
mZmZm
Z
mZddlmZmZmZmZmZmZddlmZgd�Ze�d	�Zgd
�Zedd��ZGd
d�de�ZGdd�ded�Z Gdd�de �Z!Gdd�de"�Z#Gdd�d�Z$e$�Z%Gdd�de&�Z'Gdd�d�Z(dd�Z)Gdd�d�Z*e*�Z+d d!�Z,Gd"d#�d#�Z-Gd$d%�d%e�Z.Gd&d'�d'e�Z/Gd(d)�d)e e/d�Z0Gd*d+�d+�Z1Gd,d-�d-e0�Z2d.d/�Z3Gd0d1�d1e/�Z4Gd2d3�d3e0e4d�Z5dS)4z�
    pygments.lexer
    ~~~~~~~~~~~~~~

    Base lexer classes.

    :copyright: Copyright 2006-present by the Pygments team, see AUTHORS.
    :license: BSD, see LICENSE for details.
�N)�
apply_filters�Filter)�get_filter_by_name)�Error�Text�Other�
Whitespace�
_TokenType)�get_bool_opt�get_int_opt�get_list_opt�make_analysator�Future�guess_decode)�	regex_opt)
�Lexer�
RegexLexer�ExtendedRegexLexer�DelegatingLexer�LexerContext�include�inherit�bygroups�using�this�default�words�line_rez.*?
))s�utf-8)s��zutf-32)s��zutf-32be)s��zutf-16)s��zutf-16becC�dS)N����xr!r!�B/opt/saltstack/salt/lib/python3.10/site-packages/pygments/lexer.py�<lambda>"�r%c@�eZdZdZdd�ZdS)�	LexerMetaz�
    This metaclass automagically converts ``analyse_text`` methods into
    static methods which always return float values.
    cCs(d|vrt|d�|d<t�||||�S)N�analyse_text)r
�type�__new__)Zmcs�name�bases�dr!r!r$r++szLexerMeta.__new__N)�__name__�
__module__�__qualname__�__doc__r+r!r!r!r$r(%sr(c@sneZdZdZdZgZgZgZgZdZ	dZ
dZdZdd�Z
dd�Zdd	�Zd
d�Zdd
�Zddd�Zdd�ZdS)ra"
    Lexer for a specific language.

    See also :doc:`lexerdevelopment`, a high-level guide to writing
    lexers.

    Lexer classes have attributes used for choosing the most appropriate
    lexer based on various criteria.

    .. autoattribute:: name
       :no-value:
    .. autoattribute:: aliases
       :no-value:
    .. autoattribute:: filenames
       :no-value:
    .. autoattribute:: alias_filenames
    .. autoattribute:: mimetypes
       :no-value:
    .. autoattribute:: priority

    Lexers included in Pygments should have two additional attributes:

    .. autoattribute:: url
       :no-value:
    .. autoattribute:: version_added
       :no-value:

    Lexers included in Pygments may have additional attributes:

    .. autoattribute:: _example
       :no-value:

    You can pass options to the constructor. The basic options recognized
    by all lexers and processed by the base `Lexer` class are:

    ``stripnl``
        Strip leading and trailing newlines from the input (default: True).
    ``stripall``
        Strip all leading and trailing whitespace from the input
        (default: False).
    ``ensurenl``
        Make sure that the input ends with a newline (default: True).  This
        is required for some lexers that consume input linewise.

        .. versionadded:: 1.3

    ``tabsize``
        If given and greater than 0, expand tabs in the input (default: 0).
    ``encoding``
        If given, must be an encoding name. This encoding will be used to
        convert the input string to Unicode, if it is not already a Unicode
        string (default: ``'guess'``, which uses a simple UTF-8 / Locale /
        Latin1 detection.  Can also be ``'chardet'`` to use the chardet
        library, if it is installed.
    ``inencoding``
        Overrides the ``encoding`` if given.
    NrcKs�||_t|dd�|_t|dd�|_t|dd�|_t|dd�|_|�dd	�|_|�d
�p-|j|_g|_	t
|dd�D]}|�|�q8d
S)a�
        This constructor takes arbitrary options as keyword arguments.
        Every subclass must first process its own options and then call
        the `Lexer` constructor, since it processes the basic
        options like `stripnl`.

        An example looks like this:

        .. sourcecode:: python

           def __init__(self, **options):
               self.compress = options.get('compress', '')
               Lexer.__init__(self, **options)

        As these options must all be specifiable as strings (due to the
        command line usage), there are various utility functions
        available to help with that, see `Utilities`_.
        �stripnlT�stripallF�ensurenl�tabsizer�encoding�guessZ
inencoding�filtersr!N)�optionsr
r3r4r5rr6�getr7r9r�
add_filter)�selfr:�filter_r!r!r$�__init__�s�zLexer.__init__cCs.|jrd|jj�d|j�d�Sd|jj�d�S)Nz<pygments.lexers.z with �>)r:�	__class__r/�r=r!r!r$�__repr__�szLexer.__repr__cKs*t|t�s
t|fi|��}|j�|�dS)z8
        Add a new stream filter to this lexer.
        N)�
isinstancerrr9�append)r=r>r:r!r!r$r<�s
zLexer.add_filtercCr)a�
        A static method which is called for lexer guessing.

        It should analyse the text and return a float in the range
        from ``0.0`` to ``1.0``.  If it returns ``0.0``, the lexer
        will not be selected as the most probable one, if it returns
        ``1.0``, it will be selected immediately.  This is used by
        `guess_lexer`.

        The `LexerMeta` metaclass automatically wraps this function so
        that it works like a static method (no ``self`` or ``cls``
        parameter) and the return value is automatically converted to
        `float`. If the return value is an object that is boolean `False`
        it's the same as if the return values was ``0.0``.
        Nr!)�textr!r!r$r)�r&zLexer.analyse_textc	
Csvt|t�sy|jdkrt|�\}}nu|jdkrezddl}Wnty-}ztd�|�d}~wwd}tD]\}}|�|�rI|t|�d��	|d�}nq2|durb|�
|dd��}|�	|�d�p_d	d�}|}n!|�	|j�}|�d
�rx|td
�d�}n
|�d
�r�|td
�d�}|�dd�}|�d
d�}|j
r�|��}n|jr�|�d�}|jdkr�|�|j�}|jr�|�d�s�|d7}|S)zVApply preprocessing such as decoding the input, removing BOM and normalizing newlines.r8�chardetrNzkTo enable chardet encoding guessing, please install the chardet library from http://chardet.feedparser.org/�replaceir7ruz
�
�
)rD�strr7rrG�ImportError�
_encoding_map�
startswith�len�decodeZdetectr;rHr4�stripr3r6�
expandtabsr5�endswith)	r=rF�_rG�e�decodedZbomr7�encr!r!r$�_preprocess_lexer_input�sR


���
��
�



zLexer._preprocess_lexer_inputFcs4�������fdd�}|�}|st|�j��}|S)ae
        This method is the basic interface of a lexer. It is called by
        the `highlight()` function. It must process the text and return an
        iterable of ``(tokentype, value)`` pairs from `text`.

        Normally, you don't need to override this method. The default
        implementation processes the options recognized by all lexers
        (`stripnl`, `stripall` and so on), and then yields all tokens
        from `get_tokens_unprocessed()`, with the ``index`` dropped.

        If `unfiltered` is set to `True`, the filtering mechanism is
        bypassed even if filters are defined.
        c3s&�����D]
\}}}||fVqdS�N)�get_tokens_unprocessed)rT�t�v�r=rFr!r$�streamer
s��z"Lexer.get_tokens.<locals>.streamer)rXrr9)r=rF�
unfilteredr^�streamr!r]r$�
get_tokens�s
zLexer.get_tokenscCst�)aS
        This method should process the text and return an iterable of
        ``(index, tokentype, value)`` tuples where ``index`` is the starting
        position of the token within the input text.

        It must be overridden by subclasses. It is recommended to
        implement it as a generator to maximize effectiveness.
        )�NotImplementedErrorr]r!r!r$rZs	zLexer.get_tokens_unprocessed)F)r/r0r1r2r,�aliases�	filenamesZalias_filenamesZ	mimetypes�priority�urlZ
version_addedZ_exampler?rCr<r)rXrarZr!r!r!r$r1s$;
/r)�	metaclassc@s$eZdZdZefdd�Zdd�ZdS)ra 
    This lexer takes two lexer as arguments. A root lexer and
    a language lexer. First everything is scanned using the language
    lexer, afterwards all ``Other`` tokens are lexed using the root
    lexer.

    The lexers from the ``template`` lexer package use this base lexer.
    cKs<|di|��|_|di|��|_||_tj|fi|��dS�Nr!)�
root_lexer�language_lexer�needlerr?)r=Z_root_lexerZ_language_lexerZ_needler:r!r!r$r?+szDelegatingLexer.__init__cCs�d}g}g}|j�|�D]$\}}}||jur(|r#|�t|�|f�g}||7}q|�|||f�q|r<|�t|�|f�t||j�|��S)N�)rjrZrkrErO�
do_insertionsri)r=rFZbuffered�
insertionsZ
lng_buffer�ir[r\r!r!r$rZ1s


�z&DelegatingLexer.get_tokens_unprocessedN)r/r0r1r2rr?rZr!r!r!r$r!s	rc@�eZdZdZdS)rzI
    Indicates that a state should include rules from another state.
    N�r/r0r1r2r!r!r!r$rHsrc@r')�_inheritzC
    Indicates the a state should inherit from its superclass.
    cCr)Nrr!rBr!r!r$rCS�z_inherit.__repr__N)r/r0r1r2rCr!r!r!r$rrOsrrc@s eZdZdZdd�Zdd�ZdS)�combinedz:
    Indicates a state combined from multiple states.
    cGst�||�SrY)�tupler+)�cls�argsr!r!r$r+^szcombined.__new__cGsdSrYr!)r=rwr!r!r$r?aszcombined.__init__N)r/r0r1r2r+r?r!r!r!r$rtYsrtc@sFeZdZdZdd�Zddd�Zddd�Zdd	d
�Zdd�Zd
d�Z	dS)�_PseudoMatchz:
    A pseudo match object constructed from a string.
    cCs||_||_dSrY)�_text�_start)r=�startrFr!r!r$r?ks
z_PseudoMatch.__init__NcCs|jSrY)rz�r=�argr!r!r$r{osz_PseudoMatch.startcCs|jt|j�SrY)rzrOryr|r!r!r$�endrsz_PseudoMatch.endcCs|rtd��|jS)Nz
No such group)�
IndexErrorryr|r!r!r$�groupusz_PseudoMatch.groupcCs|jfSrY)ryrBr!r!r$�groupszsz_PseudoMatch.groupscCsiSrYr!rBr!r!r$�	groupdict}rsz_PseudoMatch.groupdictrY)
r/r0r1r2r?r{r~r�r�r�r!r!r!r$rxfs


rxcsd�fdd�	}|S)zL
    Callback that yields multiple actions for each group in the match.
    Nc3s��t��D]O\}}|durqt|�tur)|�|d�}|r(|�|d�||fVq|�|d�}|durT|r>|�|d�|_||t|�|d�|�|�D]}|rS|VqLq|r^|��|_dSdS)N�)�	enumerater*r	r�r{�posrxr~)�lexer�match�ctxro�action�data�item�rwr!r$�callback�s,�����zbygroups.<locals>.callbackrYr!)rwr�r!r�r$r�src@rp)�_ThiszX
    Special singleton used for indicating the caller class.
    Used by ``using``.
    Nrqr!r!r!r$r��sr�csli�d�vr��d�}t|ttf�r|�d<nd|f�d<�tur+d��fdd�	}|Sd���fdd�	}|S)	a�
    Callback that processes the match with a different lexer.

    The keyword arguments are forwarded to the lexer, except `state` which
    is handled separately.

    `state` specifies the state that the new lexer will start in, and can
    be an enumerable such as ('root', 'inline', 'string') or a simple
    string which is assumed to be on top of the root state.

    Note: For that to work, `_other` must not be an `ExtendedRegexLexer`.
    �state�stack�rootNc	3s���rt|j�}|���|jdi|��}n|}|��}|j|��fi���D]
\}}}||||fVq'|r>|��|_dSdSrh)	�dictr:�updaterAr{rZr�r~r��	r�r�r�r.Zlx�sror[r\)�	gt_kwargs�kwargsr!r$r��s�

 �zusing.<locals>.callbackc	3st�t|j�}|����di|��}|��}|j|��fi���D]
\}}}||||fVq!|r8|��|_dSdSrh)r�r:r�r{rZr�r~r�r���_otherr�r�r!r$r��s�

 �rY)�poprD�listrur)r�r�r�r�r!r�r$r�s


�rc@r')rz�
    Indicates a state or state action (e.g. #pop) to apply.
    For example default('#pop') is equivalent to ('', Token, '#pop')
    Note that state tuples may be used as well.

    .. versionadded:: 2.0
    cCs
||_dSrY)r�)r=r�r!r!r$r?�s
zdefault.__init__N)r/r0r1r2r?r!r!r!r$r�src@s"eZdZdZddd�Zdd�ZdS)	rz�
    Indicates a list of literal words that is transformed into an optimized
    regex that matches any of the words.

    .. versionadded:: 2.0
    rlcCs||_||_||_dSrY)r�prefix�suffix)r=rr�r�r!r!r$r?�s
zwords.__init__cCst|j|j|jd�S)N�r�r�)rrr�r�rBr!r!r$r;�sz	words.getN)rlrl)r/r0r1r2r?r;r!r!r!r$r�s
rc@sJeZdZdZdd�Zdd�Zdd�Zdd	�Zddd�Zd
d�Z	dd�Z
d
S)�RegexLexerMetazw
    Metaclass for RegexLexer, creates the self._tokens attribute from
    self.tokens on the first instantiation.
    cCs t|t�r	|��}t�||�jS)zBPreprocess the regular expression component of a token definition.)rDrr;�re�compiler�)rv�regex�rflagsr�r!r!r$�_process_regex�s
zRegexLexerMeta._process_regexcCs&t|�tust|�sJd|����|S)z5Preprocess the token component of a token definition.z0token type must be simple type or callable, not )r*r	�callable)rv�tokenr!r!r$�_process_tokens�zRegexLexerMeta._process_tokencCs
t|t�r0|dkrdS||vr|fS|dkr|S|dd�dkr)t|dd��SJd|����t|t�rdd	|j}|jd
7_g}|D]}||ksRJd|����|�|�|||��qE|||<|fSt|t�r~|D]}||vs{|dvs{Jd|��qk|SJd
|����)z=Preprocess the state transition action of a token definition.�#pop����#pushN�z#pop:Fzunknown new state z_tmp_%dr�zcircular state ref )r�r�zunknown new state def )rDrK�intrt�_tmpname�extend�_process_stateru)rv�	new_state�unprocessed�	processedZ	tmp_state�itokensZistater!r!r$�_process_new_states<



�

��z!RegexLexerMeta._process_new_statec
Cs�t|t�sJd|����|ddksJd|����||vr!||Sg}||<|j}||D]�}t|t�rM||ks@Jd|����|�|�||t|���q.t|t�rSq.t|t�rm|�|j	||�}|�
t�d�j
d|f�q.t|�tuszJd|����z|�|d||�}Wnty�}	ztd	|d�d
|�d|�d|	���|	�d}	~	ww|�|d
�}
t|�dkr�d}n	|�|d||�}|�
||
|f�q.|S)z%Preprocess a single state definition.zwrong state name r�#zinvalid state name zcircular state reference rlNzwrong rule def zuncompilable regex z
 in state z of z: r��)rDrK�flagsrr�r�rrrr�r�rEr�r�r�r*rur��	Exception�
ValueErrorr�rO)rvr�r�r��tokensr�Ztdefr��rex�errr�r!r!r$r�)sD
�

&��
�zRegexLexerMeta._process_stateNcCs<i}|j|<|p
|j|}t|�D]	}|�|||�q|S)z-Preprocess a dictionary of token definitions.)�_all_tokensr�r�r�)rvr,�	tokendefsr�r�r!r!r$�process_tokendefTs
zRegexLexerMeta.process_tokendefc

Cs�i}i}|jD]_}|j�di�}|��D]Q\}}|�|�}|dur;|||<z|�t�}Wn	ty5Yqw|||<q|�|d�}|durFq||||d�<z|�t�}	Wn	ty^Yqw||	||<qq|S)a
        Merge tokens from superclasses in MRO order, returning a single tokendef
        dictionary.

        Any state that is not defined by a subclass will be inherited
        automatically.  States that *are* defined by subclasses will, by
        default, override that state in the superclass.  If a subclass wishes to
        inherit definitions from a superclass, it can use the special value
        "inherit", which will cause the superclass' state definition to be
        included at that point in the state.
        r�Nr�)�__mro__�__dict__r;�items�indexrr�r�)
rvr��inheritable�cZtoksr�r�ZcuritemsZinherit_ndxZnew_inh_ndxr!r!r$�
get_tokendefs\s6

���zRegexLexerMeta.get_tokendefscOsRd|jvri|_d|_t|d�r|jrn	|�d|���|_tj	|g|�Ri|��S)z:Instantiate cls after preprocessing its token definitions.�_tokensr�token_variantsrl)
r�r�r��hasattrr�r�r�r�r*�__call__)rvrw�kwdsr!r!r$r��s
zRegexLexerMeta.__call__rY)r/r0r1r2r�r�r�r�r�r�r�r!r!r!r$r��s#
+1r�c@s$eZdZdZejZiZddd�ZdS)rz�
    Base for simple stateful regular expression-based lexers.
    Simplifies the lexing process so that you need only
    provide a list of states and regular expressions.
    �r�ccs��d}|j}t|�}||d}	|D]�\}}}	|||�}
|
r�|dur:t|�tur2|||
��fVn|||
�EdH|
��}|	dur�t|	t�rm|	D]"}|dkrZt|�dkrY|�	�qI|dkrf|�
|d�qI|�
|�qIn,t|	t�r�t|	�t|�kr�|dd�=n||	d�=n|	dkr�|�
|d�nJd|	����||d}n3qz'||d	kr�d
g}|d
}|t
d	fV|d7}Wq|t||fV|d7}Wn
ty�YdSwq)z~
        Split ``text`` into (tokentype, text) pairs.

        ``stack`` is the initial stack (default: ``['root']``)
        rr�r�Nr�r�F�wrong state def: rIr�)r�r�r*r	r�r~rDrurOr�rEr��absrrr)r=rFr�r�r�Z
statestack�statetokens�rexmatchr�r��mr�r!r!r$rZ�s`�

��
�#��z!RegexLexer.get_tokens_unprocessedN�r�)	r/r0r1r2r��	MULTILINEr�r�rZr!r!r!r$r�s
rc@s"eZdZdZddd�Zdd�ZdS)rz9
    A helper object that holds lexer position data.
    NcCs*||_||_|pt|�|_|pdg|_dS)Nr�)rFr�rOr~r�)r=rFr�r�r~r!r!r$r?szLexerContext.__init__cCsd|j�d|j�d|j�d�S)Nz
LexerContext(z, �))rFr�r�rBr!r!r$rCszLexerContext.__repr__�NN)r/r0r1r2r?rCr!r!r!r$r�s
rc@seZdZdZddd�ZdS)rzE
    A RegexLexer that uses a context object to store its state.
    Nccs �|j}|st|d�}|d}n|}||jd}|j}	|D]�\}}}|||j|j�}	|	r�|durYt|�turG|j||	��fV|	��|_n|||	|�EdH|sY||jd}|dur�t	|t
�r�|D]'}
|
dkrwt|j�dkrv|j��qd|
dkr�|j�
|jd�qd|j�
|
�qdn1t	|t�r�t|�t|j�kr�|jdd�=n|j|d�=n|dkr�|j�
|jd�nJd	|����||jd}nHqz;|j|jkr�WdS||jd
kr�dg|_|d}|jtd
fV|jd7_Wq|jt||jfV|jd7_Wnt�yYdSwq)z
        Split ``text`` into (tokentype, text) pairs.
        If ``context`` is given, use this lexer context instead.
        rr�r�r�Nr�r�Fr�rI)r�rr�rFr�r~r*r	r�rDrurOr�rEr�r�rrr)r=rF�contextr�r�r�r�r�r�r�r�r!r!r$rZsn�



��
�#��z)ExtendedRegexLexer.get_tokens_unprocessedr�)r/r0r1r2rZr!r!r!r$rsrc	cs��t|�}zt|�\}}Wnty|EdHYdSwd}d}|D]{\}}}|dur.|}d}	|r�|t|�|kr�||	||�}
|
rP|||
fV|t|
�7}|D]\}}}
|||
fV|t|
�7}qR||}	zt|�\}}Wnty{d}Ynw|r�|t|�|ks:|	t|�kr�||||	d�fV|t|�|	7}q#|r�|p�d}|D]\}}}|||fV|t|�7}q�zt|�\}}Wnty�d}YdSw|s�dSdS)ag
    Helper for lexers which must combine the results of several
    sublexers.

    ``insertions`` is a list of ``(index, itokens)`` pairs.
    Each ``itokens`` iterable should be inserted at position
    ``index`` into the token stream given by the ``tokens``
    argument.

    The result is a combined token stream.

    TODO: clean up the code here.
    NTrF)�iter�next�
StopIterationrO)rnr�r�r��realposZinsleftror[r\ZoldiZtmpvalZit_indexZit_tokenZit_value�pr!r!r$rmSs\�
������rmc@r')�ProfilingRegexLexerMetaz>Metaclass for ProfilingRegexLexer, collects regex timing info.csLt|t�rt|j|j|jd��n|�t��|��tjf����fdd�	}|S)Nr�cs`�jd���fddg�}t��}��|||�}t��}|dd7<|d||7<|S)Nr�rr r�)�
_prof_data�
setdefault�timer�)rFr��endpos�info�t0�res�t1�rvZcompiledr�r�r!r$�
match_func�sz:ProfilingRegexLexerMeta._process_regex.<locals>.match_func)	rDrrr�r�r�r��sys�maxsize)rvr�r�r�r�r!r�r$r��s

�z&ProfilingRegexLexerMeta._process_regexN)r/r0r1r2r�r!r!r!r$r��sr�c@s"eZdZdZgZdZddd�ZdS)�ProfilingRegexLexerzFDrop-in replacement for RegexLexer that does profiling of its regexes.�r�c#s���jj�i�t��||�EdH�jj��}tdd�|��D��fdd�dd�}tdd�|D��}t	�t	d�jj
t|�|f�t	d	�t	d
d�t	d�|D]}t	d
|�qSt	d	�dS)NcssP�|]#\\}}\}}|t|��d��dd�dd�|d|d||fVqdS)zu'z\\�\N�Ai�)�reprrQrH)�.0r��r�nr[r!r!r$�	<genexpr>�s���z=ProfilingRegexLexer.get_tokens_unprocessed.<locals>.<genexpr>cs
|�jSrY)�_prof_sort_indexr"rBr!r$r%�s
z<ProfilingRegexLexer.get_tokens_unprocessed.<locals>.<lambda>T)�key�reversecss�|]}|dVqdS)�Nr!)r�r#r!r!r$r��s�z2Profiling result for %s lexing %d chars in %.3f mszn==============================================================================================================z$%-20s %-64s ncalls  tottime  percall)r�r�zn--------------------------------------------------------------------------------------------------------------z%-20s %-65s %5d %8.4f %8.4f)rAr�rErrZr��sortedr��sum�printr/rO)r=rFr�Zrawdatar�Z	sum_totalr.r!rBr$rZ�s*��
��z*ProfilingRegexLexer.get_tokens_unprocessedNr�)r/r0r1r2r�r�rZr!r!r!r$r��s
r�)6r2r�r�r�Zpygments.filterrrZpygments.filtersrZpygments.tokenrrrrr	Z
pygments.utilr
rrr
rrZpygments.regexoptr�__all__r�rrM�staticmethodZ_default_analyser*r(rrrKrrrrrurtrxrr�rrrrr�rrrrmr�r�r!r!r!r$�<module>sH
 
q'
4(aH@