Sindbad~EG File Manager

Current Path : /home/infinitibizsol/mypythonenv/lib/python3.6/site-packages/bs4/__pycache__/
Upload File :
Current File : /home/infinitibizsol/mypythonenv/lib/python3.6/site-packages/bs4/__pycache__/dammit.cpython-36.pyc

3

/ĜgƠ�@s^dZdZddlmZddlmZddlZddlZddlZddl	Z	dZ
yddlZ
Wn\ek
r�yddl
Z
Wn:ek
r�yddlZ
Wnek
r�dZ
YnXYnXYnXe
r�dd�Zndd�Zd	Zd
Ze�Zejejd�ej�ejejd�ej�d�ee<ejeej�ejeej�d�ee<dd
lmZGdd�de�ZGdd�d�ZGdd�d�ZdS)aBBeautiful Soup bonus library: Unicode, Dammit

This library converts a bytestream to Unicode through any means
necessary. It is heavily based on code from Mark Pilgrim's Universal
Feed Parser. It works best on XML and HTML, but it does not rewrite the
XML or HTML to reflect a new encoding; that's the tree builder's job.
�MIT�)�codepoint2name)�defaultdictNcCst|t�rdStj|�dS)N�encoding)�
isinstance�str�chardet_module�detect)�s�r�4/tmp/pip-build-fsllo_ck/beautifulsoup4/bs4/dammit.py�chardet_dammit+s
r
cCsdS)Nr)r
rrrr
0sz$^\s*<\?.*encoding=['"](.*?)['"].*\?>z0<\s*meta[^>]+charset\s*=\s*["']?([^>]*?)[ /;'">]�ascii)�html�xml)�html5c@s�eZdZdZdd�Ze�\ZZZdddddd	�Ze	j
d
�Ze	j
d�Ze
dd
��Ze
dd��Ze
dd��Ze
ddd��Ze
ddd��Ze
dd��ZdS)�EntitySubstitutionzFThe ability to substitute XML or HTML entities for certain characters.cCs�i}i}t�}tt�}x�ttj��D]�\}}|jd�rD|dd�}n|}||krX|||<|||<t|�dkr�t|�dkr�|dkr�q$t|�dkr�tdd�|D��r�q$t|�dkr�|j	|�q$||dj	|�q$Wt�}xJ|D]B}||}	|	s�|j	|�q�d	j
d
d�|	D��}
|j	d||
f�q�Wx0t|j��D] }x|D]}|j	|��q:W�q0Wd
dj
|�}
x*tt
j��D]\}}t|�}|||<�qpW||tj|
�fS)u�Initialize variables used by this class to manage the plethora of
        HTML5 named entities.

        This function returns a 3-tuple containing two dictionaries
        and a regular expression:

        unicode_to_name - A mapping of Unicode strings like "⦨" to
        entity names like "angmsdaa". When a single Unicode string has
        multiple entity names, we try to choose the most commonly-used
        name.

        name_to_unicode: A mapping of entity names like "angmsdaa" to 
        Unicode strings like "⦨".

        named_entity_re: A regular expression matching (almost) any
        Unicode string that corresponds to an HTML5 named entity.
        �;N��z<>&css|]}t|�dkVqdS)rN)�ord)�.0�xrrr�	<genexpr>�sz?EntitySubstitution._populate_class_variables.<locals>.<genexpr>r�cSsg|]}|d�qS)rr)rrrrr�
<listcomp>�sz@EntitySubstitution._populate_class_variables.<locals>.<listcomp>z
%s(?![%s])z(%s)�|���)�setr�sortedr�items�endswith�lenr�all�add�join�list�valuesr�chr�re�compile)Zunicode_to_nameZname_to_unicodeZshort_entitiesZ long_entities_by_first_characterZname_with_semicolon�	character�nameZ	particles�shortZ
long_versions�ignoreZ
long_entitiesZlong_entityZ
re_definition�	codepointrrr�_populate_class_variablesFsD



z,EntitySubstitution._populate_class_variablesZapos�quot�amp�lt�gt)�'�"�&�<�>z&([<>]|&(?!#\d+;|#x[0-9a-fA-F]+;|\w+;))z([<>&])cCs|jj|jd��}d|S)zpUsed with a regular expression to substitute the
        appropriate HTML entity for a special character string.rz&%s;)�CHARACTER_TO_HTML_ENTITY�get�group)�cls�matchobj�entityrrr�_substitute_html_entity�sz*EntitySubstitution._substitute_html_entitycCs|j|jd�}d|S)zoUsed with a regular expression to substitute the
        appropriate XML entity for a special character string.rz&%s;)�CHARACTER_TO_XML_ENTITYr<)r=r>r?rrr�_substitute_xml_entity�sz)EntitySubstitution._substitute_xml_entitycCs6d}d|kr*d|kr&d}|jd|�}nd}|||S)a*Make a value into a quoted XML attribute, possibly escaping it.

         Most strings will be quoted using double quotes.

          Bob's Bar -> "Bob's Bar"

         If a string contains double quotes, it will be quoted using
         single quotes.

          Welcome to "my bar" -> 'Welcome to "my bar"'

         If a string contains both single and double quotes, the
         double quotes will be escaped, and the string will be quoted
         using double quotes.

          Welcome to "Bob's Bar" -> "Welcome to &quot;Bob's bar&quot;
        r6r5z&quot;)�replace)�self�valueZ
quote_withZreplace_withrrr�quoted_attribute_value�sz)EntitySubstitution.quoted_attribute_valueFcCs"|jj|j|�}|r|j|�}|S)aSubstitute XML entities for special XML characters.

        :param value: A string to be substituted. The less-than sign
          will become &lt;, the greater-than sign will become &gt;,
          and any ampersands will become &amp;. If you want ampersands
          that appear to be part of an entity definition to be left
          alone, use substitute_xml_containing_entities() instead.

        :param make_quoted_attribute: If True, then the string will be
         quoted, as befits an attribute value.
        )�AMPERSAND_OR_BRACKET�subrBrF)r=rE�make_quoted_attributerrr�substitute_xmls


z!EntitySubstitution.substitute_xmlcCs"|jj|j|�}|r|j|�}|S)a�Substitute XML entities for special XML characters.

        :param value: A string to be substituted. The less-than sign will
          become &lt;, the greater-than sign will become &gt;, and any
          ampersands that are not part of an entity defition will
          become &amp;.

        :param make_quoted_attribute: If True, then the string will be
         quoted, as befits an attribute value.
        )�BARE_AMPERSAND_OR_BRACKETrHrBrF)r=rErIrrr�"substitute_xml_containing_entitiess


z5EntitySubstitution.substitute_xml_containing_entitiescCs|jj|j|�S)aReplace certain Unicode characters with named HTML entities.

        This differs from data.encode(encoding, 'xmlcharrefreplace')
        in that the goal is to make the result more readable (to those
        with ASCII displays) rather than to recover from
        errors. There's absolutely nothing wrong with a UTF-8 string
        containg a LATIN SMALL LETTER E WITH ACUTE, but replacing that
        character with "&eacute;" will make it more readable to some
        people.

        :param s: A Unicode string.
        )�CHARACTER_TO_HTML_ENTITY_RErHr@)r=r
rrr�substitute_html+sz"EntitySubstitution.substitute_htmlN)F)F)�__name__�
__module__�__qualname__�__doc__r0r:ZHTML_ENTITY_TO_CHARACTERrMrAr)r*rKrG�classmethodr@rBrFrJrLrNrrrrrCs$w

%rc@sHeZdZdZddd�Zdd�Zedd	��Zed
d��Z	eddd
��Z
dS)�EncodingDetectoraLSuggests a number of possible encodings for a bytestring.

    Order of precedence:

    1. Encodings you specifically tell EncodingDetector to try first
    (the known_definite_encodings argument to the constructor).

    2. An encoding determined by sniffing the document's byte-order mark.

    3. Encodings you specifically tell EncodingDetector to try if
    byte-order mark sniffing fails (the user_encodings argument to the
    constructor).

    4. An encoding declared within the bytestring itself, either in an
    XML declaration (if the bytestring is to be interpreted as an XML
    document), or in a <meta> tag (if the bytestring is to be
    interpreted as an HTML document.)

    5. An encoding detected through textual analysis by chardet,
    cchardet, or a similar external library.

    4. UTF-8.

    5. Windows-1252.

    NFcCsnt|pg�|_|r |j|7_|p&g|_|p0g}tdd�|D��|_d|_||_d|_|j|�\|_	|_
dS)a�Constructor.

        :param markup: Some markup in an unknown encoding.

        :param known_definite_encodings: When determining the encoding
            of `markup`, these encodings will be tried first, in
            order. In HTML terms, this corresponds to the "known
            definite encoding" step defined here:
            https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding

        :param user_encodings: These encodings will be tried after the
            `known_definite_encodings` have been tried and failed, and
            after an attempt to sniff the encoding by looking at a
            byte order mark has failed. In HTML terms, this
            corresponds to the step "user has explicitly instructed
            the user agent to override the document's character
            encoding", defined here:
            https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding

        :param override_encodings: A deprecated alias for
            known_definite_encodings. Any encodings here will be tried
            immediately after the encodings in
            known_definite_encodings.

        :param is_html: If True, this markup is considered to be
            HTML. Otherwise it's assumed to be XML.

        :param exclude_encodings: These encodings will not be tried,
            even if they otherwise would be.

        cSsg|]}|j��qSr)�lower)rrrrrrsz-EncodingDetector.__init__.<locals>.<listcomp>N)r&�known_definite_encodings�user_encodingsr�exclude_encodings�chardet_encoding�is_html�declared_encoding�strip_byte_order_mark�markup�sniffed_encoding)rDr]rVrZrXrW�override_encodingsrrr�__init__Xs"
zEncodingDetector.__init__cCs8|dk	r4|j�}||jkrdS||kr4|j|�dSdS)z�Should we even bother to try this encoding?

        :param encoding: Name of an encoding.
        :param tried: Encodings that have already been tried. This will be modified
            as a side effect.
        NFT)rUrXr$)rDr�triedrrr�_usable�s

zEncodingDetector._usableccs�t�}x |jD]}|j||�r|VqW|j|j|�r>|jVx |jD]}|j||�rF|VqFW|jdkr||j|j|j�|_|j|j|�r�|jV|j	dkr�t
|j�|_	|j|j	|�r�|j	VxdD]}|j||�r�|Vq�WdS)zmYield a number of encodings that might work for this markup.

        :yield: A sequence of strings.
        N�utf-8�windows-1252)rcrd)rrVrbr^rWr[�find_declared_encodingr]rZrYr
)rDra�errr�	encodings�s*




zEncodingDetector.encodingscCs�d}t|t�r||fSt|�dkrT|dd�dkrT|dd�dkrTd}|dd�}n�t|�dkr�|dd�dkr�|dd�dkr�d}|dd�}nd|dd	�d
kr�d}|d	d�}nB|dd�dkr�d
}|dd�}n |dd�dkr�d}|dd�}||fS)z�If a byte-order mark is present, strip it and return the encoding it implies.

        :param data: Some markup.
        :return: A 2-tuple (modified data, implied encoding)
        N��s��zzutf-16bes��zutf-16le�szutf-8s��zutf-32bes��zutf-32le)rrr")r=�datarrrrr\�s*
z&EncodingDetector.strip_byte_order_markcCs�|rt|�}}nd}tdtt|�d��}t|t�r@tt}ntt}|d}|d}d}	|j||d�}
|
r�|r�|j||d�}
|
dk	r�|
j�d}	|	r�t|	t�r�|	j	d	d
�}	|	j
�SdS)a�Given a document, tries to find its declared encoding.

        An XML encoding is declared at the beginning of the document.

        An HTML encoding is declared in a <meta> tag, hopefully near the
        beginning of the document.

        :param markup: Some markup.
        :param is_html: If True, this markup is considered to be HTML. Otherwise
            it's assumed to be XML.
        :param search_entire_document: Since an encoding is supposed to declared near the beginning
            of the document, most of the time it's only necessary to search a few kilobytes of data.
            Set this to True to force this method to search the entire document.
        iig�������?rrN)�endposrrrC)r"�max�intr�bytes�encoding_resr�search�groups�decoderU)r=r]rZZsearch_entire_documentZ
xml_endposZhtml_endpos�res�xml_reZhtml_rer[Zdeclared_encoding_matchrrrre�s(



z'EncodingDetector.find_declared_encoding)NFNNN)FF)rOrPrQrRr`rb�propertyrgrSr\rerrrrrT=s
-,rTc�@sVeZdZdZddd�ZdddgZgdd	gddfd
d�Zdd
�Z�ddd�Z�ddd�Z	e
dd��Zdd�Zdd�Z
�dd�d �d!�d"�d#�d$�d%�d&�d'�d(�d)�d*d2�d+d2d2�d,�d-�d.�d/�d0�d1�d2�d3�d4�d5�d6�d7d2�d8�d9dQ� ZdRddSdTdUdVdWdXdYdZd[d\d]d2d^d2d2d_d_d`d`dadbdcdddedfdgdhd2didjddkdldmdndodpd[dqdPdrdsdkddtdbdudvdwdx�d:dzd{dadSd|drd}d~dd�d2d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�dad�d�d�d�d�djd�d�d�d�d�d�d�d�d�dld�d�d�d�d�d�d�d�dud�dududududud�dudzdzdzdzd�d�d�d���Zd�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d�d��d�d�d�d�d�d�d�d�d�d	�d
�d�d�d
�d�zZ�d;�d<�d=gZe�d�dZe�d>�dZe�d?�d�d��ZdS(@�
UnicodeDammitz�A class for detecting the encoding of a *ML document and
    converting it to a Unicode string. If the source encoding is
    windows-1252, can replace MS smart quotes with their HTML or XML
    equivalents.z	mac-romanz	shift-jis)�	macintoshzx-sjis�windows-1252z
iso-8859-1z
iso-8859-2NFc
Cs�||_g|_d|_||_tjt�|_t||||||�|_	t
|t�sJ|dkrd||_t|�|_
d|_dS|j	j|_d}x,|j	jD] }	|j	j}|j|	�}|dk	r|Pq|W|s�x@|j	jD]4}	|	dkr�|j|	d�}|dk	r�|jjd�d|_Pq�W||_
|s�d|_dS)a2Constructor.

        :param markup: A bytestring representing markup in an unknown encoding.

        :param known_definite_encodings: When determining the encoding
            of `markup`, these encodings will be tried first, in
            order. In HTML terms, this corresponds to the "known
            definite encoding" step defined here:
            https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding

        :param user_encodings: These encodings will be tried after the
            `known_definite_encodings` have been tried and failed, and
            after an attempt to sniff the encoding by looking at a
            byte order mark has failed. In HTML terms, this
            corresponds to the step "user has explicitly instructed
            the user agent to override the document's character
            encoding", defined here:
            https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding

        :param override_encodings: A deprecated alias for
            known_definite_encodings. Any encodings here will be tried
            immediately after the encodings in
            known_definite_encodings.

        :param smart_quotes_to: By default, Microsoft smart quotes will, like all other characters, be converted
           to Unicode characters. Setting this to 'ascii' will convert them to ASCII quotes instead.
           Setting it to 'xml' will convert them to XML entity references, and setting it to 'html'
           will convert them to HTML entity references.
        :param is_html: If True, this markup is considered to be HTML. Otherwise
            it's assumed to be XML.
        :param exclude_encodings: These encodings will not be considered, even
            if the sniffing code thinks they might make sense.

        FrNrrCzSSome characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.T)�smart_quotes_to�tried_encodingsZcontains_replacement_charactersrZ�logging�	getLoggerrO�logrT�detectorrrr]Zunicode_markup�original_encodingrg�
_convert_from�warning)
rDr]rVrzrZrXrWr_�urrrrr`s@&



zUnicodeDammit.__init__cCs�|jd�}|jdkr&|jj|�j�}nf|jj|�}t|�tkr�|jdkrfdj�|dj�dj�}q�dj�|dj�dj�}n|j�}|S)z[Changes a MS smart quote character to an XML or HTML
        entity, or an ASCII character.rrrz&#xrr7r)r<rz�MS_CHARS_TO_ASCIIr;�encode�MS_CHARS�type�tuple)rD�match�origrHrrr�_sub_ms_charus


zUnicodeDammit._sub_ms_char�strictcCs�|j|�}|s||f|jkr"dS|jj||f�|j}|jdk	rh||jkrhd}tj|�}|j|j	|�}y|j
|||�}||_||_Wn tk
r�}zdSd}~XnX|jS)z|Attempt to convert the markup to the proposed encoding.

        :param proposed: The name of a character encoding.
        Ns([�-�]))
�
find_codecr{�appendr]rz�ENCODINGS_WITH_SMART_QUOTESr)r*rHr��_to_unicoder��	Exception)rDZproposed�errorsr]Zsmart_quotes_reZsmart_quotes_compiledr�rfrrrr��s"




zUnicodeDammit._convert_fromcCst|||�S)z}Given a string and its encoding, decodes the string into Unicode.

        :param encoding: The name of an encoding.
        )r)rDrkrr�rrrr��szUnicodeDammit._to_unicodecCs|js
dS|jjS)zhIf the markup is an HTML document, returns the encoding declared _within_
        the document.
        N)rZrr[)rDrrr�declared_html_encoding�sz$UnicodeDammit.declared_html_encodingcCs`|j|jj||��pN|r*|j|jdd��pN|r@|j|jdd��pN|rL|j�pN|}|r\|j�SdS)z�Convert the name of a character set to a codec name.

        :param charset: The name of a character set.
        :return: The name of a codec.
        �-r�_N)�_codec�CHARSET_ALIASESr;rCrU)rD�charsetrErrrr��szUnicodeDammit.find_codeccCs<|s|Sd}ytj|�|}Wnttfk
r6YnX|S)N)�codecs�lookup�LookupError�
ValueError)rDr��codecrrrr��s
zUnicodeDammit._codec�euro�20AC� �sbquo�201A�fnof�192�bdquo�201E�hellip�2026�dagger�2020�Dagger�2021�circ�2C6�permil�2030�Scaron�160�lsaquo�2039�OElig�152�?�#x17D�17D�lsquo�2018�rsquo�2019�ldquo�201C�rdquo�201D�bull�2022�ndash�2013�mdash�2014�tilde�2DC�trade�2122�scaron�161�rsaquo�203A�oelig�153�#x17E�17E�Yumlr) ����������������������������������������������������������������ZEUR�,�fz,,z...�+z++�^�%�Sr8ZOE�Zr5r6�*r�z--�~z(TM)r
r9Zoe�z�Y�!�cZGBP�$ZYENrz..z(th)z<<z(R)�oz+-�2�3�acuter��P�1z>>z1/4z1/2z3/4�AZAE�C�E�I�D�N�O�U�b�B�aZaerf�i�n�/�y)�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������s€s‚sƒs„s…s†s‡sˆs‰sŠs‹sŒsŽs‘s’s“s”s•s–s—s˜s™sšs›sœsžsŸs s¡s¢s£s¤s¥s¦s§s¨s©sªs«s¬s­s®s¯s°s±s²s³s´sµs¶s·s¸s¹sºs»s¼s½s¾s¿sÀsÁsÂsÃsÄsÅsÆsÇsÈsÉsÊsËsÌsÍsÎsÏsÐsÑsÒsÓsÔsÕsÖs×sØsÙsÚsÛsÜsÝsÞsßsàrsâsãsäsåsæsçsèsésêsësìsísîsïsðsñsòsósôsõsös÷søsùsúsûsüsýsþ)zr����������������������������������������������������������������������������������������������������������������������������������������������������������������������������r�r�rir�r�rjr�r�rhrr�utf8cCs$|jdd�j�dkrtd��|j�dkr0td��g}d	}d	}x�|t|�k�r�||}t|t�sft|�}||jkr�||jkr�xz|j	D]$\}}	}
||kr�||	kr�||
7}Pq�Wq>|d
kr�||j
kr�|j|||��|j|j
|�|d7}|}q>|d7}q>W|d	k�r|S|j||d��d
j|�S)aFix characters from one encoding embedded in some other encoding.

        Currently the only situation supported is Windows-1252 (or its
        subset ISO-8859-1), embedded in UTF-8.

        :param in_bytes: A bytestring that you suspect contains
            characters from multiple encodings. Note that this _must_
            be a bytestring. If you've already converted the document
            to Unicode, you're too late.
        :param main_encoding: The primary encoding of `in_bytes`.
        :param embedded_encoding: The encoding that was used to embed characters
            in the main document.
        :return: A bytestring in which `embedded_encoding`
          characters have been converted to their `main_encoding`
          equivalents.
        r�r��windows-1252�windows_1252zPWindows-1252 and ISO-8859-1 are the only currently supported embedded encodings.r��utf-8z4UTF-8 is the only currently supported main encoding.rrrN�)r�r�)r�r�)
rCrU�NotImplementedErrorr"rrnr�FIRST_MULTIBYTE_MARKER�LAST_MULTIBYTE_MARKER�MULTIBYTE_MARKERS_AND_SIZES�WINDOWS_1252_TO_UTF8r�r%)r=Zin_bytesZ
main_encodingZembedded_encodingZbyte_chunksZchunk_start�posZbyte�start�end�sizerrr�	detwingles<



zUnicodeDammit.detwingle)r�)r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r�)r�r)r5r)r�r�ri)r�r�rj)r�r�rhr)r�ry)rOrPrQrRr�r�r`r�r�r�rvr�r�r�r�r�r�r�r�r�rSr�rrrrrw
sbV	

	rw)rR�__license__�
html.entitiesr�collectionsrr�r)r|�stringrZcchardet�ImportError�chardetZcharset_normalizerr
Zxml_encodingZ	html_meta�dictrpr*r�rrorr�objectrrTrwrrrr�<module>sD
{N

Sindbad File Manager Version 1.0, Coded By Sindbad EG ~ The Terrorists