3 $f@sddZddlZddlZddlZddddddd d d d d ddddddddddgZddddddddd d!d"d#d$d%d&d'd(d)d*gZdddddd+dddd"d d!d,d#d$d%d-d'd(d&d.d/d0d)d*gZddd1d#ddd d!d$d%d2d3d"d&d4gZdd1d5d6d+ddd,d2d3g Zddddd d!d"dd$d%d2d3g Z ddd1ddd6ddd d!d,dd#g Z d7Z d8Z d9d:d;gZ dZd?Zd@ZdAdBZeefdCdDZeefdEdFZdGdHZGdIdJdJeZGdKdLdLeZGdMdNdNeZGdOdPdPeeZGdQdRdReeZddSlmZeddTZeddUZeddVZ dWe_dXej!_dYej"_dZe_d[ej#_d\ej$_d]ej%_d^ej&_d_ej"_d`e _ej#je j#_ej$je j$_ej%je j%_dae j'_ej&je j&_ej"je j"_eZ(GdbddeeZ)GdcddeeZ*Gdddde eZ+GdeddeeZ,GdfddeeZ-Gdgdde eZ.dhdiZ/e/[/ddkdZ0dldmZ1ddndoZ2dpdqZ3drdsZ4ddtdZ5dudZ6dvdZ7ddwdZ8dxdZ9dyZ:da;dzdZdd~dZ?ddd Z@ddd ZAdddZBeCdZDeEeDZFiZGGdddejHZIddd ZJddd ZKdddZLddddeKfdd ZMddZNddZOdaPddZQdaRddZSddZTddZUdaVddZWdddZXddZYddZZddZ[ddZ\dS)a3Parse (absolute and relative) URLs. urlparse module is based upon the following RFC specifications. RFC 3986 (STD66): "Uniform Resource Identifiers" by T. Berners-Lee, R. Fielding and L. Masinter, January 2005. RFC 2732 : "Format for Literal IPv6 Addresses in URL's by R.Hinden, B.Carpenter and L.Masinter, December 1999. RFC 2396: "Uniform Resource Identifiers (URI)": Generic Syntax by T. Berners-Lee, R. Fielding, and L. Masinter, August 1998. RFC 2368: "The mailto URL scheme", by P.Hoffman , L Masinter, J. Zawinski, July 1998. RFC 1808: "Relative Uniform Resource Locators", by R. Fielding, UC Irvine, June 1995. RFC 1738: "Uniform Resource Locators (URL)" by T. Berners-Lee, L. Masinter, M. McCahill, December 1994 RFC 3986 is considered the current standard and any future changes to urlparse module should conform with it. The urlparse module is currently not entirely compliant with this RFC due to defacto scenarios for parsing, and for backward compatibility purposes, some parsing quirks from older RFCs are retained. The testcases in test_urlparse.py provides a good indicator of parsing behavior. The WHATWG URL Parser spec should also be considered. We are not compliant with it either due to existing user code API behavior expectations (Hyrum's Law). It serves as a useful guide when making changes. Nurlparse urlunparseurljoin urldefragurlsplit urlunsplit urlencodeparse_qs parse_qslquote quote_plusquote_from_bytesunquote unquote_plusunquote_to_bytes DefragResult ParseResult SplitResultDefragResultBytesParseResultBytesSplitResultBytesZftphttpZgopherZnntpZimapZwaisfileZhttpsZshttpZmmsZprosperoZrtspZrtspuZsftpZsvnzsvn+sshZwsZwssZtelnetZsnewsZrsyncZnfsZgitzgit+sshZhdlZsipZsipsZtelZmailtoZnewszAabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+-.z!     cCstjtjdS)z,Clear the parse cache and the quoters cache.N) _parse_cacheclear _safe_quotersr!r!/usr/lib64/python3.6/parse.py clear_cache^sr#asciistrictcCs|S)Nr!)objr!r!r"_noopmsr'cCs |j||S)N)encode)r&encodingerrorsr!r!r"_encode_resultpsr+cstfdd|DS)Nc3s"|]}|r|jndVqdS)rN)decode).0x)r)r*r!r" vsz_decode_args..)tuple)argsr)r*r!)r)r*r" _decode_argstsr2cGsZt|dt}x.|ddD]}|rt|t|krtdqW|rL|tfSt|tfS)Nrz$Cannot mix str and non-str arguments) isinstancestr TypeErrorr'r2r+)r1Z str_inputargr!r!r" _coerce_argsxs  r8c@seZdZdZfZdddZdS)_ResultMixinStrz>Standard approach to encoding parsed results from str to bytesr$r%cs|jfdd|DS)Nc3s|]}|jVqdS)N)r()r-r.)r)r*r!r"r/sz)_ResultMixinStr.encode..)_encoded_counterpart)selfr)r*r!)r)r*r"r(sz_ResultMixinStr.encodeN)r$r%)__name__ __module__ __qualname____doc__ __slots__r(r!r!r!r"r9sr9c@seZdZdZfZdddZdS)_ResultMixinBytesz>Standard approach to decoding parsed results from bytes to strr$r%cs|jfdd|DS)Nc3s|]}|jVqdS)N)r,)r-r.)r)r*r!r"r/sz+_ResultMixinBytes.decode..)_decoded_counterpart)r;r)r*r!)r)r*r"r,sz_ResultMixinBytes.decodeN)r$r%)r<r=r>r?r@r,r!r!r!r"rAsrAc@sDeZdZdZfZeddZeddZeddZedd Z d S) _NetlocResultMixinBasezHShared methods for the parsed result objects containing a netloc elementcCs |jdS)Nr) _userinfo)r;r!r!r"usernamesz_NetlocResultMixinBase.usernamecCs |jdS)Nr3)rD)r;r!r!r"passwordsz_NetlocResultMixinBase.passwordcCsD|jd}|sdSt|tr dnd}|j|\}}}|j||S)Nr%%) _hostinfor4r5 partitionlower)r;hostnameZ separatorZpercentZzoner!r!r"rLs  z_NetlocResultMixinBase.hostnamecCs@|jd}|dk rr?r@propertyrErFrLrPr!r!r!r"rCs    rCc@s(eZdZfZeddZeddZdS)_NetlocResultMixinStrcCsD|j}|jd\}}}|r4|jd\}}}|sr@rQrDrIr!r!r!r"rRs rRc@s(eZdZfZeddZeddZdS)_NetlocResultMixinBytescCsD|j}|jd\}}}|r4|jd\}}}|sr@rQrDrIr!r!r!r"r`s r`) namedtuplez url fragmentz!scheme netloc path query fragmentz(scheme netloc path params query fragmentz DefragResult(url, fragment) A 2-tuple that contains the url without fragment identifier and the fragment identifier as a separate argument. z$The URL with no fragment identifier.z Fragment identifier separated from URL, that allows indirect identification of a secondary resource by reference to a primary resource and additional identifying information. z SplitResult(scheme, netloc, path, query, fragment) A 5-tuple that contains the different components of a URL. Similar to ParseResult, but does not split params. z%Specifies URL scheme for the request.z0 Network location where the request is made to. z@ The hierarchical path, such as the path to a file to download. z The query component, that contains non-hierarchical data, that along with data in path component, identifies a resource in the scope of URI's scheme and network location. z Fragment identifier, that allows indirect identification of a secondary resource by reference to a primary resource and additional identifying information. zr ParseResult(scheme, netloc, path, params, query, fragment) A 6-tuple that contains components of a parsed URL. z Parameters for last path element used to dereference the URI in order to provide access to perform some operation on the resource. c@seZdZfZddZdS)rcCs |jr|jd|jS|jSdS)N#)fragmenturl)r;r!r!r"geturlCszDefragResult.geturlN)r<r=r>r@rir!r!r!r"rAsc@seZdZfZddZdS)rcCst|S)N)r)r;r!r!r"riKszSplitResult.geturlN)r<r=r>r@rir!r!r!r"rIsc@seZdZfZddZdS)rcCst|S)N)r)r;r!r!r"riPszParseResult.geturlN)r<r=r>r@rir!r!r!r"rNsc@seZdZfZddZdS)rcCs |jr|jd|jS|jSdS)N#)rgrh)r;r!r!r"riVszDefragResultBytes.geturlN)r<r=r>r@rir!r!r!r"rTsc@seZdZfZddZdS)rcCst|S)N)r)r;r!r!r"ri^szSplitResultBytes.geturlN)r<r=r>r@rir!r!r!r"r\sc@seZdZfZddZdS)rcCst|S)N)r)r;r!r!r"ricszParseResultBytes.geturlN)r<r=r>r@rir!r!r!r"rascCs8ttfttfttff}x|D]\}}||_||_qWdS)N)rrrrrrr:rB)Z _result_pairsZ_decodedZ_encodedr!r!r"_fix_result_transcodinggs  rkTc Csft||\}}}t|||}|\}}}}}|tkrHd|krHt|\}}nd}t||||||} || S)a#Parse a URL into 6 components: :///;?# Return a 6-tuple: (scheme, netloc, path, params, query, fragment). Note that we don't break the components up in smaller bits (e.g. netloc is a single string) and we don't expand % escapes.;r)r8r uses_params _splitparamsr) rhschemeallow_fragments_coerce_resultZ splitresultrUqueryrgparamsresultr!r!r"rts cCsRd|kr,|jd|jd}|dkr6|dfSn |jd}|d|||ddfS)N/rlrrr3)findrfind)rhir!r!r"rns   rncCsLt|}x*dD]"}|j||}|dkrt||}qW|||||dfS)Nz/?#r)lenrvmin)rhstartdelimcZwdelimr!r!r" _splitnetlocs   r~cCs| stdd|D rdSddl}|jdd}|jdd}|jdd}|jdd}|jd |}||krndSx(d D] }||krttd |d d qtWdS)Ncss|]}t|dkVqdS)N)ord)r-r}r!r!r"r/sz_checknetloc..rrSrrTrf?NFKCz/?#@:znetloc 'z' contains invalid z#characters under NFKC normalization)any unicodedatareplace normalizerO)rUrnZnetloc2r}r!r!r" _checknetlocs       rcCsxtD]}|j|d}qW|S)Nr)_UNSAFE_URL_BYTES_TO_REMOVEr)rhbr!r!r"_remove_unsafe_bytes_from_urls rc Cst||\}}}t|}t|}|jt}|jt}t|}|||t|t|f}tj|d}|rj||St tt kr|t d}}}|j d} | dkr|d| dkr~|d| j }|| dd}|dddkrt|d\}}d |kod |ksd |krd |krtd |r:///?# Return a 5-tuple: (scheme, netloc, path, query, fragment). Note that we don't break the components up in smaller bits (e.g. netloc is a single string) and we don't expand % escapes.NrrTrrr3z//r[r\zInvalid IPv6 URLrfrcss|]}|dkVqdS) 0123456789Nr!)r-r}r!r!r"r/szurlsplit..)r8rlstrip_WHATWG_C0_CONTROL_OR_SPACEstripbooltypergetryMAX_CACHE_SIZEr#rvrKr~rOsplitrr scheme_charsr) rhrorprqkeycachedrUrrrgrxvr}restr!r!r"rsd          cCs<t|\}}}}}}}|r&d||f}|t|||||fS)zPut a parsed URL back together again. This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had redundant delimiters, e.g. a ? with an empty query (the draft states that these are equivalent).z%s;%s)r8r) componentsrorUrhrsrrrgrqr!r!r"rs cCst|\}}}}}}|s4|r`|tkr`|dddkr`|rP|dddkrPd|}d|pXd|}|rp|d|}|r|d|}|r|d |}||S) akCombine the elements of a tuple as returned by urlsplit() into a complete URL as a string. The data argument can be any five-item iterable. This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had unnecessary delimiters (for example, a ? with an empty query; the RFC states that these are equivalent).Nrz//r3rurrTrrf)r8 uses_netloc)rrorUrhrrrgrqr!r!r"rs    c Cs|s|S|s|St||\}}}t|d|\}}}}}} t|||\} } } } }}| |ks`| tkrh||S| tkr| r|t| | | | ||fS|} | r| r|} |} |s|}|t| | | | ||fS|jd}|ddkr|d=| dddkr| jd}n(|| jd}td|dd |dd <g}xX|D]P}|dkrdy |jWntk r`YnXn|dkrtq0n |j |q0W|d d kr|j d|t| | dj |pd| ||fS) zaJoin a base URL and a possibly relative URL to form an absolute interpretation of the latter.rrur3N...rrrr)rr) r8r uses_relativerrrfilterpop IndexErrorappendjoin)baserhrprqZbschemeZbnetlocZbpathZbparamsZbqueryZ bfragmentrorUpathrsrrrgZ base_partsZsegmentsZ resolved_pathZsegr!r!r"r sT            c CsTt|\}}d|kr>t|\}}}}}}t|||||df}nd}|}|t||S)zRemoves any existing fragment from URL. Returns a tuple of the defragmented URL and the fragment. If the URL contained no fragments, the second element is the empty string. rfr)r8rrr) rhrqsrpaqZfragZdefragr!r!r"rOs Z0123456789ABCDEFabcdefc Cs|s|jdSt|tr"|jd}|jd}t|dkr<|S|dg}|j}tdkrbddtDaxb|ddD]R}y(|t|dd ||d dWqptk r|d||YqpXqpWdj |S) z,unquote_to_bytes('abc%20def') -> b'abc def'.zutf-8rHr3rNcSs4i|],}tD]"}tt||dg||jq qS))_hexdigbytesrNr()r-rrr!r!r" usz$unquote_to_bytes..r) rr4r5r(ryr _hextobyterKeyErrorr)stringbitsresritemr!r!r"rbs*     z([-]+)utf-8rcCsd|kr|j|S|dkrd}|dkr*d}tj|}|dg}|j}x@tdt|dD],}|t||j|||||dqVWdj|S) aReplace %xx escapes by their single-character equivalent. The optional encoding and errors parameters specify how to decode percent-encoded sequences into Unicode characters, as accepted by the bytes.decode() method. By default, percent-encoded sequences are decoded with UTF-8, and invalid sequences are replaced by a placeholder character. unquote('abc%20def') -> 'abc def'. rGNzutf-8rrr3rr)r_asciirerrangeryrr,r)rr)r*rrrrxr!r!r"rs   Fc CsPi}t||||||d}x2|D]*\}} ||kr>||j| q| g||<qW|S)aParse a query given as a string argument. Arguments: qs: percent-encoded query string to be parsed keep_blank_values: flag indicating whether blank values in percent-encoded queries should be treated as blank strings. A true value indicates that blanks should be retained as blank strings. The default false value indicates that blank values are to be ignored and treated as if they were not included. strict_parsing: flag indicating what to do with parsing errors. If false (the default), errors are silently ignored. If true, errors raise a ValueError exception. encoding and errors: specify how to decode percent-encoded sequences into Unicode characters, as accepted by the bytes.decode() method. max_num_fields: int. If set, then throws a ValueError if there are more than n fields read by parse_qsl(). Returns a dictionary. )r)r*max_num_fields)r r) qskeep_blank_valuesstrict_parsingr)r*rZ parsed_resultpairsnamevaluer!r!r"r scCst|\}}|dk rszparse_qsl..=rzbad query field: %rrr+ )r)r*)r8countrOrryrrr)rrrr)r*rrq num_fieldsrrZ name_valueZnvrrr!r!r"r s4      cCs|jdd}t|||S)zLike unquote(), but also replace plus signs by spaces, as required for unquoting HTML form values. unquote_plus('%7e/abc+def') -> '~/abc def' rr)rr)rr)r*r!r!r"rs sAABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_.-c@s(eZdZdZddZddZddZdS) QuoterzA mapping from bytes (in range(0,256)) to strings. String values are percent-encoded byte values, unless the key < 128, and in the "safe" set (either the specified safe set, or default set). cCstj||_dS)zsafe: bytes object.N) _ALWAYS_SAFEunionsafe)r;rr!r!r"__init__szQuoter.__init__cCsd|jjt|fS)Nz<%s %r>) __class__r<dict)r;r!r!r"__repr__szQuoter.__repr__cCs(||jkrt|ndj|}|||<|S)Nz%{:02X})rchrformat)r;rrr!r!r" __missing__szQuoter.__missing__N)r<r=r>r?rrrr!r!r!r"rsrrucCsbt|tr8|s|S|dkrd}|dkr*d}|j||}n |dk rHtd|dk rXtdt||S)aquote('abc def') -> 'abc%20def' Each part of a URL, e.g. the path info, the query, etc., has a different set of reserved characters that must be quoted. RFC 2396 Uniform Resource Identifiers (URI): Generic Syntax lists the following reserved characters. reserved = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" | "$" | "," Each of these characters is reserved in some component of a URL, but not necessarily in all of them. By default, the quote function is intended for quoting the path section of a URL. Thus, it will not encode '/'. This character is reserved, but in typical usage the quote function is being called on a path where the existing slash characters are used as reserved characters. string and safe may be either str or bytes objects. encoding and errors must not be specified if string is a bytes object. The optional encoding and errors parameters specify how to deal with non-ASCII characters, as accepted by the str.encode method. By default, encoding='utf-8' (characters are encoded with UTF-8), and errors='strict' (unsupported characters raise a UnicodeEncodeError). Nzutf-8r%z,quote() doesn't support 'encoding' for bytesz*quote() doesn't support 'errors' for bytes)r4r5r(r6r )rrr)r*r!r!r"r %s cCsdt|trd|ks$t|tr2d|kr2t||||St|trBd}nd}t|||||}|jddS)zLike quote(), but also replace ' ' with '+', as required for quoting HTML form values. Plus signs in the original string are escaped unless they are included in safe. It also does not have safe default to '/'. r r)r4r5rr r)rrr)r*Zspacer!r!r"r Qs c st|ttfstd|sdSt|tr6|jdd}ntdd|D}|jt|s^|jSy t |Wn&t k rt |j t |<YnXdj fdd|DS)zLike quote(), but accepts a bytes object rather than a str, and does not perform string-to-bytes encoding. It always returns an ASCII string. quote_from_bytes(b'abc def?') -> 'abc%20def%3f' z!quote_from_bytes() expected bytesrr$ignorecSsg|]}|dkr|qS)r!)r-r}r!r!r"rosz$quote_from_bytes..csg|] }|qSr!r!)r-char)quoterr!r"rvs)r4r bytearrayr6r5r(rstrip_ALWAYS_SAFE_BYTESr,r rr __getitem__r)Zbsrr!)rr"r bs  cCst|dr|j}nRy t|r2t|dt r2tWn0tk rdtj\}}}tdj|YnXg} |sxr|D]j\} } t| t r|| |} n|t | |||} t| t r|| |} n|t | |||} | j | d| qtWn,x(|D]\} } t| t r|| |} n|t | |||} t| t rL|| |} | j | d| qt| t rz|| |||} | j | d| qy t| } Wn:tk r|t | |||} | j | d| YqXxJ| D]B} t| t r|| |} n|t | |||} | j | d| qWqWdj | S)a^Encode a dict or sequence of two-element tuples into a URL query string. If any values in the query arg are sequences and doseq is true, each sequence element is converted to a separate parameter. If the query arg is a sequence of two-element tuples, the order of the parameters in the output will match the order of parameters in the input. The components of a query arg may each be either a string or a bytes type. The safe, encoding, and errors parameters are passed down to the function specified by quote_via (encoding and errors only if a component is a str). itemsrz1not a valid non-string sequence or mapping objectrr) hasattrrryr4r0r6sysexc_infowith_tracebackrr5rr)rrZdoseqrr)r*Z quote_viaZtyZvatblkrr.Zeltr!r!r"rxsP                c CsJt|trFy|jdj}Wn(tk rDtdt|dYnX|S)zto_bytes(u"URL") --> 'URL'.ASCIIzURL z contains non-ASCII characters)r4r5r(r, UnicodeErrorrepr)rhr!r!r"to_bytess  rcCs`t|j}|dddkr<|dddkr<|ddj}|dddkr\|ddj}|S) z8unwrap('') --> 'type://host/path'.Nr3<>zURL:rr)r5r)rhr!r!r"unwraps   rcCsDtdkrtjdtjatj|}|r<|j\}}|j|fSd|fS)z:splittype('type:opaquestring') --> 'type', 'opaquestring'.Nz ([^/:]+):(.*)) _typeprogrecompileDOTALLmatchgroupsrK)rhrrodatar!r!r" splittypes   rcCsXtdkrtjdtjatj|}|rP|j\}}|rH|ddkrHd|}||fSd|fS)z;splithost('//host[:port]/path') --> 'host[:port]', '/path'.Nz//([^/#?]*)(.*)rru) _hostprogrrrrr)rhrZ host_portrr!r!r" splithosts  rcCs |jd\}}}|r|nd|fS)zJsplituser('user[:passwd]@host[:port]') --> 'user[:passwd]', 'host[:port]'.rSN)rV)hostuserr|r!r!r" splitusersrcCs |jd\}}}||r|ndfS)z/splitpasswd('user:passwd') -> 'user', 'passwd'.rTN)rJ)rr|Zpasswdr!r!r" splitpasswdsrcCsDtdkrtjdtjatj|}|r<|j\}}|r<||fS|dfS)z*splitport('host:port') --> 'host', 'port'.Nz(.*):([0-9]*)$) _portprogrrrrr)rrrPr!r!r" splitports  rr3c CsT|jd\}}}|s|}n2|rLy t|}Wntk rBd}YnX||fS||fS)zSplit host and port, returning numeric port. Return given default port if no ':' found; defaults to -1. Return numerical port if a valid number are found after ':'. Return None if ':' but not a valid number.rTN)rVrNrO)rZdefportr|rPZnportr!r!r" splitnports  rcCs$|jd\}}}|r||fS|dfS)z/splitquery('/path?query') --> '/path', 'query'.rN)rV)rhrr|rrr!r!r" splitquery"srcCs$|jd\}}}|r||fS|dfS)z)splittag('/path#tag') --> '/path', 'tag'.rfN)rV)rhrr|tagr!r!r"splittag)srcCs|jd}|d|ddfS)zksplitattr('/path;attr1=value1;attr2=value2;...') -> '/path', ['attr1=value1', 'attr2=value2', ...].rlrr3N)r)rhZwordsr!r!r" splitattr0s rcCs |jd\}}}||r|ndfS)z-splitvalue('attr=value') --> 'attr', 'value'.rN)rJ)attrr|rr!r!r" splitvalue6sr)rT)r)rT)T)rr)FFrrN)FFrrN)rr)ruNN)rNN)rur)r)]r?rr collections__all__rrrmZnon_hierarchicalZ uses_queryZ uses_fragmentrrrrrr#Z_implicit_encodingZ_implicit_errorsr'r+r2r8objectr9rArCrRr`reZ_DefragResultBaseZ_SplitResultBaseZ_ParseResultBaserhrgrorUrrrrsZ ResultBaserrrrrrrkrrnr~rrrrrrrrrrrrrr r r frozensetrrrr defaultdictrr r r rrrrrrrrrrrrrrrrr!r!r!r" s                 !           @  E   & ;  ,   O