3
Reu @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl
Z
ddl Z ddl m
Z
ddlmZ ddlmZ ddlmZ ddlmZ dd lmZmZmZmZmZmZm Z m Z m Z m Z m Z m!Z!m"Z"m#Z# dd
l$m%Z% dd l&m'Z' dd l(m)Z)m*Z*m+Z+m,Z, dkZ-ej. Z/dddZ0ej1dkr:dd Z2dd Z dd Z3dd Z4dld d Z5d d Z6d d! Z7d"d# Z8d$d% Z9d&d' Z:d(d) Z;dmd*d+Zd0d1 Z?d2d3 Z@d4d5 ZAd6d7 ZBd8d9 ZCd:d; ZDeEdnZFd>d? ZGd@dA ZHdBdC ZIdDdE ZJdFdG ZKdHdI ZLejMdJdK ZNdLdM ZOdodNdOZPdPdQ ZQdpdSdTZRdUdV ZSdWdX ZTdYjUdZZVeVd[ ZWeVd\ ZXd]d^ ZYd_d` ZZdadb Z[ej\dcZ]ej\ddZ^dedf Z_dgdh Z`didj ZadS )qz
requests.utils
~~~~~~~~~~~~~~
This module provides utility functions that are used within Requests
that are also useful for external consumption.
N) OrderedDict ) __version__)certs)to_native_string)parse_http_list)quoteurlparsebytesstrunquote
getproxies proxy_bypass
urlunparse
basestring
integer_typesis_py3proxy_bypass_environmentgetproxies_environmentMapping)cookiejar_from_dict)CaseInsensitiveDict)
InvalidURL
InvalidHeaderFileModeWarningUnrewindableBodyError.netrc_netrcP i )httphttpswin32c C s yt rdd l}ndd l}W n tk
r. dS X y6|j|jd}t|j|dd }|j|dd }W n tk
rz dS X | s| rdS |j d}xX|D ]P}|dkrd| krd S |j
dd
}|j
d d }|j
d
d}t j || t j
rd S qW dS )Nr Fz;Software\Microsoft\Windows\CurrentVersion\Internet SettingsZ ProxyEnableZ
ProxyOverride;z.Tz\.*z.*?)r winreg_winreg ImportErrorOpenKeyHKEY_CURRENT_USERint QueryValueExOSErrorsplitreplacerematchI)hostr& ZinternetSettingsZ proxyEnableZ
proxyOverridetest r5 /builddir/build/BUILDROOT/alt-python36-pip-20.2.4-5.el8.x86_64/opt/alt/python36/lib/python3.6/site-packages/pip/_vendor/requests/utils.pyproxy_bypass_registry0 s8
r7 c C s t rt| S t| S dS )zReturn True, if the host should be bypassed.
Checks proxy settings gathered from the environment, if specified,
or the registry.
N)r r r7 )r3 r5 r5 r6 r W s r c C s t | dr| j } | S )z/Returns an internal sequence dictionary update.items)hasattrr8 )dr5 r5 r6 dict_to_sequencec s
r; c C s2 d }d}t | dr t| }nbt | dr.| j}nPt | dr~y | j }W n tjk
rZ Y n$X tj|j}d| jkr~t j
dt t | dry | j }W n$ t
tfk
r |d k r|}Y n\X t | do|d kry&| jdd | j }| j|pd W n t
tfk
r d}Y nX |d kr$d}td|| S )
Nr __len__lenfilenoba% Requests has determined the content-length for this request using the binary size of the file: however, the file has been opened in text mode (i.e. without the 'b' flag in the mode). This may lead to an incorrect content-length. In Requests 3.0, support will be removed for files in text mode.tellseek )r9 r= r> ioUnsupportedOperationosfstatst_sizemodewarningswarnr r@ r- IOErrorrA max)o total_lengthcurrent_positionr> r5 r5 r6 super_lenl s@
rP Fc C s yddl m }m} d}xJtD ]B}ytjjdj|}W n tk
rJ dS X tjj|r |}P q W |dkrndS t | }d}t
| t r|j d}|j
j|d } y6||j| }
|
r|
d rdnd} |
| |
d fS W n |tfk
r |r Y nX W n ttfk
r Y nX dS ) z;Returns the Requests tuple auth for a given url from netrc.r )netrcNetrcParseErrorNz~/{} :asciir rB )rQ rR NETRC_FILESrE path
expanduserformatKeyErrorexistsr
isinstancer decodenetlocr. authenticatorsrK r( AttributeError) url raise_errorsrQ rR
netrc_pathflocrisplitstrr3 r login_ir5 r5 r6 get_netrc_auth s8
rh c C sB t | dd}|r>t|tr>|d dkr>|d dkr>tjj|S dS )z0Tries to guess the filename of the given object.nameNr )getattrr[ r rE rV basename)objri r5 r5 r6 guess_filename s rp c C s t jj| r| S t jj| \}}x4|rTt jj| rTt jj|\}}dj||g}q"W tj|sd| S tj|}||j kr~| S t j
}t jj|f|jd }t jj|s|j ||d}|S )zReplace nonexistent paths that look like they refer to a member of a zip
archive with the location of an extracted copy of the target, or else
just return the provided path unchanged.
/)rV ) rE rV rZ r. joinzipfile
is_zipfileZipFilenamelisttempfile
gettempdirextract)rV archivememberprefixzip_filetmpextracted_pathr5 r5 r6 extract_zipped_paths s
r c C s. | dkr dS t | ttttfr&tdt| S )a Take an object and test to see if it can be represented as a
dictionary. Unless it can not be represented as such, return an
OrderedDict, e.g.,
::
>>> from_key_val_list([('key', 'val')])
OrderedDict([('key', 'val')])
>>> from_key_val_list('string')
Traceback (most recent call last):
...
ValueError: cannot encode objects that are not 2-tuples
>>> from_key_val_list({'key': 'val'})
OrderedDict([('key', 'val')])
:rtype: OrderedDict
Nz+cannot encode objects that are not 2-tuples)r[ r r
boolr+
ValueErrorr )valuer5 r5 r6 from_key_val_list s
r c C s@ | dkr dS t | ttttfr&tdt | tr8| j } t| S )a Take an object and test to see if it can be represented as a
dictionary. If it can be, return a list of tuples, e.g.,
::
>>> to_key_val_list([('key', 'val')])
[('key', 'val')]
>>> to_key_val_list({'key': 'val'})
[('key', 'val')]
>>> to_key_val_list('string')
Traceback (most recent call last):
...
ValueError: cannot encode objects that are not 2-tuples
:rtype: list
Nz+cannot encode objects that are not 2-tuples) r[ r r
r r+ r r r8 list)r r5 r5 r6 to_key_val_list s
r c C s\ g }xRt | D ]F}|dd |dd ko4dkn rJt|dd }|j| qW |S )a Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
It basically works like :func:`parse_set_header` just that items
may appear multiple times and case sensitivity is preserved.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
To create a header from the :class:`list` again, use the
:func:`dump_header` function.
:param value: a string with a list header.
:return: :class:`list`
:rtype: list
Nr "rl rl )_parse_list_headerunquote_header_valueappend)r resultitemr5 r5 r6 parse_list_header= s (r c C s| i }xrt | D ]f}d|kr$d||< q|jdd\}} | dd | dd koVdkn rlt| dd } | ||< qW |S )a^ Parse lists of key, value pairs as described by RFC 2068 Section 2 and
convert them into a python dict:
>>> d = parse_dict_header('foo="is a fish", bar="as well"')
>>> type(d) is dict
True
>>> sorted(d.items())
[('bar', 'as well'), ('foo', 'is a fish')]
If there is no value for a key it will be `None`:
>>> parse_dict_header('key_without_value')
{'key_without_value': None}
To create a header from the :class:`dict` again, use the
:func:`dump_header` function.
:param value: a string with a dict header.
:return: :class:`dict`
:rtype: dict
=Nr r rl rl )r r. r )r r r ri r5 r5 r6 parse_dict_header] s ( r c C s^ | rZ| d | d ko dkn rZ| dd
} | sF| dd dkrZ| j ddj ddS | S ) zUnquotes a header value. (Reversal of :func:`quote_header_value`).
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
:rtype: str
r r r NrB z\\\z\"rl rl )r/ )r is_filenamer5 r5 r6 r s
$ r c C s" i }x| D ]}|j ||j< q
W |S )zReturns a key/value dictionary from a CookieJar.
:param cj: CookieJar object to extract cookies from.
:rtype: dict
)r ri )cj cookie_dictcookier5 r5 r6 dict_from_cookiejar s
r c C s
t || S )zReturns a CookieJar from a key/value dictionary.
:param cj: CookieJar to insert cookies into.
:param cookie_dict: Dict of key/values to insert into CookieJar.
:rtype: CookieJar
)r )r r r5 r5 r6 add_dict_to_cookiejar s r c C sT t jdt tjdtjd}tjdtjd}tjd}|j| |j| |j| S )zlReturns encodings from given content string.
:param content: bytestring to extract encodings from.
zIn requests 3.0, get_encodings_from_content will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)z!])flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])rI rJ DeprecationWarningr0 compiler2 findall)content
charset_re pragma_rexml_rer5 r5 r6 get_encodings_from_content s
r c
C s | j d}|d j |dd }}i }d}xh|D ]`}|j }|r2|d }}|jd} | dkr|d| j|}|| d d j|}|||j < q2W ||fS ) zReturns content type and parameters from given header
:param header: string
:return: tuple containing content type and dictionary of
parameters
r" r r Nz"' Tr rl )r. stripfindlower)
headertokens content_typeparams params_dictitems_to_stripparamkeyr index_of_equalsr5 r5 r6 _parse_content_type_header s
r c C sD | j d}|sdS t|\}}d|kr4|d jdS d|kr@dS dS )z}Returns encodings from given HTTP Header Dict.
:param headers: dictionary to extract encoding from.
:rtype: str
z content-typeNcharsetz'"textz
ISO-8859-1)getr r )headersr r r5 r5 r6 get_encoding_from_headers s
r c c sr |j dkr"x| D ]
}|V qW dS tj|j dd}x | D ]}|j|}|r:|V q:W |jddd}|rn|V dS )zStream decodes a iterator.Nr/ )errors T)final)encodingcodecsgetincrementaldecoderr\ )iteratorrr decoderchunkrvr5 r5 r6 stream_decode_response_unicode s
r c c sL d}|dks|dkr t | }x*|t | k rF| ||| V ||7 }q W dS )z Iterate over slices of a string.r N)r= )string slice_lengthposr5 r5 r6 iter_slices s r c C sv t jdt g }t| j}|rJy t| j|S tk
rH |j| Y nX yt| j|ddS t k
rp | jS X dS )zReturns the requested content back in unicode.
:param r: Response object to get unicode content from.
Tried:
1. charset from content-type
2. fall back and replace all unicode characters
:rtype: str
zIn requests 3.0, get_unicode_from_response will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)r/ )r N)
rI rJ r r r r r UnicodeErrorr TypeError)r tried_encodingsr r5 r5 r6 get_unicode_from_response s
r Z4ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzz0123456789-._~c
C s | j d}xtdt|D ]}|| dd }t|dkr|j rytt|d}W n tk
rt td| Y nX |tkr||| dd ||< qd|| ||< qd|| ||< qW dj |S ) zUn-escape any percent-escape sequences in a URI that are unreserved
characters. This leaves all reserved, illegal and non-ASCII bytes encoded.
:rtype: str
%r r rB z%Invalid percent-escape sequence: '%s'N )
r. ranger= isalnumchrr+ r r UNRESERVED_SETrr )uripartsihcr5 r5 r6 unquote_unreserved: s
r c C s: d}d}yt t| |dS tk
r4 t | |dS X dS )zRe-quote the given URI.
This function passes the given URI through an unquote/quote cycle to
ensure that it is fully and consistently quoted.
:rtype: str
z!#$%&'()*+,/:;=?@[]~z!#$&'()*+,/:;=?@[]~)safeN)r r r )r safe_with_percentsafe_without_percentr5 r5 r6 requote_uriR s r c C sl t jdtj| d }|jd\}}t jdtjtt|d }t jdtj|d |@ }||@ ||@ kS )zThis function allows you to check if an IP belongs to a network subnet
Example: returns True if ip = 192.168.1.1 and net = 192.168.1.0/24
returns False if ip = 192.168.1.1 and net = 192.168.100.0/24
:rtype: bool
z=Lr rq )structunpacksocket inet_atonr. dotted_netmaskr+ )ipnetipaddrnetaddrbitsnetmasknetworkr5 r5 r6 address_in_networkh s
r c C s& ddd| > d A }t jtjd|S )zConverts mask from /xx format to xxx.xxx.xxx.xxx
Example: if mask is 24 function returns 255.255.255.0
:rtype: str
l r z>I)r inet_ntoar pack)maskr r5 r5 r6 r w s r c C s* yt j| W n t jk
r$ dS X dS )z
:rtype: bool
FT)r r error) string_ipr5 r5 r6 is_ipv4_address s
r c C s | j ddkryt| jdd }W n tk
r8 dS X |dk sJ|dkrNdS ytj| jdd W q tjk
r| dS X ndS dS )zV
Very simple check of the cidr format in no_proxy variable.
:rtype: bool
rq r Fr r T)countr+ r. r r r r )string_networkr r5 r5 r6
is_valid_cidr s r c c sT |dk }|r"t jj| }|t j| < z
dV W d|rN|dkrDt j| = n
|t j| < X dS )zSet the environment variable 'env_name' to 'value'
Save previous value, yield, and then restore the previous value stored in
the environment variable 'env_name'.
If 'value' is None, do nothingN)rE environr )env_namer
value_changed old_valuer5 r5 r6 set_environ s
r c C s, dd }|}|dkr |d}t | }|jdkr2dS |rdd |jdd jd
D }t|jrxz|D ],}t|rt|j|rdS qb|j|krbdS qbW nD|j}|jr|d j|j7 }x&|D ] }|jj |s|j |rdS qW t
d|6 yt |j}W n t t
jfk
r d }Y nX W dQ R X |r(dS d S )
zL
Returns whether we should bypass proxies or not.
:rtype: bool
c S s t jj| pt jj| j S )N)rE r r upper)kr5 r5 r6 s z'should_bypass_proxies..Nno_proxyTc s s | ]}|r|V qd S )Nr5 ).0r3 r5 r5 r6 s z(should_bypass_proxies.. r ,z:{}F)r hostnamer/ r. r r r portrX endswithr r r r gaierror) r` r get_proxy no_proxy_argparsedproxy_iphost_with_portr3 bypassr5 r5 r6 should_bypass_proxies s<
r c C s t | |dri S t S dS )zA
Return a dict of environment proxies.
:rtype: dict
)r N)r r
)r` r r5 r5 r6 get_environ_proxies s r c C sv |pi }t | }|jdkr.|j|j|jdS |jd |j |jd|j dg}d}x |D ]}||krX|| }P qXW |S )zSelect a proxy for the url, if applicable.
:param url: The url being for the request
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs
Nallz://zall://)r r r scheme)r` proxiesurlparts
proxy_keysproxy proxy_keyr5 r5 r6 select_proxy s
r python-requestsc C s d| t f S )zO
Return a string representing the default user agent.
:rtype: str
z%s/%s)r )ri r5 r5 r6 default_user_agent s r c C s t t djddddS )z9
:rtype: requests.structures.CaseInsensitiveDict
z, gzipdeflatez*/*z
keep-alive)z
User-AgentzAccept-EncodingAccept
Connection)r r )r r rr r5 r5 r5 r6 default_headers' s
r c C s g }d}| j |} | s|S xtjd| D ]}y|jdd\}}W n tk
r^ |d }}Y nX d|j di}xP|jdD ]B}y|jd\}} W n tk
r P Y nX | j |||j |< qzW |j| q(W |S ) zReturn a list of parsed link headers proxies.
i.e. Link: ; rel=front; type="image/jpeg",; rel=back;type="image/jpeg"
:rtype: list
z '"z, * '"r )r r0 r. r r ) r links
replace_charsvalr` r linkr r r5 r5 r6 parse_header_links3 s&
r rT rB c C s | dd }|t jt jfkr dS |dd t jkr6dS |dd t jt jfkrRdS |jt}|dkrhd S |dkr|ddd tkrd
S |d dd tkrd S |dkr|dd t krd
S |d d t krdS dS )z
:rtype: str
N zutf-32r z utf-8-sigrB zutf-16r zutf-8z utf-16-ber z utf-16-lez utf-32-bez utf-32-le)
r BOM_UTF32_LE BOM_UTF32_BEBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BEr _null_null2_null3)datasample nullcountr5 r5 r6 guess_json_utf^ s*
r) c C s8 t | |\}}}}}}|s$|| }}t||||||fS )zGiven a URL that may or may not have a scheme, prepend the given scheme.
Does not replace a present scheme with the one provided as an argument.
:rtype: str
)r r )r`
new_schemer r] rV r queryfragmentr5 r5 r6 prepend_scheme_if_needed~ s
r- c C sB t | }yt|jt|jf}W n ttfk
r< d}Y nX |S )z{Given a url with authentication components, extract them into a tuple of
username,password.
:rtype: (str,str)
r )r r )r r usernamepasswordr_ r )r` r authr5 r5 r6 get_auth_from_url s
r1 s ^\S[^\r\n]*$|^$z^\S[^\r\n]*$|^$c C sf | \}}t |trt}nt}y|j|s4td| W n* tk
r` td||t|f Y nX dS )zVerifies that header value is a string which doesn't contain
leading whitespace or return characters. This prevents unintended
header injection.
:param header: tuple, in the format (name, value).
z7Invalid return character or leading space in header: %sz>Value for header {%s: %s} must be of type str or bytes, not %sN)r[ r
_CLEAN_HEADER_REGEX_BYTE_CLEAN_HEADER_REGEX_STRr1 r r type)r ri r patr5 r5 r6 check_header_validity s
r6 c C sF t | \}}}}}}|s"|| }}|jddd }t|||||dfS )zW
Given a url remove the fragment and the authentication part.
:rtype: str
@r r rl )r rsplitr )r` r r] rV r r+ r, r5 r5 r6
urldefragauth s
r9 c C s` t | jdd}|dk rTt| jtrTy|| j W q\ ttfk
rP tdY q\X ntddS )zfMove file pointer back to its recorded starting position
so it can be read again on redirect.
rA Nz;An error occurred when rewinding request body for redirect.z+Unable to rewind request body for redirect.)rm bodyr[ _body_positionr rK r- r )prepared_request body_seekr5 r5 r6 rewind_body s r> )r r )F)FzBABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~)N)r )b__doc__r
contextlibrC rE r0 r r sysrw rI rs collectionsr r r r _internal_utilsr compatr r r r r
r r r
r r r r r r r r cookiesr
structuresr
exceptionsr r r r rU whereDEFAULT_CA_BUNDLE_PATH
DEFAULT_PORTSplatformr7 r; rP rh rp r r r r r r r r r r r r r r frozensetr r r r r r r contextmanagerr r r r r r r encoder# r$ r% r) r- r1 r r2 r3 r6 r9 r> r5 r5 r5 r6 s @
' =
3 #
% =
&