HEX

File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/http.cpython-310.pyc
o

�N�g���1@sdZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlZddlZddl
Z
ddlZddlZddlZddlmmZddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Zddl!Zddl"Zddl#Zddl$Zddl%Zddl&m'm(Z)ddl*Zddl+Zddl,m-Z-ddl.m/Z/ddl0m1Z1zddl
m2Z2m3Z3dZ4Wn7e5y�zddl6m2Z2m3Z3dZ4Wn!e5y�zddl7m2Z2m3Z3dZ4Wne5y�dZ4YnwYnwYnwzddl8Z8dZ9Wne5y�dZ9Ynwzddl:Z:dZ;Wne5�ydZ;Ynwe�<e=�Z>d	ej?j@��ZAd
d�ZBd&dd
�ZCd&dd�ZDe1d�dddddddddddddddddddddddddddddddddddddddddeAddddddf0dd��ZEd'dd�ZF				d(dd�ZGdd�ZHdd�ZId d!�ZJd"d#�ZKd)d$d%�ZLdS)*z�
Utils for making various web calls. Primarily designed for REST, SOAP, webhooks
and the like, but also useful for basic HTTP testing.

.. versionadded:: 2015.5.0
�N)�
HTTPClient)�compile_template)�jinja_filter)�CertificateError�match_hostnameTFzSalt/cCs�|s|St�dt|�|�|dkrt�|tj�}n,|dkr.t�|�}tj	|d�}|�
�}n|dkr6td��|dkr>td��|d	krFtd
��t�dt|��|S)z�
    Decompress returned HTTP content depending on the specified encoding.
    Currently supports identity/none, deflate, and gzip, which should
    cover 99%+ of the content on the internet.
    z7Decompressing %s byte content with compression type: %sZdeflate�gzip)�fileobjZsdchz+SDCH compression is not currently supported�brz-Brotli compression is not currently supported�compressz*LZW compression is not currently supportedz$Content size after decompression: %s)�log�trace�len�zlib�
decompress�	MAX_WBITS�io�BytesIOr�GzipFile�read�
ValueError)�codingZpgctnt�buf�f�r�C/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/http.py�__decompressContentSs&
�

rcCsF|dkrt|t�s|r|�|jpd�}|St|t�r!|r!|�d�}|S)z%
    Decode only the result_text
    �requestszutf-8)�
isinstance�str�decode�encoding�bytes)�result_text�backend�decode_body�resultrrr�_decode_result_textss�
r&cCshd|vr(tj��}|�d|d�|���d�r(|��}|r(t|t�s(|�	|�}t
||||d�}||fS)z-
    Decode the result_text and headers.
    �Content-Typeztext/�r$r%)�email�message�EmailMessage�
add_header�get_content_type�
startswith�get_content_charsetrrrr&)r"�result_headersr#r$r%�msgZcontent_charsetrrr�_decode_result�s

�r2Z
http_queryZGET�autoZlwp�minion�Pc1fs�i}2|dur,|dkrtj�tj�tjjd��}n|dkr*tj�tj�tjjd��}ni}|s4|�	dd�}|�	dd�}3|3rCtj
j�|3�}3|�	dd�}4|�	dd�}5|5rXtj
j�|5�}5|�	d	d�}6|6rgtj
j�|6�}6|�	d
g�}7t
j�|�j|7vr~d}3d}4d}5d}6d}8|3r�|4r�|dkr�t�d�d}|5r�|6r�d
|5�d|6�d|3�d|4��}8nd
|3�d|4��}8t�d|�}9|9s�tj
j��|dkr�tdur�d|2d<t�|2d�|2St�d�}:|:�tj�|r�tjjj� ||�n|};| dur�t!|�} |!dur�|�	dd�}!|"du�r|�	dd�}"|du�rt"|||||�}t#|;|+�}<t�d|<��t�d|��dk�ret�$tj%��ret&|t'��r\|�(�}=t&|+t)��rR|D]}>|+D]}?|>|?k�rNd|=|><�qC�q?t�*dt+�,|=��n	t�*dt+�,|��|du�rt"|||||�}@t&|@t'��r{|@}n|@�-�}|du�r�i}|du�r�g}|du�r�tj�|�	dtjj.�d�}|du�r�tj�|�	dtjj.�d�}|du�rtj
j/j0�rtj�1|��r�tj
j2�3|d��}Atj
j/�4|A�}BWd�n	1�s�wYt&|Bt'��r�|�5|B�n!tj
j2�3|d ��}Atj
j/�6d!|A�Wd�n	1�swY|D]}C|C�7d�}Dt8|D�d"k�r"�q|Dd#�9�||Dd$�9�<�q|
�s=|�r=|	�r=||	f}
|*t:k�rG|*�d%�}*|*|d&<|3�rp|4�rp�dk�rpd'|v�rpt�d(�d)|d'<d*|v�rpt8|��|d*<|dk�r�t;�<�}E|
|E_=|Ej>�5|�t�*d+|Ej>�|Ej?}F|!|E_@|8du�r�|8|8d,�|E_An
|d-k�r�d}Fnd}F|du�r�|d.k�r�tBjC�D|�}FntBjC�E|�}Ftj�1|��s�|F�F�|F�4�|du�r�|du�r�iS|}d|2d/<|dk�r�i}G|&du�r�t;jGd$d0k�r�d|Gd1<nd|Gd2<|"du�r4t&|"tH��rtj�I|"��r|"|Gd<n#t&|"t)��r.tj�I|"d$��r-tj�I|"d#��r-|"|Gd<nt�d3|"�|-�r\|.�sGd4|2d<t�|2d�|2S|EjJ�|f||.|/tK�L|�fid5�|G��}Hn
|EjJ�|f||d6�|G��}H|H�M�|&du�r�|(d7|HjN�d8��|'|HjO�d9|HiS|)du�r�|H|HjOd:�St�d;t#|HjP|+��|HjN}I|Hj>}J|HjO}K|Hj?}LtQ|K||0|Hd<�}K|K|2d=<�n�|d-k�r t
jJ�R|;|�}Mt
jJjSt
jJ�T|F�g}N|�Ud>��r�|M�V�}Ot
jJ�Wd#�|Nd$<tX�s�t�Yd?�n�|!du�r�t�Yd@�n�d|Ov�r�|O�7d�\}O}ndA}tZ�ZtZj[tZj\�}P|P�]|Ot^|�f�t_j`|P| t_jadB�}Qz	tb|Q�c�|O�Wntd�y@}RzdC�et+�,|R��|2d<|2WYd}R~RSd}R~Rww|"du�r�d}St&|"tH��rXtj�I|"��rW|"}Sn#t&|"t)��rstj�I|"d$��rrtj�I|"d#��rr|"}Snt�d3|"�dStft_dD��r�t_jgjh|S�}T|N�it
jJjW|TdE��n"|M�V�||Sd$dF�}Ut8|S�d#k�r�|Sd#|UdG<tBjjjkdoi|U��|Nd$<t
jJjl|N�}V|D]}C|M�m|C||C��q��fdHdI�|M_nz|V�o|M�}HWnt
jjp�y�}RzdJtH|R�iWYd}R~RSd}R~Rww|&du�s�|)du�r|H|HjOd:�S|Hjq}It'|H�r��}J|H�s�}Ktt|K|J||0|Hd<�\}K}J|K|2d=<�nni}G|"du�rdt&|"tH��r9tj�I|"��r8|"|GdK<n+t&|"t)��r^tj�I|"d$��r]tj�I|"d#��r]|"d$|GdK<|"d#|GdL<nt�d3|"�t&|t'��rpt
j�u|�}|!�rw| |GdM<|�	dNtjjvdN�}W|�	dOtjjvdO�}X|�	dPtjjvdP�}Ytjjjwjx�yd�tj
jz�{tjjj|j}j~�}ZdQ|Zjzv}[|G�5�|||	||!d|'|(|X|Y|,ddR�
�tj
jj�|GddS�}Gz|[�r�t�|WdT�nt��}\|\j�|;fi|G��}HWnftjjjwj��y}Rz#|Rjq|2dU<tH|R�|2d<tt|Rj�j�|Rj�j>||0dV�\|2d=<}]|2WYd}R~RSd}R~RwtZj�t�t�tZj�f�yG}Rz |
du�r,d$|2dU<tH|R�|2d<t�dW|;|2d�|2WYd}R~RSd}R~Rww|&du�sR|)du�rX|H|Hj�d:�S|Hjq}I|Hj>}J|Hj�}Ktt|K|J||0|Hd<�\}K}J|K|2d=<dX|Jv�r�|du�r�t�|JdX�}L|LD]}>|F��|>��q�nd}Lt&|Jt)��r�i}^|JD]}C|C�7d�}Dd�|Dd#d���9�|^|Dd$�9�<�q�|^}Jt�dY|I�t�*dZ|J�t�*d[|F�|J�	d\d]�}_|dk�r�t�|_|K�}Kzt�*d^|K�Wnt��y�}Rzt�*d_|R�WYd}R~Rnd}R~Rww|#du�rtj
j2�3|#d`��
}`|`��|K�Wd�n	1�swY|$du�rEtj�I|$��rEtj
j2�3|$d`��
}a|a��|J�Wd�n	1�s@wY|du�rN|F�F�|du�r�tj
j/j0�r�da|Jv�r�tj
j2�3|d ��&}A|J�	dad�}B|Bdu�r}tj
j/�6db|Bi|A�ntj
j/�6d!|A�Wd�n	1�s�wY|
du�r�|I|2dU<|du�r�|J|2dc<|du�rP|ddk�r�|J�	dedf�}bdg|bv�r�dg}ndh|bv�r�dh}n
di|bv�r�di}ndj}dk}c||cv�r�dl�et+�,|c��|2d<t�|2d�|2S|dhk�r�tj
j���|K�|2dm<n5|dgk�rg|2dm<t���|K�}d|dD]
}>|2dm�it���|>���q
n|dik�r-tj
j��tj
j���|K��|2dm<nd}|%�rPtj
j2�3|%d`��
}e|e��|K�Wd�n	1�sKwY|du�rY|K|2dn<|2S)pz6
    Query a resource, and decode the return data
    NZmasterr4r#�tornado�
proxy_host�
proxy_port�proxy_username�proxy_password�no_proxyrz7Switching to request backend due to the use of proxies.zhttp://�:�@zOhttps?://((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)($|/)Fz_http.query has been set to use requests, but the requests library does not seem to be installed�error�
verify_sslT�certz!Requesting URL %s using %s methodzUsing backend: %s�POSTZ
XXXXXXXXXXzRequest POST Data: %sZcachedirzcookies.txtzcookies.session.p�rb�wb���rz
 http.query()z
User-agentr'zVContent-Type not provided for POST request, assuming application/x-www-form-urlencodedz!application/x-www-form-urlencodedzContent-LengthzRequest Headers: %s)�http�httpsZurllib2Zmozilla�test�0Zprefetch�streamzAThe client-side certificate path that was passed is not valid: %sz1formdata_fieldname is required when formdata=True)�params�files)rL�dataz	HTTP/1.0 z MESSAGE�handle)rO�bodyz"Final URL location of Response: %sr(rPrHzgmatch_hostname() not available, SSL hostname checking not available. THIS CONNECTION MAY NOT BE SECURE!z]SSL certificate verification has been explicitly disabled. THIS CONNECTION MAY NOT BE SECURE!i�)�ca_certs�	cert_reqsz3The certificate was invalid. Error returned was: {}�
SSLContext)�context)�host�port�	cert_file�key_filecs�S�Nrr��methodrr�<lambda> szquery.<locals>.<lambda>�ErrorZclient_certZ
client_keyrQZ
http_max_bodyZhttp_connect_timeoutZhttp_request_timeout�
max_body_size)
r[�headersZ
auth_usernameZ
auth_passwordrPZ
validate_certZallow_nonstandard_methods�streaming_callback�header_callback�connect_timeoutZrequest_timeout�raise_errorZdecompress_response)�to_str)r^�status)r$z$Cannot perform 'http.query': %s - %sz
Set-CookiezResponse Status Code: %szResponse Headers: %szResponse Cookies: %szContent-Encoding�identityzResponse Text: %szfCannot Trace Log Response Text: %s. This may be due to incompatibilities between requests and logging.�wz
set-cookie�Cookier_r3zcontent-typezapplication/json�xml�json�yaml�plain)rjrirkrlz9Invalid decode_type specified. Valid decode types are: {}�dict�textr)��saltZconfigZ
master_config�os�path�join�syspathsZ
CONFIG_DIRZ
minion_config�get�utilsZstringutilsrd�urllib�parse�urlparse�hostnamer�debug�re�match�networkZrefresh_dns�HAS_REQUESTSr>�logging�	getLogger�setLevel�WARNING�extr6ZhttputilZ
url_concat�
get_ca_bundle�_render�sanitize_url�isEnabledForZTRACErrm�copy�listr�pprintZpformat�
splitlinesZ	CACHE_DIRZmsgpackZHAS_MSGPACK�isfilerM�fopen�load�update�dump�splitr
�strip�	USERAGENTrZSession�authr_�cookies�verifyZproxiesrG�	cookiejarZMozillaCookieJarZLWPCookieJarZsave�__version__r�exists�requestr�StringIOZraise_for_statusZstatus_code�content�urlr&ZRequestZHTTPHandlerZHTTPCookieProcessorr.Zget_hostZHTTPSHandler�HAS_MATCHHOSTNAME�warning�socket�AF_INET�SOCK_STREAM�connect�int�ssl�wrap_socket�
CERT_REQUIREDr�getpeercertr�format�hasattrrS�load_cert_chain�append�client�HTTPSConnectionZbuild_openerr,Z
get_method�open�URLError�code�inforr2�	urlencodeZDEFAULT_MINION_OPTSZ
httpclientZAsyncHTTPClientZ	configure�argsZget_function_argspecZsimple_httpclientZSimpleAsyncHTTPClientZ
initializerNrrZfetch�	HTTPError�responserP�herror�OSError�TimeoutError�gaierror�parse_cookie_headerZ
set_cookier�UnicodeEncodeError�writerj�loads�ETZ
fromstringriZto_dictrkZ	safe_load)fr�r[rLrNZ	data_fileZheader_dictZheader_listZheader_file�username�passwordr�rZdecode_typerer_rnr�Z
cookie_jarZ
cookie_formatZpersist_sessionZsession_cookie_jarZdata_renderZ
data_rendererZ
header_renderZheader_renderer�
template_dictrIZtest_url�noderV�optsr#�	ca_bundler?r@�text_outZheaders_outZ
decode_outrKr`rarOZagent�hide_fieldsrcZformdataZformdata_fieldnameZformdata_filenamer$�kwargs�retr7r8r9r:r;Zhttp_proxy_urlr|Zrequests_logZurl_full�log_urlZlog_data�item�fieldZ
header_tpl�fh_Zsession_cookies�header�compsZsessZsess_cookiesZ
req_kwargsr%Zresult_status_coder0r"Zresult_cookiesr��handlersry�sockZsockwrap�excZ
cert_chainrTZcert_kwargsZopenerZmax_bodyrb�timeoutZclient_argspecZsupports_max_body_sizeZdownload_client�_Zresult_headers_dictrZtofZhof�content_typeZ
valid_decodes�itemsZdofrrZr�query�s�8��
���




��

�


�
��
��


��

�


��








�$�����

�
���

��
�
�����
	�$�������

�
�$���
�
��
����
��
���

��
&

���
��


��







��


�
�
r�cCs�ttd�rtjS|duri}|�dd�}|dur tj�|�r |S|�ddtjj	gi�}|�dg�D]}dD]}tj�
||�}tj�|�rJ|Sq5q1d}|D]}tj�|�r\|SqPtjj�
�ritrit��SdS)zs
    Return the location of the ca bundle file. See the following article:

        http://tinyurl.com/k7rx42a
    �__return_value__Nr��
file_roots�base)z
cacert.pemz
ca-bundle.crt)z"/etc/ssl/certs/ca-certificates.crtz /etc/pki/tls/certs/ca-bundle.crtz&/etc/pki/tls/certs/ca-bundle.trust.crtz/etc/ssl/certs/ca-bundle.crtz&/var/lib/ca-certificates/ca-bundle.pemz/etc/ssl/cert.pem)r�r�r�rtrprqr�rorsZSRV_ROOT_DIRrrru�platformZ
is_windows�HAS_CERTIFI�certifi�where)r�Zopts_bundler�Z	salt_rootrqZ	cert_pathZ	locationsrrrr�s.
��
�r�c	Cs�|duri}|durt|�}|durt�d�dS|dur#|�dd�}t�d||�t|dddd|d�|dur�t|t�r@|g}t|t�sLt�d	�dSd
}|D]L}t	j
�|�r�t�d||�z#tj
j�|d��}d
�||��f�}Wd�n1s|wYWqPty�}z
t�d||�WYd}~qPd}~wwqP|r�t�d|�z'tj
j�|d��}|�d
�|�|�Wd�WdS1s�wYWdSty�}zt�d||�WYd}~dSd}~wwdSdS)a.
    Attempt to update the CA bundle file from a URL

    If not specified, the local location on disk (``target``) will be
    auto-detected, if possible. If it is not found, then a new location on disk
    will be created and updated.

    The default ``source`` is:

        http://curl.haxx.se/ca/cacert.pem

    This is based on the information at:

        http://curl.haxx.se/docs/caextract.html

    A string or list of strings representing files to be appended to the end of
    the CA bundle file may also be passed through as ``merge_files``.
    Nz/Unable to detect location to write CA bundle toZ
ca_bundle_urlz!http://curl.haxx.se/ca/cacert.pemzAttempting to download %s to %sTF)rnrr_rer�zIA value was passed as merge_files which was not either a string or a listrDz#Queueing up %s to be appended to %s�r�
z.Reading from %s caused the following error: %szAppending merge_files to %s�az,Writing to %s caused the following error: %s)r�rr>rtrzr�rrr�rprqr�rorurMr�rrrr�r�)	�target�sourcer�Zmerge_filesZ
merge_contentrWZfcfr�Ztfprrr�update_ca_bundle5s`


�������

&����r�c
Cs�|rR|duri}|s|�dd�}tj�|i�}|�d�}|�d�}t|||||fi|��}tjj�|�r8|��}t	|��
d�rPt	|��
d�sPt	|��dd	�d	}|Stjj�
|d
��}	|	��Wd�S1siwYdS)z
    Render a template
    N�rendererz
jinja|yamlZrenderer_blacklistZrenderer_whitelistz#!z#!/r�rFr�)rtro�loader�renderrruZstringioZis_readablerrr.r�rMr�)
�templater�r�r�r�ZrendZ	blacklistZ	whitelistr�r�rrrr�s*


��$�r�cCsd}g}|�d�D]!}|��}d|vr%d|vr%|�d�D]}|�|�qq	|�|�q	g}i}d}|D]Y}|�d�}	|	d��|	d<|	d|vrp|	d|vrU|�|�i}t|	�dkrid�|	dd	��||	d<q3d
||	d<q3|d
ur}|�|�i}d}d�|	dd	��||	d<d
}q3|r�|�|�d}
g}|D]m}d	}d	}
t|�D]}||vr�q�|}|�|�}
q�d|vr�tj	�
|d�|d<|
D]
}||vr�d||<q�|d
dkr�d|d
<|ddkr�i|d<|ddkr�d|d<|�dd	�|�dd	�|�tj	jd||
d�|���q�|S)z�
    Parse the "Set-cookie" header, and return a list of cookies.

    This function is here because Tornado's HTTPClient doesn't handle cookies.
    )	�expiresrq�domain�version�httponly�secure�commentzmax-age�samesite�;�,r�F�=rrFNT)r�rVZport_specifiedr�Zdomain_specifiedZdomain_initial_dotrqZpath_specifiedr�r��discardr�Zcomment_url�restrDr�r�r�r�)�name�valuer)r�r�r��lowerr
rrr��poprGr�Z	http2timerh)r�ZattribsZmorselsr��partr�ZcookieZ	value_setZmorsel�partsZreqdr�r�r�Zreqrrrr��sp
�



� r�cCs�t|t�rItj�|�}|d}t|�dkr|d7}|dd�D]$}d}|D]}|�d�}|r9|�d�}t||�}q%t||�}q%||7}q|�d�St	|�S)z4
    Make sure no secret fields show up in logs
    rrF�?N�&)
rr�rvrw�
splitqueryr
r��_sanitize_url_components�rstripr)r�r�Z	url_compsr��pairZurl_tmpr�Z
comps_listrrrr�s 




r�cCsl|sdS|d�|�d��r!|�d�}|�|d�|t||�S|d�d�}|�|d�|t||�S)zC
    Recursive function to sanitize each component of the url.
    rDrr�z=XXXXXXXXXX&r�)r.�remover�)Z	comp_listr�r�rrrr�s
r�cCsdt��}|r
|r
||f|_|r|st�d�dS|r#d|i}t|�|_|s(d|_|r0|j�|�|S)z#
    create a requests session
    z;You cannot use both ca_bundle and verify_ssl False togetherFr�)	r�sessionr�rr>r�r�r_r�)�userr�r?r�r_rr�rrrr*s


r)NNrY)NNNN)NNTNN)M�__doc__�
email.messager)r�http.clientrGZhttp.cookiejarrrrpr�r{r�r��urllib.errorrv�urllib.parse�urllib.requestZxml.etree.ElementTreeZetreeZElementTreer�rZsalt.configroZsalt.ext.tornado.httputilZ"salt.ext.tornado.simple_httpclientZsalt.loaderZ
salt.syspathsZsalt.utils.argsZsalt.utils.dataZsalt.utils.filesZsalt.utils.jsonZsalt.utils.msgpackZsalt.utils.networkZsalt.utils.platformZsalt.utils.stringutilsZsalt.utils.urlZsalt.utils.xmlutilruZxmlutilriZsalt.utils.yamlZsalt.versionZsalt.ext.tornado.httpclientrZ
salt.templaterZsalt.utils.decorators.jinjarrrr��ImportErrorZbackports.ssl_match_hostnameZsalt.ext.ssl_match_hostnamerr~r�r�r��__name__rr�r�r�rr&r2r�r�r�r�r�r�r�rrrrr�<module>s�������

 

�
t3
�Jk