HEX

File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/files.cpython-310.pyc
o

�N�gao�@sRdZddlZddlZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlZddlZddl
ZddlZddlZddlZddlZddlmZmZmZddlmZzddlZdZWneyidZYnwe�e�ZdZ dZ!d	e!Z"d
Z#ddd
dddd�Z$dd�e$�%�D�Z&dd�Z'dd�Z(dd�Z)dd�Z*dVdd�Z+dd �Z,dWd!d"�Z-ej.dXd%d&��Z/d'd(�Z0ej.d)d*��Z1d+d,�Z2ej.d-d.��Z3ej.d/d0��Z4dYd1d2�Z5d3d4�Z6d5d6�Z7ed7�d8d7��Z8dZd9d:�Z9d;d<�Z:dWd=d>�Z;ed?�d[dAdB��Z<edC�dDdE��Z=dFdG�Z>edH�dIdH��Z?dJdK�Z@dLdM�ZAdNdO�ZBdPdQ�ZCdWdRdS�ZDdTdU�ZEdS)\z"
Functions for working with files
�N)�CommandExecutionError�
FileLockError�MinionError)�jinja_filterTF)��file)�http�https�ftpZswiftZs3)�saltr�__salt.tmp.��`�@�8�(� )�sha512�sha384�sha256�sha224�sha1�md5cCsi|]\}}||�qS�r)�.0�x�yrr�D/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/files.py�
<dictcomp>1src
CsHzt|�WdSty#}ztjd|tjd�WYd}~dSd}~ww)z 
    Remove temporary files
    z+Exception while removing temp directory: %s)Zexc_info_on_loglevelN)�rm_rf�	Exception�log�error�logging�DEBUG)�tmp�excrrr�__clean_tmp4s���r'cCsJ|��}dD]}|�d|�rdSqdD]
}|�d|�r"|SqdS)zH
    Guess an archive type (tar, zip, or rar) by its file extension
    )
�tarztar.gz�tgzztar.bz2Ztbz2Ztbzztar.xzZtxzztar.lzmaZtlz�.r()�zipZrarN)�lower�endswith)�nameZendingrrr�guess_archive_typeBs��r/cOsPd|vrd|d<|�dd�}tj|i|��\}}|dur ||fSt�|�~|S)a
    Helper function which does exactly what ``tempfile.mkstemp()`` does but
    accepts another argument, ``close_fd``, which, by default, is true and closes
    the fd before returning the file path. Something commonly done throughout
    Salt's code.
    �prefixr�close_fdTF)�pop�tempfile�mkstemp�os�close)�args�kwargsr1Zfd_Zf_pathrrrr4[s
r4c
Cs�tjj�|�D];\}}}|�|d��tj�}tj�||�}tj�	|�s(t�
|�|D]}tj�|||�}tj�||�}	t�||	�q*qdS)z�
    Recursively copy the source directory to the destination,
    leaving files with the source does not explicitly overwrite.

    (identical to cp -r on a unix machine)
    rN)
r�utils�pathZos_walk�replace�lstripr5�sep�join�exists�makedirs�shutil�copyfile)
�source�dest�root�_�filesZpath_from_sourceZtarget_directoryr.Zfile_path_from_source�target_pathrrr�recursive_copyms
��rIrc
	Cs�tj�|�s
td|����tj�tj�|��std|����tj�|�}tj�tj�|��}t||d�}t	�
||�d}|rEtj�|d�}|dksO|dkrZ|rZtj�|�rZt
||�|dksd|dkre|re	d}tjj��s~zt�|�}Wn	ty}Ynwzt	�||�Wnty�t|��w|dur�t�||j|j�t�||j�tjj�d	�}	|	r�d
}
ztjj��}
Wntt fy�Ynw|
dkr�t!tj"d��}|	|g}t#j$|||d
�Wd�n1s�wYtj�|�r�t|�dSdS)zj
    Copy files from a source to a destination in an atomic way, and if
    specified cache the file.
    z%[Errno 2] No such file or directory: )r0�dirrZfile_backupZminionZbothZmasterNZ
restoreconFZ	Enforcing�w)�stdout�stderr)%r5r:�isfile�OSError�isdir�dirname�basename�abspathr4rArBr>r?�
backup_minionrr9�platform�
is_windows�stat�mover r'�chown�st_uid�st_gid�chmod�st_mode�which�modulesZselinuxZ
getenforce�ImportErrorr�fopen�devnull�
subprocess�call)
rCrDZbackup_modeZcachedir�bname�dname�tgt�bkroot�fstatZrcon�policyZdev_null�cmdrrrrBs`
�����rBc
Cs�z	t�||�WdStyT}z?|jtjkr�zt�|�Wn#tyB}z|jtjkr8td|�d|j����WYd}~nd}~wwt�||�WYd}~dSd}~ww)z�
    On Windows, os.rename() will fail with a WindowsError exception if a file
    exists at the destination path. This function checks for this error and if
    found, it deletes the destination path first.
    zError: Unable to remove �: N)	r5�renamerO�errno�EEXIST�remove�ENOENTr�strerror)�src�dstr&rrrrm�s �����rmcCsv|dur
t|t�r|f}nd}|j|vrdS|jtjkr#t|�d���|jtjkr0td|����td�|j||j���)z�
    Common code for raising exceptions when reading a file fails

    The ignore argument can be an iterable of integer error codes (or a single
    integer error code) that should be ignored.
    Nrz does not existzPermission denied reading from z(Error {} encountered reading from {}: {})�
isinstance�intrnrqrZEACCES�formatrr)r&r:�ignorerrr�process_read_exception�s 
�

��ry�皙�����?c
#s��t|t�s
td��|dur|d}�durt���d}d�fdd�	}z�z�tj�|�r9tj�|�s9|d|�d��tjtj	Btj
B}t���|kr�zt�||�}Wn/ty�}	z#|	j
t
jkrk|d	�|	j
||	j��t�d
||�t�|�WYd}	~	n'd}	~	wwt�|d��Wd�n1s�wYt�d|�d
}dVnt���|ksJ|d�||��Wn#ty��ty�}	z|d|�d|	���WYd}	~	nd}	~	wwW|r�t�|�t�d||�dSdS|r�t�|�t�d||�ww)zJ
    Obtain a write lock. If one exists, wait for it to release first
    zpath must be a stringNz.wFcst|�d��)z'
        Raise a FileLockError
        ��
time_start)r)�msgZracer|rr�_raise_error�szwait_lock.<locals>._raise_errorzlock_fn z exists and is not a filez/Error {} encountered obtaining file lock {}: {}z(Lock file %s exists, sleeping %f secondsrKzWrite lock %s obtainedTzDTimeout of {} seconds exceeded waiting for lock_fn {} to be releasedz&Error encountered obtaining file lock rlzWrite lock for %s (%s) released�F)ru�strr�timer5r:r?rN�O_CREAT�O_EXCL�O_WRONLY�openrOrnrorwrrr!Ztrace�sleep�fdopenr rp)
r:Zlock_fn�timeoutr�r}Z
obtained_lockrZ
open_flagsZfh_r&rr|r�	wait_lock�sl�

�������� ���
�
�r�cCst�d�}t�|�|S)z#
    Returns the current umask
    r)r5�umask)�retrrr�	get_umask.s

r�c	csN�|dustjj��rdVdSt�|�}zdVWt�|�dSt�|�w)zM
    Temporarily set the umask and restore once the contextmanager exits
    N)rr9rUrVr5r�)�maskZ	orig_maskrrr�	set_umask7s�

r�cOs�z|ddvrt|d�d���Wn	tyYnwd}|�dd�rtt|�dkrLt|�}d|dvrK|d�dd�|d<d|dvrK|dd7<n(|�d	�rpd|d	vro|d	�dd�|d	<d|d	vro|d	d7<nd
|d	<d|vr�d}t|�dkr�t|�}d|dvr�d
}|�d	d�r�d|d	vr�d
}|s�t|d<|s�|�dd�s�d|d<|�d�dkr�d|�d	d�vr�t�	d�t
j|d<t|i|��}t
�r�ztj}Wnty�d}Ynwt�|��tj�}t�|��tj||B�|S)a�
    Wrapper around open() built-in to set CLOEXEC on the fd.

    This flag specifies that the file descriptor should be closed when an exec
    function is invoked;

    When a file descriptor is allocated (as with open or dup), this bit is
    initially cleared on the new file descriptor, meaning that descriptor will
    survive into the new program after exec.

    NB! We still have small race condition between open and fcntl.
    r)r��z# is not a permitted file descriptorN�binaryr��b�t�mode�rb�encodingFT�newliner�	bufferingzaLine buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used)�	TypeError�
IndexErrorr2�len�listr;�get�__salt_system_encoding__r!�debug�io�DEFAULT_BUFFER_SIZEr��is_fcntl_available�fcntl�
FD_CLOEXEC�AttributeError�fileno�F_GETFD�F_SETFD)r7r8r��f_handler�Z	old_flagsrrrraGsn
���
��
��
�rac/s���d�dd�}�d}t|g��Ri|���M}z3tdd�r;tj}�r3t�fdd�|D��r3tj}t�|��|�|VWtdd�rMt�|��tj�ntdd�r]t�|��tj�wwWd�dS1siwYdS)	z;
    Shortcut for fopen with lock and context manager.
    rr�N�waT�Zcheck_sunoscsg|]}|�dv�qS)rr)r�write�r7rr�
<listcomp>�szflopen.<locals>.<listcomp>)	rar�r��LOCK_SH�any�LOCK_EX�flockr��LOCK_UN)r7r8�filename�writingr�Z	lock_typerr�r�flopen�s&�

�
���"�r�c	os��|�dd�}|�dd�}|�dd�}t|i|���M}|d}t�|�}ttd�rC|j|ks3|j|krCdd	�||fD�rCt�|||�|dur\t�|j	�}||kr\t�
||j	|A|B�|VWd�dS1sjwYdS)
a�
    Shortcut for fopen with extra uid, gid, and mode options.

    Supported optional Keyword Arguments:

    mode
        Explicit mode to set. Mode is anything os.chmod would accept
        as input for mode. Works only on unix/unix-like systems.

    uid
        The uid to set, if not set, or it is None or -1 no changes are
        made. Same applies if the path is already owned by this uid.
        Must be int. Works only on unix/unix-like systems.

    gid
        The gid to set, if not set, or it is None or -1 no changes are
        made. Same applies if the path is already owned by this gid.
        Must be int. Works only on unix/unix-like systems.

    �uid����gidr�NrrYcSsg|]}|dkr|�qS)r�r�r�irrrr��szfpopen.<locals>.<listcomp>)r2rar5rW�hasattrrZr[rY�S_IMODEr]r\)	r7r8r�r�r�r�r:Zd_statZ	mode_partrrr�fpopen�s$�

�"�r�c
cs:�|durt�}zt�|�}Wnty*}z|dur||�WYd}~dSd}~ww|rHt�|�}|jdkrH|j|jf}||vrCdS|�|�gg}	}
|D]}tj�	||�}tj�
|�rd|	�|�qO|
�|�qO|rr||	|
fV|	D]}tj�	||�}
|s�tj�|
�s�t
|
||||�EdHqt|s�||	|
fVdSdS)z�
    A clone of the python os.walk function with some checks for recursive
    symlinks. Unlike os.walk this follows symlinks by default.
    Nr)�setr5�listdirrOrW�st_ino�st_dev�addr:r>rP�append�islink�	safe_walk)�top�topdown�onerror�followlinks�_seen�names�err�status�node�dirs�nondirsr.�	full_path�new_pathrrrr��sD���



��r�cCs&zt�|�WdStyYdSw)z
    Safely remove a file
    N)r5rprO)rgrrr�safe_rm	s
�r�cCsrdd�}tj�|�stj�|�st�|�dStjj��r0z	tjj	�
|�}Wn	ty/Ynwtj
||d�dS)zj
    Platform-independent recursive delete. Includes code from
    http://stackoverflow.com/a/2656405
    cSs6tjj��rt�|tj�st�|tj	�||�dS�)a/
        Error handler for `shutil.rmtree`.

        If the error is due to an access error (read only file)
        it attempts to add write permission and then retries.

        If the error is for another reason it re-raises the error.

        Usage : `shutil.rmtree(path, onerror=onerror)`
        N)
rr9rUrVr5�access�W_OKr\rW�S_IWUSR)�funcr:�exc_inforrr�_onerrorszrm_rf.<locals>._onerror)r�N)r5r:r�rPrprr9rUrV�stringutilsZ
to_unicoder�rA�rmtree)r:r�rrrrs�r�is_emptycCs(z	t�|�jdkWStyYdSw)z
    Is a file empty?
    rF)r5rW�st_sizerO)r�rrrr�6s
�cCs|r
tjj��r
dStS)z�
    Simple function to check if the ``fcntl`` module is available or not.

    If ``check_sunos`` is passed as ``True`` an additional check to see if host is
    SunOS is also made. For additional information see: http://goo.gl/159FF8
    F)rr9rUZis_sunos�	HAS_FCNTLr�rrrr�Bsr�cCsBdd�}t|t�st�d|t|d��dd��Stjd||tjd�S)a�
    Input the basename of a file, without the directory tree, and returns a safe name to use
    i.e. only the required characters are converted by urllib.parse.quote
    If the input is a PY2 String, output a PY2 String. If input is Unicode output Unicode.
    For consistency all platforms are treated the same. Hard coded to utf8 as its ascii compatible
    windows is \ / : * ? " < > | posix is /

    .. versionadded:: 2017.7.2

    :codeauthor: Damon Atkins <https://github.com/damon-atkins>
    cSstjj|�d�dd�S)Nrr)�safe)�urllib�parse�quote�group)Zre_objrrr�_replace[sz$safe_filename_leaf.<locals>._replacez[\\:/*?"<>|]�utf8�ascii�backslashreplace)�flags)rur��re�sub�encode�UNICODE)Z
file_basenamer�rrr�safe_filename_leafNs
�r�cCsh|stj}t|t�st|t�r|�d�}tj�|�\}}|�dd�|�|�D��}|r2|�||g�}|S)a3
    Input the full path and filename, splits on directory separator and calls safe_filename_leaf for
    each part of the path. dir_sep allows coder to force a directory separate to a particular character

    .. versionadded:: 2017.7.2

    :codeauthor: Damon Atkins <https://github.com/damon-atkins>
    r�cSsg|]}t|��qSr)r�)rZfile_sectionrrrr�{sz!safe_filepath.<locals>.<listcomp>)	r5r=rur�r�r:�
splitdriver>�rsplit)Zfile_path_nameZdir_sep�driver:rrr�
safe_filepathis	
�r�Zis_text_file�cs�dd��d��fdd�tdd�D��d}z|�|�}Wn1tyMzt|d	��
}|�|�}Wd
�n1s9wYWntyJYYdSwYnwd|vrTdS|sXd
Sz|�d�Wd
StyiYnw|�d
|�}t	t
|��t
|�dkS)a
    Uses heuristics to guess whether the given file is text or binary,
    by reading a single block of bytes from the file.
    If more than 30% of the chars in the block are non-text, or there
    are NUL ('') bytes in the block, assume this is a binary file.
    cSs
t|f�S�N)�bytes)rrrr�int2byte�s
zis_text.<locals>.int2byte�c3s�|]}�|�VqdSr�rr��r�rr�	<genexpr>�s�zis_text.<locals>.<genexpr>r�s

	r�NF�Tzutf-8g333333�?)r>�range�readr�rarO�decode�UnicodeDecodeError�	translate�floatr�)�fp_�	blocksizeZtext_characters�blockZfp2_Znontextrr�r�is_text�s6	"�����	
�rZis_bin_filec	Cs�tj�|�sdSz=t|d��-}z|�d�}|�t�}tjj	�
|�WWd�WSty:YWd�WdSw1s>wYWdStyOYdSw)z�
    Detects if the file is a binary, returns bool. Returns True if the file is
    a bin, False if the file is not and None if the file is not available.
    Fr��NT)
r5r:rNrar�r�r�rr9r��	is_binaryrrO)r:r�datarrrr�s"

����rc
CsFzt�|�WdSty"}z|jtjkr�WYd}~dSd}~ww)zS
    Runs os.remove(path) and suppresses the OSError if the file doesn't exist
    N)r5rprOrnrq)r:r&rrrrp�s���rp�
list_filescCsht�}|�|�t|�D]#\}}}|D]}|�tj�||��q|D]}|�tj�||��q"qt|�S)zS
    Return a list of all files found under directory (and its subdirectories)
    )r�r�r�r5r:r>r�)�	directoryr�rEr�rGr.rrrr
�s
�c	Cs,z	t|�dd�WSttfyYdSw)zj
    Convert the st_mode value from a stat(2) call (as returned from os.stat())
    to an octal mode.
    ���Nr)�octr�r��r�rrr�st_mode_to_octal�s
�rcCsF|durdSt|t�st|�}|�dd�}|�d��d��d��d�S)z�
    Return a mode value, normalized to a string and containing a leading zero
    if it does not have one.

    Allow "keep" as a valid mode (used by file state/module to preserve mode
    from the Salt fileserver in file states).
    NZ0o�0�"�'�)rur�r;�stripr<�zfillrrrr�normalize_mode�s
rcCs\dddddd�}t|�}t�d|�}|std��t|�d��}d	|�|�d�d
�}||S)z/
    Convert human-readable units to bytes
    r�r��rrz)�K�M�G�T�Pz^(\d+)([KMGTP])?$zFSize must be all digits, with an optional unit type (K, M, G, T, or P)ir)r�r��match�
ValueErrorrvr�r�)Z
human_sizeZsize_exp_mapZhuman_size_strrZsize_numZunit_multiplierrrr�human_size_to_bytes�s�rc	Cs,tj�|�\}}tjj��r|�dd�}n|dd�}tjj��s&t�|�}t	t
t��d��dd�}tjj��r@t�d�}nt�d�}|dd	��|�d|d	d���}tj�
|||�d|���}tj�tj�|��svt�tj�|��t�||�tjj��s�t�||j|j�t�||j�dSdS)
z%
    Backup a file on the minion
    �:rFr�Ni@Bi����z%a_%b_%d_%H-%M-%S_%Yz%a_%b_%d_%H:%M:%S_%Yr)r5r:�splitrr9rUrVr;rWr�rvr��strftimer>rPrQr@rArBrYrZr[r\r])	r:rhrfreZsrc_dirri�msecsZstampZbkpathrrrrTs&

"�rTcCsFtjd|dd��}tj�|j���Wd�S1swYdS)z�
    Detect case insensitivity on a system.

    Returns:
        bool: Flag to indicate case insensitivity

    .. versionadded:: 3004

    ZTmPT)r0rJ�deleteN)r3ZNamedTemporaryFiler5r:r?r.r,)r:Ztmp_filerrr�case_insensitive_filesystem!s
$�r%cCs�dd�}dd�}dd�}dd�}tj�|�std	��zt|d
��
}|�d�}Wd�n1s0wYWntyAtd
��w||�}|rJ|S||�rPdS||�rVtS||�r\dStd��)ao
    Detect a file's encoding using the following:
    - Check for Byte Order Marks (BOM)
    - Check for UTF-8 Markers
    - Check System Encoding
    - Check for ascii

    Args:

        path (str): The path to the file to check

    Returns:
        str: The encoding of the file

    Raises:
        CommandExecutionError: If the encoding cannot be detected
    cSs0z
|�d�t�d�WdStyYdSw)N�ASCIIzFound ASCIIFT)r�r!r�r��_datarrr�check_asciiBs
��z!get_encoding.<locals>.check_asciic
Ss�dtjj�tj�fdtjj�tj�fdtjj�tj�fdtjj�tj�fdtjj�tj	�fdtjj�d�fdtjj�d�fdtjj�d	�fdtjj�d
�fdtjj�d�fg
}|D]\}}|�
|�rlt�d|�|SqYd
S)Nz	UTF-32-BEz	UTF-32-LEz	UTF-16-BEz	UTF-16-LE�UTF-8zUTF-7z+/v8-z+/v8z+/v9z+/v+z+/v/zFound BOM for %sF)
rr9r��to_bytes�codecs�BOM_UTF32_BE�BOM_UTF32_LE�BOM_UTF16_BE�BOM_UTF16_LE�BOM_UTF8�
startswithr!r�)r(Zboms�	_encodingZbomrrr�	check_bomLs"�
�zget_encoding.<locals>.check_bomcSs&z|�d�}WdStyYdSw)Nr*FT)r�r)r(�decodedrrr�check_utf8_markersb���z(get_encoding.<locals>.check_utf8_markerscSs&z|�t�WdStyYdSw)NFT)r�r�rr'rrr�check_system_encodingjr7z+get_encoding.<locals>.check_system_encodingz
Not a filer�rNzFailed to open filer*r&zCould not detect file encoding)r5r:rNrrar�rOr�)r:r)r4r6r8rr	r�rrr�get_encoding/s0
���r9)rrr�)Nrzr{N)TNTNr�)r�)F�__doc__r,�
contextlibrnr�r#r5r�rArWrcr3r��urllib.parser�Zsalt.modules.selinuxrZsalt.utils.pathZsalt.utils.platformZsalt.utils.stringutilsZsalt.exceptionsrrrZsalt.utils.decorators.jinjarr�r�r`�	getLogger�__name__r!ZLOCAL_PROTOSZ
REMOTE_PROTOSZVALID_PROTOSZTEMPFILE_PREFIXZHASHES�itemsZ
HASHES_REVMAPr'r/r4rIrBrmry�contextmanagerr�r�r�rar�r�r�r�rr�r�r�r�rrrpr
rrrrTr%r9rrrr�<module>s��
�
;
D	
O


-1
#


'