HEX
Server: Apache
System: Linux server2.voipitup.com.au 4.18.0-553.104.1.lve.el8.x86_64 #1 SMP Tue Feb 10 20:07:30 UTC 2026 x86_64
User: posscale (1027)
PHP: 8.2.29
Disabled: exec,passthru,shell_exec,system
$ pwd: /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/
Upload Files
File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/gitfs.cpython-310.pyc
o

�N�g�1�
@s
dZddlZddlZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlZddlZddl
Z
ddlZddlZddlZddlZddlZddlmZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Zddl!Zddl"Zddl#Zddl$Zddl%m&Z&ddl%m'Z(ddl)m*Z*m+Z+m,Z,ddl-m.Z.ddl/m0Z0dd	lm1Z2dd
l$m3Z3e(dZ4dZ5d
Z6dZ7dZ8dZ9dZ:dZ;dZ<dZ=e	�>e?�Z@dZAzddlBZBdZAWn	eCy�Ynwz.ejDjE�F��r
ejDjG�Hd�dk�r
ddlImJZJeJ��r
eCd��ddlKZKddlLZLe3eKjM�ZNWneO�y'dZNYnwzeddlPZPeP�Q��eP�Rd�ddlSZSWd�n	1�sFwYe3eSjM�ZTe3eSjU�ZUeTe3d�k�ryz
ddlVZSddlWZSWn
eC�ypYn	wdd�ZXeXeSjY_ZzeSj[j\Z\Wne]�y�eOZ\YnwWn"eO�y�Z^zdZTdZUe_e^eC��s�e@�`d�WYdZ^[^ndZ^[^wwe3d �Zae3d!�Zbe3d"�Zcd#d$�Zdd%d&�ZeGd'd(�d(�ZfGd)d*�d*ef�ZgGd+d,�d,ef�Zhehegd-�ZiGd.d/�d/�ZjGd0d1�d1ej�ZkGd2d3�d3ej�ZlGd4d5�d5ej�Zmd6d7�ZndS)8zJ
Classes which provide the shared base for GitFS, git_pillar, and winrepo
�N)�datetime)�DEFAULT_HASH_TYPE)�DEFAULT_MASTER_OPTS)�FileserverConfigError�GitLockError�get_error_message)�tagify)�OrderedDict)�get_machine_identifier)�VersionZgitfs_ref_types��name��d)�pygit2)�user�password�pubkey�privkey�
passphrase�
insecure_auth)�
mountpoint�root�refzgGitPython is installed, you may wish to set %s_provider to 'gitpython' to use GitPython for %s support.zbpygit2 is installed, you may wish to set %s_provider to 'pygit2' to use pygit2 for for %s support.z�Cache path %s (corresponding remote: %s) exists but is not a valid git repository. You will need to manually delete this directory on the master to continue to use this %s remote.FT�gitz/usr/bin/git)�git_is_stubzGit is not present.�ignorez0.26.3cCs|sdStj�|��d�S)N�utf-8)rZffi�string�decode)�ptrrr�D/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/gitfs.py�__maybe_string�sr"zFailed to import pygit2z0.3z0.20.3z0.20.0c
s�tttddddtd���fdd�}|�vr!||�}|dur!t|�S�|}|dkrKt|tftf�s5t|�}t|t�rDdd�|�d�D�Sd	d�|D�Sz||�WStym}zt�d
||�t|�WYd}~Sd}~ww)zO
    Force params to be strings unless they should remain a different type
    Z
stringlist)�
ssl_verifyr�disable_saltenv_mapping�saltenv_whitelist�saltenv_blacklist�refspecs�	ref_types�update_intervalc	s`�D])}z|�d|�r|}W|SWqty+|�dt|��r)|}Y|SYqwd}|S)N�_)�endswith�	TypeError�str)�key�item�ret�Znon_string_paramsrr!�_find_global�s ���z#enforce_types.<locals>._find_globalNcS�g|]}|���qSr)�strip��.0�xrrr!�
<listcomp>��z!enforce_types.<locals>.<listcomp>�,cSsg|]}t|��qSr�r-r5rrr!r8�r9zGFailed to enforce type for key=%s with val=%s, falling back to a string)	�bool�intr-�
isinstance�list�split�	Exception�log�error)r.�valr2�expected�excrr1r!�
enforce_types�s@�

���rGcCstd|����)z7
    Fatal configuration issue, raise an exception
    zFailed to load )r)�rolerrr!�failhard�srIc@s`eZdZdZe��Z	dSdd�Zdd�Zdd�Z	d	d
�Z
dd�Zd
d�Zdd�Z
dd�Zdd�ZdTdd�Zedd��Zdd�Zdd�ZdTdd�ZdTd d!�Zd"d#�Zd$d%�ZdUd'd(�ZdUd)d*�Zd+d,�ZejdVd/d0��Zd1d2�ZdWd4d5�Z d6d7�Z!d8d9�Z"d:d;�Z#d<d=�Z$d>d?�Z%d@dA�Z&dBdC�Z'dDdE�Z(dFdG�Z)dHdI�Z*e+dJdK��Z,dLdM�Z-dNdO�Z.dPdQ�Z/dRS)X�GitProviderz�
    Base class for gitfs/git_pillar provider classes. Should never be used
    directly.

    self.provider should be set in the sub-class' __init__ function before
    invoking the parent class' __init__.
    �gitfsc
s||_||_dd�}t��dd�|_tjjj|j�|j�d�g�ddt	|d�|_
t�|�|_
|j
��D]\}	}
d|
vrLtjj�|j
|	d�|j
|	d<q3�fd	d
�|D�}|rat�dd�|��z|��Wntyxt|�t���Ynwt|t��r'tt|��|_|��tjjj||jddt	td�}|s�t�d
|j|j�t|j�|jdkr�|jdkr�d|vr�t�d|j|j�t|j�d}
�fdd�|D�D]G}d}
|tvr�|j t!vr�d�"|j||jd�t!�|j �}|jdkr�|d7}t�|�q�d�"|j||jd����}|jdk�r|d7}t�|�q�|
�r t|j�|j
�#|�n||_|��d|j
v�r9d|j
d<|jdk�rhd|j
v�rh|j�$dd�d |j
d<|j
d�%��&d!��rh|j
ddd"�|j
d<d|j
v�rtjj�|j
d��'d�|j
d<nd|j
d<d#|j
v�r�i|j
d#<n$|j
d#��D]\}	}
d|
v�r�|j
d#|	}tjj�|d�|d<�q�|j
��D]\}}|t(v�r�t)||��s�t*|||��q�t(D]}|d$k�r�t*|d%||j
|�|�+|��q�t)|d&��s#|j�d'�}zt,|}Wnt-�yt�d(|�t|j�Ynwt*|d&|�t�.d)||j|j�t/|d*�t)|d+��rXd,d
�|j0D�|_0d-d
�|j0D�}|�rXt�d.|j|jd�|�d�t1��t|j�t|jt	��smt�d/|j|j�t|j�t)|d��rx|j2|_3n t4t5|j�d0t6��}t	t7�8||j�9d1���:��d2d3��;dd%�|_3tjj<�||j3�|_=d%|_>|j�?d��r�z|�@�|_>WntA�y�t�d4|j|j�t|j�Ynwtjj<�|j3|j>�|_Btjj<�|j=|j>�|_Ctjj<�|d5|jB�|_Dtjj<�|d6|jB�|_EtFj<�G|jC��stF�H|jC�z|�I�|_JWn2tK�y?}z%d7�"|j|j|�}t|tL��r)|d87}tj|dd9�t|j�WYd}~nd}~ww|�M�|�N�tFj<�G|jD��sVtF�H|jD�|�O�tP�r�tF�Q�}tR�S|�}|j�d:d�}|j�d;d�}|�r�|�r�tjjT�UtV|�dSdSdSdS)<NcSst|�S�Nr;)r7�yrrr!�_val_cbsz%GitProvider.__init__.<locals>._val_cb�
machine_id�no_machine_id_availableZ_saltenvT)�strictZrecurseZkey_cbZval_cbrcsg|]}|�vr|�qSrrr5)�per_remote_onlyrr!r8�z(GitProvider.__init__.<locals>.<listcomp>ziThe following parameter names are restricted to per-remote use only: %s. This is a bug, please report it.�, z�Invalid per-remote configuration for %s remote '%s'. If no per-remote parameters are being specified, there may be a trailing colon after the URL, which should be removed. Check the master configuration file.�
git_pillar�__env__�basez{Invalid per-remote configuration for %s remote '%s'. base can only be specified if __env__ is specified as the branch name.Fc3s�|]	}|�vr|VqdSrLrr5)�valid_per_remote_paramsrr!�	<genexpr>Ls��z'GitProvider.__init__.<locals>.<genexpr>z�{0} authentication parameter '{1}' (from remote '{2}') is only supported by the following provider(s): {3}. Current {0}_provider is '{4}'.rKzLSee the GitFS Walkthrough in the Salt documentation for further information.zQInvalid {} configuration parameter '{}' in remote '{}'. Valid parameters are: {}.�M See the GitFS Walkthrough in the Salt documentation for further information.r��winrepor
�/�����.git����saltenvrr*r'Z	_refspecsz@The '%s' option has no default value in salt/config/__init__.py.ztThe 'refspecs' option was not explicitly defined as a configurable parameter. Falling back to %s for %s remote '%s'.�confr(cSr3r)�lowerr5rrr!r8�r9cSsg|]}|tvr|�qSr)�VALID_REF_TYPESr5rrr!r8�rSzXThe following ref_types for %s remote '%s' are invalid: %s. The supported values are: %szZInvalid %s remote '%s'. Remotes must be strings, you may need to enclose the URL in quotes�	hash_typer�ascii)�encodingz%__env__ cant generate basename: %s %s�work�linksz6Exception caught while initializing {} remote '{}': {}z Perhaps git is not available.��exc_info�cachedirZ
gitfs_remotes)W�optsrH�_get_machine_identifier�get�mach_id�salt�utils�dataZrepack_dictlistr-�global_saltenv�copy�deepcopyrc�items�urlZstrip_protorB�critical�joinr,r?r>�dict�next�iter�id�get_urlrGrI�branch�AUTH_PARAMS�provider�AUTH_PROVIDERS�format�update�rsplitrdr+r4�PER_SALTENV_PARAMS�hasattr�setattr�add_conf_overlay�_DEFAULT_MASTER_OPTS�KeyError�debug�delattrr(rer
�_cache_basehash�getattr�hashlibr�base64�	b64encode�encode�digest�replace�path�_cache_hash�_cache_basename�
startswith�get_checkout_target�AttributeError�_cache_full_basename�	_cachedir�_salt_working_dir�_linkdir�os�isdir�makedirs�init_remote�newrA�	GitPython�verify_auth�setup_callbacks�fetch_request_check�
HAS_PSUTIL�getpid�psutilZProcess�processZ"register_cleanup_finalize_function�gitfs_finalize_cleanup)�selfrn�remote�per_remote_defaultsrR�override_params�
cache_rootrHrNrb�saltenv_confZper_remote_collisionsZper_remote_confZper_remote_errors�param�msgZsaltenv_ptrr.rDZdefault_refspecsZinvalid_ref_typesrfrF�cur_pidr��	cache_dirZgitfs_activer)rRrXr!�__init__�s�
��������


�


���
���	�

�
�

���
���
�
�

���
�����
���
��zGitProvider.__init__cC�|jSrL)r��r�rrr!�get_cache_basehash�zGitProvider.get_cache_basehashcCr�rL)r�r�rrr!�get_cache_hashr�zGitProvider.get_cache_hashcCr�rL)r�r�rrr!�get_cache_basename"r�zGitProvider.get_cache_basenamecCr�rL)r�r�rrr!�get_cache_full_basename%r�z#GitProvider.get_cache_full_basenamecCr�rL)r�r�rrr!�get_cachedir(r�zGitProvider.get_cachedircCr�rL)r�r�rrr!�get_linkdir+r�zGitProvider.get_linkdircCr�rL)r�r�rrr!�get_salt_working_dir.r�z GitProvider.get_salt_working_dirc
s��fdd�}d�jv}d�jv}t�}tjj��j�r |�d�tjj�	|�D]=}|�
d�r4|dd�}|�d	d
�\}}|dkrY|rY|�d	�}	|	drO|	dn|	d
}|||�q'|dkrd|rd|||�q'|S)z�
        Return the names of remote refs (stripped of the remote name) and tags
        which are map to the branches and tags.
        csN|�jvr|��j|�dS|�jkr|�d�dS�js%|�|�dSdS)zC
            Add the appropriate saltenv(s) to the set
            rWN)�saltenv_revmapr�rW�addr$)Zenv_set�rnamer�rr!�
_check_ref7s

�z8GitProvider._get_envs_from_ref_paths.<locals>._check_refr��tagrWzrefs/�Nr]r^�remotes�r�tags)
r(�setrrrs�stringutils�is_hexrWr�rtrr�r@�	partition)
r��refsr�Zuse_branchesZuse_tagsr0rZrtyper�Zpartedrr�r!�_get_envs_from_ref_paths1s$





�z$GitProvider._get_envs_from_ref_pathsr�cCstjj�|j|d�S)N�.lk)rrrsr�r{r��r��	lock_typerrr!�_get_lock_fileXszGitProvider._get_lock_filecsd�fdd�	}t|�|�dS)zV
        Programmatically determine config value based on the desired saltenv
        rWc	s(�fdd�}�jdkr|t�d���S�j�|i���dkrl���fdd�}||�}z�j}|rB||krBt�d||�j�j||�|WStyMYnw|d	krU�j	S�j
rh|durft�d
�j�j|�|S|pk|S��vrv|���S|�jvr���j|vr�|�j|��S|t�d���S)Ncs�dvr
|�tj�S|S)N)rr)�rstripr��sep)r7rrr!�	strip_sepbszAGitProvider.add_conf_overlay.<locals>._getconf.<locals>.strip_seprKr*rcs:��vr��S|�jvr��j|vr�j|�SdSrL)ru)�tgt_env)r
r�r�rr!�_get_per_saltenvks
zHGitProvider.add_conf_overlay.<locals>._getconf.<locals>._get_per_saltenvz�The per-saltenv configuration has mapped the '%s' branch/tag to saltenv '%s' for %s remote '%s', but this remote has all_saltenvs set to '%s'. The per-saltenv mapping will be ignored in favor of '%s'.rWzXsaltenv mapping is disabled for %s remote '%s' and saltenv '%s' is not explicitly mapped)rHr�rbrp�all_saltenvsrBr�rr�rWr$ru)r�r�r�r�Zper_saltenv_refZall_saltenvs_refr)r�r�r!�_getconfasP
�
��z.GitProvider.add_conf_overlay.<locals>._getconfN�rW)r�)�clsr
r�rrr!r�[sDzGitProvider.add_conf_overlaycCsJtjj�|j|����tj�}tj�	|�r|St
�d|��|j|j
�dS)z�
        Check if the relative root path exists in the checked-out copy of the
        remote. Return the full path to that relative root if it does exist,
        otherwise return None.
        z7Root path '%s' not present in %s remote '%s', skipping.N)rrrsr�r{r�rr�r�r�r�rBrCrHr)r��root_dirrrr!�
check_root�s	�zGitProvider.check_rootcCs�g}d}tj��}tjj��sd|d<d|d<tjt	�
|�tjj��tj�|j
�|tjtjd�}|��d}|�t�}|jdkrNt�d|j|j||�|Sd}tjj�
|d	�D]}|�|�rl|�|t|�d
����qX|rzt�d|jd�|��|S)
zV
        Remove stale refs so that they are no longer seen as fileserver envs
        zgit remote prune origin�CsLANGUAGEsLC_ALL)�	close_fds�cwd�env�stdout�stderrrzOFailed to prune stale branches for %s remote '%s'. Output from '%s' follows:
%sz * [pruned] �
Nz&%s pruned the following stale refs: %srT)r��environrvrrrs�platform�
is_windows�
subprocess�Popen�shlexr@r��dirname�gitdir�PIPE�STDOUT�communicater�__salt_system_encoding__�
returncoderB�warningrHr�	itertoolsr��append�lenr4r�r{)r��cleanedZcmd_strr��cmd�outputZmarker�linerrr!�clean_stale_refs�sJ
�

��
��zGitProvider.clean_stale_refscCsN|jjjdd�durt�d�td��z|�|�W|jj��S|jj��w)z!
        Clear update.lk
        �<��timeoutF�gitfs master lock timeout!)�	__class__�_master_lock�acquirerBrC�TimeoutError�_clear_lock�releaser�rrr!�
clear_lock�s

zGitProvider.clear_lockc
sX�j|d����fdd�}g}g}zt���Wnsty�}zg|jtjkr9d�j�d��d|�d�}t�|�n3|jtj	kr^zt
���Wn.ty]}z|||�WYd}~n%d}~ww|||�WYd}~||fSWYd}~||fSWYd}~||fSWYd}~||fSd}~wwd	|�d
�j�d�j
�d�j�d
�	}t�|�|�|�||fS)z?
        Clear update.lk without MultiProcessing locks
        �r�cs(d��j�|�}t�|�|�|�dS)Nz-Unable to remove update lock for {} ({}): {} )r�ryrBr�r�)ZerrlistrFr���	lock_filer�rr!�
_add_error�s
�
z+GitProvider._clear_lock.<locals>._add_errorzAttempt to remove lock z for file (z$) which does not exist, exception : � NzRemoved �
 lock for �	 remote '�' on machine_id '�')r�r��remove�OSError�errno�ENOENTryrBr��EISDIR�shutil�rmtreerHrrqr�)r�r�r�success�failedrFr�rrr!r�sJ����
�
�
�
����

zGitProvider._clear_lockcCsNtj�|jd�}tjj��}|�|�st	�
d|�dSd}d}z|�|d�}Wntjjjy<|�
|�d}d}Ynwt	�d|j|j||j�||jkrc|�|d|j�t	�d	|j|j|j�d}zt|j|d
dd��}Wntjjjy}g}Ynwt|j�}t	�d|j|j||�||kr�|�|d
|�t	�d
|j|j|�d}z|�dd�}Wn tjjjy�|�
d�d}Yntjjjy�d}Ynwt|j���}	t	�d|j|j||	�||	kr�|�dd|	�t	�d|j|j|	�d}|�r%tjj�|d��}
|�|
�t	�d|j|j|�Wd�dS1�swYdSdS)z�
        For the config options which need to be maintained in the git config,
        ensure that the git config file is configured as desired.
        �configz&Failed to read from git config file %sFzremote "origin"ryTNz6Current fetch URL for %s remote '%s': %s (desired: %s)z&Fetch URL for %s remote '%s' set to %s�fetch)Zas_listz5Current refspecs for %s remote '%s': %s (desired: %s)z%Refspecs for %s remote '%s' set to %s�httpZ	sslVerifyz;Current http.sslVerify for %s remote '%s': %s (desired: %s)z+http.sslVerify for %s remote '%s' set to %s�wz/Config updates for %s remote '%s' written to %s)r�r�r{r�rrrsZconfigparserZGitConfigParser�readrBrCrpZNoSectionErrorZadd_sectionr�rHrryr��sortedZ
NoOptionErrorr'Zset_multivarr-r#rd�files�fopen�write)r��
git_configrcZconf_changedZremote_sectionryr'Zdesired_refspecsr#Zdesired_ssl_verify�fp_rrr!�enforce_git_config&s�

��
��
��
���
�$��zGitProvider.enforce_git_configcCs�z'|jdd��t�d|j|j�|��Wd�WS1s wYWdStyd}z1|jtjkrFt�	d|j|j|j
dd�|j�nt�	d|j|j|j
dd�|jt|��WYd}~dSd}~wty|}z
t�	d|�WYd}~dSd}~ww)	z�
        Fetch the repo. If the local copy was updated, return True. If the
        local copy was already up-to-date, return False.

        This function requires that a _fetch() function be implemented in a
        sub-class.
        r�rzFetching %s remote '%s'Na1Update lock file is present for %s remote '%s', skipping. If this warning persists, it is possible that the update process was interrupted, but the lock could also have been manually set. Removing %s or running 'salt-run cache.clear_git_lock %s type=update' will allow updates to continue for this remote.��Update lock file generated an unexpected exception for %s remote '%s', The lock file %s for %s type=update operation, exception: %s .Fz*fetch got NotImplementedError exception %s)
�gen_lockrBr�rHr�_fetchrr�EEXISTr�r�r-�NotImplementedError)r�rFrrr!r�s8(�
�

�	���zGitProvider.fetchFcCsP|jjjdd�durt�d�td��z
|�||�W|jj��S|jj��w)zO
        Place a lock file if (and only if) it does not already exist.
        rrFr)rrrrBrCr�_GitProvider__lockr	)r�r�rIrrr!�_lock�s
zGitProvider._lockc
Cs�z=t�|�|�tjtjBtjB�}t�|d��t�|tj	j
�t���d|j
�d���Wd�n1s6wYW�nxt�y�}�zj|jtjk�r�tj	j�|�|�d��E}zttj	j
�|������}Wntytd}Ynwz
tj	j
�|�����}Wnty�}zd}WYd}~nd}~wwWd�n1s�wY|jd}|j|d�}	|j|�r<|�d|�d	|	�d
|j�d|j�d|j
�d
|�d�}
|�r,|
d|�d�7}
|j
s�|r�|
d|�d|j
��7}
tj	j�|��s,|j
|k�r|
d|��7}
n*|
d7}
t�|
�|��\}}|�r!|j d|d�WYd}~S|�r%�WYd}~dSt�|
�|�r5�WYd}~dS|�r^tj	j�|��r^t�d||j||	|j
�|�rW�WYd}~dS|�rmt�d||j||	|j
�|��\}}|�r�|j d|d�WYd}~S|�r��WYd}~dSd|�d|j�d|�|��d|j
�d|��
}
tj!|
dd�t"|j|
��d}~wwd |�d|j�d|j�d!|j
�d"�	}
t�#|
�|
S)#zv
        Place a lock file if (and only if) it does not already exist.
        Without MultiProcessing locks.
        �wbr�N�rrZ_global_lockrz is enabled and z
 lockfile z is present for rz' on machine_id z with pid 'z'.z	 Process z obtained the lockz for machine_id z, current machine_id z� but this process is not running. The update may have been interrupted. If using multi-master with shared gitfs cache, the lock may have been obtained by another master, with machine_id z� but this process is not running. The update may have been interrupted.  Given this process is for the same machine the lock will be reallocated to new process r��r�rIz1Process %d has a %s %s lock (%s) on machine_id %szjProcess %d has a %s %s lock (%s) on machine_id %s, but this process is not running. Cleaning up lock file.zUnable to set rz (z) on machine_id �: TrkzSet rr)$r��openr��O_CREAT�O_EXCL�O_WRONLY�fdopenr%rrrsr��to_bytesr�rqrrr,r#r$r=�
to_unicode�readliner��
ValueErrorrHrnrr�Z
os_is_runningrBr�rr.rCrr�)
r�r�rIZfh_rF�fd_�pidrqZglobal_lock_keyr
r�rZfailrrr!Z__lock�s�������������
�����
���
�
��	������$d
zGitProvider.__lockcCs�g}g}z|jdd�}Wn.ty:}z"t�d|j|j|jdd�|jt|��|�|j	�WYd}~||fSd}~ww|durD|�|�||fS)aG
        Place an lock file and report on the success/failure. This is an
        interface to be used by the fileserver runner, so it is hard-coded to
        perform an update lock. We aren't using the gen_lock()
        contextmanager here because the lock is meant to stay and not be
        automatically removed.
        r�rr)N)
r/rrBr�rHrr�r-r��strerror)r�rr�resultrFrrr!�lock>s(
�	��
zGitProvider.lockr��?c	cs��t|t�sttjd|�d���zt|�}Wnty!d}Ynw|dkr(d}t|tftf�r4|dkr6d}||kr<|}d}d}zst��}	z|j	|dd�d}|Vd}Wn5t
tfy�}z'|rjt��||krqt|j|j��t�
d||j|j|�t�|�WYd	}~qEd	}~wwW|s�|r�d
|�d|j�d|j�d
|�d|�d�}t�
|�|j|d�d	Sd	S|s�|r�d
|�d|j�d|j�d
|�d|�d�}t�
|�|j|d�ww)z�
        Set and automatically clear a lock,
        should be called from a context, for example: with self.gen_lock()
        zInvalid lock_type 'rrrBFTr2zFA %s lock is already present for %s remote '%s', sleeping %f second(s)NzAttempting to remove 'z' lock for 'z
' remote 'z' due to lock_set1 'z' or lock_set2 'r)r>r-rr�EINVALr=r<�float�timer/rr?rBr�rHr�sleepr
)	r�r�rZ
poll_intervalZ	lock_set1Z	lock_set2�
time_startrFr�rrr!r*Zs��
��
��������
������
�zGitProvider.gen_lockcC�t���zA
        This function must be overridden in a sub-class
        �r-r�rrr!r���zGitProvider.init_remoteTcCrHrIrJ)r��
fetch_on_failrrr!�checkout�rKzGitProvider.checkoutcCrHrIrJ�r�r�rrr!�dir_list�rKzGitProvider.dir_listcCstjjj||j|jd�S)zo
        Check if an environment is exposed by comparing it against a whitelist
        and blacklist.
        )Z	whitelistZ	blacklist)rrrsr�Zcheck_whitelist_blacklistr%r&rNrrr!�env_is_exposed�s
�zGitProvider.env_is_exposedcCrH)zb
        Provider-specific code for fetching, must be implemented in a
        sub-class.
        rJr�rrr!r+�szGitProvider._fetchcCrHrIrJr�rrr!�envs�rKzGitProvider.envscCrHrIrJrNrrr!�	file_list�rKzGitProvider.file_listcCrHrIrJ)r�r�r�rrr!�	find_file�rKzGitProvider.find_filecCsh|jdkr1|jdkr1z|jWStyYnw|j�d�p%|j�d�p%d}|dkr-|jSt|�S|jS)z0
        Resolve dynamically-set branch
        rUrV�	pillarenvrbrW)rHr�r�r�rnrprWr-)r��targetrrr!r��s�zGitProvider.get_checkout_targetc	Cs�|�|�sdS|�|�}|durdS|jD]-}zd|��}t||�}Wnty5t�d|jj|�Yqw||�}|durB|Sq|j	rx|jD].}zd|��}t||�}Wntyit�d|jj|�YqIw||j	�}|durw|SqIdS)zD
        Return a tree object for the specified environment
        NZget_tree_from_z!%s class is missing function '%s')
rPrr(r�r�rBrCr�__name__�fallback)r�r��tgt_refZref_typeZ	func_name�func�	candidaterrr!�get_tree�sJ



���

��
�zGitProvider.get_treecCs\|jdvr(z|j�dd�\|_|_WdSty'|jd|_|j|_YdSw|j|_dS)zW
        Examine self.id and assign self.url (and self.branch, for git_pillar)
        )rUr\Nr^r�)rHrr@r�ryr<rcr�rrr!r�
s
�zGitProvider.get_urlc
Cs�tjj�|jd�}tj�|�r@t�d|j�zt�	|�Wnt
y9}ztjd|j|dd�WYd}~nd}~ww|��dSdS)N�
fetch_requestzFetch request: %sz%Failed to remove Fetch request: %s %sTrkF)
rrrsr�r{r�r��isfilerBr�rrrCr)r�r\rFrrr!r�s"���zGitProvider.fetch_request_checkcCsz|jWSty�g|_z|j�d�}Wnty't�d|jj�YnVwt|dd��D]2\}}z	||dg}Wnt	yHg}Ynw|j�
tjj
j|jg|d|d��R�|gf�q0z|j�d|j|dggf�Wn	t	y|Ynw|jYSw)zq
        Return the expected result of an os.walk on the linkdir, based on the
        mountpoint value.
        r]z-%s class is missing a '_mountpoint' attributeNr_r^r)Z
_linkdir_walkr��_mountpointr@rBrCrrV�	enumerate�
IndexErrorr�rrrsr�r{r��insert)r��parts�idxr/�dirsrrr!�linkdir_walk.s>���"�� �
�zGitProvider.linkdir_walkcC�dS)zX
        Only needed in pygit2, included in the base class for simplicty of use
        Nrr�rrr!r�TszGitProvider.setup_callbackscCs
d|_dS)zS
        Override this function in a sub-class to implement auth checking.
        NT��credentialsr�rrr!r�YszGitProvider.verify_authcCrHrIrJ)r��blob�destrrr!�
write_file`rKzGitProvider.write_fileN�rK)r�)r�F)r�rrB�T)0rV�
__module__�__qualname__�__doc__�multiprocessing�Lockrr�r�r�r�r�r�r�r�r�r��classmethodr�r�r�r
rr(rr/r.rA�
contextlib�contextmanagerr*r�rMrOrPr+rQrRrSr�r[r�r��propertyrer�r�rkrrrr!rJ�s\


�)
'
K
0

.j
)

xA
-
%rJcs~eZdZdZ	d�fdd�	Zddd�Zdd	�Zd
d�Zdd
�Zdd�Z	dd�Z
dd�Zdd�Zdd�Z
dd�Zdd�Z�ZS)r�z 
    Interface to GitPython
    rKc	�"d|_t��|||||||�dS�N�	gitpython�r��superr��r�rnr�r�rRr�r�rH�rrr!r�l�
�zGitPython.__init__Tc	Cs�|��|��}z	|j�d�j}Wntyd}Ynwd|dfd|dfg}|jr<|d|jdfd|jdfg7}|D]�\}}z	|j�|�j}Wn	tyTYq>w||kr_|��Sz.|jdd��|jj	�
|�t�d	|j
|j||rzd
nd�Wd�n1s�wYWn<ty�}z(|jtjkr�t|jd|j
�d
|j�d���t�d|j|j
|j�WYd}~dSd}~wty�Yq>w|��S|r�t�d||j
|j�|��|j
dd�St�d||j
|j�dS)af
        Checkout the configured branch/tag. We catch an "Exception" class here
        instead of a specific exception class because the exceptions raised by
        GitPython when running these functions vary in different versions of
        GitPython.

        fetch_on_fail
          If checkout fails perform a fetch then try to checkout again.
        �HEADNzorigin/Fztags/TrMrz+%s remote '%s' has been checked out to %s%sz as fallbackr[�Checkout lock exists for rr�?Error %d encountered obtaining checkout lock for %s remote '%s'�>Failed to checkout %s from %s remote '%s': fetch and try again�rL�DFailed to checkout %s from %s remote '%s': remote ref does not exist)r�r��repo�	rev_parse�hexsharArWr�r*rrMrBr�rHrrrr,rCr)	r�rLrX�head_shaZ
checkout_refs�checkout_refrW�
target_sharFrrr!rM�s�
�

���
���	������zGitPython.checkoutc	Cs�d}t�|j�stj�|j�|_d}n"z	t�|j�|_Wntjjy4t	�
t|j|j|j
�|YSwtjj�|jjd�|_|��|S)z�
        Initialize/attach to a remote using GitPython. Return a boolean
        which will let the calling function know whether or not a new repo was
        initialized by this function.
        FTr`)r��listdirr�rZRepo�initr�rFZInvalidGitRepositoryErrorrBrC�
_INVALID_REPOryrHrrrsr�r{Zworking_dirr�r()r�r�rrr!r��s�zGitPython.init_remotecs�t�}����}|s|S����r.z	|����}Wnty%|YSw��fdd�}ndd�}��fdd�}|��D]}t|tj�rO|�|||j	���q=��
��r]|���
���|S)zT
        Get list of directories for the target environment using GitPython
        c�tj�|�����SrL�r�r��relpathr�r�rNrr!r��z#GitPython.dir_list.<locals>.relpathcS�|SrLrr�rrr!r��c�tjjj����|dd�S�NT�Z
use_posixpath�rrrsr�r{rr�rNrr!�add_mountpoint��z*GitPython.dir_list.<locals>.add_mountpoint)r�r[rr��traverser>r�Treer�r�r)r�r�r0�treer�r�rirrNr!rO�s(

��
zGitPython.dir_listcCsdd�|jjD�}|�|�S)�n
        Check the refs and return a list of the ones which can be used as salt
        environments.
        cSsg|]}|j�qSrr�r5rrr!r8sz"GitPython.envs.<locals>.<listcomp>)r�r�r��r�Z	ref_pathsrrr!rQs
zGitPython.envscCs�|jjd}z|��}Wn
ty|��}Ynwd}|D]B}|jdurAt�d|j|j|j	|jj
dd�|jj
dd��d}q|j|j
|jfvr`t�d|j|j|j
krWdnd	|j|j	�d}q|��}|si|rkdSdS)
��
        Fetch the repo. If the local copy was updated, return True. If the
        local copy was already up-to-date, return False.
        rFNz1%s has updated '%s' for remote '%s' from %s to %s�Tz*%s has fetched new %s '%s' for remote '%s'r��head)r�r�r�AssertionErrorZ
old_commitrBr�rHr
rr��commit�flagsZNEW_TAGZNEW_HEADr�)r��origin�
fetch_resultsZnew_objsZ	fetchinfor�rrr!r+s<�
���zGitPython._fetchcst�}i}����}|s||fS����r4z	|����}Wn
ty+||fYSw��fdd�}ndd�}��fdd�}|��D]:}t|tj�sLqC|||j��}|�	|�t
�|j�r}t
��}	|�|	�|	�d�tjj�|	���}
|	��|
||<qC||fS)zJ
        Get file list for the target environment using GitPython
        cr�rLr�r�rNrr!r�Rr�z$GitPython.file_list.<locals>.relpathcSr�rLrr�rrr!r�Wr�cr�r�r�r�rNrr!r�Zr�z+GitPython.file_list.<locals>.add_mountpointr)r�r[rr�r�r>r�Blobr�r��stat�S_ISLNK�mode�io�BytesIO�stream_data�seekrrrsr��to_strr!�close)r�r�r#�symlinksr�r�r��	file_blobZ	file_path�stream�link_tgtrrNr!rRBs8

�


�zGitPython.file_listc	Cs�|�|�}|s	dSd}d}	|d7}|tkrd}nOzB||}t�|j�rNt��}|�|�|�d�t	j
j�|�
��}|��t	j
jjtj�|�|dd�}n|}t|tj�rXd}WnWntyfd}Ynwqt|tj�ru||j|jfSdS�zF
        Find the specified file in the specified environment
        )NNNNrTr^r�)r[�SYMLINK_RECURSE_DEPTHr�r�r�r�r�r�r�rrrsr�r�r!r�r�r{r�r�r>rr�r�r�r�)	r�r�r�r�ri�depthr�r�r�rrr!rSmsD


��	��zGitPython.find_filecC�0z
t�|jd|���jjWStyYdSw)zh
        Return a git.Tree object matching a head ref fetched into
        refs/remotes/origin/
        �refs/remotes/origin/N)rZRemoteReferencer�r�r�r<�r�rrrr!�get_tree_from_branch�s���zGitPython.get_tree_from_branchcCr�)zU
        Return a git.Tree object matching a tag ref fetched into refs/tags/
        �
refs/tags/N)rZTagReferencer�r�r�r<r�rrr!�get_tree_from_tag�s
�zGitPython.get_tree_from_tagc	Cs.z|j�|�jWStjjtfyYdSw)z9
        Return a git.Tree object matching a SHA
        N)r�r�r��gitdbrFZODBErrorr�r�rrr!�get_tree_from_sha��
�zGitPython.get_tree_from_shacCs@tjj�|d��}|�|�Wd�dS1swYdS�zO
        Using the blob object, write the file to the destination path
        �wb+N)rrrsr#r$r��r�rirjr'rrr!rk�s"�zGitPython.write_filerlrm)rVrnrorpr�rMr�rOrQr+rRrSr�r�r�rk�
__classcell__rrr}r!r�gs�
Y"&+,		r�cs�eZdZdZ	d%�fdd�	Zdd�Zd&dd	�Zd'�fdd�	Zd
d�Zdd�Z	dd�Z
dd�Zdd�Zdd�Z
dd�Zdd�Zdd�Zdd �Zd!d"�Zd#d$�Z�ZS)(�Pygit2z
    Interface to Pygit2
    rKc	rw�Nrrzr|r}rr!r��r~zPygit2.__init__cCs&z|��WSty|��YSw)aY
        Compatibility function for pygit2.Reference objects. Older versions of
        pygit2 use .get_object() to return the object to which the reference
        points, while newer versions use .peel(). In pygit2 0.27.4,
        .get_object() was removed. This function will try .peel() first and
        fall back to .get_object().
        )�peelr�Z
get_object)r��objrrr!r��s

�zPygit2.peelTc
s�������}d|}d|}d|}z�j�d�}Wnty/t�d�j�j�YdSwz
t	��
|�j�}WntyMt�d�j�j�YdStyWd}Ynw�j�
�}d��fd	d
�	}	�z||vr�||vr��jr��j}d|}d|}d|}||v�r��
�j�|��j�||vr��j�|��zt	��
�j�|��j�}
Wnty�tjd|�j�jdd�YWdSw||
k�r|j}t|t	��r||v�r||k�r|�d�d
}|s�t�d|�j�j�WdSd|}
|
|vr�|}
�j�|�j�|
�j�|	|dd��sWdS���WS||v�r��j�|�}t|tjtjf��s3t�d|t|��nOzt	|jj�}Wn+t�yfzt	|j�}Wnt�yctjd|�j�jdd�YYWdSwYnwt�d||�||k�r}|	|dd��s}WdS���WSWn)t�y��t�y�}ztjd|�j�j|dd�WYd}~dSd}~ww|�r�t�d|�j�j�����jdd�St�d|�j�j�dS)z�
        Checkout the configured branch/tag

        fetch_on_fail
          If checkout fails perform a fetch then try to checkout again.
        zrefs/heads/r�r�rz"HEAD not present in %s remote '%s'Nz,Unable to get SHA of HEAD for %s remote '%s'Tc
s�z4�jdd��#�j�|�|r!�j��tj�Wd�WdSWd�WdS1s-wYWdStyg}z'|jtjkrQt|jd�j	�d�j
�d���t�d|j�j	�j
�WYd}~d	Sd}~ww)
zP
            DRY function for checking out either a branch or a tag
            rMrNTr�rrr�F)
r*r�rM�resetrZGIT_RESET_HARDrrr,rHrrBrC)r�r�rF��	pygit2_idr�rr!�_perform_checkouts6
��
������z*Pygit2.checkout.<locals>._perform_checkoutz5pygit2 was unable to get SHA for %s in %s remote '%s'rkr_zMpygit2 was unable to resolve branch name from HEAD ref '%s' in %s remote '%s')r�zG%s does not correspond to pygit2 Commit or Tag object. It is of type %szTUnable to resolve %s from %s remote '%s' to either an annotated or non-annotated tagzSHA of tag %s: %sFz-Failed to checkout %s from %s remote '%s': %sr�r�r�rm)r�r�r��lookup_referencer�rBr�rHrr-r�r�rC�listall_referencesrWZcreate_referencerUr>r�r��revparse_singlerZCommitZTag�typer�rrArrM)r�rLrXZ	local_refZ
remote_refZtag_refZ
local_headr�r�r�r�Zhead_refZbranch_nameZremote_headZtag_objZtag_sharFrr�r!rM�s�
��

���


�

��

��
���

����
��zPygit2.checkoutNcsLztjrgWSWn	tyYnw|jdur!t�dt|j�gSt��	�S)zX
        Clean stale local refs so they don't appear as fileserver environments
        Nz�The installed version of pygit2 (%s) does not support detecting stale refs for authenticated remotes, saltenvs will not reflect branches/tags removed from remote '%s')
r�GIT_FETCH_PRUNEr�rhrBr��PYGIT2_VERSIONrr{r�)r��
local_refsr}rr!r��s��
�
zPygit2.clean_stale_refsc	Cs�tj�d�}|tjjtj<d}t�|j�st�	|j�|_
d}n z	t�|j�|_
Wnty>t
�t|j|j|j�|YSwtjj�|j
jd�|_|��tj�|jd�}tj�|�rittd�kri|j
j�|�|S)z�
        Initialize/attach to a remote using pygit2. Return a boolean which
        will let the calling function know whether or not a new repo was
        initialized by this function.
        �~FTr`rz0.28.0)r�r��
expanduserrZsettingsZsearch_pathZGIT_CONFIG_LEVEL_GLOBALr�r�Zinit_repositoryr�Z
Repositoryr�rBrCr�ryrHrrrsr{Zworkdirr�r(�existsr�rrZadd_file)r��homer�r&rrr!r��s$�zPygit2.init_remotec	s���fdd��t�}����}|s|S����rCz|����j}�j|}Wnty2|YSwt|tj�s;|S��fdd�}ndd�}g}|rT�||�������fdd�}|D]}|�	|||���q]��
��rv|�	��
���|S)zS
        Get a list of directories for the target environment using pygit2
        c	sxt|�D]5}|j�jvr
q�j|j}t|tj�sq|�tjj	j
||jdd��|r9�||tjj	j
||jdd��qdS)z�
            Traverse through a pygit2 Tree object recursively, accumulating all
            the empty directories within it in the "blobs" list
            Tr�N)r~rr�r>rr�r�rrrsr�r{r
)r��blobs�prefix�entryri��	_traverser�rr!r��s"����z"Pygit2.dir_list.<locals>._traversecr�rLr�r�rNrr!r�r�z Pygit2.dir_list.<locals>.relpathcSr�rLrr�rrr!r�"r�cr�r�r�r�rNrr!r�)r�z'Pygit2.dir_list.<locals>.add_mountpoint)r�r[rrr�r�r>rr�r�r)	r�r�r0r�r�r�r�r�rir�r�r�r�r!rO�s2

�
zPygit2.dir_listcCs|j��}|�|�S)r�)r�r�r�r�rrr!rQ4s

zPygit2.envsc

Cs�|jjd}|j��}i}|jdur|j|d<n	|jdur!|j|_ztj|d<Wn	ty1Ynwz
|jdi|��}WnLt	y�}z@t
|���}d|vrat|jtj
�ratjd|j|jdd�nd	|vrqtjd
|j|jdd�ntjd|j|j|dd�WYd}~dSd}~wwz|d
}Wnttfy�|j}Ynw|dkr�t�d|j||j�n	t�d|j|j�|j��}|j|d�}	|s�||ks�|	r�dSdS)r�rNZ	callbacksZprunezunsupported url protocolz�Unable to fetch SSH-based %s remote '%s'. You may need to add ssh:// to the repo string or libgit2 must be compiled with libssh2 to support SSH authentication.Trkz+authentication required but no callback setzH%s remote '%s' requires authentication, but no authentication configuredz*Error occurred fetching %s remote '%s': %sF�received_objectsz&%s received %s objects for remote '%s'z%s remote '%s' is up-to-date)r�r)r�r�r��remotecallbacksrhrr�r�r�GitErrorrrdr>�KeypairrBrCrHrr,r�r�r�)
r�r�Zrefs_preZfetch_kwargsr�rFZexc_strr�Z	refs_postr�rrr!r+<st


�
��	����
��
z
Pygit2._fetchcs��fdd��t�}i}����}|s||fS����rKz|����j}�j|}Wn
ty8||fYSwt|tj�sC||fS��fdd�}ndd�}i}|r\�||�������fdd�}|�	dg�D]}	|�
|||	���qi|�	d	i���D]\}	}
|
||||	��<q}||fS)
zG
        Get file list for the target environment using pygit2
        c	s�t|�D]Z}|j�jvr
q�j|j}t|tj�rItjjj	||j
dd�}|�dg��|�t
�||j
j�rH�j||j
jj}||�di�|<qt|tj�r^�||tjjj	||j
dd��qdS)z�
            Traverse through a pygit2 Tree object recursively, accumulating all
            the file paths and symlink info in the "blobs" dict
            Tr�r#r�N)r~rr�r>rr�rrrsr�r{r
�
setdefaultr�r�r��filemodertr�)r�r�r�r�r��	repo_pathr�r�rr!r��s*�����z#Pygit2.file_list.<locals>._traversecr�rLr�r�rNrr!r��r�z!Pygit2.file_list.<locals>.relpathcSr�rLrr�rrr!r��r�cr�r�r�r�rNrr!r��r�z(Pygit2.file_list.<locals>.add_mountpointr#r�)r�r[rrr�r�r>rr�rpr�rx)r�r�r#r�r�r�r�r�r�r�r�rr�r!rR�s4

�zPygit2.file_listc	Cs�|�|�}|s	dSd}d}d}	|d7}|tkrd}nAz4||}|j}t�|�r>|j|jj}tj	j
jtj
�
|�|dd�}n|j|j}t|tj�rLd}WnWntyZd}Ynwqt|tj�rj|t|j�|fSdSr�)r[r�r�r�r�r�rrtrrrsr�r{r�r�r>rr�r�r�r-)	r�r�r�r�rir�r�r�r�rrr!rS�s@

��	��zPygit2.find_filecC�2z|�|j�d|����jWStyYdSw)zk
        Return a pygit2.Tree object matching a head ref fetched into
        refs/remotes/origin/
        r�N�r�r�r�r�r�r�rrr!r��s���zPygit2.get_tree_from_branchcCr�)zX
        Return a pygit2.Tree object matching a tag ref fetched into refs/tags/
        r�Nr�r�rrr!r�	s
�zPygit2.get_tree_from_tagcCs.z|j�|�jWSttttfyYdSw)z<
        Return a pygit2.Tree object matching a SHA
        N)r�r�r�r�r,r<r�r�rrr!r�	r�zPygit2.get_tree_from_shacCs`ttd�krtj|jd�|_|jsdd�}||j_dSdSd|_|js.t�	dt�d��dSdS)z8
        Assign attributes for pygit2 callbacks
        z0.23.2rgc_rf)NTr)�args�kwargsrrr!�_certificate_check	r�z2Pygit2.setup_callbacks.<locals>._certificate_checkNzdpygit2 does not support disabling the SSL certificate check in versions prior to 0.23.2 (installed: z2). Fetches for self-signed certificates will fail.)
r�rrZRemoteCallbacksrhr�r#Zcertificate_check�warnings�warn)r�r�rrr!r�	s����zPygit2.setup_callbacksc	s:d�_tj��j�rdSt�fdd�tD��sdS�fdd�}�fdd�}�j�d	�\}}}|s5d
}�j}|��}|dvr?dSd
|vr�d}|�	d
�d}||kr^t
�d�j�j
�t�j�|�_t�fdd�|D��r��fdd�dD�}dD](\}	}
||	}|dur�z
tj�|�s�||
|�Wqwty�||
|�Yqwwqwtj|��_dS�fdd�|D�}||�dSd|v�rd}t�fdd�|D��}
t�fdd�|D��}|r�dS|
r�|dkr�js�t
�d�j�j
�j�t�j�t��j�j��_dS�fdd�|D�}||�dSt
�d�j�j
|�t�j�dS)a�
        Check the username and password/keypair info for validity. If valid,
        set a 'credentials' attribute consisting of the appropriate Pygit2
        credentials object. Return False if a required auth param is not
        present. Return True if the required auth parameters are present (or
        auth is not configured), otherwise failhard if there is a problem with
        authenticaion.
        NTc3s�|]	}t�|d�VqdSrL�r�r5r�rr!rY9	s�z%Pygit2.verify_auth.<locals>.<genexpr>cs(t�d�j�jd�|��t�j�dS)zU
            Helper function to log errors about missing auth parameters
            zPIncomplete authentication information for %s remote '%s'. Missing parameters: %srTN)rBrzrHrr{rI)�missingr�rr!�_incomplete_auth=	s�z,Pygit2.verify_auth.<locals>._incomplete_authcs$t�d||�j�j�t�j�dS)zN
            Helper function to log errors about missing key file
            aSSH %s (%s) for %s remote '%s' could not be found, path may be incorrect. Note that it may be necessary to clear git_pillar locks to proceed once this is resolved and the master has been started back up. A warning will be logged if this is the case, with instructions.N)rBrzrHrrI)�key_typer�r�rr!�_key_does_not_existJ	s�z/Pygit2.verify_auth.<locals>._key_does_not_existz://Zssh)r�file)rr�@rzJKeypair specified for %s remote '%s', but remote URL is missing a usernamec3� �|]}tt�|d��VqdSrL�r<r�r5r�rr!rYu	��csg|]}t�|d��qSrLr�r5r�rr!r8v	s
��z&Pygit2.verify_auth.<locals>.<listcomp>)rrrr))r^r)r�rc� g|]}tt�|d��s|�qSrLr�r5r�rr!r8�	�
��r)rrc3r�rLr�r5r�rr!rY�	r�c3r�rLr�r5r�rr!rY�	s�
�aInvalid configuration for %s remote '%s'. Authentication is disabled by default on http remotes. Either set %s_insecure_auth to True in the master configuration file, set a per-remote config option named 'insecure_auth' to True, or use https or ssh-based authentication.cr�rLr�r5r�rr!r8�	r�zEInvalid configuration for %s remote '%s'. Unsupported transport '%s'.)rhr�r��isabsry�anyr�r�rdr@rBrzrHrrIr�allr]r,rr�rZUserPassr)r�r�r�Z	transportr*�addressZrequired_paramsrZkeypair_paramsrcr�Zkey_pathZmissing_authZpassword_okZno_password_authrr�r!r�+	s�	
�

�
���
�

��

��zPygit2.verify_authcCsBtjj�|d��}|�|j�Wd�dS1swYdSr�)rrrsr#r$r%rtr�rrr!rk�	s"�zPygit2.write_filerlrmrL)rVrnrorpr�r�rMr�r�rOrQr+rRrSr�r�r�r�r�rkr�rrr}r!r��s*�

[?LF)			r�)rryc@s�eZdZdZddeedddfdd�Zdeefdd�Zd	d
�Zdd�Z	d
d�Z
dd�Zd'dd�Zd(dd�Z
d(dd�Zd(dd�Zdd�Zdd�Zd)dd �Zd)d!d"�Zd#d$�Zd*d%d&�ZdS)+�GitBasez)
    Base class for gitfs/git_pillar
    NrTc		Cs�||_|dur	|nt|_|��|dur||_|_ntjj�	|jd|j
�|_tjj�	|jd�|_tjj�	|jd�|_tjj�	|jd�|_tjj�	|jdd|j
�|_
tjj�|j�|rn|�|durf|ng|||�dSdS)a
        IMPORTANT: If specifying a cache_root, understand that this is also
        where the remotes will be cloned. A non-default cache_root is only
        really designed right now for winrepo, as its repos need to be checked
        out into the winrepo locations and not within the cachedir.

        As of the 2018.3 release cycle, the classes used to interface with
        Pygit2 and GitPython can be overridden by passing the git_providers
        argument when spawning a class instance. This allows for one to write
        classes which inherit from salt.utils.gitfs.Pygit2 or
        salt.utils.gitfs.GitPython, and then direct one of the GitBase
        subclasses (GitFS, GitPillar, WinRepo) to use the custom class. For
        example:

        .. code-block:: Python

            import salt.utils.gitfs
            from salt.fileserver.gitfs import PER_REMOTE_OVERRIDES, PER_REMOTE_ONLY

            class CustomPygit2(salt.utils.gitfs.Pygit2):
                def fetch_remotes(self):
                    ...
                    Alternate fetch behavior here
                    ...

            git_providers = {
                'pygit2': CustomPygit2,
                'gitpython': salt.utils.gitfs.GitPython,
            }

            gitfs = salt.utils.gitfs.GitFS(
                __opts__,
                __opts__['gitfs_remotes'],
                per_remote_overrides=PER_REMOTE_OVERRIDES,
                per_remote_only=PER_REMOTE_ONLY,
                git_providers=git_providers)

            gitfs.fetch_remotes()
        Nrmr�zenvs.p�hashZ
file_lists)rn�
GIT_PROVIDERS�
git_providers�verify_providerr�Zremote_rootrrrsr�r{rH�	env_cache�
hash_cachedir�file_list_cachedir�cacheZverify_cache_version�init_remotes)	r�rnr��per_remote_overridesrR�global_onlyrr�rrrr!r��	s,2����zGitBase.__init__c
s�t�|�}�fdd�tD�}�jtvr|t7}n2|rJd�d�t��}�j�d�j�d�j�d�j�d|�d	�
}�jd
kr@|d7}t�	|�t
�j�i}	t|�}
|
�t|��|
D]$}�j�d|��}|�j
vrst�	d
|�t
�j�t|�j
|�|	|<qYg�_|D]�}
�j�j�j
|
|	||�j�j�}t|d��r�j
ddkr�|jr�|��i|_|j��D]5\}}d|vr�|d}|j�|g��|�|dkr�|j|�d�||jkr�t�d�j|j�j�||_q�g}|j��D]}|�|�q�|j ��D]\}}||v�rd|v�r|j�|dg��|�q��j�|�q�i��jD]}��|�!�g��|j��q#�fdd��D�}|�rW|D]}t�	d�jd��|��t
�j��qAt"dd��jD���rh��#�dSdS)z$
        Initialize remotes
        cs2g|]}�j�j�d|��r�j�d|���qS)r*)rnrHr5r�rr!r8
s
��z(GitBase.init_remotes.<locals>.<listcomp>z{}rTz) authentication was configured, but the 'z' zb_provider does not support authentication. The providers for which authentication is supported in z are: �.rKrZr*zNKey '%s' not present in global configuration. This is a bug, please report it.r��__role�minionrrWz�The 'base' environment has been defined in the 'saltenv' param for %s remote %s and will override the branch/tag specified by %s_base (or a per-remote 'base' parameter).cs g|]}t�|�dkr|�qS)r^)r�r5)�cachedir_maprr!r8�
� zqThe following %s remotes have conflicting cachedirs: %s. Resolve this using a per-remote parameter called 'name'.css�|]}|jVqdSrL)r�r5rrr!rY�
s�z'GitBase.init_remotes.<locals>.<genexpr>N)$rvrwr�r�r�r�r{rHrBrzrIr�r�rnrGr�rr�r�r�rr�rbrxr�r��poprWr�r�values�extendrur�r��write_remote_map)r�r�rrRrr�Zglobal_auth_paramsZmsg_auth_providersr�r�Z
global_valuesr�r.r�Zrepo_objrbr�rZall_envsZ	env_namesrcr�Z
collisionsr�r)rr�r!r
s�

�

����
�


�

�	
����
��zGitBase.init_remotesc
Cs\zt�|�Wnty#}zt�d|j||�WYd}~dSd}~wwt�d|j|�dS)Nz.Unable to remove old %s remote cachedir %s: %sFz%s removed old cachedir %sT)rrrrBrCrHr�)r�r�rFrrr!�_remove_cache_dir�
s���zGitBase._remove_cache_dirccsB�t�|j�D]}|dvrqtj�tjj�|j|��r|VqdS)N)r�r�rjri)r�r�r�r�r�rrrsr{)r�r/rrr!�_iter_remote_hashes�
s���zGitBase._iter_remote_hashescCsVd}dd�|jD�}|��D]}||vr"|�tjj�|j|�p |�}q|s)|��|S)zK
        Remove cache directories for remotes no longer configured
        FcSsh|]}|���qSr)r�)r6r1rrr!�	<setcomp>�
r9z,GitBase.clear_old_remotes.<locals>.<setcomp>)	r�rrrrrsr�r{r�r)r�ZchangeZ
remote_setr/rrr!�clear_old_remotes�
s��zGitBase.clear_old_remotescCspg}|j|jfD]-}tj�|�r5zt�|�Wqty4}z|�d|�d|���WYd}~qd}~wwq|S)z(
        Completely clear cache
        zUnable to delete r3N)	r�rr�r�r�rrrr�)r��errorsZrdirrFrrr!�clear_cache�
s"���zGitBase.clear_cacher�c	Cs�g}g}|jD]6}|r+zt�|j|�sWqWnty*t�|jt|��s(YqYnw|j|d�\}}|�|�|�|�q||fS)z1
        Clear update.lk for all remotes
        r)r��fnmatchryr,r-r
r)r�r�r�Zclearedrr�rrrrr!r
�
s"
���
zGitBase.clear_lockc
s||durg}nt|t�r|�d�}n
t|t�st�d|�g}d}|jD�]}t|dd�}|r;|j|f|vs;||v�r;z�t	j
�|���d��fdd�t	���D�}|D][}||�
�kr]qTtjj
��|�}tjj
�|d	�}t	j
�|�r�ztjj�|d
��Wd�n1s�wYWqTty�}	ztjd||	dd
�WYd}	~	qTd}	~	wwt�d|�qTt	���D]]}||�
�kr�q�tjj
��|�}tjj
�|d	�}t	j
�|��rztjj�|d
��Wd�n1s�wYWq�t�y}	ztjd||	dd
�WYd}	~	q�d}	~	wwt�d|�q�|���rd}Wq$t�y:}	ztjd|j|j|	dd
�WYd}	~	q$d}	~	wwq$|S)z�
        Fetch all remotes and return a boolean to let the calling function know
        whether or not any remotes were updated in the process of fetching
        Nr:zLInvalid 'remotes' argument (%s) for fetch_remotes. Must be a list of stringsFr
rcs$g|]\}}}|stj�|���qSr)r�r�r�)r6r��subdirsr#�Zrepo_work_hashrr!r8�
s��z)GitBase.fetch_remotes.<locals>.<listcomp>r\r z#Failed to make fetch request: %s %sTrkz Failed to make fetch request: %sz2Exception caught while fetching %s remote '%s': %s)r>r-r@r?rBrCr�r�rr�r�r��walkr�rrrsr{r�r#r$rr�rrArH)
r�r��changedr�r
Zbranchesr�Zbranch_salt_dirZ
fetch_pathrFrrr!�
fetch_remotes�
s�

�
�������������
�����DzGitBase.fetch_remotesc	Cs�g}g}|jD]4}|r+zt�|j|�sWqWnty*t�|jt|��s(YqYnw|��\}}|�|�|�|�q||fS)z$
        Place an update.lk
        )r�rryr,r-rAr)r�r��lockedrr�rrrrr!rA3s"
���
zGitBase.lockcCsnddd�}|��|d<|j|d�rd|d<|jddk}|ddus)tj�|j�s=tj�|j�}tj�|�s;t�	|�d}|rn|j
dd	�}tjj
�|jd
��}|�tj�|��t�d|j�Wd�n1siwY|j�d
d�r�tjjjd|jd|jdd��}|�|tddgdd��Wd�n1s�wYztj�|j|j�WdSty�YdSw)aE
        .. versionchanged:: 2018.3.0
            The remotes argument was added. This being a list of remote URLs,
            it will only update matching remotes. This actually matches on
            repo.id

        Execute a git fetch on all of the repos and perform maintenance on the
        fileserver cache.
        FrK)rZbackendr�r�Tr
r)�ignore_cacher�zWrote env cache data to %sNZfileserver_eventsZmasterZsock_dir)rn�listenr��
fileserver�r�)rrrnr�r�r]rr�r�r�rQrrrsr#r$r%�payload�dumpsrB�tracerp�eventZ	get_eventZ
fire_eventrr"Zreap_fileserver_cache_dirrrSr)r�r�rtZrefresh_env_cacheZenv_cachedirZnew_envsr'r'rrr!r�IsB

���
��zGitBase.updatecCsdd�|jD�S)at
        Returns a dictionary mapping remote IDs to their intervals, designed to
        be used for variable update intervals in salt.master.FileserverUpdate.

        A remote's ID is defined here as a tuple of the GitPython/Pygit2
        object's "id" and "name" attributes, with None being assumed as the
        "name" value if the attribute is not present.
        cSs"i|]
}|jt|dd�f|j�qS)r
N)rr�r))r6r�rrr!�
<dictcomp>�s��z,GitBase.update_intervals.<locals>.<dictcomp>rr�rrr!�update_intervals{s	�zGitBase.update_intervalscCs d|j�d�|jvr|jd|j�d�|_ne|j�|j�d��}|s6|jdd�r,d|_nO|jdd�r5d|_nEz|��}WntyKt|���}Ynw||j	vrdt
�d|j|d�|j	��t
|j�n|dkrp|��rpd|_n|dkr{|��r{d|_t|d	�s�t
�d
|j�t
|j�dSdS)z1
        Determine which provider to use
        �	verified_�	_providerT)�quietrryz/Invalid %s_provider '%s'. Valid choices are: %srTr�z,No suitable %s provider module is installed.N)rHrnr�rp�
verify_pygit2�verify_gitpythonrdr�r-rrBrzr{rIr�)r�Zdesired_providerrrr!r�s<��

�
�zGitBase.verify_providerFcs��fdd�}ts|st�d�j�|�dSd�jvrdSg}ttkr2|��j�dt�dt�d��tjj	�
d	�sC|�d
�j�d��|rV|D]}t�|�qG|sT|�dSd�jd�j�d
�<t�d�j�dS)zX
        Check if GitPython is available and at a compatible version (>= 0.3.0)
        c�,trd�jvrt�t�j�j�dSdSdSr�)r�rrBrC�_RECOMMEND_PYGIT2rHrr�rr!�
_recommend���z,GitBase.verify_gitpython.<locals>._recommendzA%s is configured but could not be loaded, is GitPython installed?Fryz: is configured, but the GitPython version is earlier than �
. Version �
 detected.rzDThe git command line utility is required when using the 'gitpython' �
_provider.r*r+zgitpython %s_provider enabledT)
�GITPYTHON_VERSIONrBrCrHr�GITPYTHON_MINVERr�rrrsr��whichrnr��r�r,r1rrCrr�r!r.�sF�
�����zGitBase.verify_gitpythoncs�fdd�}ts|st�d�j�|�dSd�jvrdSg}ttkr2|��j�dt�dt�d��ttkrE|��j�d	t�dt�d��t	t
d
d�s\tjj
�d�s\|�d�j�d
��|ro|D]}t�|�q`|sm|�dSd�jd�j�d�<t�d�j�dS)z�
        Check if pygit2/libgit2 are available and at a compatible version.
        Pygit2 must be at least 0.20.3 and libgit2 must be at least 0.20.0.
        cr/rx)r6rrBrC�_RECOMMEND_GITPYTHONrHrr�rr!r1�r2z)GitBase.verify_pygit2.<locals>._recommendzK%s is configured but could not be loaded, are pygit2 and libgit2 installed?Frz7 is configured, but the pygit2 version is earlier than r3r4z8 is configured, but the libgit2 version is earlier than r�rzAThe git command line utility is required when using the 'pygit2' r5r*r+zpygit2 %s_provider enabledT)r�rBrCrHr�
PYGIT2_MINVERr��LIBGIT2_VERSION�LIBGIT2_MINVERr�rrrrsr�r8rnr�r9rr�r!r-�sZ�
���������zGitBase.verify_pygit2c
Cs�tjj�|jd�}zGtjj�|d��5}t���	d�}|�
d|j�d|�d��|jD]}|�
tjj
�|���d|j�d���q*Wd�n1sJwYWn
tyZYdSwt�d	|j|�dS)
z*
        Write the remote_map.txt
        zremote_map.txt�w+z%d %b %Y %H:%M:%S.%fz# z_remote map as of r�z = NzWrote new %s remote map to %s)rrrsr�r{r�r#r$r�now�strftimer%rHr�r�r�r�rrrB�info)r�Z
remote_mapr'�	timestampr�rrr!r
s$
�����	�zGitBase.write_remote_mapcCs�t��}t��|dkrFz|j|d�WStyE}z&|jtjkr,t�d�WYd}~qtjd|j|j|j	|dd�WYd}~dSd}~wwt�d|j|j	|j�dS)	z�
        Common code for git_pillar/winrepo to handle locking and checking out
        of a repo.

        fetch_on_fail
          If checkout fails perform a fetch then try to checkout again.
        r�r�g�������?NzIError %d encountered while obtaining checkout lock for %s remote '%s': %sTrkz�Timed out waiting for checkout lock to be released for %s remote '%s'. If this error persists, run 'salt-run cache.clear_git_lock %s type=checkout' to clear it.)
rErMrrr,rFrBrCrHr)r�r�rLrGrFrrr!�do_checkout!s6
�
	
���zGitBase.do_checkout)Nr�rL�Frm)rVrnrorp�PER_REMOTE_ONLY�GLOBAL_ONLYr�rrrrrr
rrAr�r)rr.r-rrCrrrr!r��	s:
�M
�


Z
2
%
+4r�cs�eZdZdZdZe��Zddedddf�fdd�	Z	dedddfdd	�Z
d
d�Zdd
d�Zddd�Z
dd�Zdd�Zdd�Zdd�Zdd�Zdd�Z�ZS) �GitFSz>
    Functionality specific to the git fileserver backend
    rKNrTc
	s�|r
tjjjj��nd}|r||jvrKt�|�}	t	t
|	�j||dur$|ng|||dur-|nt||d�|s<t
�d�|	St
�dt���|	|j|<|	St
�dt���|j|S)a�
        If we are not initializing remotes (such as in cases where we just want
        to load the config so that we can run clear_cache), then just return a
        new __init__'ed object. Otherwise, check the instance map and re-use an
        instance if one exists for the current process. Weak references are
        used to ensure that we garbage collect instances for threads which have
        exited.
        N)rrRrr�rz/Created gitfs object with uninitialized remotesz#Created gitfs object for process %sz$Re-using gitfs object for process %s)rr�extZtornadoZioloopZIOLoop�current�instance_map�object�__new__r{rGr�r�rBr�r�r�)
r�rnr�rrRrr�rZio_loopr�r}rr!rLPs(

�
�

z
GitFS.__new__cCsdSrLr)r�rnr�rrRrr�rrrr!r�szGitFS.__init__cC�|�|d�S)z@
        Return a list of all directories on the master
        rd��_file_lists�r��loadrrr!rO�szGitFS.dir_listFcsv|stj�|j|j�}|dur|St�}|jD]����}�j�	�D]}|�
|�q"|�
�fdd�|D��qt|�S)zH
        Return a list of refs that can be used as environments
        Ncsg|]	}��|�r|�qSr)rPr5�r�rr!r8�szGitFS.envs.<locals>.<listcomp>)rrr"Zcheck_env_cachernrr�r�rQr�rr�r")r�r �cache_matchr0Z	repo_envs�env_listrrRr!rQ�s
z
GitFS.envsrWcKs�ddd�}tj�|�r
|Stjj�|jd||�}tjj�|j||�d��}tjj�|j||�d��}tjj�|j||�d��}tj�|�}	tj�|�}
tj�	|	�slzt�
|	�Wntykt�|	�t�
|	�Ynwtj�	|
�s�zt�
|
�Wnty�t�|
�t�
|
�Ynw|j
D�]0}|�|�r�|�|�|�tj�s�q�tjj�|�s�||��vr�|js�q�|t|�|��d��tj�}|�|�r�tjj�|�|�|�}|�||�\}
}}|
dur�q�dd	�}tj�||�z@tjj�|d
��-}tjj�|���}||k�r||d<||d<|||�Wd�WSWd�n	1�s'wYWnt�yH}z|jtj k�r>�WYd}~nd}~wwtjj�|d
��Wd�n	1�s]wYt!�!|�D]}zt�|�W�qgt"�y|Y�qgw|�#|
|�tjj�|d��
}|�$|�Wd�n	1�s�wYzt�|�Wn
t�y�Ynw||d<||d<|||�S|S)z�
        Find the first file to match the path and ref, read the file out of git
        and send the path to the newly cached file
        r[)r��relr�z.hash.*z.hash.blob_sha1r�NcSs|dur	|g|d<|S)a!
                Add a the mode to the return dict. In other fileserver backends
                we stat the file to get its mode, and add the stat result
                (passed through list() for better serialization) to the 'stat'
                key in the return dict. However, since we aren't using the
                stat result for anything but the mode at this time, we can
                avoid unnecessary work by just manually creating the list and
                not running an os.stat() on all files in the repo.
                Nr�r)�fndr�rrr!�_add_file_stat�s

z'GitFS.find_file.<locals>._add_file_statr1rUr�r r>)%r�r�r�rrrsr{r�rr�r�r�rrr�rr�r�r�r�rQrWr��lstriprrSr"Z	wait_lockr#r$r:r!rr�globrArkr%)r�r�r�r�rVrjZhashes_globZblobshadestZlk_fnZdestdirZhashdirr�r�riZblob_hexshaZ	blob_moderWr'ZsharF�filenamerrr!rS�s�
��
�
����

����������zGitFS.find_filec	s"d�vr	��d�ddd�}hd�}t�fdd�|D��s+t�dd�|�����|S|d	s1|S|d
|d<��dd
�}tj�	|d	�}t
jj�
|d��:}|��d�|�|jd�}|rkt
jj�|�sk|�t�}|r{|r{t
jj�||�}||d<||d<Wd
�|S1s�wY|S)zG
        Return a chunk from a file based on the data received
        r�r[)rtrj>rbr��locc3��|]}|�vVqdSrLrr5�rQrr!rY
��z#GitFS.serve_file.<locals>.<genexpr>z<Not all of the required keys present in payload. Missing: %srTr�rUrj�gzipN�rbr[Zfile_buffer_sizert)rr�rBr�r{�
differencerpr�r��normpathrrrsr#r$r�r!rnZ	is_binaryrr�Z	gzip_util�compress)	r�rQrVr0Zrequired_load_keysr_Zfpathr'rtrr]r!�
serve_file
s8

�


�	�	zGitFS.serve_filec

s�d�vr	��d�t�fdd�dD��sdSd|jdi}|d}|d}|jd}tjj�|j�d	|�d
|���}z#tjj�	|d��}|�
�|d<Wd
�|WS1sWwY|WStyw}	z
|	jtj
krm�WYd
}	~	nd
}	~	wwzt�tj�|��Wnty�}	z
|	jtjkr��WYd
}	~	nd
}	~	wwtjj�||jd�|d<tjj�	|d��}|�|d�Wd
�|S1s�wY|S)zT
        Return a file hash, the hash type is set in the master config file
        r�c3r\rLrr5r]rr!rY3
r^z"GitFS.file_hash.<locals>.<genexpr>)r�rb)r[NrfrUr�rbz.hash.r`ZhsumNr>)rr�rnrrrsr�r{rr#r$r!rrrr�r�r�r,Z	hashutilsZget_hashr%)
r�rQrVr0r�r�Zlc_hash_typeZhashdestr'rFrr]r!�	file_hash+
sR

�
��������
��zGitFS.file_hashcCs�d|vr	|�d�tj�|j�s+zt�|j�Wnty*t�d|j�gYSw|d�	tjj
d�}tjj�
|j|�d��}tjj�
|jd|�d��}tj�|j|||�\}}}|dura|S|r�t�d	�t�it�d
�}	|jD]I}
tjj�|d�s�|d|��vs�|
jr�t��}|
�|d�\}}
|	d�|�|	d�|
�|	d
�|
�|d��t�d|
jt��|�qst|	d�|	d<t|	d
�|	d
<|r�tj�|j|	||�t�d�|	� |g�S|dkr�iSgS)zL
        Return a dict containing the file lists for files and dirs
        r�zUnable to make cachedir %srbz_|-z.pr	z.wNz&Start rebuilding gitfs file_list cache)r#r�rdr#r�rdz9gitfs file_name cache rebuild repo=%s duration=%s secondsz)Finished rebuilding gitfs file_list cache)!rr�r�r�rr�rrBrCr�r�rrrsr{r"Zcheck_file_list_cachernr&r�r�r�r�rQrWrErRr�rOZprofilerr"Zwrite_file_list_cacherp)r�rQZformZlc_path_adjZ
list_cacheZw_lockrSZ
refresh_cacheZ
save_cacher0r��startZ
repo_filesZ
repo_symlinksrrr!rOQ
sh
��
�

�

��
��
�
zGitFS._file_listscCrM)zb
        Return a list of all files on the file server in a specified
        environment
        r#rNrPrrr!rR�
szGitFS.file_listcCsgS)zF
        Return a list of all empty directories on the master
        rrPrrr!�file_list_emptydirs�
szGitFS.file_list_emptydirscsvd|vr	|�d�tjj�|d�s|d|��vriSd|vr(|d�d��nd�|�|d�}�fdd�|��D�S)	zQ
        Return a dict of all symlinks based on a given path in the repo
        r�rbr�r]r[r�cs i|]\}}|���r||�qSr)r�)r6r.rDr#rr!r(�
r
z&GitFS.symlink_list.<locals>.<dictcomp>)	rrrrsr�r�rQr4rOrx)r�rQr�rr#r!�symlink_list�
s
�zGitFS.symlink_listrDr�)rVrnrorprH�weakref�WeakKeyDictionaryrJrErLr�rOrQrSrdrerOrRrgrhr�rrr}r!rGHs4�3
�

g &>rGc@s&eZdZdZdZd	dd�Zdd�ZdS)
�	GitPillarz;
    Functionality specific to the git external pillar
    rUTcCs�t�|_g|_|jD]`}|j||d�}|durj|jdkr0t|d�r0|j�d�p.|j�d�p.d}n|j	r7|j	}n|j|j
kr@d}n
|��}||j
krKdn|}|jre|�
|�rd||j|��<|j�|���q
||j|<q
dS)z�
        Checkout the targeted branches/tags from the git_pillar remotes

        fetch_on_fail
          If checkout fails perform a fetch then try to checkout again.
        r�NrVr�rTrbrW)r	Zpillar_dirsZpillar_linked_dirsr�rCr�r�rnrpr�rWr�r^�link_mountpointr�r�)r�rLr�rmr�Ztgtrrr!rM�
s,
�
�
��zGitPillar.checkoutc

Cs&tjj�|��|j�}tjj�|��|����t	j
�}d}d}�zU|jddd���Att	j
|��dd��}||jkrRt�d|���t�d|�t�d|j�d	}n�td
d�|dd
�D��sht�d|�d	}nmtjj�|�srd	}ncz	tjj�|�}Wnty�t�d|�d	}YnIw||kr�t�d|||�ztjj��r�|�d�s�t	j�|�r�t�|�nt	�|�Wnty�}z
t�d||�WYd}~nd}~wwd}d	}|�r#d	}z	t�|���Wn	ty�Ynwzt	j�|�}	t	�|	�t�d|	�Wn"t�y"}zt� d|	|�WYd}~Wd�WdSd}~ww|�razt	�!||�t�d||�WWd�Wd	St�y`}zt� d|||�WYd}~Wd�WdSd}~wwWd�Wd	S1�snwYWd	St"�y�t� d|j#|j$|j#|j#|j%dd��YdSw)zv
        Ensure that the mountpoint is present in the correct location and
        points at the correct path
        Fr�
)r�r)�followlinksz2Results of walking %s differ from expected resultszWalk results: %szExpected results: %sTcss4�|]}tjj�|d�otj�|d�VqdS)rN)rrrsr��islinkr�r�r5rrr!rY�
s
�"�
�z,GitPillar.link_mountpoint.<locals>.<genexpr>Nr_z-Linkdir parents of %s are not all directoriesz Failed to read destination of %sz=Destination of %s (%s) does not match the expected value (%s)z\\z;Failed to remove existing git_pillar mountpoint link %s: %sz#Successfully made linkdir parent %sz-Failed to os.makedirs() linkdir parent %s: %sz%Successfully linked %s to cachedir %sz-Failed to create symlink to %s at path %s: %sz�Timed out setting mountpoint lock for %s remote '%s'. If this error persists, it may be because an earlier %s checkout was interrupted. The lock can be cleared by running 'salt-run cache.clear_git_lock %s type=mountpoint', or by manually removing %s.r)&rrrsr�r{r�r^r�rr�r�r�r*r?rrerBr�r�ro�readlinkrAr�r�r�r�rrr�	exceptionrr�r�rC�symlinkrrHrr�)
r�r�Z
lcachelinkZ
lcachedestZwipe_linkdirZcreate_linkZwalk_resultsZldestrFZldirnamerrr!rl�
s��
�
����
��
�
�����
�
��I��Z�
��Z��p�p�
��zGitPillar.link_mountpointNrm)rVrnrorprHrMrlrrrr!rk�
s

!rkc@s"eZdZdZdZiZddd�ZdS)�WinRepoz�
    Functionality specific to the winrepo runner

    fetch_on_fail
          If checkout fails perform a fetch then try to checkout again.
    r\TcCs8i|_|jD]}|j||d�}|dur||j|j<qdS)zN
        Checkout the targeted branches/tags from the winrepo remotes
        r�N)�winrepo_dirsr�rCr)r�rLr�rmrrr!rMfs
��zWinRepo.checkoutNrm)rVrnrorprHrtrMrrrr!rsYs
rscCs�t��}t��dd�}t�|d�}|j�rJ|���rLt|�	d��}g}d}d}zy|D]N}t
jj�
|d��<}ztt
jj�|������}WntyPd}Ynwz
t
jj�|�����}Wntyid}YnwWd�n1stwYq+||kr�||kr�|s�d|�d	�}	t�|	�|�|||f�n|�|||f�Wnty�t�d
|�Ynw|D]�\}}}zt�|�Wnxt�y7}
zk|
jtjkr�d|�d|�d
|�d|
�d�	}	t�|	�nJ|
jtjk�rzt�|�Wn;t�y}
zd|�d|�d
|�d|
�d�	}	t�|	�WYd}
~
nd}
~
wwd|�d|�d
|�d|
�d�	}	t�|	�WYd}
~
q�d}
~
wwd|�d|�d
|�d�}	t�|	�q�dSdSdS)z5
    Clean up finalize processes that used gitfs
    rOrPz/gitfs/workz**/*.lkrr1Nzgitfs lock file for pid 'a' does not contain a machine id, deleting lock file which may affect if using multi-master with shared gitfs cache, the lock may have been obtained by another master recommend updating Salt version on other masters to a version which insert machine identification in lock a file.zgitfs lock file: %s not foundz<SIGTERM clean up of resources attempted to remove lock file z, pid 'z', machine identifier 'z$' but it did not exist, exception : rz*SIGTERM clean up of resources, lock file 'z', pid 'z2'was a directory, removed directory, exception : 'rz;SIGTERM clean up of resources, unable to remove lock file 'z', exception : 'z2SIGTERM clean up of resources, removed lock file ')r�r�rorp�pathlib�Pathr��is_dirr?rYrrrsr#r$r=r�r:r;r�r<rBr�r��FileNotFoundErrorrrrrrrr)r�r�rqrRZ
file_del_listZfile_pidZfile_mach_id�	file_namer=r�rFrrr!r�qs���������
�
��������������	�����
�� �����*r�)orpr�rtrvrrrYr�r��loggingrqr�rur�rr�r�rErirZsalt.ext.tornado.iolooprrZsalt.fileserverZsalt.utils.cacheZsalt.utils.configparserZsalt.utils.dataZsalt.utils.filesZsalt.utils.gzip_utilZsalt.utils.hashutilsZsalt.utils.itertoolsZsalt.utils.pathZsalt.utils.platformZsalt.utils.processZsalt.utils.stringutilsZsalt.utils.urlZsalt.utils.userZsalt.utils.versionsZsalt.configrrr�Zsalt.exceptionsrrrZsalt.utils.eventrZsalt.utils.odictr	r
rorrerErFr�r�r�r�r:r0r��	getLoggerrVrBr�r��ImportErrorrsr�Z	is_darwinr�r8Zsalt.utils.mac_utilsrrr��__version__r6rAr��catch_warnings�simplefilterrr�r<Z
pygit2.ffiZ
pygit2.remoter"r�Zmaybe_stringrr�r�rFr>rqr7r;r=rGrIrJr�r�r�r�rGrkrsr�rrrr!�<module>s(���
�
��


�

���
��6Z�l(
@ Telegram