File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/master.cpython-310.pyc
o
�N�g�v � @ sZ d Z ddlZddlZddlZddlmZmZ ddlZddl Zddl
Zddl Zddl Zddl
ZddlZddlZddlZddlZddlZddlmZ ddlmZ ddlmZ ddlmZ e� e �Z dd � Z d
d � Z d d
� Z!G dd� d�Z"G dd� de�Z#G dd� de�Z$G dd� de�Z%dd� Z&d dd�Z'dd � Z(e d kr�ej)�*d �Z+e%e+�Z,e,�-� dS dS ) ze
salt.utils.master
-----------------
Utilities that can only be used on a salt master.
� N)�Event�Thread)�
SaltException)�CacheCli)�Process)�zmqc C s� g }t j�| d d�}t j�|�s|S t �|�D ](}t j�||�}zt|| �}|dur0|�|� W q ty@ t� d|� Y qw |S )z0
Return the running jobs on this minion
�cachedir�procNz.%s removed during processing by master process)
�os�path�join�isdir�listdir�_read_proc_file�append�OSError�logZtrace)�opts�retZproc_dirZfn_r �data� r �E/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/master.py�get_running_jobs! s
� �r c C s< t jj�| d��9}|�� }|�� |rt j�|�}n zt� | � W n t
y0 t � d| � Y nw W d � dS W d � n1 sDw Y t
|t�sPdS t jj�|d �srzt� | � W dS t
yq t � d| � Y dS w t|�s�|�d�}|r�t �d|� zt� | � W dS t
y� t � d| � Y dS w |S )z0
Return a dict of JID metadata, or None
�rbz Unable to remove proc file %s.N�pidz7PID %s exists but does not appear to be a salt process.)�salt�utils�files�fopen�read�close�payload�loadsr
�remover r �debug�
isinstance�dict�processZ
os_is_running�_check_cmdline�get�warning)r r �fp_�bufr r r r r r 8 sH �� �
��
��r c C s� t jj�� sdS | �d�}|sdS tj�d�sdS tj�d|� d��}tj� |�s+dS z"t jj
� |d��}d|� � v W d � W S 1 sFw Y W d S t
yW Y dS w )
a9
In some cases where there are an insane number of processes being created
on a system a PID can get recycled or assigned to a non-Salt process.
On Linux this fn checks to make sure the PID we are checking on is actually
a Salt process.
For non-Linux systems we punt and just return True
Tr Fz/procz/proc/z/cmdliner s saltN)r r �platformZis_linuxr) r
r r
r �isfiler r r r )r r r r+ r r r r( ` s"
(� �r( c @ s� e Zd ZdZ d dd�Zdd � Zd
d � Zd d
� Zd dd�Zdd� Z dd� Z
dd� Z dd� Z dd� Z
dd� Z d!d d �ZdS )"�MasterPillarUtila�
Helper utility for easy access to targeted minion grain and
pillar data, either from cached data on the master or retrieved
on demand, or (by default) both.
The minion pillar data returned in get_minion_pillar() is
compiled directly from salt.pillar.Pillar on the master to
avoid any possible 'pillar poisoning' from a compromised or
untrusted minion.
** However, the minion grains are still possibly entirely
supplied by the minion. **
Example use case:
For runner modules that need access minion pillar data,
MasterPillarUtil.get_minion_pillar should be used instead
of getting the pillar data by executing the "pillar" module
on the minions:
# my_runner.py
tgt = 'web*'
pillar_util = salt.utils.master.MasterPillarUtil(tgt, tgt_type='glob', opts=__opts__)
pillar_data = pillar_util.get_minion_pillar()
� �globNTc
C s� t �d| jj� |d u r t �d| jj� t| jj� d���|| _|| _|| _|| _ || _
|| _ || _ || _
tj�|�| _t �d|||||||� d S )NzNew instance of %s created.z!%s: Missing master opts init arg.z : Missing master opts init arg.z�Init settings: tgt: '%s', tgt_type: '%s', saltenv: '%s', use_cached_grains: %s, use_cached_pillar: %s, grains_fallback: %s, pillar_fallback: %s)r r$ � __class__�__name__�errorr r �tgt�tgt_type�saltenv�use_cached_grains�use_cached_pillar�grains_fallback�pillar_fallbackr �cache�factory) �selfr5 r6 r7 r8 r9 r: r; r r r r �__init__� s2 ��zMasterPillarUtil.__init__c G s� dd� |D �}| j �dd�s | j �dd�s t�d� |S |s$| j�d�}|D ] }tjj� | j |�s2q&| j�
d|� �d �}t |t �rE|||< q&|S )
Nc S � i | ]}|i �qS r r ��.0� minion_idr r r �
<dictcomp>� � z:MasterPillarUtil._get_cached_mine_data.<locals>.<dictcomp>�minion_data_cacheFZenforce_mine_cachezSSkipping cached mine data minion_data_cacheand enfore_mine_cache are both disabled.�minions�minions/�mine)
r r) r r$ r< �listr r �verify�valid_id�fetchr% r& )r> �
minion_ids� mine_datarC �mdatar r r �_get_cached_mine_data� s$ ��
�z&MasterPillarUtil._get_cached_mine_datac G s� dd� |D �}|� � }| j�dd�st�d� ||fS |s#| j�d�}|D ]:}tjj �
| j|�s1q%| j� d|� �d�}t |t
�sKt�d t|�j|� q%d
|v rU|d
||< d |v r_|d ||< q%||fS ) Nc S r@ r r rA r r r rD � rE z<MasterPillarUtil._get_cached_minion_data.<locals>.<dictcomp>rF Fz>Skipping cached data because minion_data_cache is not enabled.rG rH r zGcache.fetch should always return a dict. ReturnedType: %s, MinionId: %s�grains�pillar)�copyr r) r r$ r< rJ r r rK rL rM r% r& r* �typer3 )r> rN rR �pillarsrC rP r r r �_get_cached_minion_data� s0
� �z(MasterPillarUtil._get_cached_minion_datac C s` t �d|� tj�| jd ��}|jd�|�d| jd dd�W d � S 1 s)w Y d S )Nz%Getting live grains for minions: "%s"Z conf_file�,z grains.items�timeoutrJ )rY r6 )r r$ r �clientZget_local_clientr �cmdr )r> rN rZ r r r �_get_live_minion_grains� s �$�z(MasterPillarUtil._get_live_minion_grainsc C sb |d u ri S |st �d|� i S t �d|� tj�| j||| j| jd �}t �d|� |�� }|S )Nz2Cannot get pillar data for %s: no grains supplied.zGetting live pillar for %sZ
ext_pillarzCompiling pillar for %s) r r* r$ r rS ZPillarr r7 Zcompile_pillar)r> rC �
minion_grainsrS r r r r �_get_live_minion_pillar� s � z(MasterPillarUtil._get_live_minion_pillarc s
i }|� di �}i � i �| jrJdd� |�� D �� � fdd�|D �}t�d|� | jr/| �|��ttdd� |D ��� �t��� � t� �� � �}|S | �|���fdd�|D �}t�d |� | jrjd
d� |�� D �� ttd d� |D ��� �t��� � t� �� � �}|S ) N�
cached_grainsc S � i | ] \}}|r||�qS r r �rB rC Zmcacher r r rD � ��z7MasterPillarUtil._get_minion_grains.<locals>.<dictcomp>c � g | ]}|� vr|�qS r r rA ��cretr r �
<listcomp> � z7MasterPillarUtil._get_minion_grains.<locals>.<listcomp>z#Missed cached minion grains for: %sc S r@ r r rA r r r rD rE c rc r r rA ��lretr r rf # rg z!Missed live minion grains for: %sc S r` r r ra r r r rD ( rb c S r@ r r rA r r r rD . rE ) r) r8 �itemsr r$ r: r\ r& rJ )r> rN �kwargsr r_ �missed_minionsr )re ri r �_get_minion_grains sN �
�
�
��
�
� �
�
��z#MasterPillarUtil._get_minion_grainsc s* i }|� di ��|� di �}i � i ��jrUdd� |�� D �� � fdd�|D �}t�d|� �jr:��fdd�|D ��ttd d� |D ��� �t��� � t� �� � �}|S ��fd
d�|D ���fd d�|D �}t�d |� �jrzd
d� |�� D �� ttdd� |D ��� �t��� � t� �� � �}|S )NrR �
cached_pillarc S r` r r ra r r r rD > rb z7MasterPillarUtil._get_minion_pillar.<locals>.<dictcomp>c rc r r rA rd r r rf C rg z7MasterPillarUtil._get_minion_pillar.<locals>.<listcomp>z$Missed cached minion pillars for: %sc
�" i | ]
}|�� |� �|i ���qS r �r^ r) rA �rR r> r r rD H � � ��c S r@ r r rA r r r rD O rE c
ro r rp rA rq r r rD T rr c rc r r rA rh r r rf Z rg z"Missed live minion pillars for: %sc S r` r r ra r r r rD _ rb c S r@ r r rA r r r rD e rE )r) r9 rj r r$ r; r& rJ )r> rN rk r rn rl r )re rR ri r> r �_get_minion_pillar4 sX �
� �
�
�� ��
� �
�
��z#MasterPillarUtil._get_minion_pillarc C s^ g }t jj�| j�}|�| j| j�}|d }|s#t� d| j| j� i S t� d| j| j|� |S )NrG z1No minions matched for tgt="%s" and tgt_type="%s"z3Matching minions for tgt="%s" and tgt_type="%s": %s)
r r rG � CkMinionsr Z
check_minionsr5 r6 r r$ )r> rN Z ckminionsZ_resr r r � _tgt_to_listk s$ ��z MasterPillarUtil._tgt_to_listc C s� i }i }| � � }| jr|si S tdd� | j| j| j| jfD ��r-t�d� | j |� \}}ni }i }t�d|� | j
|d|i�}t�d|� | j |||d��}|S )a�
Get pillar data for the targeted minions, either by fetching the
cached minion data on the master, or by compiling the minion's
pillar data on the master.
For runner modules that need access minion pillar data, this
function should be used instead of getting the pillar data by
executing the pillar module on the minions.
By default, this function tries hard to get the pillar data:
- Try to get the cached minion grains and pillar if the
master has minion_data_cache: True
- If the pillar data for the minion is cached, use it.
- If there is no cached grains/pillar data for a minion,
then try to get the minion grains directly from the minion.
- Use the minion grains to compile the pillar directly from the
master using salt.pillar.Pillar
c s � � | ]}|V qd S �Nr �rB �argr r r � <genexpr>� s
� �
�z5MasterPillarUtil.get_minion_pillar.<locals>.<genexpr>zGetting cached minion data�!Getting minion grain data for: %sr_ z"Getting minion pillar data for: %s)rR rn ) ru r5 �anyr8 r9 r: r; r r$ rW rm rs )r> Zminion_pillarsr] rN �cached_minion_grains�cached_minion_pillarsr r r �get_minion_pillar� s:
��
� �� �z"MasterPillarUtil.get_minion_pillarc C sl i }| � � }|s
i S tdd� | j| jfD ��r$t�d� | j|� \}}ni }t�d|� | j|d|i�}|S )a5
Get grains data for the targeted minions, either by fetching the
cached minion data on the master, or by fetching the grains
directly on the minion.
By default, this function tries hard to get the grains data:
- Try to get the cached minion grains if the master
has minion_data_cache: True
- If the grains data for the minion is cached, use it.
- If there is no cached grains data for a minion,
then try to get the minion grains directly from the minion.
c s rv rw r rx r r r rz � s � z5MasterPillarUtil.get_minion_grains.<locals>.<genexpr>zGetting cached minion data.r{ r_ )ru r| r8 r: r r$ rW rm )r> r] rN r} r~ r r r �get_minion_grains� s"
� ��z"MasterPillarUtil.get_minion_grainsc C s& i }| � � }t�d|� | j|� }|S )z@
Get cached mine data for the targeted minions.
z Getting cached mine data for: %s)ru r r$ rQ )r> rO rN r r r �get_cached_mine_data� s
z%MasterPillarUtil.get_cached_mine_dataFc C s� g }|r |� d� |r|� d� |r|� d� |dur$|� d|� d�� |s-t�d� dS | �� }t�d d
�|�|� ||krDi }i }n| j|� \}}z�| j�d �} |D ]{}
tj j
� | j |
�s`qT|
| vreqTd |
� �} |�
|
d�} |�
|
d�}
|rz|s�|r~|
r�|r�| s�| j�| d
� n |r�|
r�| j�| d
d|
i� n|r�| r�| j�| d
d| i� |r�| j�| d� qT|dur�| j�| d�}t|t�r�|�
|d�r�| j�| d|� qTW dS ty� Y dS w )zG
Clear the cached data/files for the targeted minions.
rS rR rI Nz mine_func: '�'z,No cached data types specified for clearing.Fz Clearing cached %s data for: %sz, rG rH r T)r r r$ ru r rW r< rJ r r rK rL r �pop�flush�storerM r% r& r )r> Z clear_pillarZ clear_grainsZ
clear_mineZclear_mine_funcZ
clear_whatrN rR rV Z c_minionsrC ZbankZ
minion_pillarr] rO r r r �clear_cached_minion_data� st
������
�� ��z)MasterPillarUtil.clear_cached_minion_data)r0 r1 NTTTTN)NN)FFFN)r3 �
__module__� __qualname__�__doc__r? rQ rW r\ r^ rm rs ru r r� r� r� r r r r r/ z s4
�)
,72 �r/ c @ s e Zd ZdZdd� Zdd� ZdS )�
CacheTimerzw
A basic timer class the fires timer-events every second.
This is used for cleanup by the ConnectedCache()
c C s6 t �| � || _|| _d| _tj�| jd d�| _d S )NT�sock_dir�
con_timer.ipc) r r? r �stopped�daemonr
r r �
timer_sock)r> r �eventr r r r? ( s
zCacheTimer.__init__c C s� t �� }|�t j�}|�t jd� |�d| j � d}t� d� | j
� d�sA|� t
j�|�� |d7 }|dkr9d}| j
� d�r&dS dS )z=
main loop that fires the event every second
�d �ipc://r zConCache-Timer started� �< N)r �Context�socketZPUB�
setsockopt�LINGER�bindr� r r$ r� �wait�sendr r! �dumps)r> �contextr� �countr r r �run/ s
�zCacheTimer.runN)r3 r� r� r� r? r� r r r r r� " s r� c s( e Zd ZdZ� fdd�Zdd� Z� ZS )� CacheWorkerz�
Worker for ConnectedCache which runs in its
own process to prevent blocking of ConnectedCache
main-loop when refreshing minion-list
c s t � jdi |�� || _dS )z<
Sets up the zmq-connection to the ConCache
Nr )�superr? r �r> r rk �r2 r r r? J s
zCacheWorker.__init__c C sD t tjj�| j��� �}t| j�}|�� |� |g� t
� d� dS )zI
Gather currently connected minions and update the cache
z$ConCache CacheWorker update finishedN) rJ r r rG rt r �
connected_ids� cache_cliZ
get_cachedZ put_cacher r$ )r> Znew_minsZccr r r r� Q s
zCacheWorker.run)r3 r� r� r� r? r� �
__classcell__r r r� r r� C s r� c sH e Zd ZdZ� fdd�Zdd� Zdd� Zdd � Zd
d � Zd d
� Z � Z
S )�ConnectedCachez�
Provides access to all minions ids that the master has
successfully authenticated. The cache is cleaned up regularly by
comparing it to the IPs that have open connections to
the master publisher port.
c s� t � jdi |�� t�d� || _g | _tj�| jd d�| _ tj�| jd d�| _
tj�| jd d�| _ | � � t
� | _t| j| j�| _| j�� d| _dS ) z=
starts the timer and inits the cache itself
zConCache initializing...r� z
con_cache.ipcz con_upd.ipcr� TNr )r� r? r r$ r rG r
r r �
cache_sock� update_sock�
upd_t_sock�cleanupr �
timer_stopr� �timer�start�runningr� r� r r r? d s
zConnectedCache.__init__c C s | � � dS )z-
handle signals and shutdown
N)�stop)r> �sig�framer r r �signal_handler~ s z ConnectedCache.signal_handlerc C s` t �d� tj�| j�rt�| j� tj�| j�r t�| j� tj�| j�r.t�| j� dS dS )z,
remove sockets on shutdown
zConCache cleaning upN) r r$ r
r �existsr� r# r� r� �r> r r r r� � s
�zConnectedCache.cleanupc C sf t �d� tj�| j�rt�| jd� tj�| j�r!t�| jd� tj�| j�r1t�| jd� dS dS )z9
secure the sockets for root-only access
zConCache securing socketsi� N) r r$ r
r r� r� �chmodr� r� r� r r r �secure� s
�zConnectedCache.securec C s0 | � � | jrd| _| j�� | j�� dS dS )z(
shutdown cache process
FN)r� r� r� �setr� r r� r r r r� � s
�zConnectedCache.stopc
C s$ t �� }|�t j�}|�t jd� |�d| j � |�t j�}|�t j d� |�t jd� |�d| j
� |�t j�}|�t j d� |�t jd� |� d| j � t �
� }|�|t j� |�|t j� |�|t j� t�tj| j� | �� t�d� | j�rwz t|�d��}W n- ty� | �� Y n! t jy� } zt� d� t� |� | �� W Y d}~nd}~ww |� |�t jkr�t j �!|�"� �}t�#d|� t$|t%�r�|d kr�t j �&| j'�} |�(| � |� |�t jk�rQt j �!|�"� �}
t$|
t)�s�t� d
� ~
q{z9|
�s t�#d � W q{|
d } t$| t%��r(| | j'v�r't�#d
|
d � | j'�*| � nt$| t)��r6t�#d� | | _'W n t+�yG t�#d� ~
Y nw t�dt,| j'�� |� |�t jk�rtt j �!|�"� �} t-| d �d k�rtt.| j/�}
|
�0� | js| �� |�1� |�1� |�1� |�2� t�#d� dS )zw
Main loop of the ConCache, starts updates in intervals and
answers requests from the MWorkers
r� r� � zConCache startedr� z ConCache ZeroMQ-Error occurredNz ConCache Received request: %srG z(ConCache Worker returned unusable resultz%ConCache Got empty update from workerr z"ConCache Adding minion %s to cachez*ConCache Replacing minion list from workerz.ConCache Got malformed result dict from workerz ConCache %s entries in cache� zConCache Shutting down)3r r� r� ZREPr� r� r� r� ZSUBZ SUBSCRIBEr� �connectr� ZPoller�register�POLLIN�signal�SIGINTr� r� r �infor� r& �poll�KeyboardInterruptr� ZZMQErrorr4 � exceptionr) r r! r" �recvr$ r% �strr� rG r� rJ r �
IndexError�len�intr� r r� r Zterm)r> r� Zcreq_inZcupd_inZtimer_inZpollerZsocksZzmq_err�msgZreplyZ
new_c_datar Z sec_eventZcwr r r r� � s�
��
� �
�
�
�LzConnectedCache.run) r3 r� r� r� r? r� r� r� r� r� r� r r r� r r� \ s r� c C sl | d rt tjj�| ��� �}d}nd}d}tj�� �}|j|d|d� W d � dS 1 s/w Y dS )z%
Ping all connected minions.
rF rJ �*r1 z test.ping)r6 N) rJ r r rG rt r� rZ Z LocalClientZ cmd_async)r r5 ZformrZ r r r �ping_all_connected_minions s "�r� Fc C s� | dkr|� dd�dkr|� dd�} | �d�r |� dd�} tjj�� r)| �dd�} tj� |d d| � d��}tjj
� |d | |� z tjj �
|d ��
}|�� W d
� W S 1 sZw Y W d
S tyk Y d S w ) z
Return the master key.
�root�userZsudo_�\�_r �.Z_key�rNr0 )r) �
startswithr r r- Z
is_windows�replacer
r r rK Zcheck_path_traversalr r r r )Zkey_userr Zskip_perm_errors�keyfile�keyr r r �get_master_key- s
(� �r� c C s0 g }| D ]}t jj�||�r|�| | � q|S )zL
Check a whitelist and/or blacklist to see if the value matches it.
)r r Z stringutilsZ
expr_match�extend)Z pattern_dictZ user_namer �exprr r r �get_values_of_matching_keysF s �r� �__main__z/etc/salt/master)F).r� �loggingr
r� � threadingr r Z
salt.cacher Z salt.clientZ salt.configZ salt.payloadZ salt.pillarZsalt.utils.atomicfileZsalt.utils.filesZsalt.utils.minionsZsalt.utils.platformZsalt.utils.stringutilsZsalt.utils.verifyZsalt.exceptionsr Zsalt.utils.cacher r� Zsalt.utils.processr Zsalt.utils.zeromqr � getLoggerr3 r r r r( r/ r� r� r� r� r� r� ZconfigZ
master_configr Zconcr� r r r r �<module> sP
( +! D
�