File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/__pycache__/master.cpython-310.pyc
o
�N�gq � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl
Z
ddl Z ddl Z ddl
Z
ddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Zddl Zddl Zddl Zddl Zddl!Zddl"Zddl#Zddl$Zddl%Zddl&Zddl'Zddl(Zddl)Zddl*Zddl+Zddl,Zddl-Zddl.Zddl/Zddl0Zddl1Zddl2Zddl3Zddl4m5Z5 ddl6m7Z7 ddl8m9Z9 ddl:m;Z; ddl<m=Z= ddl>m?Z? dd l@mAZAmBZBmCZC dd
l#mDZD dd lEmFZF dd l2mGZGmHZH zddlIZId
ZJW n eK�y dZJY nw e�LeM�ZNG dd� d�ZOG dd� dejPjQjR�ZSG dd� dejPjQjR�ZTG dd� deO�ZUG dd� dejPjQjR�ZVG dd� dejPjQjR�ZWG dd � d �ZXG d d � d eX�ZYG d d � d eX�ZZdS )!z�
This module contains all of the routines needed to set up a master server, this
involves preparing the three listeners and the workers needed by the master.
� N)�DEFAULT_INTERVAL)�DEFAULT_TARGET_DELIM)� StackContext)�
TRANSPORTS)�iter_transport_opts)�RequestContext)�enable_sigusr1_handler�enable_sigusr2_handler�
inspect_stack)�tagify)� OrderedDict)�ZMQ_VERSION_INFO�zmqTFc s\ e Zd ZdZi Zdd� Z� fdd�Z� fdd�Zdd � Ze dd d ��Z
e ddd��Z � Z S )�SMasterzN
Create a simple salt-master, this will generate the top-level master
c C s$ || _ tj�| j �| _| �� | _dS )zm
Create a salt master server instance
:param dict opts: The salt options dictionary
N)�opts�salt�cryptZ
MasterKeys�
master_key�_SMaster__prep_key�key��selfr � r �?/opt/saltstack/salt/lib/python3.10/site-packages/salt/master.py�__init__\ s zSMaster.__init__c s. t � �|� |d | _|d | _|d t_d S )Nr r �secrets)�super� __setstate__r r r r �r �state�� __class__r r r k s
zSMaster.__setstate__c s& t � �� }|�| j| jtjd�� |S )N)r r r )r � __getstate__�updater r r r r r r r r" q s
��zSMaster.__getstate__c C s t jj�| j�S )z�
A key needs to be placed in the filesystem with permissions 0400 so
clients are required to run as root.
)r �daemons� masterapiZ access_keysr �r r r r Z
__prep_key| s zSMaster.__prep_keyNc C s� | j d d �� �0 | j d d jtjkr | j||dd� n | j d d jd7 _| j d d jW d � S 1 s<w Y d S )N�aes�secret�serialF)�use_lock� )r �get_lock�value�sys�maxsize�rotate_secrets)�clsr �eventr r r �
get_serial� s $�zSMaster.get_serialTc C s� t �d� |d u r i }| j�� D ]X\}}|rC|d �� � tjj�|d � �|d _ d|v r3d|d _ W d � n1 s=w Y ntjj�|d � �|d _ d|v rYd|d _ |rh|j
d|� d�did d
� q|� d �r|t � d � tjj
�|� d S d S )
NzRotating master AES keyr( �reloadr) r Zrotate_Z_keyTr )�tagZping_on_rotatez1Pinging all connected minions due to key rotation)�log�infor �itemsr, r �utils� stringutils�to_bytesr- �
fire_event�get�debug�masterZping_all_connected_minions)r1 r r2 r* Z
secret_keyZ
secret_mapr r r r0 � s4
�
� ��
�
�
�zSMaster.rotate_secrets)NN)NNT)
�__name__�
__module__� __qualname__�__doc__r r r r" r � classmethodr3 r0 �
__classcell__r r r r r S s � r c sX e Zd ZdZ� fdd�Zdd� Zdd� Zdd � Zd
d � Zd d
� Z dd� Z
dd� Z � Z S )� MaintenancezP
A generalized maintenance process which performs maintenance routines.
c sX |� dd�| _t� jdi |�� || _t| jd �| _tt�� �| _t| jd �| _ dS )z[
Create a maintenance instance
:param dict opts: The salt options
�master_secretsN�
loop_intervalZmaintenance_intervalr )
�poprG r r r �intrH �time�rotate�restart_interval)r r �kwargsr r r r � s zMaintenance.__init__c C s( | j dur | j t_t| j�}d|d< tj�|�}tj� | ji �| _ tj
j j | j|�
� | j d�| _ tj
j�| j�| _tj
jj| j| jd dd�| _tjj�| j�| _| jd rmtj
j�� smt�d | jd � t� | jd � d| _ | j� d
d�r�d}t | j�D ]
\}}|d kr�d}q~|s�d| _ dS dS dS ) a<
Some things need to be init'd after the fork has completed
The easiest example is that one of these module types creates a thread
in the parent process, then once the fork happens you'll start getting
errors like "WARNING: Mixing fork() and threads detected; memory leaked."
NT�quiet)� returners�sock_dirF��listenZmaintenance_nicenessz"setting Maintenance niceness to %d�presence_eventsZtcp) rG r r �dictr r �runner� RunnerClient�loaderrP r9 �scheduleZScheduleZfunctions_dict�minions� CkMinions� ckminionsr2 �get_master_eventr$ r% Zinit_git_pillar�
git_pillar�platform�
is_windowsr6 r7 �os�nicerT r= r )r Zropts�
runner_clientZtcp_only� transport�_r r r �_post_fork_init� s<
��
��
�zMaintenance._post_fork_initc C s | � � t�� }d}d}tt�� �}| j�dd�}t� }t�� | | jk r�t�d� |r3|| | j krKt
j j �
| j� t
j j �| j� t
j j �| j� |rS|| |krY|}| �� | �� | �� | �|� | �|� t
jj�| j� |}tt�� �}t�| j � t�� | | jk s%dS dS )z�
This is the general passive maintenance process controller for the Salt
master.
This is where any data that needs to be cleanly maintained from the
master is maintained.
Nr �git_pillar_update_intervalz Running maintenance routines)rf rK rJ r r= �setrM r6 �tracerH r r$ r% Zclean_old_jobsZclean_expired_tokensZclean_pub_auth�handle_git_pillar�handle_schedule�handle_key_cache�handle_presence�handle_key_rotater9 �verifyZcheck_max_open_files�sleep)r �start�lastZlast_git_pillar_update�nowrg � old_presentr r r �run� s2
�zMaintenance.runc C s� | j d dkrog }| j d tv rd}nd}t�tj�| j d |��D ]}|�d�s=tj�tj�| j d ||��r=|�|� q"t �
d� t j j
jtj�| j d |d �d
d ��}t j�||� W d � d S 1 shw Y d S d S )
z`
Evaluate accepted keys and create a msgpack file
which contains a list
Z key_cacheZschedrd rZ Zaccepted�pki_dir�.zWriting master key cachez
.key_cache�wb)�modeN)r r ra �listdir�path�join�
startswith�isfile�appendr6 r> r r9 Z
atomicfileZ atomic_open�payload�dump)r �keys�accZfn_Z
cache_filer r r rl s& �
�
�"��z Maintenance.handle_key_cachec C s� d}t j�| jd d�}z3t �|�}tjj�� r,t � |t j
�s,d}t � |tj tj
B � n
|jdkr4d}nt�d� t �|� W n tyH Y nw | j�d�r[|| j | jd kr[d}|rjt�| j| j� || _dS dS ) z-
Rotate the AES key rotation
F�cachedir�.dfnTi � z6Found dropfile with incorrect permissions, ignoring...Zpublish_sessionN)ra r{ r| r �statr r9 r_ r` �access�W_OK�chmod�S_IRUSR�S_IWUSR�st_moder6 �error�remove�OSErrorr= rL r r0 r2 )r rs Z to_rotate�dfn�statsr r r rn 1 s*
�
�z Maintenance.handle_key_rotatec
C sP z
| j D ]}|�� qW dS ty' } ztjddd� W Y d}~dS d}~ww )z#
Update git pillar
z*Exception caught while updating git_pillarT��exc_infoN)r^ Z
fetch_remotes� Exceptionr6 r� )r �pillar�excr r r rj O s
� ��z Maintenance.handle_git_pillarc
C sf z| j �� | j j| jk r| j j| _W n ty+ } z t�d|� W Y d}~nd}~ww | j �� dS )z(
Evaluate the scheduler
z&Exception %s occurred in scheduled jobN)rY �evalrH r� r6 r� Zcleanup_subprocesses)r r� r r r rk Y s
���zMaintenance.handle_schedulec C s� | j rK| jjdd�rM| j�� }|�|�}|�|�}|s |r0t|�t|�d�}| j�|tdd�� dt|�i}| j�|tdd�� |� � |�
|� dS dS dS )z1
Fire presence events if enabled
� )�timeout)�new�lostZchangeZpresence�presentN) rT r2 Z connect_pullr\ Z
connected_ids�
difference�listr< r �clearr# )r rt r� r� r� �datar r r rm g s
�zMaintenance.handle_presence)
r@ rA rB rC r rf ru rl rn rj rk rm rE r r r r rF � s -,
rF c sH e Zd ZdZ� fdd�Zdd� Zedd� �Zedd � �Z d
d � Z
� Z S ) �FileserverUpdatezH
A process from which to update any dynamic fileserver backends
c sB t � jdi |�� || _i | _dd l}|j�| j�| _| �� d S )Nr r )r r r �update_threads�salt.fileserver�
fileserver�
Fileserver� fill_buckets)r r rN r r r r r � s zFileserverUpdate.__init__c C s | j �� }i | _| j �� D ]w}|� d�}z| j j| }W n ty+ t�d|� Y q
w ||v rY|| �� D ]!\}}|sDt }t�d|� | j�
|t � �}|�
||fg �� |� q6q
z |� d�}| j
| }W n tyw t }t�d||� Y nw d| j�
|t � �||f< q
dS )z�
Get the configured backends and the intervals for any backend which
supports them, and set up the update "buckets". There will be one
bucket for each thing being updated at a given interval.
z.updatez/No update function for the %s filserver backendz<An update_interval of 0 is not supported, falling back to %sZ_update_intervalzQ%s key missing from configuration. Falling back to default interval of %d secondsN)r� �update_intervals�buckets�backends�servers�KeyErrorr6 r> r8 r �
setdefaultr r r �warning) r r� �backend�fstr� update_func�id_�intervalZi_ptrZ interval_keyr r r r� � sJ
���
��
���z FileserverUpdate.fill_bucketsc C s� | � � D ];\}}|\}}z|rt�d||� |f}nt�d|� d}||� W q ty? } z t�d|� W Y d}~qd}~ww dS )z,
Perform fileserver updates
z:Updating %s fileserver cache for the following targets: %sz Updating %s fileserver cacher z5Uncaught exception while updating %s fileserver cacheN)r8 r6 r> r� � exception)r� r� Z update_argsZ backend_namer� �argsr� r r r �
_do_update� s* � ����zFileserverUpdate._do_updatec C s� t � � }t�� }t � � | |k rCt�d|� | �|� t�d||� |�
|�|� W d � n1 s4w Y t � � | |k sdS dS )zV
Threading target which handles all updates for a given wait interval
zEPerforming fileserver updates for items with an update interval of %dzXCompleted fileserver updates for items with an update interval of %d, waiting %d secondsN)rK � threading� Conditionr6 r> r� �wait)r1 r� r� r� rq Z conditionr r r r# � s" �
� ��zFileserverUpdate.updatec C s� | j d r tjj�� s t�d| j d � t�| j d � tj j
� | j � | j D ] }t
j| j|| j | | j d fd�| j|< | j| �� q'| jrht| j�� �D ]\}}|�d� |�� sb| j�|� qO| jsHdS dS )z*
Start the update threads
Zfileserver_update_nicenessz'setting FileServerUpdate niceness to %dZfileserver_interval)�targetr� r+ N)r r r9 r_ r` r6 r7 ra rb r$ r% Zclean_fsbackendr� r� �Threadr# r� rq r� r8 r| �is_aliverI )r r� �name�threadr r r ru � s4 �
��
� �
��zFileserverUpdate.run) r@ rA rB rC r r� � staticmethodr� rD r# ru rE r r r r r� { s
5
r� c @ s8 e Zd ZdZdd� Zdd� Zdd� Zdd � Zd
d � Zd S )
�Masterz
The salt master server
c C s& t r tdk r t�d� t�| |� dS )z]
Create a salt master server instance
:param dict: The salt options
)r� � z�You have a version of ZMQ less than ZMQ 3.2! There are known connection keep-alive issues with ZMQ < 3.2 which may result in loss of contact with minions. Please upgrade your ZMQ!N)r r
r6 r� r r r r r r r s
�zMaster.__init__c C s� t sd S t�tj�\}}|tjkr|}t�d||� | jd }||kr-t�d|||� |}||k rct�d|� t�tj||f� zt�tj�\}}t�d||� W d S t yb t�
d|� Y d S w d S )Nz:Current values for max open files soft/hard setting: %s/%sZmax_open_filesz�The value for the 'max_open_files' setting, %s, is higher than the highest value the user running salt is allowed to set (%s). Defaulting to %s.z"Raising max open files value to %sz5New values for max open files soft/hard values: %s/%sz~Failed to raise max open files setting to %s. If this value is too low, the salt-master will most likely fail to run properly.) � HAS_RESOURCE�resourceZ getrlimitZ
RLIMIT_NOFILEZ
RLIM_INFINITYr6 r7 r Z setrlimit�
ValueError�critical)r Zmof_sZmof_hZmof_cr r r Z__set_max_open_files& sD
�
�
�
���zMaster.__set_max_open_filesc C s� g }g }zt �d� W n ty& } z|�d|� d�� W Y d}~nd}~ww | j�dd�rlddl}|j�| j�}|j sK|�d�
d � | jd
��� n!z|� � W n |j
jyk } z |�|� � W Y d}~nd}~ww | jd
sv|�d � | jd r�t j�t j� | jd
d ��s�z&|jj�d�� t �t j� | jd
d �� W d � n1 s�w Y W n ty� Y nw | j�dd��r1zdd� | j�dg �D �}W n ty� g }|�d� Y nw |�r1zKt�| j�}ddl}|D ];} | g|d< z|jjj|| d |jj j |jj j |jj j d�}
W q� |j
j�y+ } z |�|j � W Y d}~q�d}~ww W ~n~w |�s7|�r\|D ]} t!�"| � �q9|D ]} t!�#| � �qDt!�#d� t$�%|j&j'j(� dS dS )zv
Run pre flight checks. If anything in this method fails then the master
should not start up.
�/z!Cannot change to root directory (�)NZfileserver_verify_configTr zCFailed to load fileserver backends, the configured backends are: {}z, Zfileserver_backendz%No fileserver backends are configuredZ pillar_cacher� �? Zgit_pillar_verify_configc S s& g | ]}d |v rt |d t�s|�qS )�git)�
isinstance�str)�.0�xr r r �
<listcomp>� s
�z&Master._pre_flight.<locals>.<listcomp>Z
ext_pillarz�Invalid ext_pillar configuration. It is likely that the external pillar type was not specified for one or more external pillars.r� )Zper_remote_overridesZper_remote_onlyZ global_onlyz)Master failed pre flight checks, exiting
))ra �chdirr� r r r= r� r� r� r� �formatr| �init�
exceptionsZFileserverConfigErrorr{ �isdirr9 �filesZ set_umask�mkdir� TypeError�copy�deepcopyZsalt.pillar.git_pillarZgitfsZ GitPillarr� r^ ZPER_REMOTE_OVERRIDESZPER_REMOTE_ONLYZ GLOBAL_ONLY�strerrorr6 r� r� r. �exit�defaultsZ exitcodesZ
EX_GENERIC) r �errorsZcritical_errors�errr r� r� Z git_pillarsZnew_optsZrepor^ r� r r r � _pre_flightW s� ��� ��
� �� �
� ��
����
�zMaster._pre_flightc
C s\ | � � t�dtjj�� � t� t� | � � tjj
� t j
t j���� t�tjtjj�tjj�� ��tjtjdd�tjjjd�tjd< t�d� tjj
j dd�| _ g }t�d � t | j �D ] \}}tj j!j"�#|�}|j$| j d
tjid � |�%|� q^t�d � | j j&tjj'j(| j fd
d� | j �)d�r�t*| j d t+�r�d}| j d D ]
}d|v r�d} nq�|s�| j d �%di i� nd| j d vr�t�d� i | j d d< tj,�-| j | j � t�d� | j j&t.| j fdtjidd� | j �)d��rt�d� | j j&tjj'j/| j fdd� | j �)dg �}|D ]Q}t�d|� z8d �0|�1d �d d � �} |�1d �d }
t2| t3� t4� |
gd �} | �5|
�}
d |
j6� d �} | j j&|
| j f| d� W �q t7�y_ t�8d!|� Y �qw | j d" �r�t�d#� | j j&tjj9j:| j fd$d� t�;d%� t<�=d&� t�d'� i }
tjj>�?� �r�tj|
d
< | j j&t@| j | jA| jBf|
d(d� | j j&tC| j fd)d� | j d* �r�tjjDjE�F� �r�| j j&tjjDjE| j d* d+ | j d, d-| j d* �)d-i �id.�jGd/d0� nt�8d1� tHjIjJd&k�r�t�8d2� W d � n 1 �s�w Y t �Kt j
�t jLu �rt � t j
| jM� t �Kt j�t jLu �r't � t j| jM� | j �G� d S )3z6
Turn on the master server components
z$salt-master is starting as user '%s'F)�lock)r( r) r4 r' z Creating master process manager� )�
wait_for_killz!Creating master publisher processr )rN z'Creating master event publisher process�EventPublisher�r� r� Zreactor�enginesTzEnabling the reactor enginez#Creating master maintenance processrG rF )r� rN r� Z event_returnz$Creating master event return process� EventReturnZ
ext_processesz"Creating ext_processes process: %srw N���z ExtProcess(r� z(Error creating ext_processes process: %sZ con_cachez Creating master concache process�ConnectedCachez-Sleeping for two seconds to let concache restr� z&Creating master request server process� ReqServerZFileServerUpdateZ discovery�portZ interface�mapping)r� Z listen_ipZanswer�SSDPDiscoveryServer)r� z6Unable to load SSDP: asynchronous IO is not available.zRYou are using Python 2, please install "trollius" module to enable SSDP discovery.)Nr� r6 r7 r r9 �user�get_userr r �_Master__set_max_open_files�process�default_signals�signal�SIGINT�SIGTERM�multiprocessing�Array�ctypes�c_charr: r; r Z CrypticleZgenerate_key_stringZValue�
c_longlongr r �ProcessManager�process_managerr r �channel�server�PubServerChannel�factory�pre_forkr � add_processr2 r� r= r� r� r� Z
start_enginesrF r� r| �split�
__import__�globals�locals�__getattribute__rB r� r� r? r� r> rK rp r_ Zspawning_platformr� r r r� Zssdpr� Z is_availableru r. � version_info�major� getsignal�SIG_DFL�_handle_signals)r Z pub_channelsre r �chanZrine�itemZ ext_procs�proc�modr1 Z_tmpr� rN r r r rq � s�
��� �
� ��
�
� � �
�
�
� ���
�� � z Master.startc C s& | j �||� t�d� t�d� d S )Nr+ r )r� r rK rp r. r� �r �signum�sigframer r r r H s
zMaster._handle_signalsN) r@ rA rB rC r r� r� rq r r r r r r� s 1V r� c sT e Zd ZdZd� fdd� Z� fdd�Zdd� Zd d
� Zej fd d �Z
d
d� Z � Z S )r� zZ
Starts up the master request server, minions send results to this
interface.
Nc s. t � jdi |�� || _|| _|| _|| _dS )a
Create a request server
:param dict opts: The salt options dictionary
:key dict: The user starting the server and the AES key
:mkey dict: The user starting the server and the RSA key
:rtype: ReqServer
:returns: Request server
Nr )r r r r r r )r r r �mkeyr rN r r r r U s
zReqServer.__init__c s | � |� t� �||� d S �N)�destroyr r r r r r r g s
zReqServer._handle_signalsc C s� | j dur | j t_ tj�| jd d�}tj�|�rAz tjj �
� r1t� |tj �s1t�
|tjtjB � t�|� W n ty@ Y nw tjjjddd�| _g }t| j�D ]\}}tjjj�|�}|�| j� |� |� qR| jd r�tjj �
� s�t � d| jd � t� | jd � tjj� t!j"t!j#��) t$t%| jd ��D ]}d
|� �}| jj&t'| j| j(| j)|f|d � q�W d � n1 s�w Y | j�*� dS ) z(
Binds the reply server
Nr� r� ZReqServer_ProcessManagerr+ )r� r� �req_server_nicenessz/setting ReqServer_ProcessManager niceness to %dZworker_threadszMWorker-r� )+r r ra r{ r| r r~ r r9 r_ r` r� r� r� r� r� r� r� r� r� r� r� r r� r� ZReqServerChannelr� r� r r6 r7 rb r� r� r� r� �rangerJ r� �MWorkerr r ru )r r� � req_channelsrd r r
Zindr� r r r �__bindk sH
�� �
�� �zReqServer.__bindc C s | � � dS )z(
Start up the ReqServer
N)�_ReqServer__bindr& r r r ru � s z
ReqServer.runc C s2 t | d�r| j�� | j�|� | j�� d S d S )Nr� )�hasattrr� Zstop_restartingZsend_signal_to_processesZ
kill_children)r r r r r r � s
�zReqServer.destroyc C s | � � d S r )r r& r r r �__del__� s zReqServer.__del__r )
r@ rA rB rC r r r ru r� r� r r rE r r r r r� O s 0r� c s� e Zd ZdZ� fdd�Z� fdd�Z� fdd�Z� fdd �Zd
d � Ze j
j j j
d d
� �Zdd� Zdd� Zdd� Zdd� Z� ZS )r zd
The worker multiprocess instance to manage the backend operations for the
salt master.
c sN t � jdi |�� || _|| _|| _|| _d| _t�dd� �| _ t
�
� | _ dS )a.
Create a salt master worker process
:param dict opts: The salt options
:param dict mkey: The user running the salt master and the RSA key
:param dict key: The user running the salt master and the AES key
:rtype: MWorker
:return: Master worker
r c S �
ddd�S �Nr )�mean�runsr r r r r �<lambda>� �
z"MWorker.__init__.<locals>.<lambda>Nr ) r r r r r r �k_mtime� collections� defaultdictr� rK �
stat_clock)r r r r r rN r r r r � s zMWorker.__init__c s$ t � �|� |d | _|d t_d S )Nr" r )r r r" r r r r r r r � s
zMWorker.__setstate__c s" t � �� }|�| jtjd�� |S )N)r" r )r r" r# r" r r r r r r r" � s
zMWorker.__getstate__c sv t | dd�D ]}z|�� W q ty Y qw t | dd �}|d ur2z|�� W n ty1 Y nw t� �||� d S )Nr r � clear_funcs)�getattr�closer� r r r )r r r r� r& r r r r � s � �zMWorker._handle_signalsc C sb t jjj�� | _| j�� | jD ] }|j| j | jd� qz| j�
� W dS t t fy0 Y dS w )z(
Bind to the local port
)�io_loopN)
r �ext�tornadoZioloopZIOLoopr) Z make_currentr Z post_fork�_handle_payloadrq �KeyboardInterrupt�
SystemExit)r Z req_channelr r r r � s
��zMWorker.__bindc C s> |d }|d }|dkr| � |�}n| �|�}tjjj�|��)a�
The _handle_payload method is the key method used to figure out what
needs to be done with communication to the server
Example cleartext payload generated for 'salt myminion test.ping':
{'enc': 'clear',
'load': {'arg': [],
'cmd': 'publish',
'fun': 'test.ping',
'jid': '',
'key': 'alsdkjfa.,maljf-==adflkjadflkjalkjadfadflkajdflkj',
'kwargs': {'show_jid': False, 'show_timeout': False},
'ret': '',
'tgt': 'myminion',
'tgt_type': 'glob',
'user': 'root'}}
:param dict payload: The payload route to the appropriate handler
�enc�loadr' )� _handle_aes�
_handle_clearr r* r+ �genZReturn)r r� r r0 �retr r r r, � s
zMWorker._handle_payloadc C s� t � � }|| }| j| d | j| d d | | j| d | j| d< || j | jd krR| jj�|| j | j| jd�t| jd�� t �
dd� �| _|| _d S d S )
zK
Calculate the master stats and fire events with stat info
r r r+ Zmaster_stats_event_iter)rK Zworkerr� r� c S r r r r r r r r * r! z%MWorker._post_stats.<locals>.<lambda>N) rK r� r% r � aes_funcsr2 r< r� r r# r$ )r rq �cmd�endZdurationr r r � _post_stats s " ��
�
�zMWorker._post_statsc C s� t �d|d � |d }| j�|�}|si ddifS | jd r.t�� }| j| d d7 < ||�ddif}| jd rA| �||� |S )z�
Process a cleartext command
:param dict load: Cleartext payload
:return: The result of passing the load to a function in ClearFuncs corresponding to
the command specified in the load's 'cmd' key.
z&Clear payload received with command %sr6 �funZ
send_clear� master_statsr r+ )r6 ri r& �
get_methodr rK r� r8 )r r0 r6 �methodrq r4 r r r r2 - s
zMWorker._handle_clearc s� d|vr t �d|� i S |d }t �d|d � � j�|�}|s&i ddifS � jd r:t�� }� j| d d7 < � fd d
�}tt �
t |� jd ���� ||�}W d � n1 s[w Y � jd rk� � ||� |S )
z�
Process a command sent via an AES key
:param str load: Encrypted payload
:return: The result of passing the load to a function in AESFuncs corresponding to
the command specified in the load's 'cmd' key.
r6 z Received malformed command %sz$AES payload received with command %sr9 �sendr: r r+ c s � j �| d | �S )Nr6 )r5 �run_func)r� r& r r r> V s z%MWorker._handle_aes.<locals>.run_func)r� r N)
r6 r� ri r5 r; r rK r� r � functools�partialr r8 )r r� r6 r< rq r> r4 r r& r r1 B s(
�
�
zMWorker._handle_aesc C s� t jj�� sJd}| jd r0t jj�� dkr't�d| j � t
� d| jd � n t� d| j � d}|rJ| jd rJt�d | j | jd � t
� | jd � t
| j| j�| _| j�� t| j�| _| �� d
S ) z'
Start a Master Worker
Tr �rootz1%s decrementing inherited ReqServer niceness to 0r� z@%s unable to decrement niceness for MWorker, not running as rootFZmworker_nicenesszsetting %s niceness to %iN)r r9 r_ r` r r� r� r6 r7 r� ra rb r� �
ClearFuncsr r& �connect�AESFuncsr5 �_MWorker__bind)r Zenforce_mworker_nicenessr r r ru b s6
����
z MWorker.run)r@ rA rB rC r r r" r rE r r* r+ r3 � coroutiner, r8 r2 r1 ru rE r r r r r � s
r c @ s e Zd ZdZdZdd� ZdS )�TransportMethodsz�
Expose methods to the transport layer, methods with their names found in
the class attribute 'expose_methods' will be exposed to the transport layer
via 'get_method'.
r c C sH || j v r zt| |�W S ty t�d|� Y dS w t�d|� dS )zM
Get a method which should be exposed to the transport layer
z Requested method not exposed: %sN)�expose_methodsr' �AttributeErrorr6 r� )r r� r r r r; � s
�zTransportMethods.get_methodN)r@ rA rB rC rH r; r r r r rG � s rG c @ s� e Zd ZdZdZdd� Zdd� Zdd� Zd d
� Zd d � Z d
d� Z
dd� Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Zd/d0� Zd1d2� Z d3d4� Z d5S )6rD zQ
Set up functions that are available when the load is encrypted with AES
)�
verify_minion� _master_tops� _master_opts� _mine_get�_mine� _mine_delete� _mine_flush�
_file_recv�_pillar�
_minion_event�_handle_minion_event�_return�_syndic_return�
minion_runner�pub_ret�
minion_pub�minion_publish� revoke_auth� _serve_file�
_file_find�
_file_hash�_file_hash_and_stat�
_file_list�_file_list_emptydirs� _dir_list�
_symlink_list�
_file_envsc C s| || _ tjjj| j | j d dd�| _tjj�|�| _tj� | j d �| _
tj j | j dddd�| _
| �� tjj�|�| _dS )z�
Create a new AESFuncs
:param dict opts: The salt options
:rtype: AESFuncs
:returns: Instance for handling AES operations
rQ FrR � conf_fileT�ZstatesZrendZignore_config_errorsN)r r r9 r2 r] rZ r[ r\ �client�get_local_client�local�minion� MasterMinion�mminion�_AESFuncs__setup_fileserverr$ r% Z RemoteFuncsr r r r r � s �
�zAESFuncs.__init__c C sv ddl }|j�| j�| _| jj| _| jj| _| jj | _
| jj | _ | jj
| _| jj| _| jj| _| jj| _| jj| _dS )zK
Set the local file objects from the file server interface
r N)r� r� r� r �fs_Z
serve_filer\ Z
_find_filer] Z file_hashr^ Zfile_hash_and_statr_ Z file_listr` Zfile_list_emptydirsra Zdir_listrb Z symlink_listrc Z file_envsrd )r r r r r Z__setup_fileserver� s
zAESFuncs.__setup_fileserverc
C s� t jj�| j|�s dS tj�| jd d|�}zt j� |�}W n+ t
y. t � d|� Y dS t
ttfyI } z
t �d||� W Y d}~nd}~ww z |�|�dkrUW dS W n t
yn } z t �d |� W Y d}~nd}~ww t �d
|� dS ) �i
Take a minion id and a string signed with the minion private key
The string needs to verify as 'salt' with the minion public key
:param str id_: A minion ID
:param str token: A string signed with the minion private key
:rtype: bool
:return: Boolean indicating whether or not the token can be verified.
Frv rZ zzSalt minion claiming to be %s attempted to communicate with master, but key could not be read and verification was denied.z"Unable to load public key "%s": %sNs saltTzUnable to decrypt token: %szdSalt minion claiming to be %s has attempted to communicate with the master and could not be verified)r r9 ro �valid_idr ra r{ r| r Z PublicKeyr� r6 r� r� �
IndexErrorr� r� Zdecrypt)r r� �tokenZpub_pathZpubr� r r r Z__verify_minion� s8 �������zAESFuncs.__verify_minionc C s | � ||�S )ro )�_AESFuncs__verify_minion)r r� rr r r r rJ s zAESFuncs.verify_minionc s> d| j vrdS t| j d t�sdS t� fdd�dD ��r dS � d �d�r'dS | �� d � d �s;t�d
� d � dS � �d � g }| j d D ] }t �
|� d �ret| j d | t �re|� | j d | � qGd � d v r�� d �
d �� d< g }� d D ] }|�|�
� � q{|� d < | jj|� d � d � d
� �dd�dd�S )a
Verify that the passed information authorized a minion to execute
:param dict clear_load: A publication load from a minion
:rtype: bool
:return: A boolean indicating if the minion is allowed to publish the command in the load
ZpeerFc 3 � � | ]}|� vV qd S r r �r� r ��
clear_loadr r � <genexpr>. s �
�z3AESFuncs.__verify_minion_publish.<locals>.<genexpr>)r9 �arg�tgtr4 �tok�idr9 zpublish.r| r{ zOMinion id %s is not who it says it is and is attempting to issue a peer command�,ry rz �tgt_type�globT)Zpublish_validate)r r� rU �anyr} rs r6 r� rI �re�matchr� �extendr r r\ �
auth_checkr= )r rw Zpermsr� Zarg_ry r rv r Z__verify_minion_publish sJ
��
�
�z AESFuncs.__verify_minion_publishc s~ t � fdd�|D ��r
dS d� vr t�d� d t� d d� dS | �� d � d �s4t�d� d � dS d� v r=� �d� � S ) a�
A utility function to perform common verification steps.
:param dict load: A payload received from a minion
:param list verify_keys: A list of strings that should be present in a
given load
:rtype: bool
:rtype: dict
:return: The original load (except for the token) if the load can be
verified. False if the load is invalid.
c 3 rt r r ru �r0 r r rx e � � z)AESFuncs.__verify_load.<locals>.<genexpr>Fr{ �7Received incomplete call from %s for '%s', missing '%s'r| �co_name�&Minion id %s is not who it says it is!)r� r6 r� r
rs r� rI )r r0 Z verify_keysr r� r Z
__verify_loadX s
�
zAESFuncs.__verify_loadc C �( | � |d�}|du r i S | jj|dd�S )z�
Return the results from an external node classifier if one is
specified
:param dict load: A payload received from a minion
:return: The results from an external node classifier
�r| r{ FT�Z skip_verify)�_AESFuncs__verify_loadr% rK �r r0 r r r rK z s zAESFuncs._master_topsc C s i }i }| � � }|D ]
}||vrg ||< q
||d< | jd |d< | jd |d< | jd |d< |�d�r5|S | jd |d< | jd |d< | jd |d< | jd |d < | jd
|d
< | jd |d < | jd |d < | jd
|d
< | jd |d< | jd |d< | jd |d< | jd |d< |S )z�
Return the master options to the minion
:param dict load: A payload received from a minion
:rtype: dict
:return: The master options
�
file_rootsZtop_file_merging_strategyZ env_orderZ default_topZenv_onlyZrendererZfailhardZ state_topZstate_top_saltenvZ
nodegroupsZstate_auto_orderZ state_eventsZstate_aggregateZ jinja_envZ
jinja_sls_envZjinja_lstrip_blocksZjinja_trim_blocks)rd r r= )r r0 �moptsr� Zenvs�saltenvr r r rL � s4 �
zAESFuncs._master_optsc C r� )z�
Gathers the data from the specified minions' mine
:param dict load: A payload received from a minion
:rtype: dict
:return: Mine data from the specified minions
)r| rz r9 r{ FTr� )r� r% rM r� r r r rM � s zAESFuncs._mine_getc C r� )z�
Store the mine data
:param dict load: A payload received from a minion
:rtype: bool
:return: True if the data has been stored in the mine
)r| r� r{ FTr� )r� r% rN r� r r r rN � s zAESFuncs._minec C s$ | � |d�}|du r i S | j�|�S )a
Allow the minion to delete a specific function from its own mine
:param dict load: A payload received from a minion
:rtype: bool
:return: Boolean indicating whether or not the given function was deleted from the mine
)r| r9 r{ F)r� r% rO r� r r r rO � � zAESFuncs._mine_deletec C r� )z�
Allow the minion to delete all of its own mine contents
:param dict load: A payload received from a minion
r� FTr� )r� r% rP r� r r r rP � s zAESFuncs._mine_flushc sP t � fdd�dD ��r
dS t� d t�sdS | jd s dS tjj�| j� d �s*dS d| jd }d
� v rB� d
d k rBt� d � dS t
� d
�� � d
d � |kr[t� d|� d � dS d� vrnt� d� d t � d d� dS | �
� d � d �s�t�d� d � i S � �d� tj�� d �}tj�|�}tj�|�s�d� d v r�dS tj�| jd d� d d|�}tj�|��| jd �s�t�d|� dS tj�|�}tj�|�s�zt�|� W n ty� Y nw tj�|�r� d
d kr�d}nd}tjj � ||��!}� d
�r |� � d
� |� tjj �!� d
�� W d � dS 1 �s!w Y dS ) zm
Allows minions to send files to the master, files are sent to the
master file cache
c 3 rt r r ru r� r r rx � r� z&AESFuncs._file_recv.<locals>.<genexpr>)r| r{ �locFr{ Z file_recvr| i �file_recv_max_sizer� r z#Invalid file pointer: load[loc] < 0r� z�file_recv_max_size limit of %d MB exceeded! %s will be truncated. To successfully push this file, adjust file_recv_max_size to an integer (in MB) large enough to accommodate it.r{ r� r� r� z../r� rZ r� zdAttempt to write received file outside of master cache directory! Requested path: %s. Access denied.�abrx NT)"r� r� r� r r r9 ro rp r6 r� �lenr= r
rs r� rI ra �sepr| r{ �normpath�isabsr} �dirnamer� �makedirsr� r~ r� �fopen�seek�writer: r; ) r r0 r� Zsep_pathr� Zcpath�cdirry �fp_r r� r rQ � s|
��
�� �
��zAESFuncs._file_recvc s t � fdd�dD ��r
dS tjj�| j� d �sdS � d � d d< tjj| j� d � d � �d� �d��� �d �� �d
i �� �d �� �d �� �d
�d� }|� � }| j
� � | j�dd�r�| j j
�d�� d �d� d |d�� | j�d�du r�| j�d� d it� d dd�� |S )z�
Return the pillar data for the minion
:param dict load: Minion payload
:rtype: dict
:return: The pillar data for the minion
c 3 rt r r ru r� r r rx @ r� z#AESFuncs._pillar.<locals>.<genexpr>)r| �grainsFr| r� r� �envr* �pillar_override� pillarenv�extra_minion_data� clean_cache)r* r� r� r� r� Zminion_data_cache�
minions/{}r� )r� r� Zminion_data_cache_eventsTzMinion data cache refreshZrefreshrj )r� r r9 ro rp r r� Z
get_pillarr= Zcompile_pillarrn Z update_optsr% �cache�storer� r2 r< r )r r0 r� r� r r� r rR 7 s<
�
�
�zAESFuncs._pillarc C s2 | � |d�}|du r i S | j�|� | �|� dS )z�
Receive an event from the minion and fire it on the master event
interface
:param dict load: The minion payload
r� FN)r� r% rS rT r� r r r rS ` s
zAESFuncs._minion_eventc C s� |d }|� dd�dkrt�d||d d � |� dg �D ]G}|� di �}d |v rd|� d
�}|s1q |d }ztjjj| j||| j|d � W q t tj
j fyc } zt�d |||� W Y d
}~q d
}~ww q d
S )z5
Act on specific events from minions
r| r5 � Z _salt_errorz#Received minion error from [%s]: %sr� �message�eventsrZ �jid)rl Z syndic_idz)Could not add minion(s) %s for job %s: %sN) r= r6 r� r r9 �jobZ
store_minionsr rl r� r� �SaltCacheError)r r0 r� r2 Z
event_datar� rZ r� r r r rT o s0 �
������z AESFuncs._handle_minion_eventc C s | j d rd|vrt�d|d � dS d|v r_t�d� |�d�}tj�| j d d�|d ��}t j
j � |�}t j
�|||�s[t�d |d � | j d
rVt�d |d � dS t�d � ||d< zt jjj| j || j| jd
� W dS t jjy� t�d|� Y dS w )aU
Handle the return data sent from the minions.
Takes the return, verifies it and fires it on the master event bus.
Typically, this event is consumed by the Salt CLI waiting on the other
end of the event bus but could be heard by any listener on the bus.
:param dict load: The minion payload
Z require_minion_sign_messages�sigzo_return: Master is requiring minions to sign their messages, but there is no signature in this payload from %s.r| Fz*Verifying signed event publish from minionrv r� z0Failed to verify event signature from minion %s.Z drop_messages_signature_failzAdrop_messages_signature_fail is enabled, dropping message from %szLBut 'drop_message_signature_fail' is disabled, so message is still accepted.)r2 rl z,Could not store job information for load: %sN)r r6 r� ri rI ra r{ r| r� r Z serializersZmsgpackZ serializer Zverify_signaturer7 r9 r� Z store_jobr2 rl r� r� r� )r r0 r� Zthis_minion_pubkeyZserialized_loadr r r rU � sD
�
��
�� ��zAESFuncs._returnc s� � � d�}t|t�s
� g}|D ]Љ t� fdd�dD ��r q� � d�r<| jd r<d�| jd �}| jj| � d � d � tj �
| jd d �}tj �
| jd d � d
�}t j j
�||�r�tj �|�s�tj �|�d }tj �|�stt�|� t j j�|d ��
}|�d
� W d � n1 s�w Y � d �� D ]G\}} � d |d�}
|
�| � d� v r�� d |
d< d� v r�� d |
d< d� v rň d |
d< d� v rψ d |
d< d� v rو d |
d< | �|
� q�qdS )z�
Receive a syndic minion return and format it to look like returns from
individual minions.
:param dict load: The minion payload
r0 c 3 rt r r ru r� r r rx � r� z*AESFuncs._syndic_return.<locals>.<genexpr>)�returnr� r| �master_job_cache� {}.save_loadr� r� Zsyndicsr| r �wr� Nr� )r� r| � master_idr9 Zfun_args�outr� )r= r� r� r� r r� rl rP ra r{ r| r r9 ro Z
clean_path�existsr r� r� r� r� r8 r# rU ) r r0 �loadsr� rA Zsyndic_cache_path� path_nameZwfhr r r4 r r� r rV � sP
��
�
�
��zAESFuncs._syndic_returnc C s$ | � |d�}|du r i S | j�|�S )z�
Execute a runner from a minion, return the runner's function data
:param dict clear_load: The minion payload
:rtype: dict
:return: The runner function data
)r9 ry r| r{ F)r� r% rW )r rw r0 r r r rW � r� zAESFuncs.minion_runnerc C s� | � |d�}|du r i S tj�| jd d�}tj�|�s!t�|� tj�|t|d ��}tj j
� |d��}|d |� � ksFi W d � S W d � n1 sPw Y | j
�|d �S ) a
Request the return data from a specific jid, only allowed
if the requesting minion also initiated the execution.
:param dict load: The minion payload
:rtype: dict
:return: Return data corresponding to a given JID
)r� r| r{ Fr� Z publish_authr� �rr| N)r� ra r{ r| r r� r� r� r r9 r� r� �readri Zget_cache_returns)r r0 Z
auth_cacheZjid_fnr� r r r rX � s
� �zAESFuncs.pub_retc C � | � |�si S | j�|�S )aL
Publish a command initiated from a minion, this method executes minion
restrictions so that the minion publication will only work if it is
enabled in the config.
The configuration on the master allows minions to be matched to
salt functions, so the minions can only publish allowed salt functions
The config will look like this:
.. code-block:: bash
peer:
.*:
- .*
This configuration will enable all minions to execute all commands:
.. code-block:: bash
peer:
foo.example.com:
- test.*
The above configuration will only allow the minion foo.example.com to
execute commands from the test module.
:param dict clear_load: The minion pay
)� _AESFuncs__verify_minion_publishr% rY �r rw r r r rY �
zAESFuncs.minion_pubc C r� )aD
Publish a command initiated from a minion, this method executes minion
restrictions so that the minion publication will only work if it is
enabled in the config.
The configuration on the master allows minions to be matched to
salt functions, so the minions can only publish allowed salt functions
The config will look like this:
.. code-block:: bash
peer:
.*:
- .*
This configuration will enable all minions to execute all commands.
peer:
.. code-block:: bash
foo.example.com:
- test.*
The above configuration will only allow the minion foo.example.com to
execute commands from the test module.
:param dict clear_load: The minion payload
)r� r% rZ r� r r r rZ : r� zAESFuncs.minion_publishc C sF | � |d�}| j�dd�st�d|d � |S |du r |S | j�|�S )a2
Allow a minion to request revocation of its own key
:param dict load: The minion payload
:rtype: dict
:return: If the load is invalid, it may be returned. No key operation is performed.
:rtype: bool
:return: True if key was revoked, False if not
r� Zallow_minion_key_revokeFzKMinion %s requested key revoke, but allow_minion_key_revoke is set to Falser| )r� r r= r6 r� r% r[ r� r r r r[ ] s � zAESFuncs.revoke_authc C s� |� d�r i ddifS t| |�r<zt�� }t| |�|�}t�d|t�� | � W n ty; d}tjd|dd� Y n
w t�d |� d
ddifS |d krR|ddifS |d krxd
|v rx|�d�dkrn| j d dkrn|ddifS |dd|d
d�fS |ddifS )z�
Wrapper for running functions executed with AES encryption
:param function func: The function to run
:return: The result of the master function that was called
�__r9 r= z'Master function call %s took %s secondsr� zError in function %s:
Tr� zHReceived function %s which is unavailable on the master, returning FalseFrU rR r| �ver�2Zpillar_versionr+ Z send_privater� )r9 r rz )
r} r rK r' r6 ri r� r� r= r )r �funcr0 rq r4 r r r r> x s2
� �� zAESFuncs.run_funcc C s, | j �� | jd ur| j�� d | _d S d S r )r% r ri r& r r r r � s
�zAESFuncs.destroyN) r@ rA rB rC rH r rm rs rJ r� r� rK rL rM rN rO rP rQ rR rS rT rU rV rW rX rY rZ r[ r> r r r r r rD � s8 '
8"
# U)34## )rD c @ s� e Zd ZdZdZdd� Zdd� Zdd� Zd d
� Zd d � Z d
d� Z
dd� Z dd� Z dd� Z
edd� �Zd#dd�Zdd� Zd d � Zd d � Zd d!� Zd"S )$rB z~
Set up functions that are safe to execute when commands sent to the master
without encryption and authentication
)�ping�publish� get_token�mk_token�wheelrV c C s� || _ || _tjjj| j | j d dd�| _tj�| j d �| _tjj �
|�| _ tj �
|�| _tjj| j dddd�| _tj�|�| _tjj�||�| _g | _d S )NrQ FrR re Trf )r r r r9 r2 r] rg rh ri rZ r[ r\ �authZLoadAuth�loadauthrj rk rl r� ZWheel�wheel_r$ r% Z
LocalFuncs�channels)r r r r r r r � s �
�
zClearFuncs.__init__c
C sv | � |�\}}}}| jj|||d�}|�d�}|r d|iS |�d�}|dkr\| j�|�dg �|d |�di ��} | sEd|d�||�d �iS t| t�rPd| v rP| S |D ]}
|� |
d
� qRn d|v rt|d }t
j � |��
� rs| j�dd �}nt
jj�� }z|� d�} t
j�| j�} | j| |�di �|d d
�W S ty� }
zt�d| |
� d|
jj|
jt|
�d�iW Y d
}
~
S d
}
~
ww )zJ
Send a master control function back to the runner system
�r r� �usernamer� � auth_listr9 �kwarg�9Authentication failure of type "{}" occurred for user {}.�r� r� NrA T)ri �-Exception occurred while introspecting %s: %s)r� r� r� ) �_prep_auth_infor� �check_authenticationr= r\ � runner_checkr� r� rU rI r r� �AuthUser�is_sudor r9 r� r� rV rW Z asynchronousr� r6 r� r! r@ r� r� )r rw � auth_type�err_namer �sensitive_load_keysr� r� r� r� r r9 rc r� r r r rV � s\
��� ��
�����zClearFuncs.runnerc
C s | � |�\}}}}| jj|||d�}|�d�}|r d|iS |�d�}|dkr\| j�|�dg �|d |�di ��} | sEd|d�||�d �iS t| t�rPd| v rP| S |D ]}
|� |
d
� qRn d|v rt|d }t
j � |��
� rs| j�dd �}nt
jj�� }zSt
jj�| j�} |� d�} t| d d
�}
d| � �| |
|d�}| j�|t| dgd �� | jj| fddi|��}|d |d< |d |d< | j�|t| dgd �� |
|d�W S t�y } z-t�d| |� d�| |j j |�|d< d|d< | j�|t| dgd �� |
|d�W Y d
}~S d
}~ww )zI
Send a master control function back to the wheel system
r� r� r� r� r� r9 r� r� r� NrA r� )�prefixzwheel.)r9 r� r5 r� r� Z full_returnTr� �successr4 )r5 r� r� z&Exception occurred in wheel {}: {}: {}F) r� r� r� r= r\ � wheel_checkr� r� rU rI r r� r� r� r r9 r� r� r� Zgen_jidr r2 r< r� Z call_funcr� r6 r� r! r@ )r rw r� r� r r� r� r� r� r� r r� r9 r5 r� r4 r� r r r r� sr
��� ��
� ���zClearFuncs.wheelc C s" | j �|�}|st�d� dS |S )z�
Create and return an authentication token, the clear load needs to
contain the eauth key and the needed authentication creds.
z0Authentication failure of type "eauth" occurred.r� )r� r� r6 r� )r rw rr r r r r� b s
zClearFuncs.mk_tokenc C s d|vrdS | j �|d �S )zZ
Return the name associated with a token or False if the token is invalid
rr F)r� Zget_tokr� r r r r� m s zClearFuncs.get_tokenc C s\ |� di �}tj�| jd �}|�|d �s |�|d �r/t�d|d |d � dddd �iS |� d
t �}| j
� |d |� d d
�|�}|� dt � �}|� dt � �}|� dd�}|� dd�} | �
|�\}
} } }
|
dkrt| jj||
| d�}n| j�||
�}d}d|v r�|�dg �}|dur�| | | j� dd� kr�|}n|� dg �}d|
� d�}|� d�r�t�|� dddd �i}d|v r�| j�i |�|�t|d dgd �� |S |
dks�|
dk�rY|�rY| j
j||d |d |d |� d d
�|d gd �}|�s7|
d k�r|�sd!|v �rd |v �rt�d"|d |d! � t�|� dddd �i}d|v �r5| j�i |�|�t|d dgd �� |S |
d#k�rL|� d!�}||d< t�d$|� n
|
d k�rY| j�|�|d< | j� d%��sq|�sqd&d|d'�|d �d(�d)�S | �||�}|du �r�d&dd*id)�S | �|||||�}| j� d%��r�||d< | j||d+� | � |� d&|d ||d,�d)�S )-zt
This method sends out publications to the minions, it can only be used
by the LocalClient.
rN Zpublisher_acl_blacklistr� r9 zq%s does not have permissions to run %s. Please contact your local administrator if you believe this is in error.
r� ZAuthorizationErrorz Authorization error occurred.r� � delimiterrz r~ r rZ �missing� ssh_minionsFr Nr� r� rA z Authentication failure of type "z " occurred.ZAuthenticationErrorz Authentication error occurred.r� r� ry zsaltutil.find_job)rZ Z whitelist�eauthr� z8Auth configuration for eauth "%s" and user "%s" is emptyrr z Minion tokenized user = "%s"Z
order_mastersr� z.Master could not resolve minions for target {})r� rZ r� )r/ r0 zMaster failed to assign jid)r� )r� rZ r� ) r= r ZaclZ PublisherACLr Zuser_is_blacklistedZcmd_is_blacklistedr6 r� r r\ Z
check_minionsr� r� r� r� rI r� r2 r< r r� r> Z load_namer� � _prep_jid� _prep_pub�
_send_ssh_pub� _send_pub)r rw �extraZ
publisher_aclr� Z_resrZ r� r� Zauth_keyr� r� r r� r� Zsyndic_auth_listr� �err_msgr� Z
authorizedr� r� r� r r r r� u s� � ���� � �
���
�
�
�
��
�
���
�zClearFuncs.publishc C sT g }d }d|v rd}d}dg}nd|v r d}d}ddg}nd}d}| j }||||fS ) Nrr ZTokenAuthenticationErrorr� ZEauthAuthenticationErrorr� �passwordr� ZUserAuthenticationErrorr� )r rw r� r r� r� r r r r� s
zClearFuncs._prep_auth_infoc C s� |� d�r |d nd}|� dd�}d�| jd �}z
| jj| ||d�}W |S ttfyD d�|�d �d
�}t� |� d |i Y S w ) z3
Return a jid for this publication
r� N�nocacheFz {}.prep_jidr� )r� �
passed_jidzJFailed to allocate a jid. The requested returner '{}' could not be loaded.rw r r� )
r= r� r rl rP r� r� r r6 r� )r rw r� r� r� r� r� �msgr r r r� ( s ��
�zClearFuncs._prep_jidc C sP | j st| j�D ]\}}tjjj�|�}| j �|� q| j D ]}|� |� q dS )zQ
Take a load and send it across the network to connected minions
N)
r� r r r r� r� r� r� r r� )r r0 rd r r
r r r r� @ s
�zClearFuncs._send_pubc C s& t | d�stjjjj| jd�| _| jS )N� _ssh_client)r� )r r rg ZsshZ SSHClientr r� r& r r r �
ssh_clientK s
zClearFuncs.ssh_clientFc C sB | j d du r |du r t�d� tj| jj|d��� dS dS dS )zK
Take a load and send it across the network to ssh minions
Zenable_ssh_minionsTzSend payload to ssh minions)r� rN N)r r6 r> r� r� r� r6 rq )r r0 r� r r r r� Q s
�zClearFuncs._send_ssh_pubc
C sd ||d< |� di �� dt�}| j�d|i|d � |d |d |d |d |d |d ||d
�}| j�|t|d d gd �� | jd
r�d�| jd
�}d} ztjj �
| j j | �}
d|
j vrgt
�d| jd
� W n ttfy~ d} t
�d| jd
� Y nw | r�z| j j | |d ||d� W n ty� t
jddd� Y nw zd�| jd �}| j j | |d ||� W n! ty� t
�d| jd � Y n ty� t
jddd� Y nw ddi} |d |d |d |d |d d�} d | jv r�| jd | d < d |v �r|d | d < |tk�r|| d< d |v �r|d | d < d|v �r%|d | d< d |v �r0|d | d < d|v �r�d |d v �rE|d � d �| d < d |d v �rU|d � d �| d < d!|d v �re|d � d!�| d!< d"|d v �ru|d � d"�| d"< d#|d v �r�|d � d#�| d#< d|v �r�t
�d$|d |d |d � |d | d< n t
�d%|d |d � t
�d&| � | S )'z�
Take a given load and perform the necessary steps
to prepare a publication.
TODO: This is really only bound by temporal cohesion
and thus should be refactored even further.
r� rN r� rZ r~ rz r� r9 ry )r� r~ rz r� r9 ry rZ r� r� r� Z
ext_job_cacher� Tz�The specified returner used for the external job cache '%s' does not have a 'minions' kwarg in the returner's save_load function.Fz_The specified returner used for the external job cache "%s" does not have a save_load function!)rZ z,The specified returner threw a stack trace:
r� r� z]The specified returner used for the master job cache "%s" does not have a save_load function!r/ r' r4 )r9 ry rz r� r4 r� r| �toZ
ret_config�metadataZmodule_executorsZ
executor_optsZ
ret_kwargsz(User %s Published command %s with jid %sz Published command %s with jid %sz Published command details %s)r= r r2 r< r r r� r r9 r� Zget_function_argspecrl rP r6 r� rI r� r� r7 r> )
r rZ r� rw r� r� r� Z new_job_loadr� Zsave_load_funcZarg_specr� r0 r r r r� Y s� �
�
����
�
�� � �
�
�� zClearFuncs._prep_pubc C s |S )z3
Send the load back to the sender.
r r� r r r r� � s zClearFuncs.pingc C sZ | j d ur
| j �� d | _ | jd ur| j�� d | _| jr+| j�� }|�� | js d S d S r )r% r ri r� rI r( )r r
r r r r � s
�zClearFuncs.destroyc C s> | j rd S t| j�D ]\}}tjjj�|�}| j �|� q
d S r ) r� r r r r� r� r� r� r )r rd r r
r r r rC � s �zClearFuncs.connectN)F)r@ rA rB rC rH r rV r� r� r� r� r� r� r� �propertyr� r� r� r� r rC r r r r rB � s*
AM "
rB )[rC r# r� r� r? �loggingr� ra r� r� r� r. r� rK Zsalt.aclr Z salt.authZsalt.channel.serverZ salt.clientZsalt.client.ssh.clientZ
salt.cryptZsalt.daemons.masterapiZsalt.defaults.exitcodesZ salt.enginesZsalt.exceptionsZsalt.ext.tornado.genZsalt.keyZ salt.minionZ salt.payloadZ salt.pillarZ salt.runnerZsalt.serializers.msgpackZ
salt.stateZsalt.utils.argsZsalt.utils.atomicfileZsalt.utils.eventZsalt.utils.filesZsalt.utils.gitfsZsalt.utils.gzip_utilZsalt.utils.jidZsalt.utils.jobZsalt.utils.masterZsalt.utils.minionsZsalt.utils.platformZsalt.utils.processZsalt.utils.scheduleZsalt.utils.ssdpZsalt.utils.stringutilsZsalt.utils.userZsalt.utils.verifyZsalt.utils.zeromqZ
salt.wheelZ salt.configr Z
salt.defaultsr Z salt.ext.tornado.stack_contextr Zsalt.transportr Zsalt.utils.channelr Zsalt.utils.ctxr Zsalt.utils.debugr r r
r Zsalt.utils.odictr r
r r� r� � ImportError� getLoggerr@ r6 r r9 r� ZSignalHandlingProcessrF r� r� r� r rG rD rB r r r r �<module> s� �
W R ?_ \